Microsoft have given more insights on Project HoloLens, including a look at the hardware inside the headset and glimpses of a host of applications. The demonstrations took place at the company’s annual developers conference, Build, April 29.

The insights amount to our having much greater sense of the functionality of the immersive augmented reality (AR) device which was noted as becoming available within the Windows 10 ‘timeframe’. But whether that means this year, or next, unfortunately we don’t yet know.

We also learnt that Microsoft will be unveiling more about HoloLens in June at E3 2015 – an event that’s sure to present some of the best demos yet and perhaps even a release date.

Project HoloLens is fundamentally different from virtual reality (VR) devices such as Oculus Rift or HTC/Steam’s Vive (for outlines on the VR landscape see ‘The VR Race is On‘). Rather than a headset that presents its wearer with an entirely simulated visual scene, HoloLens’ transparent lenses allow you to see, hear and interact ordinarily with your environment.

What HoloLens does is to augment that scene using entirely state-of-the-art holographic technology that’s capable of mapping virtual objects and graphics into the environment. Additional technologies allow these to not only be seen, but also heard and interacted with in what Microsoft are terming ‘mixed reality’.

Microsoft HoloLens is the first fully untethered, see-through holographic computer. It enables high-definition holograms to come to life in your world, seamlessly integrating with your physical places, spaces, and things. We call this experience mixed reality. Holograms mixed with your real world will unlock all-new ways to create, communicate, work, and play.

Microsoft, Project HoloLens

Inner Workings

A complex blend of sensors embedded within the headset allows it to map its surroundings in real time – creating a 3D canvas of objects, walls and surfaces which, when viewed through the visor, it projects graphics onto.

An inertial measurement unit (IMU) consisting of an accelerometer, gyroscope, and magnetometer tracks head movement. The IMU, together with head tracking cameras, captures information about what a user is doing and where they’re doing it.

Depth sensors play a critical role in interpreting gestures used for interacting with the digital scene. Input to HoloLens is also supported by voice commands captured by a microphone array.

The headset features a photo/video camera to support sharing experiences between users and the ability to capture HoloLens moments.

The latest demonstrations also shed light on the device’s use of spatial sound technology. Developers have said that HoloLens can synthesise a binaural sound in such a way that you perceive it as having come from a specific location in the physical space around you. Apparently, a scientific model which characterises how the human ear receives sound from a specific physical location has supported this capability. Either way, it’s sure to add considerable immersion to the HoloLens experience.

All this hardware is clearly going to be gathering an astonishing amount of data, much of which must be processed in real-time. To handle this, HoloLens utilises a high-end CPU and GPU, as well as a first of its type Holographic Processing Unit (HPU).

For more information, note that we provided a detailed look at how HoloLens is working when it was first unveiled – ‘Introducing Project HoloLens‘.

 

Power & Flexibility

When Project HoloLens was first unveiled in January much of its capability was conveyed through slides and images. Now, several of those same applications have been shown in full, working form. And it’s seriously impressive.

We see for instance how properties of HoloLens’ multi-dimensional graphics are truly powerful. They can be pinned, or anchored, into a fixed location you choose, allowed to follow their own rules of movement, or pinned to a position within your field of view independent of the direction you look or your movements.

Take the HoloLens globe and Skype feature – what we saw first as pictures, we now know is a complete app, and one that will be available in Windows 10. We witness the pinning feature, as well as the placement of windows on any surface before being enlarged to a preferred scale. Check out HoloLens’ lead developer Alex Kipman narrating through this demonstration of the device.

That graphics can follow some path of movement according to rules of their own is an exciting prospect. One that hints at possibilities for forms of AI, or at the very least virtual characters, to feature in future HoloLens applications. After all, the demonstrations did feature video of a virtual puppy and a little robot. Could virtual assistants be on the horizon? Almost certainly.

Freedom of Movement

Project HoloLens entirely untethered – meaning that there’s no need for cables, or even connection to a PC. It will function as a stand-alone, battery powered device. Having said that, it’s unlikely that HoloLens will be working outdoors – since it’s not been demonstrated and Microsoft have focussed on in-door scenarios.

Applications

At the demonstration this week an emphasis was placed on the device’s working seamlessly with Windows 10. More specifically all universal apps, which currently includes Office apps, Outlook and Photos, will function with HoloLens.

Furthermore, Windows 10 will come with application programming interfaces (APIs) that support holographic computing – meaning that developers can work with APIs to extend holographic platforms.

Windows 10 is the first platform to support holographic computing with APIs that enable gaze, gesture, voice, and environmental understanding on an untethered device. With Windows 10, holograms are Windows universal apps, and all Windows universal apps can be made to work as holograms.

Microsoft, Project HoloLens

Building Project HoloLens with Windows 10 in mind is a huge hint at what Microsoft envision. Unlike many VR headsets which thus far are geared toward tapping the gaming market, Project HoloLens appears focussed on mainstream personal computing and advancing mixed reality as a principal platform for interacting with the digital world. Of course it’ll still be interesting to see if HoloLens games are demoed at E3 in June – since the possibilities here too are extraordinary.

Aside from personal computing, another new video clip from the demonstrations provides a look at what HoloLens could do for industry, in particular for design. The video conveys outcomes from collaboration between Microsoft and Trimble, a company which provides design tools for architects and engineer. We see renderings of architectural designs coming to life as holograms at various scales, rooms transformed with blueprints overlaid over existing structures, and modifications made to designs in real-time.

The scope of applications for AR technology the likes of which HoloLens is advancing is hard to foresee at this time, but to hear Microsoft’s take on it, we’re looking at the beginnings of a new era for interacting with digital media and information.

 

Additional Information

There were developments beyond new demonstrations too. Microsoft launched an official YouTube channel Project HoloLens – available here. The company also found time to update the official Project HoloLens website. A video of the Build keynote is available here.

 

Sources

Project HoloLens Official Site

Microsoft Conversations