It combines Mo-Cap and model processing at the same time

Jul 29, 2015 14:05 GMT  ·  By

Apparently showcasing Minecraft and different medical applications are just some of the things HoloLens can do, many more features remaining hidden until recently when Microsoft showed how it captures live video.

Microsoft decided to outline how HoloLens grabs live video that would work as fodder for the device's holographic capabilities. These images would then be able to be streamed over the internet so viewers owning HoloLens would actually have the feeling of watching the streamed persons live, right in front of them.

Using 106 GRB infrared cameras and a green screen around the filmed persons, the subjects will be transmitted live in the best possible lifelike representation.

It seems that the HoloLens software will give more polygonal attention to the hands and faces of the subjects, while other body parts will be a bit less precise. Microsoft aims to create a 12 Mbps MPEG file with enough details to cram into this file and be transferred online.

Unfortunately, what Microsoft needs is to expand the headset's relative narrow field of view for a convincing augmented reality experience. For now the user can have the filmed subjects themselves removed from any background and placed in their own room. If a different background is desired, it can be easily changed.

According to Road to VR, this technology isn't new at all, but what it does is that it basically compresses two sequential processes in one, doing both in the same time. In the past motion capture, systems captured the performance of their subjects and then used them to animate digital models. HoloLens does both at the same time.