Microsoft's research arm has released a video demonstrating a technology it calls IllumiRoom, designed to increase immersion when gaming by extending the environment beyond the confines of the television or monitor.
Created by combining a projector with a modified version of the Kinect motion-sensing peripheral available for the current Xbox 360, the system starts by performing a depth-scan of the room - complete with totally unnecessary but very cool grid-line animations provided by the projector. Once the system has an idea of the shape and layout of the surface surrounding the TV, plus the location and size of the TV itself, things start to get really clever.
When activated, the IllumiRoom system uses the projector to extend the game world beyond the edges of the TV or monitor. Gunfire and explosions exit the screen and travel across the room - appearing to approach the player - while environmental effects such as snowflakes or fire also appear. The system also allows for the entire game world to be projected outside the screen, either as a full-colour representation or a simplified outline-only version.
The result is undeniably incredible: even on a small-screen system, the game world appears to surround the user, covering the player's visual field with images that are not only there to increase immersion but which can also provide an edge during gaming - providing an in-game equivalent to real-life peripheral vision in a similar way to that promised by the Oculus Rift
project, but without the need to wear a silly headset.
The basics of IllumiRoom could, of course, be replicated through a standard projector and a modified rendering engine - but the Kinect-powered depth scan provides important information for setting up the system. In its demonstration, Microsoft Research shows the system being used in a realistic living-room environment - complete with a cabinet, shelves and an entertainment unit on which the TV sits. All of these objects provide an uneven surface for the projector which would normally distort the projected image, turning straight lines into a maze of zigzags that would entirely ruin the effect.
To avoid this, IllumiRoom uses the depth data from the Kinect sensor - gathered during the 'scanning room' portion of the system's setup - to pre-distort the image in a such a way that, when projected onto the uneven surface surrounding the TV, the picture appears perfect to the player. The scanning system also detects the size and position of the TV set, ensuring that the projector doesn't attempt to wash out its image with a picture of its own while creating a perfectly-centred illusion of immersion for the player.
Sadly, the one thing Microsoft Research hasn't shared is when - or, indeed, if - the IllumiRoom system will be released, but the timing of its unveiling is interesting: with Microsoft expected to launch a successor to the Xbox 360 later this year, and that purported Xbox 720 being expected to come bundled with a second-generation Kinect sensor system, it wouldn't be surprising to see IllumiRoom heading into living rooms later this year or early 2014.
In the meantime, enjoy a short but impressive demonstration of the technology - created, Microsoft claims, entirely on the IllumiRoom system with no camera trickery or post-processing effects.