Microsoft unveils IllumiRoom gaming technology
xbox, xbox 720, illumiroom, microsoft, gaming, console gaming, microsoft research, kinect, projector
Created by combining a projector with a modified version of the Kinect motion-sensing peripheral available for the current Xbox 360, the system starts by performing a depth-scan of the room - complete with totally unnecessary but very cool grid-line animations provided by the projector. Once the system has an idea of the shape and layout of the surface surrounding the TV, plus the location and size of the TV itself, things start to get really clever.
When activated, the IllumiRoom system uses the projector to extend the game world beyond the edges of the TV or monitor. Gunfire and explosions exit the screen and travel across the room - appearing to approach the player - while environmental effects such as snowflakes or fire also appear. The system also allows for the entire game world to be projected outside the screen, either as a full-colour representation or a simplified outline-only version.
The result is undeniably incredible: even on a small-screen system, the game world appears to surround the user, covering the player's visual field with images that are not only there to increase immersion but which can also provide an edge during gaming - providing an in-game equivalent to real-life peripheral vision in a similar way to that promised by the Oculus Rift project, but without the need to wear a silly headset.
The basics of IllumiRoom could, of course, be replicated through a standard projector and a modified rendering engine - but the Kinect-powered depth scan provides important information for setting up the system. In its demonstration, Microsoft Research shows the system being used in a realistic living-room environment - complete with a cabinet, shelves and an entertainment unit on which the TV sits. All of these objects provide an uneven surface for the projector which would normally distort the projected image, turning straight lines into a maze of zigzags that would entirely ruin the effect.
To avoid this, IllumiRoom uses the depth data from the Kinect sensor - gathered during the 'scanning room' portion of the system's setup - to pre-distort the image in a such a way that, when projected onto the uneven surface surrounding the TV, the picture appears perfect to the player. The scanning system also detects the size and position of the TV set, ensuring that the projector doesn't attempt to wash out its image with a picture of its own while creating a perfectly-centred illusion of immersion for the player.
Sadly, the one thing Microsoft Research hasn't shared is when - or, indeed, if - the IllumiRoom system will be released, but the timing of its unveiling is interesting: with Microsoft expected to launch a successor to the Xbox 360 later this year, and that purported Xbox 720 being expected to come bundled with a second-generation Kinect sensor system, it wouldn't be surprising to see IllumiRoom heading into living rooms later this year or early 2014.
In the meantime, enjoy a short but impressive demonstration of the technology - created, Microsoft claims, entirely on the IllumiRoom system with no camera trickery or post-processing effects.
Previous Article
44 Comments
Discuss in the forums ReplyWould love to give it a try though!
well its running on a 360...so....yeah extra gpu may not be needed.
That's likely how the trick is done: after all, if the IllumiRoom could produce an image as good as that of the TV, you wouldn't need the TV at all...
It's a tech demo, with the tech itself a very basic extension and twist on projection mapping tech we've all seen elsewhere, including Sony for their PS3 ads blended with a bit of Philips Ambilight
http://vimeo.com/34605168#
http://www.youtube.com/watch?v=MAZcDHIIXDs
This "unveiling" is not a real product and it will never be a real product. It's totally impractical as anything other than a "Aren't we clever" demo.
I don't think they're claiming everyone is going to have it tomorrow? It's a potential product?
It's far better than Ambilight, and more realistic than the PS3 thing as far as I can see?
I agree with the thinking that some of the effects were great. I especially liked when the flame and laser shots went outside the TV. That was clever. The picture above sucks as did some of the effects. And I am not sure quite how much the marketing department have messed with that end video, I suspect quite a bit. You would also need a pretty 'dark' room - the only way projectors can cope with lighter rooms nowadays are the tech built in to the pull down blinds. Normal home walls and furniture will be a whole mixture of colours, shades, materials, textures ..... A bit like the Philips ambilight TV tech. Its ok, good in a few cases, not great in most.
how wrong was i!
Because we all like to play games in well lit rooms
Gaming and watching films are why we switch the lights off!
So you only do those things in the evening? :|
I imagine it will really be for the next xbox, which should have more than enough power to do this sort of thing.
My TV is in the middle of a large white wall above my fireplace, so I can only imagine how good this will look.
Hopefully not just limited to the front wall, but would have a curve projection so it hits the side walls too. being able to spot explosions in the corner of my eye and then focusing my crosshairs on the screen to that point to have a proper look.
It is just a stepping stone to a more VR system in my opinion though, when people get use to their whole room becoming the screen, they are more likely to move onto a full VR headset when one comes out.
but it reminds me of this, which i always thought was cool;
http://www.bbc.co.uk/blogs/researchanddevelopment/2010/02/surround-video-yes-surround-vi.shtml
that tech is quite old i think
If you go to a client and bring your own projector, you can use any wall.
At home you wouldn't use it. Who spends a fortune on projector-surroundsound-blurayplayer but skimps on a screen?
Sarcasm aside, you're forgetting the brightness issue. Microsoft is showing it projecting across an entire wall, so you'd need to take that 160nit pico projector and step it back enough to project a larger image say dropping the brightness down by a factor of four to 40nit and that's assuming that your wall and furniture is painted in a nice reflective white paint like Vogel's.
So in a best case scenario you have your 40nit walls surrounding your 300-500nit screen. Those snowflakes are be going to be looking pretty dim.
Again I state that this tech demo looks cool but it's totally and utterly impractical for home use. The hardware required would be simply too expensive. And if you've splashed out on a projector bright enough to work you'll have it facing a decent screen and it won't be arranged to project over the furniture.
As for brightness: the reason pico projectors are so dim isn't so much their size - although that is a limiting factor, as you can only have a light-dump so big in a pocket-size device - but their requirement to run on battery power. A theoretical Xbox 720 IllumiRoom accessory would have no such requirement: it could be powered entirely by the console. If we assume that we also don't mind it being bigger than a normal pico projector, but small enough that it won't be awkward, and factor in the space freed up by not having a battery, there's no reason why you couldn't fit a brighter light source and bigger light dump to boost the brightness considerably.
In other words: there's absolutely nothing to stop Microsoft launching IllumiRoom as a commercially available product by the end of the year. It requires nothing in the way of technology that we don't already have available. Will Microsoft launch it by the end of the year? Who knows - I certainly don't, and neither do you.
Your side-stepping some of the laws of physics I'm afraid and underestimating the laws of economics.
You're believing that Microsoft can create a projector that outperforms anything that's been seen up to this point?
If an ultra bright projector could be produced for the money you're talking about then why hasn't any of the projector manufacturers who've been in the business for years been able to do it? What secret do you (and Microsoft) know that the rest of the worlds illumination experts don't know?
Pico-projectors are dim because to get a bright projector you have to deal with Power + Heat and dealing with those costs. You can't have a bright projector (Even an LED projector) without shifting away a lot of excess heat. And anyone with a PC knows what's needed for a decent cooling system. Bright projectors generate noise because of the cooling they need, you can't ignore cooling. Hey even my 1000lumen LED head torch states it shouldn't be used continuously while stationary due to the heat build up.
LEDs product a lot less heat than incandescent sources but LEDs still waste between 60% and 90% of their input power as heat, compared to <15% for an incandescent. LEDs waste a higher percentage of their usage but their usage is way way lower so that's mostly okay... until you start going brighter and then heat becomes an issue again...
Projectors are getting better and cheaper all the time but we're a LONG way off from having a "As bright as a LCD screen" projector available at the prices or sizes you're talking about.
The cost alone of a LED + Driver circuit + Heatsink + Fan + PSU + Projector panel + Lens + Video circuit puts your material costs beyond the range of an 'XBox Accessory' and that assuming focus is manual, keystone adjustment is software only and MS doesn't want to make a profit.
If I'm wrong. Show me an example of a commercially available projector that has appropriate brightness, at the price point you're talking about and can maintain that brightness on normal furniture. As I know that nothing exists show me something that's even half way to being suitable.
As to the whole peripheral vision side of things. This could indeed be replicated without the need to run dual screens. You just use your big-arsed HD projector with a wider in-game FOV and shifted HUD. It's all software at the end of the day you don't need a wide throw projector and a 2nd screen to do this, you can do it with the projector alone.
I'm not picking a fight here, I'm just a realist. Microsoft weren't demoing an ultra cheap state of the art 'never seen before at this price point' projection technology they were demoing a software + hardware mashup.
Again I reassert that it's a nice technology demo, but it's impractical for mass adoption due to the hardware requirements being too costly. You say I don't know this, but I do know it. This is based on commonly understood physics and by following how projector technology has progressed in the last few decades.
Edit: On a side note, I'm not really a projector expert, but I've lived with a few and there was actually a point where I'd sat and watched movies in 30% of the worlds digitally projected cinemas. I know enough to get by and to differentiate between feasible and pipe-dream, but if you know how the issues I've mentioned can be overcome I'm all ears.
You admit you've got a 1,000 lumen head-torch: that's almost twice as bright as the IllumiRoom system would need to be. A quick Google reveals that the recommended minimum projector output for a 60in screen is 160 lumens. Doubling that size to 120in - roughly the size of the projection in the demonstration video, measured by taking the size in pixels of the 11in Kinect sensor you can see in front of the TV to get a pixels-per-inch measurement and then measuring the projected image in pixels - means quadrupling the light output to 640 lumens.
Now, I believe you just said you had a 1,000 lumen head torch. So, a 640 lumen compact projector would seem perfectly feasible. Heat can be handled with a quiet fan - that's how the Xbox is cooled, after all, so what's another fan? If it's a problem, drop the output a bit: 640 lumen is the recommended brightness for a low-end device projecting to a 120in diagonal, so we can probably drop it to 400 lumen or less and still get an image that looks pretty neat with the lights off. Can't make out fine detail? Who cares? Try reading a book by holding it up to the side of your head and using just your peripheral vision - see how much fine detail you can make out then.
If you're asking me do I believe Microsoft could produce a single-purpose low-resolution projector with a 400-640 lumen output for an affordable sum, then yes. Yes I do.
So, is £160 an unreasonable price point? Possibly, possibly not - but I can't help but think £200 would be easily achievable.
If that wasn't reason enough for not doing it your way, you also have to find room for a 120in screen - like you say, nobody wants to play a game on their bookshelves, however clever the pre-distortion system might be - and another £80 for that. You also won't have any option but to 'go big' - if you're gaming in the daytime, kiss goodbye to being able to see what the hell you're doing 'cos even a 4,000-lumen projector won't make a 120in white display surface look black.
In short: you appear to be misunderstanding why the IllumiRoom exists as a concept.
Never a truer word was said.
By the way, your "Minimum recommended Lumen" link recommends projects between 700 and 12,000 lumen. I don't believe a <2000 lumen system will be anywhere close to the required brightness. Not at full wall size, through a cheap lens, off a cheap DLP panel, reflecting off the average wall.
Back to your '160 lumen' example, which your 'quick google' didn't give you the full picture.
The 160 lumen @ 60" reference you quote is not a 'recommended' table but a modified image from the publicity materials from an Aaxa HS102 Micro Projector (Also available as branded LG). It's not saying 'This is what you need' it's saying 'This is what someone is selling'
The HS102 is a £450+ 160 lumen projector can indeed display @ 60". But that's a dim image even on a white wall. Double the horizontal size and you knock the brightness back by 75%.
No way is it suitable for use in this MS demo and it costs way too much.
And your Benq... you're really quoting end of line discontinued products flogged off on ebay? Okay I'll bite. It's a projector based on incandescent tech which have very short bulb life, and the replacement bulbs are £100+.
http://compare.ebay.co.uk/like/321014155686?var=lv<yp=AllFixedPriceItemTypes&var=sbar&cbt=y
End of line obsolete tech isn't a good example to base an argument on, or an accurate example of what is possible at a given price point.
MS would need to produce a really good short throw LED / DLP projector for a fraction of the cost of anyone else has been able to do so. Incandescent projectors won't cut it for cheap home gaming use, everyone knows that bulb degradation and replacement costs can be a killer.
I say it's not possible for MS to make a system that would sit @ the right price point.
PS. Yes I have a bright head torch. It's expensive and runs hot. A head torch is basically a bulb, reflector and power/driver circuits. A projector system is a LOT more complex. I was using it as an example that it's indeed possible to drive an LED bright enough but in a projection system you need a very good way to offload heat. You can't just 'keep running through cold air' like I do with my winter head torch.
This is my final post on this by the way. There's nothing more I can say that isn't retreading the same ground.
On the whole projectors subject: I don't know much, but every portable projector I've ever used for work (and they're quite small and crap) would be perfectly bright enough to project some sort of image with the lights off.
Here's a question though: the wall behind my TV is red. Would Kinect be clever enough to alter the colour as well as the angle to produce the correct image?
Why not just use the Kinect's depth sensor? the depth sensor has a resolution of 320x240, whereas the camera has a resolution of 640x480.
Mario cart doesn't exist for the PC does it? (or for xbox)