bit-tech.net

Microsoft unveils haptic feedback monitor

Microsoft unveils haptic feedback monitor

Microsoft's Actuated 3D Display with Haptic Feedback uses a robot arm at the rear to provide a resistive force against the user's touch.

Microsoft's research arm has developed what it claims is the world's first 3D touch-screen monitor with force feedback capabilities - allowing objects on the screen to offer different 'weights' and 'textures' to the user.

Dubbed the Actuated 3D Display with Haptic Feedback, a name only a researcher could love, the project is the brainchild of Mike Sinclair, Michel Pahud and Hrvoje Benko of Microsoft's Natural Interaction Research Group. Demonstrated to the public as part of the company's TechFest 2013 event in Redmond, the system utilises the kinaesthetic haptic sense - the process of recognising objects through touch - to provide feedback a plain monitor simply cannot offer.

The system works by mounting the display - a commercial, off-the-shelf 3D monitor with multi-touch capabilities - onto a robot arm connected to the same computer. As the user pushes their finger against the screen to interact with the display, the robot arm pushes back - and can alter the strength of that feedback as required. One example used during the demonstration was an array of virtual blocks constructed from different materials: stone, wood and sponge. Each was designed to behave as realistically as possible in terms of weight and friction, with the robot arm making the user work harder to push the stone block backwards than the wood or sponge blocks.

'I had been interested in the notion of putting a robot behind something you could touch,' claims Sinclair in a blog post on the matter. 'Originally, I had wanted a robot arm with many degrees of freedom but complexity, costs, and safety issues narrowed down the options to one dimension of movement. At that point, I was sure that others must have already looked into this scenario, but after looking at the literature, it turned out no one had done this.'

The system has uses beyond crude effort-based variant, however: the arm is capable of moving in fine enough degrees that a user can feel the 'surface' of a virtual object simply by dragging a finger over the top: the stone block feels smooth compared to the sponge block, for example. During one experiment, the team blindfolded testers and got them to brush their fingers over relatively simple shapes - including two-dimensional squares and triangles and three-dimensional pyramids, cylinders and wedges. 'There were even some subjects who were 100 per cent correct [in that test],' recalls Pahud. 'That was definitely a surprise.'

The team has already considered multiple usage scenarios for the technology, beyond a simple tech demo: Pahud has already created a volumetric medical imaging system that provides fine-grained control over magnetoresonance imaging (MRI) scans from a human brain - while the remainder of the team has considered more mainstream applications. 'There’s always 3-D gaming,' Sinclair says, 'but also 3-D modelling, education, and medical. We anticipate improving the experience with crisper, more detailed feedback, such as texture.'

The team's work isn't the first to consider the importance of haptic feedback to the touch-screen experience: several patents already exist in the field, many of which are in active use today - the on-screen keyboard of most tablets and smartphones provides a small vibration when a keypress is detected as a crude means of providing haptic feedback, for example. A more impressive variant of the technology is in the Tactus Tactile Layer, unveiled in Custom PC Issue 108, which causes physical 'bubbles' to appear on the screen when the on-screen keyboard is used.

Haptic feedback predates the smartphone and tablet boom, however. From the more prosaic Microsoft Sidewinder Force Feedback Joystick, which provided complex enough feedback to require a dedicated 16-bit Intel processor on-board and the Logitech Wingman Force Feedback Mouse to the frankly bizarre Novint Falcon, engineers have been working to enhance immersion using the sense of touch for years.

Thus far, Microsoft has not indicated whether it plans to bring the Actuated 3D Display with Haptic Feedback to market as a commercial product.

14 Comments

Discuss in the forums Reply
mi1ez 3rd July 2013, 10:45 Quote
I don't understand. I've never used a touchscreen where you physically need to push it back.
Gareth Halfacree 3rd July 2013, 10:56 Quote
Quote:
Originally Posted by mi1ez
I don't understand. I've never used a touchscreen where you physically need to push it back.
That's the point: Microsoft has created something new. If it helps, imagine it as a monitor mounted on a Novint Falcon.
mi1ez 3rd July 2013, 11:05 Quote
If this means one day my phone will fight back, I want my 3210 back!
stuartwood89 3rd July 2013, 11:33 Quote
It means that typing on touch screen phones just may get tolerable.
Guinevere 3rd July 2013, 11:57 Quote
Quote:
Originally Posted by stuartwood89
It means that typing on touch screen phones just may get tolerable.

No. It's got nothing to do with touch screens phones unless you mount them on a robot arm.

This is just another research project by MS. Don't expect it to result in anything close to a real product.
Corky42 3rd July 2013, 13:52 Quote
When the article mentions 3D monitor is that a true 3D display or a stereoscopic displays ?
Blademrk 3rd July 2013, 14:13 Quote
Quote:
From the more prosaic Microsoft Sidewinder Force Feedback Joystick, which provided complex enough feedback to require a dedicated 16-bit Intel processor on-board

I still have mine but not used it in years, wonder if there's win7 drivers for it...

I remember some of the tech demos that came with it (one was as if you were moving the stick through molasses another was as if your were trying to balance the stick on the tip of a pyramid. Shame the only real application seemed to be the flight sim.
Stanley Tweedle 3rd July 2013, 14:25 Quote
Quote:
Originally Posted by mi1ez
I don't understand. I've never used a touchscreen where you physically need to push it back.

Well I already have a potential use for it. Flightsim... You're using A10 sim and looking at the instrument panel using your Track IR... you can lean forward and when you touch a virtual button in the cockpit... you feel it.

Having said that... I think a monitor on a robot arm maybe is a bit limited.

My ideal would be Oculus rift and a haptic glove with Leapmotion for accurate hand positioning.
sixfootsideburns 3rd July 2013, 14:47 Quote
Quote:
Originally Posted by stuartwood89
It means that typing on touch screen phones just may get tolerable.

Its called Swipe, and its already very tolerable. If your on a Crapple and you don't live typing, maybe you should consider switching.

Also I agree with Guinevere, this is highly irrelevant to cellular screens in the near future. Ultimately the tech sounds like it could be really awesome though. I think these concepts go well beyond gaming too. Immersive interaction (while absolutely not needed) could really change professions like drafting. Generate a 3D model, set the material, and be able to touch the screen and interact with it as a solid is a pretty awesome concept.
Gareth Halfacree 3rd July 2013, 14:48 Quote
Quote:
Originally Posted by Corky42
When the article mentions 3D monitor is that a true 3D display or a stereoscopic displays ?
Active-shutter stereoscopic - you can just about see the glasses in the picture.
Quote:
Originally Posted by Blademrk
I still have mine but not used it in years, wonder if there's win7 drivers for it...
Same - although my PC doesn't actually have a gameport anymore, so it's a bit redundant other than as a relic of a bygone era.

Gamers used to what passes for 'force feedback' these days don't know what they missed. Forget a bit of a rumble effect from a vibration motor or two; the Sidewinder Force Feedback could do so much more. Best memory: I was playing the excellent Independence War, which put you in command of a massive battle cruiser flying through space. I got into combat, and a lucky shot snuck past my shields and took out my lateral thrusters. What did the joystick do? Lose auto-center capability on the horizontal axis; if I moved the joystick vertically it would resist me as normal, fighting to return to the centre point, but it just flopped limply in the horizontal axis. When my engineers had repaired the damage, the joystick shuddered back to life and worked as normal.

Absolutely incredible stuff, and doubly so when you realise that beyond the Novint Falcon we basically have nothing like it available today.
Blademrk 3rd July 2013, 16:25 Quote
Quote:
Originally Posted by Gareth Halfacree
Same - although my PC doesn't actually have a gameport anymore, so it's a bit redundant other than as a relic of a bygone era.

Mine came with a game port to USB adapter, so I've always used it through USB.
Gareth Halfacree 3rd July 2013, 17:14 Quote
Quote:
Originally Posted by Blademrk
Mine came with a game port to USB adapter, so I've always used it through USB.
Sounds like you have the Force Feedback 2, then - mine's the original Pro model, which isn't USB compatible. Well, without a proper USB connected gameport adapter, which wasn't bundled.
longweight 3rd July 2013, 17:28 Quote
I think it was the Blackberry Bold that had a clicky touchscreen, it felt horrible to type on!

I loved the haptic feedback on my HTC Desire but I really don't miss it on my Nokia 920.

Cool idea though!
Blademrk 3rd July 2013, 20:11 Quote
Quote:
Originally Posted by Gareth Halfacree
Sounds like you have the Force Feedback 2, then - mine's the original Pro model, which isn't USB compatible. Well, without a proper USB connected gameport adapter, which wasn't bundled.

Just checked (as I've got home) yep, I've got the Force Feedback 2 (and it is native USB, not through an adapter) I also have (an extremely dusty) Microsoft Precision Pro (which uses a similar frame to yours, but without the force feedback) which has a gameport socket and a USB adapter.

Edit: Well the precision Pro works, just need to find a power cable to test the FF2 one :)

I need to dig out my Star lancer discs...
Log in

You are not logged in, please login with your forum account below. If you don't already have an account please register to start contributing.



Discuss in the forums