bit-gamer.net

Valve researching games for the deaf

Valve researching games for the deaf

Valve may be planning to introduce a new hearing-impaired character in Half-Life: Episode 3 as a love interest for Alyx.

Valve founder Gabe Newell has hinted that the developer is researching sign language implementation in games for the introduction of a new deaf character in Half-Life: Episode Three.

The news comes from a series of YouTube videos filmed at Valve headquarters which are part of a focus group Newell conducted with a hearing-impaired audience to gauge the reaction of the deaf community and to try and understand some of the issues surrounding how sign language might be introduced in games. The focus group is also attended by Valve animators who study the movements and facial expressions of participants.

While Newell admits that the interest in sign language is really just an "excuse to build the technology" and to see if characters who communicate solely through sign language is possible, it apparently may also tie in to the larger Half-Life universe.

There's talk of a new character being introduced, for example, whom Alyx knew long before the events of Half-Life 2 and whom she "had a crush on" long before Gordon showed up. The unnamed love interest communicates solely through sign language, driving Alyx to upgrade D0G to talk through sign language so that she can practice having signed conversations when he isn't around - answering the question of why the giant robot can't speak.

Valve seems especially interested in how it might incorporate signing technology into multiplayer games too, a situation where closed captioning and subtitles aren't always helpful. It'd be interesting to see that idea carried over into a stealth-military setting too of course.

Newell makes no explicit mention of incorporating signing technology into Episode Three, but has talked in the past about how Episode Three may be delayed because of exciting new technologies being incorporated into the next version of the Source Engine.

What are your thoughts on sign language being used in games? Let us know your thoughts in the forums.

30 Comments

Discuss in the forums Reply
capnPedro 17th August 2009, 11:54 Quote
I quite often play without sound, and I can't imagine what could be added with signing that isn't available with closed captions....

Oh, and Dog doesn't even have hands!
yakyb 17th August 2009, 11:56 Quote
very interesting

would be good to have a 15 minute section to teach the basics, also maybe parts later whereby the player must understand wht the NPC is saying in order to complete the level
CardJoe 17th August 2009, 12:01 Quote
I think they aren't likely to plan it so that players have to understand sing language - but I get the impression from watching the videos that it'll be likely that Alyx will use sign language to communicate with either D0G or with a deaf character and you, as Gordon, will be watching the exchange and likely relying on subtitles (if anything). Watching Alyx argue with someone else through sign language for example, then having to translate everything back to Gordon to fill him in.

Personally, I'd welcome that - it's an interesting idea and a brilliant addition to the tech. Would be fun to see the tables turned a little like that.
Bauul 17th August 2009, 12:08 Quote
Well, Vavle were very proud of the auto lip-synching technology in the first version of Source (record the sound, and the engine calculates the lip movements accordingly). Presumably they're hoping to do something similar with this: enter the text, and the engine calculates the arm movements automatically. It'd have a lot of uses outside of Half-Life too, which I imagine is where Valve are thinking long term.
Phil Rhodes 17th August 2009, 12:18 Quote
It wouldn't actually do much for the deaf audience here in the UK - American and British Sign Language are almost completely different.

I guess, though, if they're going to (presumably) motion capture it and have a virtual character do it, they could have more than one language track.
crazyceo 17th August 2009, 12:20 Quote
This actually sounds pretty cool. A practice option would be good especially for those stealth missions but any news on the developement of HL2:EP3 is good news.
CardJoe 17th August 2009, 12:24 Quote
Quote:
Originally Posted by Phil Rhodes
It wouldn't actually do much for the deaf audience here in the UK - American and British Sign Language are almost completely different.

If the tech is there as Bauul suggests though then it could easily localise itself, assuming Valve don't cater to both as they currently do with other foreign languages.
Adnoctum 17th August 2009, 12:33 Quote
Quote:
Originally Posted by Bauul
Valve were very proud of the auto lip-synching technology in the first version of Source (record the sound, and the engine calculates the lip movements accordingly)

I can tell you've never made a Source mod, because FacePoser isn't as automatic as you seem to think.
In fact, the only auto part Valve has done is to provide the tool (Face Poser) and HL2 character phenomes, which you then have to sequence manually according to your best judgement of facial movements.
And then it is a flaky bitch off her anti-psychotics as you try to troubleshoot why the &%@* it WON'T WORK!!! <demented screech - pulling of hair>
Quite frankly, there are times I genuinely would rather be hit by a car and roll the dice on permanent injury than to use FacePoser.

OTOH, exactly what you describe is used in Crytek's CryEngine2 (Crysis' engine), and it is so easy! Falling over and hitting your back-side easy. Paris Hilton easy!
mclean007 17th August 2009, 12:38 Quote
Interesting... I can imagine this spinning off into a future version of TV subtitles, with an overlaid avatar automatically converting the closed captions to signs. This could be done either at the TV station, with the rendered output overlaid on the picture as they currently do for certain programmes (e.g. Hollyoaks omnibus on T4 Sundays), or it could be rendered using hardware built into the TV. This would be a major selling point for TVs for deaf people. As a bonus, you'd be able to customise the signing variant used, and customise your avatar. How long before the first naked Lara Croft sign language avatar appears, is anyone's guess.

Not being hearing impaired, I confess I don't really understand why signed programmes are more attractive to deaf people than subtitles, but I presume you can convey more information and/or emotion with signs than with subs? They might have to add additional metainfo to subs to allow effective addition of this information, but it would still surely allow a far greater amount of signed programming than there is currently?
Ending Credits 17th August 2009, 12:44 Quote
Um? Why would you want to convert the closed captions to sign language? :p

I'm partially deaf so I like the subtitles (I can understand everything without them, it just allows me to be lazy) but I don't know sign language anyway so I can't really comment.

The main problem with subtitles is they take a long time to write so pretty worthless for news/sports programs since there's always about 10-20 delay from the words spoken to the subtitles.
bowman 17th August 2009, 13:15 Quote
Maybe the endless physics puzzles can be replaced with sign language learning. How awesome would that be, 'hey where'd you learn to sign', 'a video game'.
Star*Dagger 17th August 2009, 14:08 Quote
Awesome. Valve is at the forefront of so many aspects of PC Gaming.

On a lighter note: I often play with apparently deaf people online, especially in TF2 when I am commanding the team and the do not go where I ordered.

Yours in Louder is not better Plasma,
Star*Dagger
pizan 17th August 2009, 14:08 Quote
Just another excuse to delay episode 3
DXR_13KE 17th August 2009, 14:19 Quote
interesting... this could be adapted in a huge number of fields, cool.
Blademrk 17th August 2009, 14:30 Quote
Quote:
Originally Posted by bowman
Maybe the endless physics puzzles can be replaced with sign language learning. How awesome would that be, 'hey where'd you learn to sign', 'a video game'.

As opposed to watching Justin on cbeebies ("Look a Robot! Can you sign 'Robot'?") - spent far to long watching cbeebies with my nephew

It's a good idea (if they can get it to work)
Bauul 17th August 2009, 14:30 Quote
Quote:
Originally Posted by pizan
Just another excuse to delay episode 3

Yes, because Valve deliberatly sets out not to release their product and do everything possible not to earn any money.
Matticus 17th August 2009, 14:32 Quote
So that's why its taking so long....

In all seriousness, if the tech is being produced and could be used for a range of things outside of video gaming then its a worthwhile project to pursue. I have always fancied learning sign language, would look good on a CV. Now if I was being taught it by a video game character I would be a lot more inclined to pay attention.
MrWillyWonka 17th August 2009, 14:44 Quote
Quote:
Originally Posted by Phil Rhodes
It wouldn't actually do much for the deaf audience here in the UK - American and British Sign Language are almost completely different.

Absolutely, it would also make much more sense for Valve to actually use BSL (British Sign Language) Rather than ASL (American Sign Language) because ISL (International Sign Language) is much more similar to BSL than ASL. It is easily possible to communicate with others in Europe (and beyond) using ISL even though we don't speak the same language verbally (I have done it a number of times). So by using BSL or ISL you have a wider audience.

Being profoundly deaf myself I am fortnuate to have very good English which cannot be said for the majority for profoundly deaf people (due to the fact their listening and speaking development as a kid is slower). Sign Language if put into words would not make much sense but the expression of it gives deaf signers (not all deaf people sign!) more information to them than words ever can. For a lot of deaf people subtitles do not make sense (because of their poorer English) so the development of sign languge in games would be a great thing, especially if one day there was an option to change subtitles and speech to sign language.

Valve have won several awards for subtitling, Half Life 2 has excellent subtitling facilities which is different to HL1 which didn't have anything. http://www.deafgamers.com/ is a good website.
Quote:
Originally Posted by Matticus
So that's why its taking so long....

In all seriousness, if the tech is being produced and could be used for a range of things outside of video gaming then its a worthwhile project to pursue. I have always fancied learning sign language, would look good on a CV. Now if I was being taught it by a video game character I would be a lot more inclined to pay attention.

Its already being done by IBM in Hursley - http://www.aboutmyplanet.com/videos/virtual-translator/

Learn sign language? Many local colleges do evening courses, go to http://www.rnid.org.uk/ for mor info.
HourBeforeDawn 17th August 2009, 17:06 Quote
I think its not for functionality but for visual appeal just another layer of the game for amusement. But still seems neat.
TTmodder 17th August 2009, 19:21 Quote
Gordon's gonna be pissed :P
Matticus 17th August 2009, 19:27 Quote
Cheers for those links wonka, I might have to discuss this with my girlfriend, she is currently doing a degree in childcare, so for her learning sign language wouldn't just "look good" but could have a real benefit to her and to any children she may have to deal with in future.
kingred 17th August 2009, 22:24 Quote
gabe newell is so fat when he gets up in teh morning he gets out of both sides of the bed.
NethLyn 17th August 2009, 23:56 Quote
Quote:
Originally Posted by pizan
Just another excuse to delay episode 3

Who cares, if it's going to be the same length as other episodes. Cannot believe some people I knew who blitzed through the episodes on easy when they were two years apart. Mad. If all this stuff turns it into an add-on and it lasts a tiny bit longer, fine. Plenty of other games to play whilst we wait, DNF, it certainly isn't.
[USRF]Obiwan 18th August 2009, 12:21 Quote
I just had a clear vision: "in game character signs that the player needs to shoot the bad guy before he can fires his gu...." a damn! to late!
MrWillyWonka 18th August 2009, 13:05 Quote
Quote:
Originally Posted by HourBeforeDawn
I think its not for functionality but for visual appeal just another layer of the game for amusement. But still seems neat.

You are right but I believe it is one step closer. :)
SMIFFYDUDE 18th August 2009, 15:34 Quote
I wanna see Familiy Guy's, Greased up Deaf Guy in episode 3
Phil Rhodes 18th August 2009, 16:34 Quote
I wonder how good the motion capture tech would have to be for sign language to be intelligible. I suspect not very; it'd be the visual equivalent of an over-compressed MP3. I was involved in the design of a system to allow centralisation of BSL translators - that is, you call someone up over the internet, and talk via video link. We found that the usual video chat services often weren't good enough. Fluent BSL users often move very fast, just as fluent users of spoken language speak very fast, and the codecs have a lot of trouble keeping up; the result could sometimes be impossible to understand.

It'll be interesting to see how this affects animation data for realtime 3D rendering.
Nexxo 18th August 2009, 19:46 Quote
Quote:
Originally Posted by Phil Rhodes
It wouldn't actually do much for the deaf audience here in the UK - American and British Sign Language are almost completely different.
Mainly in alphabet signing, but many British Deaf people know how to sign-spell in ASL as well as BSL.

There are some drastic differences in conversational sign too, but for Deaf people these obstacles are easier to get around than for hearing people dealing with a different language. A lot of Deaf sign is fairly universal and discrepancies can be clarified with figurative signs. Even BSL can differ quite drastically across different areas of the UK (and even more with Ireland), and Deaf people tell me that they get around that with ease.
Rebourne 19th August 2009, 06:30 Quote
*Thumbs up*

Hopefully it works out, it would be cool to see the technology adapted to replace some of the captions as well.
1BillionHex 29th April 2011, 18:02 Quote
Talk tables are used in a wide range of games, it shouldn't be much more of a problem to use sign tables so each different set of movements for each sign language standard could be used.

The problem with captioning for many is you can't read them very well on non-HD TVs (for games consoles).

Our game currently in development will use this method and we plan to allow the engine modules responsible for converting text (and later directly from speech recognition) into sign language to be used for other purposes. With a lot of help from users, we plan to add support for as many sign languages as possible.
Log in

You are not logged in, please login with your forum account below. If you don't already have an account please register to start contributing.



Discuss in the forums