Gamescom 2010: Kinect Interview - Nick Burton, Rare's Senior Software Engineer
Written Thursday, August 19, 2010 By Dan WebbView author's profile
Founded in 1982, Rare has been through a lot in 28 years, going from developing for 8-bit machines like the ZX Spectrum to sealing an agreement to develop Nintendo-exclusive titles in 1994. Then since 2002, Rare has been a subsidiary of Microsoft who bought the company for a record $375 million.
Rare has more recently been instrumental in helping to develop Kinect, Microsoft's motion-detecting camera that has more than a few tricks under it's glossy black exterior and has been hard at work readying Kinect Sports as a first-party launch title.
We caught up with Rare's Senior Software Engineer, Nick Burton at Gamescom to chat about Kinect, Kinect Sports, the new Xbox Live Avatars and the experiences in working with Kinect's new and exciting technology.
Is it just us, or has the design of the Xbox Live Avatars changed slightly for the Dashboard Kinect Update?
No, it's not just you. Avatars are now 2.0 – that's what we say internally - and what we found with Avatars was that they have cartoon proportions and even though under-the-hood, they've got a pretty-close-to-human skeleton in terms of the bones and proportions: arms, legs, spine and so on, when you start to map a real version of you or me onto that, if the skeletal scale is off enough, the hand-eye coordination is broken. Not technologically, but it's a perception thing. Imagine that we're playing table tennis and I've got short, stubby Avatar arms, so my hand is up here. So, I go to slice the ball, and now there are two choices as a developer if I've got to use an Avatar. One is that I can say, well, what we'll do is we'll map the Avatar so its hands are in the right place, but then it ends up in a funny pose that you're not doing, so that breaks it. The other way is to say, well we'll leave these Avatars here as they are, but then we penalise the player again, because the ball would still be here, but his hand is there, so we have to have elbows and shoulders matched with yours. Now we only had to change them a bit to do this, but we've given Avatars slightly more human proportions and features.
In some ways, yes. I wouldn't want to change their proportions or anything like that in the game, but as you can see with Kinect, your Avatar is a proper physical representation of you and physically occupies the same space as you. So for something like table tennis for instance or for soccer, that's great because the hand-eye co-ordination isn't broken because I can reach for that, I can kick a football and I can imagine if the ball is there in relation to my Avatar, it's there in relation to me, if you see what I'm saying.
But then we think, why stop there? When we're thinking about the future, why don't we make the facial expressions more complicated? We could do it. It's technically possible. Why don't we have a little button you can press that makes it suddenly be wearing my shorts and T-shirt? Kinect gives us those possibilities, so we could continually layer on the fidelity. That is something I'd like to see, because it'd be pretty cool I think.
There are more sports. We've told everybody what the sports are, but we've just not let you play them yet. So you've yet to see boxing, beach volleyball and table tennis, and there are some other cool modes in there, especially cool for coming back from the pub, but we've not talked about those yet.
No, not drunken boxing. It's more the way the game plays, leveraging some of the things like the identity system with Kinect, so you don't have to worry about anything. You just go, “let me play games” and the box deals with everything so you can just have a good time and play.
Well, we've been involved with it for about two years. In fact, I was one of the first people to see it. Two years this October it'll be since we first saw it and it was so early at that time, we were blown away. It felt to me like a Fast Show moment - “does my bum look big in this?” I remember thinking, oh my word! My posture's terrible! I've really got to improve that!
Having done hardware launches in the past, like the Xbox 360 launch, GameCube launch and things like that, you expect and know things are going to change and improve, so seeing that (Kinect) there and thinking, where's that going to get to by the time we launch, was for the first-party... There's a lot of challenges there, because you're working with - in some instances very early on – some really kind of Heath Robinson (needlessly complex) devices, that you know that's going to stop at a particular point. So what we've got to do is make sure the games are being made, and we'll worry about the technology. That all firms up and it is pretty easy to use, especially when you look at some of the really hardcore technology going on underneath it. I mean, the skeletal tracking system for instance – it just goes, there's a skeleton, and that's it. There's no doing anything clever to it. It just goes, there it is, that's where it is, and there's one next to it – it knows if there are two people playing.
And then thinking of the kind of things you can do... One of the things I've said to people about Kinect is that it really feels like it's more than the sum of its parts. Now that might sound silly, but coming from a computer visual background, one of the things that was difficult to do years ago was isolate the person from all the stuff around him. If you just look at the camera feedback, if you look at the depth information, you can see there's a person-shaped thing standing in front of all this other stuff, so therefore that's the person. Then the tracking system goes, well, that's head, shoulders, knees and toes - it knows what's connected together – so you go, the head's there, we want to know who's speaking. You know roughly the direction it's coming from. I can hear something, it sounds like a voice coming from over there and there's a skeleton, he's bound by the ID system, so we know that's him. If I was just doing that with a speech recognition system, that's a really hard technical problem to solve. It's about five lines of code with Kinect.
No. It wasn't scaled back, it was productised. When you productise anything, you have to isolate the key things and identify what things are frankly really not that important, so those things fall by the wayside. We knew the skeletal tracking was the 'secret sauce' if you like, we knew we wanted an identity system, those kind of things: yes, we've got to have them. Some of the other things – I mean, I've not heard about the sign language until now to be honest – you could maybe still do that if you stood close enough to it, but pick your battles. We don't want to do that right now. Do you want to hit the world with a sign language game, or something where you can run around and do the Olympics and play football. Personally, I'd go for the football, like we did!
Yeah. The sky's the limit. I mean there's some really exciting ideas that we've got. It's technically more than the sum of its parts, so there's so many things that you could do, because you could just write a system for it. One of the things I found amazing, was finding different ways to do things like Your Shape, Ubisoft's exercise game and I never thought of doing it like that! That's brilliant! And they did it a completely different way to how I would have done it, but we were really stunned by it, they got a good result out of it. And if you think about it, there's a system that gives two developers who are geographically about 60-70 miles apart doing it totally differently, that's pretty amazing.
It's our own engine, yeah. I've spoken to quite a lot of people about, “what engines does this integrate with or what engines does the controller integrate with?” And it's very simple, it's not really a difficult proposition. You can integrate it into any engine in a couple of weeks, which is nothing.
You probably need to speak to Kudo (Tsunoda, Microsoft's Creative Director) or someone like that, because I don't really know what the decision was. All I know was, early on with the Kinect Adventures team, they were using Unreal for prototyping and I don't know whether it was analogue. For us, one or two years ago, all we were thinking was whether it would be difficult to integrate with our stuff and we only had one engine we were working with at the time. And Kinect Sports is built on our own engine and is nothing to do with Unreal. We literally took one engine we were familiar with and had it integrated with Kinect in a week. It had to be debugged and so on, and you need a game on top of it to do that, so I'd say only now is it beautifully integrated. But to get it up and running so the team can develop the game, you can pick an engine, Unreal or whatever and it's not going to be a difficult proposition.
Never say never. There are a lot of cool ideas out there still in a big stack of ideas. This is an interesting thing, because a lot of people have said to me that all of the titles for Kinect are core. I think for me as a core player, there's the initial technology infatuation. It's like smoke and mirrors or magic, the way it works. As far as looking through our back catalogue or future games, could we do what are traditionally termed core games? Yeah, totally. I mean initially one of the things I've heard and I thought was quite funny, is a lot of people talking about 3D navigation. How could you run around in Halo with Kinect, because you don't want to stand there and not run around, but there are a number of ways you can do it. At the last count we worked out eleven. I don't know if that's the ultimate golden number or if there's more or what, but there's loads of ways you could do that. There are ways you could do it where it's just you and there's ways you could do it where you potentially hold the controller with it as well. Ultimately, yes. There's totally going to be stuff for core gamers.
You could do, although I'm not saying whether we are or not. I don't know. You'd have to ask market research. The one thing I would say is I don't know whether that kind of audience would buy it. I think the kind of people who would buy it are sitting around this table right now. Games like Kinect Sports and Kinect Adventures already have so much depth and discoverability, is a gamer not going to want to do that?
We're continually, always looking at what we're going to do next. There's timing, making sure you've got the right thing at the right time and making sure you've got the right audience for the product. There's a heap of stuff that we're looking at right now, so never say never. It might be an Xbox Live Arcade game, it might be a Kinect Xbox Live Arcade game or it could be a massive opus. You'd have to get inside the mind of our Creative Director, George Andreas to find out.
Maybe Killer Instinct Kinect.
You'd have to play our boxing game and decide if you think it would.