Building huge games in Adobe AIR

Most of the fall in our office was spent working on the Yahoo! Bus Stop Derby, a fantastic project involving multi player games on huge touch screens. The campaign ended January 28th, but I thought I'd share my experience from being the Tech Lead at ACNE on this project.

Here is the background: ClearChannel is putting up a new line of interactive bus shelters in the streets of San Francisco. These bus shelters feature a 72" touch screen in portrait mode and are connected to the internet over a 3G modem built into the units. The screens are supposed to be used for interactive advertising and games, and we were given the opportunity to be the first to build an interactive experience for these screens through a partnership with Goodby, Silverstein & Partners who are the advertising agency for Yahoo!

There was a lot of buzz surrounding the campaign and I encourage you to go elsewhere if you want to read what the game is all about. Instead I will write a little bit about the challenges we faced from a technical and user experience perspective.

The user experience challenge:
The form factor in this project presented a whole new set of HCI challenges for us. We have to step away from the conventions we normally rely on when creating digital experiences for the desktop and web as there is no mouse or keyboard, and people have a very short attention span with this thing. It's more relevant to look to smartphones and tablet computers, but obviously there are major differences between the form factor on a 4" screen with multi touch capability that you can hold in your hand and standing in front of a screen that's taller than most people and interacting with much larger gestures.

Take a look at the image below and you will understand some of the challenges we had:


So the person on the left is an average size male adult. The person on the right is a 10 year old boy. How do we make sure both can play these games? There is a limit to how high the child can reach, and while the adult can reach down and reach the lower parts of the screen by kneeling or bending over, this isn't a comfortable position to be in when playing a game. Another thing is the sheer size of the visual display. If you were watching a movie on a 72" screen, you would probably want to be standing at least 8 feet away from the screen, but in our case our users literally can't stand longer than an arms length away from the screen.

We also had to be very careful with relying to much on some of the newer touch screen conventions coming from smartphones and tablet computers. Part of this is because not all bus passengers are necessarily that touch screen savvy, but also because you don't necessarily think "touch screen" when you walk up to a giant display like this. From previous observations I've made with large touch screen installations I've found that not a lot of people aren't that comfortable with walking up to one of these screens and start interacting with it.

So we had to make the user experience very simple compared to what we usually do and we can't leverage more than roughly 50% of the screen area. The goal is that everybody should be encouraged to and able to play with these things without being hardcore gamers or super tech savvy.

The technical and practical challenges:
We started development of this project before the hardware was ready, so we started by building the project based on assumptions on how it would work and on early prototypes of the actual 72" units. We decided to build the project in Adobe AIR 2.0, since that technology gave us the opportunity to develop the project very rapidly and to share the work between a large team of developers with experience on the Flash platform. I was technical director on the project and had no less than 6 Flash developers working with me. 1 developer responsible for each game, 1 responsible for the overall UI and 1 developer to help out where ever help was needed.

All the developers had experience with building games and most had experience with touch screen devices, but I was the only one who had created AIR for touchscreen devices previously, so we had to setup a working environment where the individual developers could work and test on their own computers without having to learn too much new technologies. So I was basically responsible for setting up an architecture for them to work in where they wouldn't have to worry about connecting to the Flash Media Server for multi player communication, how to run and launch the application and how to switch between the games and the UI. Half of the team were working in Flash Builder 4, and the other half were working in FDT 3 or 4. And 2 were on Windows and the rest on Mac. So the solution was to create an environment where we used ANT to build and test each individual element in the application to allow for rapid test and deployment without dependencies on the development platform. Most of the team didn't have experience with such a cross platform approach, but everybody caught up just fine with the flow without having to spend too much time learning it.

One of the big challenges with this project has been dealing with 3G connection on the units. We don't want to bother casual bus passengers with error messages about latency issues or loss of connection, so the focus in the error handling has been to make sure the impact on the user is as small as possible. So while the Internet might be crashing in the background, the goal was to make sure the user could go on playing his or her game without noticing something is wrong. Adobe AIR offers some really good solutions for dealing with this situation, as you can do offline storage using this technology. So basically every time we get an XML response back from the backend keeping track of the score, we would store the response locally. That way, next time the unit needs to check the high score, if it fails in connecting to the backend server, it will simply fall back to the local version of the high score XML stored on the local computer. Pretty neat, and the user will never notice anything is wrong.

The high score might not be completely up to date, but there is no way the user will know, and when the internet connection comes back up, the scores will be updated with the latest data, and since it was never down for very long I doubt any users would have the time to travel to one of the other bus stops to spot the inconsistency.

There were a lot of solutions like that built into the application, and all of them were built on assumptions on how we expected the units to behave in the field, so naturally we were a little anxious once the units actually hit the field, but after a little bit of going back and forth and a very solid team effort between all the stakeholders everything worked out just fine.

On the practical side we were developing for a resolution of 1080x1920, and that meant having to get monitors that could rotate to portrait mode and connecting these to our laptops. A couple of weeks into the project we were shipped a 70" screen from Korea. One of the more unusual challenges we had to deal with was how to get this 250kg beast through our office door and how to mount it against the wall, but fortunately a couple of the developers were pretty big, so we eventually managed.

This was definitely the best looking image I've ever seen on a monitor. This unit could light up the room by itself, and seeing our layout on it for the first time was really cool. To my regret we couldn't connect our PS3 to it, so we didn't really get to test the performance of the monitor. Also it made a lot of noise when it was turned on.

Unfortunately the 70" device wasn't touch screen enabled, and we had to wait another couple of weeks until we got a unit that was touch screen enabled.

Unfortunately this unit was only 47" and while this is a pretty big screen, there is still a long way up to 72", so building the games put a lot of demand on us, as we basically had to take the game back and forth between our 24" development monitors, where we did the development to the 47" touch screen to test the game play and the 70" to check out the layout. At times the developers were standing in line to get time on the big screens, but most of the time we were able to share the screens between us. For a while I even wrote code with the 70" display as my primary monitor just to try it out, but I had to stop when my eyes felt like they were about to start bleeding. That display is very bright indeed.

All in all I'm very proud of this project, and I absolutely love getting away from the usual challenges with developing for the web and mobile and really going big. The project ended Jan 28 and the screens are now part of the ClearChannel portfolio of outdoor displays for interactive advertising.