Rock Band: Founding player research

A four-player game, tested when someone had time

Rock Band let four friends play guitar, drums and vocals together as a band, on controllers built for the game. Nobody buys a game because they have to, so it would live or die on whether players loved it.

Before I joined, developers ran playtests themselves, when they could find the time. The game was in crunch, the instruments were new, and the questions were big: how do you build a real feeling of camaraderie in a band, who controls the menus when four people are playing, and what makes drumming feel authentic?

Organise it, then make it last

Take playtesting off the developers' plates, help introduce songs to players in order of difficulty, and stress-test the guitar, drum and microphone hardware before launch. Then turn that into a practice the whole studio could rely on, game after game. I came from running Linux usability studies, so I went from watching one person struggle with a desktop operating system to watching four people argue about a band.

Research as part of how the studio worked

Bring method to playtesting

I introduced formal research processes to playtesting, recruited and managed a database of playtesters by demographic, and reported on both short- and long-term issues. I also kept an inventory of controller hardware, so developers always had working instruments to test with.

Break the hardware before players do

For Rock Band 2, player stories suggested the instruments weren't surviving real living rooms. I catalogued every instrument in the studio, with its age, production run and history, ran diary studies with community members, and simulated rough use: instruments pushed under sofas, stuffed in drawers and dropped from height.

One stubborn bug, an accelerometer misfiring, turned out to come from players reclining on the sofa, which created exactly the angle that triggered it. Adjusting the sensor fixed it. Alongside the hardware work, I organised fan wishlists with the community team and ran A/B tests of gameplay and menu revisions with new players.

Make every employee a playtester

The Beatles: Rock Band was in development for more than a year before it was announced, so we couldn't bring in outside playtesters. I asked staff to refer friends and family close to the target audience, which usually meant four non-gamers meeting for the first time in the studio lobby.

I also split the whole studio into interdisciplinary groups of five, who played the game every week. Developers shared the player's experience first-hand, and teams talked to each other more. The system ran on every project after that.

Grow the team

While the in-house team worked on The Beatles, I hired my first assistant to playtest Lego Rock Band's balance and difficulty, for players as young as six. I hired for a positive attitude, a friendly manner and an analytical mind rather than research experience, because research skills are easier to teach. Onboarding them made me rethink our methods and broaden what the department offered, and it set the template for every researcher and intern who followed.

Keep partner studios in step

Several games were built by partner studios while creative direction stayed in Boston. On Rock Band Unplugged, built by Backbone Entertainment in California, I relayed research feedback between the studios, planned milestones with both production teams, and worked with QA to pick the best build before each playtest.

Teach real instruments

Rock Band 3 added real guitars and keyboards. I recruited players across every level of musical skill, compared how well they learned from different teaching prototypes, and worked with the UI team on notation people could read at song speed. After many iterations we settled on a visual language that matched chords to distinct shapes, usable even by new guitarists.

Move fast with small teams

Smaller projects needed research on tighter loops. For VidRhythm, an iOS app built by a small team on a short timeline, feedback was usually analysed the same day, and many sessions ran alongside Dance Central 2 playtests, with participants staying a few minutes extra. For Rock Band Blitz, I analysed how new and top players' tactics changed, evaluated power-up prototypes, and ran internal tournaments with QA to test the leaderboards.

A practice that outlasted its first game

• A dozentitles supported by player research, across consoles, handhelds and mobile.
• Award-winninggames, with much of the praise for how friendly and accessible they were, even for novice and casual players.
• Every employeeplaytested weekly in interdisciplinary groups, a system that ran on every project after The Beatles: Rock Band.
• First hiretrained from scratch, and a template for every researcher and intern after.

Some of the awards the games won:

• Rock Band: Family Game of the Year, Outstanding Innovation in Gaming and Outstanding Achievement in Soundtrack at the 2008 Interactive Achievement Awards (GameSpot); Best of Show, Best Social/Casual/Puzzle and Best Hardware/Peripheral at the 2007 Game Critics Awards (Engadget).
• Rock Band 2: Best Music Game and Best Soundtrack at the 2008 Spike Video Game Awards (Co-Optimus).
• The Beatles: Rock Band: Family Game of the Year at the 2010 Interactive Achievement Awards (Kotaku).
• Rock Band 3: Outstanding Achievement in Soundtrack at the 2011 Interactive Achievement Awards (Wikipedia).