In case you missed it, at the Game Developers Conference last week in San Francisco, Claritics hosted a session for game developers called “Art vs. Science: Fueling the Right Brain with the Left.” Moderated by our own Raj Pai, the session featured a panel discussion with heavy hitters including Bill Jackson of Zynga, Rajeev Behera of Playdom, Marcus Weichselbaum of TheBroth and Rob Carroll of Tapjoy.
Although the panelists didn’t always agree on how to go about analyzing and fine tuning your games, one thing there was no disagreement on was the hyper-critical role analytics plays in game design and development. Below is a recap of some of the questions that were asked and responses given:
So, which one is it? Is game design driven by analytics or gut instinct? When creating a new character, adding a new feature or even building an entirely new game, is it better to start from deep within your imagination or from a foundation supported by facts?
Not surprisingly, the answer seems to be “both.” Zynga’s Bill Jackson explained that the best innovations in game design tend to come from the gut, and that creative solutions tend to come from creative people. However, he added, “Sometimes your gut is wrong,” acknowledging that data must be used to either support a thesis before putting it into action or to test the thesis immediately upon launching the new game/feature to see how it works.
Tapjoy’s Rob Carroll agreed. He compared game design to warfare strategies, reminding us all that “As soon as you hit the battlefield, even the best laid plans go to hell.”
What are the top 2 – 3 metrics you’d advise game developers to keep an eye on?
Here the answers tended to be different based on the person’s role in the company, the stage of the game in question, the business objective of the game and other external factors. For instance, Marcus said that the two Key Performance Indicators he cares about – and the ones he looks at first thing in the morning and last thing at night – are revenue (ARPU) and users (Daily Active Users). If anything is amiss there, he knows he needs to dig deeper to figure out why.
Playdom’s Rajeev Behera, meanwhile, advised developers to keep their eye on Life Time Value above all else, because, as a stat that incorporates several elements all into one, it is a good reflection of the overall health of your game.
And Bill Jackson provided yet a third viewpoint, that user retention (return visits) should be your leading indicator because it shows whether people truly enjoy your game or not. Rob took this idea even one step further by suggesting that developers look at the user flow for the first-time user experience. Watch for where people are dropping off or getting stuck and fix those immediately, because you may be missing out on a huge opportunity to convert long-lasting fans.
Which platforms should developers be building on today?
“Not all platforms are created equal,” said Bill Jackson, urging developers to build their games to suit the unique characteristics of each platform they’re developing for. His advice was to deeply understand the limitations and capabilities of each platform in terms of virality, monetization, game design, etc. before you build for that platform. And if you don’t know it all from the start, use analytics to monitor critical activities very closely so you can figure it out quickly and iterate on your game.
Marcus agreed, admitting that his company, TheBroth, might have tried to build too soon on one platform before it really mastered the platform its games were already on. This mastery of platforms can take months or even years, and analytics is critical to understanding how a platform’s policies and features affect your acquisition, engagement and monetization strategies.
How does analytics factor into game development cycles?
Rob Carroll’s advice, especially to mobile developers, was to quickly launch a minimally viable product and then to use your analytics to understand key engagement metrics and iterate on the game quickly. That’s the way many of today’s most successful games were built, he argued, and the best way to get your game out there in the hands of real users so you can begin collecting data and feedback on what they do and don’t like about it.
Bill Jackson warned that that sets a dangerous precedent, however. One risk is that you may not be putting your best foot forward right from the get-go and, as the old adage goes, you only have one chance to make a first impression. It has also become increasingly difficult to get your game discovered in the new competitive environment, so launching a half-baked game might not be in your best interest. He did agree with Rob, however, in that he urged developers to iterate very quickly after a game’s initial launch in order to understand how to improve game design and mechanics.
Rajeev Behera also advised developers to A/B test as much as they possibly can within their games. Although A/B testing can be fairly complicated and expensive, it is the best way to determine without question what features or mechanics users like best, so that you can make smarter, more informed decisions about product updates and go from there.
What does the future of analytics look like?
Raj ended the discussion by asking each of the panelists what they’d want analytics services to look like in the future, and to a man each of them agreed that they want analytics to simplify their lives by automating as much of the analytics process as possible.
“It would be great if my analytics provider could figure out which of the two new features I launched worked better and make the adjustment on the fly for me,” said Marcus.
To that, all we can say is, stay tuned.
You can be the first to comment!
The Claritics blog offers insights and analysis on the rising social commerce wave and how app developers can leverage social intelligence for their Reach, Retention and Revenue campaigns.