so I made this 2d sidescroller called Private Joe, and got featured in FreeAppADay. 15k downloads per day for a couple of days before it subsided. Wanted to share some insight into the games from a designer's perspective. I used Playtomic to study players' behaviour. Metric #1 - Beginnings and Deaths Blue line: steep player drop rates right after Levels 1 and 6. I knew immediately why most players left after Level 1 : the user flow was horrible! The missions menu was poorly designed. Most players couldn't progress beyond Level 6 because too many died there! Orange line: most players get killed in levels 5, 6 and 10. Note: I excluded Level 10's deaths because it's a boss level. I shall write a separate article about boss level analytics soon What I didn't expect was a high number of players dying in Levels 5 and 6. I had to investigate the cause of this! First thing that comes to mind is to record the x,y coordinates where players die frequently, and plot this over the level map. Enter heatmaps! Metric #2 - Heatmaps Some definitions: StartPosition (Purple box) : Spawn position of player at start of level WeaponBazooka : Collecting this unlocks the Bazooka for the player RefuelStation: A checkpoint to emit messages ( not important here ) Ger_schutzer: Basic enemy soldier Ger_captain: Advanced enemy soldier Ger_tank2: Advanced enemy tank Notice the two colorful spots in Level 6. The first (300,200) is green-bluish, signaling a lower intensity of deaths. The killer is the 2nd, more colorful one near (600,200). What could be the reason? The enemy tank packs a powerful punch. Getting hit by a tank shell gives 3x more damage than a regular bullet from an enemy soldier. Most players die fighting the 2nd tank, because their lives were reduced too much after fighting the 1st tank. Having two health packs after the 1st tank didn't seem to help much. To reduce death numbers, I could add more healthpacks, reduce tank damage, remove the soldiers, remove the 2nd tank, etc... the options are endless for the game designer. Without analytics, I wouldn't have considered them! Metric #3 - Favourite weapons to buy Appears that the rifle (blue) is the favourite weapon, followed by the pistol (orange). Ironically, these were the two weapons I spent the LEAST development time making. I poured most time into making fancy Machine Gun recoils, Grenade mechanics and Bazooka explosion effects. Bummer. Does that mean I wasted precious dev juice doing unimportant things? No. I just had to tweak the other 7 weapons to make sure they're utilized more. This can mean: increasing firepower for the big guns, reducing the prices ( players need coins to purchase weapons ) or giving players more "trial bullets" to begin with. So now we have like a billion options to test against. This sets the scene for a future AB or multivariate testing. To do this, we can use the "GameVars" feature in Playtomic to selectively manipulate variables in-game. AB/multivariate testing is a science which deserves it's own article: more on this in future posts... Metric #4 - Free vs. paid To encourage players with zero lives to continue playing, I offered them two choices: buy 8 lives for 80 coins, then respawn at the nearest checkpoint take the free life, but respawn all the way back at the starting point "Free life" won by majority. The upside? Some people are willing to pay for lives, because it provides value: you don't need to respawn from the beginning, PLUS you won't get nagged at for at least the next 8 deaths! PS: Let me know what other metrics you'd like to see, and if I should write other posts as well.
Your analysis is interesting, thanks for sharing. I notice the data gathering is not mentioned in the AppStore description. I'm curious to know if the people buying the game (Private Joe) knew you were transmitting this information for analysis? eg: was it an option they could turn on/off or did you just collect it by default?
Interesting to see such detailed statistics - thanks for sharing From my understanding, legally, you should have a privacy policy that's accessible from the game. It shouldn't be an issue for data that cannot conceivably be used to identify users (i.e. game stats are fine, UDIDs and locations require a privacy policy). Some analytics services collect this data anyway, so you may need to have one even if you're not collecting personal data yourself. I'm not a lawyer though, this is just my general understanding ^^;
Legality and privacy are both relevant here, but also the principle that a game may be transmitting data via the owners ISP without their knowledge or informed consent. Or perhaps the transmission of this data may degrade the game experience, depending on how much data is being sent. I like the tools and the analysis they enabled, but it's important the customer is aware, even it the data quantities are tiny.
Pretty cool. Did you manage to get the ARPPU ? That's a tough one as you might need get an id per user (even if you can't identify the user himself) to figure out the ARPPU. And also, I wonder how did you do the heatmap ? Did you do the tracking and visualization stuff yourself or is it a feature of Playtomic?
This is a very good question that brings up something I don't think gets enough attention. A huge number of apps (I dare say the majority of mid-top tier apps) use analytics tools such as Flurry that record as much or even more specific data than this player behaviour tools does, often tied directly to a device ID. I've never once seen any of these apps inform users they're doing this, nor given an option to opt out. More here: http://www.troyhunt.com/2011/10/secret-ios-business-what-you-dont-know.html
I've wondered about this too. Flurry keeps posting very detailed analysis on their blog, including things like "what percentage of each age group and gender spends how much money on IAP". As a dev, that's awesome to know, but it does make me wonder how they know these things. But otherwise, fugufish, thanks for sharing. The heatmap is especially nice to have while troubleshooting your level design. Emeric, I just checked playtomic's feature list and looks like heatmap is their feature. Apparently, you just upload a shot of your level, send the x/y coordinates and it takes care of everything else.
Hi guys, I'm not 100% sure whether a privacy policy is required with Playtomic but I'll look into that tomorrow. We do not transmit or store any personally identifying information (including the device id), the closest we get to the user is an IP address and in the analytics that's converted to a country, for leaderboards and player created content it's stored as a salted hash. I'm personally a huge fan of online privacy, that's why we support the Do Not Track header (don't think that matters on mobile, but it hopefully will) and provide block instructions for people who google how to block us. For Emeric - we do have new stuff coming that we've been putting together with the help of some social game experts we're friends with that specifically looks at things like virality, arpu/arppu, retention and cohort analysis etc. Really cool, powerful stuff.
thanks, I had lots of fun looking at the heatmap results. So much to be deduced from the heat patterns!
True... especially coz the 'hard parts' become invisible to devs after playing it continuously for a few months . We've wanted to try such analytics in our games but haven't gotten around to doing it yet, which is why I found your post very interesting.