★ TouchArcade needs your help. Click here to support us on Patreon.

[LIVE DATA] from my iOS game

11-21-2011, 01:18 PM
#1
Joined: Nov 2011
Posts: 27
[LIVE DATA] from my iOS game

so I made this 2d sidescroller called Private Joe, and got featured in FreeAppADay. 15k downloads per day for a couple of days before it subsided.

Wanted to share some insight into the games from a designer's perspective. I used Playtomic to study players' behaviour.


Metric #1 - Beginnings and Deaths



Blue line: steep player drop rates right after Levels 1 and 6.

I knew immediately why most players left after Level 1 : the user flow was horrible! The missions menu was poorly designed.
Most players couldn't progress beyond Level 6 because too many died there!

Orange line: most players get killed in levels 5, 6 and 10.

Note: I excluded Level 10's deaths because it's a boss level. I shall write a separate article about boss level analytics soon

What I didn't expect was a high number of players dying in Levels 5 and 6. I had to investigate the cause of this! First thing that comes to mind is to record the x,y coordinates where players die frequently, and plot this over the level map. Enter heatmaps!


Metric #2 - Heatmaps



Some definitions:

StartPosition (Purple box) : Spawn position of player at start of level
WeaponBazooka : Collecting this unlocks the Bazooka for the player
RefuelStation: A checkpoint to emit messages ( not important here )
Ger_schutzer: Basic enemy soldier
Ger_captain: Advanced enemy soldier
Ger_tank2: Advanced enemy tank

Notice the two colorful spots in Level 6. The first (300,200) is green-bluish, signaling a lower intensity of deaths. The killer is the 2nd, more colorful one near (600,200). What could be the reason?

The enemy tank packs a powerful punch. Getting hit by a tank shell gives 3x more damage than a regular bullet from an enemy soldier. Most players die fighting the 2nd tank, because their lives were reduced too much after fighting the 1st tank. Having two health packs after the 1st tank didn't seem to help much.

To reduce death numbers, I could add more healthpacks, reduce tank damage, remove the soldiers, remove the 2nd tank, etc... the options are endless for the game designer. Without analytics, I wouldn't have considered them!


Metric #3 - Favourite weapons to buy



Appears that the rifle (blue) is the favourite weapon, followed by the pistol (orange). Ironically, these were the two weapons I spent the LEAST development time making. I poured most time into making fancy Machine Gun recoils, Grenade mechanics and Bazooka explosion effects. Bummer.

Does that mean I wasted precious dev juice doing unimportant things? No. I just had to tweak the other 7 weapons to make sure they're utilized more. This can mean: increasing firepower for the big guns, reducing the prices ( players need coins to purchase weapons ) or giving players more "trial bullets" to begin with.

So now we have like a billion options to test against. This sets the scene for a future AB or multivariate testing. To do this, we can use the "GameVars" feature in Playtomic to selectively manipulate variables in-game. AB/multivariate testing is a science which deserves it's own article: more on this in future posts...


Metric #4 - Free vs. paid




To encourage players with zero lives to continue playing, I offered them two choices:
  1. buy 8 lives for 80 coins, then respawn at the nearest checkpoint
  2. take the free life, but respawn all the way back at the starting point

"Free life" won by majority. The upside? Some people are willing to pay for lives, because it provides value: you don't need to respawn from the beginning, PLUS you won't get nagged at for at least the next 8 deaths!


PS: Let me know what other metrics you'd like to see, and if I should write other posts as well.

About me
I experiment a lot with game analytics.

My Gamasutra blog : Indie Game Analytics 101
11-22-2011, 12:58 AM
#2
Joined: Aug 2009
Posts: 568
Quote:
Originally Posted by fugufish View Post
so I made this 2d sidescroller called Private Joe, and got featured in FreeAppADay. 15k downloads per day for a couple of days before it subsided.

Wanted to share some insight into the games from a designer's perspective. I used Playtomic to study players' behaviour.


Metric #1 - Beginnings and Deaths



Blue line: steep player drop rates right after Levels 1 and 6.

I knew immediately why most players left after Level 1 : the user flow was horrible! The missions menu was poorly designed.
Most players couldn't progress beyond Level 6 because too many died there!

Orange line: most players get killed in levels 5, 6 and 10.

Note: I excluded Level 10's deaths because it's a boss level. I shall write a separate article about boss level analytics soon

What I didn't expect was a high number of players dying in Levels 5 and 6. I had to investigate the cause of this! First thing that comes to mind is to record the x,y coordinates where players die frequently, and plot this over the level map. Enter heatmaps!


Metric #2 - Heatmaps



Some definitions:

StartPosition (Purple box) : Spawn position of player at start of level
WeaponBazooka : Collecting this unlocks the Bazooka for the player
RefuelStation: A checkpoint to emit messages ( not important here )
Ger_schutzer: Basic enemy soldier
Ger_captain: Advanced enemy soldier
Ger_tank2: Advanced enemy tank

Notice the two colorful spots in Level 6. The first (300,200) is green-bluish, signaling a lower intensity of deaths. The killer is the 2nd, more colorful one near (600,200). What could be the reason?

The enemy tank packs a powerful punch. Getting hit by a tank shell gives 3x more damage than a regular bullet from an enemy soldier. Most players die fighting the 2nd tank, because their lives were reduced too much after fighting the 1st tank. Having two health packs after the 1st tank didn't seem to help much.

To reduce death numbers, I could add more healthpacks, reduce tank damage, remove the soldiers, remove the 2nd tank, etc... the options are endless for the game designer. Without analytics, I wouldn't have considered them!


Metric #3 - Favourite weapons to buy



Appears that the rifle (blue) is the favourite weapon, followed by the pistol (orange). Ironically, these were the two weapons I spent the LEAST development time making. I poured most time into making fancy Machine Gun recoils, Grenade mechanics and Bazooka explosion effects. Bummer.

Does that mean I wasted precious dev juice doing unimportant things? No. I just had to tweak the other 7 weapons to make sure they're utilized more. This can mean: increasing firepower for the big guns, reducing the prices ( players need coins to purchase weapons ) or giving players more "trial bullets" to begin with.

So now we have like a billion options to test against. This sets the scene for a future AB or multivariate testing. To do this, we can use the "GameVars" feature in Playtomic to selectively manipulate variables in-game. AB/multivariate testing is a science which deserves it's own article: more on this in future posts...


Metric #4 - Free vs. paid




To encourage players with zero lives to continue playing, I offered them two choices:
  1. buy 8 lives for 80 coins, then respawn at the nearest checkpoint
  2. take the free life, but respawn all the way back at the starting point

"Free life" won by majority. The upside? Some people are willing to pay for lives, because it provides value: you don't need to respawn from the beginning, PLUS you won't get nagged at for at least the next 8 deaths!


PS: Let me know what other metrics you'd like to see, and if I should write other posts as well.
wow, talk about in depth analysis!! pretty impressive!

11-22-2011, 01:10 AM
#3
Your analysis is interesting, thanks for sharing.

I notice the data gathering is not mentioned in the AppStore description.
I'm curious to know if the people buying the game (Private Joe) knew you were transmitting this information for analysis? eg: was it an option they could turn on/off or did you just collect it by default?

řƹȡ1
11-22-2011, 01:52 AM
#4
Joined: Oct 2011
Location: Netherlands
Posts: 127
Quote:
Originally Posted by Red1 View Post
I'm curious to know if the people buying the game (Private Joe) knew you were transmitting this information for analysis? eg: was it an option they could turn on/off or did you just collect it by default?
This, but please do share more, this was a very interesting read!

Check out my iOS game Eve of Impact and / or follow me on Twitter ;-)
11-22-2011, 05:02 AM
#5
Interesting to see such detailed statistics - thanks for sharing

Quote:
I notice the data gathering is not mentioned in the AppStore description.
I'm curious to know if the people buying the game (Private Joe) knew you were transmitting this information for analysis? eg: was it an option they could turn on/off or did you just collect it by default?
From my understanding, legally, you should have a privacy policy that's accessible from the game. It shouldn't be an issue for data that cannot conceivably be used to identify users (i.e. game stats are fine, UDIDs and locations require a privacy policy). Some analytics services collect this data anyway, so you may need to have one even if you're not collecting personal data yourself. I'm not a lawyer though, this is just my general understanding ^^;
11-22-2011, 05:37 AM
#6
Quote:
Originally Posted by Alistair Aitcheson View Post
From my understanding, legally, you should have a privacy policy that's accessible from the game. It shouldn't be an issue for data that cannot conceivably be used to identify users (i.e. game stats are fine, UDIDs and locations require a privacy policy). Some analytics services collect this data anyway, so you may need to have one even if you're not collecting personal data yourself. I'm not a lawyer though, this is just my general understanding ^^;
Legality and privacy are both relevant here, but also the principle that a game may be transmitting data via the owners ISP without their knowledge or informed consent. Or perhaps the transmission of this data may degrade the game experience, depending on how much data is being sent.

I like the tools and the analysis they enabled, but it's important the customer is aware, even it the data quantities are tiny.

řƹȡ1
11-22-2011, 09:02 AM
#7
Pretty cool. Did you manage to get the ARPPU ? That's a tough one as you might need get an id per user (even if you can't identify the user himself) to figure out the ARPPU.

And also, I wonder how did you do the heatmap ? Did you do the tracking and visualization stuff yourself or is it a feature of Playtomic?
11-23-2011, 01:11 AM
#8
Joined: Aug 2010
Location: Singapore
Posts: 98
Quote:
Originally Posted by Red1 View Post
I notice the data gathering is not mentioned in the AppStore description. I'm curious to know if the people buying the game (Private Joe) knew you were transmitting this information for analysis? eg: was it an option they could turn on/off or did you just collect it by default?
This is a very good question that brings up something I don't think gets enough attention. A huge number of apps (I dare say the majority of mid-top tier apps) use analytics tools such as Flurry that record as much or even more specific data than this player behaviour tools does, often tied directly to a device ID. I've never once seen any of these apps inform users they're doing this, nor given an option to opt out.

More here:

http://www.troyhunt.com/2011/10/secr...dont-know.html
11-23-2011, 11:34 AM
#9
I've wondered about this too. Flurry keeps posting very detailed analysis on their blog, including things like "what percentage of each age group and gender spends how much money on IAP". As a dev, that's awesome to know, but it does make me wonder how they know these things.

But otherwise, fugufish, thanks for sharing. The heatmap is especially nice to have while troubleshooting your level design.

Emeric, I just checked playtomic's feature list and looks like heatmap is their feature. Apparently, you just upload a shot of your level, send the x/y coordinates and it takes care of everything else.

I Won't Fall - an adorable new 2D platformer for iPhone and iPad

Co-Founder, Agni Studios (Facebook, Twitter)
11-23-2011, 06:23 PM
#10
Joined: May 2011
Posts: 11
Hi guys,

I'm not 100% sure whether a privacy policy is required with Playtomic but I'll look into that tomorrow.

We do not transmit or store any personally identifying information (including the device id), the closest we get to the user is an IP address and in the analytics that's converted to a country, for leaderboards and player created content it's stored as a salted hash.

I'm personally a huge fan of online privacy, that's why we support the Do Not Track header (don't think that matters on mobile, but it hopefully will) and provide block instructions for people who google how to block us.

For Emeric - we do have new stuff coming that we've been putting together with the help of some social game experts we're friends with that specifically looks at things like virality, arpu/arppu, retention and cohort analysis etc. Really cool, powerful stuff.

Playtomic - analytics & services for iOS | Unity | C++ | Android | AS3 | AS2 | HTML5