New ranking algorithm on the App Store?

Discussion in 'Public Game Developers Forum' started by fairlady, Feb 3, 2010.

  1. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    MikeSz_spokko, M of IMAK, thank you for sharing your thoughts and numbers openly here. It is starting to become more clear that mass ratings/reviews are simply an indicator of abnormally high sales volume during an app's life. In many cases, it looks like the reviews and ratings were boosted due to being available for free at some point.

    So, we'd be looking at a sliding window calculation for sales over the past several days, plus a modifier weight "bonus" to rank that may be total sales over a large period, say the previous 12 months, or the entire life of the app in the store. This sales total value is the sum of BOTH paid and free sales, MikeSz_spokko's numbers pretty much guarantee that, as Rogue Touch has sold more paid copies than Save Kitty, but has never been available for free.

    I think recent sales volume (past 3 -7 days) must be a large part of your rank still, over 50% of it at least. If it was not, all recent apps would have disappeared even if they were selling well.

    Assuming that this becomes permanent, it means that people may be able to cheat the system by switching to free for a while, gathering a hefty bonus modifier for when they switch back to paid ranking. Ugly stuff. Hope it is not true :(
     
  2. David Tiger

    David Tiger Well-Known Member

    Something strange is definitely going on. I came here searching for info because I saw Enigmo in the Top 50 games, which was very surprising. Enigmo is a game that has clearly had its run, so I was wondering what possibly could have changed that would propel it back to the top of the charts. Normally, you need to be moving well over 1000 copies a day in the US to be in the Top 50 Games.

    I did notice that it was updated on January 15, but it doesn't look like anything major. Anyway, lots of other apps moving pretty quickly.

    Somebody in another thread suggested that there may have been some national TV coverage of old app store successes. All hearsay at this point.

    Anyway, no reason for anyone to panic, but it sure is curious!
     
  3. araczynski

    araczynski Well-Known Member

    Oct 5, 2009
    1,467
    0
    36
    programmer
    omaha, ne
    i take it this implies that consumers actually care about what's in the top whatever category?

    the few times i looked through those app store top anythings, i saw a complete mess/mix of low count reviews, high count reviews, apps rated average 2 stars, and apps rated average 5 stars.

    with those kinds of 'tops', i stopped wasting my time browsing those sections.
     
  4. matt.johnson.mobile

    matt.johnson.mobile Well-Known Member

    Jun 6, 2009
    650
    0
    0
    Game Developer
    Murphy, TX
    Just curious how you locate games that interest you on the store then? I am operating under the assumption that a high percentage of consumers never bother to look beyond N&N, Whats Hot or the top 25 apps on the device.

    Thanks,
    Matt
     
  5. gabor.in.austin@gmail.com

    [email protected] New Member

    Feb 4, 2010
    4
    0
    0
    iPhone developer
    Austin, TX
    #65 [email protected], Feb 4, 2010
    Last edited: Feb 4, 2010
    I would say that word of mouth plays a large role. The sales for my "Graphing Calculator" application are always high when kids are in school, and it's really low over the weekends/spring break/holidays. I have purchased 100s of Apps, and over half of them were recommended to me by my friends, or I read about them on Twitter.

    While there are lots of people like araczynski (ignore the lists), being on the top of the app store list definitely helps. There's no reason to argue with that, and it's especially important for new applications, which need visibility.

    I've been trying to figure out the new ranking algorithm, but I'm afraid I can't add anything intelligent to this thread until I see the sales figures for Wednesday.
     
  6. schplurg

    schplurg Well-Known Member

    Perhaps Apple should have rankings for current hot apps, and a separate list for all time most popular ones.

    Take the music industry as an example. You don't see 30 year old albums (do we still use that term?) on the top charts, even if they have sold more copies over time. It would be stupid.

    Apple is so damn strange in their ways, sometimes I don't think they live in the real world at all, or any place near it for that matter..
     
  7. GlennX

    GlennX Well-Known Member

    May 10, 2009
    761
    0
    0
    UK
    Of course the sales data for yesterday still hasn't showed up. It's almost as if the sales from a day six months ago have been applied to the charts by mistake...

    I'm hoping it's a glitch
     
  8. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    In every country besides America and UK my apps are moving normally, like they would if there was no change to the ranking algorithm. It was interesting while it lasted but it looks like all the crackpot theories that said it wasn't just a glitch were wrong.

    See you again on the next Apple f*** up
     
  9. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    Not seeing any change at all for any of the 15+ apps I've been watching all day in MajicRank. Anyone else seeing changes finally?
     
  10. D-Cypher

    D-Cypher Well-Known Member

    I wonder if the change is what is effecting sales reports for Feb 3?
     
  11. I've got MajicRank too (too bad it stops working after a set number of samples and you have to delete some old ones to get it working again...)

    Anyway, On Tuesday (when all the features get changed each week) My game "The Creeps!" jumped up about 30 spots in Strategy, and maybe 40 in kids. But my game got put on the first page of "What we're playing" that day so I just assumed that was why. Two weeks ago (very shortly after I released a new version with DLC) it showed up in "What's Hot" (the games section version, not the main app store version) and it had a similar jump to about the same ranking... then after a week it moved to the second page, sales dropped, ranking dropped. Anyway, all of the seemed perfectly normal (aside from being super lucky to get two features in a row for a game that's over a year old). In the last hour it's dropped about 8 places in strategy and about 14 in kids. It has about 4000 ratings over all versions, but only 14 for the current version (a minor bug fix version just came out yesterday). I don't know if any of this helps feed the theories, but it is very confusing for sure.

    And... reports are still not up for me... I wonder if this will delay reports for today's sales as well...
     
  12. Toolism_spokko

    Toolism_spokko Well-Known Member

    May 27, 2009
    175
    0
    0
    Guys guys.. clearly you are all mistaken. Let's look at the situation here from an observers point of view shall we?

    So one day we wake up and find our apps in strange places. Some of us go WOW thats great cause our apps skyrocketed on the other hand some of us go DAMN.. and watch their precious apps drop and lose revenue.. So.. anyone seeing a pattern here?

    A major event occurs, and soon after some people are expieriencing joy while others suffering. I'm not that religious but CLEARLY, we are all dead and either in HEAVEN or HELL (sorted by developers). Don't know about you guys but I'm gonna go look for Jesus he's got to know something...
     
  13. exosyphen

    exosyphen Well-Known Member

    I am up by 42 places with Rail Adventures 3D.

    I feel sorry for all those great games made in 5 minutes that dropped out of the charts.
     
  14. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    Clearly there was something wrong with Europe in general yesterday. That's what Apple blamed the late reports on, and if you've been reading the Across Age thread, Future777 said they could not submit their game (He's from Germany I believe) and said quote:


    This is a separate incident, and likely unrelated to what we're talking about. That has been cleared up because he was able to submit, and 2/3/10 reports are in. My US sales are down 66% for Wednesday due to the massive shakeup in RPGs.

    It does seem Apple suddenly woke up and said "let's shake things up"! We have this new ranking algorithm, still in place after 2 days now. They also implemented that weird "view iTunes links" in your browser while waiting for iTunes to load up. There may be other hidden changes waiting for us to discover too.




    You admit the US and UK are still acting with this new behavior, but deny that there is a ranking change. I would suggest that you take a second and think about the size of each market. The reason why the US is affected the greatest amount, followed by the UK, and little elsewhere does not mean it is a glitch. It means that the *smaller* the app store market is in a particular country, the less movement you'll see. The US in particular is such a huge market with many millions of downloads per day, any small ranking algorithm change they make is magnified into HUGE gains or losses. In a country where you can sell a couple of copies a day and end up in the top 100 apps, there is little to amplify.



    I know that's sarcasm, but you should note all the apps made in 5 minutes each that dramatically gained position because of this... all the fake MMO point pack apps in the RPG category. And the people who made decent, "middle of the road" games that took a few months to build but were shoved way down out of sight when this older stuff, or stuff that went free for a while, was brought to the surface.

    ----------


    Ok, where does this leave us? 48 hours in, I am more inclined to believe that this is NOT a temporary ranking experiment. We might be stuck with this until they try the next idea to adjust the calculation with another weight factor.

    I'm pretty much convinced now that it is taking into account lifetime sales of the app. And that lifetime sales counts both paid and free sales. Review/ratings counts are a red herring, the apps that made the biggest gains all had huge rating counts, but that was just an indicator that they'd been downloaded an incredibly large number of times.

    MikeSz_spokko and M of IMAK have provided good data that shows paid and free downloads are being lumped together. If it was solely off paid sales the shake up we're seeing would have been less severe.

    RegularStormy, the Creeps has been in the app store for a very long time (good game too, I own it ;)), it's collected tons of sales just due to the length of availability, your regular updates, and various times being featured. If I had to guess you've probably sold over 200,000 copies by now, which gives you enough leverage to jump ranks without having gone free like some of the other apps in our study here.

    Anyway, thanks to everyone who's been sharing info and sales numbers. That's been a big help to pin down what is going on! If the charts are like this into next week then we're probably stuck with it :)
     
  15. MrBlue

    MrBlue Well-Known Member

    Sep 3, 2008
    320
    1
    0
    iPhone Developer
    Hm... if this is the new rankings algorithm, then it's a race to FREE. Awesome, pricing apps at 99c ain't good enough. Now we gotta go free. Does Apple think there are too many devs or something and it's time to drive some out of business? #$!@
     
  16. svolskiy

    svolskiy Well-Known Member

    Apr 5, 2009
    353
    0
    0
    Kiev
    I think it's a good time to change ranking algorithm in AppStore. Looks like marketing companies used previous algorithm as a hole in to Top100 for real shit applications developed by one freelancer in one week maximum.
     
  17. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    TanZen saw a 20% increase in sales overall, and a 35% increase in the US for the 3rd. The rankings have not reset themselves to normal levels in any country. If anything, TanZen's rank has gone even higher. Unfortunately, my competitor, with all the free downloads, is even higher still. So is Apple suggesting I should make TanZen free for a week, so I can move up ten more spots to have a fighting chance?
     
  18. Venan

    Venan Well-Known Member

    Jan 13, 2010
    174
    0
    0
    Obviously we'll know better in the next few days what the heck is going on, but I wouldn't necessarily assume going free is the way to go. It could simply be that the apps you are mentioning that had a free window were able to generate legitimate sales through previous loopholes in the system (i.e. free downloads helping paid rank). I think it would be pretty stupid of Apple to incent developers to not charge for their app.
     
  19. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    Honestly, that is my take-away from all this. :(

    Rather than make Tanzen free, perhaps you should switch Compression back to paid and see what that does. Less risky at the moment.

    I get the feeling they were trying for some other effects with this change, but missed this huge hole in the middle of their plan. It is possible that this ranking algorithm change would help brand new apps in the store stick around in the charts a bit longer. With a fairly strong sales start, and only days or weeks to average it across instead of months or years, a new "mildly successful" app would float in the charts a bit longer after the sales spike started to taper off.

    Apple may be trying to tame some of the volatility of that precious 3 day moving average that causes quick rises and falls when sales spike or drop off. Probably tested this new calculation on a bunch of sample app cases, and then said what the hell, lets go live with it.

    Venan, it would be great if the ones counting "free" sales in their rank held their free period back when you retained those for ranking when switching back to paid. We'd need to verify if there are any apps that got this benefit that went free for a while AFTER Apple closed that loophole. My gut says that there are, but specific examples need to be found.
     
  20. Knight

    Knight Well-Known Member

    Oct 2, 2008
    410
    0
    0
    Game Developer
    Well perhaps I can give another data point.

    7 Cities went free for 3 days in December where it racked up 150K downloads total (free). After last Tuesday when the new ranking change went into effect, it jumped from off the Top Strategy list to rank 67 or so, and it is holding steady there now.

    I do believe that the total life time downloads are being taken into account for the new ranking system, but given the current ranking of 7 Cities, I don't think going free for a limited time is that effective. I noticed that whenever I drop the price to 99 cents, I get the same effect anyway. But it all depends on the app as well.
     

Share This Page