New ranking algorithm on the App Store?

Discussion in 'Public Game Developers Forum' started by fairlady, Feb 3, 2010.

  1. fairlady

    fairlady Well-Known Member

    Dec 31, 2008
    336
    0
    0
    Owner: Fairlady Media
    Raleigh, NC
    #1 fairlady, Feb 3, 2010
    Last edited: Feb 3, 2010
    The word on the street today is that there's a new ranking algorithm on the App Store. Since we're seeing a lot of games that were popular awhile ago suddenly break into the top ranks again, devs are hypothesizing that a longer timeframe is being used to calculate rank. For example, total sales / total time on the App Store.

    Our most popular game, Spazzle (which has been out for a year), jumped up 28 spots in Family Games today. Our other games, which have not been out as long, have not seen similar jumps upward.

    Have any of you seen changes in your App Store ranking today?
     
  2. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    #2 bravetarget, Feb 3, 2010
    Last edited: Feb 5, 2010
    Don't mean to botch your excitement, but it's not a new ranking algorithm, its just a glitch. One that we've seen in the past, and it will take a moment to address and go back to normal...just as it did last week.

    edit: sp
     
  3. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    I'm not sure it's a glitch. The previous glitches involved completely random apps shooting to the top. These are very specifically older apps (the older, the more significant the jump), with a lot of ratings, moving up for no reason at all.
     
  4. MikeSz_spokko

    MikeSz_spokko Well-Known Member

    May 27, 2009
    594
    0
    16
    our Save Kitty (a year old app) suddenly, completely out of a blue, appeared in top 50 arcade games. without any "warning", no reviews, nothing. just appeared there and is right in front of Canabalt as we speak

    not that we complain about it ;)
     
  5. fairlady

    fairlady Well-Known Member

    Dec 31, 2008
    336
    0
    0
    Owner: Fairlady Media
    Raleigh, NC
    I guess only time will tell whether this is a glitch or not...

    But if it isn't there are two pretty significant ramifications. If they're using all-time sales as a factor, it will be probably be harder to 1) break into the top100 lists (especially if your game does not do well right away), and 2) use advertising to move up the rankings.
     
  6. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    Yes, as we've been analyzing on Twitter for the last four or five hours it definitely appears linked to the number of reviews and/or ratings an app has. Several junk apps like the iMob 30 point pack jumped over 180 places, from invisible on the store to the #3 RPG overnight!

    Still haven't determined the exact nature of this modifier... Appears it could be linked to reviews over the lifetime of the app compared to the total sales of that app. But there is something else in there I can't quite see yet.

    It will be very interesting to see if it is permanent or someone at Apple messing around.
     
  7. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    It's a glitch. What sort of sense does that make to apply amount of ratings to a rank? or amount of all-time sales to a rank? Think about how impossible it would be for new apps to get exposure at that point...

    Here is a solution to get a better idea of how your app is selling during this scrambled phase:

    Go to iPad Apps, iPhone Apps, Deals and Discovery at App Shopper and look up the top 100 grossing apps in your category, look at them in relation to other same-tier apps.
     
  8. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    We're not saying it makes sense. We're saying that we're seeing a pattern. All the other glitches have been completely random. This glitch does not have random results.
     
  9. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    To answer your questions- It doesn't make sense, but mathematically and analytically that appears to be exactly what Apple is doing today. It's certainly not a "glitch", but an actual change of the ranking algorithm. Whether they are just experimenting and will flip things back to normal in a day or not, who knows....

    Why don't you take your advice and check this out:
    http://appshopper.com/bestsellers/games/paid/?sort=change&dir=desc

    That should be convincing enough. For those that don't follow the link:

    Tap Tap Revenge Classic was unranked, jumped all the way to slot #10, which shouldn't be on the board anymore, except that it happens to have about 270,000 ratings!
    Enigmo was unranked, jumped all the way to slot #46, over 26,000 ratings.
    Wild West Pinball was unranked, jumped in to #56, over 70,000 ratings.

    I could go on and on. None of these guys had their sales go up 1000x overnight. It's a very specific effect, and older apps/games with more reviews (or ones that went free and back to paid, gaining tons of reviews) have all made tremendous gains in their categories.

    Of course, the sky is not falling just yet, because they may just change it all back tomorrow. But if it's still like this next week or next month, it will be very interesting to see what happens.

    Edit- Thanks LittleWhiteBear, you said it before I did, and in fewer words! :D
     
  10. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    I was getting the feeling people were starting to analyze how the "new ranking" algorithm works. The point I'm trying to express is nobody should analyze patterns or possible ranking formulas when it's going to change back to normal soon enough.
     
  11. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    #11 bravetarget, Feb 3, 2010
    Last edited: Feb 3, 2010
    Sorry, I looked around for 2 seconds and found a loophole to your theory, fast food calorie counter in healthcare & fitness skyrocketed in rank today, with only 1000 ratings? Not to mention, they are behind me in top grossing.

    These glitches that Apple experiences are never "random" so to speak, only their occurrences are.
     
  12. nattylux

    nattylux Well-Known Member

    Sep 17, 2008
    1,151
    5
    0
    Washington, DC
    @bravetarget Whether this change reverts or not, the actual rankings seen by the customers are much more important than rankings listed on some website nobody looks at. The apps that just jumped into the top 100 due to this glitch/not-glitch are going to see a massive spike in sales today - and vice versa.

    Also, the App Store is not set in stone. In the times it has changed in the past, it was just like this - something just changes, and it's up to us to figure out what happened. We're not going to see an official announcement about this.

    Given that we are detecting some patterns in the change and it's not just completely random, I personally think this isn't a glitch. But only time will tell.
     
  13. goldglover411

    goldglover411 Well-Known Member

    Apr 11, 2009
    2,436
    1
    0
    ipod touch game reviewer
    USA
    pocket god jumpped
     
  14. bryanmitchell

    bryanmitchell Well-Known Member

    Apr 18, 2009
    246
    1
    0
    My game, "Geared" jumped from 65 in Puzzle apps to 37 at about 3AM last night.
     
  15. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    Skyrocketed? It went up 7 places. We're talking about apps and games that were buried for all time outside of the top 100, possibly outside of the top 500 in some cases, that have come up out of nowhere.

    Why don't you look at "Easy Relax" currently at #13, 2 spaces behind your example, that was unranked for ages until today. It has 10975 reviews. There's plenty more amazing jumps in there to look at in your category as well- aSleep, Gym Buddy, Weight Tracker, Vibrating Massager, etc.

    There's a pattern here that can't be ignored yet. As Nattylux points out ranking algorithms have changed before and probably will change again. We can only hope this is a change that they press the UNDO button on quickly, as it really makes no sense and only opens things up to obvious abuse.

    A few new responses I had to check for accuracy:
    Goldglover411 - Yep, Pocket God jumped up as well, and they have 93,000+ ratings.
    BryanMitchell - I see you have over 8300 ratings. The game "Chopper" has about 8500 ratings and it jumped back up into the charts overnight, 40 something places.
     
  16. Little White Bear Studios

    Little White Bear Studios Well-Known Member
    Patreon Silver

    Aug 27, 2008
    2,572
    0
    0
    But it's fun! :p
     
  17. CommanderData

    CommanderData Well-Known Member
    Patreon Indie

    Yes it it :D

    And I think I'm starting to understand the "reasoning" behind this. We used to be able to issue an update to an existing app and gain exposure. They took this away. Devs complain that there is no visibility for older apps unless you're on a Top 100 list. Devs are right. Apple spends some time thinking about the problem:

    Apple Programmer #1: Hey, why don't we apply a fudge-factor to ranking based on the age of the app, you know, how long it's been in the store? That should give older apps a fighting chance to be seen.

    Apple Programmer #2: Nah, bad idea, as that will just allow any old "crapps" to float to the top because they have been submerged for ages.

    Apple Programmer #1: Ugh, nice visual, but you are right. Hey, why don't we take the app's age and incorporate the average rating?

    Apple Programmer #2: Nah, that won't work either because we let people rate on delete so everyone gets more 1 star ratings. I keep that one in there just for kicks.

    Apple Programmer #1: Ok, how about we take the amount of time it's been in the store and the NUMBER of ratings, whether they are good or bad. If it's got lots of ratings, a lot of people must have been using it at some point!

    Apple Programmer #2: Yeah, that's good enough. Let's just change this calculation right here and not tell anyone. What could possibly go wrong?
     
  18. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    I agree, it's all very exciting figuring out things Apple doesn't tell us. I'm simply the one who provides a counter-point. For instance, good observation on Easy Relax; however, that still leaves plenty of loopholes. After you investigate why Easy Relax shot up however many places, you have to determine why others with similar standings did not shoot up with it. There are so many apps that didn't get the same changes made to them, even though they have the same characteristics; hence, loophole!

    Unless there is a solid ranking formula (like the previous one (weighted/7days)) that makes sense for a small handful of apps to sky rocket in rank, then I'm sticking with the notion that it is a glitch similar to what we've seen in the past.

    Also, for the top 25 of Healthcare & Fitness, going up 7 ranks in a couple hours is definitely a skyrocket.
     
  19. simplymuzik3

    simplymuzik3 Well-Known Member

    Aug 12, 2009
    342
    0
    0
    When you say that apps jumped because of their time on the store and review amount, is this TOTAL reviews or just reviews for the CURRENT VERSION?
     
  20. bravetarget

    bravetarget Well-Known Member

    Sep 14, 2009
    330
    0
    0
    I'll be the first to propose a solid counter-point to my original counter-point:

    We all know the common ranking formula of a weighted average over a 7 day window:

    ((4 * D1) + (3 * D2) + (2 * D3) + D4 + D5 + D6 + D7)/13 = X
    Where
    D1 = Sales in the past 24 hours from now.
    D2 = Sales between 25 and 48 hours ago.
    D3 = Sales between 49 and 72 hours ago.
    etc.


    The higher X is, the higher your ranking is.

    Well, possibly they had moved that window for your weighted average from your release date (or since a change from free to paid/vice versa) until now, as opposed to the last 7 days. This is something that seems to make more sense giving the current standings.
     

Share This Page