Support our Sponsors:

Go Back   Touch Arcade > Developer Discussion > Public Game Developers Forum

Reply
 
Thread Tools Display Modes
  #41  
Old 10-18-2014, 06:36 PM
WalterM WalterM is offline
Junior Member
iPad (4th Gen), iOS 7.x
 
Join Date: Aug 2014
Posts: 15
Default

Hi Pixelosis, thanks for your comments. It is a dead thread now as far as we are concerned. So my last comment on it as a closure.

You are correct that the intent was to create a buzz but not for downloads of the app as you have suggested. Doubt anyone is naive enough to think posting in a few developer forum will move the needle in downloads even if the target audience for the app is developers. :-)

The goal of maintaining this discussion for a while was to keep it visible long enough for people to stumble on it. We have achieved our purpose of getting visibility for crowd-sourcing the analysis and/or getting additional data points from other developers for their own set of apps as they instrument them. People have privately provided some insightful comments/analysis and suggestions on the data.

Individual developers are justifiably averse to discussing specifics in public especially if it might appear unfavorable to Apple who control the fate of their apps. As you might know well, public forum threads like this devolve into conspiracy theories or thread-crapping with tangential or argumentative opinions just for the sake of being contrary.

As I have mentioned in an earlier post, we have received a number of tips (some of them mentioned above on observations not all of which we can follow up on) and a few developers are involved in collecting data of their own. This data gathering takes time to get enough data for statistical significance. That is better than we had hoped for. We will get to the bottom of this sooner or later and see where it leads to.

Much work to be done with the collected data while this strange behavior continues to happen.

Thanks again for your help in keeping the thread alive at the top of the list. It has been a pleasure interacting with you.
Reply With Quote
  #42  
Old 10-19-2014, 02:19 AM
Jez Hammond Jez Hammond is offline
Banned
 
Join Date: Oct 2012
Posts: 50
Default

Quote:
Individual developers are justifiably averse to discussing specifics in public especially if it might appear unfavorable to Apple who control the fate of their apps.
Yeah they wouldn't want to do that, it's bad for the health.

Quote:
As you might know well, public forum threads like this devolve into conspiracy theories or thread-crapping with tangential or argumentative opinions just for the sake of being contrary.
I find this quite condescending being one of the other handful of posters in your thread. Good look in your quest to find the Higgs particle hiding with a wallpaper scraper.
Reply With Quote
  #43  
Old 10-19-2014, 02:24 AM
Jez Hammond Jez Hammond is offline
Banned
 
Join Date: Oct 2012
Posts: 50
Default

Quote:
Originally Posted by Jez Hammond View Post
Good look in your quest to find the Higgs particle hiding with a wallpaper scraper.
*luck (ironic typo)
Reply With Quote
  #44  
Old 10-19-2014, 03:06 AM
WalterM WalterM is offline
Junior Member
iPad (4th Gen), iOS 7.x
 
Join Date: Aug 2014
Posts: 15
Default

Quote:
Originally Posted by Jez Hammond View Post
I find this quite condescending being one of the other handful of posters in your thread. Good look in your quest to find the Higgs particle hiding with a wallpaper scraper.
Jez, my apologies for what my words "threads like this" may have implied to you and other participants. All of my interactions earlier with you in this thread should show clearly that I was nothing but respectful to your comments as well as others. "Topics like this" may be a better phrasing of what I intended to say.

What I meant to say in that poorly worded statement was that trying to come up with speculative theories by discussing them in a public forum is futile because it devolves into such content (which you can see in many of the threads here that attempts to discuss speculative or unsubstantiated theories on the ratings/reviews and manipulations). This is why I tried to avoid speculations on who and why of this pattern or encouraging such, rather just focusing on the objective data analysis process to entice people to participate. It turned out to be more of a private participation for reasons you do understand.

Fortunately, it has been somewhat easier than detecting the elusive Higgs particle with a wallpaper scraper (love the metaphor!), more like deciphering intentional graffiti left on a wall. :-)

Peace.
Reply With Quote
  #45  
Old 10-19-2014, 05:52 AM
Pixelosis Pixelosis is offline
Senior Member
 
Join Date: Jan 2013
Posts: 137
Default

OK, well before you close it definitely,

Quote:
Originally Posted by WalterM View Post
Leaving no doubt that there are things going on within the App Store that is affecting the app, the user ratings for the app completely froze (no ratings left during this week were added to the app ratings) last week for exactly a week
Probation period? Who knows.

Quote:
and 29 ratings appeared in the same sweep this morning after all of them disappeared for a short period. Perhaps the algorithm/entity "managing" our app ratings took a vacation. :-)
This already occured the 6th of August.

Quote:
Code:
Oct 1 08:30:01 |18 9 1 0 1 |4.6104
Oct 1 16:30:01 |0 0 1 0 -1 |4.61142
and the +1, -1 pattern has resumed as you can see above

Jez, regarding your comment about sync problems, the above is not a sync problem. What we are measuring is more like packet loss than packet delay and not affected by delays when measured and averaged over time.

Also regarding your comment on other developers not commenting, perhaps not publicly but we have been quite heartened by the private communications we have received that has helped us progress on this.
Why do they remain private?
Why not post publicly and simply retain some sensitive information?
How in the hell pointing out that you've suffered -1s in some odd fashion as well be considered an attack against Apple's interests or whatever?

Quote:
For example, a helpful and smart reader of a forum where we posted, imported the data to an Excel spreadsheet to visualize it that showed a better way to look at it than the raw numbers and combined with the tip from another that there may just be obfuscating behavior, allowed us to process the data that led to a breakthrough.

As a sneak preview, if we plot the average ratings over the lifetime of that log including all ratings registered, we get the following graph



The downtrend is visible but not much details of how exactly it happens.
Hold on.
First of all, that graph shows peaks at ~4.375. But there's not such value in the log at the corresponding date. It also shows four peaks above the 4.6350 mark but there simply are not that many in the log.
Is the graph a curve that arcs beyond the points (up or down), as if using beziers or something similar?

Secondly, the downtrend was already occuring in the old log, even before the odd -1s started to appear. Your app's average rating starts at 4.660, sometimes a bit more, sometimes less.
By the end of it, the average ratings hover at 4.630~4.629.
Nothing new here.
The lowest average rating found in the freshier log happens to be 4.60591 and is collected on the 5th of October, yet surrounded by many averages in the 4.7xx~4.8xx ballpark. You actually lost more decimal points throughout the entire old log's course than during the new one, and considering the length of the older one, it seems clear that it covers a shorter timeframe.
In other words, your app's ranking sank more on a shorter and earlier period of time, before the -1s, than after, during a long period of time while exposed to a phenomenon you consider to be detrimental to your app's ranking.

And that's hardly the only point that is completely at odds with your claims and conclusions.

Quote:
However, if we filter out the ratings from the single account moving from rating to rating at almost every sweep with no net additions or removals, we get a much clearer picture like the following:
I'm not sure how to understand this sentence.

Quote:


This allowed us to notice some very interesting repeating patterns (in the shaded areas) based on what appears to be a triggering activity and a good idea of how the ratings declined algorithmically (and clustered) during that time. The details of what this reveals will be the center-piece of our report.
I'm afraid there's nothing to see. There's nothing glaring to be seen on this graph.
The shaded areas just look as random as they get to me.

Quote:
Tips received from forum members have indicated a lot of suspicious patterns they have noticed for themselves before I posted this thread, not all of which we will be able to explore or verify on our own.
Why don't they post openly about it? They don't even have to share the name of their apps.

Quote:
But even allowing for some paranoid suspicions, there appears to be enough of a feeling amongst more than a few that what is happening in the App Store ratings is not strictly on the level both for apps doing well and apps who mysteriously lose downloads/ratings.

We are just not sure what is real and what is not real in the App Store any more!

Some of the more credible observations (which can be verified in a few instances) and unrelated to our logs we have received from readers include:

1. App rating totals that happen to go up in each sweep with such regularity (seems almost always multiples of 2) that it seems automatic. Reviews appear like a quota system with regularity (e.g., exactly 2 or exactly 3 reviews per day) often all in the same sweep. Apps go through this phase for a fixed period and then suddenly nothing even if nothing else has changed.
Recurring low amounts of ratings are fairly common. Things stagnate at some point, I wouldn't see any unnatural pattern.

Quote:
2. One liner generic reviews that appear with regularity for certain apps which appear to be automatically generated and very often connected to the quota in 1 above. Looking at the review history of the names associated with these reviews (in iTunes) show a pattern of reviewing apps with similar review quota in operation and these reviewing accounts all appear to have similar delays between reviews (typically many months). There is a strong possibility that many of these are all fake accounts and come from a large pool of accounts used to create those reviews. We are working on a script to crawl through these kinds of short reviews and form a graph of the apps related via such reviews (and their rankings) just out of curiosity.
This is more interesting, especially if those one liners tend to sound rather negative. Examples would be nice though.

Quote:
3. Atypical rating activity clustered around Tue evening/Wed early mornings or month ends. Not sure what happens in the App Store on Wednesdays or at the beginning of the month. Coincidentally, the freeze for our app reported above also happened Wed-Wed.

It gets curiouser and curiouser...
I don't see such facts from your logs. The greater moments of rating posting are rather random. There were some happening on Wednesday, OK, but, others on Friday, some others right during Sat-Sun, etc.
I also checked the months' ends, there are no clear visible peaks in rating numbers.
Reply With Quote
  #46  
Old 10-19-2014, 03:13 PM
Jez Hammond Jez Hammond is offline
Banned
 
Join Date: Oct 2012
Posts: 50
Default

Quote:
Originally Posted by WalterM View Post
Jez, my apologies for what my words "threads like this" may have implied to you and other participants. All of my interactions earlier with you in this thread should show clearly that I was nothing but respectful to your comments as well as others. "Topics like this" may be a better phrasing of what I intended to say.

What I meant to say in that poorly worded statement was that trying to come up with speculative theories by discussing them in a public forum is futile because it devolves into such content (which you can see in many of the threads here that attempts to discuss speculative or unsubstantiated theories on the ratings/reviews and manipulations). This is why I tried to avoid speculations on who and why of this pattern or encouraging such, rather just focusing on the objective data analysis process to entice people to participate. It turned out to be more of a private participation for reasons you do understand.

Fortunately, it has been somewhat easier than detecting the elusive Higgs particle with a wallpaper scraper (love the metaphor!), more like deciphering intentional graffiti left on a wall. :-)

Peace.
Ok Walter sounds fair enough with the misunderstanding, I think you swallowed a dictionary though because raising 'tangential' issues and potential thread-contributor-abuse seem to conflict - so it was natural for me to listen which is the case. Indeed you have politely dismissed my theories - which I figured the thread was requesting so as to eliminate any by recreating some localised re-scrape test (for example).

I get it all the time too though so I'm happy to see what might have been at best touché was merely a context issue after all.

I think people should be free to discuss experiences in public, instead of letting them always go away. If people commented on my old glitch then perhaps a "probably an overly active auto-censor was incorrectly confirmed" would be fine from anyone in the public house. But I think like the 23% thing (for example) it is not auto corrected back...oh tangential, though I think before your reply the thread was feeling a little military, it is yours to set as you wish of course.

Ha yes that metaphor would make a great cartoon! Hm about the also mentioned a few times 'fingerprints' (as I described them), the ones I was getting by superimposing the graph over itself at various power-of-two levels, seemed to be repeated too often...BUT then I realise that what else is this kind of data going to look like and I didn't bother posting the layers. I still think this is all too negligible to worry about but you say you're on to something.

Good closure. Keep us posted with any result or not. Peace.
Reply With Quote

Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Copyright 2012, TouchArcade.com, LLC.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Copyright ©2008 - 2011, TouchArcade.com. Privacy Policy / DMCA Copyright Agent