Question regarding beta-testing and ad-hoc distro.

I always use iPhone Configuration Utility for binaries, profiles, UUID extraction, and speed of use. Just be careful *not* to lose a saved game/data as it only has install/uninstall -> quickly allowing updated binaries to be installed (replaced) with same version number.

So apart from the saves it does the same thing. You can always request access to saves via iTunes' File Sharing...but that defeats the purpose of using iCU.

 

I use testflightapp and you don't need iTunes to install or receive data, be it diagnostic data, crash logs or whatever.

 

Most that I have beta tested for have used testflight which does not require itunes. 2 developers weren't using testflight, and I asked them to, and they did and seem to like using it.

Otherwise, some have used the udid install method off of a website. You just send them your udid via email, then they add you to their list, then you go to their website and it authenticates you and allows you to download the game.

I don't believe apps really use any real time diagnostic error checking, but maybe I am wrong. I have always sent in bug reports or wishlists to the developer directly directly though email.

Anyway, beta testing is a good way for developers to get their games checked out in a through manner since TA has a lot of avid gamers, and to gain a following of fans for the game.

The most recent game I beta tested for was Wimp, and they actually implemented 5 of my suggestions, so that was cool. Feels good to have an opportunity to actively make games better.

Anyone know if the big game house developers like Chillingo, EA, or Gameloft have beta testing programs we can get on? Would be curious to see if their methods of beta testing would be different. Or maybe they do it all in house.

 

I remember years ago EA would hire local beta testers to work full time at their location in the SF Bay Area. I think I even applied once. I'd imagine they do it the same way still.

 

Well in big game company there is beta testing and playtesting.
- Beta testing is finding bugs. Tester do that all day. It's not fun, because you have to play the same game, and repeat tasks, many many times. -> you are hire full times but expect to be fired at the end of the project.
- Playtesting: friend and familly of developpers. They are invited for one day to try the game (free food).

With online game they do an open-beta, which is a kind of beta testing and playtesting.
But most game company has already done a small scale playtesting in house before an openBeta.
In that case, open beta are useful to tweak things that occured only when there is many players (matchmaking etc..). But big gameplay principle are generally already freezed.

From my point of view, the only benefit of being a tester in a big company is that you have a bigger chance to be hire for something else (game design, programmer)

 

My view on some of these definitions is a bit different.
Finding bugs for me is part of the system test (in the US they often call this QA).
Improving game design is part of the playtesting.

Both are IMHO part of a beta test. I'd even say that system testing (QA) is just labour. You can do it on your own if you have the capacity or you have a few externals helping you out (recommended). You don't need a broad audience because it is not about opinions but about bugs.

Playtesting OTOH is where you really need a broader audience because opinion matters and you want to get honest feedback. I strongly advise not doing playtesting with your family/friends only. While this is nice to get some usability tests where you can observe their behaviour, it is far from sufficient.

Often, they are excited that you - their friend, son, father, brother whatever - develops a game and their feedback will almost always be far too positive. Also, they may not be representative for your target audience (you make a game for girls and test it with your male friends, or your parents play your first person shooter, a non-gamer friend plays a hardcore game and finds it way too hard, a hardcore gamer finds your casual game too simple).

Overall, the playtesting is the real deal when working with (potential many) beta testers. They don't know you, they play a game and judge it and their feedback is honest and often more brutal. Finding the right testers is another story.

I'm even splitting the test in some internal system test where I really try to iron out all bugs (at least those I can find) and only then shove it down to beta where the target is more on questions about difficulty curve, playability, making it fun, polish, etc. This is practically the opposite of your definition.

It's hard to focus on multiple things at the same time and having beta testers focusing on bugs and playability at the same time often results in strictly bug filing with little feedback on playability.

Unfortunately, this is often the case. Testing is seen as a door-opener or as an area for low-skilled people and not as a rewarding activity. The result is that big companies often have a lot of bad personnel in testing that can't come up with good test results. I've seen so many system tests go wrong, having no test cases or following test cases by word, too much focus on different test cases and no focus on representative data variation, not knowing or understanding the expected outcome, no automation, bad regression tests, ignore non-functional requirements (performance), I've seen so many things go wrong in system test in my 25 yrs in the SW industry, maybe the only area that goes worse is requirements engineering.

Testing requires skill and a lot of organization, I wish the software industry would accept that fact.