★ TouchArcade needs your help. Click here to support us on Patreon.

Preparing for beta tests

07-22-2012, 07:01 PM
#1
Joined: Jan 2012
Location: Warsaw, Poland
Posts: 34
Preparing for beta tests

Hi there!

I'd like to ask you for an advice (from developers and beta testers): what is the best way to prepare beta tests.

I read several threads on this forum (even from 2008!) and I discussed this topic with a professional developer (but he works with AAA titles, so indiedev(s) can have different rules), so I have "an ideal plan" in my mind

Because I'd like to start with beta tests soon (in 2-4 weeks) I'm trying to prepare a proper environment for this process, but I don't know if some of items are all right:

1. Builds: I'm going to prepare TWO builds:
a) "Full" build for testers who are my friends
b) "Lite" build for unknown testers - all most the same functionality which the Lite version of NanoLooper is going to have
(even if the game is somehow cracked/hacked during beta tests, they will not have the full functionality)

2. Builds distribution: I'll use TestFlight for ad hoc builds.
I know that developers can distribute builds by they own, but from my perspective this is the easiest way.

3. Reporting issues: I've already set up bug tracking system - Mantis Bug Tracker.
I'd like to keep all issues in one place - all testers can see bugs reported by others (avoiding duplicates!), and they'll be able to track if something is fixed. But maybe this is too complicated for some guys?

4. General feedback: It can be done in a) MantisBT (by setting up special issue's Category)
b) or in any survey/poll system (like LimeSurvey or LittlePoll)
If I go with MantisBT, all testers will be able to see others' feedback

5. Testers community: I've crated a forum for NanoLooper with sub-forum for testers.
I don't know if this is too much or not? Maybe testers won't have time for this? I can create accounts for them but there is a chance they don't want to go over there, log in and post any feedback.

6. Beta testers: I'll start a thread in Upcoming iPad Games or in Upcoming iPhone and iPod touch Games, putting description, screenshots and probably video.
a) after a while I'll ask for volunteers in this thread
b) I have a private list of experienced beta testers on TA (list was created during my investigations through TA threads) - so I'll try to send PMs to them and ask about help

7. Test scenarios: I'm thinking about several test scenarios to prepare, and distribute them (via e-mail/testers' forum/MantisBT) to all beta testers.
This can be very helpful to check if any crucial functionality is stable (or not)

8. Cycles: I'd like to have at least 2 cycles, but if I have enough time, and beta testers will be still available, I'll have 3 or even 4 cycles.
Two cycles are "must be" - I'd like to ask testers for verification of all fixes (MantisBT can show its advantage in this area)


So... does anyone have other ideas or any suggestions?


-=Agnes=-
07-22-2012, 07:21 PM
#2
Joined: Jun 2012
Posts: 151
All of these sound easy to do and not complicated at all, including the bug reporting.

Though it sounds like you have a little trust issues which is reasonable. Maybe having some prerequisites for beta testers like 50-100 posts in TA, previous beta testing experience, or anything else I can't currently think of can make you feel safer?

Just not sure how much beta testing users can do if they're testing a lite version of the game.
07-23-2012, 04:09 AM
#3
Joined: Sep 2009
Location: Shanghai, China
Posts: 2,032
Mostly you should give out some rewards to the testers.

Extensive Knowledge of CBT. Just send me a PM!
07-24-2012, 04:39 PM
#4
Joined: Jan 2012
Location: Warsaw, Poland
Posts: 34
Thanks guys for answers!

Quote:
Originally Posted by iSky View Post
Though it sounds like you have a little trust issues which is reasonable. Maybe having some prerequisites for beta testers like 50-100 posts in TA, previous beta testing experience, or anything else I can't currently think of can make you feel safer?
I read a lot about piracy (when I was investigating anti-piracy tricks) and I'm ready to be fu**ed by hackers/crackers, but... I'd like to avoid this situation during beta tests.

I'm still thinking about the "lite" version for beta tests. Maybe I'll take into consideration postcount to decide who is going to get the full/lite version.

Quote:
Originally Posted by saiyukialex View Post
Mostly you should give out some rewards to the testers.
Understood, but as I know there are some testers (on TA) who usually want to "help" and nothing more

Is there something like a "good practice" to re-distribute through all beta testers (or at least those who gave the most valuable feedback) the final build after Apple approves/reviews it?
(developer can use the same build which was sent to Apple for an approval process)

-=Agnes=-
07-25-2012, 05:52 PM
#5
Joined: Jun 2012
Posts: 151
Quote:
Originally Posted by AgnesDev View Post
Thanks guys for answers!



I read a lot about piracy (when I was investigating anti-piracy tricks) and I'm ready to be fu**ed by hackers/crackers, but... I'd like to avoid this situation during beta tests.

I'm still thinking about the "lite" version for beta tests. Maybe I'll take into consideration postcount to decide who is going to get the full/lite version.



Understood, but as I know there are some testers (on TA) who usually want to "help" and nothing more

Is there something like a "good practice" to re-distribute through all beta testers (or at least those who gave the most valuable feedback) the final build after Apple approves/reviews it?
(developer can use the same build which was sent to Apple for an approval process)

-=Agnes=-
I recently found out Apple sends the developer promo codes once the game is approved and set to release. You could give the beta testers a promo code to your game. Besides promo codes, I don't think there is any other option except the beta testers buying the game.
07-26-2012, 01:12 AM
#6
Joined: Jan 2012
Location: Warsaw, Poland
Posts: 34
Quote:
Originally Posted by iSky View Post
I recently found out Apple sends the developer promo codes once the game is approved and set to release. You could give the beta testers a promo code to your game. Besides promo codes, I don't think there is any other option except the beta testers buying the game.
I will send promo codes to some sites asking for a review (e.g. TA editors)

But I can compile the game once again using the same provisioning profile which I used for compilation for Apple approval process.

But the question is: if those guys will have an access to the same leaderboards/achievements as players who buy a game in App Store.

I wanted to use Ad Hoc provisioning profile but unfortunately in this case the game connects to sandbox Game Center server

-=Agnes=-