I start with a roughly planned concept. The concept includes at least the game mechanics. Some kind of story idea. And some basic decisions as 2D or 3D / multiplayer or single player.
Then I implement the mechanics and check how much fun it is to play. Point out where the problems are. These implementation uses proxy graphics and no animation.
After testing I refine the concept. This happens several times. It is finished when gameplay feels smooth.
For arcade games it works well.
I don't have a clear rule when to start implementing. For implementing the first concept the rule is: Keep it simple and don't waste time on details!