Inferno+ soundtrack: http://radiangames.bandcamp.com/album/inferno-soundtrack "Green" is particularly awesome.
Tap Sonic is a great music game IMO, especially for players of the DJMAX series on the PSP. The drawback is that you've to pay for credits to play the songs, but even for me just a $4.99 pack could last for a long time.
Tone Sphere... Just got this one, and WOWZERS!! I'm totally in LOVE WITH IT!! here's a vid - Definitely a game fans of the genre should check out. =oD
Duh. Just realised the thread is titled "Best music game" not "best game music". Hence my previous post. Apologies Kevinms14... Besides the games already mentioned - Beat sneak bandit, mad acorn, pulse - that I also appreciate, I'd like to recommend Penny Time and Russian Dancing Men. The last one is really one of a kind (hilarious!) and one that, i believe, absolutely everyone should try out. If you like that game's soundtrack, there is also the Weebl Musicator (free) which is just a fun way to interact with said soundtrack, though not in a gamey way, I must admit.
Tried Mad Acorn yesterday, and it's pretty fantastic. I'm pretty amazed by how little attention it's gotten.
The sequel to Miku Flick should be out within the next few months. It will come out worldwide this time rather than releasing in Japan and then everywhere else months later. Bongo Touch 1 and 2 are a lot of fun as well.