Are you using "Play All" while still playing the shuffled list? That is the only way I can see this happening...
You might consider setting your smart button to "Shuffle All" - it might work out easier for you.
It is pitch black. You are likely to be eaten by a grue.