It’s because all five notes are adjacent to each other on the circle of fifths. Notes that are a fifth apart (adjacent on the circle of fifths) are consonant when played together. Limiting it to 5 notes makes it so that no two notes would sound too dissonant together. The more colors you try to use from the 12 chromatic notes, the greater potential for clashing colors. (The circle of fifths is kind of like a color wheel in that colors on opposite sides of the circle are maximally dissimilar to each other, so limiting yourself to 5 consecutive colors on the wheel means no colors are opposite each other)
