They seem to be linked to the color of the text. The only thing I was able to find was changing to the desired color, and then setting the track to a custom virtual tag, but that that leaves the rest a different color like this:
So I'm wondering if there is a way to change the track number and time to another color, or just a simpler way that I seem to be missing?