This is not really about 'how to remove locked files'.
You can easily remove the file ending with a dot with the same software that created it, namely MusicBee...
The issue is that MusicBee will create a file ending with a dot, which by Windows is an illegal action.
And it looks like that is the cause of the bug that the OP reported here.