I do not believe that you can prevent video files showing in the folder browser view.
My recommendation would be to use the Music node of the library where you have more control and configuration options - the video files will be in the Video node (or can be sent there). You can also put audiobooks in the Audiobook node etc. It is worth experimenting with this - you have nothing to lose (well maybe some time).
Other than that you will have to move the video files to another folder.
This needn't be time consuming - using 'Tracks' view you can set '.ext' as a displayed field, sort by that and then select all video types to move in one go (or maybe two or three).
If the video files are well tagged you can send to an organised folder structure (right click - Send To - Folder (Move) - Move Files to Organised Folder). You can preview what is going to change before proceeding but, as always, try this with one or just a few files first so that, just in case it is not a happy ending, it will be simple to put them back.