Author Topic: Multithreaded encoding  (Read 7376 times)

infogulch

  • Guest
I use LAME for my encoding with MusicBee, and it works great. The only problem is that it takes a long time to sync my multi-thousands of tracks to my device. I looked and LAME seems to be using ~25% processing power on my computer when running, which makes sense because I have a four-core processor.

It would be really nice if MusicBee could start multiple instances of LAME simultaneously (maybe an option). This would make encoding much faster for all of us that have multi-core processors.

You'd probably have to implement two queues, one for encoding, and one for copying to make it as fast as possible. (This would make it faster even for people with one core.)

:)


infogulch

  • Guest
I was thinking if you used queues and multiple instances of LAME, you could avoid having to go through the trouble of making MusicBee itself multithreaded.

But in this case you would still get a much improved syncing speed without all the extra work.

Blinghound

  • Sr. Member
  • ****
  • Posts: 319
+1, good idea, can multiple instances of lame use more than one core natively? Anyone know how dbpoweramp encodes on multiple cores?

infogulch

  • Guest
Ok, to clarify, this would NOT make LAME use multiple cores simultaneously for encoding a single file. And neither would musicbee need to be multithreaded.

The title of the topic is currently "Multithreaded encoding". I think a better term for it would be "Encoding multiple files at once". (and I may change the topic title so it's less confusing)

This suggestion is to start multiple instances of LAME.exe, each encoding a different file.

For example, if you have four cores, you could have four LAME.exe's running at once:

Lame.exe #1: encoding file A, mainly on core 1
Lame.exe #2: encoding file B, mainly on core 2
Lame.exe #3: encoding file C, mainly on core 3
Lame.exe #4: encoding file D, mainly on core 4

Note that none of these individual instances are multi-threaded (i.e. they each still only use one thread), but the operating system automatically assigns cores to a process based on need, so they will all run on different cores naturally.

Lets say file C is short and finishes faster than the others and exits. Musicbee would automatically start another instance of Lame.exe that encodes file E, and then it would copy file C to wherever it needs to go. And so on and so forth until they're all encoded.

All other things aside, this would make encoding at least [YourNumberOfCores] times faster than it currently encodes. But even if you only have one core, the separation of starting LAME on the encoding process from the copying of the file to the device would still make it faster.

Blinghound

  • Sr. Member
  • ****
  • Posts: 319
This would certainly be useful, and I wouldn't have thought it would be very hard to implement.

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34369
i cant see any issues with this - infact the cd ripping is already using this approach, so i just need to rework some of the code in the device synch

jimmanningjr

  • Jr. Member
  • **
  • Posts: 87
+1  I transcoded some flac to .ogg the other night and noticed that it was only handling one file at a time and on my Cpu meter only one of two cores was near maxed ...so I have been thinking about this lately..count my vote for this...I know foobar does it like that...For us multi core users it would make transcoding so much faster...hmmm at least 2wice as fast for 2 core. I was wondering why it was not done like this in the beginning??? Are not most computers these days at least Dual Core ?

bbutlerau

  • Guest
i know EAC does this where you can say how many instances to use 4 for quad core or 8 for quad with HT  :D
sadly EAC is closed source if memory serves so no luck "getting inspiration" from there

rognvaldr

  • Guest
i cant see any issues with this - infact the cd ripping is already using this approach, so i just need to rework some of the code in the device synch

I'm not sure multithreaded cd ripping would be such a good idea. In fact, I'm pretty sure it isn't and it wouldn't bring any benefits, just drawbacks.

Come to think about it I'm also pretty sure you didn't mean what I thought you meant. You're not implying reading multiple songs from a cd at the same time right? Please tell me no. lol

Rogn.

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34369
it reads the cd data serially and has a separate queue for encoding which will run in multiple threads if the encoding starts falling behind the reading

SDG

  • Guest
+1 for two queues - copying queue can transfer songs that already in lossy format, while encoding queue transcode lossless files.

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34369
this is done for the next update.
there is a recent setting already in the file converter preferences that allows you to specify the number of threads and priority for encoding. The device synch now respects that setting. Running the file converter manually/ cd ripping already uses that setting.


Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34369
its already in
http://www.mediafire.com/?c85i0bixc9nor11

and will be included in the weekly update probably tonight