System Out of Memory Error When Converting Tracks [SOLVED]

[SpirosG]

I am recently unable to convert any track. I always get a "System Out of Memory" error. It's not related to the latest patch, as I was facing the same problem with the previous one as well.
Maybe it is related to my usual "Huge library" issue, but just in case it's something else, I thought it would be worth reporting.

Current RAM usage is 2.5GB.

Tried to reduce the encoding threads to two, but the result was the same.

Code

27/10/2024 15:50:25 - 10.0.22631.0 - 3.6.9065.26432D - System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.StubHelpers.ValueClassMarshaler.ConvertToManaged(IntPtr dst, IntPtr src, IntPtr pMT)
   at #=zyY0lp654OtpzR8pSgXEW3mU=.#=zBe$bJMn_I$AL.#=zX5ie5$9zOWKa(Int32 #=zAOL0gm$W8mkM, #=znVB7n6kPBYz$7RQFFw==& #=zfS0Nl4zI5wvM)
   at #=zyY0lp654OtpzR8pSgXEW3mU=.#=zBe$bJMn_I$AL.#=zNQFAt1g=.#=zFOxlzc8FY79K.MoveNext()
   at #=zyY0lp654OtpzR8pSgXEW3mU=.#=zBe$bJMn_I$AL.#=zNQFAt1g=.#=zXECJ4SmXTqqd.#=zxQ3kR6$LNiRsQQV9Hw==.MoveNext()
   at #=zlYphP1hCbp4wkwpgveLRyOPosVfi.#=z5SaSZB56ICm$huZT$Q==(Dictionary`2 #=zAkG2f7VNXwGv)
   at #=zy0BKQ3fabdqjTx41e$3TnZY=.#=zCyukTmm2GnF45OWOtA==.#=zx3Kmb46IbnWy(IList`1 #=zeQUkREM=)
   at #=zbGQH3VGxTakaeviFhnlZhIM=.#=zssz154O1wguJ(Object #=ze2c5LeI=)

27/10/2024 15:51:08 - 10.0.22631.0 - 3.6.9065.26432D - System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.String.Concat(String str0, String str1)
   at #=zyY0lp654OtpzR8pSgXEW3mU=.#=zBe$bJMn_I$AL.#=zNQFAt1g=.#=zFOxlzc8FY79K.MoveNext()
   at #=zyY0lp654OtpzR8pSgXEW3mU=.#=zBe$bJMn_I$AL.#=zNQFAt1g=.#=zXECJ4SmXTqqd.#=zxQ3kR6$LNiRsQQV9Hw==.MoveNext()
   at #=zlYphP1hCbp4wkwpgveLRyOPosVfi.#=z5SaSZB56ICm$huZT$Q==(Dictionary`2 #=zAkG2f7VNXwGv)
   at #=zy0BKQ3fabdqjTx41e$3TnZY=.#=zCyukTmm2GnF45OWOtA==.#=zx3Kmb46IbnWy(IList`1 #=zeQUkREM=)
   at #=zbGQH3VGxTakaeviFhnlZhIM=.#=zssz154O1wguJ(Object #=ze2c5LeI=)

29/10/2024 00:15:56 - 10.0.22631.0 - 3.6.9065.26432D - System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.String.Concat(String str0, String str1)
   at #=zyY0lp654OtpzR8pSgXEW3mU=.#=zBe$bJMn_I$AL.#=zjCmYMYc=.#=zBkxw0z_TBlCQ(String #=zxx6yKbo=)
   at #=zyY0lp654OtpzR8pSgXEW3mU=.#=zBe$bJMn_I$AL.#=zNQFAt1g=.#=zFOxlzc8FY79K.MoveNext()
   at #=zyY0lp654OtpzR8pSgXEW3mU=.#=zBe$bJMn_I$AL.#=zNQFAt1g=.#=zXECJ4SmXTqqd.#=zxQ3kR6$LNiRsQQV9Hw==.MoveNext()
   at #=zlYphP1hCbp4wkwpgveLRyOPosVfi.#=z5SaSZB56ICm$huZT$Q==(Dictionary`2 #=zAkG2f7VNXwGv)
   at #=zy0BKQ3fabdqjTx41e$3TnZY=.#=zCyukTmm2GnF45OWOtA==.#=zx3Kmb46IbnWy(IList`1 #=zeQUkREM=)
   at #=zbGQH3VGxTakaeviFhnlZhIM=.#=zssz154O1wguJ(Object #=ze2c5LeI=)

UPDATE

It seems that the culprit, once again, was a playlist nested into folders. The rules were to load any track that the artist tag starts with and then nothing, so once again, an auto-playlist loading all of my huge library. The strange thing is that it was in a playlist folder that hasn't been touched since 2019 -The "2019 Year" folder contains playlists with music selections categorized by genre or other criteria- so after 2019 there were no changes, and I've had no problems until recently. This folder is also part of my secondary —smaller— library, which hasn't been modified for the last month, that I export as an iTunes library to load it into the Traktor DJ app, and it's not in there.

I have no idea how I created this auto-playlist, I am sure I wouldn't create a "whatever" auto-playlist, but on the other hand i am also sure MusicBee is not creating playlists by itself out of nowhere.


By the way to identify the playlist, I had to edit all of the auto-playlists through Notepad++ at once and change the LiveUpdating="True" to LiveUpdating="False". This made all the auto-playlists "static", so a .snap file had to be created for each one. The one for this playlist was around 600MB which, let's say, seemed a bit odd.

[Mayibongwe]

Maybe it is related to my usual "Huge library" issue, but just in case it's something else, I thought it would be worth reporting.

It's definitely related to that. You have 3 million tracks and Steven had this to say for just a third of that size:

if you want to keep using MB, i think you are going to need to split your library into two - even 1 million tracks is pushing the limits of what I had in mind with MB

Also I am wondering if such users couldn't solve their problems by simply using two or three portable installs of MusicBee.

working with music and being a DJ who frequently plays a lot of open-format sets and doesn't usually pre-plan his sets
requires having access to all of my tracks, acappelas, instrumentals, samples, effects, etc simultaneously, so having different instances is not really helpful.

This was your reasoning for not splitting out your library - which is fine for playing music.
But for editing operations -- where this exception mostly echoes from -- you have no choice but to split the library.

I don't think future bug reports relating to out-of-memory errors will be justified so long as you hold that many files under a single library/install.

[sveakul]

I'm assuming you've already tried the modification described in the thread below?
https://getmusicbee.com/forum/index.php?topic=22778.0

[SpirosG]

Maybe it is related to my usual "Huge library" issue, but just in case it's something else, I thought it would be worth reporting.

It's definitely related to that. You have 3 million tracks and Steven had this to say for just a third of that size:

if you want to keep using MB, i think you are going to need to split your library into two - even 1 million tracks is pushing the limits of what I had in mind with MB

Also I am wondering if such users couldn't solve their problems by simply using two or three portable installs of MusicBee.

working with music and being a DJ who frequently plays a lot of open-format sets and doesn't usually pre-plan his sets
requires having access to all of my tracks, acappelas, instrumentals, samples, effects, etc simultaneously, so having different instances is not really helpful.

This was your reasoning for not splitting out your library - which is fine for playing music.
But for editing operations -- where this exception mostly echoes from -- you have no choice but to split the library.

I don't think future bug reports relating to out-of-memory errors will be justified so long as you hold that many files under a single library/install.

I've already created a new library for such actions. However, that's not the case I reported, as I mentioned. I'm reporting this in case there's something else causing the problems.

[SpirosG]

I'm assuming you've already tried the modification described in the thread below?
https://getmusicbee.com/forum/index.php?topic=22778.0

Yes, the app has already been "patched."

Strangely, removing all or most of my auto-playlists from the main library resolves the issue.

The behavior is odd because with all the playlists loaded, MusicBee appears to increase the use of RAM until it reaches its limit, from 2.5-2.7GB to 3.6-3.9GB whenever I attempt to convert files. In the case of the auto-playlist being removed, the additional demand for RAM when I try to convert files is 100-200MB.

[TateB]

Do you have "automatically refresh the matching tracks" ticked or unticked on your auto-playlists?  What happens to your memory usage while converting files, if you uncheck that on your auto-playlists?

Not entirely the same thing (but maybe similar?) - I discovered a bunch of songs in certain of my regular playlists that I had previously deleted (through musicbee, not file explorer or another app), which were still showing up with the "missing file" icon.  When i removed these files from the playlist, performance appeared to increase, particularly when first loading the library after the app opens or when accessing the particular playlist that had these "missing" files stored.

[Mayibongwe]

Strangely, removing all or most of my auto-playlists from the main library resolves the issue.

'Strange' implies an unexpected outcome. I don't think that's the case here, given:

Again because of the huge number of tracks in your library, loading each auto-playlist will use a lot of cpu

[SpirosG]

Do you have "automatically refresh the matching tracks" ticked or unticked on your auto-playlists?  What happens to your memory usage while converting files, if you uncheck that on your auto-playlists?

Not entirely the same thing (but maybe similar?) - I discovered a bunch of songs in certain of my regular playlists that I had previously deleted (through musicbee, not file explorer or another app), which were still showing up with the "missing file" icon.  When i removed these files from the playlist, performance appeared to increase, particularly when first loading the library after the app opens or when accessing the particular playlist that had these "missing" files stored.

Yes, I think most of them are refreshing automatically.
There are too many to edit them one by one, but I probably can convert all of them to m3u through the general settings. I'll try to give that a try.

[SpirosG]

Strangely, removing all or most of my auto-playlists from the main library resolves the issue.

'Strange' implies an unexpected outcome. I don't think that's the case here, given:

Again because of the huge number of tracks in your library, loading each auto-playlist will use a lot of cpu

Nothing to do with it as the problem appeared out of nowhere and there was no change to my Auto-playlists. No new tracks were added to the main library for the last month, but the problem appeared two to three weeks ago.
Finally, I was able to sort ithis one out as it appeared to be an auto-playlist that actually contained all of my tracks, so I deleted it and the problem was solved.
My mistake was that I forgot to update that topic.

[sveakul]

Finally, I was able to sort ithis one out as it appeared to be an auto-playlist that actually contained all of my tracks, so I deleted it and the problem was solved.

With 3 million tracks yep you could see why that was a problem, ouch!!

[SpirosG]

Finally, I was able to sort ithis one out as it appeared to be an auto-playlist that actually contained all of my tracks, so I deleted it and the problem was solved.

With 3 million tracks yep you could see why that was a problem, ouch!!

This was about the problem I had back then with the high CPU usage, not the one here.
I had no idea this was in my playlists, and the problem was that it was nested in folders, so it wasn't easy to find.
But yeap, HUGE ouch, and it was refreshing all the time even when I was just playing a track.

[SpirosG]

It seems that the culprit, once again, was a playlist nested into folders. The rules were to load any track that the artist tag starts with and then nothing, so once again, an auto-playlist loading all of my huge library. The strange thing is that it was in a playlist folder that hasn't been touched since 2019 -The "2019 Year" folder contains playlists with music selections categorized by genre or other criteria- so after 2019 there were no changes, and I've had no problems until recently. This folder is also part of my secondary —smaller— library, which hasn't been modified for the last month, that I export as an iTunes library to load it into the Traktor DJ app, and it's not in there.

I have no idea how I created this auto-playlist, I am sure I wouldn't create a "whatever" auto-playlist, but on the other hand i am also sure MusicBee is not creating playlists by itself out of nowhere.



By the way to identify the playlist, I had to edit all of the auto-playlists through Notepad++ at once and change the LiveUpdating="True" to LiveUpdating="False". This made all the auto-playlists "static", so a .snap file had to be created for each one. The one for this playlist was around 600MB which, let's say, seemed a bit odd.