Author Topic: Re-Locate missing files for whole albums / multi files  (Read 16236 times)

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
i expect what is happening is you have imported playlists where the referenced file doesnt exist and hence the only thing MB knows about the file is the filename.
Exactly.
But isn't that the idea for having a relocate functionality...? i.e. IF a file is appearing as a dead link in a playlist, which is true for every time when I would need a 'Re-Locate missing files' functionality, isn't it also a given that the file is not to be found under the previous URL anymore...?
I think I don't understand how this is meant to work if you can only use this for files that aren't missing.
Churs.
c.

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34312
when i say MB matches on filename, i mean MB matches the filename excluding the folder path eg. c:\qwerty\a.mp3 would match to c:\asdfgd\a.mp3

this has the tweak as i described above:
http://musicbee.niblseed.com/V3_1/MusicBee31_Patched.zip

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Yea, I assumed it was for fileName.ext pattern only and ignoires the path part for the match (but even that also rules out that it will ever find cases where fileNameA.mp3 is meant to be replace with fileNameA.flac, right ?)

I'll give it a try tonight if it finds the 'easiest assumable case' files for case 3:
1: filename
see screenshot part 3 - despite the exact same name in playlist and library, it has not found the file althoigh the file in the new location is in the library as well

What does it take as a basis for matching ? Library contents that are cached on first match attempt (hence the faster search on second/third/etc match attempt) ?

Also - is no one else interested in this feature ? Would be good to get feedback from other people on this as well!

Churs.
c.
Last Edit: April 07, 2017, 04:19:28 AM by alec.tron

psychoadept

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 10691
I'm excited about it but don't have anything to use it with right this second.
MusicBee Wiki
Use & improve MusicBee's documentation!

Latest beta patch (3.5)
(Unzip and overwrite existing program files)

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Heyhey,
I just tried the last MB patch with the same missing files.

First I tried the easiest case, the same files as in screenshot part 3 - and again, despite the exact same name of the missing track in the playlist and the same fileName existing in the library (in a different location), it has not found the files with the same name and the match GUI remained empty.
But.
If I now hand-navigate to the folder and point to the file in the new location for 1 file, it now immediately matches all 13 files of the album correctly (after the match, the matching column of the GUI still lists previous fileName as title, and 0kb size for the missing file, but when hand-pointed to the new locations file it seemed to succeed on fileName match maybe ?).
Success.
Awesome!

So I tried the same on a more difficult case (example 2 in the screenshot) where the fileName has changed (from "01 Ode.mp3" to "01-Nils Frahm-Ode.mp3" ) and in that case it did not find the match obviously, but also after assigning the first match by hand, it did not auto match the rest of the album neither (as missing size [0kb] and actual fileSize were different AND the fileName has changed as well...). Again, not sure how these cases of missing files are meant to work with the currently implemented hard-coded ruleset.

Same for case 1 (example 1 in the screenshot) - no dice.


A thought though - In order to match tracks that lie outside of the matching logic as it is currently implemented, I would really wish you would be willing to expose the match rules in the GUI (via tickboxes maybe ? ) and add a alphabetical / number based matching pattern as well (eg resulting in 2 match lists, source & target or some such, and the user can move items up/down in for quick corrections by hand in each to establish the match quickly themselves) for users to be able to have another option when the 3 that are currently available fail - i.e. for cases with missing files for which fileName and/or extensions change - which for me is the vast majority of cases...

Hope feedback is welcome on this & nice to see this coming together! Thanks!

Churs.
c.
Last Edit: April 07, 2017, 09:35:12 AM by alec.tron

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34312
i have made one small change for the next update.
If the function is used on a playlist displayed in the main panel, then MB will now attempt matching on valid files already in the library.

To summarise the matching process:

1. If the missing file has never been a file that MB has known about and never cached any information eg. a file referenced in an imported playlist that was missing right from the start
   a. - MB will match on filename only, including file extension

2. If the file was known about by MB and hence MB has some information cached then the matching is:
   a. - MB will firstly try to match on filename. If a single file is matched then the following criteria (b&c) must also match and no other files will be considered for matching. If not matched, other files of the same file extension will be considered but the following criteria (b&c) must match
   b. - the file size must match
   c. - the artist and title must match

As mentioned above, the files considered as candidates might include files in the library depending on whether you are running this function against a playlist or not.

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Sweet. I'll update and give it a try again.
But, as it stands and how I understand what you have implemented, this approach can not work on missing files when:
- the fileName changes
- the file extension changes (from .mp3 to .flac)

For the case when a filePath changes, but fileName & fileExtension stay the same, this is a welcome addition, but this will cover less than 10% of my missing-files-cases as I will continue to switch from mp3 to flac where & when I can, and update the fileName-patterns as I go along as well.

There is a chance to work around this by purely relying on the metadata (artists & title) matching in cases where fileName & extension changes and both new & old are inside the MB library.... which requires the user to be rather rigorous when managing replacing whole albums with different filenames and/or fileExtensions.  So I'll see if that's a path worth taking over the next weeks when I start replacing more things again.
I would still really hope for an additional alphabetical/sorting based batch match to be able to fix cases for missing albums in playlists... but since you never commented on the suggestion, it seems like this is nothing you see value in, so I'll stop mentioning it.

Churs.
c.

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Heyyeh.
I just changed 2 names of albums (i.e. path/URL changed), no other change - upon restart, MB did the following:

1) flagged the files under the previous path as missing. Since this was as expected & intended, I confirmed to not remove them as I would like to replace them with the files living under a new path.
2) MB's start up scan also found the files under the new name & added them to the Inbox, which is to be expected as well.
So far so good.

I then went ahead and:
3) use 'Locate Missing Files...' to point 1 file to the new location, which then picked up all other files, since neither name or .ext changed, and worked as expected replacing all old URLs with the new.
4) I then removed the previously detected 'new' files from the inbox (as they were added through the 'Locate Missing Files...' action above).

But, to my suprise, when restarting MB, it now still adds the 'new' files to the inbox (despite them having been added to the library via 'Locate Missing Files...' )

I had 2 assumptions here:
- once I pointed missing old files to the new location via 'Locate Missing Files...', these files would fully replace the old files, and therefore would not be considered 'new' after being added to the library via 'Locate Missing Files...'.
And additionally - and please confirm that this is correct (this only occurred to me when the above happened and I assumed this as a given - as if not this could be rather destructive to my 100s of playlists...)  IF I use 'Locate Missing Files...' on an album, and tracks from this album occur in a playlists, when replacing tracks or a full album via 'Locate Missing Files...' this should and will also update each occurence in a playlist with these 'replaced' files.
Correct ?

Cheers.
c.

ps. this is on 3.1.6306, and on previous tests, the replaced-files did not re-appear in the inbox.

pps. turns out, looking some more into this, the replaced files are gone from the library... so it makes sense that the inbox is picking them up again... I'll do the same again and see if I can reproduce any of it...

ppps. looks like this is reproducable...
I added a dummy folder with files.
Added it to MB library,
then renamed the folder,
restarted MB,
which then picked up the changes and suggested them through inbox as new files.
IF I now go to library and point the missing files from the library to the new path/URL,
AND THEN go to the inbox to delete them from the inbox (after which, the relinked files in the library are still there at this point pointing to the new/replaced files),
AND IF I THEN restart MB - the replaced files are gone from the library without a trace, neither new or old are to be found, and the files used for the 'Locate Missing Files...' action, are again added to the inbox.
I might try NOT to delete from inbox, then restart and see if things look more healthy...

pppps.
The trick is indeed - after replacing them in the library, ignore the inbox files & DO NOT delete them from there - this seems to be where MB chokes. IF I do not delete them, but just restart MB at this point, then the replaced files remain in the library and after the restart, the new'old files are not shown in the inbox anymore.
Last Edit: April 23, 2017, 12:11:09 AM by alec.tron

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
And additionally - and please confirm that this is correct (this only occurred to me when the above happened and I assumed this as a given - as if not this could be rather destructive to my 100s of playlists...)  IF I use 'Locate Missing Files...' on an album, and tracks from this album occur in a playlists, when replacing tracks or a full album via 'Locate Missing Files...' this should and will also update each occurence in a playlist with these 'replaced' files.
Correct ?


Unfortunately, the assumption above I wanted to clarify, that 'Locate Missing Files...' functionality would also correctly replace the playlist entries of replaced files, was wrong from what I can see after testing this.
So in case people are using the 'Locate Missing Files...' functionality on files that are in playlists as well be aware - You will need to fix your playlists by hand after running 'Locate Missing Files...' for the same files as well!!

Interestingly, this seems to be correct at first - i.e. after a running 'Locate Missing Files...' on files that also are in a playlist, and the library files have been replaced with the files (same fileName.ext again to be within the legal scope of how this is implemented) as expected, checking the playlist where they occur also shows them in the 'new'/re-Located path/URL.
BUT - when restarting MB after the above steps, this is where things start to get odd - I then see the files in the library correctly pointing to the new location, whereas the files in the playlists, which were pointing to the new location before the restart, are now pointing back to the old location, and have become missing files again. And I do write all playlists to static .m3u playlists in my MB preferences and this works reliably usually.

See:



Bottom line for me here - since the replace functionality is limited to cases which I do not regularly have in my workflow (i.e. as I mostly update from mp3 to flac files and rename files as well occasionally) - having the replace 'Locate Missing Files...' that also fixes up platylists entries for the replaced files, tjhis was the selling point for me on this despite the limited functionality as is.
Since that does not work, I will not attempt to use this anymore as is.
Sing out if you intend to do more work on this and I'll happily test again.

Churs.
c.
Last Edit: April 23, 2017, 12:28:34 AM by alec.tron

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34312
it should be updating playlists but i will check that out in the next few days

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34312
i have confirmed MB should be updating playlists and does so in my testing.
I think in your case you have a huge number of large playlists and it might be the case the library playlist index is corrupted.
I suggest you close MB, delete the file "MusicBeeLibrary.pidx" which is in the root music folder and then restart MB will rebuild it when the file locate command is actioned. It will take some time to rebuild the indices though

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
i have confirmed MB should be updating playlists and does so in my testing.

Sweet - and that is true pre & post MB restart for you ?
I found it very odd that it adjusts the playlist corretctly first off, but does not manage to write it. That to me, as someone with a lot of as well as large playlists, is worrying that it can choke like this without me as a user being aware that it didn't finish what it was meant to (a completely uneducated guess/hunch was that this might play into it as well - https://getmusicbee.com/forum/index.php?topic=21594.0 ).

I'll flush the playlist index, let it re-index and test it again.
Is there any indication to us end-users that we can use to tell when the playlist indexing process has finished ?
This was another rather opaque thing in MB for me, where I just had to wait and sit to get certain indicators which hint at the process probably having finished (i.e. the indicator for that for me was seeing aggregated playlists in parent folders.... but, that might be a completely wrong assumption ?! Also, since you never commented if the aggregated view [of the playlists inside parent folders] is an intentional feature we can rely on & use, or an unintentional byproduct that's better not to be relied on and used...? [actual case - it's easier to fix missing files in playlists by hand in aggregated view rather than in each playlist...])

Cheers.
c.

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
I did the same test twice as well, and after flushing "MusicBeeLibrary.pidx" & recreating it, the replaced files remained in playlist under their new path.

One oddity though:
- it took 2 MB restarts for MusicBeeLibrary.pidx to be rewritten. After deleting MusicBeeLibrary.pidx & restarting MB I was giving MB time until processor use and disc read seemingly came to a stop (I assumed playlist indexing was finished then), so after restart, I then pointed 'Locate Missing Files...' dialogue to the new file location. Then checked playlist files, and they were replaced as expected & as before. I then shut MB down (again waiting for all read/write activity seemingly having finished). After that there was no MusicBeeLibrary.pidx file in the folder (I assumed it would be written on every MB shut-down just as the iTunes xml is). So I gave it another few minutes, then restarted MB; and to my suprise (since the MusicBeeLibrary.pidx wasn't written to disc) the playlist files kept their new location. I then shut down MB again, and then the new MusicBeeLibrary.pidx suddenly was written & appeared.
Any idea why it wouldn't have written it on first shut-down - and how it still kept the playlist change ?

Also, since this was somewhat erratic and impossible to catch unless one checks all files in playlist before & after the MB restart when using
 'Locate Missing Files...' functionality, which is not realistic imo - is there a chance to have MB flag a corrupted MusicBeeLibrary.pidx file ?

In case this helps you, I still have the previous MusicBeeLibrary.pidx file which as you said might be the culprit. If you want to have a look at that, sing out!

Cheers.
c.
Last Edit: April 25, 2017, 10:25:27 AM by alec.tron

psychoadept

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 10691
I just tested this out for the first time, because I had to rerip a whole CD at a higher bitrate.  So I deleted the old files and tried to point musicbee at the new files in the Ripped Files folder.  Well, that way I would have had to do each one manually.  So I moved the files into the folder where the old ones had been, and all but a couple of them had the same filenames anyway so I didn't have to "locate" them.  But the two I did match up with the locate files command lost their play counts.

Could this be adjust to have an option for MusicBee to look in a designated folder for matching files, rather than file by file?  Could be used for mass replace as well as locating missing files.
MusicBee Wiki
Use & improve MusicBee's documentation!

Latest beta patch (3.5)
(Unzip and overwrite existing program files)

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34312
for the next update, i have enhanced it to remember play/skip counts, and date added from the original missing file (if that data was cached in the musicbee library)