Author Topic: iTunes xml parser for Traktor / Serato / Rekordbox  (Read 14695 times)

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Heya,
this is not a plug-in, but an extra helper nontheless, so if this is better placed somewhere else, please move this.

A while ago I wrapped my python based xml parser script that I use to push my MusicBee grouping & playlists to Traktor / Serato / Rekordbox into a usable package (an .exe file via pyinstaller... hence the size as it needs a fair few python libraries...).

This has been tested & works with:
Traktor 2.6.7 +
Serato SSL 2.5
Serato SDJ 1.9+
Serato SDJ Pro / 2.0.0+
Rekordbox 5.2+
Virtual DJ 5

Not supported / known issue:
Traktor 2.11.x+
There's a bug in Traktor (yet again) with files that have diacritic marks and/or umlaute ( Ä/ä, É/é  etc).... and reading an xml file with such inside will crash the latest Traktor version... you can remove all diacritic marks & umlaute & the xml will then work [but, unless you know what you are doing, this is not recommended...].



Grab it from here:
https://payhip.com/b/F4b7

Upon running the exe, it will show a console window, and a rather fugly gui. Point it to the xml you want to reformat, it then reformats it and saves it as *name*.Reformated.xml in the same place . Copy and/or rename this file to what Traktor / Serato / Rekordbox is reading in as set inside the respective programs' preferences, and you should see all your MusicBee groups & playlists inside your favourite digital dj software.

If there's issues let me know, although I can not guarantee full and/or prompt support as this just comes as is / as beerware...
Churs.
c.
Last Edit: September 14, 2018, 08:01:00 AM by alec.tron

zimakali

  • Newbie
  • *
  • Posts: 1
So I've noticed that Play Count in iTunes doesn't get updated when I play tracks in Traktor. Obviously. But, is there a way that we can update the iTunes Library xml, based on the Traktor Collection xml? I'm guessing this could be done in an Excel spreadsheet that does a vlookup on imported iTunes/Traktor xml's held in separate worksheets in tle xls workbook? I assume the .mp3 filename is the same in both the iTunes/traktor xml, so, could you copy the Play Count field from Traktors xml to the Play Count field in the iTunes xml? You'd need to run the vlookup/macro in the spreadsheet regularly in order to have your iTunes Play Counts reflect the actual plays in traktor - but even if you only ran it once a week/once a month, at least then you'd have data to work with in Smart playlists based on Play Count value...
PlexKodiLucky Patcher
Last Edit: June 17, 2018, 05:41:49 PM by zimakali

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Heya,
unfortunately there's quite a few fields in the xml that Traktor is not capable to read, or does in a bad way if at all.... and each field in Traktor does have its' own kinks & oddities when I looked into any of them in detail (I have an open bug ticketr with NI that Traktor is not capable of displaying 'Mix Artis' at all in an xml node...).
No idea about the Play Count tho, so you'd have to dig into that one yourself.
As for the same filename... that's a no as well... the iTunes xml encodes all unusual characters into a numeric character reference (NCR), so a diacritic character é in a file would be &#233 in the xml...
Good luck.
c.

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
I uploaded a new version here:
https://payhip.com/b/F4b7
as the MusicBee iTunes xml syntax has changed slightly in recent weeks.

This has been tested & should work with:
MusicBee version 3.2.6827+
-> https://getmusicbee.com/forum/index.php?topic=24754.msg149916#msg149916

And the resulting xml with groups should then work in:
Traktor 2.6.7 +
Serato SSL 2.5
Serato SDJ 1.9+
Serato SDJ Pro / 2.0.0+
Rekordbox 5.2+
Virtual DJ 5

Churs.
c.
Last Edit: September 14, 2018, 08:02:16 AM by alec.tron

Rekordbox

  • Newbie
  • *
  • Posts: 4
Hello Alec.tron,


thank you very much for your itunes xml parser. I tried the new version today for the first time and I' ve got following error:

XMLSyntaxError: PCDATA invalid Char value 16, line 33230, column 3 (line 33230).

Maybe I made a mistake?


Is it possible to integrate it directly into Musicbee by Steven?
 

Kind regards


alec.tron

  • Sr. Member
  • ****
  • Posts: 752
XMLSyntaxError: PCDATA invalid Char value 16, line 33230, column 3 (line 33230).

Maybe I made a mistake?
Unlikely ;)
There's many encoding/formatting pitfalls... especially when there's special characters in filename and/or file metadata.
Could you pm me " line 33230, column 3 (line 33230)" of the xml ? I can have a look and see if I can reproduce it. But no promises here...

Is it possible to integrate it directly into Musicbee by Steven?
That's a Steven question - he wasn't too keen on fudging around with xml syntax in the past and said he would leave it as is, so I took it upon myself to find the formatting that works for the DJ apps I need it in. Happy to pass on the knowledge though if Steven would integrate it directly into MB.
Churs.
c.

Icepat

  • Newbie
  • *
  • Posts: 5
Hello! I saw your script a few days ago and i decided to give it a try! So, i made a new backup of my iTunes Library and give it a try!

I had two main issues. The first one, sometimes people insert text inside the track number, disc number and year. That give me some errors with your script. I am a web developer (HTML/CSS) so, i'm ok with XML ;) I corrected all the tags one by one and finally got it works.

But i had a second problem. The artist tag didn't show inside Serato, but i find the problem. The problem was that i use MusicBee... in French! ;) MusicBee compile the <artist> tag as <artiste> and the <year> tag as <année>. I just changed the language of MusicBee to English US and everything works! Yay!

Finally, i can get rid of iTunes after all those years! MusicBee is so powerfull, fast and light! I love this software!

Thank you for your script! I will send you a beer soon! ;)

And i hope MusicBee will export playlist folders and sub-folders in the XML soon!

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Awesome, thanks for the feedback & sharing what you have found to troubleshoot the issues (I wonder if a french iTunes version would do the same, i.e. change the xml keys...).
Good to hear the parser is helping others as well!
c.
Last Edit: September 24, 2018, 09:12:44 AM by alec.tron

Icepat

  • Newbie
  • *
  • Posts: 5
No, iTunes in french compile the XML with English tags. To solve this problem, MusicBee should use the original column names, not the translated ones! :) Not all the tags are translated. I identified Artist and Year, but maybe other tags are translated too. For example, Genre in English is Genre in French too, but not in Spanish and some other languages maybe.

Icepat

  • Newbie
  • *
  • Posts: 5
Hello! A little update here! I tried to open the iTunes Library from the Denon Engine Prime software and it didn't work. Even directly from MusicBee without your parser.

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Heya,
looks like Denon is then employing yet another slight variation/interpretation of the itunes xml scheme, which probably works for denon <-> apple/itunes, but 3rd parties seem to have problems, i.e. MusicBee in this case...
I can't help much with that as I do not have or use Denon/Prime, but you could try reverse engineering the denon xml - i.e. see if it works in Serato/RB/Traktor, then use one entry of a few files & playlists to generate the smallest case possible, and scan the xml key/name variations for oddities see what happens if you change/add/remove things...
Good luck.
c.

Squints76

  • Newbie
  • *
  • Posts: 1
Hi, I used your parser to good effect but tried it a second time and got "exceptionin Tkinter callback errors"... Which I guess are caused by corrupt tag data, is that right? How do I locate and edit these problems so the Parser works again?

Regards...

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Heya,
Tkinter is the python UI toolkit I used for the parser UI.

Do you still have the previous XML ? (where you used the 'parser to good effect' ?)
If you do - is it replicate-able with one or more specific XML files ?
Then it indeed is an issiue with the xml (i.e. ''dodgy'' metadata might have been introduced... especially binary frames can cause issues here, or non-ascii encoding...).
If you have one xml that works, and another similar one that doesn't you could try to find the line that causes the issue. If you're in luck, sometimes the offending line/metadata is printed in the shell - have a look there as well.
That's all I can think of for now.
Churs.
c.

Barbie-qu

  • Jr. Member
  • **
  • Posts: 69
Hey alec.tron,
I've read various posts by you in different forums and i'm wondering if you have any more tips  about integrating MusicBee in a Denon Engine Prime workflow.
I posted here, and I was hoping you might have some advice
https://getmusicbee.com/forum/index.php?topic=32214.msg177468#msg177468

As Icepat experienced, the import into Denon Engine Prime software failed for me. Do you know any workarounds?

Cheers.

UPDATE: i ran a test importing the iTunes xml file generated by MusicBee of a smaller size library (450 tracks), and it was successfully read by Engine Prime. As you and other members have noted DJ software is sensitive to the variations in XML encodings. Not sure what the difference is between the two library files because my main library is 18k tracks, so i can't easily find the differences.
Last Edit: July 16, 2020, 10:03:40 PM by Barbie-qu

alec.tron

  • Sr. Member
  • ****
  • Posts: 752
Heya Barbie-qu,
Usually, pretty much in all cases, it was oddly formatted metadata from files, or the file names/paths themselves... so the hard part is finding the offensive files first, then fix whatever makes the target host crash when consuming the xml...

If you are really interested in solving this for your library & use-case, here's 2 approaches, 1 requiring a bit of leg work, the other knowing your way around xmls & formatting standards:
RouteA:
Take the whole xml with all of the 18k  files that causes problems, and start dissecting it. i.e. with Notepad++, duplicate it, name it partA & partB, and remove the last 9k track entries from A, and the first 9k from B, and run them each. IF you only have 1 offensive file, you now have 1 functioning, and one crashing xml. Now split the crashing xml one further down into parts, until you can pinpoint the crash to separate files.
Fix the files (i.e. remove filename & path oddities, as well as metadata - brute force: flush/remove all metadata...).

Alternatively, RouteB, requiring a bit of formatting/coding knowledge - find tools that you're comfortable with that let you find UTF-8 encoding violations in xmls.
Fix the files you find.

After both you should have a functioning xml with 18k tracks... and you have learnt what breaks Denon Prime xml imports specifically ;)

Hope this helps.
c.
Last Edit: July 17, 2020, 03:22:03 PM by alec.tron