Author Topic: LyricsReloaded (Updated)  (Read 164166 times)

frankz

  • Sr. Member
  • ****
  • Posts: 3840
Honestly, please someone fork this and take it over and update the original author's documentation and whatever else needs to be done.  I just don't have the bandwidth for it every time a site changes a line of code or every time someone wants to add a lyric site with a handful of obscure songs at this point. 

When I first looked at this, there were a handful of scripts that were orphaned in the original forum thread and only available to people who knew where to find them.  The original plugin author was MIA, so I learned rudimentary C# and built them in. 

My life is much different now than it was then. If Redearth or someone else wants to take the baton, by all means you have my blessing.  Make a new thread and link it here and I'll bow out.  Take the wheel.

Redearth

  • Jr. Member
  • **
  • Posts: 38
Honestly, please someone fork this and take it over and update the original author's documentation and whatever else needs to be done.  I just don't have the bandwidth for it every time a site changes a line of code or every time someone wants to add a lyric site with a handful of obscure songs at this point. 

When I first looked at this, there were a handful of scripts that were orphaned in the original forum thread and only available to people who knew where to find them.  The original plugin author was MIA, so I learned rudimentary C# and built them in. 

My life is much different now than it was then. If Redearth or someone else wants to take the baton, by all means you have my blessing.  Make a new thread and link it here and I'll bow out.  Take the wheel.

Hey frankz, thanks for all the work you've done, man. I hope you're not feeling overwhelmed by any of this just cos your fork was the last update.
My comments about the plugin needing updates to survive were for the community, I had no expectations from you.
Regardless of who has the fork, it's open source, community effort and anyone can submit pull requests, but I don't see that anyone has.

I looked through the code and it's all pretty straight-forward. I actually tried getting it running yesterday just to play with it a bit, but it was taking too long trying to get it work with the old dependencies and such.
It's possibly something I could take, but I'm really wondering if it's worth it. Is there any way to know how many people actually use it? People don't seem to be clamoring for it, and even with improvements, users generally have to have technical skill and be willing to update the config files...

sveakul

  • Sr. Member
  • ****
  • Posts: 2438
Man I am "clamoring" for it..  frankz made this sucker work well past its shelf life, but it's obvious how hard it must be to stay on top of the changes these sites make.  Redearth I encourage you to give it a shot, there are PLENTY of users who would take advantage of this but like me just don't have the technical chops to write the code.  As an inspiration I can refer to the complete resurrection of lyrics searching in the Foobar player when ONE GUY with the required time took the challenge to write the foo_multisource plugin that currently has ELEVEN working sources--Genius, Deezer, Metal Archives, etc etc.  It would be so cool to see MusicBee in the same place.  Please give it a go.

frankz

  • Sr. Member
  • ****
  • Posts: 3840
The plug-in has been downloaded 11,300 times in three years the since I first updated it - I'd say it's definitely worth the effort of keeping it updated if someone is so inclined.

The problem is that my "skills" are such that I must relearn a whole lot every time I go to do anything to it.  I code so infrequently that I have to look everything up whenever I change anything and I've got to spend time reacquainting myself with Visual Studio and how to compile it and how to publish it to github every time.  There was a time when I could devote an hour on an odd occasional Saturday morning to getting myself together and getting it done, but I can't reliably steal that time away from things now like I used to be able to.

Even with the forum these days - if I can provide a quick navigation tip or answer I do it but I rarely have the bandwidth to sit down and look at someone's complicated issue and think it through.  I wish I did.

I will say that I have serious doubts about the long-term longevity of the framework the plug-in uses to get lyrics.  It seems that lyrics in plain text at easily predictable URLs are becoming more rare, and throttling of requests is becoming more common, and I don't see any indication that either of those two trends reversing.  If someone is really going to do something that goes beyond maintenance and make it sustainable, it may be worthwhile to study what it is about that foobar plug-in that makes it works so well.

Redearth

  • Jr. Member
  • **
  • Posts: 38
I will say that I have serious doubts about the long-term longevity of the framework the plug-in uses to get lyrics.  It seems that lyrics in plain text at easily predictable URLs are becoming more rare, and throttling of requests is becoming more common, and I don't see any indication that either of those two trends reversing.  If someone is really going to do something that goes beyond maintenance and make it sustainable, it may be worthwhile to study what it is about that foobar plug-in that makes it works so well.

Yeah, after giving it a bit more of a look, I just don't see this as being viable much longer, unfortunately. The code is just too simple, and even with the few changes I could make to improve the capabilities in lyric parsing, the config files would be even more complex and still require frequent upkeep for as long as they even continue to work.

But the bigger problem is that the YAML files just aren't sufficient for adding lyric sites to LyricsReloaded. They simply aren't powerful enough to specify enough information to the plugin's generic parser. Trying to move forward with YAML would be an unmaintainable mess. The simple truth is that each lyric site needs its own parser written in code.

You're also right that relying 100% on page scraping won't be good long term, but adding API access capability to this plugin for sites that allow it would be another major addition. That with the YAML change, which I think are necessary, is essentially writing an entirely new lyric plugin from scratch.


Man I am "clamoring" for it..  frankz made this sucker work well past its shelf life, but it's obvious how hard it must be to stay on top of the changes these sites make.  Redearth I encourage you to give it a shot, there are PLENTY of users who would take advantage of this but like me just don't have the technical chops to write the code.  As an inspiration I can refer to the complete resurrection of lyrics searching in the Foobar player when ONE GUY with the required time took the challenge to write the foo_multisource plugin that currently has ELEVEN working sources--Genius, Deezer, Metal Archives, etc etc.  It would be so cool to see MusicBee in the same place.  Please give it a go.

foo_multisource isn't actually a plugin, it's a collection of parsers he wrote for the Lyric Show Panel 3 plugin, which he did not write. The plugin is much more complex and powerful than LyricsReloaded, but it still requires updates to the parsers whenever non-API lyric sites change their page structures. The author of foo_multisource even said his parsers aren't stable, and he has put in a ton of work into creating and maintaining them. They happen to work really well right now because he literally just released an update.

Also, it looks like he might rewrite the parsers for the foo_openlyrics plugin since Lyric Show Panel 3 is dead. foo_openlyrics is also a great plugin, but it's also substantially more complex and powerful than LyricsReloaded, but still requires parser upkeep.

So, sorry man, I just don't see it happening for MusicBee. Foobar attracts a very passionate and dedicated dev community.

Havokdan

  • Full Member
  • ***
  • Posts: 237
Perhaps it is the case of MusicBee reshaping the system of searching for lyrics to allow the creation of these parsers and their changes.

Steven

  • Administrator
  • Sr. Member
  • *****
  • Posts: 34313
to keep this topic focused on the lyrics plugin i have split the open-source discussion to: https://getmusicbee.com/forum/index.php?topic=35516.0

LazR

  • Jr. Member
  • **
  • Posts: 20
foo_multisource isn't actually a plugin, it's a collection of parsers he wrote for the Lyric Show Panel 3 plugin, which he did not write. The plugin is much more complex and powerful than LyricsReloaded, but it still requires updates to the parsers whenever non-API lyric sites change their page structures. The author of foo_multisource even said his parsers aren't stable, and he has put in a ton of work into creating and maintaining them. They happen to work really well right now because he literally just released an update

Honestly I think it would be easier to use this plugin on Foobar, save all lyrics to correct tag and then run in Musicbee as normal. If I had any sense of C#/++ I would try to put something together that uses the Genius API to scrape but I wouldn't even know how to get started on that.

Redearth

  • Jr. Member
  • **
  • Posts: 38
foo_multisource isn't actually a plugin, it's a collection of parsers he wrote for the Lyric Show Panel 3 plugin, which he did not write. The plugin is much more complex and powerful than LyricsReloaded, but it still requires updates to the parsers whenever non-API lyric sites change their page structures. The author of foo_multisource even said his parsers aren't stable, and he has put in a ton of work into creating and maintaining them. They happen to work really well right now because he literally just released an update

Honestly I think it would be easier to use this plugin on Foobar, save all lyrics to correct tag and then run in Musicbee as normal. If I had any sense of C#/++ I would try to put something together that uses the Genius API to scrape but I wouldn't even know how to get started on that.

It would be involved, but if Musicbee had more dev support for this, porting the plugin from Foobar to use MusicBee's API seems like the best solution. Haven't looked into it, but it seems the parsers could then be used between plugins without changes.
Last Edit: September 22, 2021, 08:31:37 AM by Redearth

crisp

  • Newbie
  • *
  • Posts: 5
I think I'm getting a yaml for the actual Genius too, but it's really complicated. Just nothing else works anymore, this plugin needs updating.

I think I have a yaml that works for Genius (until they change their formatting again).
Code
name: Genius

variables:
    artist:
        type: artist
        filters:
        - strip_diacritics
        - lowercase
        - [replace, "!!!", "chk-chik-chick"]
        - [regex, '(?<=\W|\s)+(feat.+|ft[\W\s]+|(f\.\s)).+', ""]
        - [regex, '\.+|,+|(\W+(?=$))|(^\W+)', ""]
        - [regex, "'", ""]
        - [regex, '(?<=[a-z0-9%])[^\sa-z0-9%]+(?=[a-z0-9%]+)', "-"]
        - [regex, '((?<=\s)([^a-z0-9\s-])+(\s|\W)+)|((?<=\w)([^a-z0-9-])+(\s|\W)+)', " "]
        - [strip_nonascii, -]
    title:
        type: title
        filters: artist

config:
    url: "https://genius.com/{artist}-{title}-lyrics"
    pattern: ['<div class="Lyrics__Container.*?">(?<lyrics>.*)<div class="Lyrics__Footer.*?">']

post-filters:
- utf8_encode
- entity_decode
- [regex, "<br/>", "\n"]
- strip_html
- clean_spaces

I haven't tested it extensively, but it works well when the URL is generated right. A few test failures I found:
1) Artist "X, the Y" is logged by the plugin as "X". I suspect this happens before the plugin regexes anything (I turned off the filters to check), maybe Musicbee just gives the plugin the segment before the first comma.
2) "X (Y)" is passed to the plugin as "X". Similar issue as before, but with parentheses. In my test case, Y wasn't a featured artist. Curiously, "X (Y) (ft. Z)" was correctly logged as "X (Y)". Is it only the last parenthesized phrase that's removed?
3) "Cygnus....Vismund Cygnus" is regexed to "cygnusvismund-cygnus", while the Genius URL expected "cygnus-vismund-cygnus". Other titles with ellipses remove them altogether, maybe this needs special handling.

sveakul

  • Sr. Member
  • ****
  • Posts: 2438
I think I have a yaml that works for Genius (until they change their formatting again).
Wow, this actually works!!  Thank you, crisp.  FAST, too.  Let's hope there is a solution for your code that maintains across their site changes--so far, so good!

Addendum:  been using it all afternoon and it's still drawing lyrics like crazy, have moved Genius to the top of the list in my search order.  Formatting looks good too!
Last Edit: October 16, 2021, 10:28:16 PM by sveakul

sveakul

  • Sr. Member
  • ****
  • Posts: 2438
For those using the LyricsReloaded(Updated) plugin out of the box and are not sure how to avail themselves of crisp's new yml so genius.com can be searched, follow the steps below.  They have been given in general before by frankz elsewhere in the thread.

1.  From crisp's post (https://getmusicbee.com/forum/index.php?topic=25406.msg194999#msg194999) copy the text inside the "Code" box and paste it to an empty text file, then rename the file "genius.yml."

2.  Find the path illustrated below; this is for a Portable installation of MusicBee, so if you have the Installer version the path shown will probably be in your C:\user\(username)\appdata path somewhere;  if you have the Store version I think you're SOL due to folder permissions.



3.  Place the genius.yml file inside the "providers" subfolder.

4.  Restart MusicBee, go to Prefs/Tags(2)/lyrics, and press the "..." button to the right of the selection box.

5.  In the Lyrics Providers box, scroll down to find "Genius", check it, move it to the desired search order position using the arrows, and hit the "Update" button.  Now Save and Close.

No guarantee how long before changes at the Genius website may screw things up, but for now, crisp's new yml is pulling lyrics abundantly!

Redearth

  • Jr. Member
  • **
  • Posts: 38
I think I have a yaml that works for Genius (until they change their formatting again).

Nice job! Thanks for getting it to work. I should have posted the yaml I had back in August as now I see it was just missing the <br> to "\n" conversion...
Problem was the log kept saying lyrics were successfully found, but nothing was displayed in MusicBee, and I didn't see a way to log or debug the output.

Perhaps Steven can clarify, but the three issues you brought up are due to how MusicBee passes the artist information to the plugin. When the plugin is called by MusicBee, the very first thing it does is print "Lyrics request: <artist> - <title> - <album> - <providerName>" to the log with the exact strings MusicBee provides, so there's nothing you can do using a config file right now.

The only thing I'd add to your config at the moment is [replace, "&", "and"] as a filter. You can also use br2nl instead of [regex, "<br/>", "\n"] in the post-filter.
Last Edit: October 19, 2021, 10:07:33 PM by Redearth

Redearth

  • Jr. Member
  • **
  • Posts: 38
I also fixed Musixmatch, and added the minor tweaks to crisp's fix for Genius.
For the sake of easy copy-pasting, I'm putting them both here.

Genius:
Code
name: Genius

variables:
    artist:
        type: artist
        filters:
        - strip_diacritics
        - lowercase
        - [replace, "!!!", "chk-chik-chick"]
        - [replace, "&", "and"]
        - [regex, '(?<=\W|\s)+(feat.+|ft[\W\s]+|(f\.\s)).+', ""]
        - [regex, '\.+|,+|(\W+(?=$))|(^\W+)', ""]
        - [regex, "'", ""]
        - [regex, '(?<=[a-z0-9%])[^\sa-z0-9%]+(?=[a-z0-9%]+)', "-"]
        - [regex, '((?<=\s)([^a-z0-9\s-])+(\s|\W)+)|((?<=\w)([^a-z0-9-])+(\s|\W)+)', " "]
        - [strip_nonascii, -]
    title:
        type: title
        filters: artist

config:
    url: "https://genius.com/{artist}-{title}-lyrics"
    pattern: ['<div class="Lyrics__Container.*?">(?<lyrics>.*)<div class="Lyrics__Footer.*?">']

post-filters:
- br2nl
- strip_html
- utf8_encode
- entity_decode
- clean_spaces


Musixmatch:
Code
name: Musixmatch

variables:
    artist:
        type: artist
        filters:
        - strip_diacritics
        - lowercase
        - [regex, "'", ""]
        - [regex, "/", " "]
        - [regex, '\s&(?=\s)', " "]
        - [regex, '(?<=\W|\s)+(feat.+|ft[\W\s]+|(f\.\s)).+', ""]
        - [regex, '[^\sa-z0-9]\s*', ""]
        - [strip_nonascii, -]
    title:
        type: title
        filters:
        - strip_diacritics
        - lowercase
        - [regex, " '|' |/", " "]
        - [regex, "'", " "]
        - [regex, '\.+|,+|/+|(\W+(?=$))|(^\W+)', ""]
        - [regex, '\s&(?=\s)', " and"]
        - [strip_nonascii, -]

config:
    url: "http://www.musixmatch.com/lyrics/{artist}/{title}"
    pattern: ['<p class="mxm-lyrics__content.*?">(?<lyrics>.*?)<div [^>]*"lyrics-report".*?>', s]

post-filters:
- [regex, "<script.*?</script>", "", s]
- strip_html
- utf8_encode
- entity_decode
- clean_spaces

(I would think this would also fix Musixmatch-Asian, but I'm not a good tester for it. If anyone wants it, I can add that too.)
Last Edit: October 19, 2021, 10:21:31 PM by Redearth

sveakul

  • Sr. Member
  • ****
  • Posts: 2438
I also fixed Musixmatch, and added the minor tweaks to crisp's fix for Genius.
For the sake of easy copy-pasting, I'm putting them both here.
Thanks Redearth!  I can confirm both of these yml's are working fine drawing lyrics here, MUCH appreciated.  For those who don't know, MusicBee will also draw lyrics from radio stations who send properly formatted track/artist metadata, and now these yml's are knockin' them out right and left.

Also for others: so as to not get the old existing Musixmatch mixed up with Redearth's new one on the Lyrics Providers priority list, I renamed the new yml-based one "musixmatch-new.yml", and changed the first line inside the file to "name: Musixmatch-new" which is how it shows up now on the list.

Redearth, as long as you're on a roll(!), how about seeing what you can do with AZLyrics, DarkLyrics, or even metal-archives (which would be better than DarkLyrics)?  Same plea goes out to crisp.