Author Topic: LyricsReloaded  (Read 238807 times)

Havokdan

  • Full Member
  • ***
  • Posts: 237
Here is how to add Genius.com as a provider with this plugin.

1. Save the following code as a text file named "genius.com.yml"

2. Place the file in <MusicBee's appdata folder>/mb_LyricsReloaded/providers/ folder (the folder already exists if the plugin is enabled)

3. Now you will see "Genius" entry from the lyrics providers list in Preferences> Tags(2)> lyrics setting.
Indeed, thank you, redwing, for posting that extremely helpful Genius.com provider template. I started using it awhile ago and eventually realized that it was having issues retrieving lyrics for some tracks/artists with various punctuation patterns, so I decided to take a look at the log and see what was up. And then I accidentally spent waaaaaaaay too much time discovering all the annoying variations in how lyrics sites format artist+title strings and flexing my puny regex muscles, with the aid of Pythex.org.

The result of my nerdocity is a modified Genius.com provider config + a config for Musixmatch.com. Neither are 100%, of course, but I tested it with a bunch of stupid character-containing bands and song titles and, while I'm quite sure that both configs could be significantly cleaned up by utilizing some of the built-in plugin filters and less terrible regex skillz, they seem to get the job done most of the time. They will, however, both still fail on song titles with parentheses, as the plugin strips everything on and after the 1st "(" before being scrubbed by the provider config (plus, feat. <artist> patterns are all over the place, so F that).



Anyway, here's my modified code for Genius (see redwing's original instructions for how to install)
Code: genius.com.yml
name: "Genius"

variables:
    artist:
        type: artist
        filters:
        - strip_diacritics
        - lowercase
        - [replace, "!!!", "chk-chik-chick"]    # !!! (Chk Chk Chk) artist exception
        - [regex, '(?<=\W|\s)+(feat.+|ft[\W\s]+|(f\.\s)).+', ""]
                                                # ^ Strip F./ft/eat/uring + everything after
        - [regex, '\s&(?=\s)', " and"]          # Replace " &" with " and"
                                                    ## ^ Currently superfluous (for ARTIST) bc plugin preemptively replaces "&" with "and"
                                                        ### ^ But still necessary for <title> "&" replacement
        - [regex, '\.+|,+|(\W+(?=$))|(^\W+)', ""]   # Strip "." + "," + beginning/end of string non-word chars
        - [regex, "'", ""]                      # Separate expression bc of annoying way YAML escapes '
        - [regex, '(?<=[a-z0-9%])[^\sa-z0-9%]+(?=[a-z0-9%]+)', "-"]
                                                # ^ Replace medial non-alphanumeric char(s) with single "-" e.g., f**k > f-k
        - [regex, '((?<=\s)([^a-z0-9\s-])+(\s|\W)+)|((?<=\w)([^a-z0-9-])+(\s|\W)+)', " "]
                                                # ^ Clean up any remaining successive non-alphanumeric chars before strip_nonascii
                                                    ## ^ Long & stupid bc couldn't figure out YAML-friendly non-capturing "(?:...)"
                                                        ### ^ EZ version: [regex, '\W+(?:\W)(?<!$)', " "]
        - [strip_nonascii, -]
    title:
        type: title
        filters: artist

config:
    url: "http://genius.com/{artist}-{title}-lyrics"
    pattern: ['<div\s+class="lyrics"[^>]*?>(?<lyrics>.*?)</div>', s]

post-filters:
- strip_html
- clean_spaces
- utf8_encode



For Musixmatch, copy/paste below code to a text file named "musixmatch.com.yml", save to MusicBee's <appdata folder>/mb_LyricsReloaded/providers/, open Musicbee and add new Musixmatch provider from Preferences > Tags (2) > Auto-tagging > Lyrics [...]
Code: musixmatch.com.yml
name: "Musixmatch"

variables:
    artist:
        type: artist
        filters:
        - strip_diacritics
        - lowercase
        - [replace, "!!!", "artist-46206"]      # !!! (Chk Chk Chk) artist exception
        - [replace, "+/-", "p%m"]               # +/- artist janky exception (step 1)
        - [regex, '(?<=\W|\s)+(feat.+|ft[\W\s]+|(f\.\s)).+', ""]
                                                # ^ Strip F./ft/eat/uring + everything after
        - [regex, "'", "%27"]                   # URL encode "'" bc MM is cool w/ dat shit
        - [regex, '\s&\s(?=the)', " and "]      # ONLY if succeeded by "the", replace " & " with " and "
                                                    ## ^ Currently superfluous bc plugin preemptively replaces "&" with "and"
                                                        ### ^ So MM will choke if <artist> contains "& (?!the)"
        - [regex, '(?<=[a-z0-9%])[^\sa-z0-9%]+(?=[a-z0-9%]+)', "-"]
                                                # ^ Replace medial non-alphanumeric char(s) with single "-" (except "'") e.g., M.I.A. > M-I-A.
        - [regex, '\W+(?=$)', ""]               # Strip end-of-string non-word chars
        - [regex, '((?<=\s)([^a-z0-9\s-])+(\s|\W)+)|((?<=\w)([^a-z0-9-])+(\s|\W)+)', " "]
                                                # ^ Clean up any remaining successive non-alphanumeric char(s) before strip_nonascii
                                                    ## ^ Long & stupid bc couldn't figure out YAML-friendly non-capturing "(?:...)"
                                                        ### EZ version: [regex, '\W+(?:\W)(?<!$)', " "]
        - [strip_nonascii, -]
        - [replace, "-27", "%27"]               # Fix "'" URL encoding after strip_nonascii
        - [regex, '(?<=^)p-m', "-"]             # Janky replace for artist +/- after strip_nonascii (step 2)
    title:
        type: title
        filters:                                # Not using identical artist filters due to different treatment of "&" in titles vs artist names
                                                # Musixmatch strips "&" from <title> ALWAYS; from <artist> too, *unless* "& the", then replaced with "and"
                                                    ## ^ see "&" treatment note above, under 'artist'
        - strip_diacritics
        - lowercase
        - [regex, '(?<=\W|\s)+(feat.+|ft[\W\s]+|(f\.\s)).+', ""]
                                                # ^ Strip F./ft/eat/uring + everything after
        - [regex, "'", "%27"]                   # URL encode "'" bc MM is cool w/ dat shit
        - [regex, '(?<=[a-z0-9%])[^\sa-z0-9%]+(?=[a-z0-9%]+)', "-"]
                                                # ^ Replace medial non-alphanumeric char(s) with single "-" (except "'") e.g., f**k > f-k
        - [regex, '\W+(?=$)', ""]               # Strip end-of-string non-word chars
        - [regex, '((?<=\s)([^a-z0-9\s-])+(\s|\W)+)|((?<=\w)([^a-z0-9-])+(\s|\W)+)', " "]
                                                # ^ Clean up any remaining successive non-alphanumeric char(s) before strip_nonascii
        - [strip_nonascii, -]
        - [replace, "-27", "%27"]               # Fix "'" URL encoding after strip_nonascii

config:
    url: "http://www.musixmatch.com/lyrics/{artist}/{title}"
    pattern: ['<span\s+id="lyrics-html"[^>]*?>(?<lyrics>.*?)</span>', s]

post-filters:
- utf8_encode

* I included my comments, just in case anyone wants to take stab at updating or modifying my very silly regex and wants to know what the hell I was thinking (delete at will - they have no effect on functionality).

So, yeah, with both Genius and Musixmatch as my #1 & #2 lyrics providers, I wanna say like 90% of my 25K songs now come with bonus words. Yay and thanks again redwing and, of course, quick_wango!


* Edit: 2015.09.02 10:30 PST - added featured artist stripping to both configs

For some reason both of these don't show up. I saved each text in a separate yml file (via notepad) and put them in <Roaming/MusicBee/LyricsReloaded/Providers/

Then I restarted MB and they aren't in the Tags (2) Lyrics section. Any clue? Using the latest MB (2.5.5721)

You saved with the name indicated in the above guide? Me also did not show up, then I put the name the colleague pointed out, it worked, I thought the file name was not important, only the extension.

Moshi_

  • Full Member
  • ***
  • Posts: 133
  • http://www.last.fm/user/Moshi_

For some reason both of these don't show up. I saved each text in a separate yml file (via notepad) and put them in <Roaming/MusicBee/LyricsReloaded/Providers/

Then I restarted MB and they aren't in the Tags (2) Lyrics section. Any clue? Using the latest MB (2.5.5721)

I might be just nitpicking here but that path is wrong.

It's suppose to be: AppData\Roaming\MusicBee\mb_LyricsReloaded\providers

Your path is missing the bold part.

Bigmelwalter

  • Jr. Member
  • **
  • Posts: 106

For some reason both of these don't show up. I saved each text in a separate yml file (via notepad) and put them in <Roaming/MusicBee/LyricsReloaded/Providers/

Then I restarted MB and they aren't in the Tags (2) Lyrics section. Any clue? Using the latest MB (2.5.5721)

I might be just nitpicking here but that path is wrong.

It's suppose to be: AppData\Roaming\MusicBee\mb_LyricsReloaded\providers

Your path is missing the bold part.

Big Thanks!!!!!!!!!!!!!! There are two LyricsReloaded folders

lelecureuil

  • Guest
thanks so much @emmaoninternet, @redwing and of course @quick_wango!


manu1433

  • Guest
So, I'm trying to add genius in there as well and am not getting it to show up. Was wondering if somebody could give me some pointers.

I am saving it in the correct location, so that doesn't seem to be the issue.

Alright, so all I have to do is open notepad, copy/paste the code into the document and title the file "genius.com.yml"

or is there a step that I am simply missing?

manu1433

  • Guest
So, I'm trying to add genius in there as well and am not getting it to show up. Was wondering if somebody could give me some pointers.

I am saving it in the correct location, so that doesn't seem to be the issue.

Alright, so all I have to do is open notepad, copy/paste the code into the document and title the file "genius.com.yml"

or is there a step that I am simply missing?

Disregard this.

I just realized that I forgot to save it as a YML file instead a regular text file.

Jabronie mistake on my part.

emmaoninternet

  • Jr. Member
  • **
  • Posts: 25
* Edit: 2015.09.11 13:10 PST - updated both configs to handle curly apostrophes like straight ( -> ' )
Updated provider code(s) in my orig post because curly apostrophes are silly and trollish.
Last Edit: September 11, 2015, 09:32:11 PM by emmaoninternet

PancakePenguin

  • Jr. Member
  • **
  • Posts: 21
I tried adding the musixmatch and genius providers but they don't work. I did some digging and found this in the log file


17/09/2015 07:33:52 [DEBUG] Loading config from file: C:\Users\LJ\AppData\Roaming\MusicBee\mb_LyricsReloaded\providers\genius.com.yml
17/09/2015 07:33:52 [INFO] Provider loaded: Genius
17/09/2015 07:33:52 [DEBUG] Loading config from file: C:\Users\LJ\AppData\Roaming\MusicBee\mb_LyricsReloaded\providers\musixmatch.com.yml
17/09/2015 07:33:52 [INFO] Provider loaded: Musixmatch
17/09/2015 07:34:01 [DEBUG] Received a notification of type PluginStartup
17/09/2015 07:34:03 [DEBUG] gzip compression detected
17/09/2015 07:41:26 [DEBUG] Lyrics request:  - Childish Gambino - All the Shine - Camp - Genius
17/09/2015 07:41:26 [DEBUG] Lyrics request:  - Childish Gambino - All the Shine - Camp - Genius
17/09/2015 07:41:26 [DEBUG] Lyrics request:  - Childish Gambino - All the Shine - Camp - Musixmatch
17/09/2015 07:41:26 [DEBUG] Lyrics request:  - Childish Gambino - All the Shine - Camp - Rap Genius
17/09/2015 07:41:26 [INFO] Rap Genius tries to load the lyrics...
17/09/2015 07:41:26 [DEBUG] The constructed URL: http://rapgenius.com/childish-gambino-all-the-shine-lyrics
17/09/2015 07:41:26 [INFO] Musixmatch tries to load the lyrics...
17/09/2015 07:41:26 [DEBUG] The constructed URL: http://www.musixmatch.com/lyrics/childish-gambino/all-the-shine
17/09/2015 07:41:28 [DEBUG] gzip compression detected
17/09/2015 07:41:28 [DEBUG] lyrics found

emmaoninternet

  • Jr. Member
  • **
  • Posts: 25
I tried adding the musixmatch and genius providers but they don't work. I did some digging and found this in the log file
Hmm, just from the log snippet you pasted, that all looks like it would be working ok, although it doesn't show the constructed URL for the separate Genius provider, though it may have been cut off below (+ logging seems to be a little inconsistent re: what actually gets printed for every lyric-search/transaction, especially the first search after startup).

Testing on the same song, it was able to find lyrics from both sources and, given the content/formatting, loaded them from Genius, so not sure what might be happening on your end. I tried enabling the built-in RapGenius provider (which I normally don't) to see if that was messing with anything, but haven't noticed any adverse effects. Besides just making sure you're on the latest version of this plugin/MusicBee, maybe rearrange your lyrics provider preference ordering in Tags (2) to put Genius + Musixmatch at #1/2  and disable the LyricsFetcher plugin if you have it.

Also try re-search/loading some lyrics via right-click "search next provider" "redo search", etc. Even if it finds lyrics from multiple sources, it will only show for one, but see if anything interesting is generated in the end of the log. *In the lyrics right click context menu, it will say 'Lyrics Source: MUSICBEE PLUGIN' when it's used a lyrics provider from this plugin.

Moshi_

  • Full Member
  • ***
  • Posts: 133
  • http://www.last.fm/user/Moshi_
I would also advise on disabling built-in RapGenius plugin.

PancakePenguin

  • Jr. Member
  • **
  • Posts: 21
It seems to be working now. I disabled the built in Rap Genius provider and put musixmatch and genius at the top.


StarkRaving

  • Guest
I have a question that isn't strictly speaking a LyricsReloaded issue, but I thought this would be the best place to get some help.

I have many, many songs that contain the following piece of text in place of the lyrics:

<a href=http://lyricsfly.com>Find lyrics on LyricsFly.com</a>

Is there a way to target these songs and perform a "Delete Lyrics and Redo Search"?  The songs exist across a very large array of artists and albums.

[EDIT] With the help of forum members, I was able to find a solution.  The Additional Tag Tools plugin allows me to do a find/replace on songs containing the above string.

Last Edit: November 04, 2015, 01:06:14 AM by StarkRaving

quick_wango

  • Jr. Member
  • **
  • Posts: 108
  • Software Engineer
Great to see that people are still liking my plugin.
Thanks to emmaoninternet for the good providers and helping others here.

I kind of lost interest in the project as I moved entirely to Linux and I am thus unable to use MusicBee.
But for that reason I made it both open source on GitHub (including open feature requests) and as extensible as possible right from the beginning.

However if someone wants to take the project over, feel free to contact me (contact address is on my github profile).
If someone wants to sponsor me to keep developing the plugin, contact me in the same way.

Otherwise the development of the plugin from my side officially ends now.