Author Topic: LyricsReloaded (Latest)  (Read 66103 times)

frankz

  • Sr. Member
  • ****
  • Posts: 3868
IMO this is a logical solution.  Those folks who want strict but maybe limited results can use the listed providers, and those who may want to loosen things up and take a chance with free text searching are still able to do so using Google as a lyric source, arranging them in the order of their choosing.  Great thinking.

sveakul

  • Sr. Member
  • ****
  • Posts: 2503
Sadly, the check I thought I'd be able to add turned out to be a lot harder than it sounded.
So I've reverted Genius, Musixmatch and Lyrics Freak back to their original states - redownload the plugin.
1.  Does this mean Genius and Musixmatch are still integrated into the DLL with Google, but are "doing their own thing" just like yml files (saw they are still removed from the providers yml zip)?

2.  Is Deezer still intergrated into the DLL?

Mayibongwe

  • Sr. Member
  • ****
  • Posts: 1098
  • Heal The World
Yep, they are all still there. The change is only in the way they do their searching. Otherwise, they are set to work exactly how they used to before.
Users from Europe though (and possibly other regions that we do not yet know of) aren't able to use google as a source, due to something the plugin itself doesn't have control over.

Regarding the built-in yml sources missing from the providers zip file:
people can still access them from the configs folder in the source files. Although there's not much need to do so.
Last Edit: August 06, 2022, 07:15:40 PM by Mayibongwe
I already spend hours on end on social media. Might as well spare a few of those to a greater purpose here.

sveakul

  • Sr. Member
  • ****
  • Posts: 2503
Yep, they are all still there. The change is only in the way they do their searching. Otherwise, they are set to work exactly how they used to before.
Don't mean to pester you, but can you clarify "exactly how they used to before"?  Does that mean Genius and Musixmatch function the same way as when they were included as separate yml files?  And as Deezer made its only appearance as intergrated in the DLL, what search strategy is in use there?

Mayibongwe

  • Sr. Member
  • ****
  • Posts: 1098
  • Heal The World
Sorry for the ambiguity. Yes, that means they are back to using the sites' search functions.
Before reverting back, I thought I'd make them use the google search engine as it appeared to match some songs which wouldn't have turned up when using a site's direct searching.
I didn't account for the fact that this would also return incorrect lyrics whenever there wasn't any available match.

Deezer's searching hasn't changed from what it was using when I introduced it.
It is still using the search loader (google search engine), but I suppose I should also make it use its own searching (for more accurate results).

I personally hadn't encountered an issue where incorrect lyrics were returned.
So I had no clue at first how much of an inconvenience it could cause with regards to the provider returning incorrect lyrics.
And I myself do not have MusicBee set to auto-save the lyrics. So I would have been aware in the event where incorrect lyrics were returned.
But for those who have that setting enabled, they would not be aware and I understand why people would not want a provider returning incorrect lyrics when it didn't find a match.
I already spend hours on end on social media. Might as well spare a few of those to a greater purpose here.

hiccup

  • Sr. Member
  • ****
  • Posts: 7936
I'm still seeying discrepancies between using the build-in MusixMatch vs. my yml version:

build-in finds nothing, vs. yml finds correct lyrics:

Ivan Lins - Dinorah, Dinorah
Jacques Brel - Les Bonbons 67
Carla Bruni - Quelqu'un m'a dit

build-in finds wrong lyrics, vs. yml finds correct lyrics:

Simone - Pétala

build-in finds wrong lyrics, vs. yml finds no lyrics (which is the correct behaviour here):

Sharon Robinson - Sustenance
Simone - Fantasia

the yml I'm using:
Code
name: Musixmatch (new)

variables:
    artist:
        type: artist
        filters:
        - strip_diacritics
        - lowercase
        - [regex, "'", ""]
        - [regex, "/", " "]
        - [regex, '\s&(?=\s)', " "]
        - [regex, '(?<=\W|\s)+(feat.+|ft[\W\s]+|(f\.\s)).+', ""]
        - [regex, '[^\sa-z0-9]\s*', ""]
        - [strip_nonascii, -]
    title:
        type: title
        filters:
        - strip_diacritics
        - lowercase
        - [regex, " '|' |/", " "]
        - [regex, "'", " "]
        - [regex, '\.+|,+|/+|(\W+(?=$))|(^\W+)', ""]
        - [regex, '\s&(?=\s)', " and"]
        - [strip_nonascii, -]

config:
    url: "http://www.musixmatch.com/lyrics/{artist}/{title}"
    pattern: ['<p class="mxm-lyrics__content.*?">(?<lyrics>.*?)<div [^>]*"lyrics-report".*?>', s]

post-filters:
- [regex, "<script.*?</script>", "", s]
- strip_html
- utf8_encode
- entity_decode
- clean_spaces

Mayibongwe

  • Sr. Member
  • ****
  • Posts: 1098
  • Heal The World
Did you re-download the plugin (v1.1.15.5)? Yesterday, I changed it back to how it worked at first.
The yml is exactly how it appears on yours (except that it also has that line which phred mentioned prior - which replaces an ad-line with a new line).
I already spend hours on end on social media. Might as well spare a few of those to a greater purpose here.

hiccup

  • Sr. Member
  • ****
  • Posts: 7936
Did you re-download the plugin (v1.1.15.5)?
I was convinced I had the latest one, but checking it, it said 1.1.15.4
I now downloaded 1.1.15.5, and the discrepancies are gone.
Thnx!

hiccup

  • Sr. Member
  • ****
  • Posts: 7936
I thought it might be interesting to get an idea of the success rate of finding correct lyrics for a couple of lyrics provider that the plugin can use.
After doing tests using around 100 songs of some various genres and languages, this is what it looks like for me:

          TEMPORARILY REMOVED
It was showing incorrect 'wrong lyrics' percentages
I'll post an updated version probably later today


Some of the 'smaller' providers were able to find some more obscure lyrics that the 'bigger' providers couldn't, so for me the next step will be to try and decide on the combination (and order) of providers to set to get the best average results.

edit,
No, I'm fooling myself, it's going to become more difficult.
I'll probably try to find lyrics providers that have lyrics for many of my not-found songs, and try to create (or edit existing) yml's to make those work.
If someone can point me to some tutorials/explanations on the workings of these yml's (perhaps somewhere in this thread, or in help files, or on github) it's appreciated…
Last Edit: August 07, 2022, 01:39:17 PM by hiccup

Mayibongwe

  • Sr. Member
  • ****
  • Posts: 1098
  • Heal The World
Those Genius stats are insane and I assume they're from v1.1.15.5 as 15.4 would definitely not have had a 0% for incorrect lyrics.

The downside with the google search engine is that it is biased towards English results.
Using the song examples that you've provided, I tried to make the correction that I'd said I'd make - but ended up not due to me rolling back to the first yml workings.
Tracks without a comment indicate that there was a correct match.

Ivan Lins - Novo tempo
Margriet Eshuijs Band - Black Pearl
Chico Buarque - Fortaleza
Simone - Geraldinos e Arquibaldos
Ane Brun - Du gråter så store tåra (the english translation was ahead of the original in the google search results)
Nana Caymmi - Doce Presença
Walter Becker - Door Number Two
Ivan Lins - Dinorah, Dinorah
Jacques Brel - Les Bonbons 67
Carla Bruni - Quelqu'un m'a dit (version listed at the top of the results has an English translation alongside and the yml's pattern is not built for that)
Simone - Pétala
Sharon Robinson - Sustenance (correctly returns a no match like the original)
Simone - Fantasia (the original cannot match it bcoz the artist on the site is 'Suzane & Simone', but with this, the match would occur)

..............................

Overall comments:
- The musixmatch yml below (which makes use of google searching) would mostly suit those who have most of their collection in English.
- There also would be a side effect in which songs that aren't available on the website, would return incorrect lyrics instead.
- Other users are right in that this behaviour would not be ideal as the default.
Code
name: Musixmatch_Google
loader: search

variables:
    artist:
        type: artist
        filters:
        - lowercase
        
    title:
        type: title
        filters: artist

config:
    identity url: "https://www.google.com/search?q=Musixmatch+{title}+{artist}"
    identity pattern: ['(?<identity>https://www.musixmatch.com.*?)%3Futm_|(?<identity>https://www.musixmatch.com.*?)["&]', 's']
    lyrics url: ""
    lyrics pattern: ['<p class="mxm-lyrics__content.*?">(?<lyrics>.*?)<div [^>]*"lyrics-report".*?>', s]

post-filters:
- [regex, "<script.*?</script>", "", s]
- [regex, '<div class="inline_video_ad_container_container">', "\n", s]
- strip_html
- utf8_encode
- entity_decode
- clean_spaces
Last Edit: August 07, 2022, 01:31:51 PM by Mayibongwe
I already spend hours on end on social media. Might as well spare a few of those to a greater purpose here.

Mayibongwe

  • Sr. Member
  • ****
  • Posts: 1098
  • Heal The World
Google:
I know 2000+ songs is a lot to go through, but it also would have been nice if phred could analyze a few of those and mention a few circumstances where the provider would have returned incorrect lyrics.

It'd be nice to know some of the circumstances in which this provider is bound to fail.
So far, I am aware that anything not English likely won't be found (and will leave room for incorrect lyrics being returned).

Lyrics Freak:
Having changed back to the site's search function yesterday, Idk for sure but I think the incorrect lyrics percentage would be a lot lower.
Were you using 1.1.15.4 for it? I think with 15.5, the searching would be a lot more stricter.

Deezer:
The high 'incorrect lyrics' percentage would unfortunately be something we have to live with as there is no other way I can do the searching.

I'll probably try to find lyrics providers that have lyrics for many of my not-found songs, and try to create (or edit existing) yml's to make those work.
If someone can point me to some tutorials/explanations on the workings of these yml's (perhaps somewhere in this thread, or in help files, or on github) it's appreciated…
In the 2nd post of this thread, I linked back to frankz's documentation post.
In the add-on readme, I linked the plugin's original author's github documentation page (which is no different to frankz's linked post though).
If that's not sufficient, people can ask whatever needs explanation and anybody on here with an answer would happily assist.
Last Edit: August 07, 2022, 01:21:34 PM by Mayibongwe
I already spend hours on end on social media. Might as well spare a few of those to a greater purpose here.

hiccup

  • Sr. Member
  • ****
  • Posts: 7936
Lyrics Freak:
Having changed back to the site's search function yesterday, Idk for sure but I think the incorrect lyrics percentage would be a lot lower.
Were you using 1.1.15.4 for it? I think with 15.5, the searching would be a lot more stricter.

Deezer:
The high 'incorrect lyrics' percentage would unfortunately be something we have to live with as there is no other way I can do the searching.

Testing all of this has been slightly messy and complicated. (updating the plugin, VPN yes/no, using correct yml's, don't forget to empty cache, etc.)
So I wouldn't be too surprised if your suspicions about the wrong lyrics percentages being wrong are valid.

I'll re-test those and report back…

Mayibongwe

  • Sr. Member
  • ****
  • Posts: 1098
  • Heal The World
I'll re-test those and report back…
That would be appreciated.
In the end, we should look for solutions that return more lyrics while keeping the incorrect percentages as minimal as possible (preferably avoidable at all, if possible).
I already spend hours on end on social media. Might as well spare a few of those to a greater purpose here.

hiccup

  • Sr. Member
  • ****
  • Posts: 7936
I'll re-test those and report back…
That would be appreciated.
In the end, we should look for solutions that return more lyrics while keeping the incorrect percentages as minimal as possible (preferably avoidable at all, if possible).
The 'wrong lyrics' percentages were indeed very wrong. My apologies for that. (also to Deezer and Lyrics Freaks ;-)
I'll see if I can maybe post new stats tonight. (going out now)

If Phred or anyone else has a (not too large) list of songs as a suggestion to add to my current sample song set, let me know.
The more variety, the better for its purpose.

What would e.g. add variety to my selection would be hip-hop/rap (I don't have much of that)
What would not be useful is music from artists such as The Beatles, U2, Simply Red, Adele, etc.
The lyrics for their music will easily be found by most providers.

phred

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 9391
it also would have been nice if phred could analyze a few of those and mention a few circumstances where the provider would have returned incorrect lyrics.
I will spend some time double-checking lyrics that were retrieved from Google over the past couple of days, but I'm certainly not going through 2000 files. I'll do what I can to check a hundred or so.

Meanwhile hiccup's analysis is very well done. Don't expect the same from me. I'll provide the number of lyrics retrieved from Google and how many were correct or not.
Download the latest MusicBee v3.5 or 3.6 patch from here.
Unzip into your MusicBee directory and overwrite existing files.

----------
The FAQ
The Wiki
Posting screenshots is here
Searching the forum with Google is  here