Why has SongKong split album and why am I still gettting duplicates

[meaning]

ok I send you the last duplicates run as well, which was run using " * Song is a duplicate if has same: Same MusicBrainz song and same album (any version) and sounds the same"

you get it by email.

[paultaylor]

Right, but that is not what I asked you run and that is not going to work because the all the songs on that particular album have different mbrecordingids, so the songs you consider the same have different mbrecordingid and therefore wiil not be considered a duplicate using the Same MusicBrainz Song part of criteria, in this case you can only find duplicates using the Sounds the Same criteria.

This is what I said in earlier post

However in this confusing case (with all songs having the same name) it turns out that all songs have different MBRecordingId . This means that either they are all different version of the song or more probably that there are same versions of songs but when the 2nd version of song was added to MusicBrainz it was not merged with existing but added as a new song.

So this will happen sometimes. What you could do as a one-off is try to using Sounds the same only option, but his will not prevent songs from different albums (compilation, original album) being deleted so should run on preview first, and check results. but it will detect duplicates of songs that are sonically the same.

But it will not work for first track because at only 5 seconds long is too short to safetly identify by Acoustid

[meaning]

yeah, well, why it got merged like this remains a mystery, and I undertstand I could only rely on sounds the same, but damn running this on a large set of tracks sounds scary as hell.

[paultaylor]

@meaning

yeah, well, why it got merged like this remains a mystery

Do you mean why are both copies of the same album in same folder?

The short answer is because this is how your rename mask has said to rename the files. Now there is some logic in the code (which I should revisit) to rename files if there is a clash, but because one version has been matched to this version of the release https://musicbrainz.org/release/f20fda28-957f-4bc5-964a-77d1bf5197d3 which has all songs having title untitled and the other did not there is no filename clash to prevent all songs being put in one folder.

@meaning

and I undertstand I could only rely on sounds the same, but damn running this on a large set of tracks sounds scary as hell.

I was only recommending running it as one-off on this artist and preview only first, so I agree you should not run it on all your files, just trying to explain to you why the duplicates were not found

[meaning]

I ran it on a single artist. It removed all the duplicates. + some tracks that appeared on both EP and albums. So, it does the job, but does it “too well”.

I’ll resnatch the impacted albums.

Is there something we can do in songkong, to allow deletion of tracks located in the SAME folder only ? this would make sure we can use this accousticID setting while NOT deleting tracks that are located in separate folders? It would then only remove the “duplicates” sound the same tracks that are located in the same dir ?

Also, was "musibrainz album / any version / sound the same not suposed to delete these duplicate tracks based on their accousticID as well ?

[paultaylor]

@meaning

I ran it on a single artist. It removed all the duplicates. + some tracks that appeared on both EP and albums. So, it does the job, but does it “too well”.

I did warn you earlier:

but his will not prevent songs from different albums (compilation, original album) being deleted so should run on preview first

I usually run Delete Duplicates in preview mode first.

@meaning

Is there something we can do in songkong, to allow deletion of tracks located in the SAME folder only ? this would make sure we can use this accousticID setting while NOT deleting tracks that are located in separate folders? It would then only remove the “duplicates” sound the same tracks that are located in the same dir ?

I think in the general case the solution is the combined Same song and same album (metadata only) and sounds the same option that i mentioned earlier which provides a safer method, however that would not work for this particular case because the song titles are different. So maybe as you suggest a new option that restricted to find duplicates within folder would work similar to the Find Duplicates within Format option, could call it Find Duplicates within same Folder option

@meaning

Also, was "musibrainz album / any version / sound the same not suposed to delete these duplicate tracks based on their accousticID as well ?

Yes, but because their musicbrainz recoridng id was different it failed. Although all songs had the same MusicBrainz Release Group Id duplicates did not have same mbrecordingid therefore cannot determine same MusicBrainz Song first.

Now instead of using mbrecordingid we could look at title field, but then we are relying on textual metadata this is less safe and wouldn’t work in this case anyway. Also would be confusing for user, since we have these clearly defined MusicBrainz Ids we should use them whilst providing less robust options for getting those hard to find duplicates if user wants to accept the greater risk.

[paultaylor]

Ok raised https://jthink.atlassian.net/browse/SONGKONG-2517

[meaning]

cool ! I assume this should be doable in this way :

1. you add a general folder (lets say letter A folder)
2. songkong processes the sub folders and will only take care of duplicates found in same folders only

nice one !

[paultaylor]

Yes, that is correct.

[meaning]

So, in the end, two things would improve the duplicates tasks to be run :

1. the possibility to call a specific profile using the cli
2. run the following duplicates tasks in this order : 1. same MB and same album (all versions) / 2. same discogs album / 3. sounds the same in same album folder

running these 3 tasks would get rid of most (if not all) of the dupes.

[paultaylor]

1. is now done in SongKong 10.0, also we have added Same song and same album (metadata only) and sounds the same option which i think is what you actually need rather than same album folder option.

SongKong 10.0 Nowhere released 13th of November 2023

We are pleased to announce the release of SongKong 10.0 Nowhere on the 13th of November 2023 [nowhere] This is a major release with a new task called Match to One Album Match to One Album The original idea of SongKong was to allow fix of whole music collection with one click. But over the years we realized that many customers like to work one album at a time and with the existing solution you cannot check the match before it is saved to your files. It would not be feasible to check results…

[meaning]

Thanks for this new version, it’s always appreciated to see the new features coming in (and you picking a nice album to name the new release).

I’m trying this now on a few artists folders I have albums with double or triple tracks, I’ll keep you posted !

Just to be sure, about point 1 : how can I call the specific profiles from the cli ? can you point me to the right documentation ?

I still belive the best way to process the collection is to :

1. fix tracks -> matched tracks goes to matched folder -> unmatched tracks goes to unmatched folder
2. first duplicates run -> same MB same album (any version)
3. second duplicates run -> same Discogs, same album (any version)
4. third duplicates run -> Same song and same album (metadata only) and sounds the same
5. rename tracks to their respective letter/artists/album (format) folders

Does this feel right to you ?

I have the feeling that after a well run fix task, processing the dupes using “Same song and same album (metadata only) and sounds the same” only could be a faster option, while still being safe due to the fact it has to be same track, same album AND sound the same ?

[paultaylor]

@meaning

Just to be sure, about point 1 : how can I call the specific profiles from the cli ? can you point me to the right documentation ?

Simply add

-p fullpath_to_property_file

to specify a profile to use

I’m going to expand the tutorial to explain some of the new features, the help has not been updated it is considerable effort to update the help and I am considering whether it would be better to move the standard help into the forum, that way I could update as and when instead of being tied to a release, could also get feedback from customers, what do you think ?

@meaning

I still belive the best way to process the collection is to :

1. fix tracks -> matched tracks goes to matched folder -> unmatched tracks goes to unmatched folder
2. first duplicates run -> same MB same album (any version)
3. second duplicates run -> same Discogs, same album (any version)
4. third duplicates run -> Same song and same album (metadata only) and sounds the same
5. rename tracks to their respective letter/artists/album (format) folders

Does this feel right to you ?

I think so, but I would test 4 further since because it is not using MusicBrainz/Discogs ids maybe still a risk of deleting songs you dont want to delete, worth checking the results to see ensure happy with it.

[meaning]

@paultaylor

Simply add

-p fullpath_to_property_file

OK, I’m abroad for a few days, but this is something I’ll add to “Autokong” and test once back.

I believe it’s beter to keep the 3 duplicates tasks running in a row, it’s just a matter of time, but will secure the duplicates moves.

@paultaylor

I am considering whether it would be better to move the standard help into the forum, that way I could update as and when instead of being tied to a release, could also get feedback from customers, what do you think ?

Yes, sounds like a good idea indeed.

[meaning]

I have one extra question for you @paultaylor.

In this very specific case :

See how the mp3 encoder decided to put brackets around the track titles ?

If I run the duplicates task using " Same song and same album (metadata only) and sounds the same", I believe it will not identify the “bracket” tracks as dupes. Reason is simple, they are not named the same.

So, back to my old request, and based on the fact you changed the way sonkong fixes things on a per folder basis, can I assume I now safely can run a duplicates task using “sounds the same only” on, lets say, the full BOC folder, and that only tracks that sounds the same located in the same folder will be identified as dupes ?

By this I mean, it shoud NOT delete files that have the same name and are located in different alums folders. Did I correctly understand that this is how SK will behave from now on ?

[paultaylor]

@meaning

If I run the duplicates task using " Same song and same album (metadata only) and sounds the same ", I believe it will not identify the “bracket” tracks as dupes. Reason is simple, they are not named the same.

Correct.

@meaning

So, back to my old request, and based on the fact you changed the way sonkong fixes things on a per folder basis, can I assume I now safely can run a duplicates task using “sounds the same only” on, lets say, the full BOC folder, and that only tracks that sounds the same located in the same folder will be identified as dupes ?

No, sorry no changes here, I have not yet implemented the Only Match Within Folder option so it would not be safe unless you ran on single album folder.

[paultaylor]

@meaning

can you point me to the right documentation ?

Added this page to the tutorial Tutorial 57: SongKong Command Line

[paultaylor]

Only Match Within Folder now added in new release SongKong 10.1 Placebo released 13th of Decemeber 2023

[meaning]

Thanks for this @paultaylor,

so now, I need to adapt the script to add this “same folder only” scan at the very end of its process. (before the rename task of course)

so here is wat needs to be done as far as I understand ;

add several duplicates finder tasks types, using the profile syntax. where I pick each type of duplicates tasks to run.
finish with one additionnal “same folder only” and for that specific job, select “sounds the same” only. This should get rid of the remaining dupes at that point.

I’ll have a look into it and will update you.

[paultaylor]

Yes, that sounds correct.