Well, now this is a mess :)

[meaning]

First thing first,

I obviously keep my tracks secure in the cloud.

As you do know, @paultaylor, I took some time to migrate my music locally, and spent quite some time fixing tracks.

I ran a duplicates finder job and it achieved yesterday.

Today I’m checking the results, and I can say it’s at least, unconvincing

this is just an extract, but as you can see songkong did delete stuff everywhere and I am now missing tracks in almost all my albums. randomly.

I guess my only option is now to re-snatch all my music from the cloud, back locally, and start all over again. trying to figure out what went wrong.

I send you the support files of my last duplicates run, as this is the one that most likely processed the extracts of above screenshots.

[meaning]

Only thing I could think about is that all these deleted tracks were part of compilations (???).

That would explain why some albums are missing tracks. Hopefully.

I’ve seen this :

I’d like to always keep tracks, if they are located in both albums and compilations. so what’s next ? I re-download all my music, and fix the tracks, and put allthe “various artists, compilations, etc.” to a dedicated folder outside of the one I will be processing for duplicates ?

[meaning]

I will re-snatch everything from the cloud and start fresh…

Can you please let me know, once done, how to make sure that next time I run a track fixing job:

everything that is classical goes in a specific folder called “Classical” outside of the general music folder.
everything that is Various artists, goes to a specific folder called “Various Artists” outside of the general music folder.

Thanks !!!

[paultaylor]

Hi, I will come back to rename masks for Fix Songs but i can see the problem

Report 91 (run on 1M files) has Song is a duplicate if has same set to Same MusicBrainz song and same album (specific version e.g. same country/date) which is reasonable

But report 92 (run on 2M files) has it set to Same song (metadata only) so this option only uses metadata rather than MusicBrainz Ids and is purely identifying duplicate songs without caring about the same song on different albums !

Did you pick this option on purpose or by accident ?

[meaning]

@paultaylor

Did you pick this option on purpose or by accident ?

Well, this is funny, as I usually don’t re-check these settings anymore, and assume they remain the same each time I click on “start” again.

For some reason, the setting changes from “same album/version” to “same song”, which fucked up 1 million files

So, I deleted everything yesterday, and will re-snatch all from scratch. It will take weeks…

Now, I think it is interresting to see why these settings may have changed between two tasks, without clear human action.

By the way, something worth to put in another jira ticket -> the "When find duplicates delete folder and any sub folders if empty or only contains the following files " filter works great, but I noticed that it is case sensitive. so, files named *.jpg were removed, but *.JPG were not. I think it would be nice to make this work whatever the format (upper/lower case) is, don’t you agree ?

[paultaylor]

Hi, yes I know you have spent alot of effort and your diligence in doing this has been very helpful in improving SongKong.

@meaning

Well, this is funny, as I usually don’t re-check these settings anymore, and assume they remain the same each time I click on “start” again.

But I’m really not aware of any issue with it changing this setting without user interaction, can you try a few preview runs on a few files to see if you can make it happen again because at them moment I feel that maybe you did change it by accident.

Some good news is we also improved the speed of Fix Songs, so it should be quicker this time although the speed increase not as dramatic as with Delete Duplicates.

@meaning

By the way, something worth to put in another jira ticket -> the "When find duplicates delete folder and any sub folders if empty or only contains the following files " filter works great, but I noticed that it is case sensitive. so, files named *.jpg were removed, but *.JPG were not. I think it would be nice to make this work whatever the format (upper/lower case) is, don’t you agree ?

Maybe as addtional option, but think would be dangerous to do by default.

[meaning]

well, yeah, but really, it’s not such an issue, I just need to be patient again, the time to resnatch stuff.

And yeah, we’ll reach the perfect songkong soon

And hell yeah ! I was thinking about reaching out to you to see if I could get a specific access to the AWS in order to have a faster processing of my next fix job. Does this need an update on the docker side, or is it a change you operated at AWS level ?

[paultaylor]

@meaning

And hell yeah ! I was thinking about reaching out to you to see if I could get a specific access to the AWS in order to have a faster processing of my next fix job. Does this need an update on the docker side, or is it a change you operated at AWS level ?

It was these changes https://jthink.atlassian.net/browse/SONGKONG-2370 https://jthink.atlassian.net/browse/SONGKONG-2387 made in SongKong 8.10 Frank

[paultaylor]

@meaning

everything that is classical goes in a specific folder called “Classical” outside of the general music folder.
everything that is Various artists, goes to a specific folder called “Various Artists” outside of the general music folder.

I’m not sure if you still need this now you realize why Delete Duplicates went wrong ?

But I will give brief answer, all music being fixed has to share base folder at some point, and when files are saved will use this base folder unless you set new base folders with Move Matched/Move Unmatched. So if your music is in /music, and you want classical to be in /classical then your base folder folder would have to be / and therefore your rename mask would have to add literal /music and /classical to path. Whereas if you wanted classical in own subfolder under /music e.g /music/classical then you can have /music as base folder and would only have to add /classical as literal to path if the album is a classical one.

We have existing masks such as IsClassical/HD/AlbumArtistorArtist/Album/DiscIfMultiDisc/Track - Title that demonstrate adding a path only if something true, the rename mask is

addClassical()
+ addHD()
+ ifnotempty2(albumartist,artist,'/')
+ ifnotempty(album,'/')
+ ifmultidisc('Disc ' + ifnotempty(pad(discno,2),'/'))
+ ifmultidisc(ifnotempty(pad(discno,2),' - '))
+ ifnotempty(pad(trackno,2),' - ')
+ title

and the isClassical() function is

function addClassical()
{
 return isclassical =='1' ? 'Classical/' : 'PopRock/';
}

so under the base folder this creates either a Classical or PopRock folder, you get the idea.

[meaning]

@paultaylor

I’m not sure if you still need this now you realize why Delete Duplicates went wrong

Well, it has nothing to do with the failed task. It’s just that I noticed that a lot of crap VA’s needs to be curated appart.

And regarding Classical music -> I hated the fact I had a lot of “artists” folders containing the composer + a crazy long list of musicians, conductor, etc., I’d rather have them all in a sub-section.

[paultaylor]

Okay does my answer make sense for you then?

[meaning]

yes it does. I have no issue putting classical releases in their /music_fixed/classical/, it’s allright ! I just want to avoid generating thousands of “artists” that actually will only have a single album occurence due to the fact they are a mix of composer_conductor_etc.

[paultaylor]

Fair enough, note there is a Remove Composer from Album Artist option on the Classical tab, with that enabled the Album Artist should then only contain Conductor, Orchestra, Principle Performers meaning you get a bit less variation.