Songkong putting mp3 files in existing flac folders

[paultaylor]

@meaning

any clue why the rollback / revert feature is not working at all ?

No I am sorry it should have worked (unless you ran Empty Database since running Fix Songs Report 13) and if you try to rollback /music/music_processed and it works that would also delete your other recent Fix Songs tasks that were run against Flac so that is no good.

But could you try this, just select a subfolder containing mp3s processed in report 13 and set Find Songs to that are currently in selected locations because I want to see if Undo Changes is not working at all for you or only if trying to undo from original location for moved files.

You could try the /music_processed/!!!/ folder

[paultaylor]

Re Dropsonde issue, you have two Flac versions of the album, and the bitrate of the songs in the (1) folder were generally slightly higher so these were kept and they were removed from other folder.

But for Sherbrooke and In the Shape of a Flute the bitrate was the same, so therefore it picked the file with the earliest creation date as that was the first criteria to differ

But one folder also contains Mp3, for example there only seems to be an Mp3 version of Fall in, fall Out in the original folder, so in these cases the mp3 is removed in preference to Flac

[paultaylor]

I think moving Earliest Creation Date up to 3rd in the list just behind Audio Format may be the best way to get consistency in deleting files form the same folder.

[meaning]

Undo Changes Report 28

Options

Current Profile: Default

Find songs : that are currently in the selected locations

• Found 12 songs previously fixed by SongKong
• Of these found 12 songs that had outstanding changes undone by SongKong
• Songs matched in 1 minutes 1 seconds
• Undo Changes Report 28 started at Nov 9, 2022, 8:45:59 PM
• Undo Changes Report 28 finished at Nov 9, 2022, 8:47:01 PM

[meaning]

OK in the meanwhile I’ve ran another duplicates task on that !!! folder.

And one more time it messed up things. (Thanks god all these files are safely backed up in the cloud)

As an example, I now do have “Let it be blue” folders, one containing track 5, and another one containing all other ones, except track 5.

If you want an example that can be read in the reports, have a look at Aphex Twin folder.

I can see this issue in any artist folder that was processed. A track in folder 1, other ones in folder 2, …

I understand that we are acting on “tracks” but this is not what I actually want to achieve. I want to tell songkong to check the tracks, compare them and see if they are the same (Ex: same tracks in folder 1 and 2), if so -> delete the folder that has the lowest encode quality (Ex; keep flac, delet mp3 version).

If a folder has the original tracks + extra tracks, I want songkong to keep best rip of the full album that contains the extra tracks.

For now, I end up with folder with a few, or even a single track, which will totally screw my library. I will restart a fix job for now, as these are taking time, and I’ve changed the renaming scheme to add mp3 to the folders.

Can we please check what we can do to achieve exactly what is expected as described above ? Keep 1 version of each album, each time in the best possible audio format, instead of getting files deleted here and there ?

Thanks !

[paultaylor]

Okay so it seems to just be not working if moved and you specify the orginal location, I will raise an issue. In the meantime you could go through the folders in Report 13 and send them all to Undo Changes but that is rather cumbersome, though maybe you could script it.

Or just run Fix Songs with the new renaming mask so that the Mp3 files will be moved to new folder, although I realize that will take some time but if you temporarily uncheck Search for a Musicbrainz and Update/Search from Discogs should be alot quicker.

[paultaylor]

@meaning

I understand that we are acting on “tracks” but this is not what I actually want to achieve. I want to tell songkong to check the tracks, compare them and see if they are the same (Ex: same tracks in folder 1 and 2), if so -> delete the folder that has the lowest encode quality (Ex; keep flac, delet mp3 version).

It does do that, but if sometimes the lowest quality track is in folder 1, and sometimes it is in folder 2 so I dont see your solution.

But I did suggest modifying the Criteria Order, and I think ordering as follows

• Highest Bit Depth
• Highest Quality Audio Format
• Earliest Filename Creation Date
• Orignal Filename
• Highest Bitrate

(others dont really matter)

then running with Album( Same Version) first would be good start.

Lets try that (maybe in in preview only) and lets see the results, i,.e it wont delete all duplicates but I think it will delete the correct ones.

Criteria will also work better for Album (Any version), but not always work, we need to add the longest album criteria.

[meaning]

Yes I did read your advice and used a test folder to test it.

But unfortunatelty it kept messing up things, an extract:

You can see the result in the screenshots I’ve provided yesterday.

As far as I can see, it deletes files because they appeared earlier on my hard drives, which could be an option if my mp3 files were actually “older” than the flac ones. This is true, indeed, but the transfer of these files to my server makes the usage of this unpredicatable, as I have copies of the same album, copied at different times. So you clearly can see songkong deletes files based on that criteria and therefore keeps one file here, another file there.

So I’m afraid of running the next job to be honest

[meaning]

now the good news is this :

So at least plex/roon wont add album with double tracks due to the combination of mp3/flac files in the same folder(s).

[paultaylor]

@meaning

As far as I can see, it deletes files because they appeared earlier on my hard drives, which could be an option if my mp3 files were actually “older” than the flac ones. This is true, indeed, but the transfer of these files to my server makes the usage of this unpredicatable, as I have copies of the same album, copied at different times. So you clearly can see songkong deletes files based on that criteria and therefore keeps one file here, another file there.

So I’m afraid of running the next job to be honest

Hmm, this is wrong it should be keeping the one with earliest creation date, not deleting it. I tested this and it is working for me so the problem is not simply that i have the logic wrong way round

So maybe best to leave Delete Duplicates for now until I resolve this. But would be good if you could at least run in preview only and send support files to see what results are, also there is no problem with mp3 files since they will always be deleted before flac because lower quality format.

[paultaylor]

@meaning

now the good news is this :

So at least plex/roon wont add album with double tracks due to the combination of mp3/flac files in the same folder(s).

Im not sure what you are showing me here.

[meaning]

just showing that albums in mp3 format are now correctly moved to /album_name (Mp3)/

[meaning]

OK I run another duplicates check now.

here are my settings :

preview mode, to avoid messing thing up again

I send the support files now.

In short, I really think these earliest/latest settings are messing up things and brings me to have these folders with a few tracks here and there.

I’d rather keep one single occurence of the album, only based on count of tracks + best possible quality.

[paultaylor]

Thanks, so I looked at the Occer release and it found two duplicate keys, and for both it kept the ones with the earliest file creation date which was in /music_processed/Arovane/Occer - Silicad (3), so that is right isnt it ?

Are any of the duplicates deleted in that preview actually wrong, I cant see any ?

But concerned about your previous example where it wasn’t keeping earliest file creation date, I have gone through my code and haven’t yet been able to work out why it didn’t keep the earliest one, and I cannot reproduce the issue. I wonder if the problem can only occur when files are actually deleted rather than in preview, but still doesn’t make any sense why that would be.Do you think you could run Delete Duplicates again just against /music_processed/Arovane first in preview only again, and then with preview disabled so we can see if the results are consistent with the first time ?

I think if File Creation Date did work properly that would generally work because usually all files in one folder will be copied over at about the same time, and they will all be earlier than if a later version of the album is created. But what would be better maybe would be Parent Folder Creation Date, since all files in a folder will have the same value., and then if File Creation Date came after that would deal with duplicate files in same folder.

Then if I had another criteria based on value of the Track Total field, that would prefer songs that are part of longer version of album.

[meaning]

Sure, I made a backup of my Arovane folder and will run it again in preview, then without preview. Same settings. I’ll shoot the support files. but for you to know already the order of the filters :

@paultaylor

I think if File Creation Date did work properly that would generally work because usually all files in one folder will be copied over at about the same time

Are you 100% sure songkong, during the move process, will process each album one after the other ? Then I guess this should work, indeed. But then it will keep the best quality over the most recent file, right ?

[meaning]

WOW, I’m checking the last report and please do so as well.

as you can see, the real deal (no preview mode) did delete some stuff, but I can also see some albums that have multiple copies were not affected at all :

If you go to the report, starting the “Ve Palor” line, you’ll end up to 404’s. each time !

and looking at folders contents, here we go again :

this one only has tracks 3 and 4 left

this one only has track 2

this one has 1 and 5

and so on.

unfortunately I cannot use the duplicate finder as it ends up with such results.

even scarier is this “Ve Palor” album.

I am analysing both the processed artist folder and the backuped one.

if I open all the remaining folder of Ve Palor in the processed folder, I can see this :

in the above folder, track 1 is missing

All other folders are now empty.

But when I go to the backup folder, Track 1 is not missing. So it was removed, but I don’t have a clue why this happened. And it is nowhere to find anymore in the processed folders. Scary.

[paultaylor]

@meaning

But when I go to the backup folder, Track 1 is not missing. So it was removed, but I don’t have a clue why this happened. And it is nowhere to find anymore in the processed folders. Scary.

The error you reported earlier reoccurred and this prevented completion of report, still struggling to work out quite why it is happening.

Okay, thanks for your patience, I have enough to work on so I think probably best if you just carry on with Fix Songs whilst I work on some fixes for Delete Duplicates.

[meaning]

Thank you Paul. I’ll leave the fix song task run for a few days.

[paultaylor]

Okay I think all issues fixed in SongKong 8.6 Post released 15th Novemeber 2022

[meaning]

I am now using the (Mp3) tag in folder names so I’m afraid I’m not gonna be able to test that one back.