Loaded Songs Count dont match when run FixSongs from Cmdline

[meaning]

@paultaylor

So MusicBrainz Song Only matches should not set the IsClassical field, raised https://jthink.atlassian.net/browse/SONGKONG-2503

Cool! in the meanwhile I’ll keep the super restrictive settings, it will already help me move some fully identified albums, and next time, when this will be fixed, there will be less tracks to analyse. Please remember we discussed Classical (is classical) but that I am also using the “Is compilation” tag to put VA’s in /Compilation. I guess this will also be problematic and needs to be considered aside “Is classical”

@paultaylor

we dont currently consider Discogs matches for Delete Duplicates task.

Is there a security reason therefore ? Are Discogs matches not considered as “full albums” ? I paste this again, just to understand, last column definitely indicates all these tracks were matched on discogs, and have the very same discogs release ID.

@paultaylor

Regarding duplicates Same MusicBrainz song and same album refers to MusicBrainz album, since it has not be matched to MusicBrainz album it would not be considered as a potential duplicate, we dont currently consider Discogs matches for Delete Duplicates task.

This is reassuring.

[paultaylor]

@meaning

Cool! in the meanwhile I’ll keep the super restrictive settings, it will already help me move some fully identified albums, and next time, when this will be fixed, there will be less tracks to analyse

Okay, but as I said above this doesnt prevent this problem occurring.

I would recommend you hold of running the automated rename task until we have resolved all Fix Songs issues, Im going to review your latest run in a bit.

@meaning

but that I am also using the “Is compilation” tag to put VA’s in /Compilation. I guess this will also be problematic and needs to be considered aside “Is classical”

No, I have checked this and this is okay, we don’t set this field for a MusicBrainz Song Only match

@meaning

Is there a security reason therefore ? Are Discogs matches not considered as “full albums” ? I paste this again, just to understand, last column definitely indicates all these tracks were matched on discogs, and have the very same discogs release ID.

No it just that MusicBrainz is our primary database and haven’t had time to add Discogs. We could certainly do it but with Discogs we could only add the Same Discogs song and same Album (specific version) because Discogs doesn’t have as advanced relationships between albums as MusicBrainz does,

Discogs album is considered a matched release in the context of Fix Songs task

[meaning]

@paultaylor

Okay, but as I said above this doesnt prevent this problem occurring.

OK so you recommend me to keep settings like this, right ?

and by the way, since this night, around 2am I’d say, Jthinkserver times out:

22/09/2023 08.04.39:GMT:JthinkSearchServer:doQuery:SEVERE: Read timed out
java.net.SocketTimeoutException: Read timed out

This happens when all tracks has been loaded and songkong starts to query your server. Might be a good idea to have a look at it

[paultaylor]

@meaning

OK so you recommend me to keep settings like this, right ?

I’m not necessarily recommending it just point out to you that keeping the more restrictive criteria will not prevent the isclassical problem. However, yes I would keep the less restrictive settings because if all songs in your grouping are matched to an MusicBrainz album that is better them not being matched to an album. And if they are all matched that should mean it is indeed the correct album and you just have an incomplete rip, or you have matched to the right album/release just not to the right version (because the exact version you have is not in database)

But before that what would be really good is once server fixed is you reran against same folder with more restrictive setting because would be good to compare how many of the extra matches second time round were because less restrictive setting and how many were just because found some more matches because of metadata added first time.

I’m going to check the results in FixSongsReport213 and FixSongsReport216 now because there were a couple of things in your original FixSongsReport0213 that didn’t seem quite right.

@meaning

and by the way, since this night, around 2am I’d say, Jthinkserver times out:

You are correct, never seen this error before seems to be issue with Apache server at front end which is not something i usually have to deal with

https://jthink.atlassian.net/browse/SONGKONG-2504

Im starting new server, takes a while to initalize though, should take about 20 minutes.

[paultaylor]

Server is working okay now.

[paultaylor]

So in FixSonsgReport213 you had only configured to match incomplete releases i.e noOfSongsInGrouping==noOfSongsInAlbum and if I look at the releases Matched to MusicBrainz that worked fine, although there were not many matches they were all complete matches

But if I look at Matched to Discogs there are some matches where the grouping has matched to a release with more songs then grouping (orange badge)

Check the folder we can see it only has three songs, and so it should not be matching to this 5 song release

I think the issue is related to the fact that one of the songs was matched MusicBrainz song only and that has split the grouping allowing a Discogs single song match, I need to look into this more

BTW there is also the case where Matched to Discogs shows a badge in orange indicating partial badge, but it is actually valid but occurs when the Discogs track listing includes subtracks

So this is correct, but I need to adjust report so doesnt show these as incomplete, Discogs subtracks are quite awkward to deal with.

[paultaylor]

The second issue I noticed is that the subsequent RenameFiles214 has only moved the 174 songs that were matched to MusicBrainz releases in FixSongs213, but it should have moved the 23,549 songs matched to Discogs releases

I cannot currently replicate this issue, it is working correctly for me.

[paultaylor]

@paultaylor

I cannot currently replicate this issue, it is working correctly for me.

Okay found the issue, it is a regression introduced in SongKong 9.0, see https://jthink.atlassian.net/browse/SONGKONG-2507

[meaning]

well, the next version of songkonw will have some fixes

OR, if you tell me it’s easy to fix, and an intermediar fix is coming, I’m pausing my fixes for now.

[paultaylor]

I have fixed that particular issue and the classical issue but unlikely to do a new release today though.

[meaning]

Whenever you want/can my dear @paultaylor. thank you so much for being that responsive fixing bugs!

[paultaylor]

@paultaylor

So this is correct, but I need to adjust report so doesnt show these as incomplete, Discogs subtracks are quite awkward to deal with.

Raised as https://jthink.atlassian.net/jira/software/c/projects/SONGKONG/issues/SONGKONG-2505 but quite obscure so probably wont fix this for a while.

[paultaylor]

SongKong is multi-threaded and task are submitted onto various queues, we currently have some logging to monitor this to ensure all task are completed and just used it to fix a bug. But it is only for debugging really so I have moved it out of songkong-user.log to a new log called songkong-process.log

But it occurred to me maybe there is something you use for the script, so this is the sort of output I was going to move

29/09/2023 08.30.32:BST:SEVERE: >>Submitted:StartMatcher to MusicBrainzUpdateSongAlreadyMatched:G114:Foldername:The Wall - Copy:isPartOfMultiDisc:false:NoOfSongs:26::Queue:15:totalSubmitted:33:totalDone:18
29/09/2023 08.30.56:BST:SEVERE: >>WorkDone:MusicBrainzUpdateSongAlreadyMatched:G48:Foldername:Delicate Sound of Thunder:isPartOfMultiDisc:false:NoOfSongs:15::Queue:1:totalSubmitted:34:totalDone:33
29/09/2023 08.31.10:BST:SEVERE: >Closing Latch:
29/09/2023 08.31.10:BST:SEVERE: >MainAnalyser Completed
29/09/2023 08.31.10:BST:SEVERE: >MainAnalyser Shutdown:true
29/09/2023 08.31.10:BST:SEVERE: >MainAnalyser Terminated
29/09/2023 08.31.10:BST:SEVERE: >SongSaver Shutdown:true
29/09/2023 08.31.10:BST:SEVERE: >SongSaver Finished

Can you check if this would cause any issues for you.

[meaning]

nope, I am actually only using the output of the cli to generate the notifications. Thanks for having thinked about this paul !

[meaning]

Is something heppening today ?

The tasks are super slow. It looks like only one core is used atm, and all my disks are spn down :

fixing a fodler with less than 7k files took almost 5 hours.

[paultaylor]

Server is all running okay, maybe something specific to this folder, perhaps you could send me reports and logs, I see there is one error logged.

[paultaylor]

Hi, seems there was a problem with AcousticBrainz server, but seems to be okay now https://acousticbrainz.org/

If still having problems you can try disabling Update Mood and other acoustic attributes such as BPM option then SongKong should no longer need to access this 3rd party server

[meaning]

it’s something else, look at the rename task, nothing is happening, literally :

29/09/2023 12.59.53:GMT:MonitorExecutors:outputTraceForThread:SEVERE:...

FYI, here are the settings I’m using :

But as you can see, songkong is not doing anything during the rename task for that folder. I will try to skip to the next one. To see if things are improving.

[paultaylor]

Hmm, you reported it for the Fix Songs task so I don’t understand the reference to the Rename Files task. The best thing to do is always send me report and logs, without both of those very difficult to work out problem. You can do this either by running Create Support Files immediately after failure or zipping up your archival, unfortunately there is some issue for me trying to download myself if the on the fly zip is greater than 15mb.

SongKong 9.6 Whirlpool released 29th of September 2023 has some of your requested fixes and improvements.