Here’s my status report.
Interestingly, I have about 39000 tracks, but the report only seems to have analyzed about 26000 of them. Not sure why. It started working on the report when the Completion bar was only about 2/3 of the way across. Running on Synology / Docker.
Support files pointed to this topic!
Thanks!
Hi, there are some errors on the Warning tab that is why not all songs were processed
and in the log we see
insert into SongDiff (diff, recNo) values (?, ?) [23505-199]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:457)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:427)
at org.h2.message.DbException.get(DbException.java:205)
at org.h2.message.DbException.get(DbException.java:181)
at org.h2.mvstore.db.MVPrimaryIndex.add(MVPrimaryIndex.java:131)
at org.h2.mvstore.db.MVTable.addRow(MVTable.java:546)
at org.h2.command.dml.Insert.insertRows(Insert.java:180)
at org.h2.command.dml.Insert.update(Insert.java:132)
at org.h2.command.CommandContainer.update(CommandContainer.java:133)
at org.h2.command.Command.executeUpdate(Command.java:267)
at org.h2.jdbc.JdbcPreparedStatement.executeUpdateInternal(JdbcPreparedStatement.java:200)
at org.h2.jdbc.JdbcPreparedStatement.executeUpdate(JdbcPreparedStatement.java:154)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:147)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:208)
It seems somehow it is trying to use the same id for a song, are you running two invocations of SongKong or something or I wonder if it is hitting some sort of Docker restrction.
I will review the status report I have but if you could try and rerun it and select all fields in the Show these fields in the Status Report option that would be helpful.
Because of the errors encountered running SongKong the barcharts are wrong, they have been calculated agianst the total library (39,000) rather than the processed songs (27,000)
The actual figures are that basic metadata is good with Artist, Album, Title and Track No at 99%
But Album Artist is only at 72% indicating an issue, these appear to be mainly the albums not matched to MusicBrainz
55% of songs have been matched to MusicBrainz but only 3% of songs have a SongKong Id indicating the matching was done by another application such as MusicBrainz Picard.
We can see the albums not matched by selecting Browse then Browse by Artist/Album, and setting the Matched to Album Filter and select With Full Album Match. So the first step would be to run Fix Songs against your music collection, it can be run in Preview mode first to ensure you are happy with the results.
The file structure looks okay with folder representing albums but there are many albums not matched to MusicBrainz, the Folder structure should not be modified at this point it should only be modified once the metadata has improved.
For the albums already matched to MusicBrainz there are some albums that have been matched incompletely, there are fifty artists that have albums that have not been matched, these are usually worth investigating. Select Matched, then Matched to MusicBrainz and set the Matched to Album filter to Without Full Album Match to see them. Further investigation shows that there are different issues, here we just show a few examples
Incorrect Version of Album match
Either Missing Tracks on your system or version with your tracklisting not available online
Missing Discs on your system
Matched to multiple versions of release
There is one Discogs album where the complete album has not been matched, these are usually worth investigating. Select Matched, then Matched to Discogs and set the Matched to Album filter to Without Full Album Match to see them. Further investigation shows that
Ok, this is incredibly helpful. Thank you.
I did a bunch of work today on a number of unmatched albums, and I just reran the status report on the full library and then uploaded support files (they should have just been sent.
One process / workflow question - once you’ve run fix songs (or rename) in preview, is there any way to “execute the changes” or do you have to go back and run again not in preview? I have been going through and fixing albums progressively, and once I’m done I hate to have to go back and “do it again” because it does take some amount of time, and presumably we’ve already figured out what exactly to do. But maybe I’m oversimplifying!
I have a few questions that have come up as specifics:
I went through the folder 2023Q3 and I think I’ve managed to match everything except for one single album (see 2 below). Is there any way to run rename files on all except that album? It looks like 3 of the tracks are matched (though I can’t see to what), and I don’t want to separate the tracks. But I’d like to make some progress moving things over without getting stuck.
When I try to match to one album for this release: https://www.discogs.com/release/7025147-Mozart-Artur-Rubinstein-Wallenstein-Golschmann-Piano-Concertos-Nos-17-23 I keep getting the error “Unable to find a valid mapping for release 7025147”. I don’t think it’s in MusicBrainz - I can’t find it after a lot of trying. It’s on Amazon https://www.amazon.com/Arthur-Rubinstein-Mozart-Concertos-Nos/dp/B000003FL2 but that’s about all I can find, and the CD is long buried in my attic. I could create a MusicBrainz release in order to be done with it, but that feels a little crazy if I can avoid it. Is the Discogs release just too skeletal? FWIW, this album is in “/music/flac/2023Q3/2023-08-25/Mozart - Artur Rubinstein , Wa” and I’m fairly certain that’s the right release.
Is there a way to rename files that aren’t matched? If, for instance, I can’t get this Rubinstein one to match, is there a way to force it to use the metadata I have or manual metadata to run the same filename mask?
Many many thanks!
John
Hi, so no you cannot do that currently.
The history of this is my other tagger Jaikoz does work like this, all the changes are made in preview and then you save changes as you wish, but I was finding that customers were not checking results before saving, and then if there was an issue it was only discovered when they were playing back the music.
So I took a different approach with SongKong, the chnages are made as they go along, this is also useful when matchin large collections because if anything goes wrong the stuff already processed is already saved, and because not all stored in memory the memory reuirements are low compared to Jaikoz. And because the changes are also stored in database you can use Undo Fixes task to revert the chnages at a later date
So as I said it wasnt part of the original design, and also it was expected that customers would do collection in one go not one album at a time (if that is what you mean by progressively), and also once we display report we just have a standalone html report it is not intended to deal with further requests.
But thinking about it a bit more I supose we could add a Apply Preview task that takes any preview task (Fix Songs/MetaGrater ecetera) and apply the changes as recorded in the database so I will look into this idea further because you are not the first to ask for this.
I went through the folder 2023Q3 and I think I’ve managed to match everything except for one single album (see 2 below). Is there any way to run rename files on all except that album?
If Rename Files Based on Metadata is set to Yes, if matched to Release then only renamed if matched to a Discogs or Musicbrainz release so if this one is not it will not be renamed.
When I try to match to one album for this release: https://www.discogs.com/release/7025147-Mozart-Artur-Rubinstein-Wallenstein-Golschmann-Piano-Concertos-Nos-17-23
Have you tried
Using Match To One Album and specifying the url in the Enter MusicBrainz url or Discogs url… field and then enabling the Force allow low scoring match (this field is only enabled if have entered a url)
Yes, set Rename Files Based on Metadata to Yes, for all files - but this can be a dangerous option so use carefully.
I will look at this tomorrow.
So use Browse by Folder and navigate to the folder, and as you say three are matched, But they are matched song only, we have identified the song but not identified an album that works for all six songs, remember a song can be on multiple albums
But select a song
And select the MB Recording Id field
and you can see the album this song is found on. It is on two different albums but neither seems to match the album you have.
I couldnt see Match to One Album in the latest support files, maybe you ran it later but try using Match To One Album and specifying the Discogs url in the Enter MusicBrainz url or Discogs url… field and then enabling the Force allow low scoring match (this field is only enabled if have entered a url).
If it still doesnt allow match the most likely culprit is a big difference in the length of at least one of the tracks and the track times recorded in the Disocgs release.
If it still fails please rerun Create Support Files
I had a look at your complete Status report, the problems are still the same there are just more examples and I dont think it changes my advice
Hi Paul-
So maybe I don’t understand the workflow. Let’s say that I run status report, and I find that, for instance, the Aaliyah album has been matched to the incorrect release (DVD, wrong # of tracks).
So I go to “Match to One Album”, and I find the correct release (either your algorithm finds it, or I go and research it directly and force the match to the MB URL). So I go through, and it looks good, and I hit save. Do I then go through and “Fix Songs”? I find that often when I do that, it seems to revert to the originally matched album, and when I eventually go to rename files & move folders, it messes up the structure because it’s matched to the wrong album. But I’m certain that I picked the correct album in Match to One Album and then hit save. Can that process become undone? I feel like I need a very step-by-step what to do for each of these use cases, or at least each of the big ones, and thus far I haven’t found exactly what to do, button by button, in the tutorials.
Thanks,
John
I think the issue is that on Fix Songs you changed the For Songs Already Matched option to Rematch so that it ignore any existing matches, if you change it back to Update Metdata Only then it will only try to match songs that have not already been matched.
But also note if have already run Match to Album on an album then there is no need to run Fix Songs, they basically do the same thing but Fix Songs can be run against multiple folders at once.
Ok, that’s helpful.
So do I need to run “fix songs” at all on a folder that is all matched to music brainz releases?
I guess here’s my question.
I have a folder with 100 albums in it (say, /music/flac/2023Q1). 3 of them have incorrect matches and 4 have missing tracks etc, and I find all of these by going to “Browse by Artist/Album” and selecting “Without Full Album Match”. The other 93 albums are “green”. I go through and I run Match to One Album on the 3 incorrect matches and hit save. And the 4 missing tracks are real, so i have a decision to make about whether to accept them as incomplete or delete the albums, or just let them move to my “unmatched” directory.
But do I then run “fix songs” on the whole directory as a big batch job? That seems to undo the work I’ve done in some cases. Do I want to run “fix songs” on the rest of the directory? I think I do, because I want to run the “Roon” profile. But it seems to undo any work I’ve done. I could run it in preview and then try to spot any issues that occur, but I’m unsure how to selectively apply them and have the files moved when they’re right.
Maybe my workflow is wrong - when I have an individual album that is well matched, I should have SongKong Roon profile, rename and move the files during that match, and that way they won’t be in the directory any more when I run the batch “Fix Songs” on all the remaining fully matched albums?
This tool is SO powerful, but the actual workflow is confusing and requires me to understand what each component does and how they interact in a way that is a lot. Any explanation step-by-step of what you do in your own library, or an example of what exactly you’d do in this case, would be very very much appreciated.
Hi, I think you are overthinking, its simpler than this
If you mean matching the wrong album again after you have matched to the right album that is simply because as I said earlier you have set For songs already fully matched to Rematch
when it should be set to Update Metadata Only
Do I want to run “fix songs” on the rest of the directory? I think I do, because I want to run the “Roon” profile
But both Match to One Album and Fix Songs should have a Roon profile, if for some reason Match to One Album does not you could always create one.
So lets just try and categorise the differences
My Recommended Work Flow is simply:
Thank you, super helpful.
So the painstaking part is after an initial full library “Fix Songs”, where you go through and make sure that albums didn’t mismatched.
Ok, now knowing that, what I should do is probably go back to a full backup before I started messing around. Is there a way to make SongKong “forget” about all the work / reports / change history it’s already done? I want to “start fresh” on a new version of my library (I was always prepared to do this).
Thanks!
Yes, although you only really need to look at the one listed in the Matched to MusicBrainz/Discogs Release as incomplete and in your original status report these were not matched by SongKong but another tool, so may be less after running Fix Songs. Could you let me know what tool you originally used to match with please because Im working on a few issues identified as part of reviewing your Status report, and I need to make a decision about the issue related to the missing musicbrainz trackid field and what I do depends partly on what tool caused that issue in first place.
what I should do is probably go back to a full backup before I started messing around. Is there a way to make SongKong “forget” about all the work / reports / change history it’s already done?
You can run Undo Fixes and this will undo make any changes made by SongKong since the last time the database was created (by default database only recreated when update to new SongKong version)
You can remove existing reports with Admin:Delete reports
You can keep changes but remove SongKong memory of them by recreating database with Admin:Empty Database
Whether good idea to do any of this is a different question, unless things are messed up I would just run Fix Songs (maybe preview first) and take it from there.
So I think I previously used MusicBrainz Picard, but I’ve had this library for a longtime and tried various things so I can’t be certain. The question is whether there’s valuable data or whether I should use MetaGrater and clean it out and allow SongKong to do appropriate full matching based on metadata and folder structure.
I’m going to send you a status report now on this thread on my full library with full file names (this is a backup). And before I start in on the workflow you describe (Fix Songs on all to Roon profile, then match to one Album on all incomplete matches + decisions about what to do for incomplete albums, then when everything is great finally rename all) I’d love your opinion on whether the former matching I’ve done is worth keeping or cleaning out. Also happy to try that on a copy or just definitely not if you think it’s crazy to do).
Thanks!
John
I havent seen your latest support files yet but SongKong uses a combination of:
for identification.
So if you remove the metadata you have removed one valuable source of information, and this will probably mean that SongKong will not be able to match some albums that are sucessfully matched. For example although acoustic fingerprints are very useful for matching songs, if your song is the first time that acoustic fingerprint has been submitted there will be nothing to match the song to, and acoustic fngerprints are not so useful for matching the right album
Also, you had many albums not matched to MusicBrainz so if you have already applied Picard to whole library it is likley that many of these albums are not in MusicBrainz, may not be in Discogs or Bandcamp either and therefore SongKong will not be able to match them. So if you remove their metadata then you may end up with albums with no metadata at all!
When I last looked at your songs it seemed for the ones that has been matched that most had already been matched to the right albums, only a few to possibly the wrong version of right album and didnt seem to be any matched to completely wrong albums. So I dont think removing the metadata would be helpful, removing metadata is only a good idea if it is completely wrong and fooling SongKong into matching songs to the wrong albums.
So really the best thing to do would be run Fix Songs in Preview mode on whole collection then send me support files so we can see what results we have.
Ok, thank you again for the advice. Will do.
Turns out my NUC 10i7 shut down due to thermal throttling after 15 minutes of running Fix Songs on the whole library. Not a bad way to find out - I’m new to Linux and this is an older machine, and it hasn’t gotten to writing anything so non-destructive I think. I have to futz a bit - new fan and thermal paste, update the bios. Won’t take me too long, back soon.
EDIT: Actually, while I figure out my NUC thermal issues, I can run SongKong in a docker container on my Synology 918+ NAS, which is WAY less powerful, but will actually finish (albeit slower). So started with restoring a complete separate backup of my library before I started messing around with SongKong, running Fix Songs on the whole library at one go with Roon Profile and will then run Status Report and share back when done.
Small point, but Fix Songs report is essentially a Status Report therefore no need to run a Status Report against same songs if you have just run Fix Songs.
