SongKong Jaikoz

SongKong and Jaikoz Music Tagger Community Forum

Risks of running a renaming of unidentified tracks?

#1

Hi @paultaylor,

So now that I ended up with one identified music folder, renamed…

I still have the backlog of albums/tracks that songkong did NOT identify.

here is my question, what’s the risk of running another fix tracks job on that one, and simply use tracks tags to rename the files ?

If I do so, will I still be able to run other fix jobs on these already renamed files ? and get them identified the day bandcamp / beatports gets implemented ?

Is there also a risk I could see a “Various_artists - blabla” album renamed and moved in specific artists folders ? like one track per artist, per folder ?

#2

You could use Yes, if has metadata but I think there is a high risk some files will be renamed not to your liking, perhaps you should in Preview Mode and look at the results. But why rename them, do you want them moved to /music_processed or kept in /music_not_processed, you can move them without renaming them. In fact if not identified just renaming them wont move them to /music/processed since they are still not identified.

But yes this would not stop identification at later point.

Yes, there is.

1 Like
#3

ok, I am running another fix track job now, as you told me it could identifiy previously unidentified tracks.

1.162.800 tracks to go.

Question, for future runs, is there really no way to enable more concurrent checks by enabling extra workers ? I just spent like 4 months finalizing my initial run, and I think this one might run for another month.

Other question, as I will never end fixing tracks, is it safe to install another instance of songkong dedicated to finding duplicates ?

So I will keep one instance that will keep fixing tracks, and the other one will regularly check my “processed” folder for duplicates. Do you confirm this will work even without the database information of instance A ? The duplicates (instance B) will then base itself on the tracks that were renamed and moved by instance A

#4

It does processing in parallel making use of your multiple cpus, but when you have a powerful machine with many cpus the bottleneck can be file I/O when saving and renaming files, becuase this doesnt benefit from more cpu power.

It does not need the database, however first time it will have to load the metadata for the songs into its own database, this will be slower than reading from the database. Of course you dont want the two instances to interefere with each other so as you are using docker make sure their config mapping is to different physical locations.

#5

ok.

So, I’ve got 61,5% of my music identified, renamed and moved to their respective artists folders in /music_processed after a first run.

that means I still have 38,5% of albums which were not identified. I am now running a duplicates job on these 61,5%.

What do you think about these results ? Are they in aligned with usual results you are getting ?

What now ? Should I keep running fixes on the remaining 1682235 items that composes the 38.5% ?

#6

Lower than usual, normally in the 85% range, but you have alot of unsigned artists. You just need to run Fix Songs once more on the unidentified albums.

Also I think that you have some albums identified but there was a problem moving/renaming them so kept in original location, so these will probably get fixed this time.

#7

Ok in the meantime, I am running a duplicates job.

But damn it is slow as hell, how can it be ?

image
image972×554 74.1 KB

This is what was processed after barely an hour.

No, really, this can’t be normal, right ? How can it take that long to simply delete duplicates that were already matched?

#8

So assuming different instance what is has to do is read the file metadata into database for each folder of songs, and if any causes a duplicate with what is already loaded then it has to work out which one to keep and delete the other one.

Does seem very slow, could it be file I/O again ?
If you have Fix Songs running in a different instance loading files, renaming and saving files could that be causing a bottleneck making it hard for Delete Duplicates to load files, I dont know I have never tried to run SongKong twice at the same time on the same machine, and I dont use Docker much. Maybe better to try Delete Duplicates on your Mac, or just have one instance and wait for Fix Songs to complete ?

#9

nope, there is no secondary instance running at all at this time. It was just a question I was asking.

This is my usual songkong instance here, processing the already matched, renamed and moved tracks.

I am super suprised it only processed this amount of files in more than 5 hours

image
image900×470 70.5 KB

at this speed, it will take months to only remove the duplicates.

#10

Yeah, okay that isnt right.

Now, either there is issue with yours in particular, or there is some performance issue with the way I redesigned Delete Duplicates over the last few releases, but its not jumping out at me. I will do some analysis but it may take a while as I have other things I need to get done.

#11

Although, its no longer multithreaded, the new algorithm was working unreliably when multiple folders were being processed at the same time so I had to change it, but it shoukdnt be thnis slow even with only one thread.

#12

Well, unraid let me have a quick look to all my drives write/reads :

image
image2628×870 263 KB

there is definitely no I/O issue here. :smiley:

#13

Also, the debug log shows this all the time :

09/12/2022 17.00.29:CET:SessionUtil:commitAndClose:SEVERE: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.jthink.songlayer.SongFile#495010]
org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.jthink.songlayer.SongFile#495010]
at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:2541)
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3285)
at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:3183)
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3525)
at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:159)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:465)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:351)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:350)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1258)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:425)
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101)
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:177)
at com.jthink.songkong.util.SessionUtil.commitAndClose(SessionUtil.java:51)
at com.jthink.songkong.analyse.duplicates.DeleteDuplicatesLoadFolderWorker.loadFiles(DeleteDuplicatesLoadFolderWorker.java:119)
at com.jthink.songkong.analyse.duplicates.DeleteDuplicatesLoadFolderWorker.call(DeleteDuplicatesLoadFolderWorker.java:61)
at com.jthink.songkong.analyse.duplicates.DeleteDuplicatesLoadFolderWorker.call(DeleteDuplicatesLoadFolderWorker.java:31)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
09/12/2022 17.00.29:CET:Errors:addError:SEVERE: Adding Error:Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.jthink.songlayer.SongFile#495010]
java.lang.Exception
at com.jthink.songkong.analyse.general.Errors.addError(Errors.java:28)
at com.jthink.songkong.util.SessionUtil.commitAndClose(SessionUtil.java:80)
at com.jthink.songkong.analyse.duplicates.DeleteDuplicatesLoadFolderWorker.loadFiles(DeleteDuplicatesLoadFolderWorker.java:119)
at com.jthink.songkong.analyse.duplicates.DeleteDuplicatesLoadFolderWorker.call(DeleteDuplicatesLoadFolderWorker.java:61)
at com.jthink.songkong.analyse.duplicates.DeleteDuplicatesLoadFolderWorker.call(DeleteDuplicatesLoadFolderWorker.java:31)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)

#14

Ok, that shouldn’t be happening

#15

Can you send me a copy of that log file please.

#16

sure, but, are the logs of a running job sent if I send you the support files ? Or do I have to cancel the running job ?

To be honest, I’d rather stop it now and figure out with you why the duplicates finder is taking so long. It’s silly i’ll have to wait like 30 days to finalize this task.

#17

You have to cancel jobs to run support files, that is why I suggested just sending the log file, but if don’t mind stopping the job then it’s fine to do that and then run Create Support Files

#18

I’ve sent you the logs in a PM + uploaded the support files this morning, since task was running too slow anyway.

#19

Hey paul,

Did you find something in these logs ?

#20

Havent had time yet, been busy with other things but will look later today.

1 Like