SongKong and Jaikoz Music Tagger Community Forum

meaning · 1 October 2022 10:41

Hi there,

I am processing a quite big amount of files. this morning I did notice songkong docker was stopped.

Here are the logs I could get

    at com.jthink.songkong.analyse.analyser.SongSaver.completeSongs(SongSaver.java:1551)
    at com.jthink.songkong.analyse.analyser.SongSaver.saveSongsToFile(SongSaver.java:1612)
    at com.jthink.songkong.analyse.analyser.SongSaver.saveChanges(SongSaver.java:277)
    at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:244)
    at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:77)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
    at java.base/java.lang.Thread.run(Thread.java:832)

java.io.FileNotFoundException: /songkong/Reports/images/_music_processed_The Philadelphia Orchestra, Arturo Toscanini_Mendelssohn; Debussy; Respighi The Complete Philadelphia Orchestra Recordings 1941-42 (disc 3 Ein SommernachtstraumIncidental Music to A Midsummer Night’s Dream, Opp. 21 & 61 - Symphony No. 6, Op. 74 Pathetique).jpg (File name too long)
at java.base/java.io.RandomAccessFile.open0(Native Method)
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:347)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:261)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:216)
at java.desktop/javax.imageio.stream.FileImageOutputStream.(FileImageOutputStream.java:69)
at java.desktop/com.sun.imageio.spi.FileImageOutputStreamSpi.createOutputStreamInstance(FileImageOutputStreamSpi.java:55)
at java.desktop/javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:419)
at java.desktop/javax.imageio.ImageIO.write(ImageIO.java:1549)
at com.jthink.songkong.analyse.analyser.FolderImageGenerator.generateFolderImage(FolderImageGenerator.java:87)
at com.jthink.songkong.analyse.analyser.FolderImageGenerator.generateFolderImagesForSongs(FolderImageGenerator.java:55)
at com.jthink.songkong.analyse.analyser.SongSaver.completeSongs(SongSaver.java:1551)
at com.jthink.songkong.analyse.analyser.SongSaver.saveSongsToFile(SongSaver.java:1612)
at com.jthink.songkong.analyse.analyser.SongSaver.saveChanges(SongSaver.java:277)
at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:244)
at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:77)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
java.io.FileNotFoundException: /songkong/Reports/images/_music_processed_The Philadelphia Orchestra, Arturo Toscanini_Mendelssohn; Debussy; Respighi The Complete Philadelphia Orchestra Recordings 1941-42 (disc 2 La merThe Sea - Iberia (Images, No. 2) - Feste romaneRoman Festivals - Romeo et Juliette, Op. 17 Sherzo).jpg (File name too long)
at java.base/java.io.RandomAccessFile.open0(Native Method)
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:347)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:261)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:216)
at java.desktop/javax.imageio.stream.FileImageOutputStream.(FileImageOutputStream.java:69)
at java.desktop/com.sun.imageio.spi.FileImageOutputStreamSpi.createOutputStreamInstance(FileImageOutputStreamSpi.java:55)
at java.desktop/javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:419)
at java.desktop/javax.imageio.ImageIO.write(ImageIO.java:1549)
at com.jthink.songkong.analyse.analyser.FolderImageGenerator.generateFolderImage(FolderImageGenerator.java:87)
at com.jthink.songkong.analyse.analyser.FolderImageGenerator.generateFolderImagesForSongs(FolderImageGenerator.java:55)
at com.jthink.songkong.analyse.analyser.SongSaver.completeSongs(SongSaver.java:1551)
at com.jthink.songkong.analyse.analyser.SongSaver.saveSongsToFile(SongSaver.java:1612)
at com.jthink.songkong.analyse.analyser.SongSaver.saveChanges(SongSaver.java:277)
at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:244)
at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:77)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
java.io.FileNotFoundException: /songkong/Reports/images/_music_processed_Maurice Ravel, Claude Debussy, Camille Saint-Saens, Albert Roussel; Academy of St Martin in the Fields Chamber Ensemble, Skaila Kanga_Ravel Introduction et allegro - Debussy Danse sacree et danse profane - Sonate - Saint-Saens Fantaisie - Roussel Serenade.jpg (File name too long)
at java.base/java.io.RandomAccessFile.open0(Native Method)
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:347)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:261)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:216)
at java.desktop/javax.imageio.stream.FileImageOutputStream.(FileImageOutputStream.java:69)
at java.desktop/com.sun.imageio.spi.FileImageOutputStreamSpi.createOutputStreamInstance(FileImageOutputStreamSpi.java:55)
at java.desktop/javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:419)
at java.desktop/javax.imageio.ImageIO.write(ImageIO.java:1549)
at com.jthink.songkong.analyse.analyser.FolderImageGenerator.generateFolderImage(FolderImageGenerator.java:87)
at com.jthink.songkong.analyse.analyser.FolderImageGenerator.generateFolderImagesForSongs(FolderImageGenerator.java:55)
at com.jthink.songkong.analyse.analyser.SongSaver.completeSongs(SongSaver.java:1551)
at com.jthink.songkong.analyse.analyser.SongSaver.saveSongsToFile(SongSaver.java:1612)
at com.jthink.songkong.analyse.analyser.SongSaver.saveChanges(SongSaver.java:277)
at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:244)
at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:77)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
java.io.FileNotFoundException: /songkong/Reports/images/_music_processed_Albert Roussel; Paul Paray, Georges Tzipine, Leonard Bernstein, Sandrine Piau, Michel Moragues, Melos Ensemble, Yvonne Lefebure_Roussel Suite en fa - Sinfonietta - Symphonie N°3 - Deux Poemes de Ronsard op. 26 - Serenade, op. 30 - Trois Pieces, op. 49.jpg (File name too long)
at java.base/java.io.RandomAccessFile.open0(Native Method)
at java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:347)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:261)
at java.base/java.io.RandomAccessFile.(RandomAccessFile.java:216)
at java.desktop/javax.imageio.stream.FileImageOutputStream.(FileImageOutputStream.java:69)
at java.desktop/com.sun.imageio.spi.FileImageOutputStreamSpi.createOutputStreamInstance(FileImageOutputStreamSpi.java:55)
at java.desktop/javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:419)
at java.desktop/javax.imageio.ImageIO.write(ImageIO.java:1549)
at com.jthink.songkong.analyse.analyser.FolderImageGenerator.generateFolderImage(FolderImageGenerator.java:87)
at com.jthink.songkong.analyse.analyser.FolderImageGenerator.generateFolderImagesForSongs(FolderImageGenerator.java:55)
at com.jthink.songkong.analyse.analyser.SongSaver.completeSongs(SongSaver.java:1551)
at com.jthink.songkong.analyse.analyser.SongSaver.saveSongsToFile(SongSaver.java:1612)
at com.jthink.songkong.analyse.analyser.SongSaver.saveChanges(SongSaver.java:277)
at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:244)
at com.jthink.songkong.analyse.analyser.SongSaver.call(SongSaver.java:77)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
Killed

Any clue why this is happening ?

Also, as it did crash I have no report or whatsoever to check

paultaylor · 30 September 2022 07:49

Could you please run Create Support Files, this will then send me all the log files.

meaning · 30 September 2022 08:49

Done

FYI, I started a new fix job. Thing is, as the previous one did not finalize correctly, I guess SK reads the remaining tracks in some file. As the files were already moved, I expect this number to be false. (files were already moved to the processed folder, but are still considered as “non processed” by SK).

Thanks !

paultaylor · 30 September 2022 08:56

Hi, I dont follow. I think you have SongKong configured to only move if matched, not sure why you say they are non processed.

meaning · 30 September 2022 10:00

indeed, here is what I meant :

I ran songknog to move matched released -> 3.3 million tracks remaining
It ran for several hours (last moved folder was Sept 29 at 8.25pm)
after the crash -> I run the same process again (a lot of files were moved to the process folder at step 1) -> SK tells me 3.3 million tracks still needs to be processed.

paultaylor · 30 September 2022 10:28

So if you had 3.3 million files in one folder, and some were moved to matched folder surely that means there are now less files in the original folder and therefore SongKong would not be able to find 3.3 million files in the original folder anymore

Are you sure you actually have a Move Folder configured and some files were actually moved ?

paultaylor · 30 September 2022 10:28

Looked at your logs files, I can see no errors except a few timeouts and a few issues so assume it did just crash without warning, what are you running docker version on because there should be some logs you can look at within the docker container itself.

meaning · 30 September 2022 15:29

Running the docker from my unraid server.

it’s stuck not even listing anything since 9am.

look :

Still I can see it’s working :

if I connect the docker bash, which log can I look at to get real time analysis of the running tasks ? (Eg: processed files, etc.)

thanks

paultaylor · 30 September 2022 16:02

No point continuiing, I would just stop it and restart. If that doesnt work it is possible the crash corrupted the database in which case select the Empty Database option and then rerun.

meaning · 30 September 2022 16:12

There is really no way to run a tail -f on a log to see if something is happening ?

If I remove the DB, I guess SK will “forget” bout already matched albums, not a big deal really… but worth to know.

paultaylor · 30 September 2022 16:16

Okay, you can run a tail on songkong_user0-0.log if you want to get a summary of what is happening if you wish, but I think it has gone wrong and nothing is happening

SongKong saves albums as they are matched, so already matched album will remain already matched and hence if you rerun SongKong o these files they don’t need to be rematched, it will just update the metadata with any changes due to changes being made to your formatting options. You can skip step this by setting For songs already matched to Ignore as well.

meaning · 30 September 2022 16:40

it is working, look

This is really a snapshot and I can see files are been processed. htop confirms this as well.

But in the webUI, nothing is displayed, and no single file was moved since yesterday 8.35pm.

Do you advise me to stop the job, remove the DB and restart ?

meaning · 30 September 2022 16:43

I decided to stop the job, and look :

I loaded approx 50k tracks, but did not rename or move anything

paultaylor · 30 September 2022 18:14

Have you checked your options?

paultaylor · 1 October 2022 09:40

Actually looking at screenshot I can see it hasn’t even saved or completed processing of any files, regardless of if they have been renamed or not. You should check you havent run out of disk space because SongKong internal database and reports can get large, and then need to empty database and retry.

paultaylor · 1 October 2022 10:41

Also songkong_debug0-0.log gives more detail, may show the errors

meaning · 2 October 2022 06:28

Well, I ran another task (fix) and it ran for a few hours, I could see it processing the files, and moving files as well to my processed folder, and… it crashed again.

the debug log is not giving any error.

The docker totally stops though. and shows the same java errors and “killed” as posted previously. So it seems to docker completely stops instead of keeping running.

I have plenty of disk space available for the db :

As my docker container vdisk was limited to 100GB and I’ve noticed this :

I might have reached the max space of the vdisk. I’ve raised it to 500GB and will let you know if it runs longer after that.

update:

This mornign no crash (yet) but one more time, it loads the same amount of tracks should they have been moved already or not :

The log shows that it is processing a lot of files, and all of them are already matched it seems.
Is there a logical order in the way SK processes the files ? or will it process them randomly ?
For me this looks like the files were moved in the previous job, but are still in the DB (as the docker crashed) and therefore re-processed anyway by SK?

here is a snippet of the userlog : https://pastebin.com/zenCxiAv

paultaylor · 2 October 2022 06:39

It hasn’t done much and Completed is still set to zero so something not right. Would be best if you cancel job and then send me full support files again.

Any idea what is different in your setup between this run and earlier run that worked?

meaning · 3 October 2022 08:46

I just left it running this time, and it ended up moving files. I really believe that due to the docker crash, SK DB was not up to date, and started all over again.

Now, it leads me to one question. What if I want to cancel the job for whatever reason (Eg: running out of disk space on destination drive), once I will click cancel, will all changes be reverted ? Or will the job simply be stopped, and all already processed files noted as processed ?

paultaylor · 3 October 2022 08:54

When song gets to completed stage that means any changes made to the song have been saved, so the changes made will not be reverted. If you rerun the job it will not have to rematch previously identified songs but it doesn’t skip over them entirely because you may have changed your options and therefore changes need to be to the songs (e.g enable/disable Add Audio Format to album Title), or maybe the MusicBrainz album matched to has had info added since last run SongKong.

Once you start the task you can pause it if for example you need to free up more disk space, but cancelling the job entirely and then restarting from where you left it is not a workflow directly supported by SongKong.

SK crashes. Memory issue?