SongKong Jaikoz

SongKong and Jaikoz Music Tagger Community Forum

Running songkong cmdline in script in Docker environment

Hi, okay I see the error, sorry there was another issue with the rename masks. Because there is only one example mask I only could see what happened when song was not Classical so it seemed okay, but when I modified the mask to reverse the logic

isclassical != '1' ? 'Classical/' : ifnotempty2(albumartist.substring(0,1),artist.substring(0,1) , '/')

I could see the issue, the rest of the mask is being included as last part of the ternary operator.

What we need is some brackets around the ternary operator to give us

Rename mask

 (isclassical == '1' ?  'Classical/'  :  ifnotempty2(albumartist.substring(0,1),artist.substring(0,1) , '/') )
+ ifnotempty2(albumartist,artist,'/')
+ ifnotempty(album + ' (' + audioformat + ',  ' + mbreleasetype +') '  
+ (albumversion.length > 0 ? ' (' + albumversion + ')'  : ''),
'/')
+ ifnotempty2(albumartist,artist,' - ')
+ ifnotempty(album,' - ')
+ ifmultidisc(ifnotempty(pad(discno,2),' - '))
+ ifnotempty(pad(trackno,2),' - ')
+ title
+ ifnotempty(pad(trackno,2),' - ')
+ title

Compilation Rename Mask

   ( isclassical == '1' ? 'Classical Compilations/' :  'Compilations/')
    + ifnotempty2(albumartist,artist,'/')
    + ifnotempty(album + ' (' + audioformat + ',  ' + mbreleasetype +') '  
    + (albumversion.length > 0 ? ' (' + albumversion + ')'  : ''),
    '/')
    + ifnotempty2(albumartist,artist,' - ')
    + ifnotempty(album,' - ')
    + ifmultidisc(ifnotempty(pad(discno,2),' - '))
    + ifnotempty(pad(trackno,2),' - ')
    + title
1 Like

It appears this one is functioning correctly! I’m now going to reprocess the entire ‘music_matched’ folder that was previously mishandled. All of these compilations will be organized into their dedicated folders. I’ll keep you updated if I encounter any issues, but I’m quite optimistic!

Thanks again for the time you spent helping me with these masks paul ! :slight_smile:

Ah, I was full of hopes, but looking at the renamed files, for classical releaes, I notice the track names repeats the track name 3 times :

EDIT : it also does the same weird 3 times track naming for the albums (non compilations) :

Can you run in preview mode and see if get same results when look at results in report.

Strange it appears 3 times, but I can see you left this twice in the masks :

  • ifnotempty(pad(trackno,2),’ - ')
  • title
  • ifnotempty(pad(trackno,2),’ - ')
  • title

I’ll keep only one of them and will see if I can get rid of these repeated track titles

Yes that’s just a typo

Allright, obviously, the title got repeated each time this was present in the mask :

  • ifnotempty(pad(trackno,2),’ - ')
  • title

Corrected it and running it again :wink:

We are seeing the light at the end of the tunnel !

Two things.

Firstly, I looked at your songkong-debug0-0.log files and there seemed to be a problem with finding some files

e.g

12/09/2023 13.25.38:CEST:AbstractLoadFilesWorker:createMapOfFileToAudioFile:SEVERE: Failed AddSongToDatabase for file(1):/music/Music_dump/01-2015/oscar mulero/Grey Fades to Green/09 Dreams of Happiness.mp3:Unable to find:/music/Music_dump/01-2015/oscar mulero/Grey Fades to Green/09 Dreams of Happiness.mp3
java.io.FileNotFoundException: Unable to find:/music/Music_dump/01-2015/oscar mulero/Grey Fades to Green/09 Dreams of Happiness.mp3
	at org.jaudiotagger.audio.AudioFile.checkFileExists(AudioFile.java:224)
	at org.jaudiotagger.audio.AudioFile.checkFilePermissions(AudioFile.java:241)
	at org.jaudiotagger.audio.mp3.MP3File.<init>(MP3File.java:382)
	at org.jaudiotagger.audio.mp3.MP3FileReader.read(MP3FileReader.java:39)
	at org.jaudiotagger.audio.AudioFileIO.readFile(AudioFileIO.java:356)
	at org.jaudiotagger.audio.AudioFileIO.read(AudioFileIO.java:192)
	at com.jthink.songkong.fileloader.worker.AbstractLoadFilesWorker.createMapOfFileToAudioFile(AbstractLoadFilesWorker.java:154)
	at com.jthink.songkong.fileloader.worker.AbstractLoadFilesWorker.addNewFilesSubList(AbstractLoadFilesWorker.java:306)
	at com.jthink.songkong.fileloader.worker.AbstractLoadFilesWorker.addNewFiles(AbstractLoadFilesWorker.java:289)
	at com.jthink.songkong.fileloader.worker.LoadFolderWorker.loadFiles(LoadFolderWorker.java:575)
	at com.jthink.songkong.fileloader.worker.LoadFolderWorker.call(LoadFolderWorker.java:89)
	at com.jthink.songkong.fileloader.worker.LoadFolderWorker.call(LoadFolderWorker.java:42)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
	at java.base/java.lang.Thread.run(Thread.java:832)

I don’t know why this, are these networked files and the network went down, are you moving some files whilst running task in SongKong, or are your running more than copy of SongKong at same time ?

Whatever the reason, this may be the reason it doesnt seem to do anythig after a while because failing to load the majority of files.

Secondly, your latest dump files still do not create any RenameFiles reports, but it looks like they were created and I have tested locally they get added when I try it myself. So could you show me the contents of SongKongs report directory to see if I can make any sense of it.

I’ve observed that, possibly due to unsuccessful file transfers over the years, some FLAC files appear to be corrupted. This could be because they lack a header or because their file size is too small.

It seems that SongKong might struggle to handle these corrupted files efficiently. While this is understandable, as it’s not the primary purpose of SongKong, I took the initiative to write a script today. This script scans all my files and relocates them if the FLAC file is found to be corrupted.

import os
import subprocess
import sys
import multiprocessing
import shutil
import time

total_files_analyzed = 0
total_corrupted_files = 0

def check_flac_file(file_path, output_folder, log_file):
    global total_files_analyzed
    global total_corrupted_files

    total_files_analyzed += 1

    if os.path.getsize(file_path) < 1024:
        with open(log_file, 'a') as log:
            log.write(f"Corrupted FLAC file (too small): {file_path}\n")
        total_corrupted_files += 1
        return "File too small"

    result = subprocess.run(["file", "--mime-type", file_path], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)

    if "audio/flac" not in result.stdout:
        with open(log_file, 'a') as log:
            log.write(f"Corrupted FLAC file (no header): {file_path}\n")
        total_corrupted_files += 1
        return "No FLAC header"

    return None

def scan_folder_for_flac(input_folder, output_folder, log_file):
    with open(log_file, 'w') as log:
        log.write("Starting FLAC file checking script...\n")

        for root, _, files in os.walk(input_folder):
            for file in files:
                if file.endswith(".flac"):
                    file_path = os.path.join(root, file)
                    log.write(f"Checking {file_path}...\n")

                    corruption_reason = check_flac_file(file_path, output_folder, log_file)

                    if corruption_reason:
                        # Recreate the directory structure in the output folder
                        relative_path = os.path.relpath(root, input_folder)
                        new_output_folder = os.path.join(output_folder, relative_path)
                        if not os.path.exists(new_output_folder):
                            os.makedirs(new_output_folder)

                        shutil.move(file_path, os.path.join(new_output_folder, os.path.basename(file_path)))
                        log.write(f"Corrupted FLAC file moved ({corruption_reason}): {file_path}\n")

def log_progress(log_progress_file):
    while True:
        with open(log_progress_file, 'a') as log:
            timestamp = time.strftime('%Y-%m-%d %H:%M:%S')
            log.write(f"{timestamp} - {total_corrupted_files}/{total_files_analyzed}\n")
        time.sleep(10)

if __name__ == "__main__":
    if len(sys.argv) != 4:
        print("Usage: python3 flac_checker.py /music /corrupted_flacs /app/flac_checker.log")
        sys.exit(1)

    input_folder = sys.argv[1]
    output_folder = sys.argv[2]
    log_file = sys.argv[3]

    log_progress_file = "/app/progress.log"
    
    p = multiprocessing.Process(target=log_progress, args=(log_progress_file,))
    p.start()

    scan_folder_for_flac(input_folder, output_folder, log_file)

    p.terminate()

this is a quick way to get rid of all these files. the script checks each file for a missing header or an under 1kB filesize, if one of these two conditions is met, it moves the file, together with its whole folder structure, to a destination folder.

the log will then clearly show which files are checked :

Checking /music/01-2021/Aleksi Perälä - Midnight Sun 4 EP-1 (2021) APMU39EP1 - FLAC 24 96000/01. FI3AC2139290.flac...
Checking /music/01-2021/Aleksi Perälä - Midnight Sun 4 EP-1 (2021) APMU39EP1 - FLAC 24 96000/03. FI3AC2139310.flac...
Checking /music/01-2021/Aleksi Perälä - Midnight Sun 4 EP-1 (2021) APMU39EP1 - FLAC 24 96000/02. FI3AC2139300.flac...

and the files that were moved, and why :

Corrupted FLAC file moved (No FLAC header): /music/01-2021/Bernard Szajner & Zed - Some Deaths Take Forever (2021) [WEB 24bit FLAC] {Cortizona}/02. Bernard Szajner & Zed - Ritual.flac
Corrupted FLAC file moved (No FLAC header): /music/01-2021/Bernard Szajner & Zed - Some Deaths Take Forever (2021) [WEB 24bit FLAC] {Cortizona}/03. Bernard Szajner & Zed - Execute.flac
Corrupted FLAC file moved (No FLAC header): /music/01-2021/Bernard Szajner & Zed - Some Deaths Take Forever (2021) [WEB 24bit FLAC] {Cortizona}/04. Bernard Szajner & Zed - Ressurector.flac

and the progress log shows the amount of corupted files moved against the amount of analysez tracks :

root@Tower:/mnt/cache/appdata/flac_checker# tail -f progress.log
2023-09-13 17:21:40 - 0/0
2023-09-13 17:21:50 - 0/0
2023-09-13 17:22:00 - 0/0
2023-09-13 17:22:10 - 0/0
2023-09-13 17:22:20 - 0/0
2023-09-13 17:24:29 - 0/0
2023-09-13 17:24:39 - 0/486
2023-09-13 17:24:49 - 0/620
2023-09-13 17:24:59 - 0/715
2023-09-13 17:25:09 - 0/824
2023-09-13 17:25:19 - 0/935
2023-09-13 17:25:29 - 0/1052
2023-09-13 17:25:39 - 0/1156

I’ll keep it running in order to get rid of all these files before I run songkong again.