No, the script starts the container, runs the script, analyses the process, and will send a notification once the process of the folder is done (using pushover as this is the notification agent I usually use).
Once done, the script stops the docker container, and starts over (skipping to the next folder)
But doesn’t starting the container automatically start SongKong, as it is started at end of the Dockerfile?
fair point. I create a dedicated DockerFile now and will run it as Songkong-cli ^^
ok @paultaylor now have a dedicated docker running called songkong-cli, it doesn’t start songkong.sh at all at start. I can try to run my script again BUT this is a fresh install, and I have no profile configured.
this is the profile songkong seems to use (in the "regular docker) :
#Tue Aug 15 09:27:25 CEST 2023
acousticFingerprint=false
actionIfSongAlreadyMatched=0
addNonAlbumReleaseTypesToTitle=false
classicalFieldsModifyIfEmpty=
classicalFieldsNeverModify=
classicalTrackArtist=4
discSubTitleInTitle=0
discogsGenreFromOption=0
discogsGenreGroupingFromOption=0
discogsGenreGroupingMaxFieldsOption=1
discogsGenreGroupingOverwriteOption=3
discogsGenreMaxFieldsOption=1
discogsGenreOverwriteOption=0
embedArtwork=2
featuredAlbumArtists=1
featuredArtists=1
fieldsModifyIfEmpty=BPM,KEY,MOOD,
fsArtwork=1
identifyClassical=0
ignoreFolderMetadataWhenSongMatching=false
isAddArtistArtwork=false
isAddAudioEncodingToReleaseTitle=false
isAddComposerToAlbumTitle=true
isAddComposerToMinimGroup=false
isAddComposerToMinimWork=false
isAddHdToReleaseTitle=false
isAlwaysUseArtistType=false
isApplyClassical=true
isDefault=false
isFindBackCover=false
isFindFrontCoverart=true
isIgnorePreviouslyCheckedFiles=false
isModifyIfSongOnlyMatch=true
isRemoveClassicalAlbumArtist=false
isSetBoxsetRoonAlbumTag=false
matchAllSongsOnReleaseOnly=true
matchByFolderOnly=true
maxImageSize=1200
minImageSize=200
modifyAcousticAnalysis=false
modifyArtwork=true
modifyGenres=true
movementToTitle=0
noRandomFolders=false
numberPadding=1
preferredMedia=278
preferredReleaseDateType=2
preview=false
profileName=Other runs
putOriginalReleaseYearIntoYear=false
saveArtworkFilename=folder
saveOperaFormat=0
searchDiscogs=true
searchMusicBrainz=true
storeYearOnlyInDateFields=true
translateForeignArtists=true
useArtistName=true
useArtistNameFromRecording=false
useDiscogs=true
useRecordingTitle=false
useReleaseGroupTitle=true
workToGroup=2
how do I make sure I create this profile and keep all its settings before I invoque songkong.sh in the newly cretaed docker container ?
EDIT: it turns out I cannot start java from inside the container I just built.
How does your DockerFile differ from mine ?
I’m asking mysef the same exact question. here is the modified docker file content :
FROM adoptopenjdk/openjdk14:alpine AS build
RUN /opt/java/openjdk/bin/jlink --module-path=/opt/java/openjdk/jmods \
--add-modules java.desktop,java.datatransfer,java.logging,java.management,java.naming,java.net.http,java.prefs,java.scripting,java.sql,jdk.management,jdk.unsupported,jdk.jcmd,jdk.crypto.ec,jdk.dynalink \
--output /opt/songkong/jre
RUN apk --no-cache add \
curl \
tini
RUN mkdir -p /opt \
&& curl http://www.jthink.net/songkong/downloads/current/songkong-linux-docker.tgz?val=11522| tar -C /opt -xzf - \
&& find /opt/songkong -perm /u+x -type f -print0 | xargs -0 chmod a+x
RUN rm -fr /opt/java
FROM alpine:3.11
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
# Votre code pour la configuration d'Alpine, l'installation de GLIBC, etc...
RUN mkdir -p /opt
COPY --from=build /opt/songkong /opt/songkong
RUN apk --no-cache add \
ca-certificates \
curl \
fontconfig \
msttcorefonts-installer \
tini \
&& update-ms-fonts \
&& fc-cache -f
EXPOSE 4567
# Config, License, Logs, Reports and Internal Database
VOLUME /songkong
# Music folder should be mounted here
VOLUME /music
WORKDIR /opt/songkong
ENTRYPOINT ["/bin/sh"]
Hi, took me a while to remember how docker works but okay its simpler that than after all.
This is all I did on Synology
Launched standard songkong/songkong Image but as well as configuring /music and /songkong folders I went to the Environment tab and modified the Command option from
-r
to
-m /music
Then once it started, i selected Details on the container and then Terminal and could see it had started correctly
confirmed this, by going to Log tab
and then once completed processing it automatically terminates
So from command line I think you can use standard image and just need to run something like
docker run -v /CONFIGFOLDER:/songkong -v /MUSICFOLDER:/music songkong/songkong -m /music/DATEFOLDER
replacing CONFIGFOLDER, MUSICFOLDER and DATEFOLDER with actual values.
Note when running SongKong task from command line it will use the profile last selected for the task. So if you want to use a different profile, you would need to first start task with that profile in the WebUI and start task (and then cancel if you wish since profile will now be set for that task).
This is a task for tomorrow morning ! thanks for helping here ! 
problem using docker run is that each time we run docker run, a new ocntainer is created. therefore, it is impossible to use any existing profile.
also, running docker run again and again leaves a ton of unused containers on the server.
I’me trying to run the task using docker exec instead :
Starting to process folder: /music/Music_dump/03-2023/01-03
Sending Pushover notification: Starting to process folder: /music/Music_dump/03-2023/01-03
Executing the command: docker exec f8ccfc3f1ad4 /opt/songkong/songkong.sh -m /music/Music_dump/03-2023/01-03
but this leads to the same behavior as the previous time. The process is simply not starting. I think something needs to be changed inside songkong code in order to allow to start a task using docker xec command.
so here is how I fixed this (somehow). I’ve use the following function to actually run a docker, that gets deleted once the process ends. this to avoid having multiple stopped docker containers piling on my server :
def run_songkong(relative_path):
# Use docker run with --rm to start SongKong in a new container for each folder and delete the container afterwards
container_name = "songkong_{}".format(relative_path.replace('-', '_')) # Naming the container for clarity
cmd = "docker run --rm --name {} -v /mnt/cache/appdata/songkong:/songkong -v /mnt/user/MURRAY/Music:/music {} -m {}".format(container_name, DOCKER_IMAGE_NAME, os.path.join(DOCKER_FOLDER, relative_path))
log_action("Executing the command: {}".format(cmd))
process = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while True:
line = process.stdout.readline().decode("utf-8").strip()
if "Reports:" in line:
log_action("SongKong finished processing for folder: {}".format(relative_path))
break
output, error = process.communicate()
now this has one major issue. as the docker container is pulled each time, I cannot keep any preferences. It means I have to use the default profile to fix my tracks, as it is setup when you inisially install songkong.
If you have a good idea on how to pass the settings through this command paul, I’m taking your advices !
here is how it looks like in the terminal :
So, it works. it starts a container, runs he task, sends pushover notifications, and stops the container / remove it, then goes to the next folder, and so on. I simply need to figure out with you how to make the profile settings persistant. For the fix songs task, it is not super important. But once i’ll move to the duplicates mover / finder, ti will become really important.
of course, as it uses the “default” songkong folders (Eg: it creates the reports in the Report folder and. so on), I might want to simply remove all the profiles, and only keep one. That might do the trick, as songkong won’t have any other profile to use ! I’ll test that later on.
@paultaylor re you sure there is no way to pass arguments to sonkong in cli mode ? as I told you, creating a new docker makes it impossible to pick a template. it would be convenient to allow users to select if they want, Eg, re-check accounsticID for file previously matched or not.
Just run once in gui mode select profile and set options, start task and then cancel, then cmdline will use the last selected profile with last selected options. As long as you have /config configured to a real folder those settings will be preserved regardless of creating or restarting docker containers.
it is running for 5 days now, it ate a lot of CPU and 40GB of ram while fingerprinting files.
but now, I have another “issue”. Since it stopped fingerprinting files, I noticed the CPU usage dropped to almost nothing, and RAM is only used at 8GB still.
yesterday noon (12h30 to be perfectly clear) I was here :
this morning, 8:52am, here :
so it only did save 17913 tracks in 20h of run time. Does that seems possible for you ? what could be slowing down things this way ?
The very weird thing is it actually saved files way faster during the fingerprinting / files check process. Once it was done and fingerprinting didn’t raise anymore, things did slow down. It’s like if SK did stop using the CPU’s somehow.
long story short, I saved approx 300k files in 4 days, while the fingerprinting checks were operated, and now, I calculated it runs at approximately 35k files saved per day.
any clue ?
Fingerprinting is very cpu intensive, it maybe if no fingerprinting to do the delay/bottleneck is getting results from server instead since we have to limit how many calls can be made to server by single instance of application
But also SongKong is only going to actually save the file if changes made. So if file already had fingerprint and was already matched and nothing changed in match then the file doesn’t need to be saved. So what is key is the value of the Completed bar not the Saved bar.
But I thought the idea of the script was to work on one folder at a time, and then stop container and restart because we know SongKong seems to slow down over time with very large number of files.
But this screenshots seem to indicate you are running SongKong in one go on the whole thing again ?
Not really, I have one of my “monthly” folders that contains a big amount of data. The numeber you see represents half of my library and is the “big piece”. but it’s still a monthly folder. all the other ones (next ones) are way smaller.
the script I wrote is identifying folders that have a date format, and will process each of them in a newly started songkong instance. once songkong finished its task, the script detects it, stops the docker container, and goes to the next folder, and start again.
And indeed, it looks like songkong starts to slow down at some point. Question is…why. as you can see it literally ain’t doing anything atm.
Now, I could eventually use the rename feature of songkong, in order to move the files that do have fingerprints and were correctly identified in another folder, that might reduce the monthly folder size upfront next run. as only fully identified albums will be moved and renamed, I guess it might be helpfull. right ?
Can you eventually let me know if these settings seems to be allright ?
I selected FirstLetterOfartists in order to get “smaller” folders, instead of ONE gigantic folder containing thousand of artists.
Now, regarding compilations / various artists, I’d like to get them all in a specific folder called “Various”, the way to get this done is still unclear to me.
in short here is the structure I would like to get :
Music/Releases/Artists/ArtistLetters/Artists/Album/…
Music/Releases/Classical/ArtistLetters/Artists/Album/…
Music/Releases/Compilations/ArtistLetters/Artists/Album/…
Okay it s not really a monthly folder in any sensible way if half your collection is in one folder, since the whole point of the script was to break up your huge collection into more manageable chunks.
I have no idea, it would be helpful if you could send the logs they may give an answer. But I suspect it is to do with something getting too large, therefore the best thing you could do is break up this massive folder to allow you to complete processing.
I hesitate at this stage to get RenameFiles involved, could you just use the linux cmds to break this folder into subfolders of approximately 100,000 songs each ?
well, this specific folder /was/is a mess. I decided to start a rename task in order to move the already identified albums (if all matched to a release).
I’ve told the renamer to move the “marched” items to music_matched and the unmatched items to music_unmatched.
But damn, look what happens now :
boxsets are merged like you can see above.
some fully matched releases only do have a single track:
and the good old flac and mp3 mixes are back :
and I also do have strange rename behavior like this :
is it me, of should I normally ONLY see perfectly organized and renamed files in there as all these files are supposed to be part of a unique release ? with its unique discogs / musicbrainz tag ?
Now, I do undertstand the mp3 / flacs laying in the same folder issue, as these are actually identified as the same album.
But why do I have an album, matched, with a single track which is track5 ?
Why can I see an album in last example where I have twice the same track, once identified as track 09 from disk one, while it is also identified as track 3 of a single disk release ?
why are albums sometimes put in two separate folders,and sometimes not ?
why do I clearly have fully matched albums that are moved, that misses most of their tracks ?
Why are two totally different releases identified as a single one, and put in a single folder as we can see here ?
these are just a few examples as I am browsing the folders songkong is currently moving.
I believe it’s no big deal as I plan to let it run till the end, and I’m only moving files, therefore, I can rollback using the rollback feature of Songkong, right ? But generally, I’m not super comfy with what I’m seing happening.
Hmm, well I was hoping you were going to send me the logs and I did warn against using Rename Files at this point because you should only be renaming files if you have checked and are happy they are matched correctly, that is why we split Rename Files from Fix Songs task There are alot of recent improvements in the FixSongs report to allow you to check things like this quickly.
Having said that if you only have Rename Files set to Move files then the folder structure and filename should remain the same, only the base folder should change. But from what you say above it looks like you have renaming configured as well .
Regarding Box sets It looks like maybe you are using a rename mask that doesn’t use DiscNo to separate discs
In cases where you have only matched one track to release it is not clear if you only have one track of the release, or tracks from a folder have been incorrectly matched to multiple releases. This is easy for you to determine using the Fix Songs report, Browse by Folder section.
If you are renaming and you have multiple versions of the same album then you should probably make audioformat part of the folder structure.
Yes, but it will rollback all changes to the selected folders since the songs were loaded into current SongKong database, so this could include matching/fingerprinting as well as renaming.
Really to help you further I need your support files, and I hope it has a valid FixSongs report and RenameFiles report because these reports contain all the information needed to work out what SongKong has done and are much more useful to then the screenshots posted. The screenshots do not explain why something has happened only the end result.
nice
Support files are on their way 
one thing worth mentioning is that I selected to put my artists inside their own Letter (to reduce the size of my final rename folder, and keep it as “separated” as possible.
Worth to mention as well that aside my LETTER folders, I have tons of folders that are totally not put in a letter folder, sorry one more screenshot needed :
Alltogether, I’d like to be able to trust songkong and not have to go throug this process again and again. it should simply move the albums that were fully matched, without any issue. as this is the meaning of it.
by the way I analyzed the Drukqs folder with picard. the promo (5 tracks) and vinyl versions of the album were merged in a single folder.
here is how it looks like once you import it in a media manager (here, navirome) :
another very straightforward example is OK computer, you can see 3 versions where merged in a single folder by songkong :
here is how it looks like FOLDER wise :
But this time, once imported in navidrome, you can see the album is displayed as 3 distinct releases ! I’m missing something…
Obviously, I’d rather see the Druqks release and its promo been displayed as two separate releases !
Sure, but you are not simply moving the files you also are renaming them and my suspicion is your rename mask is not correct for what you want to do, anyway will find out when look at reports.
Also you say simply move albums that were full matched, when you say album do you mean the collection of songs in a folder on your hard drive or do you mean the MusicBrainz/Discogs album the songs have been matched to?
By album i mean the MusicBrainz/Discogs album. So SongKong will move songs that have been matched to an album (not just a song), and this should usually mean that all songs in folder have to be matched to the same album and therefore all these songs will get moved to same location. But if you only have some of the songs from the album then all songs files can be matched to the album but the album can not be fully matched.
