Make SongKong a multi step program

[avan]

This is my first run on my NAS server. I have a lot of files. Songkong was exiting due to unknown reasons. I run in folder by folder with simple automation.
while read line;
do echo “$line”;
./songkong.sh -m “$line” >> nohupMedia02.out 2>&1 ;
done < fileListMedia02

The fileListMedia02 is the list of song folder at the lowest folder level with an entire list of 485 subdirectories. I rerun songkong again at the top folder and it ran smoothly.
Is it possible to make songkong run in batches instead of trying to accomplish a lot with smaller memory? Just saying that not everyone can run processes using command line utilities.

I recently purchased both Jaikoz and Songkong, both are using similar approach with same amount of memory utilization. It will be very useful to make songkong to work on one directory at a time and then aggregate using multiple passes.

[paultaylor]

Actually memory utlization is very different, Jaikoz has to store all the metadata for the songs loaded in memory, as such there is a limit to how many songs can be loaded. Whereas SongKong does work on folder by folder basis and only loads the metadata for the currently processed folder(s) into memory, however it makes use of concurrency so will usually be processing more than one folder at a time.

So SongKong can be used on systems with limited memory, however by default it can use plenty of memory available on the system, on linux if you look at your songkong.sh file it should start something like

java -Xms150m -Xmx1800m

the -Xmx defines the max amount of memory it can use, in most cases you should be able to reduce to 500m without problems.

If you run the remote ui and select Create Support Files from the web-browser that will send me your log files and reports and then I may be able to see what the problem is. Since we have added the ability to control SongKong from a web-browser the need for a command line has reduced, although I see it could be useful for some automated processes.

[avan]

There is only one Java program. I showdown all others except core OS processes.

!

[avan]

Just an update, I am not sure why it is consuming a lot of memor?

[paultaylor]

The total amount of heap memory uses is specified by -Xmx as I have explained, there is also additional stack memory used to run the application but heap memory is the most important aspect. So Im not really sure what you are asking me?

[avan]

Thank you for looking into it. I was able to tag all my music within a day. Even though memory was consuming all of the swap space, that didn’t stop the program working as expected.

[paultaylor]

Great, but it would only swap space if the max memory setting meant that there wasn’t enough physical memory to fulfill request, interested to know how much physical memory the machine had.

[avan]

I figured out a possible case:
BTFRS file system, java is capable of handling it however, not all the same. Some JRE’s take more resources to manage IO operations with BTFRS file systems. Btfrs is an excellent choice for NAS appliances but it is relative new technology in computer science terms.

I will keep looking into it while listening to my music library. Thank you for following up with the issue.