SongKong and Jaikoz Music Tagger Community Forum

OneTrueDabe · 14 January 2020 12:15

I have a 8-Core Mac Pro w/ 4GB of RAM, and upwards of 86,500 songs in my iTunes Library.

Unfortunately, loading these into Jaikoz (obviously) blows out the JVM heap.

I edited “/Applications/Jaikoz.app/Contents/Info.plist” and bumped the “-Xmx” size up as high as I could – which was about “-Xmx=2124m” – but that still wasn’t enough to load my entire library.

What I ended up having to do was request the JVM to operate in 64-bit mode. Again, in “Info.plist”:

    <key>JVMArchs</key>
    <array>
         <string>x86_64</string>
    </array>

(I’m told that will degrade back to 32-bit if no 64-bit architecture is available.)

Question: Will this have any impact (positive OR negative) on Jaikoz itself?

I searched the Forum but didn’t see any mention, one way or the other, of using a 64-bit JVM. (And I’m posting this in case others run into the same issue.)

Cheers!

–
:- Dabe

paultaylor · 14 January 2020 13:06

Ive not done any work with the 64bit JVM yet, so let us know how you get on.
However I wouldn’t recommend loading so many tracks in one go, it means Jaikoz has alot of work to do just managing the indexes ecetera.

OneTrueDabe · 14 January 2020 13:06

Altogether, I’d say 64-bit was a non-issue – things ran well.

Yeah, I guess I came to the same conclusion… In the end, I decided it was easier to deal with my collection in ~20k batches: 0-D, E-J, K-R, and S-Z.

Each batch took about ~5 minutes to load, ~9 hours to “Autocorrect Tags from MusicBrainz”, and another ~5 minutes to save.

–
:- Dabe

Nocturnal · 14 January 2020 13:06

I find that Jaikoz usually runs out of memory not because of too many files. That just makes Jaikoz really slow because it constantly has to search through and sort the list of loaded files. The problem of not enough memory lies with the mipcore.exe program that extracts the audio from compressed form to wav-form for analysis. When you have an audio file of one hour (a long DJ set, for example), that means 44100 Hz x 16 bit x 2 channels x 3600 seconds = 606MB

This comes on top of all your running programs, the operating system, and Jaikoz itself (which is also using a lot of memory for the file list and all the ID3 information, even more so when you have a lot of cover art).

One thing you can do to reduce the memory usage of Jaikoz, is disable the column that displays the cover art.

I’m currently doing a batch of 60 000 files. The loading of the files went fine. 30 000 of these already have an acoustic ID saved, so for the first step (analysis with genpuid, acoustic analysis) this leaves 30 000 files.

This is on a computer with 3GB of RAM. Windows XP and the JRE are 32 bit. Because everything is 32 bit, the maximum heap I can assign to Java is around 1600MB (currently using -Xmx1612m). This is not a bug in Jaikoz, but a limitation of the 32 bit memory addressing system. For more information for Windows on this limitation, see http://support.microsoft.com/kb/924054.

paultaylor · 14 January 2020 13:06

Thanks for the info, its worth pointing out that if your OS is 64bit such as Windows Vista or OSX Leopard there is also a 64bit version of Java freely available.

mcsetsr · 14 January 2020 13:06

64 JRE 1.6_06 Vista 64 and still no luck with setting the max memory past 2048MB on command line and can not use batch file to load app. With the 2048MB setting I can load 25-26k songs before running out. Seems the application uses double memory when running the 64 jre even at startup it used 122MB without load of 1 song.

If I change this to 4096 it loads a couple soungs then says out of memory.

“C:\Program Files (x86)\Jthink\Jaikoz\jaikoz.exe” -Jmaxheap=2048000000

Also seems if you have some corrupt music files…they use tons of memory. Jaikoz does tell you it is corrupt, so at least I have been able to delete those from my library

paultaylor · 14 January 2020 13:06

[quote=mcsetsr]
Also seems if you have some corrupt music files…they use tons of memory. Jaikoz does tell you it is corrupt, so at least I have been able to delete those from my library :)[/quote]
Maybe you are hitting another corrupt file which it is failing to load, as I stated in another post if you keep stalling at the same point despite increasing your memory try missing out the problem folder. If you have any of these corrupt files that you have identified as increasing the memory load in Jaikoz in your recycle bin perhaps you could send me so I can indentify further.

Im going to write a simple memory analysis test that we can use to identify memory problems when loading specific files.

paultaylor · 14 January 2020 13:06

Have found a bug in Java which can occur when loading large PNG images, the net resuklts oif this is Jaikoz can give an OutOfMemoryError when it hasnt actually run out of heap memory.

For the technically minded more details here:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6687968

Ive done a fix/workaround for Jaikoz 2.5.0 Beta 2/Final which will be available early next week.

heartspeace · 14 January 2020 13:06

Is this still true or have you gone to be able to do 64 bit so we can really use our machines? (8 cores 12 gigs here).

[quote=paultaylor]Ive not done any work with the 64bit JVM yet, so let us know how you get on.
However I wouldn’t recommend loading so many tracks in one go, it means Jaikoz has alot of work to do just managing the indexes ecetera.[/quote]

paultaylor · 14 January 2020 13:06

Jaikoz does work on 64bit JVM

MAzE5h1p69wB · 14 January 2020 13:07

Does it work with memory?

Because for some reason 3.7.1 keeps crashing when I try to specifically tell Java to please do feel free to use the resources in my computer, make yourself at home and work you piece of shit virtual machine.

@echo off
set CLASSPATH=
java -jar lib\\jaikoz.jar -l2 -m2 -f -Xms150m -Xmx3200m -XX:MaxPermSize=150m

06/08/2010 10.47.46:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
LoadingFiles:WARNING: Memory low:init = 16777216(16384K) used = 233072576(227609
K) committed = 259522560(253440K) max = 259522560(253440K)
06/08/2010 10.47.47:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
LoadingFiles:WARNING: Memory low after Garbage Collect:init = 16777216(16384K) u
sed = 180555400(176323K) committed = 259522560(253440K) max = 259522560(253440K)

06/08/2010 10.47.47:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
LoadingFiles:WARNING: Memory low:init = 16777216(16384K) used = 246557536(240778
K) committed = 259522560(253440K) max = 259522560(253440K)
06/08/2010 10.47.47:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
LoadingFiles:WARNING: Memory low after Garbage Collect:init = 16777216(16384K) u
sed = 246893272(241106K) committed = 259522560(253440K) max = 259522560(253440K)

06/08/2010 10.47.47:com.jthink.jaikoz.monitoring.MemoryManager:checkMemoryWhilst
LoadingFiles:SEVERE: Memory too low:init = 16777216(16384K) used = 226006856(220
709K) committed = 259522560(253440K) max = 259522560(253440K)

EDIT: Or is it Windows’ prefetch and standby memory being retarded?

paultaylor · 14 January 2020 13:07

Your memory settings dont seem to have taken effect, are you editing jaikoz.bat but then running Jaikoz.exe, you need to run Jaikoz.bat

MAzE5h1p69wB · 14 January 2020 13:07

Could you point out to me what is wrong in this jaikoz.bat that I am using?

@echo off
 set CLASSPATH=
 java -jar lib\\jaikoz.jar -l2 -m2 -f -Xms150m -Xmx3200m -XX:MaxPermSize=150m

And yes these settings do take effect.

Also it seems that jaikoz / java is using the “Free” memory, then requests more which released “Standby” memory that superfetch fills with stupidity geared for dumb applications like office, uses a lot of cpu in posting these requests each for a minuscule amount of ram such as “gimme 4 MiB more” and stops after a set number of times.

EDIT: Oh and then it craps all over itself because it runs out of perm memory and helpfully informs that maybe the VM needs to be taken out back and shot in the head.


        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
bstractAudioFileBridge.java:121)
        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
FileBridge.java:60)
06/08/2010 11.27.47:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Uncaught throwable:PermGen space on thread:AudioFileLoader_Worker_7 caught by
JaikozThreadGroup
06/08/2010 11.28.19:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Terminating application because a serious error has occurred
java.lang.OutOfMemoryError: PermGen space
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.getDeclaredMethod(Unknown Source)
        at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
        at java.io.ObjectStreamClass.access$1700(Unknown Source)
        at java.io.ObjectStreamClass$2.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.io.ObjectStreamClass.<init>(Unknown Source)
        at java.io.ObjectStreamClass.lookup(Unknown Source)
        at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
        at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
        at java.io.ObjectInputStream.readClassDesc(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at java.util.HashMap.readObject(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
        at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)

        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
bstractAudioFileBridge.java:121)
        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
FileBridge.java:60)
06/08/2010 11.27.41:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Uncaught throwable:PermGen space on thread:AudioFileLoader_Worker_4 caught by
JaikozThreadGroup
06/08/2010 11.28.19:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Uncaught throwable:PermGen space on thread:AudioFileLoader_Worker_1 caught by
JaikozThreadGroup
06/08/2010 11.28.20:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Terminating application because a serious error has occurred
java.lang.OutOfMemoryError: PermGen space
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.getDeclaredMethod(Unknown Source)
        at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
        at java.io.ObjectStreamClass.access$1700(Unknown Source)
        at java.io.ObjectStreamClass$2.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.io.ObjectStreamClass.<init>(Unknown Source)
        at java.io.ObjectStreamClass.lookup(Unknown Source)
        at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
        at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
        at java.io.ObjectInputStream.readClassDesc(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at java.util.HashMap.readObject(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
        at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)

        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
bstractAudioFileBridge.java:121)
        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
FileBridge.java:60)
06/08/2010 11.28.16:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Uncaught throwable:PermGen space on thread:AudioFileLoader_Worker_2 caught by
JaikozThreadGroup
06/08/2010 11.28.16:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Uncaught throwable:PermGen space on thread:AudioFileLoader_Worker_5 caught by
JaikozThreadGroup
06/08/2010 11.27.52:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Terminating application because a serious error has occurred
java.lang.OutOfMemoryError: PermGen space
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.getDeclaredMethod(Unknown Source)
        at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
        at java.io.ObjectStreamClass.access$1700(Unknown Source)
        at java.io.ObjectStreamClass$2.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.io.ObjectStreamClass.<init>(Unknown Source)
        at java.io.ObjectStreamClass.lookup(Unknown Source)
        at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
        at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
        at java.io.ObjectInputStream.readClassDesc(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at java.util.HashMap.readObject(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
        at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)

        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
bstractAudioFileBridge.java:121)
        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
FileBridge.java:60)
06/08/2010 11.27.52:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Uncaught throwable:PermGen space on thread:AudioFileLoader_Worker_6 caught by
JaikozThreadGroup
06/08/2010 11.28.22:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Terminating application because a serious error has occurred
java.lang.OutOfMemoryError: PermGen space
06/08/2010 11.28.22:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Terminating application because a serious error has occurred
java.lang.OutOfMemoryError: PermGen space
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(Unknown Source)
        at java.lang.ClassLoader.defineClass(Unknown Source)
        at java.security.SecureClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.access$000(Unknown Source)
        at java.net.URLClassLoader$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at com.mchange.v2.c3p0.impl.NewPooledConnection.handleThrowable(NewPoole
dConnection.java:432)
        at com.mchange.v2.c3p0.impl.NewProxyStatement.executeQuery(NewProxyState
ment.java:52)
        at com.jthink.jaikoz.db.Db.runQuery(Db.java:412)
        at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:69)

        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
bstractAudioFileBridge.java:121)
        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
FileBridge.java:60)
        at com.jthink.jaikoz.data.AudioFileFromDirectoryAndFilesLoader$TaskWorke
rThread.run(AudioFileFromDirectoryAndFilesLoader.java:660)
06/08/2010 11.28.23:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Terminating application because a serious error has occurred
java.lang.OutOfMemoryError: PermGen space
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.getDeclaredMethod(Unknown Source)
        at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
        at java.io.ObjectStreamClass.access$1700(Unknown Source)
        at java.io.ObjectStreamClass$2.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.io.ObjectStreamClass.<init>(Unknown Source)
        at java.io.ObjectStreamClass.lookup(Unknown Source)
        at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
        at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
        at java.io.ObjectInputStream.readClassDesc(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at java.util.HashMap.readObject(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
        at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)

        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
bstractAudioFileBridge.java:121)
        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
FileBridge.java:60)
06/08/2010 11.28.35:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Terminating application because a serious error has occurred
java.lang.OutOfMemoryError: PermGen space
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
        at java.lang.Class.getDeclaredMethod(Unknown Source)
        at java.io.ObjectStreamClass.getPrivateMethod(Unknown Source)
        at java.io.ObjectStreamClass.access$1700(Unknown Source)
        at java.io.ObjectStreamClass$2.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.io.ObjectStreamClass.<init>(Unknown Source)
        at java.io.ObjectStreamClass.lookup(Unknown Source)
        at java.io.ObjectStreamClass.initNonProxy(Unknown Source)
        at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
        at java.io.ObjectInputStream.readClassDesc(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at java.util.HashMap.readObject(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at com.jthink.jaikoz.db.Db.readBlobAsObject(Db.java:549)
        at com.jthink.jaikoz.db.SongCache.getSongFromDatabase(SongCache.java:76)

        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadSongFromDatabase(A
bstractAudioFileBridge.java:121)
        at com.jthink.jaikoz.data.AbstractAudioFileBridge.loadFile(AbstractAudio
FileBridge.java:60)
06/08/2010 11.28.35:com.jthink.jaikoz.manipulate.RunAndWaitForGUIUpdate:updateGU
I:SEVERE: RunAndWaitForGUIUpdate:invocationExceptionPermGen space
06/08/2010 11.28.50:com.jthink.jaikoz.manipulate.RunAndWaitForGUIUpdate:updateGU
I:SEVERE: RunAndWaitForGUIUpdate:invocationExceptionPermGen space
06/08/2010 11.28.45:com.jthink.jaikoz.manipulate.RunAndWaitForGUIUpdate:updateGU
I:SEVERE: RunAndWaitForGUIUpdate:invocationExceptionPermGen space
Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
erminated
Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
erminated
Java HotSpot(TM) 64-Bit Server VM warning: 06/08/2010 11.29.13:com.jthink.jaikoz
.JaikozThreadGroup:uncaughtException:SEVERE: Error in JaikozThreadGroup.uncaught
Exception(),continue termination
java.lang.OutOfMemoryError: PermGen space
06/08/2010 11.29.34:com.jthink.jaikoz.JaikozThreadGroup:uncaughtException:SEVERE
: Error in JaikozThreadGroup.uncaughtException(),continue termination
java.lang.OutOfMemoryError: PermGen space
Exception java.lang.OutOfMemoryError occurred dispatching signal UNKNOWN to hand
ler- the VM may need to be forcibly terminated
Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
erminated
Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
erminated
Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError
occurred dispatching signal UNKNOWN to handler- the VM may need to be forcibly t
erminated

MAzE5h1p69wB · 14 January 2020 13:07

Well I increased the perm store thing which java uses because it can’t relate to hardware in this case memory:

JAIKOZ.BAT

java -jar lib\\jaikoz.jar -l2 -m2 -f -Xms150m -Xmx3200m -XX:MaxPermSize=2000m

But it still won’t work even though there is a gig of standby memory and 200 mib of free memory right there waiting for it.

Anything I can do to help debug further?

As a suggestion though, I would recommend that possibly writing an application that can use memory would be a good idea.

EDIT: Oh and it barfs at around 20000 songs, and disabling show artwork does not help of course.

paultaylor · 14 January 2020 13:07

In your initial log it says

max = 259522560(253440K)

i.e 253 MB, which doesnt match up with your jaikoz.bat file, so thats why Im asking are you actually running jaikoz.bat rather than jaikoz.exe.

But aside from that if it is continually barfing at one point despite increasing memory then it is likely it is falling over on one specific file because of a problem decoding the image, not displaying the image column does not have any effect the image is still decoded.

I have put in workarounds for this, but maybe there is a another case that needs fixing. Run your load a few times and if it continually fails at one point try removing the subfolder where it is failing, then try loading just that folder and see if you can break it consistently and send me some details.

The Permgen problem is a deficiency in Java that I am going to have work around, I use String.intern() to ensure only one copy of strings found in the metadata is stored in memory thus reducing the total memory used. However unfortunately it doesnt store this in Heap but in Permanent and its difficult to assess how much Perm may be needed, I may remoce use of intern().

MAzE5h1p69wB · 14 January 2020 13:07

All of these files have previously been run through Jaikoz, and been processed with the setting delete id3v1 always write v2 in utf-8.

At this point I am quite frustrated, given the issues with disk-queue, memory, not saving tracks with correct filenames, and so on.

It would be good if you could fetch a few tens of thousands of music files from the internet, and set yourself the goal of making it possible to run Jaikoz against more than a few thousand files, no matter what tiny little errors are present in them or what happens.

paultaylor · 14 January 2020 13:07

[quote=MAzE5h1p69wB]All of these files have previously been run through Jaikoz, and been processed with the setting delete id3v1 always write v2 in utf-8.
[/quote]
Dont understand the relevance

Jaikoz is complex software, yes its not perfect but ive been working to resolve your issues and have already provided you with fixes for some of the problems you mention.

I think if you limit yourself 10,000 songs you should be okay, and that is going to save you lot of time over anhy other method, also if you could resend your support files it will give me some clues about whther you are hitting bad image problem.

Maximum Memory Size