SongKong Jaikoz

SongKong and Jaikoz Music Tagger Community Forum

Two issues opening files...

Hi,

When I try to open a folder I get an open dialog that doesn’t seem to understand NFS shares. The path /usr/local/export/mp3 is mounted NFS from another machine and has a bunch of stuff under it. Yet in the open dialog it won’t let me browse down below the mp3 folder.

Next, if I try to open this folder and tick the “Include Sub Folders when searching for Audio Files” it starts to work…

A progress dialog says “N Files loaded so far”. The UI behind is updated with data that looks good.

Then some sort of exception happens and there’s a dialog that says “Unexpected Problem with Jaikoz, please report problem to support@jthink.net”.

Interestingly, if you don’t click “OK” immediately, the progress dialog beneath continues to increment the count of files loaded. As soon as you click OK on the exception the process is killed.

Thx,
Scott

Is this with Jaikoz 2.3.1 or jaikoz 2.4.0 beta ?

This is v2.3.1 build 1044

Thx,
Scott

Ok, can you send me your jaikozuser0-0.log and jaikozdebug0-0.log files so I can see the exception in full.

Ok I sent you a private message with the URL to get these.

One thing I noticed while looking through the logs was that it seems to think there are a lot more MP3 files under the root dir than I do:

From the logs:

My count:

AFAIK there are no links (soft or hard) under here. Find conforms this. Even when I include directories in the count it comes out ~8600. So I have no idea where it got 21k files.

Thx,
Scott

Scott, I sent you a message because the download you made available is corrupted, could do check and retry please.

Scott, I think there maybe symblic links in the folders (rather than the mp3s) can you check for me.

I have found a bug in Jaikoz in dealing with symblic links that I have fixed which will be in the third (and hopefully final) beta in a few days.

[scott@wannabe:/usr/local/export/mp3]% find . -type l
[scott@wannabe:/usr/local/export/mp3]% 

No sym links living under the root directory added to Jaikoz, sorry. Also no hard links.

Thx,
Scott

Scott

Ok, I need you to run a test for me, please download foldertest from this page

http://www.jthink.net/jaikoz/jsp/test/start.jsp

and run it against your nfs drive

the output should tell us where things are going wrong

The output lists stuff like “Reading:/directory/name: Found 11 directly under this folder”.

Seems to make good progress until here:

eading:/usr/local/export/mp3/Beck/1994 - Mellow Gold:Found 12 directly under this folder
Reading:/usr/local/export/mp3/Beck/1996 - Odelay:Found 13 directly under this folder
Reading:/usr/local/export/mp3/Beck/1998 - Mutations:Found 11 directly under this folder
Reading:/usr/local/export/mp3/Beck/1999 - Midnight Vultires:Found 11 directly under this folder
Reading:/usr/local/export/mp3/Beck/2002 - Sea Change:Found 12 directly under this folder
Reading:/usr/local/export/mp3/Beck/2005 - Guero:Found 12 directly under this folder
Reading:/usr/local/export/mp3/Beck/2005 - Guero:Found 12 directly under this folder
Reading:/usr/local/export/mp3/Beck/2005 - Guero:Found 12 directly under this folder

It gets stuck in an infinite loop reading that dir over and over. Here’s a guess as to why:

[scott@tiger:/usr/local/export/mp3/Beck/2005 - Guero]% ll
total  70M
drwxrwxr-x     2 scott    1000         1.0k Nov 18 23:02 ./
drwxrwxr-x     9 scott    1000          512 Dec 27 20:56 ../
-rw-r--r--     1 scott    1000         4.7M Apr 13 13:19 01 - Beck - E-Pro.mp3
>>> -rw-r--r--     1 scott    1000         5.0M Apr 13 13:19 02 - Beck - Qu? Onda Guero.mp3 <<<
-rw-r--r--     1 scott    1000         4.8M Apr 13 13:19 03 - Beck - Girl.mp3
-rw-r--r--     1 scott    1000         6.8M Apr 13 13:19 04 - Beck - Missing.mp3
-rw-r--r--     1 scott    1000         3.8M Apr 13 13:19 05 - Beck - Black Tambourine.mp3
-rw-r--r--     1 scott    1000         6.1M Apr 13 13:19 06 - Beck - Earthquake Weather.mp3
-rw-r--r--     1 scott    1000         4.5M Apr 13 13:19 07 - Beck - Hell Yes.mp3
-rw-r--r--     1 scott    1000         6.2M Apr 13 13:19 08 - Beck - Broken Drum.mp3
-rw-r--r--     1 scott    1000         5.9M Apr 13 13:19 09 - Beck - Scarecrow.mp3
-rw-r--r--     1 scott    1000         5.7M Apr 13 13:19 10 - Beck - Go It Alone.mp3
-rw-r--r--     1 scott    1000         6.0M Apr 13 13:19 11 - Beck - Farewell Ride.mp3
-rw-r--r--     1 scott    1000         4.3M Apr 13 13:19 12 - Beck - Rental Car.mp3
-rw-r--r--     1 scott    1000         5.5M Apr 13 13:19 13 - Beck - Emergency Exit.mp3
-rw-r--r--     1 scott    1000         2.4k Nov 15 08:49 AlbumArtSmall.jpg
-rw-r--r--     1 scott    1000         9.2k Nov 15 08:49 AlbumArt_{9E7068E2-CB10-4B0F-8C70-0BE3BC5D457C}_Large.jpg
-rw-r--r--     1 scott    1000         2.4k Nov 15 08:49 AlbumArt_{9E7068E2-CB10-4B0F-8C70-0BE3BC5D457C}_Small.jpg
-rw-r--r--     1 scott    1000          796 Nov 15 08:49 Beck - Guero.m3u
-rw-r--r--     1 scott    1000         9.2k Nov 15 08:49 Folder.jpg
-rw-r--r--     1 scott    1000          38k Nov 18 23:02 cover.jpg

That file has some wacky unicode character in it: \351. Maybe that’s an e with an accent on it?

Thx,
Scott

Scott, thanks for the quick reply. If you copied the problem folder to your local drive does the test program/jaikoz still hang or does it work when not nfs mounted.

And does the test prog work on the nfs mount if you temporariliy remove that folder ?

Ive updated the folder test program, please try it again to see if it nows works - Im waiting for a reply before issuing the next beta so would appreciate it if you could do it as as soon as you can…

Hi,

  1. When I copy the files to the local drive (not NFS) the foldertest program works great. The local drive is an HFS partition on OSX.

  2. If I remove that folder on NFS it works until it gets to the next folder that contains wacky Unicode chars:

Reading:/usr/local/export/mp3/Bjork/1993 - Debut:Found 11 directly under this folder
...

That one is interesting in that the file with the strange char is not an MP3 – it’s an M3U file which contains char \366.

  1. The new foldertest.jar seems to behave the same way as the old, unfortunately. It gets stuck on the same Bjork directory when from NFS and it succeeds when reading local drive with strange chars.

Thx,
Scott

Thats a shame, can you do a ll of the offending folder i cant see the problem file that you mentioned. Could you also confirm that the problem file in the beck Folder was in fact the beck Q?onda file , by just removing that soliatarty fiel rather then the folder.

I have another idea but because I cant replicate your problem its just a guess, so should be another Folder test along shortly.

Ive added a new test for you to try.

[scott@wannabe:/usr/local/export/mp3/Bjork/1993 - Debut]% ll
total  67M
drwxrwxr-x     2 scott    users        1.0k Nov 18 23:02 ./
drwxrwxr-x     5 scott    users         512 Nov 19 22:28 ../
-rw-r--r--     1 scott    users        5.7M Apr 15 19:48 01 - Bjork - Human Behaviour.mp3
-rw-r--r--     1 scott    users        6.7M Apr 15 19:48 02 - Bjork - Crying.mp3
-rw-r--r--     1 scott    users        6.4M Apr 15 19:48 03 - Bjork - Venus As A Boy.mp3
-rw-r--r--     1 scott    users        4.6M Apr 15 19:48 04 - Bjork - There's More To Life Than This.mp3
-rw-r--r--     1 scott    users        6.3M Apr 15 19:48 05 - Bjork - Like Someone In Love.mp3
-rw-r--r--     1 scott    users        5.4M Apr 15 19:48 06 - Bjork - Big Time Sensuality.mp3
-rw-r--r--     1 scott    users        7.4M Apr 15 19:48 07 - Bjork - One Day.mp3
-rw-r--r--     1 scott    users        5.4M Apr 15 19:48 08 - Bjork - Aeroplane.mp3
-rw-r--r--     1 scott    users        6.8M Apr 15 19:48 09 - Bjork - Come To Me.mp3
-rw-r--r--     1 scott    users        6.8M Apr 15 19:48 10 - Bjork - Violently Happy.mp3
-rw-r--r--     1 scott    users        4.9M Apr 15 19:48 11 - Bjork - The Anchor Song.mp3
>>> -rw-r--r--     1 scott    users         780 Nov 15 08:49 Bj?rk - Debut.m3u <<<
-rw-r--r--     1 scott    users        4.7k Nov 18 23:02 cover.jpg

I ran your foldertest on the NFS server (FreeBSD, UFS partition) and it worked great. So this appears to be an issue with reading NFS directory entries with strange characters in the filenames.

I tried your test of removing the M3U file from the folder above (with the strange char). This causes the foldertest program to progress beyond that point… all the way to the next folder which is “1995 - Post”, another folder with a funny Bj?rk filename.

Thx,
Scott

Aha, I just tried your newest foldertest.jar and it “works”. I believe it’s skipping over files with strange unicode in them because the filecounts at the end differ:

Found 6717 Audio Files

Found 6747 Audio Files

But it makes progress and reads the whole tree now.

Thx,
Scott

FWIW, I dug into the foldertest code looking for the discrepancy between the NFS and local counts and the bug is weird. It seems like when files on an NFS volume have funny characters in Sun’s java.io.FileFilter code is invoked with the File argument of the parent directory instead of the file itself:

Under NFS:

Under /usr/local/export/mp3/Enya/1991 - Shepherd Moons
FILTER: "enya - shepherd moons.m3u" is  _NOT_ ok.
FILTER: "01 - enya - shepherd moons.mp3" is ok.
FILTER: "02 - enya - caribbean blue.mp3" is ok.
>>> FILTER: "1991 - shepherd moons" is  _NOT_ ok. <<<
FILTER: "03 - enya - how can i keep from singing.mp3" is ok.
FILTER: "05 - enya - angeles.mp3" is ok.
FILTER: "07 - enya - book of days.mp3" is ok.
FILTER: "08 - enya - evacuee.mp3" is ok.
>>> FILTER: "1991 - shepherd moons" is  _NOT_ ok. <<<
FILTER: "10 - enya - marble halls.mp3" is ok.
FILTER: "11 - enya - afer ventus.mp3" is ok.
FILTER: "12 - enya - smaointe....mp3" is ok.
FILTER: "cover.jpg" is  _NOT_ ok.
FILTER: "06 - enya - no holly for miss quinn.mp3" is ok.
Found 10 directly under /usr/local/export/mp3/Enya/1991 - Shepherd Moons

All the “1991 - shepherd moons” are actually MP3 files that show up fine when reading from a local mount and are parsed successfully by the filter.

It seems like the way File.listFiles works is:

863   public File[] listFiles(FilenameFilter filter)
864   {
865     String[] filelist = list(filter);
866     
867     if (filelist == null)
868       return null;
869 
870     File[] fobjlist = new File [filelist.length];
871 
872     for (int i = 0; i < filelist.length; i++)
873       >>> fobjlist [i] = new File(this, filelist [i]); <<<
874 
875     return fobjlist;
876   }

This is lexical – the new array of File objs are created from strings. Looking into the implementation of File.list(null) it seems like it uses something called a VMFile which seems to be returning null for filenames with some unicode chars in them under NFS:

  public String[] list(FilenameFilter filter)
758   {
759     checkRead();
760 
761     if (!exists() || !isDirectory())
762       return null;
763     
764     // Get the list of files
765     String files[] = VMFile.list(path);
766     
767     // Check if an error occured in listInternal().
768     if (files == null)
769       return null;
770 
771     if (filter == null)
772       return files;
773     
774     // Apply the filter
775     int count = 0;
776     for (int i = 0; i < files.length; i++)
777       {
778         >>> if (filter.accept(this, files[i])) <<< // files[i] == null here
779       ++count;
780         else
781       files[i] = null;
782       }
783 
784     String[] retfiles = new String[count];
785     count = 0;
786     for (int i = 0; i < files.length; i++)
787       if (files[i] != null)
788         retfiles[count++] = files[i];
789 
790     return retfiles;
791   }

I give up, this is job for someone who knows a lot more about java than I. :slight_smile:

Anyway, mostly fixed with your latest. Thanks.

Scott

Thanks Scott, yes the problem files are ignored because of this code.

//recursive nfs test
if(file.equals(parent))
{
      return false;
}

which prevents the infinite recursion. The fix for this release will just have to ignore the problem files because seems like a bug in java, might take a while to get a workaround.