SongKong Jaikoz

SongKong and Jaikoz Music Tagger Community Forum

Monitor Watch Folder - Linux

#22

There is nothing different in the code apart from the extra logging from previous version

image
image1648×418 45.2 KB

so I have no idea why it is now working.

Maybe interesting to see if NFS version working, because if not we may now log an exception that we would not before.

We removed rename and move files from Fix Songs because it caused too many problems and put them into Rename Songs. We then added move files back into Watch Songs because there is still need to move the files out of the watch folder as processed, but we havent included rename files, so it will only modify Base Folder not Sub Folder/Filename parts. This restriction is similar to Delete Duplicates, this cannot be performed as part of Watch Songs either, but can be run later on as required.

#23

It did not work when I pointed it back at the NFS directory, I have uploaded support files. Job #92 is the one. I did expect this to work because my understanding about notifications to filesystem changes with NFS is that as long as everything is occurring on the same box then the notifications should be made to the OS. I fully understand that a notification would not happen if a remote system made a change to a file or directory. It sounds like I might need to improve my understanding here.

Interestingly, I then reverted back to the local fs for the next job and it worked just fine again, I did not upload files for this one.

We then added move files back into Watch Songs because there is still need to move the files out of the watch folder as processed, but we havent included rename files, so it will only modify Base Folder not Sub Folder/Filename parts.

Roger that.

#24

Hi, I looked at the logs and I can see that report 92 did not start properly, because some of the logging is missing but there is no stacktrace so I cannot see why it has happened.

I noticed that after the original failed run on NFS (report 80), you didn’t restart SongKong when running tasks 81-84, you only restarted when I built the modified version of SongKong and that is when it started working. So I am thinking that maybe running against NFS breaks things, and can only be resolved by restarting SongKong. Its not clear if after the failed report 92 described above if you restarted SongKong before running successful report 93.

In Java the code can throw Errors and Exceptions, and usually code should not catch Errors and the code I modified only catches Exceptions, but I am going to build a new version that also catches Errors in case the NFS attempt is throwing an error that we are not seeing for you to retry.

1 Like
#25

Just uploaded a new version of SongKong 9.1.1 for Linux, so please try re-downloading and installing latest version against NFS, About will now show build date of 07/06/2023 rather than 06/06/2023

#26

So, I’m totally with you here about something breaking SongKong and the task is not starting correctly and is not throwing errors. In fact I noticed that when I tried to run a “Fix Songs” task last night (#94) I was experiencing the same issue with it not detecting any files in the directory, this is not something that I expected as the Fix Songs functionality has been rock solid - I had to restart SongKong to get it working and #95 ran just fine.

Now, to this mornings test… Task 97 was run against a NFS mounted directory. It completed successfully. I have just uploaded files.

I’m now wondering, what is breaking SongKong?

#27

Okay, thanks checked your logs but since the only task run since new build worked there is nothing interesting to see. If it starts not working again please send the support files then.

#28

You have the logs from yesterday, I wonder if there’s any indication what happened to cause #94 to not work?

I’m in the office today so won’t have the chance more until this evening. I’ll put it through it’s paces later and upload files if appropriate.

Thanks!

#29

No, there is nothing. That is why I did new build to try and capture more info.

1 Like
#30

Hey Paul,

So it took a few tries to recreate the symptoms but I persisted. I ended up with task #105 and #106 failing.

  • #105 watch folder failed to detect files.
  • #106 fix songs failed to detect files.
  • Restart SongKong
  • #107 fix songs succeeded.
  • #108 watch folder succeeded.

Support files have been uploaded. Please let me know if you need more tests.

#31

Okay, I noticed there was an error with #104 with trying to move files

image
image1009×716 38.6 KB

This error should never occur and I wonder if this is the cause of the problems, since there are no new errors in logs.

Anyway I need to fix this but cannot reproduce the issue so I have added more debugging and would be grateful if you try re-downloading and installing latest version against NFS, About will now show build date of 08/06/2023 rather than 07/06/2023 , then resend support files when fails (or you get these errors on an otherwise successful run like report 104).

Edit:Actually the selected folder was /mnt/nvme_data/music but error refers to /home/paul/music/, is there something weird with your mount point, is it a symbolic link ?

#32

There is nothing weird with the mount point, it is a straight NFS mount using fstab. I had noticed while testing yesterday that there was possibly some unpredictability of when the browser retained the previous path from one task to the next and was wondering whether maybe there was a temp file issue going on. I hadn’t seen it enough to notice any consistency but had made a mental note to pay more attention.

Now for testing this morning…

The issue presented itself quickly enough, I had noticed yesterday that changing target directory is more likely to bring the symptoms on. This morning task 109 was pointed at the NFS share and ran fine. I decided to then change to the local filesystem for task 110 and sure enough the symptoms presented themselves. I just performed the support file upload.

#33

I just uploaded some additional support files for 111-113 which were fix song tasks. Task 111 did not pick up any files in the directory but 112 and 113 did. Hopefully you can ascertain what I was testing for by looking at the log files. I don’t want to confuse things by typing too much :slight_smile:

#34

So I think that may be the issue, it certainly seems likley to be the issue causing the base folder error Im going to try and track that down.

But I didnt find any other errors in the logs

So if we believe the problem is now occurring when change the Watch Folder, my question would be do you continue to have problems if you dont modify the Watch Folder ?

#35

I will test for that over the weekend.

#36

Hey Paul,

I think that I have something for you…

I cleared browser cache, ensured browser is fully updated with nothing pending and restarted SongKong.

I now know that these symptoms indicate that the watch task is looking at the wrong directory. Task 122 was started and the directory selected was /home/paul/music but it didn’t pick anything up when I copied files there. I then decided to keep the task running and copy files into /mnt/nvme_data/music - it picked these files up. Obviously it shouldn’t have done this because it wasn’t configured to watch this directory. I noticed on the report for 122 that there are some errors referencing /mnt/nvme_data/music as well so hopefully your additional logging has captured something useful.

Note: This also impacted my next “Fix Songs” task (123 - not included in support files upload) in a similar way - I had to terminate and start the songkongremote script again to get this to work.

#37

Hi, just to let you know I have found the issue. Basically first time run Watch Songs it starts a WatchService to listen to that folder. The problem comes when you stop the task because the WatchService doesnt get shutdown, and because of this when you run Watch Songs next time it carries on using the original WatchService instead of creating a new one.

I am working on a fix, there are a few other things I have to consider.

1 Like
#38

Hi, just uploaded a fix so please try re-downloading and installing latest version, About will now show build date of 09/06/2023 rather than 08/06/2023

#39

Hey Paul, it appears to me that this is now fixed. I did upload the support files in case there is anything of value in there for you to validate but I was unable to get it to fail. Thank you for the excellent support here, it is very much appreciated.

Note: I did have a corrupted database and I just deleted it, I don’t consider this to be related to issue at hand.

1 Like
#40

And now released for all formats as SongKong 9.2 Brutalism released 26th of June 2023

1 Like