Monitor Watch Folder not seeing added folders?

[ramonesfan]

I am trying to get the Monitor Watch Folder feature to work, but seem to be having trouble.

I am running SongKong via Docker on my Synology NAS. The folders are on my Roon server’s internal SSD, mapped to the Synology NAS via CIFS. All manual SongKong tasks work as expected with this setup.

Here’s what I’ve been doing:

1. I set the watch folder:
   

2. I set the move folder:
   

3. I set a delay of 45 seconds to allow plenty of time for copy operations to complete:
   

   

4. I start the task (which appears to start successfully)
   

   

5. I have a folder on my local machine’s desktop (was not in the watch folder prior to task start). It is an artist folder (A Tribe Called Quest) with one album subfolder (The Anthology) in it.
   

   

6. I copy the local folder first to the remote folder (but not in the watch folder) just to make sure we don’t exceed the delay value.
   

   

7. Finally, we move the album folder into the designated watch folder. SongKong can see the newly-copied folder.
   

   

But, no matter how long I wait, the monitor watch folder task does not appear to see/process the new folder. The task just keeps running (with no notice of any files processed) and no moving of the folder I’m trying to process. After a while, I stop the task, and int always shows 0 folders/files processed.

I have created a support file for this thread. Any help appreciated!

[paultaylor]

Hi, it does sound like you are doing it correctly, and I have checked the log files and there are no errors. I will do some testing here.

[paultaylor]

Hi, okay so

• I ran Monitor Watch Folder in Windows desktop mode and it worked
• Switched to Remote mode and ran Monitor Watch Folder and it did nothing
• Tried Monitor Watch Folder again it did nothing
• Tried Monitor Watch Folder again and it worked !

Looking at the logs when it failed it didnt seem to complete initilization but no exceptions were caught. So I think there is some kind of timing bug that can prevent the initlization working, if it doesnt complete then it will not listen for any folders being sent over.

Will continue working on it.

[paultaylor]

Ok, actually i cannot get it to fail at all now, having re-read your post I think the issue maybe that you are monitoring a remotely mounted Roon folder not a folder that actually exists on the Synology NAS and it think this might mean the WatchService (provided by Java via underlying filesystem) doesnt work

FileSystems.getDefault().newWatchService()

Can you make the monitored folder a local folder on your Nas and see if that works

[ramonesfan]

Am doing two tests.

Test #1
I have pointed my Mac OS SongKong to the same mounted folder on my NAS.

The Monitor Watch Folder task seems to pick up the folder and processes the files found.

The files are then moved as expected.

So, the Mac version seems to have no problem seeing/processing the files using the same mounted folder on the NAS. I have submitted logs from the Mac version so you can take a look.

[ramonesfan]

Test #2

Using Docker version, changed file location to a local folder on the NAS.

Dropped a couple of folders in.

SongKong sees the files and starts processing!

Folders are moved as expected

I will submit support files for this as well.

I do notice one oddity in the docker version. Even though I have changed environment variables to point at a completely different folder and stopped/started the container, the docker version still seems to be retaining (in the GUI, anyway) file location information for the previous folder (see below)

I am assuming that the Mac version and the Docker version are both Java programs and both use FileSystems.getDefault().newWatchService(). If that’s the case, is there a reason that the Mac version would be able to see the remotely mounted folder via the NAS while the Docker version can’t? Could it be related to the Doicker version (apparently, at least in the GUI) retaining incorrect file location info?

[ramonesfan]

Hmmm…I see in the Java documentation for FileSystems.getDefault().newWatchService() that…

If a watched file is not located on a local storage device then it is implementation specific if changes to the file can be detected. In particular, it is not required that changes to files carried out on remote systems be detected.

I’m assuming “implementation-specific” is equivalent to “OS specific” and, if that’s the case, it could be that MacOS facilitates detection of changes, while the Synology OS does not. That would be a bummer!

[paultaylor]

Yes that is what I was getting at. But in your mac test were you monitoring a folder that was local to nas or monitoring the roon server folder you failed to get working with the nas , if the former you should try against roon for a fair test.

The issue is maybe the mechanism used to remote mount the filesystem, so it may be fixable. But since the monitored folder is a temporary dropping off point for files it seems the solution for you would be to remotely mount the nas folder onto your roon server so then you could continue to populate folder as you do already but then nas would have no issue seeing folder.

Regarding the Selected Folder anomally SongKong is simply showing the last folder selected, it doesn’t know that behind the scenes that /music has now been mapped to a different location.

[ramonesfan]

@paultaylor

But in your mac test were you monitoring a folder that was local to nas or monitoring the roon server folder you failed to get working with the nas

In my Mac test, it was the same Roon server folder that didnt work on Synology, so it was an “apples to apples” comparison…Mac OS can detect the changes on the Roon server, Synology seemingly can’t.

Just to see if a different linux-based OS running Docker might provide the Java watch code the file system support it apparently needs to monitor a remote folder, I set up docker on a Raspberry Pi and am doing some testing (although I ran into a different anomaly…please see my most recent thread, and sorry for throwing another docker anomaly at you!)

[paultaylor]

Ok, what about my idea of remote mounting a synology folder on roon instead?

[ramonesfan]

I use a NUC running Roon ROCK, and all my files are stored there. For testing purposes, I can add a Synology folder to Roon’s Library, but my ultimate goal is to keep all my music files stored on the Nucleus (internal SSD or attached USB M.2), so I’m trying to avoid storing any library files (other than periodic backups) on the NAS.

[paultaylor]

But isn’t this a watch folder for transient files not yet in your roon library rather than your roon music library?

[ramonesfan]

Yes, it’s a transient directory on the Roon NUC SSD, but that directory is not included in the folders Roon watches to create/maintain the Library. I prefer to have that “staging” folder on the local NUC SSD so that when moving processed folders to their final destination in a Roon library folder, it’s a very quick move operation. My general desired workflow is:

1. Rip CD via dbPoweramp on Mac to Roon staging folder (ripping is already slow, so we make the slower SMB transfer of the files from Mac to NUC during what is already a slower process).
2. Move folders from NUC “staging folder” to SongKong monitor folder (very fast move operation as the files are already on the desired physical drive)
3. After verifying results of SongKong match, move SongKong processed folders into permanent Roon Library folder (again, on the same physical drive so the move operation is fast).

[ramonesfan]

Quick note. I got Docker running on an intel-based Ubuntu machine, and tested the Monitor Watch Folder function (watching the same SMB share) on that platform (rather than Synology). Unfortunately, adding folders to the watch folder did not trigger anything. I had hoped Ubuntu had a more sophisticated underlying filesystem than Synology OS and could detect changes (like Mac OS can), but apparently that is not the case.

[paultaylor]

When SongKong processes the folder on the nas it has to read the existing metadata from the files, and then when it identifies the music it has to write the new metadata to the files. Depending on what music format you are using and how much space has been allocated for tagging this may involve having to rewrite the whole file. This process is much quicker if the files are local to SongKong. Although you have SongKong configured to rename/move files this happens after you have first save changes to existing files.

So the whole process is likely to be quicker if your watch folder is local to the nas.

[paultaylor]

@ramonesfan

So, the Mac version seems to have no problem seeing/processing the files using the same mounted folder on the NAS. I have submitted logs from the Mac version so you can take a look.

So I have looked at your logs, but in this setup it works fine so there is nothing to learn here.

So the conclusion is simply that using a remote watch folder is not guaranteed to work, but does work on Macs and doesnt work on Linux systems. Since this is provided by the Java api there is nothing I can do to make it work on a Linux system, (and possibly there is nothing Java can do to make it work without changes on Linux).

So your option is either to not use remote watch folder or run SongKong on MacOS instead of your Nas.