Remove folders after deleting duplicates

[paultaylor]

Have been implementing a solution, would like some initial feeback before release

Added checkbox When find duplicates delete folder if empty or only contains the following files, so if enabled when deleting (or moving) duplicates it will check the folder containing the deleted song and do the folowing.

• Check folder and subfolders for any remaining audio files, if any found dont delete folder.
• Otherwise, traverse the folder and subfolders and for each sub folder delete any files that match the entries in the list (* is a wildcard)
• If subfolder now empty delete it.
• If parent folder now empty delete it

[meaning]

This is indeed a very good start, but I would recommend to allow the end user to add his/her own files/folders to your list.

As an example, almost every album folder I do have contains a subfolder, that contains json files.

So if I was running the above, 99% of my “empty” folders would remain in place.

so, lets say a user has this kind of structure

/Album_Name/Extra_user_folder/files

then it would be great to have an “add” button that would allow the user to specify that if the album folder contains a subfolder named “Extra_user_folder”, we can totally delete it.

[paultaylor]

Hi, you can add to the list, its not actually a list but an editable text file, just with one entry per line. I was going to have as a list but think just alot easier to be able to type in directly.

So if your folder has a subfolder names Extra_user_folder, and within that there is a file called info,json, you could add info,json to the list, that file would be deleted and if Extra_user_folder was now empty that subfolder would be deleted.

What you cannot currently do is add Extra_user_folder to the list and have SongKong delete that folder regardless of its contents. It would be technically possible to do this but it sounds rather dangerous, better to specify actual files that can be removed and then remove folders that become empty as a result of this.

[meaning]

If there is a way to add files types to be monitored for deletion. Then it’s great !

One extra thing, I think it might be good to allow the user to run this task appart from the duplicates finder.

this to allow him/her to have to fix folder that were emptied of their music files in a previous run?

[paultaylor]

@meaning

One extra thing, I think it might be good to allow the user to run this task appart from the duplicates finder.

this to allow him/her to have to fix folder that were emptied of their music files in a previous run?

Okay that is a good idea, would also be useful for tidying up after Fix Songs ectera. Add a new task called Empty Folder Deleter, but adding a new top level task is quite alot of work so one for the future - https://jthink.atlassian.net/browse/SONGKONG-2363

[paultaylor]

Now released in SongKong 8.6 Post released 15th November 2022

[meaning]

I’ll run a test on the god old Arovane folder I’ve backed-up and will let you know how it ends.

Hopefully, the new “most tracks” setting will avoid the issue I was running into -> a few tracks in one folder, a few others in the other. I’ll keep you posted

thanks again Paul !

[meaning]

I am missing the “Album with most tracks” and “Erliest folder creation date” settings @paultaylor.

I believe this was not correctly added to the docker image.

[paultaylor]

Hi, sorry seems to be some issue with Linux based versions looking into it now

[paultaylor]

Working on new build now, see https://jthink.atlassian.net/browse/SONGKONG-2364 that should replace existing 8.6 within the hour, then later will release SongKong 8.6.1 for those who have already installed SongKong 8.6 so they get notification of fix version without needing to check this forum.

[paultaylor]

@meaning okay new docker image available for Intel nas, I just tested it on synology nas and it worked for me. You can check you have new version by selecting About and you’ll see Build Date will say 17/11/2022 (yes I know its one day out)

[meaning]

This is in :

This also:

Seems GOOD !

[meaning]

OK I ran a few tests.

here are my settings :

*.cue
*.log
*.txt
*.nfo
*.m3u
*.sfv
*.html
*.htm
*.rar
*.json
*.jpg
*.jpeg
*.png
*.gif
*.bmp
*.pdf
*.ini
*.db

Here is one of the folders that should have been deleted:

as you can see both *.jpg, *.nfo and *.pdf are part of my list. But the folder remained there.

I also run into another issue. As I am using rclone to copy files (fuse FS) I have a ton of .fuse_hiddenXXXXXX hidden files in my folders. I cannot get rid of these folders as the *.fuse_hidden* mask is not working. IT even avoids other masks to get taken in consideration.

I also have some “empty” files that are just named without any extension, that were previously used to announce the genre and size of a music release :

I wonder how I could get rid of these. I’ve tried *]-[*, but it did not work.

I send you the support files now.

[paultaylor]

@meaning

as you can see both *.jpg, *.nfo and *.pdf are part of my list. But the folder remained there.

I wonder did this folder actually contain any audio before you ran Delete Duplicates?

When a file is deleted we then check if we can delete its folder (and subfolders) , but if this folder never contained any music files then we wouldnt check to delete it.

[meaning]

You are right, I just checked the original folder, and it was not containing any music files.

On how to delete files names that way -> .fuse_hiddenXXXXXXXX
using *fuse_hidden* did the trick on these ones (*.fuse_hidden* didn’t)

But as soon I do add *]-[* to the list, none of the folders gets deleted.

[meaning]

More generally, I think there should be an option to allow deleting folders that do not contain music. maybe the day the feature will be available as a top feature.

In the meanwhile, can you please share the command that gets run to delete the files? Can I simply put it in a bash script to get things deleted ?

[paultaylor]

@meaning

On how to delete files names that way -> .fuse_hiddenXXXXXXXX
using fuse_hidden did the trick on these ones (.fuse_hidden didn’t)

But as soon I do add ]-[ to the list, none of the folders gets deleted.

Okay. the code actually use a Glob, a simplified version of regular expressions. We take the list and make it into a single glob

e.g

*.jpg
*.png

becomes

{*.jpg, *.png}

But both regular expressions and globs are too complex for most users to bother about, so we just mention the wildcard.

However, [ and ] and - are special characters in a glob, so you need to escape them with backslash character

e.g try this

*\]\-\[*

But its not clear to me why

*.fuse_hidden*

would not work

[paultaylor]

@meaning

More generally, I think there should be an option to allow deleting folders that do not contain music. maybe the day the feature will be available as a top feature.

Yes that is going to be new top level task Delete Folders, also useful to run after Fix Songs

@meaning

In the meanwhile, can you please share the command that gets run to delete the files? Can I simply put it in a bash script to get things deleted ?

Its Java code, Im not running a linux command

[meaning]

unfortunately, after checking all the folders that got processed for duplicates, I did figure out that the issue “some tracks here and some tracks there” is still present.

In Arovane, I have 4 instances of Radius 2

respectively :

Radius 2
Radius 2 (1)
Radius 2 (2)
Radius 2 (3)

Before the duplicates job here is each folders content :

root@Tower:/mnt/user/MURRAY/Music_processed/Arovane_Backup/Radius 2# ls -lah
total 131M
drwxrwxrwx 1 root root 4.0K Nov 10 11:49 ./
drwxrwxrwx 1 root root 4.0K Nov 10 11:50 …/
-rw-rw-rw- 1 root root 23M Nov 10 11:49 01\ -\ Derveed.flac
-rw-rw-rw- 1 root root 9.4M Nov 10 11:49 01\ -\ Derveed.mp3
-rw-rw-rw- 1 root root 15M Nov 10 11:49 02\ -\ Lenc-Crade.mp3
-rw-rw-rw- 1 root root 7.4M Nov 10 11:49 03\ -\ Dwaard.mp3
-rw-rw-rw- 1 root root 35M Nov 10 11:49 04\ -\ Inglued.flac
-rw-rw-rw- 1 root root 12M Nov 10 11:49 04\ -\ Inglued.mp3
-rw-rw-rw- 1 root root 23M Nov 10 11:49 05\ -\ Quandt.flac
-rw-rw-rw- 1 root root 7.8M Nov 10 11:49 05\ -\ Quandt.mp3
-rw-rw-rw- 1 root root 146K Nov 10 11:49 Cover.jpg
drwxrwxrwx 1 root root 23 Nov 10 11:49 Covers/
-rw-rw-rw- 1 root root 2.6K Nov 10 11:49 Folder.auCDtect.txt
-rw-rw-rw- 1 root root 939 Nov 10 11:49 album.nfo
-rw-rw-rw- 1 root root 41K Nov 10 11:49 cover.jpg
-rw-rw-rw- 1 root root 59K Nov 10 11:49 folder.jpg

Raduis 2 (1):

root@Tower:/mnt/user/MURRAY/Music_processed/Arovane_Backup/Radius 2 (1)# ls -lah
total 80M
drwxrwxrwx 1 root root 113 Nov 10 11:47 ./
drwxrwxrwx 1 root root 4.0K Nov 10 11:50 …/
-rw-rw-rw- 1 root root 38M Nov 10 11:47 02\ -\ Lenc-Crade.flac
-rw-rw-rw- 1 root root 37M Nov 10 11:47 03\ -\ Dwaard.flac
-rw-rw-rw- 1 root root 5.3M Nov 10 11:47 cover.jpg
-rw-rw-rw- 1 root root 59K Nov 10 11:47 folder.jpg

Radius 2 (2):

root@Tower:/mnt/user/MURRAY/Music_processed/Arovane_Backup/Radius 2 (2)# ls -lah
total 160M
drwxrwxrwx 1 root root 199 Nov 10 11:49 ./
drwxrwxrwx 1 root root 4.0K Nov 10 11:50 …/
-rw-rw-rw- 1 root root 23M Nov 10 11:49 01\ -\ Derveed.flac
-rw-rw-rw- 1 root root 38M Nov 10 11:49 02\ -\ Lenc-Crade.flac
-rw-rw-rw- 1 root root 37M Nov 10 11:49 03\ -\ Dwaard.flac
-rw-rw-rw- 1 root root 35M Nov 10 11:49 04\ -\ Inglued.flac
-rw-rw-rw- 1 root root 23M Nov 10 11:49 05\ -\ Quandt.flac
-rw-rw-rw- 1 root root 5.3M Nov 10 11:49 cover.jpg
-rw-rw-rw- 1 root root 59K Nov 10 11:49 folder.jpg

root@Tower:/mnt/user/MURRAY/Music_processed/Arovane_Backup/Radius 2 (3)# ls -lah
total 197M
drwxrwxrwx 1 root root 39 Nov 10 11:49 ./
drwxrwxrwx 1 root root 4.0K Nov 10 11:50 …/
-rw-rw-rw- 1 root root 38M Nov 10 11:49 .fuse_hidden0035a52a0000a790
-rw-rw-rw- 1 root root 23M Nov 10 11:49 01\ -\ Derveed.flac
-rw-rw-rw- 1 root root 38M Nov 10 11:49 02\ -\ Lenc-Crade.flac
-rw-rw-rw- 1 root root 37M Nov 10 11:49 03\ -\ Dwaard.flac
-rw-rw-rw- 1 root root 35M Nov 10 11:49 04\ -\ Inglued.flac
-rw-rw-rw- 1 root root 23M Nov 10 11:49 05\ -\ Quandt.flac
-rw-rw-rw- 1 root root 5.3M Nov 10 11:49 cover.jpg
-rw-rw-rw- 1 root root 59K Nov 10 11:49 folder.jpg

After running the job I still have 3 folders remaining
Radius 2, Radius 2 (1), Radius 2 (3)

here are their content at that time :

root@Tower:/mnt/user/MURRAY/Music_processed/Arovane/Radius 2# ls -lah
total 58M
drwxrwxrwx 1 root root 201 Nov 16 15:31 ./
drwxrwxrwx 1 root root 65 Nov 16 15:32 …/
-rw-rw-rw- 1 root root 35M Nov 16 15:29 04\ -\ Inglued.flac
-rw-rw-rw- 1 root root 23M Nov 16 15:29 05\ -\ Quandt.flac
-rw-rw-rw- 1 root root 146K Nov 16 15:29 Cover.jpg
drwxrwxrwx 1 root root 31 Nov 16 15:29 Covers/
-rw-rw-rw- 1 root root 2.6K Nov 16 15:29 Folder.auCDtect.txt
-rw-rw-rw- 1 root root 939 Nov 16 15:29 album.nfo
-rw-rw-rw- 1 root root 41K Nov 16 15:29 cover.jpg
-rw-rw-rw- 1 root root 59K Nov 16 15:29 folder.jpg

root@Tower:/mnt/user/MURRAY/Music_processed/Arovane/Radius 2 (1)# ls -lah
total 80M
drwxrwxrwx 1 root root 113 Nov 16 15:29 ./
drwxrwxrwx 1 root root 65 Nov 16 15:32 …/
-rw-rw-rw- 1 root root 38M Nov 16 15:29 02\ -\ Lenc-Crade.flac
-rw-rw-rw- 1 root root 37M Nov 16 15:29 03\ -\ Dwaard.flac
-rw-rw-rw- 1 root root 5.3M Nov 16 15:29 cover.jpg
-rw-rw-rw- 1 root root 59K Nov 16 15:29 folder.jpg

root@Tower:/mnt/user/MURRAY/Music_processed/Arovane/Radius 2 (3)# ls -lah
total 66M
drwxrwxrwx 1 root root 122 Nov 16 15:31 ./
drwxrwxrwx 1 root root 65 Nov 16 15:32 …/
-rw-rw-rw- 1 root root 38M Nov 16 15:29 .fuse_hidden0035a52a0000a790
-rw-rw-rw- 1 root root 23M Nov 16 15:29 01\ -\ Derveed.flac
-rw-rw-rw- 1 root root 5.3M Nov 16 15:29 cover.jpg
-rw-rw-rw- 1 root root 59K Nov 16 15:29 folder.jpg

As you can see, initial state, I had 3 over 4 folders that got the 5 full tracks of that EP. For some reason, I end up with 3 folders, 1 contains track 4 and 5, the next one track 2 and 3, and the last one got track 1. So the release is split in 3 folders instead of been kept in a single one.

[meaning]

@paultaylor

e.g try this

OK that did the trick thanks !

it works, then I guess we’ll only have two issues left.

1. be able to delete folders not containing any music files
2. figure out why we keep having this “one files here and there” issue