Naim WAV Genre Import

[SC]

Hi Paul –

Just been tinkering with Songkong (Melco license, on Mac) to test out the Naim import option as I’d like to tag copies of all of my Core’s WAV rips and then add additional metadata from there etc and get everything as I would like…

Just did a couple of tests and don’t know if I’m hitting an issue or if it’s user error – but should the Naim Import function import Genres that have been applied by the Core (whether they be automatic or user edits)…? I don’t seem to be getting them coming through to the saved files.

A quick example here, showing the text with the original meta.naim file –
You can see three entries for Genre (note one of these is a ‘sub genre’ which the Naim App lists a few of and allows to be added – these then present quite nicely as sub sections within the higher level genre)…However, none of these Genres are being tagged once processed by Songkok (have checked with Songkong and Metadatics)…

At the Start of the Naim Import action there is a checkbox for Add/replace Genres, I have had this checked and unchecked, no difference…I also note it says ‘See genre tab for more options’ but there doesn’t appear to be one…?

Finally, whilst looking around the text of the meta.naim file (not that I can understand much!) I noticed there’s actually quite a lot of metadata in there (certainly far more than the Naim Core and App present) but that again isn’t being processed and added to the files by the Songkong action i.e all these ‘performer’ credits…None of these Musician Credits were embedded in the saved files –

Appreciate any thoughts and feedback as to what should/can be done by the Naim Import action…My understanding was that the action would import all the metadata contained with the separate external files used by Naim i.e not just the basic tags, but user edits and any other data that was contained…?

Before I go ahead and run this over all my library I would just like to be confident that everything that is currently there will be transposed to the saved files, instead of having to check each and every album manually and tweak etc…

Thanks in advance,

[SC]

Actually, regardless of the Naim Import action results, the more I look into Genres the more confused I get…! I’ve read some of the Songkong instructions and help sections etc, and reread, and still finding it confusing…!

How ‘should’ sub-genres be handled…? Or is that a function of Grouping…?
eg If I want ‘Pop/Rock > Alternative/Indie Rock’ (as can be applied on the Naim App) how is this best done…?
I know I can add multiple Genres (in Songkong using ‘;’ ?) but this isn’t the same as a ‘sub genre’ I believe, as per the example above…?

Frustratingly confused…!

Thanks, SC

[paultaylor]

@SC

A quick example here, showing the text with the original meta.naim file –
You can see three entries for Genre (note one of these is a ‘sub genre’ which the Naim App lists a few of and allows to be added – these then present quite nicely as sub sections within the higher level genre)…However, none of these Genres are being tagged once processed by Songkok (have checked with Songkong and Metadatics)…

It looks for the Genre field, but not within user section. Within the meta section then can be competing multiple sources, we use in order Rovi/MusicBrainz/Freedb/Defaults.

If you could email support@jthink.net your full meta.naim I can give you a definitive answer.

At the Start of the Naim Import action there is a checkbox for Add/replace Genres, I have had this checked and unchecked, no difference…I also note it says ‘See genre tab for more options’ but there doesn’t appear to be one…?

It will only add Genres if you have Update Genres enabled, and the Genre is in the Genre list. There is one copy of the genre list that is shared by all profiles and tasks so we moved it to Preferences:Genres

@SC

Finally, whilst looking around the text of the meta.naim file (not that I can understand much!) I noticed there’s actually quite a lot of metadata in there (certainly far more than the Naim Core and App present) but that again isn’t being processed and added to the files by the Songkong action i.e all these ‘performer’ credits…None of these Musician Credits were embedded in the saved files –

Currently we add performers that are defined at release level but not at track level. Ive riased issue to consider this further https://jthink.atlassian.net/browse/SONGKONG-2664

Please note, there is no specification for the meta.naim file so we have just had to analyse example files and work out the best way to extract as much information as possible. For items such as performers a subsequent run with Fix Songs will be able to match most songs to MusicBrainz and extract addtional performer information from there.

[SC]

Thanks Paul, always appreciate your responses and assistance.

@paultaylor

It looks for the Genre field, but not within user section. Within the meta section then can be competing multiple sources, we use in order Rovi/MusicBrainz/Freedb/Defaults.

If you could email support@jthink.net your full meta.naim I can give you a definitive answer.

Will email you the same example meta.naim file in a short while for you to have a look. It just happened to be one of the first ones I experimented with and then hit the issue early on…

Re Genres and ‘user’ section, could this function be added possibly…? I didn’t realise they would be listed separately as such within the meta.naim file, just presumed they’d be appended…
It’s rare that I will just leave the Genre as added by the Core as they tend to be rather basic or inaccurate, so will often add additional genres or delete/tweak etc (as best possible via the Naim App metadata editing on a rip)…Before I purchased Songkong I understood (perhaps misunderstood) and thought I’d read that the Naim Import action included any User Applied Edits, so I presumed this would naturally include Genres…?
My own personal preference would be that any user edits take priority over the other sources listed in the file (i.e Rovi etc) as obviously this indicates that I have made a deliberate and desired edit…If I subsequently then want to run a 2nd pass of Fix songs/Match Album and apply any more/different Genres then obviously that is still an option…
To be honest, it would be a really useful, arguably important, tweak of the Naim Import action…

@paultaylor

It will only add Genres if you have Update Genres enabled, and the Genre is in the Genre list. There is one copy of the genre list that is shared by all profiles and tasks so we moved it to Preferences:Genres

OK, understood. So just to be clear, this one Genre list is used by each and every action incl Naim Import, there’s no separate lists for each action…?
So I presume I have to add to the list and include and Genre wording that the Naim Core/App use exactly as they have it…?

@paultaylor

Currently we add performers that are defined at release level but not at track level. Ive riased issue to consider this further https://jthink.atlassian.net/browse/SONGKONG-2664

OK, sounds good. TBH, that one wasn’t such a huge deal (though would be good if it did get added etc), it was just that it was another example of data within the file that I noted was being carried over, hence why it stopped me in my tracks and started wondering just how much of the original+user edit Naim data was indeed being extracted…It would be ideal that everything that is there is extracted in some form…

@paultaylor

Please note, there is no specification for the meta.naim file so we have just had to analyse example files and work out the best way to extract as much information as possible. For items such as performers a subsequent run with Fix Songs will be able to match most songs to MusicBrainz and extract addtional performer information from there.

Yeah, I appreciate that – good old Naim! As I said above, I appreciate I can always run a second pass etc which I’d potentially be happy to do in particular scenarios…In my head I see it as a 2 step process – extract all that the meta.naim file has within it, importantly including any/all user edits, as an initial stage…and then potentially running a further lookup if desired…Personally, what I don’t want to do is run a Fix Songs on all the Naim library from the very start and potentially change any metadata as I currently have it based upon the matches that Musicbrainz matches to etc, I sort of want to have a little more careful control – this was one of the reasons why I was happy the single Album Match action was added…Makes it all a lot slower on a large library obviously, but does allow the finite control/match Vs the relatively automated Fix Songs…

OK, thanks again…Will email that file soon
SC

[SC]

Re the subject of Sub Genres as I mentioned earlier above, one thing I have discovered is that the Naim App does NOT seem to appear to read the Grouping Tag…So this doesn’t seem to be a solution for adding sub genres, for me anyway (basically, I’m wanting to tag all my original Naim ripped WAV files as ID3, adding extra metadata, tweaking etc…then re-add to the Core (Downloads folder along with my HD files there) and, for now, continue using the Naim App…)

I’ve discovered, I THINK, that if multiple Genres are within the Genre Tag field (I did this via Metadatics) AND the wording of the Genres matches what the Naim App uses, then it will see and display those genres as sub genres…Hope I’m making sense here.

Eg:

I have this James download tagged with THREE separate Genres: Alternative & Indie (which is a main level genre on the Naim App), Pop/Rock also a main level and then Alternative/Indie Rock which appears as a ‘sub genre’ option when scrolling through Pop/Rock genre in the Naim App when editing metadata…So with these three genres in the metadata (applied externally via Metadatics) the album DOES list in the Naim app when searching within the Genre section just as it would if I had applied the Sub Genre to a rip on the Core via the App…i.e Appears under Alternative & Indie genre and also under the general Pop/Rock and then sub filtered to Alternative/Indie Rock…Screen grabs as per below.

This method only, so far, seems to work if the genre wording is exactly the same as the default genre options within the Naim App, in other words I don’t seem to be able to add a non default genre as per the Naim App and then it appears as a sub genre under Pop/Rock…

Example of Naim applied tagging via Naim App showing sub genre of Pop/Rock (on a different album)…

A different download file, the James album, tagged via Metadatics, with Three separate Genres tags (ensuring same wording as per Naim tags) –

The main ‘high level’ Genres as appear in the Naim App. The James album does appear within the Alternative & Indie list due to the ‘Alternative & Indie’ genre tag…

And it ALSO appears within the Pop/Rock Genre, which when filtered via ‘Alternative/Indie Rock’ then shows the album…

As I said, the wording of the Genre has to be exactly the same as the Naim defaults for it to be listed as a sub genre of the high level genre…

Incidentally, that same James album, when viewed within Songkong, shows the 3 Genre tags separated by x3 ;;; – can you explain how this works and why it is x3, I thought just a single ‘;’ was used as a separator…?

Thanks again, will leave you to digest…!
SC

[paultaylor]

@SC

Will email you the same example meta.naim file in a short while for you to have a look. It just happened to be one of the first ones I experimented with and then hit the issue early on…

Re Genres and ‘user’ section, could this function be added possibly…? I didn’t realise they would be listed separately as such within the meta.naim file, just presumed they’d be appended…
It’s rare that I will just leave the Genre as added by the Core as they tend to be rather basic or inaccurate, so will often add additional genres or delete/tweak etc (as best possible via the Naim App metadata editing on a rip)…Before I purchased Songkong I understood (perhaps misunderstood) and thought I’d read that the Naim Import action included any User Applied Edits , so I presumed this would naturally include Genres…?
My own personal preference would be that any user edits take priority over the other sources listed in the file (i.e Rovi etc) as obviously this indicates that I have made a deliberate and desired edit…If I subsequently then want to run a 2nd pass of Fix songs/Match Album and apply any more/different Genres then obviously that is still an option…
To be honest, it would be a really useful, arguably important, tweak of the Naim Import action…

Thanks for the meta.naim file, for this file it would read genre from the rovi section

"rovi": {
    "version": "2",
    "authority": "rovi",
    "authorityVersion": "2",
    ........
    ........
    "descriptors": [
        {
            "type": "genre",
            "id": "MA0000002613",
            "name": "Pop\/Rock",
            "weight": "10"
        }
    ],

However, notice the escaped / between Pop and Rock, we dont do any special processing that may also prevent it being read in.

So I checked the code and originally Naim Import was for the older Naim UniitServe, and this uses amginfo.xml, cddbinfo.txt and UserEdits.xml and we do process the UserEdits.xml file which contains any user edits.

For the newer UnitiCore they only use a meta.naim file and I don’t think we had any examples of user edits and have not coded for user edits, so if you could possibly find some files (or make some edits) with user changes to album titles and song titles I can then add processing of user edits as part of https://jthink.atlassian.net/browse/SONGKONG-2665 becuse I don’t have a UnitiCore myself.

@SC

OK, understood. So just to be clear, this one Genre list is used by each and every action incl Naim Import, there’s no separate lists for each action…?
So I presume I have to add to the list and include and Genre wording that the Naim Core/App use exactly as they have it…?

Yes, because I cannot see a usecase for when you would want to allow a Genre that was provided by MusicBrainz but if the same Genre was in Naim you would not want to allow it.

@SC

Personally, what I don’t want to do is run a Fix Songs on all the Naim library from the very start and potentially change any metadata as I currently have it based upon the matches that Musicbrainz matches to etc, I sort of want to have a little more careful control – this was one of the reasons why I was happy the single Album Match action was added…Makes it all a lot slower on a large library obviously, but does allow the finite control/match Vs the relatively automated Fix Songs…

I certainly think you should run Naim Import first, but what you can do with Fix Songs is add all fields to Only Modify the fields If Empty so that Fix Songs only adds metadata rather than changing metadata, could also run Preview Only.

[paultaylor]

@SC

Re the subject of Sub Genres as I mentioned earlier above, one thing I have discovered is that the Naim App does NOT seem to appear to read the Grouping Tag…So this doesn’t seem to be a solution for adding sub genres, for me anyway (basically, I’m wanting to tag all my original Naim ripped WAV files as ID3, adding extra metadata, tweaking etc…then re-add to the Core (Downloads folder along with my HD files there) and, for now, continue using the Naim App…)

So Im not that familiar with Naim App but I think its a UPnP app and therefore can only show the metadata provided to it by the uPnP server. So to clarify are you using it with a Naim UnitiCore, Melco Server or something else becuase if using Naim UnitiCore I think it justs ignore ID3 tags and reads from the meta.naim file

@SC

Incidentally, that same James album, when viewed within Songkong, shows the 3 Genre tags separated by x3 ;;; – can you explain how this works and why it is x3, I thought just a single ‘;’ was used as a separator…?

How multiple fields are stored in the file depends on the underlying format. So if using ID3 (for mp3, wav,aiff) this is completely different to if using Vorbis Comment (for ogg/flac), your tagger usually hides this detail away from you so you can use the same method whatever the underlying format.

For ID3 multiple values are seperated by the null character (unicode 0000) but this does not relate to a keyboard character so it is not easy to enter. In SongKong we allow manual editing of multiple values using three semicolons (;;;) and displays them using (;;;), I’m guessing metadatics uses a single semi-colon (;) but in both cases when the changes are saved to file a null character is actually used. We use ;;; instead of ; in SongKong to allow for cases where you want to actually use a ; as part of the text, remember this multi value support applies to all fields not just genres.

[SC]

Hi Paul!

Many thanks. OK, will digest what you’ve written and likely reply with more tomorrow…Can also send you more files etc…
I’m +8hrs GMT so going to turn brain off for a bit…!
More soon,
SC

[SC]

@paultaylor

For the newer UnitiCore they only use a meta.naim file and I don’t think we had any examples of user edits and have not coded for user edits, so if you could possibly find some files (or make some edits) with user changes to album titles and song titles I can then add processing of user edits as part of https://jthink.atlassian.net/browse/SONGKONG-2665 becuse I don’t have a UnitiCore myself.

Yes, correct. I have a Core (and a US as well actually) – I can send you as many examples of the generated meta.naim files as you would like, with and without user edits…How many would give you a good sample…?

SC

[paultaylor]

Ten with some user edits would be great

[SC]

Will send through tomorrow, be there first thing for you….

[SC]

Sent through earlier today Paul, was a 16mb .zip attachment, so hopefully came through ok…?

[SC]

@paultaylor

However, notice the escaped / between Pop and Rock, we dont do any special processing that may also prevent it being read in.

Yes, I did see that and noted it being unusual…But I wasn’t really sure what it meant code wise and the purpose…? Is this something that can be accommodated with the Naim Import action…?

@paultaylor

So I checked the code and originally Naim Import was for the older Naim UniitServe, and this uses amginfo.xml, cddbinfo.txt and UserEdits.xml and we do process the UserEdits.xml file which contains any user edits.

For the newer UnitiCore they only use a meta.naim file and I don’t think we had any examples of user edits and have not coded for user edits, so if you could possibly find some files (or make some edits) with user changes to album titles and song titles I can then add processing of user edits as part of https://jthink.atlassian.net/browse/SONGKONG-2665 becuse I don’t have a UnitiCore myself.

Yes, you are correct. I have both a UnitiServe (currently unused) and a Core and yes there was a change between the two in the external files that Naim use to store/edit metadata and rip info etc…With the Core there are now just two files generated, a meta.naim file and a rip.naim file (the latter I think is just a record of the actual CD rip stats)…So all metadata, whether it be the auto lookup from the Core and any subsequent ‘user’ edits are now stored within the meta.naim file…

I just presumed the file method re the Core was fully covered by the Naim Import action as it’s been out some time now (2019 I believe) and also presumed other users would have come across any issue with user edits in this time…I guess there’s not many of us…!

As requested, I’ve sent you through a good selection of sample files, some with minor user edits and some with extensive due to the Core not returning any lookup results etc…

Is it going to be possible to have the Naim Import action process both types of proprietary metadata files ie legacy (UnitiServe, HDX etc) and the current Core…? As I say I have both and still do have some older US rips to process at some point but I also wouldn’t want any potential users to have to choose between one option or the other…?

@paultaylor

I certainly think you should run Naim Import first, but what you can do with Fix Songs is add all fields to Only Modify the fields If Empty so that Fix Songs only adds metadata rather than changing metadata, could also run Preview Only.

Sounds good…

SC

[SC]

@paultaylor

So Im not that familiar with Naim App but I think its a UPnP app and therefore can only show the metadata provided to it by the uPnP server. So to clarify are you using it with a Naim UnitiCore, Melco Server or something else becuase if using Naim UnitiCore I think it justs ignore ID3 tags and reads from the meta.naim file

Yes, UPnP and I agree that it’s dependent upon the server, however experience shows me that the Naim App is also somewhat picky in what it exactly ‘shows’ – I’ve been through all this years ago with MinimServer and the Unitiserve…

To clarify for current usage, I’m using the Naim Core and using the Naim iOS App.

Re ID3 tags: No, the Core definitely DOES read them, but only in a certain scenario. Just to explain, in case you are not familiar – if the Core rips a CD and stores the files (folder=Music) within it’s internal ‘store’ (which can also be external NAS etc) then it generates it’s own propriety metadata file (meta.naim) and this file and the actual audio files are Read Only access via the Core direct (altering them externally can mess up the database etc)…the only supported way to alter the applied metadata and make any user edits is via the Naim App, which obviously then makes changes within the meta.naim file…

However, there is also a ‘Downloads’ folder within the Core’s store – this is intended for users to add their own rips, HD downloads etc etc and has full read/write access…This is actually the only way to import HD downloads to your Core. And obviously, by nature, this has to accommodate a range of potential file types and metadata tagging solutions – otherwise it just wouldn’t work!..

This is all much the same between the current Units Core and the older US/HDX etc, just that the external metadata files used by each have changed slightly…

So, I can confirm that as the ‘Downloads’ folder is where I put my HD WAV downloads, that the Core does read ID3 tagging, it’s just that it doesn’t use this method for it’s own generated files via ripped CDs – it will default to the meta.naim file if present (I was advised by Naim support to ensure no such meta.naim file exists in any folder if I want the Core to read other embedded tags)…The James Yummy example I showed at the start of this thread was a WAV download which I tweaked the ID3 tags via Metadatics and then imported into the Core’s Download folder…So it’s definitely reading ID3 OK…But as I say, I have a feeling the Naim App is being picky in just what/which of those tags it’s using and displaying…I know from testing yesterday, it doesn’t seem to do anything with the Grouping tag.

My aim is to just populate the audio files (both Core generate rips and my own downloaded HD files) with all the metadata I can/possible…what the Naim App then picks up and use is another story and one which I’ll perhaps address at a future date…But as long as my archive of files is in a state that I’m content with, then that’s my ideal…

@paultaylor

How multiple fields are stored in the file depends on the underlying format. So if using ID3 (for mp3, wav,aiff) this is completely different to if using Vorbis Comment (for ogg/flac), your tagger usually hides this detail away from you so you can use the same method whatever the underlying format.

For ID3 multiple values are seperated by the null character (unicode 0000) but this does not relate to a keyboard character so it is not easy to enter. In SongKong we allow manual editing of multiple values using three semicolons (;;;) and displays them using (;;;), I’m guessing metadatics uses a single semi-colon (;) but in both cases when the changes are saved to file a null character is actually used. We use ;;; instead of ; in SongKong to allow for cases where you want to actually use a ; as part of the text, remember this multi value support applies to all fields not just genres.

Ah ok, understood now. Good to know. I’m not sure this is documented anywhere…? I was aware that a single ‘;’ was often used as a separator as well as sometimes ‘//’ …but I see the logic of what has been implemented in Songkong now that you have explained it and as you say, allows the use of those characters within text if desired…

If there’s anything further you would like to know re the Core’s operation to help out on all this just give me a shout and I’ll try to assist…!

Many thanks again,
SC

[paultaylor]

@SC

Sent through earlier today Paul, was a 16mb .zip attachment, so hopefully came through ok…?

Thankyou , all received.

@SC

Yes, I did see that and noted it being unusual…But I wasn’t really sure what it meant code wise and the purpose…? Is this something that can be accommodated with the Naim Import action…?

Yes, just something I hadnt considered so need to check it.

@SC

I just presumed the file method re the Core was fully covered by the Naim Import action as it’s been out some time now (2019 I believe) and also presumed other users would have come across any issue with user edits in this time…I guess there’s not many of us…!

Yes, and many customers don’t actually contact us when they have an issue.

@SC

Is it going to be possible to have the Naim Import action process both types of proprietary metadata files ie legacy (UnitiServe, HDX etc) and the current Core…? As I say I have both and still do have some older US rips to process at some point but I also wouldn’t want any potential users to have to choose between one option or the other…?

There is a hierachy, if its finds meta.naim it will use that, if not and it finds amginfo.xml it will use that and useredits.xml, if it doesnt find that but find cddbinfo.txt it will use that and useredits.xml. So basically it uses the best one.

@SC

My aim is to just populate the audio files (both Core generate rips and my own downloaded HD files) with all the metadata I can/possible…what the Naim App then picks up and use is another story and one which I’ll perhaps address at a future date…But as long as my archive of files is in a state that I’m content with, then that’s my ideal…

Okay, thanks for the clarification

@SC

Ah ok, understood now. Good to know. I’m not sure this is documented anywhere…? I was aware that a single ‘;’ was often used as a separator as well as sometimes ‘//’ …but I see the logic of what has been implemented in Songkong now that you have explained it and as you say, allows the use of those characters within text if desired…

In ID3v24 null terminated strings are allowed for all text frames, from https://id3.org/id3v2.4.0-frames

4.2. Text information frames

The text information frames are often the most important frames,
containing information like artist, album and more. There may only be
one text information frame of its kind in an tag. All text
information frames supports multiple strings, stored as a null
separated list, where null is reperesented by the termination code
for the charater encoding. All text frame identifiers begin with “T”.
Only text frame identifiers begin with “T”, with the exception of the
“TXXX” frame. All the text information frames have the following
format:

 <Header for 'Text information frame', ID: "T000" - "TZZZ",
 excluding "TXXX" described in 4.2.6.>
 Text encoding                $xx
 Information                  <text string(s) according to encoding>

In the older IDv2.3 specification multiple null terminated fields were not supported but these days it has become common for ID3v23 as well. Officially only some fields support multiple values by using ‘/’ but this is not wel supported and is not treally multiple fields.
e.g>

TPE1
The ‘Lead artist(s)/Lead performer(s)/Soloist(s)/Performing group’ is
used for the main artist(s). They are seperated with the “/”
character.

Remember in SongKong the ‘;;;’ just denotes multiple values ‘;;;’ isnt actually stored in the files, that is format dependent and for ID3 we use null character.