Query escaping issue?

[Nocturnal]

        at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:149)
        at com.jthink.jaikoz.indexed.DataIndexer.recNoColumnMatchesSearch(DataIndexer.java:684)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.compareValues(MusicBrainzRESTQuery.java:923)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.calculateUnnormalisedAlbum(MusicBrainzRESTQuery.java:1031)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.calculateUnnormalisedScore(MusicBrainzRESTQuery.java:1148)
        at com.jthink.jaikoz.manipulate.TrackWithUnnormalizationScore.<init>(TrackWithUnnormalizationScore.java:23)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.findMatch(MusicBrainzRESTQuery.java:380)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.updateFromOnlineDatabase(MusicBrainzRESTQuery.java:217)
        at com.jthink.jaikoz.manipulate.TagFromMusicBrainzRowAnalyser$WorkerThread.analyse(TagFromMusicBrainzRowAnalyser.java:336)
        at com.jthink.jaikoz.manipulate.TagFromMusicBrainzRowAnalyser$WorkerThread.run(TagFromMusicBrainzRowAnalyser.java:289)
30/06/2007 19.15.25:SEVERE: There was a problem submitting a query to MusicBrainz for Record Number 25,672 with filename Metallica - Stone cold crazy.mp3
java.lang.RuntimeException: Unable to do perform reno match search:\\??:Cannot parse '\\??': '*' or '?' not allowed as first character in WildcardQuery
        at com.jthink.jaikoz.indexed.DataIndexer.recNoColumnMatchesSearch(DataIndexer.java:701)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.compareValues(MusicBrainzRESTQuery.java:923)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.calculateUnnormalisedAlbum(MusicBrainzRESTQuery.java:1031)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.calculateUnnormalisedScore(MusicBrainzRESTQuery.java:1148)
        at com.jthink.jaikoz.manipulate.TrackWithUnnormalizationScore.<init>(TrackWithUnnormalizationScore.java:23)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.findMatch(MusicBrainzRESTQuery.java:380)
        at com.jthink.jaikoz.manipulate.MusicBrainzRESTQuery.updateFromOnlineDatabase(MusicBrainzRESTQuery.java:217)
        at com.jthink.jaikoz.manipulate.TagFromMusicBrainzRowAnalyser$WorkerThread.analyse(TagFromMusicBrainzRowAnalyser.java:336)
        at com.jthink.jaikoz.manipulate.TagFromMusicBrainzRowAnalyser$WorkerThread.run(TagFromMusicBrainzRowAnalyser.java:289)

Seems like I have to redo 25 000 lookups, since it crashed

[Focher]

From experience, I would not recommend trying to work with so many files at once. I actually split mine up alphabetically, did all the lookups, and am now in the process of putting the artists and albums back into order for import into iTunes. It’s more work, for sure.

[paultaylor]

Appears to be a problem processing records with ? in it, Ill look into it further.

It is unfortunate that Jaikoz crashed, Im suprised I would have thought it would just return an error. But Focher is correct asking Jaikoz to hold 25,000 records in memory is asking alot, I think when I introduce a File/open using a tree structure customers will find it easier to process subsets of data the time.

There would be some benefit to gaining from having Jaikoz Save Acoustic Ids to file as they are processed on the basis that the acoustic id is ALWAYS correct, takes longer to generate than any other task and I cant see any reason why somebody would not want to save it. However this would break Jaikozes rule of only persisting changes as and when the customer demands and could cause confusion. I could add a setting ‘Save Acoustic Id to File as Created’ which would save this attribute (but no others) automatically, but this would mean that the status flag would revert to unchanged as soon as it had been saved. What do people think ?

[Nocturnal]

Appears to be a problem processing records with ? in it, Ill look into it further.

I can provide the files if you can’t find the error on the output alone. Just ask

It is unfortunate that Jaikoz crashed, Im suprised I would have thought it would just return an error. But Focher is correct asking Jaikoz to hold 25,000 records in memory is asking alot,

The memory really isn’t an issue when looking up MB ID’s. Jaikoz uses about 400 MBytes of RAM under Windows when 28 000 files are loaded, if I remember correctly. Generating PUID’s is different, because in addition to the memory Jaikoz uses, the “mipcore” process that decodes the songs for analysis uses something at least 10 MBytes of RAM per minute of MP3. So if you have a mix of 60 minutes, it can get pretty memory-consuming

I think when I introduce a File/open using a tree structure customers will find it easier to process subsets of data the time.

I realize processing them in parts is smarter, but I was away from my computer over the weekend so I hoped it would have finished when I got back. Bad gamble

There would be some benefit to gaining from having Jaikoz Save Acoustic Ids to file as they are processed on the basis that the acoustic id is ALWAYS correct, takes longer to generate than any other task and I cant see any reason why somebody would not want to save it. However this would break Jaikozes rule of only persisting changes as and when the customer demands and could cause confusion. I could add a setting ‘Save Acoustic Id to File as Created’ which would save this attribute (but no others) automatically, but this would mean that the status flag would revert to unchanged as soon as it had been saved. What do people think ?

I would certainly use that option. Perhaps it would be easiest for the user when you don’t make a setting out of it. Instead, when you press the “Retrieve acoustic id’s” button, you could have it pop up a dialog with the radio button options of either saving the ID as soon as a file finishes (new behaviour), or just use the old behaviour.

[Focher]

I would also vote in favor of saving the acoustic id at lookup.

Perhaps an overall more flexible set of options that let’s you specify which info to save automatically?

[paultaylor]

Perhaps an overall more flexible set of options that let’s you specify which info to save automatically?

Im not sure what else I would like to see automatically saved, Acoustic ids are the only thing that Jaikoz can guarantee to be correct., what else would you like to see ?

[Focher]

I was thinking that you could just choose which attributes force a save of the update. This would be good for us who are willing to take the risk of a somewhat automated mass update and save the time of waiting for a file save / directory reorg at the end of the process.

[paultaylor]

Like ‘Save on Acoustic id Updated’, ‘Save on Album name’ updated … rather than ‘Save during Retrieve Acoustic Id’, ‘Save during tag from MusicBrainz’.

That sound more complicated to me , and if the saves are attribute level you could end up with multiple saves of the same file, unless extra precautions were taken.

With the original ‘Save during Retrieve Acoustic Id’ idea would it be expected that any changes to the file so far were saved or only the Acoustic id was saved. For example if the user had manullay edited the artist name before running ‘Retrieve Acoustic Ids’ would that be saved. My own view is it should not be, the save option should only apply to the Acoustic id itself.

[Focher]

I was thinking of a more comprehensive workflow method that should probably be combined with the defined behavior you can define for the Auto Correct function.

Currently, you can tell Jaikoz what order to perform tasks and then it performs these tasks in sequence. For example:

1. Retrieve acoustic ID
2. Match Musicbrainz tags
3. Correct Artist
4. Correct filename
   etc

I would change two things. First, I would add a “Save” action that could be added at any step of the way - you could even have it save after every task if you wanted to. Second, I would allow the user to change the behavior so that instead of performing the tasks on all files before moving to the next task, allow the user to specify that those tasks run in sequence per file.

I guess I don’t see the risk there if you handle duplicate files the same way you currently do (append a number to the name).

[paultaylor]

[quote=Focher]I was thinking of a more comprehensive workflow method that should probably be combined with the defined behavior you can define for the Auto Correct function.

Currently, you can tell Jaikoz what order to perform tasks and then it performs these tasks in sequence. For example:

1. Retrieve acoustic ID
2. Match Musicbrainz tags
3. Correct Artist
4. Correct filename
   etc

I would change two things. First, I would add a “Save” action that could be added at any step of the way - you could even have it save after every task if you wanted to.
[/quote]

Yes this idea was put forward previously at http://www.jthink.net/jaikozforum/posts/list/151.page

I will add a save task, however this will save all changes to the file and I dont expect most customers actually use the Autocorrecter so i think I will also add a save option specifically for the Create Acoustic ids solution. Performance will be better anyway if the save is incorporated into the task.

Second, I would allow the user to change the behavior so that instead of performing the tasks on all files before moving to the next task, allow the user to specify that those tasks run in sequence per file.

I cant do this because the local correct tasks such as Correct Artist work by comparing all the files and selecting the most popular option, they do not work record by record.

I guess I don’t see the risk there if you handle duplicate files the same way you currently do (append a number to the name).

I think you misunderstood me my point about multiple files was not that there was a risk of data corruption, it was that performance is going to deteriate with multiple saves of the same file because saving to disk is quite a slow task.

[paultaylor]

org.apache.lucene.queryParser.ParseException: Cannot parse ‘\??’: ‘*’ or ‘?’ not allowed as first character in WildcardQuery

Fixed in Jaikoz 1.10