Author Topic: XMPlay and the UnExoticA (Amiga) music collection - possible STIL support  (Read 1419 times)

XtC

  • Posts: 6
(I registered this account back in 2009, but this seems to be my first post, so...)

Ey up!

I'm XtC, one of the main Amiga music rippers from the ExoticA website.

I contacted Ian Luck back in 2009 about possible support within XMPlay for an Amiga version of the C64 HVSC STIL.  Back then at ExoticA, we were possibly going to make it web based and indexed on the MD5 of a given music module file.  Ian liked the idea, but sadly the interest from the ExoticA team fizzled out and I'm not a Perl coder.  So the project stopped.

So I decided to do it all myself instead, but differently!

In it's place, I've created a flat database structured almost exactly like the C64 HVSC STIL.  So a typical entry would look like this: -

Code: [Select]
### Game/Brooke_Jason/Out_Run/jcb.out_run ##########################################################
    MD5: a4d1300ae531cfd0fff3c34789e6f88d
  XPATH: Game/Brooke_Jason/Out_Run/jcb.out_run
 XTITLE: Out Run
  XCOMP: Jason Brooke
  XYEAR: 1988 Probe / U.S. Gold
XFORMAT: Jason Brooke
   XRIP: This soundtrack was disk ripped by XtC.
(#1)
  TITLE: Magical Sound Shower [from arcade version]
 ARTIST: Hiroshi Miyauchi
(#2)
  TITLE: Splash Wave [from arcade version]
 ARTIST: Hiroshi Miyauchi
(#3)
  TITLE: Passing Breeze [from arcade version]
 ARTIST: Hiroshi Miyauchi

There are additional fields to compensate for the lack of file headers in most Amiga music modules.

The database currently only references music files available from the UnExoticA collection, but there are nearly 11,000 entries in it!

Since no player currently exists to use this new STIL, I guess it's technically in beta.  Would any of the XMPlay/Delix developers be interested in supporting this please?

Cheers,

XtC

Fraggie

  • Posts: 710
Hi XtC, that's a great idea (back in the day I was thinking about retrieving some of this information from the LhA comments, but never got around to it...).

I'm willing put this into delix, but since most of exotica is Protracker/Soundtracker I guess it would require a more comprehensive solution on XMPlay level maybe? (I find XMPlay's MOD reproduction better than delix's.)

Anyway, do you have a sample of your "STIL" file?

saga

  • Posts: 2179

XtC

  • Posts: 6
Hi XtC, that's a great idea (back in the day I was thinking about retrieving some of this information from the LhA comments, but never got around to it...).

I'm glad you like the concept.  I started this back in 2009, but held off waiting for the rest of the ExoticA team to join in.  But as mentioned, momentum fizzled out!  :-\


I'm willing put this into delix, but since most of exotica is Protracker/Soundtracker I guess it would require a more comprehensive solution on XMPlay level maybe? (I find XMPlay's MOD reproduction better than delix's.)

Thank you.

Yes, the Protracker/Soundtracker modules certainly do make up most of the collection - but there's also a massive amount of non-tracker music in the collection too.

The database contains both the MD5 and the UnExoticA file path.  The idea being that the end user can have the individual music files stored anywhere they like and aren't forced to stick to the UnExoticA folder structure - if that makes sense?


Anyway, do you have a sample of your "STIL" file?

I've just put together this crude wiki page on ExoticA...

http://www.exotica.org.uk/wiki/UnExoticA/UTIL

The gist of this new UTIL (aka STIL) is that it follows the same rules as the STIL, but has extra headers.

Please let me know if you think there may be ways I could refine the structure, since this is really the first attempt at this.  :)

saga

  • Posts: 2179
While MD5 is still widespread, maybe it would make sense to offer a more... modern alternative as well.
For inspiration: https://softwareengineering.stackexchange.com/questions/49550/which-hashing-algorithm-is-best-for-uniqueness-and-speed (I personally use FNV-1a for pattern data hashing at ModArchive.)

XtC

  • Posts: 6
While MD5 is still widespread, maybe it would make sense to offer a more... modern alternative as well.

Fair point.  Currently there haven't been any collisions with MD5 that weren't genuine duplicate files in this collection.

We could always provide more than one version of this UTIL.txt file, that was either indexed by MD5 or by another hash algorithm, if that is what the player author prefers?

I stuck with MD5 as I noticed XMPlay already utilised this for the HVSC collection.

saga

  • Posts: 2179
Providing two variants would work; or just putting both hashes in the database would also work but maybe bloat the size a bit.

Fraggie

  • Posts: 710
Please let me know if you think there may be ways I could refine the structure, since this is really the first attempt at this.  :)
The structure is fine (though I have never been a fan of the STIL layout, it's a pain to parse from a LL language :) ).

But I have some comments on the contents:
1) Information on instruments/sample files is redundant, I think it brings no new information, just makes the file bigger.
2) XTITLE gives game title, but nothing tells the particular song title, e.g. differentiate between "rjp.title" and"rjp.ingame". I guess that's what NAME tag would be for?

E.g. for "/Game/Olsen_Jesper/Lollypop/jo.hiscore" I would like XMPlay to display "Jesper Olsen - Lollypop - Hiscore" in the playlist hiding the filename. Does that make sense?

XtC

  • Posts: 6
The structure is fine (though I have never been a fan of the STIL layout, it's a pain to parse from a LL language :) ).

If our Perl wizard was able to commit the time, this probably would have been in XML format if used as a standalone file. Otherwise it would have been on our wiki pages as "microformat for semantic annotation".  Sigh.  :-\


But I have some comments on the contents:
1) Information on instruments/sample files is redundant, I think it brings no new information, just makes the file bigger.

No worries about the individual instruments, they're more for ExoticA's benefit. At some point BuZz may integrate this new UTIL into Modland like he did with HVSC, then they would make a bit more sense.

I'll remove them from the UTIL for now.  I'll create another exhaustive one for fanatics!  :)


2) XTITLE gives game title, but nothing tells the particular song title, e.g. differentiate between "rjp.title" and"rjp.ingame". I guess that's what NAME tag would be for?

E.g. for "/Game/Olsen_Jesper/Lollypop/jo.hiscore" I would like XMPlay to display "Jesper Olsen - Lollypop - Hiscore" in the playlist hiding the filename. Does that make sense?

That's right, the NAME tag definitely needs further populating and I've just realised I also didn't include a field for the actual file name on UnExoticA! It's in the XPATH field, but I should probably make it one less thing to have to parse for.  The point of the file name is to help someone identify a file if it happens to have been renamed by another website.

With regards to MD5,etc., how are you actually parsing this text file?  Is it better to be in MD5 sort order or would another method be better?


Fraggie

  • Posts: 710
That's right, the NAME tag definitely needs further populating and I've just realised I also didn't include a field for the actual file name on UnExoticA! It's in the XPATH field, but I should probably make it one less thing to have to parse for.  The point of the file name is to help someone identify a file if it happens to have been renamed by another website.
Sounds redunant to me.

With regards to MD5,etc., how are you actually parsing this text file?  Is it better to be in MD5 sort order or would another method be better?
Using a dictionary/map. Since it's sorted by hash on input, order in the file doesn't really matter. However I suppose it's easier to read for a human if the entries are sorted by path.

XtC

  • Posts: 6
As suggested, I've uploaded a revised version of the UTIL that doesn't contain instrument, sample or replay data. That removes nearly 1,900 entries.   :)

It's now sorted by path rather than by MD5, to make it easier for humans to read.

Any missing NAME field data will be slowly populated over time, as this database evolves.  ::)

Thanks for the suggestions and feedback so far.

XtC

  • Posts: 6
Ey up!

I'm still working on populating the NAME field and have done 2,434 so far.  Turrican 2 was probably the most detailed version of this.

We've been given approval from the HVSC UTIL team to share some of their comments where relevant, as some Amiga music from composers such as Rob Hubbard, Ben Daglish, David Whittaker, etc. was also used in Amiga versions of the same game.

For the time being, the UTIL.zip file will contain two versions, with and without the sample/instrument data.

https://www.exotica.org.uk/wiki/UnExoticA/UTIL

Please let me know if the structure needs any changes, and thanks again for your time.

Cheers,

XtC