[SM] SPARTA: Samus Palette, Art & Repoint Tool App (formerly Super Munchtroid)

[Munchy]

That is highly disappointing.   :stern:

I dunno man :neutral: thanks for your PM. It's pretty intimidating code; not beyond my abilities, I just - whoof - it's a lot of code; very very fiddly code haha!  Let me mull it over later; knowing me, if I have the "eureka" tonight, I'll probably wanna write it! I'm a little confused as to why JSM didn't add a visual editor and pointer stuff.  They've written soooo much of what you'd need to do so; heck, I'm almost tempted to fork their project and write the rest in Java (ugh, I loath Java).  Paint tools are kind of GUI 101 by contrast to all the pointer math going on in a ROM cartridge.  My suspicion is that the repointing stuff wouldn't be that hard once you've actually got all the samus sprites figured out.

[Munchy]

27/03 Munchtroid v2

I've been working out the basis of how I'm going to read stuff efficiently. Munchtroid v1 was a blunt object because I just churned it out. V2 is gonna take stream manipulation more seriously (ie. actually reading byte buffers selectively and only converting to hex outputs if/when absolutely necessary; in JS hex can only be represented as strings most of the time so the lowest common denominator is the `number` format which (ugh) effectively evaluates to `int`/`float`; that's ok though, you can still do bitwise math and so on with it. I've also decided to throw out the framework I was using to build the front end of the application.  I was using a framework called Quasar with VueJS; it was starting to bug me.  It was kind of slow and made annoying design decisions; quite a lot of my time has been spent figuring out what the best approach would be instead. Aaanyway, yeah, decisions made; it'll boot a looot faster now even though the UI will look a bit more primitive.  I'm going for function over aesthetics though.

The most important thing is that I've written a nice tight asynchronous stream reading routine that'll help me jump from ROM location to location.  i've also written a reverse-byte-order tool.  Both are working nicely so the pointer math shouldn't be any bother at all!  The nice thing about JS - in spite of its discrepancies - it's so quick to write compared to Java.  I was tempted to upgrade the JVM tool but - even though I'm well versed in Java and other C-like languages - I truly cannot be naffed to learn Swing (a very old desktop UI development library for Java); it's a big antiquated tool chain, I just don't stand to gain much useful knowledge for future projects by learning it.  Plus, that'd require reading through the whole of the existing project which - frankly - is kind of boring.  Meh

so yeah, I'll dip in-and-out of this thread as progress trots along.  I'm definitely up for working on this though, it's a harder project than I initially foresaw, but - meh - it ain't rocket science. thanks to @Crashtour99 for sharing some docs on tilemapping I pretty much get how the animations actually stick together now; gotta code me some glue.

[Munchy]

28/03

Yesterday, I managed to nail loading tilemaps with a function that simply supplies samus' top/bottom table and a pose offset.  So that's one of the big computations out of the way.

@Crashtour99 &/ @SMILEuser96 any idea how super metroid knows how many frames to load for each pose? I looked in the frame delay table, tilemap pointer table and the DMA Index table but there's not really any obvious pattern that delineates how many frames to load (ie. each pose followed by FF FF or 00 00 etc).  Thanks to JSM I can manually input the frames if absolutely necessary but - obviously - it'd save me a heap of time if there was some way of loading the frame count computationally; i figure it's in there somewhere!

[Munchy]

I found this stub in the JSM code:

Code Select Expand


frameSpeedPointer = getAnimationFrameSpeedPointer(stream, getPoseIndex());
        stream.setPosition(BitHelper.snesToPc(frameSpeedPointer));
        int animationDuration = stream.readUnsignedByte();
        numberOfFrames = 0;
        while (animationDuration < 0xF0) {
            numberOfFrames++;
            animationDuration = stream.readUnsignedByte();
        }


Yeah, actually, cross referenching that, that makes sense, ignore me! I get it now!

[Crashtour99]

any idea how super metroid knows how many frames to load for each pose?

The thing to remember is that the game only ever has ONE frame loaded at a time, no matter which pose or transition animation it's running.
Realistically the game doesn't care how many frames an animation has.  The key is in the RAM map.
7E:0A96 - 7E:0A97    Samus Animation Frame
That keeps track where in the current animation processing is taking place.  The animations are loaded dynamically on a frame by frame basis.
It'll first check to see if any button presses tell it to change to a different pose via the transition table.
If not, it'll check the animation frame counter to see if it needs to go to the next animation frame.
If so, it'll use the animation frame value for indexing to load the next sequence of graphical and tilemap updates, as well as the next frame delay.
The frame delay values are what determine how long an animation is, as there are code pointers at the end of every string (well, actually they're index values for a table of code pointers).  They tell the game if it needs to loop back to the start of the animation, transition to a different pose, etc...  Positive values (00-79) are frame delays, negative values (80-FF) are for code.  However long the string of frame delay values is, is how many frames of animation there are for any pose/transition, and the game doesn't know how long that string is until it gets to the code pointer at the end.  It's astoundingly blind when you think about it.

[Munchy]

29/03

The complete tilemap, every sprite for every frame of Samus' top or bottom for a given pose (e.g. "samus moonwalking facing left"), now loads through a single function. That's a biiig chunk of code out of the way!  Now I've gotta do step2 and match that up with the DMA entries.  All being well, that shouldn't be too difficult.  Then I can start actually rendering the data to the screen!  It'll be one of those things that just happens all of a sudden most likely! anyway, it's hella late, so I'm gonna hit the sack! ciao!

[Munchy]

31/03

I've managed to get the VRAM loaded into hex now.  So that's pretty cool.  HOWEVER, after a little confusion, I think I've realised that JSM - which I've used as my frame of reference for nearly all this stuff - has a mistake in its VRAM interpretation.  It looks like they get it right enough that it doesn't break their animation schema; but their VRAM data is loading 1 tile too many (which it shouldn't be if I've read @Crash's docs correctly).   This can be evidenced by a tile from part2 appearing at the end of part1 in their VRAM map and the presence of a seemingly irrelevant tile from a different pose loading at the end of part2.

Just in case I'm wrong however, please would you sanity check @Crashtour99?

given the sample in your document ...
-----------------------------------------------------------------------
E0 93 9E C0 00 40 00    ;11
...
The first 3 bytes are a long pointer to the actual graphics in the ROM.  The next 4 bytes define how much data to transfer for parts 1 and 2.
$9E93E0 = long pointer to gfx
#$00C0  = how many bytes of data to transfer for part 1
#$0040  = how many bytes of data to transfer for part 2
-----------------------------------------------------------------------
... are the following assertions correct?

part 1 would be exactly:  $c0 / $20 = 6 tiles in length
part 2 would be exactly: $40 / $20 = 2 tiles in length and begin on the tile adjacent to where the 6 tiles transferred for part1 left off.

That certainly seems to check out with my logic and the data i've cross referenced in Munchtroid V1.0 (and it also looks right when I look at the sprite models in JSM).

[Munchy]

01/04

today I mapped what - hopefully - will be the tile data into a format I can read to the screen as visual data.  this is the first iteration of the model for that.  I'm gonna do all the bit maths when I next get a chance!

Code Select Expand

Convert()
  .vramToTiles(samus)[ // frame#
  { // frame metadata
    top: {
      _address,
      _id,
      parts: [ // VRAM parts
        [ // VRAM part 1
          { // tile metadata
            _address,
            _id,
            tile: [ // each tile's data / 0x10
              [ // tile low words
                ['lowBits', 'highBits'],
                ...
              ],
              [ ... as above ] // tile high words
            ]
          }
        ],
        [ ... asAbove ] // VRAM part 2 (see part 1)
      ]
    }
    bottom: { ...asAbove } // see top
  },
  ...
]


anyhow, as it is, this'll call a single word needed to build up a single line of pixels in a single tile:

Convert()
  .vramToTiles(samus)[0]
  .bottom
  .parts[0][0]
  .tile[0][0][0]

I had wanted to do some clever property assignment to the arrays rather than having objects all the way down, unfortunately, I was missing the data when it was transferred over to the front end side of the software (the file handling side of the software uses nodeJS, while the front end uses Babeled ES^6; sometimes they disagree!)

[Munchy]

02/04

OK, big step:

here's a nice old data layout of a tile:

Code Select Expand


Samus VRAM
  [ [ 7, 3, 0, 0, 0, 3, 7, 7 ],
    [ 7, 0, 0, 0, 0, 0, 7, 5 ],
    [ 7, 0, 0, 0, 0, 0, 7, 5 ],
    [ 7, 0, 0, 0, 0, 0, 7, 5 ],
    [ 7, 0, 0, 0, 0, 0, 7, 5 ],
    [ 0, 0, 0, 0, 0, 0, 0, 7 ],
    [ 0, 0, 0, 0, 0, 0, 0, 7 ],
    [ 0, 0, 0, 0, 0, 0, 0, 7 ],
    _id: 954400,
    _address: '$e9020' ]


and here's the same 8x8 tile rendered in munchtroid v1.0 (apologies for blurriness)



in case you can't see the resemblance between the array and the image: paint-by-numbers  :lol:

so now that's all dealt with, I should be able to start on rendering the tiles to the screen :)

also, after a bunch of umming and ahhing, i've decided to simplify the data model for samus further and to do all her pixel generation directly after loading her data via DMA.  the trade off is that - if Super Munchtroid ever expands to become an enemy editor as well, it'll need a bunch of extra code to do the tile work.

For the purposes of rendering Samus, at least, this should greatly increase performance.

anyhow, here's what the data object looks like now:

Code Select Expand


[
  {
    top: {
      _address,
      _id,
      parts: {
        part1: {
          _address,
          _id,
          tiles: [
            // below is a sample tile for illustrative purposes
            {
              _address: '$e9020',
              _id: 954400,
              data: [
                [ 7, 3, 0, 0, 0, 3, 7, 7 ],
                [ 7, 0, 0, 0, 0, 0, 7, 5 ],
                [ 7, 0, 0, 0, 0, 0, 7, 5 ],
                [ 7, 0, 0, 0, 0, 0, 7, 5 ],
                [ 7, 0, 0, 0, 0, 0, 7, 5 ],
                [ 0, 0, 0, 0, 0, 0, 0, 7 ],
                [ 0, 0, 0, 0, 0, 0, 0, 7 ],
                [ 0, 0, 0, 0, 0, 0, 0, 7 ]
              ]
            },
            ...
          ]
        },
        part2: { // see part1
          ...
        }
      }
    },
    bottom: { // see top
      ...
    }
  }
]


the goal will be to develop something that feels a lot like JSM but with a full paint editor like TLP.

anyway, that'll do for tonight! ciao!

[Munchy]

03 & 04/04

Last couple of days have been spent tweaking tilemap stuff and finally making a start on the front end.  Here's a demo of the tree you'll use to load a frame of animation. Note I've also added flexible windows.

[Munchy]

07/04

I was well on my way to having something presentable but I've hit a massive GL issue of some kind when running my software :/ I'm at the point where I'm considering switching out to a new build. I still have all my code so there's no problems on that front but it'll take me a while to migrate.  Probably I'll just jump back to using the Quasar Electron build since I haven't had any problems of this kind with it.  I cannot emphasize how incredibly annoyed I am right now.  f******g Chrome has naffed something up and I have neither the time nor inclination to get the minutiae necessary to report the issue to God-knows-who. Ridiculous, I can't even run Super Munchtroid right now, I just get a blank screen and some kind of half-assed error message with no meaningful stack trace. ugh.

[Munchy]

07/04 evening

I've been bed-ridden with some crappy stomach bug today, so I thought - meh - I'll go at this problem hammer and tong. After a lot of reading around and experiments, I couldn't really fix the problem; so I migrated the whole thing into a Quasar build of Electron (which seems to be a lot more stable though a little slower on boot). 

Here is a gif of a not very optimized demo build. I need to fix the palettes; they're off, it's probably a high/low byte thing I wasn't aware of.



Right now I'm just loading the VRAM data with every request, this vram stuff will be moved into its own page most likely.  It's useful for me as a frame of reference however.

anyway, so it's starting to look like a thing at long last  :heheh:

[edit about 5 minutes later]
did some wiring to the Promises under the hood, yeah, it's about 5 times faster than this gif now :)

[Quietus]

I know you're not getting inundated with responses, but don't let it put you off. I suspect many are happy with your regular progress updates, and are just waiting to get their grubby mitts on a working version. :^_^:

[Crashtour99]

I know you're not getting inundated with responses, but don't let it put you off. I suspect many are happy with your regular progress updates, and are just waiting to get their grubby mitts on a working version. :^_^:

^ this...  If it's even half as good as I think it'll be, it'll blow JSM out of the water.
Especially if its going to have an integrated gfx editor. 

[Munchy]

09/04

thanks for the kind words folks! Given the quality of the hacks out there, i figured people were just as excited as I am  :grin: and right now I'm onto the really fun bit of making this project!

ooooh boy I'm so glad I wrote this using Vue, the whole reactivity thing is really paying me back. In the gif below you're just seeing what's going on on-screen, under the hood, absolutely everything from the tile, to the ROM pointer of the tile to each and every pixel in the tile is being dynamically stored as a variable withing my templates. So, this means - thanks to some careful planning at my end - it'll be super easy to wire changes back into your ROM/render the tile to a pixel editing screen/manage undo and redo (hopefully)/probably even let me hilight exactly which tile of a user-friendly samus sprite you're editing.

anyways, it's early morning in Colombia and I'm travelling somewhere kind of remote tomorrow. Internet depending, therefore, I might be radio silent for a couple weeks but - believe me - I can and will be working on this offline between then and now so there'll be stuff to look forward to.

[Munchy]

10/04

No pictures today (I have very limited internet so gonna keep it streamlined!)
- Fixed Samus' palette; just some byte ordering stuff, no biggy.
- Ironed out some bugs here and there
- and, most important, have created a meaningful way of reading tilemaps that will help me easily convert back and forth.

This has also given me some serious thought about the scope for allowing users to do more advanced things.  For example, you decide you wanna pull in 4 tiles to a sprite that currently only has 1; no problem. likewise, completely possible to quickly do stuff like flip a whole sprite horizontally/vertically (surely very useful for some forms of jumping and morph-balling) and it doesn't even require overwriting the existing images.  It's 1 bit and the snes does the rest!

There's maybe even scope (though this would be a long way down the line) for having advanced stuff like VRAM repointing; ergo, add new frames to Samus (obviously you'll have to wire em up! that's a WHOLE other story!)

[Munchy]

11/04

Tackling the first BIIIG bug of the project. Could well have had something to do with the problems I was having before. Space jump seems to be reading the DMA data incorrectly. When it tries to jump to the long tile offset pointer it's 3 bytes off. Took me aaages to trace that! (Had to start at the top of the program and work backwards eliminating different processes as they passed various tests. My suspicion is that it's something to do with multiplying against a zero DMA index ... I'll figure it out tonight hopefully... All things considered, everything has gone incredibly smoothly thus far on SMunch 2.0!

[Munchy]

12/04

LOOOOL, fixed it! I mistyped one of the DMA addresses as "98D7" when it should have been 9BD7!!  :razz: :razz: :razz: oh berrrother that is way too easy to do haha!

ok, back on with the interesting stuff!

[Munchy]

Little by little... including me showing off the space jump which no longer crashes the program! the cross will house a sprite mock up.  To it's right I'll probably put the paint screen...

[Munchy]

13/04

first legible sprite of Samus (as she would appear in game) is now rendering to screen (either 8x8 or 16x16) for every frame of animation. I've still gotta put the others in and handle hflip and vflip.  If I get some time tomorrow I'll try and crack on with that. I'm also trying to do some work to manage complexity a bit as Sprite rendering gets very labyrinthine very quickly.   Ideally I want something that will be easy to re-remember in the future/easy for others to tinker with.

oh also ... note-to-self: I need to figure out why spin jump is only rendering one frame of animation instead of the 8 or so it should do.

[Quietus]

This is already looking tantalisingly close. I'll look forward to the first decent replacement after the initially flurry of nude, furry, and other oddities being released. :grin:

[Munchy]

This is already looking tantalisingly close. I'll look forward to the first decent replacement after the initially flurry of nude, furry, and other oddities being released. :grin:

I'm conscious that I'm opening up our community to some probably dreadful remodeling.  :lol:

[Munchy]

14/04

ooookay, I'll just leave this here:

[Crashtour99]

That's some good lookin progress man!  Lookin forward to this.

[Munchy]

15/04

A big bug fixing sesh this evening.

@Crashtour99 I think you'll be interested in this... Here are some of the discoveries I've made:
1. If the game doesn't want to render a top/bottom half of VRAM, the ROM is just bunged with a bunch of zeros instead of a pointer to a tilemap.  The normal spin jump is a good example of this.

2. Sometimes (most strikingly, for about half of all morph ball frames) the tilemaps for the bottom half of samus actually load the tops of her VRAM; no idea what the reasoning was behing that, but there you go.  If you try and use the indexes in the bottom, frames go missing and/or Samus' foot appears in the morph ball  :lol:

3. There are some very weird and very not-functional poses in the collection (that almost certainly don't actually get used in the game); and I have to have my sprite emulation cope with this by expecting missing data in a number of places.  The prize for most-broken goes to pose $45 which is complete chaos!

anyhow, I've caught all the big gotchas (everything seems to be rendering as I would expect it to without throwing exceptions and all the frames match those in JSM)

soooo, yeah, hopefully I'll make a start on the editor in the next few days.  Along the way I've been thinking hard about some of this stuff; I'm definitely gonna need a faq with some expectation management... even with a dedicated editor, Samus will take time to edit and editing her will require some trial and error as sometimes your change to one tile in one piece of VRAM will actually edit said VRAM in a bunch of different places.   Then there's the issue of which poses are actually used by the game and which are who-knows-what.  Not to put people off, just a reminder that we're still dealing with a legacy piece of hardware that wasn't designed to be retrospectively changed!

on a lighter note, here's some grappling!