Converting SMILE to a modern language

[someperson]

Edit 1:
Here's the repo: https://github.com/dansgithubuser/dansSuperMetroidLibrary

Edit 2:
This thread is now also for feature requests.

Original post:
I downloaded this version of the SMILE source:
http://jathys.zophar.net/files/smile.zip
I installed vb6, am learning Visual Basic, and slowly converting SMILE to C++. Not all of it -- just enough to create a library that would enable easy creation of new editors. My end goal is a better randomizer than what's out there. I'll be converting things like rom reading, writing, and functions that ensure legal edits.

I just want to make sure I'm not repeating anyone else's work. I'm going to send JAM (the SMILE JX guy) a message. Does anyone know of anything else I should check out? Thanks!

[someperson]

I see there's a Cheesy C++ Super Metroid editor too. Has anyone managed to compile it?

[Metaquarius]

I see there's a Cheesy C++ Super Metroid editor too. Has anyone managed to compile it?

Yep. From this repo : http://code.google.com/p/cheesie/
It has been a while though, I can't remember whether I edited some files so as to make it compile on VS, you should get a bunch of warnings only. And don't forget to change the ROM path in mainpanel.cpp. If you intend to use it as a base for something else, please get rid of Cheesie's Compresch module, AFAIK, lunarCompress is a lot faster when it comes to compress stuff. :^_^:

[someperson]

Hi! Thanks for the info. I'd appreciate any dumps of little tidbits like that, if anyone isn't sure if it'll be useful. Like, is lunarCompress written by the SMILE guy? Is Compresch written by the Cheesy guy? (I think you just said it was. It would make sense since he says he's interested in data formats in the repo description). My current plan is to start with Cheesy and verify it with SMILE. So I'll be going through the code deep, which means I should be able to optimize it. I bet there's a good enough reason Compresch is slower, but I can try to mimic SMILE when speed seems like an issue.

The problem I had with compiling was wxWidgets. Admittedly I don't have much experience with it, but when I get compiler (not linker) errors that aren't "cannot find header file" I get upset and blame it on someone else. In particular, wxMemoryDC does not have a member function StretchBlit (SMClass.cpp, line 409). Today I'm going to try to rip out the ROM loading guts and make my own simple display with SFML (I'm not interested in an interface right now, but drawing levels is a good sanity check).

To be clear, my efforts are open source -- I just don't have anything to show right now. I guess I should set some kind of schedule. I'll make and post a repo once I'm happy with loading (and before I start on writing/editing).

[Zhs2]

No, Lunar Compress is written by the Lunar Magic guy. Compresch may be custom, but I dunno. Either way, interdpth (the author of Cheesie) had at one time stipulated needing some additional advice understanding where and why things were as they were in Super Metroid, but of course that whole lack of communication that happens when interdpth and his beta testers or others who may advise not being all around at the same time causes projects to linger (this is the entire story of Zero Fission and Double Helix as well!) Doesn't look like the eventual release of the SMILE source has pushed Cheesie any further, either.

[interdpth]

Hi! Thanks for the info. I'd appreciate any dumps of little tidbits like that, if anyone isn't sure if it'll be useful. Like, is lunarCompress written by the SMILE guy? Is Compresch written by the Cheesy guy? (I think you just said it was. It would make sense since he says he's interested in data formats in the repo description). My current plan is to start with Cheesy and verify it with SMILE. So I'll be going through the code deep, which means I should be able to optimize it. I bet there's a good enough reason Compresch is slower, but I can try to mimic SMILE when speed seems like an issue.

The problem I had with compiling was wxWidgets. Admittedly I don't have much experience with it, but when I get compiler (not linker) errors that aren't "cannot find header file" I get upset and blame it on someone else. In particular, wxMemoryDC does not have a member function StretchBlit (SMClass.cpp, line 409). Today I'm going to try to rip out the ROM loading guts and make my own simple display with SFML (I'm not interested in an interface right now, but drawing levels is a good sanity check).

To be clear, my efforts are open source -- I just don't have anything to show right now. I guess I should set some kind of schedule. I'll make and post a repo once I'm happy with loading (and before I start on writing/editing).

You will need to update to the latest branch of wxWidgets, also I think this copy may be latest? I don't know I worked on it for another week a few months back but the lack of people wanting Ceres turned me off. Alternatively you can make a windows only port and just call StretchBlt from WinApi. Please do use the code and make something of it. :)

[someperson]

I thought I got the most recent version of wxWidgets, but I don't know. I figured a google search for the function would easily turn up results if it was ever in a version. I'm not trying to put down your code. Libraries just bother me a lot and one of my goals is to make some code that does SM ROM reading and THAT'S IT, so anyone can very easily include (literally just #include and add a .cpp to the project) it into their own efforts. In any case I'm having an easy enough time tracing your code myself  :^_^:. I only ever wanted to compile it to be sure it worked. But now that you're here, I can ask directly!

Anyway, I'm disinclined to use Lunar Decompress, because it comes with a DLL and I can't find the source for the DLL. Just some source that uses it. Is your Compresch code at parity with Lunar Decompress? I'm not worried about the speed thing mentioned earlier right now. Just worried about functionality.

Also while I'm here, can I also ask for a quick overview of the resources you used while making Compresch? If it's not too in depth I'd like to sanity check your code as I go.

May as well ask one more thing if it's no too much trouble. Looking at Compresch_StdBlock::Decompress, am I to understand that an SM ROM is segmented into a bunch of blocks with differing compression and that indicate their compression type up front?

[interdpth]

Compresch was made by my friend named Disch. MathOnNapkins helped with all the SM portion of the lib

[Metaquarius]

Worried about functionnality, eh ? LunarCompress has countless functions which makes life easier for programmer, not only compression stuff. Unless you want to reinvent the wheel and spend months in this low-level stuff, you just can't ignore such nice library. Adding it to your project is as easy as including new source file btw.

Decompression is mostly required to read correctly graphic set and level data. Otherwise, you can write any fancy code you like. My WxWidgets build is 2.8.12. I guess interdpth has used Wx and Compresch to make Chessy cross-platform in the first place.

You might be interested in this too : http://forum.metroidconstruction.com/index.php/topic,1183.0.html

[interdpth]

Worried about functionnality, eh ? LunarCompress has countless functions which makes life easier for programmer, not only compression stuff. Unless you want to reinvent the wheel and spend months in this low-level stuff, you just can't ignore such nice library. Adding it to your project is as easy as including new source file btw.

Decompression is mostly required to read correctly graphic set and level data. Otherwise, you can write any fancy code you like. My WxWidgets build is 2.8.12. I guess interdpth has used Wx and Compresch to make Chessy cross-platform in the first place.

You might be interested in this too : http://forum.metroidconstruction.com/index.php/topic,1183.0.html

Yup, both choices were made for multiplatform support.

[someperson]

I've gone through the decompression. Barring any stealth bugs, I think I'm good. It's a lot simpler when you do SM only.

I'm stuck at tile sets. interdpth (or anyone), do you know what tsa stands for? For example, gameImage.h line 18:
vector<u16> TSA;

In my head, this is what makes sense: rooms have 2d arrays of tile pointers, so given a big graphic of all the tiles, and a palette, you can turn a tile pointer into the corresponding graphic. But there's these TSA things that are involved too and I can't figure out what they would be for. SMILE seems to call them tables. But arrays of tiles and tables of tiles are the same thing.

[snarfblam]

I don't know about SM in particular, but a TSA (tile square assemble iirc) a.k.a. metatile a.k.a. combo etc. is a block made out of tiles. For example, a TSA could be a 16x16 px block made up of four 8x8 px tiles. TSAs would be defined in a table (a.k.a. array, etc.).

[interdpth]

Aight, so TSA is Tile Squardoid Assembly or something, it's what I learned when I first started anywho, it's a tile block.


You have the TileGFX in vram, TilePal in Pal RAM, and then you have the tileblock info

so 1 ingame tile is 16x16, this takes 4 8x8s to make
or somethine

anyway



so you have map ram, this is where the tileblock info gets sent too




So, when you're looking at your tileset that is all tileblock info.

[someperson]

Hey guys, thanks for the tips!

I'm currently working on Ceres. Does anyone know what's going on with its tiles or where I could find out? SMILE doesn't handle it correctly and interdpth said he never got around to it.

After I make sure all rooms load correctly (I'm only looking at tiles and doors) I'm going to make and post a repo.

[someperson]

Posted a repo. It's in the first post.

Edit: Still waiting on anyone who knows about Ceres.

[P.JBoy]

Could you be more specific with what you want to know about Ceres?

[Zhs2]

Bump. You mentioned Ceres graphics earlier on IRC, and then you left like seven minutes before Jathys showed up. It seems Ceres graphics are interlaced. More details from a conversation log Jathys had: http://jathys.zophar.net/files/ceres_format.txt

[someperson]

Thanks, that log helps. I wont have time to play with it until tonight. Still not sure I understand though.

P.JBoy: Specifically, how do you get from a Super Metroid ROM to displaying Ceres? What is the difference between all other graphics and Ceres graphics? Interlaced usually means you draw even lines and then odd lines, I don't think this is what is meant here. If the graphics and TSA are interlaced, then instead of reading two addresses, I have to read one. Which is it, the usual location of the TSA or the usual location of the graphics? Or is it something totally different?

For clarity (I hope), this how graphics usually are:
tileSetPointer=0x7E6A2+roomState.tileSet*9;
Unique room elements are at tileSetPointer+3, compressed
Common room elements are at 0x1C8000u, compressed
The TSA is at tileSetPointer+6, compressed
More of the TSA is at 0x1CA09D, compressed, but only if it's not Ceres (I forgot I did this until just now -- will play with it later)
The palette is at tileSetPointer, compressed

[someperson]

I think I've addressed the slow compression issue. So unless there's a feature in Lunar Magic (related to Super Metroid) that isn't in Dan's Super Metroid Library, welcome to open source compression about as good as Lunar Magic.

I didn't finish Ceres yet. I went ahead to ROM writing, because now I can change stuff in Ceres and see what happens, to help identify what data is what. So I'm coming back to Ceres now.

I got some discouraging responses on IRC with respect to Ceres. Remember: I'm not making an editor and I'm really not making your idea of an editor. I am making a Super Metroid library, which may be used in, among other things, editors. Editors are things that have interfaces and help people move data around intuitively. Libraries help people who make editors easily move data around, so they can concentrate on the human side of things. So, if it's in Super Metroid, it belongs in a Super Metroid library. This keeps the library agnostic -- making a SMILE library is pointless, because you might as well put it into SMILE. Making a Super Metroid library is not pointless, because then anyone can use it to create their idea of an editor.

I'm still waiting on getting the updated SMILE source from JAM. I'm frustrated that it wasn't uploaded to github or sourceforge upon creation, but I understand there might be complications I don't know about. To me, the only difference is the time it will take to complete this project.

[someperson]

OK. So here's all the tile sets in Super Metroid:

Code Select Expand


# tsa    graph  pal    description

0 20b6f6 1d4629 212d7c Crateria
1 20b6f6 1d4629 212e5d red Crateria

2 20beee 1d7911 212f43 fungus Crateria and live old Tourian
3 20beee 1d7911 213015 dead old Tourian

4 20c5cf 1dae9e 2130e7 wrecked ship
5 20c5cf 1dae9e 2131a6 unpowered wrecked ship

6 20cfa6 1de6b0 213264 pink, green, and blue Brinstar

7 20d8dc 1e25aa 21335f green metal near Kraid and red Brinstar
8 20d8dc 1e25aa 213447 tunnel to statue room

9 20e361 1ec3f9 2135e4 red Norfair, green bubble Norfair, middle Norfair, lower Norfair
10 20e361 1ec3f9 2136bb pink Norfair

11 20f4b1 1f3130 21383c Maridia
12 21055f 1f678d 21392e icy Maridia

13 211b01 1fd414 213aed Tourian
14 211b01 1fd414 213bc1 Mother Brain chamber

15 21275e 203004 214104 Ceres rooms normal
16 21275e 203004 2141e3 Ceres rooms exploding

17 21275e 20622a 214104 Ceres shaft normal (mode 7)
18 21275e 20622a 2141e3 Ceres shaft exploding (mode 7)
19 21275e 208da9 214104 Ceres Ridley room normal (mode 7)
20 21275e 208da9 2141e3 Ceres Ridley room exploding (mode 7)

21 21227b 20060b 213c9c Save room, statue room
22 21227b 20060b 213d7b Save room
23 21227b 20060b 213e58 Save room
24 21227b 20060b 213f3d Save room
25 21227b 20060b 214021 Save room

26 20e189 1e5ff0 213510 Kraid
27 20f3af 1efe2a 213798 Crocomire room
28 21160d 1f9dea 213a2c Draygon room


tsa    graph  description

1ca09d 1c8000 common room elements
21227b 1d4629 Crateria
20beee 1d7911 fungus Crateria and old Tourian
20c5cf 1dae9e wrecked ship
20cfa6 1de6b0 pink Brinstar
20d8dc 1e25aa metal, red Brinstar, and way to Kraid
20e361 1ec3f9 Norfair
20f4b1 1f3130 Maridia
211b01 1fd414 Tourian
21275e        Ceres
       203004 Ceres non-mode-7 rooms
       20622a Ceres shaft (mode 7)
       208da9 Ceres Ridley flying (mode 7)
21227b 20060b Save room, statue room
20e189 1e5ff0 Kraid
20f3af 1efe2a Crocomire room
21160d 1f9dea Draygon room
       182000 Ceres Ridley room (uncompressed)


So, the tile sets 17-20 have mode 7 information where usually there is typical tile graphics. Mode 7 information includes both graphics and how to place the tiles, which is usually in the tsa and level data (addresses not listed). Tile sets 17-20 are used in 2 rooms: Ceres shaft and Ceres Ridley room (two versions each: normal and exploding). In the Ceres shaft room, mode 7 is used to rotate the room as it explodes. The level data marries up with the mode 7 representation, but if you change it around, nothing happens. So it seems like it's there but not used. The Ceres Ridley room is different, though. Mode 7 is used to rotate Ridley as he flies away from you (toward the screen). The level data contains tile information, which is definitely used because if you edit it, the edit comes through in the game. The tsa is definitely used because if you scramble it, the tiles in that room get scrambled. But I cannot figure out what graphics that tsa is referring to. I tried scrambling the Ceres shaft mode 7 graphics (20622a), and I tried scrambling the typical Ceres graphics (203004), but neither have any effect on the Ceres Ridley room.

I grabbed bsnes and snes9x debug versions. The plan is to figure out what is being read while in the Ceres Ridley room. In normal rooms, I could find the tiles in VRAM with the bsnes VRAM viewer. In Ceres Ridley room, I could find everything but the tiles.

What I want at this point (although I'm open to suggestions other than "give up") is a list of addresses being read from while in Ceres Ridley room. Does anyone know how to do this? Does anyone know what a snes9x usage file is? Does anyone know of any bsnes or snes9x debug tutorials?

Once I have a list of read locations, I can check against SMAll. If it's code, whatever. If it's not, hm, maybe it's the graphics.

[Crashtour99]

IIRC the tiles for Ceres Ridley's room are uncompressed in the ROM, and are located near Ridley's sprite gfx along with the metroid larva/container.  Odds are they're transferred to VRAM along with the metroid's/Ridley's/door gfx (since the doors are enemies) during normal room loading stuffs (there's probably a special routine to take care of this since it's only done in 1 room).  They probably did it this way because the room's gfx are the mode 7 image of Ridley flying toward the screen.

[P.JBoy]

The usage files are cool, just click the 'Gen. Offsets' button and the debugger will generate all the addresses read, written or executed to in a text file (for example, here's the list of RAM addresses not written to / read from in a full playthrough I did)

[interdpth]

IIRC the tiles for Ceres Ridley's room are uncompressed in the ROM, and are located near Ridley's sprite gfx along with the metroid larva/container.  Odds are they're transferred to VRAM along with the metroid's/Ridley's/door gfx (since the doors are enemies) during normal room loading stuffs (there's probably a special routine to take care of this since it's only done in 1 room).  They probably did it this way because the room's gfx are the mode 7 image of Ridley flying toward the screen.

Shaking rooms do the same

[someperson]

Crashtour99: Found the tiles! Thank you!

P.JBoy: Ah, gotcha, thanks!

Interdpth: Oh dear, just when I thought I was getting to the bottom of this. Thanks for the info!

[someperson]

interdpth, what do you mean by "shaking rooms"? I can't find any other weird rooms. I tried the speed booster room, but that one's fine. Are you talking about the Ceres shaft?