[SM] SMILE RF wont import most tiletables correctly - what's going on here?

[Pluribius]

I'm running the latest version of SMILE RF (using auto-updater and stuff). I exported the default Tileset 02 tiletable, moved the pointer via HxD, and then re-imported it - and when I tried to re-import it, the program insists that the compressed size is only 3 bytes large. The entire table (sans the CRE) displays as glitch blocks. I've tried to import into this rom's tiletables the exports from several other ROMs, same results. I tried importing 03 and 04 and 06 into their respective tilesets as well - all failures, the same sort of glitchiness. Only importing 00 or 01 works. I opened the tiletable files in a hex editor next - they look fine in the editor as far as I can tell. I checked all the pointers and they seem to be in order. I'm not using space that doesn't exist - the hex editor confirms that there's buttloads of space at the end of the ROM, and I expanded it a number of times after witnessing this issue (didn't help). SMILE JX gets the same results.
What the heck is going on here, and how do I fix it?

Possibly related - SMILE RF frequently crashes when I import palettes. It might be related to trying to import palettes to bugged tiletables.

Side note: I didn't blank the palettes and tiletables former locations in the ROM, nor did I remove them from tileset_entries, is that a problem?
Side note 2: Are there limits on what banks the SCE tile tables can access? I heard tile tables were unrestricted by banks, but if there are limits that I didn't know about and I'm breaking them, that could explain a few things. (right now, the broken tables start at 304E00, though the CRE table pointer points to 32D600 and that works just fine, so I'm a bit dubious of this).

Edit: This problem seems to have begun after I repointed the CRE. I did so with an export/import (a direct copy could be troublesome since the table was overwriting something else, because newbie) - could that be the cause, if the CRE was using more space than it had allocated (because this was someone else's hack and they goofed)?

[begrimed]

If that's the case, then your fix should be pretty easy. You should be able to just copy/paste a range of bytes, then from there you'll need to make the CRE tile tables manually so you can be sure to stay within the tile table data size limit.

Repointing the CRE tile table: repointing the CRE tile table will allow you to use all 41 of the white XXXX tiles in it, but NOT the line of eight blank black tiles that sit directly on top of the gray door cap tiles. CRE pointers work a bit differently, but the concept of repointing them is exactly the same. The maximum size of the CRE tile table is 2048 bytes ($800 if you're counting in hex).

The CRE's tile table pointer is located at offset 1683D. The data at this pointer location is "B9 85 48 A9 9D A0" - which is really just the LoROM address $B9:A09D (PC: 1CA09D). Ignore the 85 48 49. If you'll look in tileset_entries.txt near the top, you'll see 1CA09D and 1CA634. 1CA09D is where the CRE's tile table data is located, and 1CA634 is where it ends. With "Block: 1CA09D-1CA633 / Length: 597" highlighted in HxD, copy those bytes to your clipboard, then replace all of them with free space. Next, find $800 bytes of free space somewhere to paste-write these copied CRE tile table bytes; I'm going paste them at offset 1C0100. After that, go to offset 16830 and replace the bytes B9, 9D A0 with B8, 00 81 and save the ROM.

Lastly, open tileset_entries.txt and delete 1CA09D, and then underneath 1C0100, add 1C0900 and save the file. Done! The gray "CRE Table" box should say 1C0100, and if you click the tileset editor's Save button, the "CRE Table Compressed Size" numbers should now say #### / 2048.

Repointing the CRE tile sheet: repointing the CRE tile sheet most likely won't be something you need to do unless you're doing a lot of thorough graphic editing. The two rows of white X tiles along the bottom of the CRE tile sheet can NOT be edited, even after repointing! The maximum size of the CRE graphic sheet is 12288 bytes ($3000 if you're counting in hex).

The CRE's tile sheet pointers are located at offset 16415, and at 16797. The data at both of these pointer locations is "B9 85 48 A9 00 80" - which point to $B9:8000 (PC: 1C8000). Ignore the 85 48 A9. Highlight bytes 1C8000-1CA09C, copy them to your clipboard, replace them with free space, then paste-write them at offset 1C0900. Go to offsets 16415 and 16797, replace B9, 00 80 with B8, 00 89 at both of those spots and save the ROM. Open tileset_entries.txt again, replace 1C8000 with 1C3900 and save the file. Done.

IMPORTANT: QuickMet will appear glitched after you've repointed either of the CRE. This is because QuickMet's generated test ROM, QuickMet.SMC, still uses the old CRE pointers. This is easily fixed. If you've repointed CRE, open the "QuickMet.bin" file that's loctated in the SMILE RF\Files\TestRoom\ folder with a hex editor, and change the data at offsets 3C (CRE tile sheet) and 53 (CRE tile table) to match the data that is at 16415/16797 (CRE tile sheet) and 1683D (CRE tile table) in your ROM.

Edit: crashes when you import palettes shouldn't ever happen, though, unless you've got unsaved edits open in HxD while trying to import, so that might be a legit program error.

[Pluribius]

I couldn't get this figured out, so I gave up and tried again from a backup copy from before everything broke.

I redid all the pointers there (and did a direct-copy on the CRE table this time, even taking into account the extra space it took up).
Now SMILE RF won't open the rom. "Runtime Error 9: Subscript out of range." It's trying to open a room in the rom but there's nothing at the pointers yet because I was going to import everything from within SMILE RF itself. Unfortunately since I repointed literally every tile table it can't find a room with a valid tile table to load so it refuses to load the ROM (or so I assume).
I'll try to direct copy one of the tile tables over though.

Meanwhile, though, my original problem utterly stumped me and I can only assume it got corrupted or something. HxD was open during the importing/exporting process, though I'm pretty sure I had no unsaved edits.

EDIT: I think I found the problem
Some of the "free space" I was trying to write to was actually occupied, it was just tiny chunks of data scattered throughout a vast open field of emptiness. I don't know what all is pointing to it, but there was probably a conflict.
Anyway, though, I don't have to reconvert all the pointers because conveniently the freespace stops being partially-occupied around 308000 PC (as opposed to the 300000 PC i was trying to use) so I can just add 1 to one of the bytes in each pointer to bump each address up by 8000 PC.

[Pluribius]

Ok, problem is now solved, I wrote to actual free space this time. While I initially got a bunch of runtime error 9s, that was because SMILE technically loads a room on startup, even if there are no valid tileset tables or palettes available (because you were going to import them directly after modifying the pointers). After direct copying the first four tilesets (tables and palettes) into the correct places (I know the first room uses one of those four), everything works and now I can just import the rest.
Thanks for the help, begrimed.