[SM] Layer 3 custom haze woes

[monktroid]

From what I've gleaned, there isn't a lot of scope for simply writing a new HDMA fade.  My limited guess, as someone relatively new to the 65816, is that Super Metroid uses A LOT of HDMA channels.  Basically, I've run a bunch of tests effectively injecting the simple HDMA stub from the SMW HDMA tutorial into SM.  some tests replaced existing FX1 routines (causing the game to crash the moment you enter the room with said effect).  I also tried assigning it to our old friend the false X-ray block BTS 03.  the moment samus touches said block: the screen either goes black or freezes on a slightly brighter colour and the game crashes. 

Qwertie mentions the indirect tables that SM uses for stuff like the temperature mirages in Norfair.  I'm not convinced I'd need to use one of these tables because I only want to create a static fade (I'm hoping to set it behind a layer 2 background as a sort of fake parralax effect).  I would expect, therefore, that the SMW example would work in much the same way in SM.  Here's a primitive attempt using the Fool XRay.

Is it that the destination register is simply not available? Is it that the HDMA channel isn't available? What did I miss?

Code Select Expand


LoRom

org $9498B2
  DW $B300 ;Fool XRay BTS $03
  JSR Freespace

org $94B300
Freespace:
  REP #$20 ; 16 bit A
LDA #$0000 ; $43X0 = 00
STA $4330         ; $43x1 = 00
LDA #LVL1BRIGHT ; get pointer to brightness table
STA $4332         ; store it to low and high byte pointer
PHK         ; get bank
PLY         ;
STY $4334         ; store to bank pointer byte
SEP #$20         ; 8 bit A
LDA #$08         ; Enable HDMA on channel 4
TSB $0D9F              
        RTS ; return

LVL1BRIGHT:
db $0C,$0F
db $0C,$0E
db $0C,$0D
db $0C,$0C
db $0C,$0B
db $0C,$0A
db $0C,$09
db $0C,$08
db $0C,$07
db $0C,$06
db $0C,$05
db $0C,$04
db $0C,$03
db $0C,$02
db $0C,$01
db $0C,$00
db $00


[P.JBoy]

I'm guessing your PLY is corrupting the stack because it pulls back two bytes where PHK only pushes one. I would also advise putting a PHP and PLP at the start and end of that function, I would expect the game wants a 16-bit accumulator when that function returns

[PHOSPHOTiDYL]

You don't need that jsr freespace.
The dw $B300 is your pointer in this situation.

[Quote58]

So, here's the thing monktroid. Super Metroid isn't short on channels (as that tutorial mentions, you can make very good use of only a couple of channels), and there's actually an unused FX in the game if you want to just add your own FX in. You also do not have to use indirect hdma, but the routines set up for using hdma use indirect because many fx's in the game (scrolling sky, power bombs, xray, heat background, etc.) need dynamic tables and it's set up well for that. The reason your code crashes isn't because of anything like that, it's just the code itself. However, before I explain why, I just want to mention that HDMA really, really shouldn't be the first big thing you try to tackle with asm. It's one of the parts of SM that is the least documented, the least well understood and utilized, and although some parts are pretty straightforward, other parts of hdma can get very complex. Mode 7 is the other really big part of the game that has little documentation and has only been properly utilized by a handful of people so far.

That all being said, your code doesn't work because to move from smw code to sm we have to do a little translating. For starters, the registers in the tutorial's code assume 8bit when entering, while super Metroid will in most places you put the code, be in 16 bit. The first problem is exactly what PJboy mentioned, which is the PHK PLY. Instead, we can just go LDY #$0080 : STY $4334 or SEP #$30 : LDY #$80 : STY $4334, either one is fine, but the second also sets up 8bit for the following LDA #$08. The second problem is that smw has a whole different ram map from super Metroid, so whenever you see their code storing to an address in 7E/7F, you need to double check what that address is supposed to be. In this case, 0D9F in smw is the address for which HDMA channels are active, but in super Metroid it's not even documented, but it's likely to do with position calculations, judging from context. In super Metroid, the address for enabling hdma channels is $0085. There are other addresses that do similar things for specific places, but $85 is the general purpose one and will work for this. The last problem is simply where to run the code.
As Phosphotidyl mentioned, $B300 is a pointer to code for the $03 air fool xray tile, and the JSR after it is overwriting the $B400 pointer. However I suggest not using air fool xray for this. You can test it with a run every frame routine, of which there are lots, if you need to test it between different rooms, but otherwise the best place to put it initially is in a layer1_2 pointer for the room. This is a routine that will run when you enter the room, and isn't used super often so most rooms it won't matter if you replace it.
To do so, you put your code in free space at the end of bank $8F (E9A0 for example), and then replace the Layer1_2 pointer that you can find in the pointer window in smile, with the pointer to your code, and remember to RTL at the end of it. Then whatever room you set the pointer in will run your HDMA code.
For an example of a simple fade like the one from the tutorial, here's code that works in super Metroid (you also need to make sure that the gamestate is normal gameplay, or else going to the subscreen will mess everything up) using a direct table the same way as the tutorial:

Code Select Expand


org $8FE9A0
REP #$30
LDA $0998 : CMP #$0008 : BEQ + : RTL           ;if normal gameplay
+
LDA #$0000 : STA $4330
LDA #Brightness : STA $4332
LDY #$0080 : STY $4334
SEP #$20
LDA #$38 : TSB $0085                                    ;enable hdma channels
REP #$20
RTL

Brightness:                                                             ;fades into the screen from top and bottom
db $04,$00
db $04,$01
db $04,$02
db $04,$03
db $04,$04
db $04,$05
db $04,$06
db $04,$07
db $04,$08
db $04,$09
db $04,$0A
db $04,$0B
db $04,$0C
db $04,$0D
db $04,$0E
db $04,$0F
db $62,$0F
db $04,$0F
db $04,$0E
db $04,$0D
db $04,$0C
db $04,$0B
db $04,$0A
db $04,$09
db $04,$08
db $04,$07
db $04,$06
db $04,$05
db $04,$04
db $04,$03
db $04,$02
db $04,$01
db $04,$00
db $00

Btw, that tutorial is fantastic, thank you for linking it. It doesn't go over certain functions that are used by super Metroid, but it covers the basics.

[monktroid]

Thanks so much Quote58. That's an awesome response! I'm a sucker for jumping in at the deep end haha! I wondered if the RAM map had something to do with it but, like you say, trying to piece everything together from sketchy resources and various disassemblies is a tall order! I'll have a proper read through your code tomorrow and see what happens :)

another useful resource from SMWCentral (though obviously you'll need to translate the code part of the output) is "Effect Tool" which will generate HDMA tables for you from a super friendly GUI:  https://www.smwcentral.net/?p=section&a=details&id=11479

smwcentral is something of a treasure trove for useful ASM resources!  FYI, Noobies: If you search on there, someone has written the mother of all ASM intro guides in a nice PDF format!

[benox50]

oh my gawd, this could mean amazing rad visuals  :bounce:

[monktroid]

oh my gawd, this could mean amazing rad visuals  :bounce:

that's the plan! I think ShyGuy Express' brother had done some funky stuff with visuals too.  Forget what his demo was called.

[Smiley]

I want to point out that SM has a routine that makes it really easy to set up HDMA effects. The routine is at $888435, and you only have to call it with a JSL. After the JSL you have arguments for $4300, $4301 and a PLM-like instruction list. For example, below is the code I used for the wavy backgrounds in SMChallenge:

Code Select Expand


lorom

org $8FEA00 ;Layer0, because this only needs to run once :D
;88:8435 Set up a DMA entry. Arguments are stored after the JSL that calls this:
;values for 4300/4301 (2 bytes total), 2 byte pointer to plm-esque instruction list

JSL $888435 ;Set up HDMA
DB $42,$0E ;Indirect 1 reg write twice, Layer1 vertical
DW #Inst ;Pointer to instructions

JSL $888435 ;Set up HDMA
DB $42,$0F ;Indirect 1 reg write twice, Layer2 horizontal
DW #Inst2 ;Pointer to instructions
RTS

Inst: ;X holds current HDMA slot
DW $8655 : DB $8F ;Source is this bank
DW $866A : DB $7F ;Table is in bank $7F
DW $85B4 : DL MoveTable ;Copies table from ROM to RAM and sets HDMA table pointer.
DW $85B4 : DL Rotate ;Initial update to avoid graphical glitches
Loop:
DW #$0001 : DW IndirTable
DW $85B4 : DL Rotate
DW $85EC : DW #Loop


Inst2: ;X holds current HDMA slot
DW $8655 : DB $8F ;Source is this bank
DW $866A : DB $7F ;Table is in bank $7F
DW $85B4 : DL MoveTable2 ;Copies table from ROM to RAM and sets HDMA table pointer.
DW $85B4 : DL Rotate2 ;Initial update to avoid graphical glitches
Loop2:
DW #$0001 : DW IndirTable2
DW $85B4 : DL Rotate2
DW $85EC : DW #Loop2



print pc
MoveTable:
PHA : PHY : PHX
LDA $18C0,x : AND #$00FF : TAX

LDX #$001E
-
LDA Table1,x : STA $7FFE02,x
DEX : DEX : CPX #$0000 : BPL -


PLX : PLY : PLA : RTL

MoveTable2:
PHA : PHY : PHX
LDA $18C0,x : AND #$00FF : TAX

LDX #$001E
-
LDA Table,x : STA $7FFF02,x
DEX : DEX : CPX #$0000 : BPL -


PLX : PLY : PLA : RTL

Rotate: ;Rotates offset values. There are then added to layer2 x position, and IndirTable points to the results
PHX
LDX #$001E

LDA $7FFE20 : PHA
-
LDA $7FFE00,x : STA $7FFE02,x
CLC : ADC $0915 : STA $7FFA02,x
DEX : DEX : CPX #$0000 : BNE -

PLA : STA $7FFE02
CLC : ADC $0915 : STA $7FFA02

PLX : RTL

Rotate2: ;Rotates offset values. There are then added to layer2 x position, and IndirTable points to the results
PHX
LDX #$001E

LDA $7FFF20 : PHA
-
LDA $7FFF00,x : STA $7FFF02,x
CLC : ADC $0917 : STA $7FFB02,x
DEX : DEX : CPX #$0000 : BNE -

PLA : STA $7FFF02 : CLC : ADC $0917 : STA $7FFB02


PLX : RTL


print pc
Table1: ;Just values, no scanline counts
DW $0000, $0001, $0001, $0002
DW $0002, $0001, $0001, $0000
DW $FFFF, $FFFF, $FFFE, $FFFE
DW $FFFD, $FFFE, $FFFE, $FFFF

Table: ;Just values, no scanline counts
DW $0001, $0002, $0003, $0005
DW $0003, $0002, $0001, $0000
DW $FFFF, $FFFE, $FFFC, $FFFB
DW $FFFC, $FFFE, $FFFF, $0000


IndirTable: ;Scanline counts and pointers to the above values
DB $1F : DW $FA02

DB $02 : DW $FA02
DB $02 : DW $FA04
DB $02 : DW $FA06
DB $02 : DW $Fa08

DB $02 : DW $FA0A
DB $02 : DW $FA0C
DB $02 : DW $FA0E
DB $02 : DW $FA10

DB $02 : DW $FA12
DB $02 : DW $FA14
DB $02 : DW $FA16
DB $02 : DW $FA18

DB $02 : DW $FA1A
DB $02 : DW $FA1C
DB $02 : DW $FA1E
DB $02 : DW $FA20

DB $02 : DW $FA02
DB $02 : DW $FA04
DB $02 : DW $FA06
DB $02 : DW $Fa08

DB $02 : DW $FA0A
DB $02 : DW $FA0C
DB $02 : DW $FA0E
DB $02 : DW $FA10

DB $02 : DW $FA12
DB $02 : DW $FA14
DB $02 : DW $FA16
DB $02 : DW $FA18

DB $02 : DW $FA1A
DB $02 : DW $FA1C
DB $02 : DW $FA1E
DB $02 : DW $FA20

DB $02 : DW $FA02
DB $02 : DW $FA04
DB $02 : DW $FA06
DB $02 : DW $Fa08

DB $02 : DW $FA0A
DB $02 : DW $FA0C
DB $02 : DW $FA0E
DB $02 : DW $FA10

DB $02 : DW $FA12
DB $02 : DW $FA14
DB $02 : DW $FA16
DB $02 : DW $FA18

DB $02 : DW $FA1A
DB $02 : DW $FA1C
DB $02 : DW $FA1E
DB $02 : DW $FA20

DB $02 : DW $FA02
DB $02 : DW $FA04
DB $02 : DW $FA06
DB $02 : DW $Fa08

DB $02 : DW $FA0A
DB $02 : DW $FA0C
DB $02 : DW $FA0E
DB $02 : DW $FA10

DB $02 : DW $FA12
DB $02 : DW $FA14
DB $02 : DW $FA16
DB $02 : DW $FA18

DB $02 : DW $FA1A
DB $02 : DW $FA1C
DB $02 : DW $FA1E
DB $02 : DW $FA20

DB $02 : DW $FA02
DB $02 : DW $FA04
DB $02 : DW $FA06
DB $02 : DW $Fa08

DB $02 : DW $FA0A
DB $02 : DW $FA0C
DB $02 : DW $FA0E
DB $02 : DW $FA10

DB $02 : DW $FA12
DB $02 : DW $FA14
DB $02 : DW $FA16
DB $02 : DW $FA18

DB $02 : DW $FA1A
DB $02 : DW $FA1C
DB $02 : DW $FA1E
DB $02 : DW $FA20

DB $02 : DW $FA02
DB $02 : DW $FA04
DB $02 : DW $FA06
DB $02 : DW $Fa08

DB $02 : DW $FA0A
DB $02 : DW $FA0C
DB $02 : DW $FA0E
DB $02 : DW $FA10

DB $02 : DW $FA12
DB $02 : DW $FA14
DB $02 : DW $FA16
DB $02 : DW $FA18

DB $02 : DW $FA1A
DB $02 : DW $FA1C
DB $02 : DW $FA1E
DB $02 : DW $FA20

DW $0000


IndirTable2: ;Scanline counts and pointers to the above values
DB $1F : DW $FB02

DB $02 : DW $FB02
DB $02 : DW $FB04
DB $02 : DW $FB06
DB $02 : DW $FB08

DB $02 : DW $FB0A
DB $02 : DW $FB0C
DB $02 : DW $FB0E
DB $02 : DW $FB10

DB $02 : DW $FB12
DB $02 : DW $FB14
DB $02 : DW $FB16
DB $02 : DW $FB18

DB $02 : DW $FB1A
DB $02 : DW $FB1C
DB $02 : DW $FB1E
DB $02 : DW $FB20

DB $02 : DW $FB02
DB $02 : DW $FB04
DB $02 : DW $FB06
DB $02 : DW $FB08

DB $02 : DW $FB0A
DB $02 : DW $FB0C
DB $02 : DW $FB0E
DB $02 : DW $FB10

DB $02 : DW $FB12
DB $02 : DW $FB14
DB $02 : DW $FB16
DB $02 : DW $FB18

DB $02 : DW $FB1A
DB $02 : DW $FB1C
DB $02 : DW $FB1E
DB $02 : DW $FB20

DB $02 : DW $FB02
DB $02 : DW $FB04
DB $02 : DW $FB06
DB $02 : DW $FB08

DB $02 : DW $FB0A
DB $02 : DW $FB0C
DB $02 : DW $FB0E
DB $02 : DW $FB10

DB $02 : DW $FB12
DB $02 : DW $FB14
DB $02 : DW $FB16
DB $02 : DW $FB18

DB $02 : DW $FB1A
DB $02 : DW $FB1C
DB $02 : DW $FB1E
DB $02 : DW $FB20

DB $02 : DW $FB02
DB $02 : DW $FB04
DB $02 : DW $FB06
DB $02 : DW $FB08

DB $02 : DW $FB0A
DB $02 : DW $FB0C
DB $02 : DW $FB0E
DB $02 : DW $FB10

DB $02 : DW $FB12
DB $02 : DW $FB14
DB $02 : DW $FB16
DB $02 : DW $FB18

DB $02 : DW $FB1A
DB $02 : DW $FB1C
DB $02 : DW $FB1E
DB $02 : DW $FB20

DB $02 : DW $FB02
DB $02 : DW $FB04
DB $02 : DW $FB06
DB $02 : DW $FB08

DB $02 : DW $FB0A
DB $02 : DW $FB0C
DB $02 : DW $FB0E
DB $02 : DW $FB10

DB $02 : DW $FB12
DB $02 : DW $FB14
DB $02 : DW $FB16
DB $02 : DW $FB18

DB $02 : DW $FB1A
DB $02 : DW $FB1C
DB $02 : DW $FB1E
DB $02 : DW $FB20

DB $02 : DW $FB02
DB $02 : DW $FB04
DB $02 : DW $FB06
DB $02 : DW $FB08

DB $02 : DW $FB0A
DB $02 : DW $FB0C
DB $02 : DW $FB0E
DB $02 : DW $FB10

DB $02 : DW $FB12
DB $02 : DW $FB14
DB $02 : DW $FB16
DB $02 : DW $FB18

DB $02 : DW $FB1A
DB $02 : DW $FB1C
DB $02 : DW $FB1E
DB $02 : DW $FB20

DW $0000



[monktroid]

Code Select Expand


org $8FE9A0
REP #$30
LDA $0998 : CMP #$0008 : BEQ + : RTL           ;if normal gameplay
+
LDA #$0000 : STA $4330
LDA #Brightness : STA $4332
LDY #$0080 : STY $4334
SEP #$20
LDA #$38 : TSB $0085                                    ;enable hdma channels
REP #$20
RTL

Brightness:                                                             ;fades into the screen from top and bottom
db $04,$00
db $04,$01
db $04,$02
db $04,$03
db $04,$04
db $04,$05
db $04,$06
db $04,$07
db $04,$08
db $04,$09
db $04,$0A
db $04,$0B
db $04,$0C
db $04,$0D
db $04,$0E
db $04,$0F
db $62,$0F
db $04,$0F
db $04,$0E
db $04,$0D
db $04,$0C
db $04,$0B
db $04,$0A
db $04,$09
db $04,$08
db $04,$07
db $04,$06
db $04,$05
db $04,$04
db $04,$03
db $04,$02
db $04,$01
db $04,$00
db $00

Hi Quote57.  I'm finding that this code scrambles the game.  I'm also a little unsure why in

Code Select Expand

LDA #$38 : TSB $0085
you open channels 3, 4 and 5.  I've played around with various locations at the end of the bank and it doesn't seem to be happy!

[monktroid]

SMILEuser96 posts Amazingness

also, this is a thing of beauty! thanks!

[Quote58]

Don't know why it'd be scrambling the game but keep in mind that this stuff needs the FX1 of the room to not be active. However, the code itself does work:
[spoiler][/spoiler]
But I had it running through a different routine rather than the layer1_2, so I just added an org to free space in 8F, I didn't test it since that's the way I had done it originally. I just now checked and it isn't working as a layer1_2 pointer, though I'm not sure why, sorry about that, would've checked to make sure it was working but I was short on time. It should also be RTS not RTL btw, my bad.
The 38 is just because I had been using multiple channels in a different test and forgot to limit it when I removed the other stuff. It doesn't hurt anything to do that, but obviously only use 1 channel if you only need 1.

edit: I just realized it doesn't work because layer1_2 only runs once after the room loads, sorry! The routines smiley's code calls obviously set it up the way the game normally does and those will keep it active, but if you want to do it by directly changing the registers yourself you have to have the routine run every frame you want it active

[monktroid]

Hiya Quote58, I apply the asm and it screws with the tile map. I've tried XKAS at command line and via smile. Both just result in the tile map being scrambled and the game crashing on load... weird (and that's after I swap the RTS out).  Pretty sure it's unheadered and pretty sure it's [JU]... not really sure what else I can do with it!  Was the screenshot you gave above using something you'd written in the past or something fresh from the code you've just written?

[Quote58]

Now that's really weird. The screenshot I took was using the code I pasted there, but it was being run from my hud (I use it for testing stuff because I can just add it as an extra routine since it's completely rewritten, and it runs every frame (it's also indirectly called when loading the room, which is also a useful place for testing things)). The code as is should run once at the start of the room that points to it, but since it doesn't use the indirect routines it shouldn't show anything for more than a frame.
Not sure why it would change the tile map of the room, unless you put it in the wrong pointer in smile maybe? It's the Layer1_2 pointer at the bottom of the pointers window.

[monktroid]

think I'mma try the same code on some hacked roms to see if the behaviour is native to my rom.  If it isn't my next step will be to try some other versions of Xkas. Isn't there a newer more reliable compiler than xkas now anyway? I forget it's name, I think i have it somewhere....

[monktroid]

Right, I'm getting there ..!
I'm fairly confident the compiler was playing silly beggars and causing the tile table to scramble.  I switched from XKAS to ASAR (which, from what I've seen thus far should be everyone's go to instead of xkas, it's way more robust and will not write if it encounters compilation errors or duff headers; 1 thing to bear in mind, your unheadered file must explicitly have the .sfc and NOT .smc file extension or it'll basically throw a tizzy)

I've demoed Scyzers health restore test on the rom with ASAR as the compiler and it definitely works.  In lieu of knowing hijack spots well enough, the closest i've managed to get to running Quote58's haze e.g. is by pointing fake-xray BTS03 to Quote58's code.

this does not crash at runtime, however, when samus walks into the fake xray 03 blocks, the screen goes black until the player moves her out of the block.

my dumb and humble questions therefore are:
1) what is a good room-specific hijack point? (I'm testing this example in the room directly to the left of the landing site: $792FD)  Don't mind if it's specific to that room, although ideally I wanna use one of the unused "FX Layer Type" slots.

2) what would cause the screen to simply black out? I have no FX on, transparency is normal.  The game still runs, soundFX and all, it just knocks the lights out!

Code Select Expand

org $9498B2
DW $B300

org $94B300
JSL $8FE9A0
RTS

org $8FE9A0
REP #$30
LDA $0998 : CMP #$0008 : BEQ + : RTL           ;if normal gameplay
+
LDA #$0000 : STA $4330
LDA #Brightness : STA $4332
LDY #$0080 : STY $4334
SEP #$20
LDA #$38 : TSB $0085                                    ;enable hdma channels
REP #$20
RTL ; I kept the RTL given we're jumping banks in this instance.

Brightness:                                                             ;fades into the screen from top and bottom
db $04,$00
db $04,$01
db $04,$02
db $04,$03
db $04,$04
db $04,$05
db $04,$06
db $04,$07
db $04,$08
db $04,$09
db $04,$0A
db $04,$0B
db $04,$0C
db $04,$0D
db $04,$0E
db $04,$0F
db $62,$0F
db $04,$0F
db $04,$0E
db $04,$0D
db $04,$0C
db $04,$0B
db $04,$0A
db $04,$09
db $04,$08
db $04,$07
db $04,$06
db $04,$05
db $04,$04
db $04,$03
db $04,$02
db $04,$01
db $04,$00
db $00

Lastly, I wanna shout out how grateful I am for all the helpful comments.  You guys are awesome!

[Quote58]

Really didn't expect the code I pasted to cause so many problems, sorry about that. This time I did it on a vanilla rom so there's no chance that it's something to do with the other code I have running. This is running through a simple hijack right before the hud updates samus' health counter, which means it happens every frame while the game is running. I tested it on a standard unheadered vanilla rom with xkas gui (just adds a small gui for compiling with xkas) version 2.2.3. This works in any room but if there's another fx it has a problem because this is affecting layer 3, which is the same layer that other FX's use as well.

Code Select Expand

lorom

org $809B8B : JSR HDMA_test
org $80CDA0
HDMA_test:
LDA $0998 : CMP #$0008 : BEQ + : LDA $09C2 : RTS
+
REP #$30
LDA #$0000 : STA $4330
LDA #LVL1BRIGHT : STA $4332
LDY #$0080 : STY $4334
SEP #$20
LDA #$08 : TSB $0085
REP #$20

LDA $09C2 : RTS

LVL1BRIGHT:
db $04,$00
db $04,$01
db $04,$02
db $04,$03
db $04,$04
db $04,$05
db $04,$06
db $04,$07
db $04,$08
db $04,$09
db $04,$0A
db $04,$0B
db $04,$0C
db $04,$0D
db $04,$0E
db $04,$0F
db $62,$0F
db $04,$0F
db $04,$0E
db $04,$0D
db $04,$0C
db $04,$0B
db $04,$0A
db $04,$09
db $04,$08
db $04,$07
db $04,$06
db $04,$05
db $04,$04
db $04,$03
db $04,$02
db $04,$01
db $04,$00
db $00

[monktroid]

awesome, that worked! tah very muchly!

[monktroid]

I'm curious to know if it is possible to set the haze transparency behind other layers.  All evidence would seem to suggest that the answer is no, but I'm pretty sure I've seen something similar done with layer 3 before.  In fact: this!
https://youtu.be/b-HDkmmGHyA?t=6m51s

though ShyGuy seems to suggest this is playing around with some BTS trickery. I'm still unsure how you'd get it to behave with different scroll speeds like that.

[benox50]

Well in SMW, they said that using color math can make it affect FG and BG, so if you remove the color math it should not affect any layer and be behind everything. I'm curious to know if its possible too in SM, I'm sure that is. I think Smiley did it

In the video you linked its not HDMA, its layer 3 only. You can have all 3 layers + HDMA if you want. And have them at different depth,
you can even make the gradient scroll :o

So logically you could have what you can see in the video + gradient on top or behind or in the middle

[Smiley]

Anomie's register doc: http://www.romhacking.net/documents/196/

Specifically, especially $2131:

Code Select Expand


2130  wb+++- CGWSEL - Color Addition Select
        ccmm--sd

        cc = Clip colors to black before math
            00 => Never
            01 => Outside Color Window only
            10 => Inside Color Window only
            11 => Always

        mm = Prevent color math
            00 => Never
            01 => Outside Color Window only
            10 => Inside Color Window only
            11 => Always

        s     = Add subscreen (instead of fixed color)

        d     = Direct color mode for 256-color BGs

        See the sections "BACKGROUNDS", "WINDOWS", and "RENDERING THE
        SCREEN" below for details.


2131  wb+++- CGADSUB - Color math designation
        shbo4321

        s    = Add/subtract select
            0 => Add the colors
            1 => Subtract the colors

        h    = Half color math. When set, the result of the color math is
            divided by 2 (except when $2130 bit 1 is set and the fixed color is
            used, or when color is cliped).

        4/3/2/1/o/b = Enable color math on BG1/BG2/BG3/BG4/OBJ/Backdrop

        See the sections "BACKGROUNDS", "WINDOWS", and "RENDERING THE
        SCREEN" below for details.


2132  wb+++- COLDATA - Fixed Color Data
        bgrccccc

        b/g/r = Which color plane(s) to set the intensity for.
        ccccc = Color intensity.

        So basically, to set an orange you'd do something along the lines of:
            LDA #$3f
            STA $2132
            LDA #$4f
            STA $2132
            LDA #$80
            STA $2132

        See the sections "BACKGROUNDS" and "WINDOWS" below for details.

[monktroid]

Anomie's register doc: http://www.romhacking.net/documents/196/

Specifically, especially $2131:

Code Select Expand


2130  wb+++- CGWSEL - Color Addition Select
        ccmm--sd

        cc = Clip colors to black before math
            00 => Never
            01 => Outside Color Window only
            10 => Inside Color Window only
            11 => Always

        mm = Prevent color math
            00 => Never
            01 => Outside Color Window only
            10 => Inside Color Window only
            11 => Always

        s     = Add subscreen (instead of fixed color)

        d     = Direct color mode for 256-color BGs

        See the sections "BACKGROUNDS", "WINDOWS", and "RENDERING THE
        SCREEN" below for details.


2131  wb+++- CGADSUB - Color math designation
        shbo4321

        s    = Add/subtract select
            0 => Add the colors
            1 => Subtract the colors

        h    = Half color math. When set, the result of the color math is
            divided by 2 (except when $2130 bit 1 is set and the fixed color is
            used, or when color is cliped).

        4/3/2/1/o/b = Enable color math on BG1/BG2/BG3/BG4/OBJ/Backdrop

        See the sections "BACKGROUNDS", "WINDOWS", and "RENDERING THE
        SCREEN" below for details.


2132  wb+++- COLDATA - Fixed Color Data
        bgrccccc

        b/g/r = Which color plane(s) to set the intensity for.
        ccccc = Color intensity.

        So basically, to set an orange you'd do something along the lines of:
            LDA #$3f
            STA $2132
            LDA #$4f
            STA $2132
            LDA #$80
            STA $2132

        See the sections "BACKGROUNDS" and "WINDOWS" below for details.

Dude, you are a veritable treasure trove of knowledge! This document is amazing! thanks so much!

[P.JBoy]

Anomie's docs ain't got nothing on no$sns' docs

[monktroid]

The SMW Effect Tool is a thing of joy and wonder that saves SO much time. Now I'm getting stuff done... my GOD. I have some cool ideas floating around my melon and I'm fairly confident I can do em!!

[monktroid]

Hmmmmmmmmmmmmmmmmmmmm.....!

So I'm trying to translate a wobbly BG2 from the SMW Effect Tool and - while I've managed to stop it crashing the game (I'm using room 79f2d as my test bed) - it doesn't make BG2 animate. 

In effect, the animation is supposed to give the room's background a kind of swampy lava-lamp type effect.  The animation is easy to recreate in the Effect tool, choose vertical mode and have a play.

A curious side effect is that the mushrooms in an adjacent room have started flashing in places(!!!!??)

Furthermore, when I check the HDMA flag in Geiger it seems to be suggesting that data is being sent to Reg $2111 (layer3 horizontal scrolling).  But I've specifically set it to $2110 soooo... eh?!

Here is the code. I've been pouring over it for hours now. those bits that aren't generated by the effect tool are marked with an asterisk.  I've followed it through end to end in the debug and - with all the crashes fixed - I can't see anything I wouldn't expect from the program. Please, I'm sure I'm just missing something obvious; put it down to inexperience.  Any help enormously appreciated:

Code Select Expand

lorom
org $8F91D3
JMP InitCode  ; * original location expects RTS so used JMP here rather than JSL.     
org $8FE9A0   ; * some space
InitCode:               ;\  This section is to be used in the INIT code of levelASM   
   print pc   ; * debug spot.
   REP #$20             ; |
   LDA #$1002           ; | Use Mode 02 on register 2110
   STA $4330            ; | 4330 = Mode, 4331 = Register
   LDA #$C802           ; | Address of HDMA table below
   STA $4332            ; | 4332 = Low-Byte of table, 4333 = High-Byte of table   
   SEP #$20             ; |
   LDA.b #$7F           ; | Address of HDMA table, get bank byte *** Monktroid:  this doesn't zero the high byte of the accumulator, I'm wondering if it's sending something it shouldn't to 4335? any ideas?
   STA $4334            ; | 4334 = Bank-Byte of table
   LDA #$08             ; |
   TSB $0085  ; * Enable HDMA channel 3
   RTS                  ;/  End HDMA setup  

;The Table takes up 172 bytes of the free RAM
;It ranges from $7FC802 - $7FC8AD (both addresses included)
org $809B8B : JSR MainCode ; * hijack samus' health.
org $80CDA0
MainCode:              ;\  This section is to be used in the MAIN code of levelASM 
  LDA $0998 : CMP #$0008 : BEQ + : LDA $09C2 : RTS ; * test game mode
  +
  SEP #$30
  LDY #$00             ; | Y will be the loop counter.
  LDX #$00             ; | X the index for writing the table to the RAM
  LDA $13              ; | Speed of waves
  LSR #1               ; | Slowing down A
  STA $00              ;/  Save for later use.

  PHB : PHK : PLB      ;\  Preservev bank
.Loop:                  ; | Jump back if not finished writing table 
   LDA #$04             ; | Set scanline height
   STA $7FC802,x        ; | for each wave
   TYA                  ; | Transfer Y to A, to calculate next index
   ADC $00              ; | Add frame counter
   AND #$0F             ; |
   PHY                  ; | Preserve loop counter
   TAY                  ;/  Get the index in Y

   LDA.w .WaveTable,y   ;\  Load in wave value
   LSR                  ; | Half only
   CLC                  ; | Clear Carry for addition
   ADC $20              ; | Add value to layer Y position (low byte)
   STA $7FC803,x        ; | store to HDMA table (low byte)
   LDA $21              ; | Get high byte of X position
   ADC #$00             ; | Add value to layer X position (low byte)
   STA $7FC804,x        ;/  store to HDMA table (high byte)

   PLY                  ;\  Pull original loop counter
   CPY #$38             ; | Compare if we have written enough HDMA entries.
   BPL .End             ; | If bigger, end HDMA
   INX                  ; | Increase X, so that in the next loop, it writes the new table data...
   INX                  ; | ... at the end of the old one instead of overwritting it.
   INX                  ; |
   INY                  ; | Increase loop counter
   BRA .Loop            ;/  Repeat loop

.End:                   ;\  Jump here when at the end of HDMA
   PLB                  ; | Pull back data bank.
   LDA #$00             ; | End HDMA by writting 00...
   STA $7FC805,x        ; | ...at the end of the table.
   REP #$30             ; | reenable native mode.
   print pc                ; * debug point to observe the routine ending successfully.
   CLC                     ; * Kill the carry inherited from above :P It's code assembled by a GUI program so go fig.
   LDX #$1400 : LDY #$07C6 ; * restore original indices' values.   
   LDA $7E09C2 : RTS       ; * Load samus' health stuff that we hijacked and return.

.WaveTable:             ;\ 
   db $00               ; |
   db $01               ; |
   db $02               ; |
   db $03               ; |
   db $04               ; |
   db $05               ; |
   db $06               ; |
   db $07               ; |
   db $07               ; |
   db $06               ; |
   db $05               ; |
   db $04               ; |
   db $03               ; |
   db $02               ; |
   db $01               ; |
   db $00               ;/ 

[Quote58]

I took a look at the output of effect tool a while back and it's really not as good a reference point as the asm file smiley provided. I'm not saying you can't translate the code from effect tool, but it would be more efficient to work with the routines already in super Metroid. They have checks and balances, and they use the correct addresses that super Metroid chooses to mirror to the correct registers. However if you're like me and just prefer to make your own routines so that you can change aspects of it without repointing and rewriting code, then I suggest studying the code from the program as well as the code smiley provided, and using the documentation available from qwerty/no$sns/anomie to write your routine from scratch, so you can start small and work upwards.

That all being said, wavy backgrounds are actually pretty simple, so I would use the program to the get the h/v values you want (since it has a nice visual and sliders for getting the right values), then use smiley's asm to learn how to format all the tables/instructions, and then write a routine that stores to the registers according to the documentation.

As for the code you have there, I'll read through it later but room 792fd has a bg pointer and a fx1 'a' pointer, so I would try first with a room that for sure has nothing else going on that could affect it, or at least turn all fx's off first.