DEHEXTRA discussion (split from Pr+ UMAPINFO thread)

I am all for this DEHEXTRA lump idea.

Paging @Graf Zahl btw.

11 minutes ago, plums said:

I hope once this format gets finalized it can also be applied to Heretic. HHE was never fully developed and support for it in modern ports is basically limited to Chocolate/Crispy Heretic; having a DEH-like tool that works in Crispy Heretic, Eternity, and GZDoom would be fantastic.

That would presumably demand implementing Heretic codepointers, which can be a problem.

10 hours ago, Shadow Hog said:

You'd need to change a lot of pointers, whether to expand the size of the tables in-place or to move them outside their normal location to, say, the end of the executable's filesize, but I suppose it's doable, yes.

Yeah, I made a hack for the press beta years ago which expanded the states table like this (it loaded a complete replacement state table from a file on disk to a malloc'ed buffer, then patched all references to states[] to go there instead).

Budko's Doom+ hacks change data section size in exe header and patch all references to visplanes[], drawsegs[], etc. to expanded buffers in memory after the origina end address of data section.

7 minutes ago, Cacodemon345 said:

I am all for this DEHEXTRA lump idea.

 

Paging @Graf Zahl btw.

It's not really a lump, more like an expansion for DEHACKED, adding extra "empty" states/actors for modding.

And I believe he already added it to GZDoom: https://github.com/coelckers/gzdoom/commit/f647f3c1321f0bcf4a42e334390989f1e9d79bf3

10 hours ago, plums said:

I hope once this format gets finalized it can also be applied to Heretic. HHE was never fully developed and support for it in modern ports is basically limited to Chocolate/Crispy Heretic; having a DEH-like tool that works in Crispy Heretic, Eternity, and GZDoom would be fantastic.

Was HHE source code ever released? (DHE 3.1 3.0a source was)

edit: nope it seems :(

33 minutes ago, Ferk said:

I guess in theory it could be possible to make it dynamic implementation-side by simply reallocating them whenever the Thing/Frame id given is higher than the current number of things, without introducing "NEW".

Absolute values will limit to only one DEHACKED patch at a time and will cause problems with multiple patches.

I'm talking about a syntax like this (please ignore the extra fields):

14 minutes ago, fraggle said:

Before we start considering forks, I'll just say that I'd be open to including this in DEH9000.

 

I'd like to see DEH9000's state parser expanded to something like a DECORATE-lite so that people can write vanilla-compatible patches (Or Boom/MBF-compatible, etc.) in a nicer format while being able to take advantage of DEH9000's features like the state auto-reclaim. It's potentially tricky in some ways though and parts of it need to be carefully considered.

DelphiDoom's and FPCDoom's ACTORDEF script (similar to DECORATE) is a convenient way to write dechacked patches, since the only thing the ACTORDEF parser does is to convert the ACTORDEF script to a dehacked patch. Then the dehacked subsystem parses the auto-generated dehacked script and does all the job.  And by using the -devparm parameter it prints the dehacked script to the log file.

9 minutes ago, xttl said:

 

Was HHE source code ever released? (DHE 3.1 source was)

AFAIK no. Chocolate Doom had to reverse-engineer support for it: https://www.doomworld.com/forum/topic/50676-chocolate-heretic-hhe-support/

Also if the source was released someone probably would have fixed it; the last HHE release doesn't entirely work properly with SotSR.

@Gez I meant an equivalent but incompatible implementation for Heretic; I don't expect Doom ports to implement all the Heretic code pointers, or vice versa.

38 minutes ago, jval said:

Absolute values will limit to only one DEHACKED patch at a time and will cause problems with multiple patches. 

I'm talking about a syntax like this (please ignore the extra fields):

You mean you'll reset the "NewFrame xx" and "NewThing xx" counts whenever a new patch is loaded and keep them separate? In theory the same could be done with the Extra region if a port really wanted to support that (you can have "Thing 150" in one patch be different than the "Thing 150" from the other patch).

That will keep the things/frames in that section independent on each patch (which sometimes might not be what you want, actually).

While I also at first thought about that, I think making mandatory the support for some dynamic "NewFrame/NewThing" syntax would increase the complexity and reduce the chances of this becoming a standard thing that all ports can easily support.

You can always have a third party tool to merge dehacked patches that use these EXTRA ids, or have some engines actually do that merge by themselves in the way I was suggesting.

3 hours ago, Altazimuth said:

Yeah OK so most of the 4 are worthless. There's the [HELPER] block which is only useful if you use -dogs so it isn't useful at all.

The other three are more interesting, as they appear to have been planned by Ty Halderman.

• [SPRITES]: You can change the 4-letter prefix of any sprite type. Seems possibly handy.
• [SOUNDS]: The SOUNDS block can change the name of any sound. Seems weird and kinda niche but maybe useful? I dunno the EE documentation mentions EDF sound mnemonics specifically, but it's worth noting the DOOM EDF sound names are all the exact same as their BEX counterparts.
• [MUSIC]: The MUSIC block can change the name of any music lump used by the game. This would have been useful in the early 2000s... but it's 2020 and UMAPINFO makes this pointless.

I think these allow for great subtle tweaks in terms of puzzles, i feel, some how.

I do have some silly question though:

• With the new extended states, how was a DeHacked patch achieved? Through WhackEd or? Because obviously DOS DeHackEd only supports a limited amount of states in the first place, so i am curious to how VGA's mods were achieved in the first place.

2 hours ago, esselfortium said:

Alright, I've split this into its own thread, as suggested :)

Thank you Essel for this :)

1 hour ago, esselfortium said:

I can see where having states point to each other could be useful to save time in some situations if you're editing the states table directly, but I don't think it's necessary.

 

Rather than worrying too much about hand-editing the states table, which is going to still be a pain in the ass no matter what the defaults are, I think the ideal goal for this would be to have fraggle's DEH9000 python library expanded or forked to support BEX/MBF/EXTRADEH patches. The end result there would effectively be a crossport Decorate-lite, allowing modders to use a nice easily-readable text format that "compiles" to EXTRADEH like it already does for vanilla DEH. Would provide many of the benefits of a new cross-port-compatible content definition system without requiring ports to actually implement yet another new content definition system on top of the ones they have already. :)

It all sounds a lot like the old Fusion port with its many many additional states.

On that note i like to reinsert the idea of a Chocolate Dehacked again. I originally referenced it for a direct Windows like implementation of DOS Dehacked (So with all its features like DoomHack) but DEH9000, as powerful as it is, i can imagine could be repurposed for something similar, and include these BEX/MBF/EXTRADEH

(I wish we could shorten the new state extentions to a 3 letter word, like DEH and BEX are) :P

EDIT: This is halfway what xttl is also suggesting in some form.

38 minutes ago, fraggle said:

Before we start considering forks, I'll just say that I'd be open to including this in DEH9000.

 

I'd like to see DEH9000's state parser expanded to something like a DECORATE-lite so that people can write vanilla-compatible patches (Or Boom/MBF-compatible, etc.) in a nicer format while being able to take advantage of DEH9000's features like the state auto-reclaim. It's potentially tricky in some ways though and parts of it need to be carefully considered.

Please see above. Its great that you are open to this. And i fully side with the points brought forward - DEH9000 has seen some amazing examples (Currently working on getting a few onto /idgames from TDRR) but putting the best parts forward is definitely something worthwhile. And a extended DeHacked patch that builds on top of vanilla DeHackEd (Instead of inventing an entirely new format like BEX, despite the similarities) can mean a lot for vanilla Doom in general.

44 minutes ago, xttl said:

Heh, if this stays simple enough, it might even be possible to implement support for it in vanilla by exe hacking. (possibly requiring having to run an external program which converts from text to binary format first, but hey, you need to run DHE to apply DEH files to vanilla anyway)

Perhaps the recent launcher VULD by @JadingTsunami could be of use here if that's the case? VULD already uses DeHacked (and DeuSF) to automatically apply DeHackEd patches, i figure some kind of crispy mode (pardon the pun) could be made that takes these Extended DEH states into account.

(I realize i sound like an Ideas Guy here, but at the very least i am very willing to test anything of this sorts if needed. Vanilla/pseudo-vanilla stuff is something i like to test in general)

Suggestion: infinite ammo with non-melee attack codepointer = shouldn't subtract ammo from anywhere. A weird omission in BEX patches, since it shouldn't be that hard to implement.

10 minutes ago, Ferk said:

 

You mean you'll reset the "NewFrame xx" and "NewThing xx" counts whenever a new patch is loaded and keep them separate? In theory the same could be done with the Extra region if a port really wanted to support that.

Though you'll be unable to have patches that modify other patches, which might be sometimes desirable.

I can't see why patches modifying other patches wouldn't work if you keep the same loading order. I'll also mention another idea, why limit frame and thing id's to numbers and don't use aliases and let the engine decide internally for the actual id.

Quote

Still, that's the kind of things that would increase the complexity and reduce the chances of this becoming a standard thing.

I don't thing, from a developer side of view, that this is that much more complex to implement. The only thing that must be done is to reallocate the internal tables, and that will happen only at load time (assuming we do no have the extreme need to load on demand, during gameplay, a dehacked patch).

6 minutes ago, Da Werecat said:

Suggestion: infinite ammo with non-melee attack codepointer = shouldn't subtract ammo from anywhere. A weird omission in BEX patches, since it shouldn't be that hard to implement.

It'd definitely make sense to have some new entries in the Misc section to get around some of Doom's hardcoded weapon behaviors, like the fact that the BFG and rocket launcher weapon slots don't auto-fire like other weapons. (Which in effect means that if you've put something that isn't similar to a BFG or a rocket launcher in that slot, your weapon sometimes doesn't fire when you would expect it to. Not great.)

3 minutes ago, jval said:

I can't see why patches modifying other patches wouldn't work if you keep the same loading order

Your example implied that the count for NewFrame would be reset when loading a different DEHACKED (if it didn't then it would not work the way you wanted, I think). And if you reset the count you were using for reference then you lose the reference.

Aliases would work, but they would still be more complex than simply expanding the tables.

1 minute ago, Ferk said:

Your example implied that the count for NewFrame would be reset when loading a different DEHACKED (if it didn't then it would not work the way you wanted, I think). And if you reset the count you were using for reference then you lose the reference.

Let's say that we have 1000 hardcoded states. The example adds 4 new states. Now we have 1004 states. To alter the example patch with another we can still use absolute values, and that will work if we preserve the correct load order.

eg remove the BRIGHT from NewFrame 4:

Frame 1004
Sprite Subnumber = 4

1 minute ago, Ferk said:

Aliases would work, but they would still be more complex than simply expanding the tables.

IMO not that complex, also why does it have to be too-simple to be a standard? It's nothing compared to a full working dehacked parser.

13 minutes ago, jval said:

To alter the example patch with another we can still use absolute values

At that point you are losing the advantage you wanted to get with the idea of making it relative. Why go through all the effort?

13 minutes ago, jval said:

IMO not that complex, also why does it have to be too-simple to be a standard? It's nothing compared to a full working dehacked parser.

IMHO, the alias idea is better than the NewFrame count, and in a way it's a superset of EXTRA (kinda like numeric aliases).

So why don't we see what the reception of the DEHEXTRA is in the modding community before deciding whether going the extra step for aliases is worth it? Specially if someone can come up with a dehacked that works in vanilla for this.. or maybe chocolate support?

So uhh... I added most of this stuff to EE. New frames, states, mobjs. The only thing I've not done is put dummy mobjs in the range of dehnum 146-150. In the UMAPINFO fork these're MT_GIBDTH, MT_STEALTHBABY, MT_STEALTHVILE, MT_STEALTHBRUISER, and MT_STEALTHHEAD—all of which are just dummies. Retro, on the other hand, has MT_GIBDTH, BT_BLUEBLOOD, MT_GREENBLOOD, MT_FUZZYBLOOD, and MT_TRAIL here. Two cents from relevant parties on what to do here would be appreciated.

Outside of that if there's any other new deh features I haven't added them. I'd like to make suggestions on what I feel is sane but I'd like to have port authors on-board with just the basics first.

45 minutes ago, Altazimuth said:

So uhh... I added most of this stuff to EE. New frames, states, mobjs. The only thing I've not done is put dummy mobjs in the range of dehnum 146-150. In the UMAPINFO fork these're MT_GIBDTH, MT_STEALTHBABY, MT_STEALTHVILE, MT_STEALTHBRUISER, and MT_STEALTHHEAD—all of which are just dummies. Retro, on the other hand, has MT_GIBDTH, BT_BLUEBLOOD, MT_GREENBLOOD, MT_FUZZYBLOOD, and MT_TRAIL here. Two cents from relevant parties on what to do here would be appreciated.

 

Outside of that if there's any other new deh features I haven't added them. I'd like to make suggestions on what I feel is sane but I'd like to have port authors on-board with just the basics first.

Sick. So now Retro, Crispy and Eternity have the extended states.

I do question however how much reinventing the wheel is done here though - Like mentioned, Fusion already did a very similar effort aeons ago, but it never got put to use, so in that sense it seems odd that a decade or two later it gets mileage.

1 minute ago, Redneckerz said:

Sick. So now Retro, Crispy and Eternity have the extended states.

Also GZDoom and PrBoom-plus (um).

9 minutes ago, Redneckerz said:

Sick. So now Retro, Crispy and Eternity have the extended states.

 

I do question however how much reinventing the wheel is done here though - Like mentioned, Fusion already did a very similar effort aeons ago, but it never got put to use, so in that sense it seems odd that a decade or two later it gets mileage.

Being in the right place at the right time seems to be one of the most important factors in feature adoption around here.

This implementation now having support from all the major ports that run Boom stuff (and even some that don't!) is great news, and means that wads can start taking advantage of it in the same way that they've done for MBF's expansions. The informal "boom-plus" standard has grown once again :)

1 hour ago, Altazimuth said:

So uhh... I added most of this stuff to EE. New frames, states, mobjs. The only thing I've not done is put dummy mobjs in the range of dehnum 146-150. In the UMAPINFO fork these're MT_GIBDTH, MT_STEALTHBABY, MT_STEALTHVILE, MT_STEALTHBRUISER, and MT_STEALTHHEAD—all of which are just dummies. Retro, on the other hand, has MT_GIBDTH, BT_BLUEBLOOD, MT_GREENBLOOD, MT_FUZZYBLOOD, and MT_TRAIL here. Two cents from relevant parties on what to do here would be appreciated.

 

Outside of that if there's any other new deh features I haven't added them. I'd like to make suggestions on what I feel is sane but I'd like to have port authors on-board with just the basics first.

The best would be to mark these mobjs as "RESERVED -- DO NOT USE" and disallow dehacked touching them.

the GIBDTH is, as said before, an Easter egg that has been disabled in this fork; stealth monsters have been implemented in PrBoom for all of two days way back then and have been disabled ever since; the Doom Retro blood things are there to support its colored blood feature, and I suppose MT_TRAIL is a rocket trail actor.

None of these things are part of the discussed standard, and I do not think Eternity should try to implement either.

8 hours ago, Ferk said:

I don't know if devs would consider it worth of standard (I think the way it's done in EE and GZDoom is much more complex and featureful), but I can make a PR for PrBoom+ this weekend for "Dropped Item" to initiate discussion on that.

I went ahead and made a PR for PrBoom+ about the "Dropped Item" field for Thing the same way Doom Retro did:

    https://github.com/coelckers/prboom-plus/pull/104

Just wondering if that would be an acceptable way for other ports to support items (or things) being dropped by mobjs on death.

A_Spawn was technically a good universal way to specify any drop, and without being bound to any specific frame during the death animation.

10 hours ago, Ferk said:

Just wondering if that would be an acceptable way for other ports to support items (or things) being dropped by mobjs on death.

Let's say I would accept it for Crispy. *hint*

It shouldn't be too hard to tie it to ZDoom's DropItem list in GZDoom.

31 minutes ago, fabian said:

Let's say I would accept it for Crispy. *hint*

Submitted :)

Hmm.. it might also be interesting to de-hardcode things like these which Graf had actually already done in a branch from prboom-plus:

   https://github.com/coelckers/prboom-plus/commit/f5f8dea10bbb6ae20796c4459b39e68e8f4b6586

Looks like he was already planning a decorate-lite. I'd say it might make more sense to expand DEH than to make yet another syntax. Specially when you can have things like deh9000 or ACTORDEF script "compile" into deh for the syntax sugar and flexibility.

14 minutes ago, Ferk said:

Submitted :)

Impressive, thank you! 

12 hours ago, esselfortium said:

Being in the right place at the right time seems to be one of the most important factors in feature adoption around here.

Agreed. It is interesting/funny how this was already implemented all the way back in 2017, but slowly but surely it has come around to this place and this time. There is a ton of potential for this stuff, though i am not quite understanding how things like Black Ops/Smoothed Weapons were achieved, since you obviously need some sort of editor that supports al these extended features?

12 hours ago, esselfortium said:

This implementation now having support from all the major ports that run Boom stuff (and even some that don't!) is great news, and means that wads can start taking advantage of it in the same way that they've done for MBF's expansions. The informal "boom-plus" standard has grown once again :)

Boom Plus has a nice ring to it :) If that is going to be the defacto name sake for ports that support these extended features (Which is Retro, Crispy, PrBoom Plus UM, GZDoom and Eternity) then i have to ask: When can this be set in stone? I realize that you need most of the authors involved to be in agreement with this.

1 hour ago, Ferk said:

Hmm.. it might also be interesting to de-hardcode things like these which Graf had actually already done in a branch from prboom-plus:

   https://github.com/coelckers/prboom-plus/commit/f5f8dea10bbb6ae20796c4459b39e68e8f4b6586

I went ahead and made another PR for crispy with that as a proposal... opinions?

https://github.com/fabiangreffrath/crispy-doom/pull/636

I'm wary of adding additional flags for the "Bits" field though... since changing the Bits field would be hairy compatibility wise, and I think other ports have done it in possibly conflicting ways.

Some time ago when I added the other flags to the PrBoom libretro core I added a Bits2, but it still internally is just extending the flags and it might not be ideal, also "Bits2" would conflict with EE and probably others:

https://github.com/libretro/libretro-prboom/blob/9cbbebe4023230881e3c22dde25c251deacfdbbf/doc/dehacked.txt#L165-L177

• Drop items: Easy (I have code that should do this, though it's uncommitted).
• The stuff you just made the PR to Crispy for: A bit of a faff but should be doable I guess?
• More flags: NOPE. NOPE. NAH. NUH-UH. HERE BE CLASHES.

If we wanna do bits/flags we're gonna have to think of a way to do it that's somewhat abstracted probably, because I think even within the first set of flags there's clashes between ports. Hell, EE has exposed "Bits2" to deh since 1999.

Any additional flags field should be abstracted, for sure, since EE and GZDoom already have their own hugely expanded flags sets that these would have to get internally mapped to. Having it be a new extradeh-specific field that just provides a text-based list would also be useful in the sense that it wouldn't run out of room after an arbitrary point. It would be really nice to have additional flags, though! Doom has so much hardcoded stuff that makes deh work extra fiddly and restrictive, and new flags could do a lot to mitigate that.