The Force Engine Version 1.0 Release

Greatly appreciate your continued work on this.

11 hours ago, lucius said:

Version 1.09.4 has been released, read the blog post at https://theforceengine.github.io/2023/07/31/Version-1-09-04.html.

 

Various different shots with bloom with different settings.

 

8-bit versus 8-bit interpolated comparisons. The 8-bit interpolated mode removes most of the banding while still using the color map - so colormap tricks used to emulate colored lighting and fog still work.

 

And last but not least are the Smooth Vue animation feature and Closed Captions as well, see the blog post for animations and also for more and better quality images: https://theforceengine.github.io/2023/07/31/Version-1-09-04.html.

Keep up the good work!

It's mindblowing to see more new features that add even more value to the gameplay, apart from the main planned ones. I especially dig the accesibility-oriented Closed Captions feature, which highlights ImGui's potential use case.

Thanks everyone. :)

There has been a small point release (1.09.41) that fixes a few issues with version 1.09.4. You can find it at the same download link: https://theforceengine.github.io/downloads.html

Now that this release is out of the way, I'm focusing on the next release - version 1.09.5 (see the blog post for more details about upcoming plans).

1 hour ago, lucius said:

As many of you have seen, Nightdive has announced their Dark Forces Remaster. They have partnered with me to help them in a technical advisory role to help with the project, which includes exchanging information and providing some TFE code. So I have written a FAQ to answer likely questions.

 

https://theforceengine.github.io/2023/08/22/Nightdive-Remaster-FAQ.html

Congrats on your involvement!

It is always nice to see game studios collaborate with fan projects rather than trying to have them shut down.

2 hours ago, Rudolph said:

It is always nice to see game studios collaborate with fan projects rather than trying to have them shut down.

Every so often, common sense prevails.  This must be one of those rare times...

9 hours ago, lucius said:

As many of you have seen, Nightdive has announced their Dark Forces Remaster. They have partnered with me to help them in a technical advisory role to help with the project, which includes exchanging information and providing some TFE code. So I have written a FAQ to answer likely questions.

 

https://theforceengine.github.io/2023/08/22/Nightdive-Remaster-FAQ.html

Congratulations.

Nightdive seems to be genuinely interested in working with fan communities when possible. Unfortunately, that isn't always possible due to the mandates of the client or publisher.

The SDKs for XBox / PS / Switch are proprietary and *cannot* be linked against GPL code. This is ultimately why Nightdive have to use their own engine, Kex. In the case of Quake I & II, even though the source is available, iD, as owners of the copyright, can licence a closed-source version of the game. For Quake II, the "game" is separate from the engine and communicates via an API, which is why some code is available for Quake II Enhanced but this is a special case unique to Quake II.

As for The Force Engine, it's probably a matter of sharing "knowledge" (implementation details), I can't imagine they can include GPL code unless they intend to open-source the entire game. Either way, it's good news for Dark Forces fans. Even with a remaster available, it will be a proprietary engine that won't get ported any further. TFE will remain the only solution for ongoing improvement and porting.

4 hours ago, Kroc said:

As for The Force Engine, it's probably a matter of sharing "knowledge" (implementation details), I can't imagine they can include GPL code unless they intend to open-source the entire game. Either way, it's good news for Dark Forces fans. Even with a remaster available, it will be a proprietary engine that won't get ported any further. TFE will remain the only solution for ongoing improvement and porting.

As stated in the FAQ, "Only code reverse-engineered or written by [lucius] has been shared with Night Dive," which makes lucius owner of the copyright for that code and able to relicense it as needed.

2 hours ago, Shepardus said:

As stated in the FAQ, "Only code reverse-engineered or written by [lucius] has been shared with Night Dive," which makes lucius owner of the copyright for that code and able to relicense it as needed.

That is correct and is the approach I took. I put this in the FAQ in case anyone had concerns about using contributor work or assets, which would obviously require permission from said contributors.

Apologies for going a bit off-topic, but... @lucius, do you ever intend to revisit your Daggerfall project sometime in the future? While I appreciate what the Daggerfall Unity project is doing, there's no surpassing of how accurately you reconstructed the original Dark Forces renderer and generally kept close to the source material's inner workings. XnGine is a very nice engine of its time, and it's a shame that Bethesda lost the sources and no real effort is made to revive it.

I have considered an "XnGine" project - similar to TFE. The goal would be to support all XnGine games (over time), rather than just Daggerfall. That would allow me to start with something simpler, like FutureShock (which I also really like) rather than jumping straight into maximum complexity.

That said, it is currently just an idea. My focus will remain on TFE for quite some time yet, since I still need to finish Outlaws support and version 2.0. I don't plan on taking on any large projects until TFE version 2.0 is out and development is stable (aka in a maintainable state). It will depend on how well TFE is doing, when I want to jump onto another large project, and whether people in general would even be interested (especially with Daggerfall Unity). Though I think there would still be a fair amount of interest in other games, like Future Shock, so that last point is probably less of a concern.

Ultimately I have learned to keep scope manageable for these types of projects and avoid taking on too many projects at once, or they don't get done (which is a really common issue).

52 minutes ago, lucius said:

I have considered an "XnGine" project - similar to TFE. The goal would be to support all XnGine games (over time), rather than just Daggerfall. That would allow me to start with something simpler, like FutureShock (which I also really like) rather than jumping straight into maximum complexity.

I absolutely love Future Shock, amazingly well ahead of its time, and undeservedly overshadowed by its contemporaries. The XnGine revival that you describe would be a wonderful thing.

Wishing you the best of luck in bringing TFE to its full potential! Cheers!

This is not aimed at lucius themselves, but I wish there were a "ScummVM for FPSes". We have lots of separate projects to painstakingly reverse engineer various FPSes but all these projects struggle in their own ways and a lot of effort must go into building the basics of input & rendering when that could be reused. I wish we could combine these things so that when the engine overall improves or gets ported, lots of games automatically gain the benefits.

3 hours ago, Kroc said:

This is not aimed at lucius themselves, but I wish there were a "ScummVM for FPSes". We have lots of separate projects to painstakingly reverse engineer various FPSes but all these projects struggle in their own ways and a lot of effort must go into building the basics of input & rendering when that could be reused. I wish we could combine these things so that when the engine overall improves or gets ported, lots of games automatically gain the benefits.

 That was the ultimate goal of the rewrite of Doomsday. Sadly it never came to fruition. 

9 hours ago, Kroc said:

This is not aimed at lucius themselves, but I wish there were a "ScummVM for FPSes". We have lots of separate projects to painstakingly reverse engineer various FPSes but all these projects struggle in their own ways and a lot of effort must go into building the basics of input & rendering when that could be reused. I wish we could combine these things so that when the engine overall improves or gets ported, lots of games automatically gain the benefits.

Perhaps not even close enough, but I just remembered about Doomity, which might still be worth checking.

On 8/25/2023 at 5:19 AM, Kroc said:

This is not aimed at lucius themselves, but I wish there were a "ScummVM for FPSes". We have lots of separate projects to painstakingly reverse engineer various FPSes but all these projects struggle in their own ways and a lot of effort must go into building the basics of input & rendering when that could be reused. I wish we could combine these things so that when the engine overall improves or gets ported, lots of games automatically gain the benefits. 

Having spent a lot of time looking at old FPS engines and more recently some time looking at ScummVM and SCUMM specifically, I think the biggest differences are that adventure games have way simpler technical demands that lend themselves to a common "everything engine". Scumm was definitely upgraded over the years (1987-1997) to do many new things, and even fully rewritten once, but there weren't any real "paradigm shifts" in hardware capabilities that changed everything the way Wolf3D -> Doom -> Quake -> onward did. The latter technical leaps meant massive changes to data formats, simulation, rendering, networking, the works. As plenty of projects have shown you can definitely "emulate" Doom in Quake and even kinda vice versa, but FPS are so dependent on things like how the lower level controls and collision feel, how levels are constructed, and how game logic is defined. At the point that you're writing entirely different code paths to cover those differences, you might as well have different engines with a few shared libraries, which is what we have anyway across all the different source ports. I don't get the impression that duplication of effort is a major problem there. It's understandable to wonder about though!

On 8/27/2023 at 9:44 AM, JPL said:

 

Having spent a lot of time looking at old FPS engines and more recently some time looking at ScummVM and SCUMM specifically, I think the biggest differences are that adventure games have way simpler technical demands that lend themselves to a common "everything engine". Scumm was definitely upgraded over the years (1987-1997) to do many new things, and even fully rewritten once, but there weren't any real "paradigm shifts" in hardware capabilities that changed everything the way Wolf3D -> Doom -> Quake -> onward did. The latter technical leaps meant massive changes to data formats, simulation, rendering, networking, the works. As plenty of projects have shown you can definitely "emulate" Doom in Quake and even kinda vice versa, but FPS are so dependent on things like how the lower level controls and collision feel, how levels are constructed, and how game logic is defined. At the point that you're writing entirely different code paths to cover those differences, you might as well have different engines with a few shared libraries, which is what we have anyway across all the different source ports. I don't get the impression that duplication of effort is a major problem there. It's understandable to wonder about though!

I have tried the "all in one" engine thing myself - and it is a bit surprising how little can be shared between similar FPS games. Obviously, if they use the same engine, that helps a lot (see Doom source ports that support Hexen, Heretic, etc. or even TFE in the future with Dark Forces and Outlaws). But beyond that, if you want to accurately represent the game and its quirks, the best you can do is share the framework - like Nightdive does with Kex.

And really, if all you are sharing is the framework it seems you are better off with separate projects that share libraries or code so you can just take what you need and avoid bloat. Often, based on the capabilities of the game, you will want to customize your application anyway.

That is how I feel about it now anyway, though my opinions a decade ago were obviously different. :)-

It's important to note as well that ScummVM is not an "everything engine", it's a collection of engines bundled together. It's kind of like MAME or RetroArch in that respect.

Can something like that be done for shooters? Certainly! But it wouldn't look like GZDoom which has one unified engine for all games; it'd be more like a common front-end with multiple back-ends. So upgrades in the front-end would be shared by everything, but each back-end would need its own maintenance. ScummVM's got a huge team to handle all this; nothing comparable with our source ports where you usually have one or two main developers with maybe half-a-dozen frequent contributors if you're lucky.

A common frontend, ala MAME or RetroArch is still a valuable thing because it ensures game specifics are separate from hardware. We have a port of Doom to PSP, but will there ever be one for TFE? If the front end were all shared then this would problem would be moot. Good point regarding how different the "feel" of FPSes can be and that this wouldn't translate to shared code. That said, data-driven design could be used to parametrise behaviour much as Doom has been redefined via PWADs and Dehacked.

This is all off topic however, so I'll leave it at that.

I have not posted in this thread for some time, but TFE has continued development.

I posted a 2023 Retrospective and 2024 Plans post on the blog - https://theforceengine.github.io/2024/01/01/2023-Retrospective-2024-Plans.html - that summarizes the year and plans for 2024.

The TFE built-in level editor has also made a lot of progress, as seen in these videos I recently posted:

1 hour ago, lucius said:

I have not posted in this thread for some time, but TFE has continued development.

 

I posted a 2023 Retrospective and 2024 Plans post on the blog - https://theforceengine.github.io/2024/01/01/2023-Retrospective-2024-Plans.html - that summarizes the year and plans for 2024.

 

The TFE built-in level editor has also made a lot of progress, as seen in these videos I recently posted:

ooh, this editor looks really nice...you're doing god's work here, heh

btw, i have a question. knowing star wars fans, i think that something that could probably cause a massive explosion in modding for dark forces is the ability to create custom enemies and weapons a la dehacked and its various extensions for doom, that way you could heavily expand the game's bestiary to include all sorts of stuff from the star wars universe. however, i've been told by someone far more knowledgeable with all of this that the way the force engine handles states is uh...messy, to say the least. so, do you think there's any possibility that something of that sort could happen with this? perfectly understandable if it's not

I think you meant the way Dark Forces handles states... but yeah, that is an issue. The plan is a two-pronged approach:

* Expose the hardcoded data as external text files and allow those to be overridden.

   This will already allow a lot of options given the modular way that Dark Forces handles "logics" (behaviors). This will also allow for weapons to be changed, items to be changed, etc.

* Support scripting and expose the ability to add new behaviors through scripts as well as using the built-in behaviors. Scripting support is already in the engine to some degree, but it isn't exposed yet to modders.

5 hours ago, lucius said:

I think you meant the way Dark Forces handles states... but yeah, that is an issue. The plan is a two-pronged approach:

 

* Expose the hardcoded data as external text files and allow those to be overridden.

   This will already allow a lot of options given the modular way that Dark Forces handles "logics" (behaviors). This will also allow for weapons to be changed, items to be changed, etc.

* Support scripting and expose the ability to add new behaviors through scripts as well as using the built-in behaviors. Scripting support is already in the engine to some degree, but it isn't exposed yet to modders.

ah, yeah, sorry. i meant the jedi engine, not the force engine. my bad.

either way, it's nice that that's in the works. it'd be cool to see a lot more maps coming out for this game, cuz it has tons of modding potential :)

Digital Foundry has released their in-depth review about the Dark Forces remaster:

I wonder, that they prefer the remaster over TFE, but I think this could be because the remaster brings new/updated assets, as textures etc.?

I would like to know how the MIDI soundtrack is handled in TFE. Is it like in the original DOS version under DOSBox or was there a similar approach taken like in the remaster?

Also, are the micro stutter also present in TFE, based on tickrates?

In general I would have liked that Digital Foundry had showed more comparisons with TFE.

I know, I could launch TFE myself to find answers to my questions, but I thought it would be also nice to give new life to this thread. Also I have not so much time to turn on my PC currently.

Anyway congrats for this great source port, the collaboration with Nightdive and for the great work on a new editor. I was always sad to see, when modding for an old game became almost forgotten. I hope the old websites hosting Dark Forces mods get never lost.

On 2/26/2024 at 11:39 PM, Kyle07 said:

Digital Foundry has released their in-depth review about the Dark Forces remaster:

 

I wonder, that they prefer the remaster over TFE, but I think this could be because the remaster brings new/updated assets, as textures etc.?

 

I would like to know how the MIDI soundtrack is handled in TFE. Is it like in the original DOS version under DOSBox or was there a similar approach taken like in the remaster?

 

Also, are the micro stutter also present in TFE, based on tickrates?

 

In general I would have liked that Digital Foundry had showed more comparisons with TFE.

 

I know, I could launch TFE myself to find answers to my questions, but I thought it would be also nice to give new life to this thread. Also I have not so much time to turn on my PC currently.

 

Anyway congrats for this great source port, the collaboration with Nightdive and for the great work on a new editor. I was always sad to see, when modding for an old game became almost forgotten. I hope the old websites hosting Dark Forces mods get never lost.

(I removed the video embed from the quote to save space)

* Midi is handled via iMuse, reverse-engineered from the original. The Remaster iMuse code is actually from TFE (with permission/agreement), but they had some initial issues because not all of the code was properly brought over/merged into their code base. Those issues have been fixed after launch though. The Remaster uses a different library for low-level OPL emulation, but the same "driver" level code (again reverse-engineered from the original and used with permission). They use a different default sound font as well. So the Remaster and TFE will sound slightly different, but they both use iMuse.

* The microstutter based on tickrates also exists in TFE, though it doesn't seem to manifest as strongly (probably differences in the way threading/synchronization is handled between projects).

* Digital Foundry did a comparison in a previous video, unfortunately, it wasn't very thorough or up to date.

Version 1.10.000 was finally released. It has been a long time since the previous release, so there are a large number of changes - see https://theforceengine.github.io/downloads.html for the list.

• Remaster HD Asset support, use the new Enhancements menu to gain access.
• MOD_CONF.txt support to enable TFE setting overrides per-mod and override which assets are allowed to be replaced by HD assets.
• Many bug fixes - crashes, alpha-blending and sorting fixes, anisotropic filtering fixes, etc.
• An optional "solid wall" fix for moving sectors (so you fall out of them when jumping).
• Option to be able to step-up on second altitude stairs/steps like regular steps, making 3DO based stairs much nicer to navigate - such as in Havkov 2024.
• Quality of life improvements, such as new cheats, controller improvements, crouch toggle option, and several Accessibility improvements.

Other notes

• This release does not include the Level Editor, that is the focus of the next major release.
• It does not include Outlaws enhancements yet, those will happen after the Level Editor has been released.
• Except for hot fixes and major issues, my focus will be on the Level Editor alpha release until it is finished ("soon").