Raytracing in GZDoom possible?

On 5/14/2022 at 8:11 AM, Edward850 said:

are you prepared to pay someone else to do the work for you?

Who, and how much?

On 5/16/2022 at 9:20 AM, Graf Zahl said:

It is a well known fact that the Doom engine is not good at handling wide open maps. The Build engine isn't either. It's a testament to modern CPU performance that we can get away with maps like Frozen Time, BoA or Elementalism these days. To get more juice out of these we'd need some radical changes to how the map needs to be processed - and these cannot be easily grafted onto existing maps and require educating the mappers to be aware of the needed restrictions.

 

The classic BSP approach is right at its limit - current CPU and CPU cache performance won't allow more. We may get 20% more if we could finally ditch OpenGL and fully multithread the renderer, but with a current rate of 15-20% of graphics hardware not capable of using modern APIs and still lots of old dual core CPUs around that's out of the question for the time being.

Someone somewhere should have laughed so hard on that "Testament" to modern CPUs, of course it might be true that modern CPUs could have been much better but that's why developers exist, they do something to make it work fast and flawlessly with contemporary technology.

Nowadays it easily goes beyond million of polygons on the screen without a need in super-duper technology of dinosaur old Doom BSP, especially a type of which is still 2D. Apparently, if the Doom BSP traversal doesn't scale up nicely to the modern world, with is ultra detailed, full 3D huge spaces, then it wasn't intendend to live up to this day.

Let me tell it again from a perspective of a modern mapper, if my map has somewhat 80 000 polygons (pathetic amount even for 2005) and in 3ds max it literally has 1200 FPS under 10-20% GPU load (GTX 1060) and 2% of CPU load (Ryzen 5 1600) without any super culling technology of 60's called "BSP", especially the one used in Doom, which kind of traversal is still 2D, right?

This is simply ridiculous and hugely impractical to suggest people getting beasty computers that intended to play AAA games on ultras just to cope with fucking FrozenT.

I think everybody should stop using this technique and convert to modern polygon based approaches with collision detections, lightmaps and all modern visibility determination methods, I believe you can even simulate Doom sector lighting coming in all flavors with lightmaps, if this aesthetic is needed so bad.

I think only total refactoring from the ground up will take Doom into modern world. I'm not sure what playsim does but if it also doesn't scale up nicely, kill it.

By the way, if FrozenT was converted to a 3D model and placed into a GZDoom level it would only depend on your GPU how many FPS you'd get, I made three screenshots to demonstrate it specially, scale 0.25 (Potato - 774FPS) 1920x1080 native Full HD (209FPS), and scale 2.0 (4K Ultra HD - 70FPS).

See how blazing fast it gets when there is no limiting factors like playsim and bsp? 774 FPS!

It all just needs two things.

1. Organized team work.

2. Effort to simulate all these things that contribute to the feeling of "Doom".

Now, at least, I think you should get rid of old doom bsp code and old culling techniques that slow it all down.

I thought it didn't need to be said that rendering a static model is a far cry from having, you know, a working level, but here we are. Anyway, Helion is that way.

41 minutes ago, Shepardus said:

I thought it didn't need to be said that rendering a static model is a far cry from having, you know, a working level, but here we are. Anyway, Helion is that way.

I didn't know about this port up until now, thanks, I'll give it a check!

Doom, Quake, Duke Nukem 3D all of them don't have a dynamic geometry except for some texture and floor/ceiling height changes, and maybe duke nukem and zdoom having polyobjects that simply rotate it but don't change. Can't it be done like if for example we look at an elevator sector that changes floor height fro and about simply subtracting the resulting floor height from initial and then creating a big model of that elevator that moves up and down.

Well it all depends what is the direction gzdoom team wants to reach and the complexity of this job. Same thing for helion source port for an eventual ACS implementation. If author is interested or wants to keep the port on a "vanilla side".

Of course those are open source projects and everyone is free to fork them and do whatever they want, or help with contributions.

IIRC selaco team did some modifications on the rendering part of the gzdoom engine. Once selaco is out they will release their fork since they have to respect the GPL license. 

I think a lot of people are missing the fact that there's a difference between theoretically possible and then what it takes to move a code base with 25 years of work on it to such a state.

Unlike Helion, GZDoom has to support the complete feature set of current GZDoom. That includes dynamic lights, portals, everything boom supports and everything zdoom supports. It even needs to take the build engine into consideration, since it shares render backend with Raze. Its users are also on a wide variety of hardware spread over 3 rendering technologies: OpenGL, OpenGL ES and Vulkan.

GZDoom is also a mature product. Unlike a new source port that sets no expectations from its current users (since it has none), GZDoom's users are used to a polished finished product. If it broke half the maps it used to be able to run out there, but now technically ran everything 5 times as fast, I absolutely guarantee you that there would be more people unhappy than happy.

Will GZDoom ever go "BSP less"? Probably not to the degree Helion does it, but maybe some day it will have something that lets it draws things more efficiently. It would be much easier for us do that with the Vulkan backend, since it does things more compatible for such approaches, but how many people on older computers do you think will celebrate if we removed all OpenGL support in the process?

9 minutes ago, dpJudas said:

I think a lot of people are missing the fact that there's a difference between theoretically possible and then what it takes to move a code base with 25 years of work on it to such a state.

 

Unlike Helion, GZDoom has to support the complete feature set of current GZDoom. That includes dynamic lights, portals, everything boom supports and everything zdoom supports. It even needs to take the build engine into consideration, since it shares render backend with Raze. Its users are also on a wide variety of hardware spread over 3 rendering technologies: OpenGL, OpenGL ES and Vulkan.

 

GZDoom is also a mature product. Unlike a new source port that sets no expectations from its current users (since it has none), GZDoom's users are used to a polished finished product. If it broke half the maps it used to be able to run out there, but now technically ran everything 5 times as fast, I absolutely guarantee you that there would be more people unhappy than happy.

 

Will GZDoom ever go "BSP less"? Probably not to the degree Helion does it, but maybe some day it will have something that lets it draws things more efficiently. It would be much easier for us do that with the Vulkan backend, since it does things more compatible for such approaches, but how many people on older computers do you think will celebrate if we removed all OpenGL support in the process?

I can't agree on GZDoom "polishness". Portals and other stuff are actually half-done workarounds to overcome limits imposed by id Tech 1. The line portals still break the experience sometimes as some monsters can't see through them, they are not linked like stacked sectors and they're slow sometimes. The fact it spans across so many APIs is great though, cause that makes it playable on mobile devices.

You still can't have different closed sectors stacked at the top of each other so it doesn't cross each other linedefs like it was done in Build btw, it becomes of a huge inconvenience and slow down to edit structures with 3D floors that way, cause once drawn it fuses together and each newly introduced linedef needs to be edited manually after if one needed to change the first "stage" geometry in the bottom. I once addressed it to Graf and he said it's something that needs to be done on the map editor side but even if multilayer editing was ever introduced to doom builder, the upper geometry consisting of 3D floors would have still becoming dependend on geometry located below and splitted into pieces.

What's about some vanilla maps like FrozenT and Planisphere 2, Helion has smooth gameplay across the whole time except for some strange microstutters but the overall framerate locked hard on 75FPS (my monitor limitations) all on OpenGL which old computers love. The CPU and GPU load were staying stable low and multithreaded during all that time which is cool, while the experience is so close to vanilla, it's a wonder.

To me Helion just proved that the Doom game can be simulated really well without its old quirks and run relatively smooth across the majority of modern computers, even the older ones.

Here is also a thing that all current ports are unable to do: transfer doom experience over highly detailed true 3D worlds without ugly squareness and blockiness imposed by current editing techniques which is still a 2D drawing. Helion just proved it's possible and this is the future. That way doom editing will get more 3D, like if you could have more freedom to those drawn 2D sectors, stack them like if they were actual geometry, rotate them, extrude, bevel and so on, maybe even AI doing some work for mapper, importing 3D objects with true or simplified collisions so the room for an artistic vision increase dramatically.

Tell me why I need zdoom and boom stuff then?

9 minutes ago, Vadim said:

Tell me why I need zdoom and boom stuff then?

Compatibility.

TBH you are looking at the wrong engine if you don't want blockiness and squareness.

And they run doom. There is no need to create complex worlds: there are better tools for that like Unity or UE.

And might be not of interest among source ports developers because they like that, in the end, it's a passion they have.

and

Quote

To me Helion just proved that the Doom game can be simulated really well without its old quirks and run relatively smooth across the majority of modern computers, even the older ones.

as dpJudas said is because Helion does not have a 25 years codebase, but it's completely new without all the complexity GZDoom has.

Removing bsp from GZDoom is more complex and requires a lot of work ( if that's possible).

1 hour ago, Vadim said:

Tell me why I need zdoom and boom stuff then?

Indeed. If you find GZDoom so unpolished and Helion checks all the boxes of your needs, then why do you complain about GZDoom? Just launch Helion already!

I don't want to shit on Helion, because it is a very cool project, but there's just a very big difference between supporting Doom's original features and then what boom and zdoom added over the years. As a simple example, Helion can render all sprites in one pass because there's only one type of sprite in vanilla. GZDoom sprites are multiple types and can have different blend modes. Drawing the entire level as a single mesh is a lot easier when you have no dynamic lights and your level isn't potentially full of high polycount 3d models as some GZDoom maps have. And what about the portals? You can see portals in portals in portals - drawing the level mesh once isn't going to be enough. Then there are the camera textures...

Is it theoretically impossible to fix all those things? No, it is not. But don't pretend it is a trivial matter just because Helion showed they could do it for the subset of maps they support.

5 hours ago, dpJudas said:

Indeed. If you find GZDoom so unpolished and Helion checks all the boxes of your needs, then why do you complain about GZDoom? Just launch Helion already!

 

I don't want to shit on Helion, because it is a very cool project, but there's just a very big difference between supporting Doom's original features and then what boom and zdoom added over the years.

I guess as occasional playtest dummy for Helion (Which isn't a port!) i should stress that Helion isn't mean't to compete with GZDoom. GZDoom supports a shitton of things mean't for modders and creators alike. Helion, by contrast, is just an engine that wants to render Doom really really fast.

It is more comparable in philosophy to Ironwail, the Quake port that primarily focuses on speed and renderer optimizations.

Helion's idea is also a dangerous one to execute: Because it does not use Doom's renderloop (and its inherent limitations), its behavior needs to be exactly the same to the original or it will cause errors. Thus what Helion currently does is the result of several years of development before @hobomaster22 and Water went public with it.

It does not support GZDoom's most advanced (and heavy!) maps because those rely on the feature set that GZDoom provides. So anything with heavy modelling or post processing, that will likely eat Helion.

But in the tier below that, Boom-compatibility and what not, that's where Helion shines, albeit with less visual fidelity.

FWIW: The way Helion does it (Forfeiting Doom's rendering paradigm altogether) is likely how i would go with this aswell if my goal was to have the best performance out there. For most users, GZDoom is perfectly adequate because it simply brute forces through most WADS. But for those WADS that hit Doom's rendering limits, Helion is there (And it shows. The performance uptick is just ridiculous at this point)

TLDR: GZDoom and Helion are two different product targetting two different goals. They are not and should not be compared to eachother.
 

2 hours ago, dpJudas said:

Indeed. If you find GZDoom so unpolished and Helion checks all the boxes of your needs, then why do you complain about GZDoom? Just launch Helion already!

 

I don't want to shit on Helion, because it is a very cool project, but there's just a very big difference between supporting Doom's original features and then what boom and zdoom added over the years. As a simple example, Helion can render all sprites in one pass because there's only one type of sprite in vanilla. GZDoom sprites are multiple types and can have different blend modes. Drawing the entire level as a single mesh is a lot easier when you have no dynamic lights and your level isn't potentially full of high polycount 3d models as some GZDoom maps have. And what about the portals? You can see portals in portals in portals - drawing the level mesh once isn't going to be enough. Then there are the camera textures...

 

Is it theoretically impossible to fix all those things? No, it is not. But don't pretend it is a trivial matter just because Helion showed they could do it for the subset of maps they support.

2 hours ago, dpJudas said:

Indeed. If you find GZDoom so unpolished and Helion checks all the boxes of your needs, then why do you complain about GZDoom? Just launch Helion already!

 

I don't want to shit on Helion, because it is a very cool project, but there's just a very big difference between supporting Doom's original features and then what boom and zdoom added over the years. As a simple example, Helion can render all sprites in one pass because there's only one type of sprite in vanilla. GZDoom sprites are multiple types and can have different blend modes. Drawing the entire level as a single mesh is a lot easier when you have no dynamic lights and your level isn't potentially full of high polycount 3d models as some GZDoom maps have. And what about the portals? You can see portals in portals in portals - drawing the level mesh once isn't going to be enough. Then there are the camera textures... 

  

Is it theoretically impossible to fix all those things? No, it is not. But don't pretend it is a trivial matter just because Helion showed they could do it for the subset of maps they support. 

Helion supports almost all Boom features (except for two or three exotic ones) and we do support a subset of ZDoom features. It supports full 3D checks. You can stack monsters on top of each other and crush them, stack them on a lift, we have rocket jumping etc. But I don't think this entirely detracts away from your point, I just wanted to make that part clear.

The base of Helion was built using some of the ZDoom structure for decorate, etc. it would be an insane amount of work to get anywhere near feature parity with GZDoom. Our main focus is Boom/MBF21 and Vanilla since it covers the vast majority of maps anyway and is currently a reasonable feature set for Helion. Mappers make exceptionally large Boom compatible maps all the time that choke on modern hardware in all ports and Helion is designed to solve this problem. This is the reason we started with a brand new code base as we didn't want to be tied down to archaic bloated code (and we also wanted to use a managed language). Static rendering or not, I don't think most realize what an insane lift it was to get a game like Doom functional from scratch and reasonably support what players and mappers expect from the game. The amount of quirks and bugs as features is the stuff of nightmares.

I'm not convinced adding dynamic lights 3d models would bring rendering to it's knees since none of these things are the bottleneck. Obviously, adding more complex rendering will drop the performance but it's not a performance killer like the BSP tree. Modern video games do this all the time. Even something like Zwift is capable of rendering more complex scenes on tablets, old apple TVs and apparently will even run on my iPhone.

45 minutes ago, dpJudas said:

Indeed. If you find GZDoom so unpolished and Helion checks all the boxes of your needs, then why do you complain about GZDoom? Just launch Helion already!

I've been complaining about GZDoom, I didn't even know Helion existed before Shepardus told me about it yesterday. I've been doing a custom project for GZDoom for years and when I stumbled across a thing that bottlenecks CPU (which is BSP and clipping), I started asking on letting me remove it for certain maps or zones because modern GPUs will eat it for breakfast but seems like all my questions are simply getting ignored or replied in a manner to shut me up (it's not gonna work attitude). They won't even allow me to try it!

48 minutes ago, dpJudas said:

Drawing the entire level as a single mesh is a lot easier when you have no dynamic lights and your level isn't potentially full of high polycount 3d models as some GZDoom maps have. And what about the portals? You can see portals in portals in portals - drawing the level mesh once isn't going to be enough. Then there are the camera textures...

My levels have huge loads of 3d models and some do have stacked sectors, line portals and guess what happens? BSP and clipping make it magnitudes worse, especially when you look below or above because the clipper is 2D and higher or lower you look the broader frustrum becomes and more geometry that old clipper needs to process, the more apparent slowdown becomes, on a modern desktop system.

I had another approach on mind btw, creating a huge 3D blockmap for a level and prerendering 360 panoramas then cull geometry, 3D actors mesh and some level decorations like torches in LZDoom manner. Then showing these panoramas instead of culled stuff, even replacing skybox with it. That could speed everything up dramatically on every system... if not the question on rendering monsters occured. I could only suggest having zbuffer rendered into 360 panoramas to to define whether monster in the distance rendered or not (if obstructed), maybe those zbuffer values could be converted to map coordinates somehow? Like monster actor is closer than a value converted from prerendered zbuffer - then render the monster, if a zbuffer value is less - then don't render it?

48 minutes ago, Vadim said:

I've been complaining about GZDoom, I didn't even know Helion existed before Shepardus told me about it yesterday. I've been doing a custom project for GZDoom for years and when I stumbled across a thing that bottlenecks CPU (which is BSP and clipping), I started asking on letting me remove it for certain maps or zones because modern GPUs will eat it for breakfast but seems like all my questions are simply getting ignored or replied in a manner to shut me up (it's not gonna work attitude). They won't even allow me to try it!

Say what? I can't really remember any PR that you sent that got rejected.

Here's the thing. Nobody is against improving the performance of GZDoom. Nobody. What we are against is someone telling us to do it and pretend it is such a trivial thing to do, then act like we are the assholes when we choose not to do it for you for free. Keep in mind any reply you have gotten has always been given in the context of modifying current GZDoom's code to do what you request. Thus Helion is not a proof of anything in that regard, since it explicitly choose not to do that.

Totally agree with dpJudas

2 hours ago, Vadim said:

They won't even allow me to try it!

Do you go by a different name on GitHub? I don't see any rejected pull requests by your name. https://github.com/ZDoom/gzdoom/pulls?q=is%3Apr+is%3Aclosed

1 hour ago, dpJudas said:

Say what? I can't really remember any PR that you sent that got rejected.

 

Here's the thing. Nobody is against improving the performance of GZDoom. Nobody. What we are against is someone telling us to do it and pretend it is such a trivial thing to do, then act like we are the assholes when we choose not to do it for you for free. Keep in mind any reply you have gotten has always been given in the context of modifying current GZDoom's code to do what you request. Thus Helion is not a proof of anything in that regard, since it explicitly choose not to do that.

It's hard to implement, that's why I don't simply say it in imperative tense but suggesting something that would actually work and it's not just for me, it's for your audience too.

People usually don't care what technology used they want it smooth and especialy they know that doom is an old game that doesn't need a latest kind of hardware to play it nicely. Guess how they will treat you if offered to get a beasty pc to play your wad.

Helion made it clear to me that it's worth to keep certain optimizations in mind as the overall speed this renderer works is faster than all known doom source ports summed up, on OpenGL, without Vulkan and this is what going to potentially expand your audience.

9 minutes ago, Edward850 said:

Do you go by a different name on GitHub? I don't see any rejected pull requests by your name. https://github.com/ZDoom/gzdoom/pulls?q=is%3Apr+is%3Aclosed

I never asked anything on GitHub and been doing so at forum.zdoom.org
Is this my greatest mistake?

11 minutes ago, Vadim said:

I never asked anything on GitHub and been doing so at forum.zdoom.org

Is this my greatest mistake?

What do you mean ask? You said you were being stopped from trying, yet I can't find any code from you. GZDoom is an open source project, you are never stopped from trying anything.

4 minutes ago, Edward850 said:

What do you mean ask? You said you were being stopped from trying, yet I can't find any code from you. GZDoom is an open source project, you are never stopped from trying anything.

I don't code, this is what developers are for.

11 minutes ago, Vadim said:

I don't code, this is what developers are for.

ok Karen

14 minutes ago, Vadim said:

I don't code, this is what developers are for.

You do understand that in saying this, what you've just done is acknowledged that you don't know how to do the things you are demanding other people do, and thus don't understand what you're talking about? Because if you knew how these things worked you would be able to provide code for it, but as you yourself admit you don't know how to code, well...

12 minutes ago, Gez said:

ok Karen

"I told the manager how to fix the problem, but they won't do it! Then I told the manager that I would fix the problem, but they won't even let me try because I don't know how to fix the problem!!!"

Probably making a fork for GZDoom to work with Raytracing APIs would be a better approach than integrating it in the proper GZDoom, as Raytracing is still a niche PC gaming subject that only a few commercial games support it, and very less open-source projects use it.

34 minutes ago, Vadim said:

I don't code, this is what developers are for.

Lets be clear then:

• If you submitted a patch on Git, show us where.
• If you made a suggestion on ZDoom.org, show us where.
• You say you werent allowed to try submitting a feature request. Where weren't you able to? Github, or Zdoom forums?

18 minutes ago, Redneckerz said:

Lets be clear then:

• If you submitted a patch on Git, show us where.
• If you made a suggestion on ZDoom.org, show us where.
• You say you werent allowed to try submitting a feature request. Where weren't you able to? Github, or Zdoom forums?

zdoom forums,  about that distance culling and env map baking technique. I called it nonsense back then because I wanted volumetric spheres with distant geometry baked into their textures, I think having just one is enough.

https://forum.zdoom.org/viewtopic.php?p=1178138#p1178138

36 minutes ago, Edward850 said:

You do understand that in saying this, what you've just done is acknowledged that you don't know how to do the things you are demanding other people do, and thus don't understand what you're talking about? Because if you knew how these things worked you would be able to provide code for it, but as you yourself admit you don't know how to code, well...

Not exactly that I don't know how it works but I can't implement it. By the way, there was another thing I asked a question about whether it's possible to disable that BSP traversal and clipper at all. I know that according to "stat rendertimes" cvar it shows that BSP/Clip takes a long time before it all sent to GPU, thus bottlenecking your graphics card and lowering FPS.

https://forum.zdoom.org/viewtopic.php?t=77089

another link: https://forum.zdoom.org/viewtopic.php?t=70639

Btw, I also suggested on including LOD support for 3D models and it's been quiet for a while.
https://forum.zdoom.org/viewtopic.php?t=76755

17 minutes ago, Vadim said:

Not exactly that I don't know how it works but I can't implement it.

These are one in the same. Either you know how it works it or you don't, and you can't present any code to show for it. After all you have no proof of work, you can't prove these concepts are compatible, and you have no code to show for it.

2 minutes ago, Edward850 said:

These are one in the same. Either you know how it works it it doesn't, and you can't present any code to show for it.

I disagree, shoud I really provide the developers with full proof of concept set then? What can be easier to understand that I simply want to take what LZDoom already does under "Display->OpenGL->Sprites/walls distance culling" and instead of showing empty skybox that remains in the place where geometry is culled, show a pre-rendered frame?

4 hours ago, hobomaster22 said:

I'm not convinced adding dynamic lights 3d models would bring rendering to it's knees since none of these things are the bottleneck. Obviously, adding more complex rendering will drop the performance but it's not a performance killer like the BSP tree. Modern video games do this all the time. Even something like Zwift is capable of rendering more complex scenes on tablets, old apple TVs and apparently will even run on my iPhone.

I didn't mean to imply it would bring a correct implementation to its knees. But currently GZDoom builds a light list for the entire mesh (each line or flat). If the mesh is the entire level, that light list would now be all lights in the map. With that many lights per surface the performance would tank. It is just one small example of how there's a lot of things that needs to be redesigned for it to actually scale.

3 minutes ago, Vadim said:

I disagree, shoud I really provide the developers with full proof of concept set then? What can be easier to understand that I simply want to take what LZDoom already does under "Display->OpenGL->Sprites/walls distance culling" and instead of showing empty skybox that remains in the place where geometry is culled, show a pre-rendered frame?

What is being asked is proof of work. We need code and performance statistics of some kind of test done in GZDoom between its existing implementation and a new one, otherwise all you're doing is coming in and saying "People should code this because I say you should", which is a colossal waste of everyone's time. Nobody likes an ideas guy.

@Vadim:

Here's an offer: If you pay me a year's salary to exclusively work on GZDoom, I am fairly certain that I can get it to a state where those gargantuan Boom maps can be played with at least 150 fps.

But as things stand I have to spend the vast amount of my programming time on financing my life, and no magic wand can do anything to increase the time being spent on an unpaid hobby project.

And rest assured, it does not look different for all other potential contributors.

I hope that is clear enough for you to digest.

3 minutes ago, Edward850 said:

What is being asked is proof of work. We need code and performance statistics of some kind of test done in GZDoom between its existing implementation and a new one, otherwise all you're doing is coming in and saying "People should code this because I say you should", which is a colossal waste of everyone's time. Nobody likes an ideas guy.

Sad, if I could code this would happen without any fuss on forums.

5 minutes ago, Graf Zahl said:

@Vadim:

 

Here's an offer: If you pay me a year's salary to exclusively work on GZDoom, I am fairly certain that I can get it to a state where those gargantuan Boom maps can be played with at least 150 fps.

But as things stand I have to spend the vast amount of my programming time on financing my life, and no magic wand can do anything to increase the time being spent on an unpaid hobby project.

And rest assured, it does not look different for all other potential contributors.

 

I hope that is clear enough for you to digest.

 

 

Now I got it, thanks.