Native Ambient Occlusion (SSAO) shader

Need help editing the game? Check for help here!

Moderators: Darksaber, General_Trageton, Forceflow

Re: Native Ambient Occlusion (SSAO) shader

Turgidson
Cadet 3rd Class
Posts: 38
Joined: Sun Aug 25, 2019 10:02 pm

Post by Turgidson » Wed Oct 02, 2019 10:04 pm

keiranhalcyon7 wrote:
Wed Oct 02, 2019 9:52 pm
The cockpit images up thread look pretty good, but IMO the change in the ISD images is all but unnoticeable due to the game engine's lack of shadows (which are probably a pipe dream to add).
In a way, many pipe dreams have already come true... Who knows? I don't think it's likely to happen, but I wouldn't bet against it. :-D

And if not XWAU, maybe the X-Wing engine rebuilding project...

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Wed Oct 02, 2019 10:05 pm

keiranhalcyon7 wrote:
Wed Oct 02, 2019 9:52 pm
blue_max wrote:
Wed Oct 02, 2019 6:19 pm
The other problem is that close objects will always interact with distant objects anyway because the back faces of closer objects are culled and the depth values for those faces are lost. So I'm not sure if there's anything that can be done about that...
There's always the chance that Jeremy knows where back face culling is done and can disable it with a hook. I imagine the render pipeline portion of the code is rather complex, though.
After thinking about it, disabling backface culling probably won't help in this case; but it will help in other areas -- like computing shadow maps and enabling true 6dof in VR. In any case, a hook like that would be very useful.
keiranhalcyon7 wrote:
Wed Oct 02, 2019 9:52 pm
The appropriate solution would probably be baked AO maps... which may not be totally out of the question for ddraw to load on the side.
Inside ddraw we have no direct knowledge about vertex order, so matching it with an AO map would probably be rather difficult. I may be wrong; but something like that sounds like something that would go in the OPT and probably would require some hooks to send more vertex data to ddraw. Still, if we could figure this out, we could also load per-vertex smooth normals and then we could enable normal mapping and specular highlights...

User avatar
DarHan
Cadet 1st Class
Rebel Alliance
Posts: 160
Joined: Mon Dec 05, 2005 12:01 am
Contact:

Post by DarHan » Wed Oct 02, 2019 10:34 pm

blue_max wrote:
Wed Oct 02, 2019 9:32 pm
I know DarHan posted some config in the Bloom thread; but it didn't work for me when I tried it and I haven't gone back to fiddle with it to make it work.
I actually decided to investigate this more just right now.

With Jeremy’s latest ddraw and the config I posted in the other thread, ReShade’s AO is functional on my computer only if MSAA is disabled in ddraw.cfg

With the bloom shader active, ReShade’s AO doesn’t work regardless of whether MSAA is active.

(my ReShade config have non-standard values for the TextureSearchPaths and EffectSearchPaths parameters, so check those before you try to use the file I posted).
“Red Four, nothing’s gone wrong yet.”
— Hobbie Klivian

User avatar
keiranhalcyon7
Ensign
Posts: 274
Joined: Tue Jan 02, 2018 6:41 am

Post by keiranhalcyon7 » Thu Oct 03, 2019 4:38 am

Turgidson wrote:
Wed Oct 02, 2019 10:04 pm
And if not XWAU, maybe the X-Wing engine rebuilding project...
By which I assume you mean XWVM. Yes, they get a bunch of stuff for minimal programming effort simply by virtue of using a modern game engine/platform.
blue_max wrote:
Wed Oct 02, 2019 10:05 pm
After thinking about it, disabling backface culling probably won't help in this case; but it will help in other areas -- like computing shadow maps and enabling true 6dof in VR. In any case, a hook like that would be very useful.
I think you'd also need frustum culling disabled for those - which I think is to say, disable all the culling.
Inside ddraw we have no direct knowledge about vertex order, so matching it with an AO map would probably be rather difficult. I may be wrong; but something like that sounds like something that would go in the OPT and probably would require some hooks to send more vertex data to ddraw. Still, if we could figure this out, we could also load per-vertex smooth normals and then we could enable normal mapping and specular highlights...
I was thinking more along the lines of AO texture maps with 1:1 correspondence to the opt textures. I thought that was how baked AO works, but now I'm starting to second-guess myself.

Bman
Lieutenant JG
Posts: 702
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Thu Oct 03, 2019 6:30 am

Great Blue_Max. That shadowing effect/correction looks a whole lot better on the ISD pics.
W-I-P: ISD-II, XQ-1 Platform1, (Imp) Escort Carrier, Misc, & TFTC.

User avatar
DTM
Lieutenant Commander
XWAU Member
Posts: 1311
Joined: Tue Apr 22, 2003 11:01 pm
Contact:

Post by DTM » Thu Oct 03, 2019 2:52 pm

Finally I had the chance to play and try your work (bloom shader without reshade): wonderful !!! :god:
Adding native Ambient Occlusion... :tv:

Great work!!!

MTD industries
WIPs: REBCP version 2

User avatar
ual002
Ensign
Galactic Empire
Posts: 488
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Thu Oct 03, 2019 3:21 pm

One of our newer forum posters pointed out that the 32 bit hook is whats causing the directional light to be washed out. Is this correct, at this point I have several test builds and cant narrow down what is causing it? In fact I didn't even notice it until it was pointed out.
For the glory of his majesty Emperor Palpatine! Image Image Image Image

User avatar
DarHan
Cadet 1st Class
Rebel Alliance
Posts: 160
Joined: Mon Dec 05, 2005 12:01 am
Contact:

Post by DarHan » Thu Oct 03, 2019 3:59 pm

I noticed it a while ago but didn't mention it because I thought it was a known issue.

Anyway, slightly off-topic, but on my setup the bloom shader has introduced the gameplay element of “blinded by cockpit sparks” :D
“Red Four, nothing’s gone wrong yet.”
— Hobbie Klivian

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Fri Oct 04, 2019 6:41 am

@DarHan: Thanks for the tip! I'll try again later to see if I can get SSAO working with Reshade as I would like to have some point of reference. Maybe you could post some pictures? I'm specially interested to see if SSAO is only applied to objects at a specific distance. Also: I could probably enable another independent bloom setting for the cockpit sparks... they're probably too bright :D

@keiranhalcyon7: Yes, disabling frustrum culling is probably what I meant; and yes, now that I think about it, maybe I could load pre-baked AO textures after all... and maybe also load other textures to enable normal mapping in the future. Maybe...

@DTM and @Bman: Thanks for your comments guys!

@ual002: I also didn't notice the light problem. Let us know if you figure out anything.

So, I went ahead and split the objects into distant and close objects. What this means in practice is that I can get reasonably decent SSAO when perspective-correction is turned off and foreground objects don't obscure background objects. This makes more sense with pictures. This is the regular SSAO, notice how the foreground X-Wing obscures the background ISD:
ssao6-regular.jpg
And this is with dual SSAO:
ssao6-dual.jpg
I believe the problem is solved differently in regular games -- a threshold is introduced to attenuate the effect with distance. This could also work here; but then we would see SSAO disappearing on distant objects when the foreground overlaps them. With dual SSAO there are two depth buffers and this doesn't happen -- and this is all done in one SSAO pass so the performance penalty is negligible. Unfortunately, in the exterior view the foreground objects are rendered first, so the background still has a hole in it; but I still think it's better this way (the cockpit is rendered last, so dual SSAO works even better there).

I think I can release a preview this weekend. It won't be perfect; but I'll keep polishing it.
You do not have the required permissions to view the files attached to this post.

User avatar
DarHan
Cadet 1st Class
Rebel Alliance
Posts: 160
Joined: Mon Dec 05, 2005 12:01 am
Contact:

Post by DarHan » Fri Oct 04, 2019 3:15 pm

I just took some screenshots of ReShade’s SSAO. They’re visibly far less detailed than your version. I also noticed that the rendering distance is very short, I literally need to fly against the ISD’s hull before AO becomes visible on it.
https://imgur.com/a/fDmE3YW
“Red Four, nothing’s gone wrong yet.”
— Hobbie Klivian

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Fri Oct 04, 2019 4:26 pm

Thank you DarHan! Funny how people never mention this limitation of SSAO: it only works on a relatively narrow set of distances. I guess this isn't much of a problem for regular games; but here we can have detailed stuff both inside the cockpit and a few kilometers away.

I should also take a note that the pictures you posted are much smoother. I know my implementation has banding and a few other issues. I'll be fixing those problems as well.

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Fri Oct 04, 2019 6:36 pm

I think this effect is more noticeable with the 32-bit mode hook:
ssao7.jpg
ssao8.jpg
ssao9.jpg
You do not have the required permissions to view the files attached to this post.

User avatar
DarHan
Cadet 1st Class
Rebel Alliance
Posts: 160
Joined: Mon Dec 05, 2005 12:01 am
Contact:

Post by DarHan » Sat Oct 05, 2019 9:23 pm

Huh. Weird. When I opened this thread last night, only one picture showed.

I really like the shot of the Corvette with a heavy AO effect. The shot of the ISD kinda emphasizes the washed-out ambient lighting mentioned earlier, though.
“Red Four, nothing’s gone wrong yet.”
— Hobbie Klivian

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Sun Oct 06, 2019 1:22 am

Alright, here's the first release of ddraw.dll with SSAO:

https://www.dropbox.com/s/xwjtg8432p972 ... 7.zip?dl=0

I've tested it on my 10-year-old-laptop with Win7 and it seems to work fine, although it sometimes takes a few seconds to fully close. Not sure why (maybe my laptop is too old?)

I've included two configs one for high-quality but lower performance and one for medium quality but high performance (this is the default config file). Performance and quality can be fine-tuned and I've provided a bunch of settings. (These settings may change in the future, though).

Darksaber: I also changed the names of the config files as per your previous requests. Let me know if I missed anything.

The blur filter for SSAO in this release is not yet depth-aware and I think that's causing some halos -- I'll fix that in future releases. However, I'm using Pascal Gilcher's sampling pattern and it's so damn good that a single blur pass yields a very smooth SSAO effect. It's the same pattern used in MXAO, so at least the quality is close in that area. This is how it looks:

ssao10.jpg
Speaking of MXAO, I was finally able to see it in action on my old laptop (thanks DarHan! Your config and the tips helped me get it working). SSAO looks good; but stops working at mid-range and the IL component just looks... well.. weird. Anyway, I'll try to implement that bit in my shader too in a future release.

Lately, I've been thinking about redoing the whole illumination/shading for XWA. I believe the game is still computing per-vertex Gouraud shading and I think maybe that can be improved. So perhaps in the not too distant future we'll simply be able to set the illumination to "None" in the in-game settings and just let ddraw do all the shading... Maybe...
You do not have the required permissions to view the files attached to this post.

User avatar
DarHan
Cadet 1st Class
Rebel Alliance
Posts: 160
Joined: Mon Dec 05, 2005 12:01 am
Contact:

Post by DarHan » Sun Oct 06, 2019 11:53 am

The halo effect does indeed get noticeable depending on circumstances. See around the asteroid here.
https://i.imgur.com/Ct6sUh1.jpg
https://i.imgur.com/c6IynFW.jpg
These were taken using the high quality preset.

I also noticed that space debris feature a dark halo at close range, though I took it more as a reminder to disable space debris. :D

EDIT: visible AO around warhead trails (or rather, AO isn’t computed through the transparent parts of the trail)
https://i.imgur.com/2GMmYRX.jpg
“Red Four, nothing’s gone wrong yet.”
— Hobbie Klivian

User avatar
Darksaber

Fleet Admiral (Administrator)
Posts: 10319
Joined: Mon Jan 10, 2000 12:01 am
Contact:

Post by Darksaber » Sun Oct 06, 2019 2:07 pm

Yeah I noticed a Halo straight away in the hangar, there seems to be a white or lighter band around an object, then the darker halo, as seen in DanHan's images, I think it shows more with the black_level = 0.2 I set mine to 0.6, it doesn't remove it but it makes it less noticable.
“You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”
- John Lydgate

Good Things Come To Those Who Wait....
Darksaber's X-Wing Station

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Sun Oct 06, 2019 6:24 pm

Yeah, I new about the halo, I'm trying to fix that; but I didn't know about the warheads or space debris. Thanks for pointing that out, I'll fix that.

User avatar
Trevor
Cadet 1st Class
Posts: 188
Joined: Thu Dec 04, 2014 7:11 pm

Post by Trevor » Sun Oct 06, 2019 7:13 pm

unfortunalty I have some bad news... AO and bloom are not working and my zbuffer is disabled (objects are drawing in painters order)

Trev

User avatar
DarHan
Cadet 1st Class
Rebel Alliance
Posts: 160
Joined: Mon Dec 05, 2005 12:01 am
Contact:

Post by DarHan » Sun Oct 06, 2019 7:40 pm

@blue_max — My bad. I decided to check again and it turns out the issue with the warhead trail was because I forgot to turn off ReShade’s SSAO before using yours. Using both at he same time also caused the effect to “inverse lag” (SSAO from one frame is displayed with the next render frame). Disabling ReShade’s SSAO solved that. Now the only issue with the warhead trail is the halo.

@Trevor — I tested this too. The Z-buffer gets messed up if MSAA is enabled in Ddraw.cfg. Turning off MSAA restores the Z-buffer (I use ReShade for MSAA, which allows me to bypass that isue).
“Red Four, nothing’s gone wrong yet.”
— Hobbie Klivian

User avatar
Trevor
Cadet 1st Class
Posts: 188
Joined: Thu Dec 04, 2014 7:11 pm

Post by Trevor » Sun Oct 06, 2019 11:07 pm

YAY it works
turning off blur stops the halos

turning up the samples helps compensate but the cocpit turns all spotty/moiry like its got a transparent overlay mesh that is z-fighting with the base mesh
Trev

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Mon Oct 07, 2019 6:11 am

Thanks for the additional information guys.

@Trevor: The SSAO blur should remove the Moire patterns you're seeing -- although it will bring back the halos. Can you try toggling that setting to see if it fixes the patterns you're seeing at least?

User avatar
Trevor
Cadet 1st Class
Posts: 188
Joined: Thu Dec 04, 2014 7:11 pm

Post by Trevor » Mon Oct 07, 2019 7:31 pm

sorry yes, that was the result of disabling the blur, but I chose the lesser of 2 evils (the blur causes halos because its blur radius is wayyy to large, eg a thin line of shadow blurs to a fat thick smudge)

I also tried setting the sample frame size to 0.5 (2x supersample) but that didn't work. it showed a quarter screen of AO with lines running up and down the screen.

Trev

p.s. can you add an ini option for g_bShowSSAODebug so we can see the AO only

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Mon Oct 07, 2019 9:16 pm

Thanks for confirming that the blur fixes the pattern, Trevor. The blur is currently being done on a 4x4 pixel mask, so it's already small; but it's not depth-aware yet. I believe the halos will go away once I add the depth into it. Just out of curiosity are you scaling the window size in some way? Are you also running SSAO with SteamVR, perhaps?

For g_bShowSSAODebug just press Ctrl + Alt + D -- that should show the SSAO mask.

Kampher
Cadet 3rd Class
Posts: 34
Joined: Tue May 09, 2006 11:01 pm

Post by Kampher » Tue Oct 08, 2019 1:15 am

Between this and the Bloom shader, XWA is getting a heck of a face-lift. Seeing these screenshots makes me wish we had a higher poly Star Destroyer though. I think a lot of the newer releases make it look aged in comparison. But I'm off topic.

I find myself checking progress updates every day. I can't wait to get some time to test this out.

blue_max
Ensign
XWAU Member
Posts: 331
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Sat Oct 12, 2019 7:53 pm

I'm about to release a new version. I've fixed the halos (as much as that's possible), made the blur angle- and depth-aware, and the SSAO itself is now also depth aware. I've fixed all the reported artifacts and a few others. The missiles/warheads is an interesting case because it's a mix of solid geometry and transparency. The solid part still casts occlusions and I believe that's how it should be; but I've removed the artifact for the transparent textures.

I also added a few more flags for the bloom so that, for instance, cockpit sparks can be configured separately.
ssao11.jpg
ssao12.jpg
You do not have the required permissions to view the files attached to this post.

Post Reply