DAT File/Backdrop Transparency (SOLVED!!!)

Want to edit the game, build your own craft and missions? Here you'll find help, tools, guides and people to discuss with.

DAT File/Backdrop Transparency (SOLVED!!!)

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sat Jun 29, 2019 8:39 pm

Out of curiosity and an attempt to understand how the game handles transparency I started messing with the backdrop files. I'm sure many of you have explored this ad-nauseum in the quest to create the super backdrops patch, but I know the game can handle transparency in the concourse art as I have already experimented with it. With the dat files however the game seems insistent on replacing a specific color in the dat files. Below I create a mockup of what I was trying to address. On the left is what we have now, and on the right was my intended concept and attempt at the same star backdrop but with transparent edges to better fit in the new colored starfields.

Image

However after loading the .png into the XWA Dat Editor and saving, it seems that no matter what setting I provide, the end result fills in the transparency and crops the edge.

Image

I wonder what information you guys could impart to help me understand this. I was really hoping the task of feathering the different star and planet backdrop image edges was just a load of work someone didn't want to do, something I am more than capable of, to help out. Alas I get the impression you have already been down this road and hit a road block.
Last edited by ual002 on Sun Jun 30, 2019 6:39 pm, edited 1 time in total.
Image Image Image Image Image

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sat Jun 29, 2019 8:45 pm

I imagine JeremyaFr might have some insight into what exactly his editor does when you load in a PNG with transparency and save.

EDIT: Correction, it looks like the dat editor preserves the transparency just fine, its the engine that doesn't seem to know what to do with it. Reopening a modified, saved dat file seems to keep transparency just fine.
Image Image Image Image Image

User avatar
Darksaber
Vice Admiral
Posts: 10931
Joined: Mon Jan 10, 2000 12:01 am
Contact:

Post by Darksaber » Sat Jun 29, 2019 9:07 pm

After DTM released I think the first version of the Super backdrops, I tried exactly the same thing, exacting each and every image (and there are loads), feathering the edges of the planets, moons and so on to remove the edges, then inserting and replaced all of DTM's original images, only to test it in game and be very disappointed at the result.

So it was another project that found it's self in the recycle bin :(
“You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”
- John Lydgate

Good Things Come To Those Who Wait....
Darksaber's X-Wing Station

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sat Jun 29, 2019 9:13 pm

Shame, I'm sure there's an answer if we dig. Well, I still have a few of my test backdrops just in case we figure it out.
Image Image Image Image Image

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sun Jun 30, 2019 4:18 pm

I just now went so far as to save my work as a BMP with an alpha channel. (I think I did it right) I was hoping the game would prefer an alpha channel for feathering. Unfortunately it does not. There has got to be a solution. JeremyaFr, do you think this is an engine problem or something that can be solved with a tweak to the DAT editor?
Image Image Image Image Image

User avatar
JeremyaFr
XWAU Member
Posts: 3921
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sun Jun 30, 2019 6:04 pm

The problem comes from the game engine.
When a backdrop with transparency is loaded, it is converted to 1555 format.

You can try this patch:

Code: Select all

At offset 03156A, replace 7406 with 9090.
This will use 4444 format instead of 1555 format.

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sun Jun 30, 2019 6:09 pm

Ok, I am unfamiliar with the exe patcher but I will give it a shot and report back.
Image Image Image Image Image

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sun Jun 30, 2019 6:30 pm

Well, HOT DAMN IT WORKED. The feathered edges are gonna take a little tweaking, and they have a bit of a gradient but I personally prefer that over hard, black edges.

Image

Jeremy you are a master.

Now that begs the question Darksaber, do you still have all your transparent versions or do I need to go thru them with DTM or something?
Image Image Image Image Image

User avatar
Darksaber
Vice Admiral
Posts: 10931
Joined: Mon Jan 10, 2000 12:01 am
Contact:

Post by Darksaber » Sun Jun 30, 2019 7:04 pm

Just looking through my files, and I have the Planet.dat I changed sometime ago, I've just added the different starfields that are in v2 and created a new Planet2.dat

The problem is the Nebula.dat there is a lot of black background with those images, see we are just saving images from DTM's Dat's, he has the original images that might already have alpha transparent backgrounds, so instead of stepping on his toes, contact DTM first and ask your question.

Until then I'll hold off sending you the files, also he might want do it his self, dunno, but better to ask permission first :D

Also nice find Jeremy
“You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”
- John Lydgate

Good Things Come To Those Who Wait....
Darksaber's X-Wing Station

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sun Jun 30, 2019 7:15 pm

Nah, its totally fine, makes sense, plus why do double work potentially. As a test, I modified the low orbit backgrounds to remove the starfield and have a transparent atmospheric haze. Problem is, at that resolution, the gradient transition is obvious. I did this hoping I could have a large gas giant rising over the horizon, sadly I don't think that will be possible.

The below image is layered as follows:
1. Spherical Starfield Backdrop, up, down, left, right, front, back.
2. Gas giant
3. Retouched low atmosphere backdrop (removed starfield with photoshop), down, left, right, front, back

Image
Image Image Image Image Image

User avatar
Darksaber
Vice Admiral
Posts: 10931
Joined: Mon Jan 10, 2000 12:01 am
Contact:

Post by Darksaber » Sun Jun 30, 2019 7:27 pm

interesting, but yes it doesn't work, nevermind it was a nice idea

Just remembered DTM is on holiday at the moment, so he can do anything for the time being until he gets back home :)
“You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”
- John Lydgate

Good Things Come To Those Who Wait....
Darksaber's X-Wing Station

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sun Jun 30, 2019 7:38 pm

Lol,
Welcome back DTM, here's a shit-ton of work.
Hahaha.
Image Image Image Image Image

User avatar
keiranhalcyon7
Lieutenant JG
Posts: 599
Joined: Tue Jan 02, 2018 6:41 am

Post by keiranhalcyon7 » Sun Jun 30, 2019 10:49 pm

That's some severe banding. These aren't still 256-color images, are they?

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Sun Jun 30, 2019 11:34 pm

That would be a question for JeremyaFr, as I think it might happen in the exe conversion. The banding areas are fresh from new parts of the original image saved as a png.
Image Image Image Image Image

User avatar
Driftwood
Admiral (Moderator)
Posts: 2174
Joined: Wed Oct 22, 2003 11:01 pm
Contact:

Post by Driftwood » Sun Jun 30, 2019 11:48 pm

Yeah that doesn't look ideal, but on the other hand I don't necessarily think it looks terrible either.

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Mon Jul 01, 2019 12:56 am

Nice, almost there. Above patch, is that related to the .Dat type "25" project?

Ual002, these are from my notes Jeremy wrote in past, just to bring you up to date if this is helpful . . .
"In dat files, images are 8-bits indexed colors (each color is 24-bit R8G8B8 ), with or without an 8-bit or 1-bit alpha channel. The palette contains up to 256 colors.
The main image types in dat files are:

type 7: 8-bit indexed colors and 1-bit alpha, rle compressed
type 23: 8-bit indexed colors and 8-bit alpha, rle compressed
type 24: 8-bit indexed colors and 8-bit alpha
type 25: similar to type 24 (currently isn't functional within the game engine) ????

Concourse images use type 7, 23.
In-flight images use type 7, 24, 25.

If a concourse image is not type 7 or 23, then the image is not drawn.
If an in-flight image is not type 7, 24, or 25, then the image is not loaded into a texture and the game crashes.

When importing an image, the editor will set the type to 24. In the dll, I added support for a type 25 which is 32-bit. Since this type is not functional in-game (WIP), I disabled it in the editor. So the imported images are reduced to up to 256 colors (each color being 24-bit). The current algorithm generates up to 128 shades per channel."



.
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
Phoenix Leader
Rebel Alliance
Posts: 437
Joined: Wed Aug 08, 2018 2:20 pm

Post by Phoenix Leader » Mon Jul 01, 2019 12:13 pm

Edit: used PM for this thing
Last edited by Phoenix Leader on Tue Jul 02, 2019 6:30 am, edited 1 time in total.

User avatar
Trevor
Lieutenant JG
Posts: 541
Joined: Thu Dec 04, 2014 7:11 pm

Post by Trevor » Mon Jul 01, 2019 8:09 pm

The reason for the banding is Jeremy provided a variation on the 16bit format, 4444 = 16bit
what you want is 8888 = 32bit

Trev

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Mon Jul 01, 2019 8:19 pm

I will try again tonight and update you with the info you provided Trev, thanks.
Image Image Image Image Image

User avatar
Darksaber
Vice Admiral
Posts: 10931
Joined: Mon Jan 10, 2000 12:01 am
Contact:

Post by Darksaber » Mon Jul 01, 2019 8:30 pm

Thanks Trev, but we would really need to know what the equivalent offset value would be to 8888
JeremyaFr wrote:
Sun Jun 30, 2019 6:04 pm

Code: Select all

At offset 03156A, replace 7406 with 9090
This will use 4444 format instead of 1555 format.
Any idea what the value would be?
“You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”
- John Lydgate

Good Things Come To Those Who Wait....
Darksaber's X-Wing Station

User avatar
JeremyaFr
XWAU Member
Posts: 3921
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Mon Jul 01, 2019 8:52 pm

Yes.
But for now, it doesn't work. The game crashes when setting 32-bit format for backdrops.
I will try to solve it.

User avatar
Darksaber
Vice Admiral
Posts: 10931
Joined: Mon Jan 10, 2000 12:01 am
Contact:

Post by Darksaber » Mon Jul 01, 2019 9:01 pm

Thanks for the answer Jeremy, I thought there must be a reason why, but what we have now is a massive step up from what we did have :2thumbs:
“You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”
- John Lydgate

Good Things Come To Those Who Wait....
Darksaber's X-Wing Station

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Mon Jul 01, 2019 9:53 pm

Ah, I misread Trev's info at a quick glance.
Image Image Image Image Image

User avatar
JeremyaFr
XWAU Member
Posts: 3921
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sat Jul 06, 2019 12:00 pm

JeremyaFr wrote:
Sun Jun 30, 2019 6:04 pm

Code: Select all

At offset 03156A, replace 7406 with 9090.
This will use 4444 format instead of 1555 format.
Hello,
I've added this patch to XwaExePatcher.

User avatar
JeremyaFr
XWAU Member
Posts: 3921
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Sun Jul 14, 2019 3:58 pm

Hello,
Here is an in-game comparison of pixel formats:

xwa_dat_pixel_format.png
You do not have the required permissions to view the files attached to this post.

Post Reply