The Dynamic Cockpit Project

Want to edit the game, build your own craft and missions? Here you'll find help, tools, guides and people to discuss with.
Post Reply

The Dynamic Cockpit Project

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Mon Jul 15, 2019 6:20 am

DC-1.jpg
DC-2.jpg
DC-3.jpg
The link to the video is here: https://www.youtube.com/watch?v=D0PUKpi ... e=youtu.be

Looks like it's possible to map the HUD to cockpit textures while the game is running, thus providing a "Dynamic Cockpit" that is fully-functional -- and 3D.

In VR, the HUD is often lost to the edges of the screen, and I think this is a nice solution to that problem; but it also works for the regular version of the game (and yes, it would also work for TrackIR users).

This is a prototype right now. To make this truly generic, it will require some amount of work. Right now, I'm just wondering if there's any interest in this project because it will most likely require some help from the community -- and also permission from the creators of the original cockpit textures because they will have to be modified a little bit.
You do not have the required permissions to view the files attached to this post.

User avatar
Camerad
Cadet 3rd Class
Posts: 28
Joined: Wed Dec 25, 2013 12:07 pm

Post by Camerad » Mon Jul 15, 2019 7:04 am

That's way cool with the HUD stuff in the cockpit dashboard or whatever it's called.
Windows 10 Pro 64-bit, i7-10700KF, 64GB 3200MHz DDR4, Geforce RTX 2080 Super 8GB

User avatar
keiranhalcyon7
Lieutenant JG
Posts: 599
Joined: Tue Jan 02, 2018 6:41 am

Post by keiranhalcyon7 » Mon Jul 15, 2019 8:30 am

:shock:
:megaeek:
This is it. This is the major way in which XwA rolled back functionality from the previous games. Is there interest? You better believe there's interest!

Are you overwriting entire textures in the cockpit opt, or compositing the HUD textures into the existing textures? I'm wondering what the best way would be to standardize things so cockpit opts could be created compatibly.

User avatar
Vince T
Fleet Admiral (Administrator)
Posts: 14045
Joined: Fri Apr 27, 2001 11:01 pm
Contact:

Post by Vince T » Mon Jul 15, 2019 9:27 am

keiranhalcyon7 wrote:
Mon Jul 15, 2019 8:30 am
:shock:
:megaeek:
What he said! This is brilliant!

I feel a sudden urge to rework all my fighter and shuttle models ...
Your ship, Captain. I need a drink. - Vince Trageton
Vince T's X-Wing HQ - where the bad guys get their gear

User avatar
Darksaber
Vice Admiral
Posts: 10931
Joined: Mon Jan 10, 2000 12:01 am
Contact:

Post by Darksaber » Mon Jul 15, 2019 11:31 am

Very interesting :D
“You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”
- John Lydgate

Good Things Come To Those Who Wait....
Darksaber's X-Wing Station

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Mon Jul 15, 2019 3:56 pm

I'm so into this. Just to think that every cockpit will be a little different in not just look and feel but actual function, especially if paired with Track IR or VR. I'm just worried about the amount of work required to update cockpits. I guess I need to start learning how to do that stuff in the 3D realm to help out if we ever get to that stage.

In my own little HUD project I'm trying to figure out how to increase the radar sizing. I can increase the frame size but the content of the actual radar isn't moving or scale-able. Have you figured out how to adjust that stuff? I'm running 1920x1080 and I'd like to be able to increase the scale of the screens as a first goal, then actually move the placement of them in a 5760x1080 triple wide setup. Moving all the screen edge hud objects would be a goal for that actually.
Image Image Image Image Image

User avatar
JeremyaFr
XWAU Member
Posts: 3922
Joined: Mon Jan 18, 2010 5:52 pm
Contact:

Post by JeremyaFr » Mon Jul 15, 2019 4:27 pm

Cool :thumbs:

User avatar
Darksaber
Vice Admiral
Posts: 10931
Joined: Mon Jan 10, 2000 12:01 am
Contact:

Post by Darksaber » Mon Jul 15, 2019 7:04 pm

Just thought of a problem with this, your testing the dynamic cockpit with the VR thingy you have installed which you're developing right? Well not everyone including myself have these occular goggle thingys you use, we play XWA with a static screen, my point is the POV of the XWing looks like this on a static screen
flightscreen2.jpg
You see the problem yet? If the HUD elements are mapped to the 3D cockpit model your not going to see the screens, unless you start pressing the keypad buttons to look up and down, or unless the POV is changed so you see more of the screens like so
flightscreen3.jpg
Which actually looks better lol

Also we can see where the radar, shield, laser bars, central hud screen and the lefthand hud screen are, but where is the righthand and hud screen and the top central screen, where do these disappear too?

Just a few friendly concerns from the statically screen challenged :D
You do not have the required permissions to view the files attached to this post.
“You can please some of the people all of the time, you can please all of the people some of the time, but you can’t please all of the people all of the time”.”
- John Lydgate

Good Things Come To Those Who Wait....
Darksaber's X-Wing Station

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Mon Jul 15, 2019 7:15 pm

So, I think this is only going to be usable for VR and Track IR people. Those of us with that tech that are familiar with more modern sims realize that 6DOF and checking cockpit instruments can be a little more laborious, but much more immersive. I don't see the traditional HUD ever going away. It might influence the ergonomics of future cockpit designs, but there's no way we can push everyone to get VR or simple head tracking to permanently change everyone's play experience.

The real question is going to be whether or not cockpit elements can be sort of packaged and distributed among cockpit authors so shared elements represent displays for specific information, then if those elements can have stock imagery replaced with live HUD elements. And finally, if everyone shared the same cockpit opt, can both types of players connect and fly together at the same time? One VR/Headtracking guy with live hud data in their cockpit VS a guy with stock/static cockpit instruments and a traditional HUD.

We need to consider that stuff now.
Image Image Image Image Image

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Mon Jul 15, 2019 7:19 pm

Another option is to simply mirror the hud info into a live cockpit, then the VR people just turn off original HUD elements while the traditional guys leave it on.
Image Image Image Image Image

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Mon Jul 15, 2019 7:43 pm

Alright, where do I start? Well, I guess this means there's interest, so that's good. I'll try to answer all of your questions; but I can't do that in one go, so let's see...

* This should be usable by *anyone* out there, whether you're running VR, TrackIR or a traditional screen: I can change the POV and viewing angle a little bit, inside ddraw.dll, and I can make that configurable, so that people don't have to modify the POV or the cockpits at all and still be able to use this. However, having said that, if the community wants to update the cockpits to change the POV or provide larger elements, that would help too.
* Things feel a bit larger in VR and you can zoom it; but yes, some of the elements look small. I can overload the up-arrow key so that people can zoom-in if needed even if they're not running VR/TrackIR.
* The main reason the radars are that small was that I couldn't find a place big enough to put them; but they can be placed anywhere, and they can be made arbitrarily large -- but they will become pixelated after a certain threshold.
* The missing HUD elements are mainly because this is a prototype and I'm lazy: this proved to be much harder than I thought at first, so I quickly realized I would need help from the community; but why bother if no one cared? Anyway, looks like the community cares, so I'll keep working on it.

So, let me repeat this: this project is for *everyone*; not just the VR/TrackIR crowd. I want everybody to enjoy this. As I said, I can modify the POV within ddraw and I can make that configurable, so that shouldn't be a barrier either.

Also, if you want to help, you'll only need basic image manipulation skills and some basic math, that's all. No OPT or 3D skills needed (that's only for people who want to modify the POV or the cockpit; but that's not necessary).

I'll post more details soon :)

User avatar
Driftwood
Admiral (Moderator)
Posts: 2174
Joined: Wed Oct 22, 2003 11:01 pm
Contact:

Post by Driftwood » Mon Jul 15, 2019 8:59 pm

I'd like to see this continued to be figured out if possible. I'm also curious what it takes to set this up in a given opt since if the bugs get worked out, I could see wanting to impliment this in mine.

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Mon Jul 15, 2019 9:45 pm

There's other potential applications if we can ever add new things in too. Being an Idea man, you'll have to forgive me for dreaming.
1. Sfoil State
2. Slam on/off
3. Weapon link status
4. Auto convergence
5. Landing struts (if we ever separate them)
6. Shield Percentages
7. External Cameras??? (Cockpits for Moldy crow and Dx9 possible?)
Image Image Image Image Image

User avatar
keiranhalcyon7
Lieutenant JG
Posts: 599
Joined: Tue Jan 02, 2018 6:41 am

Post by keiranhalcyon7 » Mon Jul 15, 2019 10:17 pm

ual002 - Don't forget a pre-XwA-style throttle slider. ;) And external cameras could find use in TIE cockpits, in line with the way the prior games displayed the non-forward view directions.

Personally, I'm prepared to sacrifice some movie authenticity for the sake of immersion (the same tradeoff made by TG in X-Wing/TIE Fighter/Xvt). I realize though that that opinion may not be shared by everyone, so it might become necessary to look into the difficulty of providing both the current cockpits and new "instrumentation-ready" cockpits as an installer option (sorry, DS). If it's possible to package the original cockpit and the new cockpit in a single opt (e.g, by selecting which set of elements to use via the FG color mechanism), this may boil down to a single global setting in a text file.

Multiplayer - I vaguely recall that all players having the same set of ships (and versions thereof) is a hard requirement for multiplayer to work. But would it be an issue to have different cockpit opts and/or cockpit opt textures, which (should) have no bearing on network operations or gameplay?

Stage 2 should totally be destruction graphics for the cockpit instrumentation, with a reworking of the cockpit sparks to emit them from the destroyed instruments...

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Mon Jul 15, 2019 11:10 pm

No OPTs were harmed in the making of the movie above :D

Let me provide a bit more information. I'll post pictures later; but here's the gist of how this is done: on every frame, the game switches from rendering 3D to rendering 2D. This usually means the game will start rendering HUD/GUI stuff. I keep track of the state of each frame in the VR mod, so I can detect this change. When this happens, I stop rendering to the screen, and render to an off-screen buffer. Once the frame is done, I cut out pieces of this off-screen buffer and paste them on top of specific cockpit textures. Then, to cover the cutouts, I cover these textures with new textures where I've punched some transparency holes so that the HUD can show through.

I mentioned the VR mod; but this mod can run without VR and it basically runs the original ddraw.dll while keeping track of the state. So that's how non-VR users can still use the dynamic cockpit.

This change is completely optional and can be activated/de-activated using cfg files (well, it will be configurable, right now a lot of it is hard-coded). It would even be possible to show the dynamic cockpit *and* the regular HUD on top of everything. Or maybe display a combination of both.

For multiplayer I don't *think* there will be any issues because the OPTs should be the same. The additional textures are loaded by ddraw at runtime as needed.

Destruction graphics sounds like a nice idea. Another thing that is possible is adding animations too; but we're not there yet. Good ideas for a future project, though.

User avatar
Trevor
Lieutenant JG
Posts: 541
Joined: Thu Dec 04, 2014 7:11 pm

Post by Trevor » Mon Jul 15, 2019 11:12 pm

Holy..... WOW this is FANTASTIC!!!

You bet there's even more interest :P

Trev

Bman
Lieutenant Commander
Posts: 1167
Joined: Mon Apr 05, 2004 11:01 pm

Post by Bman » Mon Jul 15, 2019 11:18 pm

Excellent project. :-)
W-I-P: TFTC, MC Viscount Cr., ISD-II Avenger, NL-1 Platform, Ton-Falk Esc. Cr., & Misc.

User avatar
ual002
XWAU Member
Posts: 983
Joined: Wed Sep 24, 2008 2:23 am

Post by ual002 » Tue Jul 16, 2019 12:41 am

keiranhalcyon7 brings up a good point about destroyed instruments. How do your changes affect that? If the traditional HUD elements that fail also fail in the live cockpit, do you have the ability to replace a "broken" screen in the live cockpit with a static "destroyed instrument bmp"?
Image Image Image Image Image

User avatar
Phoenix Leader
Rebel Alliance
Posts: 437
Joined: Wed Aug 08, 2018 2:20 pm

Post by Phoenix Leader » Tue Jul 16, 2019 12:41 am

blue_max wrote:
Mon Jul 15, 2019 11:10 pm
No OPTs were harmed in the making of the movie above :D

Let me provide a bit more information. I'll post pictures later; but here's the gist of how this is done: on every frame, the game switches from rendering 3D to rendering 2D. This usually means the game will start rendering HUD/GUI stuff. I keep track of the state of each frame in the VR mod, so I can detect this change. When this happens, I stop rendering to the screen, and render to an off-screen buffer. Once the frame is done, I cut out pieces of this off-screen buffer and paste them on top of specific cockpit textures. Then, to cover the cutouts, I cover these textures with new textures where I've punched some transparency holes so that the HUD can show through.
Amazing. So you essentially paste the HUD informations on top of some cockpit textures, plus a few graphics adjustments to make the thing more immersive.
The only problem is that if you stop rendering to the screen when the switch from 3D to 2D triggers, you can no longer display the classic HUD.
Is it possible to do the same thing without stopping rendering to the default screen?
I mean the best possible experience would be having both the old HUD and the dynamic cockpit, but to implement this combination the solution is to render the 2D part both to the main screen and to the buffer.

User avatar
Jaeven
XWAU Member
Posts: 578
Joined: Mon Mar 30, 2015 3:18 am

Post by Jaeven » Tue Jul 16, 2019 1:05 am

This sounds really interesting. Looking forward to how this develops.

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Tue Jul 16, 2019 1:11 am

ual002 wrote:
Tue Jul 16, 2019 12:41 am
keiranhalcyon7 brings up a good point about destroyed instruments. How do your changes affect that? If the traditional HUD elements that fail also fail in the live cockpit, do you have the ability to replace a "broken" screen in the live cockpit with a static "destroyed instrument bmp"?
Right now, I don't think I can tell between a destroyed instrument and one that was simply turned off. For that I would probably need to inspect the PlayerData with a hook (BTW, anyone knows how to get access to XWA's heap inside ddraw.dll?). I think it's possible; but not right now.

@Phoenix Leader: I think I can render both the dynamic cockpit and the regular HUD on top of everything. I can make this configurable too. As in "render this to the cockpit; but leave these other things alone".

User avatar
keiranhalcyon7
Lieutenant JG
Posts: 599
Joined: Tue Jan 02, 2018 6:41 am

Post by keiranhalcyon7 » Tue Jul 16, 2019 1:29 am

Question - what happens if you turn off part of the HUD (e.g. tap 'end' to shut off the CMD/cannon power indicators)? Do parts of the dynamic cockpit that depend on that part of the HUD go blank?

Where I'm going with this - I think it would probably be better overall to implement this as a hook (outside ddraw.dll) that redirects the rendering of each of the (what, 20-ish?) items of HUD data to its own buffer, copy them to the cockpit appropriately, and then separately copy them to the HUD buffer if the containing HUD panel is on. Then you'd also have access to the entire XWA heap to access system status data, or whatever else you need, via the same mechanism as the other hooks. (Although, I'm not sure what would stop you from using that same mechanism to gain access to XWA's heap from within ddraw.dll. All the dlls are loaded in basically the same way, right?)

Edit: How are you accessing the cockpit opt textures if you don't have access to the XWA heap?

User avatar
blue_max
XWAU Member
Posts: 2295
Joined: Wed Mar 20, 2019 5:12 am

Post by blue_max » Tue Jul 16, 2019 5:00 am

Thanks for the suggestion, keiranhalcyon7. I'm still probably a couple of weeks away from being able to release this to the public, so this is the right moment to provide all kinds of suggestions. Not all of them might be doable; but I do listen and I'll try to do things.

Anyway, I think what you're proposing is actually harder to do. The Direct3DDevice and Context are created in ddraw. Can they be shared with a hook? Maybe; but I don't know if it's safe to do so. On top of that, it's probably not a great idea to use independent buffers for each HUD element. XWA renders things in weird ways. For instance, I think the text on the left and right message panels are rendered at the same time, so it's not easy to split them while they are being rendered. I'm not opposed to using hooks, though; but I'm not sure how they can share the same DirectX device. It's probably easier to do something so that ddraw can access XWA's heap?

If someone can figure out a hook for the whole 3D engine... well, that would be amazing and would solve a number of problems :)

If you turn off any HUD element, the corresponding dynamic cockpit element just displays the blank background.

Regarding access to textures: XWA loads ddraw and then sends commands to create textures through ddraw. I can't see the code that loads the textures from disk; but I can see the flow of data that is sent to DirectX to create the textures. I'm applying a simple CRC algorithm to this data stream and attach it to each texture. This CRC works as a unique identifier that I can use to figure out what the game is about to render. So, for instance, when I see CRC XXX and I know that is the targeting computer texture, then I know that I have to add some cut out from the HUD buffer and then cover it with another custom texture.

BTW, since I'm working with texture CRCs, that means that if every rebel alliance ship (for instance) uses the exact same texture in its targeting computer, then all these ships will get a functional dynamic cockpit element when this project is enabled.

Finally, I've also got to make this clear: for the moment, you'll need to use my version of ddraw in order to use this feature. There are a number of things that have to be modified in ddraw to enable this: State tracking, CRCs, and modified Shaders (and most likely, other stuff that I'm forgetting right now). I'm happy to share my code with others and I can copy the relevant bits of code to other versions of ddraw to enable them with this feature too; but I only have so much bandwidth. So, for now, I'll most likely continue to work on my version. I hope this isn't a showstopper. Just yesterday I thought no one would really care about this project and now it looks like there's quite a bit of support (Thank you guys!). So, if you have a favorite version of ddraw -- for whatever reason -- and absolutely would want to see this feature in that version, just let me know.

... on the other hand, if we can figure out how to do this using hooks, then the version of ddraw won't be a problem after all :P

yaglourt
Recruit
Posts: 7
Joined: Thu Apr 26, 2012 10:21 am

Post by yaglourt » Tue Jul 16, 2019 6:28 pm

Blue-max, when will you stop to amaze the old XWA community :schockiert: :lachtot:
I can't wait to try your new dynamic cockpit !
Tho is it readable enough for you in VR ?

User avatar
DTM
Fleet Admiral (Administrator)
Posts: 2119
Joined: Tue Apr 22, 2003 11:01 pm
Contact:

Post by DTM » Tue Jul 16, 2019 6:43 pm

Do you wonder if there is interest in this project ??? Do we realize that this is a revolution for XWA? We have been working for years to make XWA a more modern and playable flight simulator. This is the purpose of the XWAUP project. So let me answer your question: yes, this project is VERY interesting.

Ok, you have all my attention and support to apply HUDs screens to all XWAUP cockpits and make this technology accessible to all OPT authors to apply HUDs to all extra-XWAUP crafts.

As for XWAUP copyright, I have to speak with my XWAUP project partners, but I think it won't be a problem. Many cockpits need to be updated...

As for the workforce, to change the cockpits and make the screens more readable, I immediately offer myself available. But I warn you that I don't understand anything about programming. You only give me input, what form and position the screens must have and which textures I must use to make them operational, and I do. :D

Post Reply