feat: dump IW5 GfxWorld and clipMap_t as JSON

[eithan1231]

IW5's GfxWorld and clipMap_t never had dumpers, this should hopefully add some support for it

MW3 got an update recently that moved it from x86 to x64, which broke compatibility with a heap of file formats. I may push a change for that over the next few days. This PR does not include any changes or that.

I'm not across all of this repo's conventions yet, so happy to correct anything that's off.

[Laupetin]

hey :) thanks for creating a PR.

Yeah, the reason why GfxWorld and clipMap_t do not have any dumpers or loaders (yet - for any game) is that those assets are very complicated, very far from their original source format (as they are the compiled result of Radiant) and are not particularly feasable to edit manually like this.
The plan is currently to give these two assets support in terms of dumping and loading from file in the same way official mod tools handle them: as a .d3dbsp file.

That file contains all the radiant compiled data related to the map itself and includes not only GfxWorld and clipMap_t but in case of IW5 also these assets: ComWorld, GlassWorld, PathData, VehicleTrack, MapEnts, FxWorld.
There is an open PR for implementing this format for IW3 in #834 which i did not find the time to review yet 😅
The idea would be to bring an implementation similar to it to other games as well like IW5 to handle dumping and writing map assets.

The reason why i wouldn't use json in this case would be:

• there is a known, canonical format for these asset that fits the compiled format in this case
• for some games there is official mod tools support for working with these files
• jsons strengths are in that they are human-editable and readable
  • map assets are not really feasable to manually edit by hand with a text editor (besides map ents partly maybe)
  • there are large binary blobs that are not really efficient and practical to make human readable (which i have also seen you have as external file)

but maybe you can elaborate on what your goals are with this PR? 😅
The idea behind almost all dumpers is to write the asset into a format that can be edited optionally and later written back to a fastfile without any data being lost.
From what i can see the PR at this time only dumps a small subset of the available data of both assets?
Is there a specific usecase you have for only dumping this exact data?

[Laupetin]

Regarding the x86 to x64 change: Yes, i am aware, it is unfortunately not a particular small change to the fastfile format.
Fastfiles contain assets in an almost memory-dump like representation. So when the game changes from x86 to x64, a lot of the struct layouts also change with it. For example mostly because 64bit uses 8 byte big pointers, while 32bit only has 4 byte big pointers. This drastically changes the layout for every struct that uses a pointer already, it also changes alignment in the struct as well.
There is also other ways in which pointer sizes change things and also, some assets also change data layout besides that in addition.

So all in all, it's not a small change 😅 While OAT is already almost ready to handle x64 games natively, there is a bunch of work to be done in creating an integration between the x86 versions of IW5 and the x64 and also not having full duplicates of all of the games functionality.
So it's not a small task, but i am working on it on the side.

[eithan1231]

From what i can see the PR at this time only dumps a small subset of the available data of both assets?
Is there a specific usecase you have for only dumping this exact data?

Yeah I have only needed this small subset of data, game engines are definitely not my area of expertise. Everything you've said makes sense, I might have a read of the IW3 implementation.

I've been porting all of the CoD maps to be explorable within your browser, and so far have only required the attributes described in this PR. The choice for JSON, was that browsers love it haha.

[Laupetin]

That's a pretty cool project :) And also already looks very good. Also using the clipmap to be able to wall around then?

Im afraid the json part doesnt fit upstream oat though as there are other plans for implementing gfxworld support that probably fit the usecase of it as mod tools better 😅 As described above.

[eithan1231]

Yeah no worries at all, happy to close off this PR.

Using Clipmap to determine where players can go, just to be truthful to the original maps. I'm surprised how much you can run within modern browsers, I've effectively got all of the MW3 maps running without many issues. The most difficult part has been lighting/lightmaps. There's certainly a few things that the old quake/cod engines do in very abnormal ways, bridging that technical gap is a pain.

I've managed to get it looking pretty decent though, below are some more recent screenshots. Most maps are running at 40-60 FPS on mobile, which I certainly was not expecting.