OpenSMACX v0.2.3

[scient]

I do wonder about different versions of Windows SDK biting you in the ass.

I did run into a similar issue with redirecting mem_get() and env_open() between current Windows SDK and the one that SMACX is compiled with. There was an incompatibility that would cause the game to crash on start up. I tracked it down to free() in section of code I hadn't decompiled. That's why I have two additional "_old" versions of those functions inside general.cpp. The classes I've fully decompiled, including destruction, use the non-old versions. I then redirect the exe to call the old version that does a direct call to old SDK version of _fopen and malloc.

I'm not sure what version of SDK I have on my VS 2015 VM but it is not up to date so likely one that is a couple years old at this point. I don't think I'm using anything that is version specific that would require setting hard SDK limits.

[bvanevery]

Nevertheless standardizing on your compiler and your SDK, even if other people "could" build it some other way by changing VS version and .sln settings, is good for your community development.  When someone comes to you with a weird crash, you get to ask them, "Does it work with VS 2017 and SDK version X" ?  Then everyone has to meet the same threshold of yep, it's sane, there's actually a problem other than me borking my dev environment.

Since you are doing weird things to dlls, I think this kind of engineering discipline does actually matter.  You aren't just writing "portable ANSI C code that runs everywhere".

[scient]

Added CodeBlocks project file thanks to induktio and cleared up applicable warnings when using gcc. Still chipping away at Veh/unit functions.

[scient]

I realized to continue working on Veh/unit functions that I needed a better understanding of how the underlying map code works. There are various inaccuracies in all the previous analysis that I've come across (mainly 63456_SMAX-MP.txt) in how it is handled internally. For those interested, have a look at map.h/map.cpp. They really did like to pack everything into as little bits possible. There is still one byte per tile (offset +6) that doesn't seem to be used. I think some of the default .mp files have it set but I haven't looked into it more. Otherwise, the only part I haven't fully documented are some flags in bit2.

Still chugging along with more Veh functions. Getting closer to being able to work on more combat related code that I know there are issues with.

[vidyajames]

A source port would be amazing, hopefully it can implemented with a cross-platform library like SDL to bring support to Linux and others.

[bvanevery]

Feel free to get started on that.  I mean, a bunch of people looking at each other to do stuff, never got anything done in Open Source.

Myself, I'm not going to lead such a thing.  I have 25 years of $0 Open Source experience.  I know why things get done and why they don't.  I just spent 6 full time person months, 1 calendar year, coughing up my mod.  It doesn't even touch code! yet that's how much work it was in the real world.  And I did it.  All of it.  But man that will give you burn out.  Not throwing another year of my life away on something that makes me $0.  This is hardly the 1st time I've done this sort of thing either.  At least this time, I crossed the finish line and got people to use it and it looks like they will keep using it.  Next step for me is getting money for something.

[vidyajames]

I would be willing to assist, I do have experience with SDL and some comparable libraries. It's unclear if this code is supposed to be a pure vanilla reverse engineering project, or something that could be ported to SDL itself. Do you think it would be worth it to port this code as-is or should that come later?

[bvanevery]

It's unclear if this code is supposed to be a pure vanilla reverse engineering project, or something that could be ported to SDL itself.

Scient wrote in his readme.md:

A project to decompile SMAC/X to C++ with the long term goal of creating a full open source clone.

I think you can see that in principle, SDL can be applicable.  Particularly if you somehow manage to apply it.

In practice, you have a hell problem on your hands.  The 3D voxel game artwork is in a proprietary format that nobody has ever completely reversed engineered.  Some person(s) got started on that, and made enough progress to be able to insert a texture map into a model.  But that's hardly the same as being able to render or edit those models.  It's a real showstopper piece of work, jousting at crap like that.  It's where I definitely begged off.  For the effort to be spent, I'd be far better off writing a rendering display for a 4X TBS game from scratch.  In which case, why should I spend years messing with SMAC?  I'd rather spend years messing with a spiritual successor to SMAC, that I can make money selling.

Rather than trying to display the original artwork, it might be easier to just create similar artwork from scratch, in a modern open source usable 3D graphics format.  I did start researching what format would be applicable.  glTF seems like the leading candidate, particularly for an open source crowd.

I'm not clear on what open source 3D engines can display glTF, nor how well various 3D editors handle or export glTF.  I have been resisting learning Blender for years, because to me it looks like a real turd.  So much complexity to do things.  Not that I really know any ways to produce 3D art assets easily.  I'm rather much a dunce at that, even though I've written parts of 3D libraries and 3D device drivers before.  I'm even a visual artist in traditional media such as acrylics, but I have no digital art skills whatsoever.  It just looks like such a godawful chore that I simply never want to get started on learning Blender, the thing makes me want to puke.  I've wondered whether the upcoming Blender 2.80 will alleviate any of that.  I tend to research "Blender things" every several months when I remember I have this core problem of how to produce game assets.  Then I tend to get distracted by real life, like fixing cars, or making small amounts of money to keep my head above water.

Also I don't personally believe in Linux development, as there's no road to money that way.  I did Linux recently for 3 years solid, waiting for the Steam Machines to turn into something.  They didn't.  I had hoped for a better development environment under Linux, with better tools and ecosystem, and I didn't get it.  It was crap, at least for consumer software and game development.  I bet it's like the greatest network and embedding platform ever, but for consumer stuff, it's crap.  I begrudgingly went back to Windows, which is the epicenter of the 4X TBS game market.  Real customers use those systems, and real development will surely be in some kind of DirectX.

Don't get me started too much on OpenGL, it's a nightmare.  It's also dying and people would be fools to start any new project with it.  If some 3D engine already did OpenGL well enough and had the job done, then that might work ok, but it would also need to do glTF.  I can't tell you if anyone in open source has risen to that bar.  I knew more about such things about 6 to 9 months ago, at which time I said, to hell with this ain't gonna happen.

I was also deeply spending piles of time on my SMACX AI Growth mod back then.  I had a reason to avoid any and all pointless technical distractions.

Vulkan is future looking in the Linux world at least.  Problem is, nobody else gives a damn about it.  Apple went with Metal.  Windows has had and will continue to have DirectX.

If WebGL is actually usable for anything, and someone's got a 3D engine for it, and someone's got glTF support, that would be worth knowing.

The strongest open source game development engine out there right now seems to be Godot, but it is 2D oriented more than 3D.  Also what that mainly buys you is smartphones, which I really don't think is the point of the 4X TBS genre.

Anyways this is a very long way of saying, I can see why people would be motivated to simply hack on the game binary, rather than try to tackle the difficult problem of making an open source clone.  But hey, if your research and due diligence actually yields some kind of viable answer, and you produce some kind of proof of concept like 1 piece of unit artwork up on a screen, then you da man (or woman, or whatever).

[bvanevery]

I am now remembering that Godot is doing glTF, they are proponents of it.

[vidyajames]

While I'm a proponent of Godot due to it being FOSS, I don't think it'd be a good fit for a source port. It's a full on engine/workflow that would be more trouble to get everything working within it's "node system".

I completely overlooked the 3D models in the game, which would make SDL less than great for this use-case since SDL is 2D-centric unless you write raw OpenGL, which would hurt future Vulkan support. There are other libraries that have less experience with, but do lend itself towards mixing 2D and 3D within it's framework (ie. SFML, Allegro, raylib).

All of these libraries support OpenGL ES, and using tools like emscripten can be transpiled to WebAssembly and played in a browser. Easily portable to mobile and consoles, etc.

[bvanevery]

But the point is Godot does glTF, their core developer is into it.  A solution is needed for the 3d art assets.  I seriously don't think you or anyone else is going to manage to reverse engineer the proprietary voxel format.  Unless that kind of grungy problem really really turns you on, and you're fully prepared to spend a really really long time banging your head on that, just because you're that kind of person.  Semi-sane people would make new artwork from scratch.  Completely sane people would move along and not take on any of this SMAC open source idea at all.

If you have some other way to come up with proof of concept on the 3D art assets that doesn't use glTF and/or Godot, power to you.  But it's the major roadblock.

[vidyajames]

I do think replacement 3D assets would be the way to go. raylib supports glTF: https://github.com/raysan5/raylib/pull/842

[bvanevery]

Those comments on the raylib glTF implementation say it's pretty bare bones.  Only a diffuse texture, no animations.  That means someone has to do the work of coming up with all of that in the real world.  You know of anyone out there with a more complete glTF implementation?  You may not be thrilled about the organizational pain Godot would cause you, but raylib would clearly cause you glTF pain at this point.  I think it's too much to expect that all that kind of stuff is just going to get done in open source land and be good / robust.  That feature landed on raylib 13 days ago.

[vidyajames]

It is very early, but we're also talking about a project that is also bare-bones, and hypothetical. I'm unsure why glTF specifically is important, other than to be especially bleeding edge in context to a 20 year old game. Godot is certainly not the only implementation of glTF 2.0, and is available as a stand-alone library: https://github.com/syoyo/tinygltf

I've worked a bit with Godot, but I've never tried GDNative (their method using C/C++ code) so I can't vouch for how well it works. The idea of re-implementing the whole game using their Node system seems to me like creating something from scratch using basic game logic derived from reverse engineering, instead of repackaging the reverse engineer code into a cross-platform graphics/audio/input library.

[bvanevery]

I'm unsure why glTF specifically is important,

Because I just did due diligence on candidate 3D file formats and that's what I came up with as far as what would be a good idea nowadays.  Look you can read a wiki entry about glTF if you don't know much about it, why it might be important, and what its value might be.  You're free to come up with any other proof of concept option especially if you personally are going to do the work.  That's how open source goes, the tools that some author likes are what get used because they actually did the work.  I can't quote you chapter and verse of glTF right now, I did that research 6 to 9 months ago and I'm going on memory.  I know I'm not stupid and there's a reason I was considering that one.

other than to be especially bleeding edge in context to a 20 year old game.

Look if you just got the memo on SMAC having a proprietary voxel format that you're not going to get working in the real world, don't cast aspersions on other people's research or motives.  If you need to convince yourself that it's not actually possible or wise to reverse engineer SMAC's voxel format, do so.  The problem is obvious to any 3D person with experience, and I am one of those.  I've been paid to do it, if a long time ago.  I went bankrupt trying to write my own SMAC-like game 20 years ago, and this isn't my 1st open source rodeo, not by a long shot.

Godot is certainly not the only implementation of glTF 2.0,

I'm sure.  The question is, what's out there that's real world right now, not some moon mission of additional work before one has got any results to show for it.  I didn't get as far as researching every glTF option available.  Also 6 to 9 months ago, there were very few.  Things may have changed since then.

The idea of re-implementing the whole game using their Node system seems to me like creating something from scratch using basic game logic derived from reverse engineering, instead of repackaging the reverse engineer code into a cross-platform graphics/audio/input library.

I don't know why you think you're going to be able to get a front end for no work.  Maybe if you dig around in the SMAC binary you'll find some kind of old school dumb frame buffer implementation.  Maybe.  Maybe PRACX has some clues about how such things are put together.