Getting into modding is a painful experience for a newcomer especially when on a clock (as in: Ain’t no time for this) so here is what I understood so far.

Finding resources

EgoSoft is one of the few companies that still hold on to a forum. They may be on social media but that’s just as news outlet. This is somehow also where the modding community resides, when it usually would find a place on it’s own “elsewhere”. Clever move and I applaud – at least I don’t have to join some private group on Facebook.

Sadly the forum has no search function to speak of especially not for a technical search or, gasp, a code search. So most mod developers usually host their stuff elsewhere – like scattered on GitHub, and only talk about their mods here and there. Connecting the dots (and users) is up to the initiate. In fact it helps slightly to utilize an external search engine that does a better job compared to the forum search itself. Go figure.

Eventually I stumbled by accident or luck over a Confluence installation that seems to act as some sort of Wiki and has indeed some pointers on modding for various X titles at https://www.egosoft.com:8444/confluence/ – and I am still uncertain if this is intended to be public or not. Sadly the X4 modding articles in there are short and of course available in English only. That wouldn’t be an issue if Confluence wouldn’t stubbornly insist in trying to render a page in my native language first, informing me helpful every time that no such entry exists and makes me switch the language every single time back to English. Well, I guess it is a community driven documentation system so I could scratch my own itch and translate stuff. Thing is I should understand what I’d be talking about in the first place, no?

Turns out that most know-how for X4 modding can in fact be learnt from X3 and X Rebirth in particular. Both are precursors and partially sandboxes for X4, from my understanding, so a lot of the information does also apply to X4. Sometimes with slight differences.

The most helpful place however is an unofficial Discord channel where many of the mod developers hang out and they do seem to be really friendly to newcomers asking the same old questions over and over again. I won’t bother with the invite link as this one is subject to change all the time but it can indeed be found with the dreaded forum search.

The extension file formats (and formatting)

Anyway, let’s dive into some details. Mods, or as it should be called nowadays “extensions”, consist usually of so called MD Scripts, that’s short for Mission Director (and not Markdown) written in well formed XML Syntax, and LUA scripts (that seems to be the gaming industry standard – at least I keep hearing that). LUA itself is explained in great detail in the online documentation of LUA while game specifics are listed in varying detail on the Confluence mentioned above but that’s subject for another article. Also various assets may be floating around in the extension folders.

Now in theory XML and LUA are completely system independent so modding with different systems in mind should be a no brainer, right? Right?

Well, of course nothing is ever that easy. Thankfully the user CulunTse took the burden and wrote an article on all the caveats encountered so far: https://forum.egosoft.com/viewtopic.php?t=380912 (Steps to make your mod work on Linux+Mac) – it’s for X-Rebirth but the gist applies for X4 as well. So when writing mods make sure to use lower case only and don’t use special characters at all. Best not even use a white-space just to be sure.

<rant>Again Windows f****s us all over being the only system that is fine with "A==a" being true. Not even JavaScript manages this. And since X4 is a game and the majority of gamers (and mod devs) play on Windows we have to suffer from this yet again. The average mod is simply not compatible and broken for e.g. Linux users. Guess who gets the blame. The worst part is that another generation of developers will not see any problem with this behaviour.</rant>

So now we learnt that an extension consists mostly of XML and LUA files, so how comes that mods downloaded from Steam, e.g. to learn from, are riddled with various TXT files instead rendering your linter of choice useless because it won’t automatically parse a text file? Well that seems to be a Steam Workshop limitation not allowing certain file… extensions. So developers started to rename their files when uploaded to Steam. Yay, more confusion for man and machine (as in mankind – c’mon, it’s a lame allusion!).

Why I mentioned Steam now? Well, learning from existing extensions is the way to go. Also since some simply don’t work for Not-Windows users it’s up to myself to debug. The places to get extensions is usually from NexusMods (no thanks, still angry that they lost my user data years ago – still used by scammers) and Steam. Avid readers of my ramblings may know my especially sour spot of having purchased X4 on GOG (https://beko.famkos.net/2021/03/17/x4-cradle-of-humanity-for-linux-and-gog/) so downloading from the Steam Workshop is not as straight forward as it is for others. There are various so called “steam workshop downloader” that easy the pain somewhat.

But wait. The Steam Workshop file comes as DAT file. What is that again when I just talked about TXT files? Well, this is from my understanding a format by EgoSoft designed for Steam Workshop files for X Rebirth – also called a XRWS file. At least from https://github.com/Lighting/XRWSunpack – a tiny little project that helps unpacking the DAT files but has to be compiled before use. That’s usually a matter of issuing the make command after checking out the repository (or download the release file if you’re feeling lucky). Fair warning: It is somewhat rigid in the way the DAT file has to be named so you may have to adjust that by renaming. Ymmv.

Armed with that knowledge I was able to download and extract the awesome extensions_fireandsmoke_v107.dat extension that really spices up space fights with the effects we know and love from X Rebirth (or not). This one I could also drop in my user space folder under ~/.config/EgoSoft/X4/extensions/x4_fireandsmoke/ (again: case sensitive, important). The folder name matters because it is also hardcoded into the extension files itself and will fail to load various assets if changed. Interesting design choice.

Why this is noteworthy? Well apparently this does not work for all kinds of extensions. Some seem to work only when put in the game path /path/to/X4_Foundations/game/extensions/ where the game also stores official DLCs. Better keep that in mind. The gist seems to be that user space extensions are limited in functionality to prevent nefarious mods. Or so I hear. Maybe EgoSoft simply never got it working properly. There is more (conflicting) info hidden deep in that Confluence mentioned above.

One extension to rule them all (and a pipe)

So why go through all that trouble when an extension can simply be downloaded from e.g. GitHub? Well, I learnt the hard way: Also not as simple. For example many mods rely on one very important extension that can be found in the repository https://github.com/bvbohnen/x4-projects – a wild mix of various extensions and even a Pipe Server (more on that later). It’s the indeed impressive extension sn_mod_support_apis featuring a clever way to work around some UI modding limitations in X4, allowing lazy loading of further LUA scripts and even introduces a Pipe Server to interact with the game from the outer world – mostly used for more complex hot keys. A dream coming true and used by many other extensions as well.

Sadly it didn’t really work out of the box when checked out from GitHub and put in place at /path/to/X4_Foundations/game/extensions/sn_mod_support_apis/. There were various reasons for this. First of all: Case sensitive again. The XML files in the md/ folder must be lower case or X4 will simply ignore the files. Easily fixed though.

The next problem wasn’t that easy to identify and the reason for this is hidden in plain sight in this titbit of information from the synopsis:

A workaround is to load in custom lua files alongside the egosoft lua. This is done by editing one of a handful of ui.xml files in the ui/addons folders, adding the path to the custom lua file. These ui.xml files cannot be diff patched. The lua file must be given an xpl extension, and this xpl and the ui.xml must be packed in a “subst” cat/dat.

bvbohnen/x4-projects

Where this arcane know-how was acquired from in the first place I do not know. The gist is that some XML files are happily read by X4 (and can even be hot reloaded) while some can not. The ui.xml falls into the not so much category and since I have no idea how to create a cat/dat file (yet) I had to scrape the “subst” files from a release (Steam Workshop, NexusMod, GitHub release, you name it). Without it’s simply not read and ignored and this is also why no single debug line will ever be logged to give the (weary) initiate a hint what may be wrong.

After that mods relying on sn_mod_support_apis started working (or throwing traces at least). Awesome! Onwards to Pipes! Or Not! Because this part is Windows only. Why? Well, the Pipe Server uses a LUA feature to load a library from disk providing that pipe feature. And that project is written in C, compiled as a separate DLL and relying on Windows, of course. That makes even sense and I really can not blame the author for scratching the own itch only here. See https://github.com/bvbohnen/x4-projects/blob/master/extensions/sn_mod_support_apis/lua/c_library/winpipe.lua for details and to be fair the extension is written in a way that other features do still work so it’s not a total roadblock and in theory I can go back to be a happy gamer at this point.

Alas I want that Pipe feature, of course, so I have to come up with my own library at this point. It’s not a complex file but my C days are long past. So to spice this up I needed a crash course on how LUA is supposed to work. Script wise and all and I don’t think I was prepared for all this.

Diving into the unknown (What is LUA/JIT anyway?)

LUA is basically… ah frell, go and look it up yourself. In the end LUA scripts are interpreted by a VM. That would be LuaJIT (JIT – Just In Time) on Linux and this is why X4 is shipped with a file named libluajit-5.1.so.2. Sadly this is not the particular version of LuaJIT. It stands for an ABI compatibility version. In theory at least. My first goal to get the idea was to grab the source and compile my own libluajit. Should be a no brainer, right? Little did I know when I checked out the project from https://github.com/LuaJIT/LuaJIT.

I run a hexedit on the distributed library to get an idea what version is used and came up with 2_1_0_beta3. The commit 8271c643c21d1b2f344e339f559f2de6f3663191 of the LuaJIT project is tagged with that version so I went with that first.

Compile went smooth, beside some warnings, but X4 would stop dead throwing a Fatal Error at me that I’ve never seen before. Well, let’s fast forward to HEAD and try again and this time the game started but became stuck in main menu with unresponsive entries and missing labels (some said “Processing…”). So obviously X4 is not running vanilla LuaJIT and since this is under MIT licence I don’t think they even have to provide modifications. What now?

Thankfully some fellow gamer on Telegram, who doesn’t want to be named, pointed me in the right directions due to experience with LUA. An article over at https://developer.x-plane.com/article/luajit/ explains some major caveats with this and sure enough once I knew what I was looking for I found evidence here and here and here. (Yes, they have Jira as well – who would have guessed).

Lua engine was upgraded to LuaJIT 2.1 which comes with performance improvements as well as new language features (incl. some added Lua 5.2-specific features as well as some Lua 5.3 ones.

Just looking at the number of forks of LuaJIT on GitHub makes me dizzy so I went with the first recommendation https://github.com/openresty/luajit2 that also addresses the memory issue and also edited the file src/Makefile enabling some LUA 5.2 features by commenting in the line: XCFLAGS+= -DLUAJIT_ENABLE_LUA52COMPAT

And guess what, X4 launched with this and also started an older save game of mine just fine. I guess this works so I’ll keep that in mind in case I need some monkey patching to try stuff.

Next on that list? Find out how to write a loadable C library for LUA and adapt that Pipe Server.

At least gaming itself is easy as pie on Linux in 2021. Modding? Now so much.

I was eager for the second expansion set of “Cradle Of Humanity” since I enjoyed the Split Vendetta expansion a lot. Sunk many nights into this sandbox so I pre-ordered the DLC on GOG (Good Old Games) and watched that countdown to release ticking down. Oh boy, was I disappointed when the timer reached zero and I got nothing while people who bought on Steam already enjoyed the DLC. As usual GOG leaves Linux players standing in the rain so I called it a day and checked again on the next day when I was presented with this:

X4: Foundations patched, Split Vendetta was not, Cradle Of Humanity nowhere to be seen.

So apparently they finally managed to roll out version 4.0 of the main game but missed the first DLC Split Vendetta. What could possibly go wrong. Cradle Of Humanity is still nowhere to be seen. This didn’t change until now, one day after release and the time of writing of this article. Oh GOG, I am so done with this. And I even expected this, joking around weeks before the release date that this will be two weeks later for GOG users. Again.

Curiously I checked the Downloads for Windows next and guess what: The DLCs were all there and also on version 4.0!

So I did what every Linux tinkerer would do. I checked out what is really in the DLC files by extracting the contents using innoextract. To my delight I could not find anything operating system related in there so I threw all the Windows DLCs into my version 4.0 base game folder and extracted both DLCs.

The required files are:

  • setup_x4_cradle_of_humanity_4.00_(64bit)(45636)-1.bin
  • setup_x4_cradle_of_humanity_4.00(64bit)(45636).exe
  • setup_x4_split_vendetta_4.00(64bit)(45636)-1.bin
  • setup_x4_split_vendetta_4.00(64bit)_(45636).exe
cd /games/linux/X4_Foundations/game
innoextract --exclude-temp --extract setup_x4_split_vendetta_4.00_\(64bit\)_\(45636\).exe 
innoextract --exclude-temp --extract setup_x4_cradle_of_humanity_4.00_\(64bit\)_\(45636\).exe

When I run the game now I found both DLCs registered in the Extensions menu and could start a new game as Terrain fraction. Whop whop, here we go.

No, I will never buy a recent game on GOG again. Especially not as pre-order. I wish I could migrate this to my Steam account. This is the DRM free revolution. As usual, the joke is on the paying customer.

There’s a weird issue with (snap) on that starts when using voice chat causing really bad lag and short freezes (input, rendering, everything) that became worse over time. My journal filled up with looping messages from appindicator causing this.

appindicatorsupport(at)rgcjonas.gmail.com[2514]: discord1, Impossible to lookup icon for 'discord1_12-panel'

Followed by a JS exception and trace:

JS ERROR: Exception in callback for signal: icon: Error: Argument 'filename' (type filename) may not be null

When I finally found the cause of this I went on looking for a solution and it seems like the unsung hero @3v1n0 fixed this long standing bug like 8 days ago: https://github.com/ubuntu/gnome-shell-extension-appindicator/commit/745c66a73e0a15a870e92e5aa461e2e9e646b899

Here is a more coherent report on this: https://bugs.launchpad.net/ubuntu/+source/gnome-shell-extension-appindicator/+bug/1849142

Fun thing is: I only have that indicator because Discord would eventually crash without trying to access this.

Now it’s patched and gone – back to 😁

Man, this is a 180° turn for me. When I started out with Linux the GPUs where usually troublemakers and I kinda got used to throw moar power at it to solve the problem. Spent nights fiddling with Elsa Winner or 3Dfx Vodoo or some ATI cards (that eventual became AMD). When laptops of mine could no longer be used because AMD simply dropped support for perfectly fine hardware I was really never again buying from it again.

The background for finally ripping out the heart of my Linux PC is basically this issue: https://beko.famkos.net/2020/01/17/computer-fallen

NVRM: GPU 0000:01:00.0: GPU has fallen off the bus.

This is followed by a frozen X server rendering all HID interfaces dead until reboot. It happened once or twice a week. NVIDIA support has no idea and while the card is still fine and up for any task I finally decided to get a more recent GPU hoping that the problem will be gone (and not be an issue from the mainboard).

So here I am in 2020 ripping out the heart of my Linux PC.

The decision to try AMD again after a decade was basically made because I read so much positive news on their open source drivers and general good support by Mesa nowadays. Since nothing about the old fglrx days is valid any more this is sort of a jump into cold water for me 🙂

I decided for the slightly older RX 5600 XT 14Gbps 6GB (THICC III Pro) edition by XFX that seems to be good for 1080p gaming and this is close to my main display resolution of 1920×1200. While I never heard of XFX before I was hooked by NO RGB and that tiny vBios switch it has offering a backup bios. That’s a feature I like in my mainboards as well.

Speaking of I heard a lot of confusion on said vBioses on this series so I digged deeper on this topic. Thankfully a lot of the legwork was already done for me by André Almeida who describes the process for Linux PC on https://andrealmeid.com/post/2020-05-01-vbios2/ after a lot of research in part 1.

With the help of the mentioned tool amdvbflash I was able to drag the following vBios information out of the GPU:

AMDVBFLASH version 4.71, Copyright (c) 2020 Advanced Micro Devices, Inc.

    Product Name is :    NAVI10 A1/A2 D1990301 XLE 6GB 300e/875m 
    Device ID is    :    731F
    Bios Version    :    017.001.000.068.000000
    Bios P/N is     :    113-170WCNAVIXLE6
    Bios SSID       :    5710
    Bios SVID       :    1682
    Bios Date is    :    03/27/20 21:25 

The extracted rom of the active vBios (switch was set on position closer to power connector) has the sha1sum 9ce7ecc9625d7ff39b3b08c45916b6c2e3bf4a8c and is according to the flashing tool valid and signed. I understood it’s a bad idea to flash with an unsigned rom because the GPU will probably refuse to boot. I’d upload it to techpowerup that seems to collect such roms and allows hassle free downloading but their extract and upload tool seems to be for Windows PC only.

XFX has vBios roms for the 12Gbps variant on it’s website but currently none for this one so it seems it is up to date already.

Installing it was a breeze. I upgraded to Fedora Workstation 32 before changing the GPU to make sure I get more recent drivers and that was it. System booted up just fine and the card worked out of the box. Unlike NVIDIA I didn’t need to download a specific driver first or add some further repository. There’s this nice tool CoreCtrl that shows me a power consumption of only 14W while the card is in idle with zero spinning fans. That’s right – no noise! When not in use this card consumes next to no resources which begs the question how I’m going to heat my man cave from now on 😀

CoreCtrl in action

This surprised me as well. All the cool bits are laid out for me to play with at /sys/class/drm/card1/device/. I mean I’m not much into over-clocking but it’s all there. This brings me to benchmarking the new GPU. I’m not some YTer so you’ll only get the Unigine Superposition benchmark with basically irrelevant OpenGL (I know of no nifty Vulkan benchmark like this yet) for Linux PC gaming.

The tool picked up the wrong model (it really is a FX 5600 XT)

That’s rad! My old Titan X has it’s stronger side in other features but managed only a total result of 3055 without over-clocking on this benchmark.

I’ve not much gaming experience with it yet. I just made sure that Valheim (beta) and X4 Foundations works fine with it. There seems to be some sound issue with Discord + Fullscreen that I found so far but I solved this by switching to windowed mode on the games seeing no difference in FPS.

The one notable issue I have compared with NVIDIA is that I have to set the environment variable DRI_PRIME=1 or games will pick up the integrated GPU of the i5-8600K. This is probably because I’ve a display connected to it as well so it’s active. Going to play with this a little until I get the idea. Will need some adapters first tho 😅

Update: Just as guessed. Once I had all my adapters in place the integrated GPU was not needed any more and does no longer show up. No need to use DRI_PRIME=1 for each 3D application any more.

“Releases for PC”. Oh we Linux gamers know this phrase well. So it seems that is taking another approach now instead of the usual “PC is not Windows”. Liam changed the wording on the news slightly:

Beyond a Steel Sky to release for Linux PC during July

https://www.gamingonlinux.com/2020/06/beyond-a-steel-sky-to-release-for-linux-pc-during-july

I like that approach and I shall also use “Linux PC” in the future. And not just for my .

 

Things were going smooth for a while. The pirates didn’t stop harassing my ships so I decided for drastic action. We slacked their station in the system.

The Scale Plate Pact station is no more

That sure put a damper on their actions in the sectors nearby. Surviving ships were hunted down, boarded and taken over. It was glorious. The fleet was unstoppable for a while and with each carefully picked battle it grew.

Gate duty helped to train the fleet further and a lot of sweet loot to improve and modify ships further could be collected.

The highlight was two Xenon K jumping into the sector to be annihilated by my fleet of two destroyers, plenty of small fighters and an auxiliary ship used as bullet sponge.

Battling Xenon K at the gates

This was when I decided to contact the Split while extending my mining business all over the galaxy big time. I started with a bad reputation towards Split – that’s kinda default for most races in the X4 universe – so I had a hard time to make them like me… somewhat. This was mostly achieved by destroying enemy ships near to their stations.

Meanwhile I bought several Magnetar mining ships so keep the needed raw materials on my prospering stations coming. Everything was going nicely and I decided to claim the sector Heretics End.

While I was busy building my stations to do so my new not-exactly-enemies decided to go to battle in freakin Argon Prime. Four Rattle Snake destroyers went basically unchallenged into the Argon heart and took out the Wharf. I didn’t want to interference since I just made not-exactly-enemy with the Split. From here it went downhill fast. Almost all of Argon Prime was wiped out – and I had bought most of my ships there. This! Hurt! Business!

I had not much time to mourn though. While I was busy with my new stations in Heretics End I was ended by a very surprising Xenon attack myself. First two more K ships showed up that I hardly managed to destroy – especially since most of my fleet was busy rescuing some Teladi trading station from another K.

While I was busy counting my losses _it_ showed up. The worst nightmare X4 has to offer: The Xenon Branch 9 Destroyer I entered the sector and all I had to offer were two half built stations, some Interceptors and my badly damaged Argon Behemoth destroyer.

Xenon Branch 9 Destroyer I

I tried to keep my distance while waiting for my fleet to arrive. This worked for a while but it eventually was going to destroy my station so I tried to get into it’s blind spot. That was my last stand. Out of mines and turrets, low on interceptors and badly damaged my destroyer went down with me on the bridge.

I accept this glorious defeat – for now 😀

I wonder for a suited format for a series of log entries for my X4 adventures. Mayhap I should simply edit and append as the story [of my sandbox] unfolds.

Spent most of the day hunting pirates that seem to develop a taste for my ships. The salvaged Cerberus Vanguard is a huge help with this and whenever a pirate surrenders my own support fleet of Minotaur grows. Did I mention that I just love the design of this ship? It’s such a great all-round asset that can even haul some cargo on occasion or act as small carrier.

I also stumbled over another really tiny vault. Took some time to unlock all it’s secrets but I’m starting to get the hang of it.

Raiding a tiny vault

Talking about raiding I had some notable encounters as well. A Raider tried to steal from the current head quarter. That was a mighty stupid idea and the Raider was toast before I even got close to it.

Scale Plate Pact Raider going down in a blaze

And another had the nerve to attack my medium freighters during their duty of making profit. When I finally had enough I called in the fleet and together we started a boarding operation. That was a long and ugly fight mostly because due to a bunch of Xen fighters trashing our party. Lost one small heavy fighter and 45 marines but in the end the ship was ours. Good thing the Cerberus brought a repair drone along to fix the busted engines, by the way.

I also identified a choking point in Napileos’ Fortune VI where a lot of the pirate traffic seems to originate. It’s an empty and unclaimed sector so I started building an Administrative Center here hoping to get this pest under control.

Claiming Napileos’ Fortune VI

I wonder if it makes sense to start more stations here, because a lot of other huge stations are within 4 sectors of this – the typical maximum distance for auto traders to go – and a lot of traffic is passing through this already.

Oh yeah – and I finally got all the parts to assemble the SINZA device. It’s IMHO not really obvious that it simply activated with Shift+4 because I tried to install it as modification or ship drive first.

I played the visual novel game Ken Follett's The Pillars of the Earth. I admit I didn’t even look up the details before when I got all three parts dead cheap as a bundle. I’ve a very faint memory of reading the book/s but that’s all. It looked like a decent point-and-click adventure available for and the idea was to play it via Steam Link in the living room with the kids around. Hint: Don’t do that.

The story is an emotional roller-coaster not shy of splattering blood all over the scene. There are dramatic moments where decisions have to be made [in time] but also peaceful chapters and fun moments. Some have to be spotted and can be missed. The story builds up slowly based on character development and decisions made. Or so it feels. All strings come together in the end and some scenes may change in detail but the overall outcome is probably the same. I’d have to read up on this or do another play-through to be sure though.

It’s not a difficult game. There are no riddles (minigames) to be solved. The only minigame included is some sort of timing game (“quick action”) where one has to click at the right moment. That was mostly annoying but mercifully simply reset the scene when it really mattered so one could try again.

Depiction of a town in 12th century England

The character style may be an issue for some. Animations are not very smooth and there seems to be no lip sync. Sometimes the animations don’t fire at all. The audio however is very good and makes up for this. Music and scenery are awesome. A lot of research went into this, unlike most games, and the depiction of 12th century England looks adequate [to me]. Since this is a hobby of mine I’m really thrown off if this does not match up in games [or movies]. I also catched the vibe of architectural love for cathedrals that I can relate to. While it’s timberframed buildings for me I can certainly understand the fascination. I visited Guédelon some years ago after all 😉

So if you like stories for your do yourself a favour and get this game. It’s worth every penny.

As Cities: Skylines comes up to the five year mark, it's on a big sale by Liam DaweLiam Dawe (gamingonlinux.com)
Paradox Interactive and Colossal Order are celebrating Cities: Skylines, as it's coming up for the five year mark since it took the city-building world by storm. What a storm it was too, not much can really come close to just how fun Cities: Skylines actually is. Frankly, it's become the definitive ...

Cities Skylines is a very special game. I sunk way too many hours into this and still enjoy it on occasion.

There’s a nifty plugin that allows to overlay a png image with transparency so one could hobble together a map with imported heights data from e.g. NASA and overlay it with streets, rivers and train tracks from e.g. Google Maps.

This results in recreation of real cities within the engine bringing the hardware to it’s limits.

There are also hundreds of downloadable assets in the workshop.