Blog 3.0

2025 Retrospective

Mon, 29 Dec 2025 00:00:00 +0000

More Of The Same

In my last retrospective I talked about developing Ephemeris and learning Blender, those have continued to be the dominant themes for 2025 too.

Ephemeris Kessler: Orbital Warfare

~~Ephemeris~~ Kessler: Orbital Warfare is a realistic simulation of near-future space combat. If this sounds like something you’d be interested in playing, come and chat with us on Discord!

I’ve been calling this idea “Ephemeris” for years, this was only ever a working title and it was always my intention to come up with a new name eventually. Unfortunately another game named Ephemeris was released this year and that forced me to come up with something. I’ve settled on Kessler: Orbital Warfare.

I have been keeping daily notes on progress since mid 2024, and I intend to keep doing this. You can find the 2025 notes here. Personally I’ve found taking notes like this has three primary benefits, in order of importance:

On days when I’m feeling like I haven’t achieved much I can look at the huge list of work achieved recently and realise that I am actually making good progress. This *really helps with motivation.
Since I write down everything that I do, I have become much more aware of what work is involved in a project. This really helps me estimate the scope of work in advance, and correctly estimate how long it will take.
When working on a system that I haven’t touched for a while, all of the notes are still there. For me this is the last important point because my notes aren’t well structured as documentation, and I very heavily comment my code so that’s usually a better reference.
It makes writing the next section of this post is much easier!

2025 Progress

I’ve made some great progress in 2025, each of these items probably represents around 2 weeks work with many smaller items getting done along the way:

Selected multiplayer framework (Mirror)
Multiplayer lobby system
Multiplayer entity management (spawning/destruction)
Orbital data sync over the network
Multiplayer time control
UI development (building core UI code)
Physics sim for thermals
Tactical view
Core systems simulating sensors
AESA RADAR sensor, using this system
Bullet simulation (capable of simulating tens of thousands of bullets)
Bullet rendering
Core systems for collision detection (Octree, Sweep & Prune)
Multiplayer sync for bullets
Deep research on armour damage models
Built armour designer minigame (IMPACT). More on this below.
Clicking on orbits
Core systems for HUD
Engine burn gizmo (like the KSP manoeuvre node gizmo)
Scheduling engine burns using gizmo

Overall I’m incredibly happy with progress in 2025! For years I have been working on building out a lot of bits of technology that I knew I would need for a realistic space game; full scale solar system simulation, rendering of planets, n-body orbital integration, rendering lines etc. This year it feels like that has changed - many of the features I listed above were building on top of that core tech, it feels like I’m finally building a game instead of a space-game-engine. I think that trend is going to continue in 2026, lots of plumbing together existing features into a game.

2026 Goals

My major milestone goal for this year is to release an alpha version of the game with basic multiplayer orbital combat in the first half of 2026. If you’re interested in playing that alpha, join us on Discord!

IMPACT Minigame

In previous years I’ve taken part in the CYLON gamejam and I’ve built some games I’m really proud of (see my games list). Unfortunately the CYLON community was based around a game called STARBASE which has died, and CYLON has died with it. I decided to work on a minigame for a month instead.

IMPACT (Impact Modelling Program for Armor Configuration Testing) is a spacecraft armour design and test program, styled as an alt-history 1970’s joint DoD/NASA program to develop armour for orbital warships. the game is available to download for free here. Join us on Discord to show off your designs!

This is based on the research I did into the physics of hypervelocity impacts in September and uses the damage model I have developed for Kessler.

Blender

In mid 2024 I started learning Blender, and in 2025 it’s become a really major thing for me. I’ve learned a lot, and I’m starting to feel like I could actually build game-ready assets at a usable quality level!

Open Source Software

Myriad.ECS

Myriad.ECS is a high performance ECS built in C#. I use Myriad in Kessler, so it’s used every day and has a lot of useful features that I’ve discovered the need for.

Changes in 2025

Lots of optimisations for speed and memory usage
Disposable components receive a callback when an entity is destroyed
Hierarchical transform system
Various internal optimisations for systems that walk entity relationships (like a hierarchy)
Delegates support in map/reduce queries
Queries which collect results into a collection
PhantomNotifier components receive a callback when entity becomes a phantom
Resumable queries with cursors
HasComponent overloads for checking multiple components all at once
Overloads for delegate queries that take no type params (filtered entirely by QueryDescription)
Delegate chunk queries
Added ability to enable/disable system groups
Unsafe accessors for entities/components in chunks (useful for integrating with Unity job system)
Delayed entity resolving from CommandBuffer (add entities to a list when the buffer is eventually executed)
Tests, tests and more tests. I find it fun to drive up test coverage!
Global chunk/archetype IDs
Job safety system (basically per archetype locking). Designed to integrate with the Unity safety system.

LLamaSharp

LLamaSharp is a C# wrapper around llama.cpp. This is not my project alone, but I became one of the lead maintainers in 2023 and I’ve continued working on it this year.

Fox4

Fox4 is an experimental AI pilot for VTOL VR, built on the AI Pilot framework built by Strikeeaglechase. Fox4 was intended as a project to learn about reinforcement learning - the core idea was to learn air combat through self-play, in the same way as AlphaGo learned Go to a superhuman level. The pilot part is built in C#, that collects data logs and the Python part learns from those logs. Unfortunately I wasn’t able to train the pilots to maintain level flight, even with a lot of rewards tweaks. I’d like to come back to learning about RL sometime (maybe through this project, maybe through something else).

HandySerialization

HandySerialization is a binary serialization library I have developed for use in game networking, it provides precise control over byte-level and bit-level packing of data.

I did a lot of work for this in April when I was developing the encoding scheme for orbital data, the final result is a general purpose lossless predictive encoder for sequences of doubles, which can compress orbital data down to about 33% of it’s original size! No doubt next time I work on anything with multiplayer this library will see more features added.

Wasm2cs

Wasm2cs is a compiler from WASM to C# code.

This isn’t a serious project that I’m using for anything, but it’s a cool idea so I wanted to share it. Imagine if you could take a native C++ dependency, compile it to WASM and then import that into a C# project as plain old C# code. No native dependencies, no WASM engine dependency. I think this idea came about when I read about WasmBoxC which is a system for sandboxing bits of C code by compiling C -> WASM -> C. The final code is just normal C code you can use, but it’s sandboxed!

The end result of wasm2cs works for a simple program that writes "Hello World" to the terminal, but I never managed to get it working for more complex programs like the asyncified Python runtime for Serpent.

Interesting Stuff

Some other interesting things that I encountered in 2025 in no particular order:

VRChat
Bigscreen Beyond 2
Banner of the Stars
- I’ve seen it before, but I rewatched the anime and reread all of the Manga this year. A couple of my Blender projects have been BotS ships.
Anime Cels for sale
Realsense D421
- Traditionally 3D cameras are very expensive, but this one is just £75!
VTubers
- While watching tutorial videos on Blender for how to build a VRChat avatar I came across this video. That sucked me into the world of VTubers!
- Some favourites:
  - Smellizabeth, currently rebranding to Aeri Andromeda. Gaming, Blender, art.
  - Pikat. Art, gaming.
  - Ellie_Minibot. Robotics engineer with an interest in AI.
  - Vedal987. Develops the AI Vtuber Neuro-sama, fascinating AI project!
  - Ironmouse. Obviously.

2024 Retrospective

Sat, 28 Dec 2024 00:00:00 +0000

Sticking With Unity

In my last retrospective I talked about the big Unity backstab and how I was considering moving away from Unity as my primary game engine. Fortunately Unity recovered from this pretty rapidly and they seem to be on a much better course now, so I have decided to stick with Unity.

Ephemeris

Ephemeris is a simulator of near-future realistic space combat. If this sounds like something you’d be interested in playing, come and chat with us on Discord!

In 2022 Ephemeris was really just a series of vaguely related side projects. I was playing with things like accurate integrators and rendering of astronomical scale scenes in Unity. In 2023 this became a bit more serious, I was still working on Placeholder Software as my main job, but I was more seriously experimenting with Ephemeris related things to see if it was feasible. In 2024 Ephemeris has become my full time, bringing together a lot of those prototypes into one coherent project.

I have been keeping daily notes on progress since June this year, and I intend to keep doing this. Now that I’ve been doing it for a few months it’s hard to imagine not keeping notes like this! You can find the 2024 notes here. So what did I work on every month?

June

One major aspect of Ephemeris is multiplayer. In May and June I experimented with a custom networking system called HandyNetworking. This was intended as a flexible front-end which could be plugged into different backends (e.g. LiteNetLib, SteamNetworkingSockets, Epic Online Services etc). I didn’t really like the end result of this experiment and I scrapped it, multiplayer is something I’ll come back to look at later (in Novemeber 2024).

July

In July I did a lot of work setting up the scene framework in Unity. Working out what layers I need for various things - for example very large things like planets and very small things like spacecraft are on different layers and are composited together to appear as if they’re in the same shot. This also involved building a manager for the camera system, which handles switching modes from looking at large things (planets) to looking at small things (spaceships).

I also ported across an old experiment for rendering symbols, using instanced rendering to draw thousands of symbols together. This will be used a lot in Ephemeris - showing all of the other things in space such as ships and missiles. In fact a game like this is almost entirely about symbols on the map!

August

In the previous months I had been experimenting with a FUI (Futuristic User Interface) framework to use in Ephemeris. In August I started developing a small game, using this framework for the HUD. This game was a docking simulator, where you have control of a pod in space and must dock with a nearby space station.

This ended up being a great project - it helped development of the FUI framework as I had to polish bits to make them usable and develop entirely new UI controls as needed for the game. Additionally I play tested the game with some friends and learned a lot about UI design by paying careful attention to how they used the UI, what bits they missed, and what things they misunderstood.

My initial prototype just used the built in Unity physics but ideally I want something separate from Unity for the main game, so I also investigated various other C# physics engines. BEPU would be my preferred choice, but it’s not compatible with the runtime version available in Unity. Jitter2 has the same problem, but if we go all the way back to Jitter1 there is a C# port to netstandard2.1. The code of this port is very messy, so I started refactoring this, removing unneeded features and cleaning up the code (e.g. using .NET vectors instead of a custom vector type).

September

For most of September I was participating in the 4 week long Cylon Gamejam. See Cylon Game Jam for details on the game I built. Once I got back to working on Ephemeris at the end of the month I resumed the Jitter refactoring and experimented with some ideas for thruster control (determining which thrusters to fire for a target movement/rotation).

October

Given how much I felt that I learned about UI design and play-testing from the docking simulator game in August, I decided to build another docking sim minigame. This time inside of the main Ephemeris project using all of the technology I have developed for the game - this means the pod and the station would be on true n-body orbits, the entire solar system would be in the sim, there would be properly sized planets visible in the background etc. This would be a good way to do a “shakedown test” on all of these components I have built.

A big part of this turned out to be about improving precision. The Ephemeris simulation uses a variable timestep for the n-body simulation - timesteps can be as small as 100ms or as large as 10 minutes. Of course rendering needs to show something once every 16.6ms (60 FPS), there is an interpolation system that takes the nearest two points from the integrator and calculates the current position. Due to the absolutely enormous speeds and positions involved (e.g. 30,000m/s at 149,597,870,700m from the Sun) a tiny percentage error in this interpolation can result in huge absolute errors. Normally this isn’t a problem, but if you have 2 bodies orbiting near each other - for example during docking - those errors can cause the two bodies to oscillate back and forth.

At the start I was using a quadratic bezier interpolation with positional oscillations of about 500m (approximately 1.5% error relative to the velocity). I developed a new “kinematic interpolation” system which assumes constant jerk (change in acceleration) which pushes that error down to about 0.1m, which is about 0.0003% error!

Another source of error is engine burns - when a burn is created at short notice the points being interpolated can suddenly change during interpolation. Rather than a smooth wobble this causes a sudden jerk. Improving the magnitude of the wobble reduced the jerkiness but did not entirely eliminate it, so I added another system which detects this jerk and basically smooths it out over multiple frames.

November

If October was all about deep technical work for the docking sim minigame, November was the opposite. I spent all of this month working on high level things like menu UI, HUD UI, sound effects, visuals, settings management, scene management etc. This work was slow going, but I learned a lot and all of these things are of course necessary for the full game!

December

I started another prototype for multiplayer, this time using Mirror instead of my own framework. Mirror is more tied to GameObjects than I would like and this isn’t a perfect fit for Ephemeris (which uses my own ECS), but it’s a much more robust system which I’ve already used for multiple games in the past.

The Future

In 2025 I’ll continue working on Ephemeris, with the aim of getting an early-access prototype out to some players. If you’re interested in that, join us on Discord!

Cylon Game Jam

This year we ran another Cylon community gamejam. This year we all suggested a topic and then voted on them to generate a ranked list of themes. Every game required the top theme and could freely pick at least one other from the list:

Running Out Of Space (Required)
Asymmetry
The Space Race
Light
Outer Space
Primitive Primitives
Reveal
Switch Off
Stringa Superstes
Paper
Car Alarms
No way back

Each year we do one of these jams I set myself a personal goal that I consider almost impossible to achieve, to really push myself. My game this year was a pretty unique idea that I’ve never seen done before - a battle royale real time strategy game! Players all start off on a plane and drop into the map wherever they want, there are various pickups scattered around the map which they fight over, the map slowly shrinks and the last person alive is the winner.

A lot of my previous gamejam games have been playable and fun, but things like menus were very lacking. One thing I really wanted to get right this year was to fix that and have a great game-feel all the way through. I picked a style I like - a pixel art retro-style inspired by Command & Conquer - and really worked hard to make the whole game fit that artistic direction. For example the menu was something I kept coming back to and tweaking - it’s the first thing anyone will see about the game and making it look professional goes a long way to setting expectations!

Overall I’m very happy with how the game turned out. It’s a genuinely fun game and as far as I’m aware is a fairly fresh take on real-time strategy. Obviously there’s more that could be done to develop the game - I really wanted to add more unit types - but as a 3 week gamejam entry I think it’s a great success.

If you would like to try the game out it is available to download here. If you want to play multiplayer you will have to open port 7777 on the host.

Blender

One of the problems I have always had with my games is art. I’m capable of building the technical parts of a game, but I can’t really make my own art, so I always have to use a very basic art style with pre-existing assets I can purchase. I decided to start on the path to solving this problem by learning blender. Obviously I don’t really expect to be able to make all of my own assets - that requires real artistic skill that takes years to develop - but if I ever want that I’d better get started now! I’ve made a lot of practice pieces this year:

Neptune’s Pride (Collective Game)

At the start of the year I played a game of Neptune’s Pride with a group of 11 players. NP is a fantastic strategy game - the mechanics are simple enough that no one is going to lose a game due to not understanding the mechanics, but deep enough to be interesting. Winning or losing a game of Neptune’s pride ultimately depends almost entirely on your ability to conduct diplomacy, organise alliances (and maybe backstab at just the right moment).

In this game I spawned right in the middle of the map, which is a very difficult starting position. I had to ensure I had good diplomatic relations with almost all of the players in the game at all times! My empire was very long and thin so was very exposed to invasion everywhere - there was no single “front line” I could defend.

The following video is the entire game from my point of view (it may skip some ticks if I forgot to record the game at that tick).

0m 0s: Spawn, I’m yellow circles
1m 0s: Invade green circles
1m 10s: Discover green squares (west)
1m 10s: Ally with light blue squares (north)
1m 25s: Ally with light blue circles (east)
1m 30s: Begin a coordinated invasion of dark blue circles with my ally light blue circles
1m 31s: Continue invading green circles in the south as well, coordinating with green squares to not accidentally attack them
1m 45s: Ally with green squares
2m 30s: At some point I discovered that blue squares had betrayed me (he was offering other players money and tech to attack me). Begin a coordinated invasion with my allies: orange circles, green squares and blue circles.
3m 31s: While everyone else is cleaning up the last of blue squares I begin a lightning invasion of red circles.
3m 57s: Tensions with blue circles were high, as a “warning shot” I destroy the last dark blue circle “haven” system. This triggers war with light blue circles.
4m 45s: green squares invades dark blue squares, I use this as leverage to ally with dark blue squares.
5m 10s: I finally manage to persuade dark blue squares to backstab his ally light blue circles. This breaks the organised resistance and allows orange circle and myself to start making real progress in the north east.
There are now 2 major alliances. Both alliances are in the process of invading smaller empires, it becomes a long stalemate at this point.
- Me (Yellow Circles) + Orange Circles
- Green Squares + Purple Circles
8m: We break the alliance and begin attacking green squares. My fleets push deep into his core worlds in the west while orange circles slowly advances from the north. There are skirmishes along the south east line with some planets exchanging hands but nothing major.
10m 30s: The original core worlds of green squares have been entirely conquered, invasion of purple circles (who was allied with green squares) starts.
My main concern now is orange circles backstabbing me. I am making slower progress at the front because all over the map I have large garrison fleets building up. I deliberately setup my main resupply lines to pass right through/near core orange worlds so I could rapidly divert and invade if necessary.
Victory!

Open Source Projects

Myriad.ECS

Last year I built PROTOLOGIC, a programming game that I was building outside of Unity specifically to learn new engines. One of the things I used with PROTOLOGIC was an ECS (Entity Component System) called Arch.

I like the overall design of Arch except for one major issue - whenever there is a trade-off to be made between a dangerously unsafe API and a tiny bit more speed, they will always choose speed. I wasted a lot of time chasing down very weird bugs in PROTOLOGIC caused by this - a mistake in one bit of code can corrupt the world and then cause completely unrelated bugs elsewhere.

To address this, I started my own C# ECS project: Myriad.ECS.

I’ve been using Myriad.ECS in my projects this year, including Ephemeris, and I think it’s a very nice framework to work with. The API doesn’t include many footguns and offers several different querying techniques which are suitable in different circumstances. There are some special component types that have special powers such as; phantom components (making tracking of destroyed entities easy), disposable components (making resource management easy) and relational components (naming setting up links between entities easy).

Serpent / Wazzy

Wazzy is an implementation of the WASI spec (WebAssembly System Interface). It is written in pure C# and is designed to be easily extensible with custom implementations of parts of the spec.

Serpent is built on top of Wazzy. Using Serpent it’s easy create an entirely self-contained Python environment. The Python code executes within WASM and asynchronously yields back to C# at opportune moments (e.g. if time.sleep(100) is called that will not block C#). My original goal with this was to create a safe Python environment which can be used as a “tool” for language models to write an execute code.

LLamaSharp

LLamaSharp is a C# wrapper around llama.cpp. This is not my project alone, but I became one of the lead maintainers last year and I’ve continued working on it this year.

In 2024 my major contribution to LLamaSharp was the development of the BatchedExecutor which is an entirely new low-level abstraction around language models. The BatchedExecutor is designed to expose all of the power of llama.cpp in a safe way, for example multiple parallel sequences evaluated all together in one batch is as simple as:

var executor = new BatchedExecutor(/* parameters here */);

var seq1 = executor.Create();
seq1.Prompt([ /* tokens */ ]);
var seq2 = executor.Create();
seq2.Prompt([ /* tokens */ ]);

await executor.Infer();

var logits = seq1.Sample();

Sequences can be easily saved and loaded, forked into 2 sequences with the same prefix (which internally share the same space in memory), the KV cache can be accessed and manipulated (e.g. to implement context shifting), sequences can even be prompted with embeddings directly which allows things like LLava.

My long term goal for 2025 is to rewrite many of the higher level parts of LLamaSharp to operate on top of the BatchedExecutor, this will reduce the overall complexity of the project by implementing it all in one place and should offer more power to advanced users since they can always build on top of BatchedExecutor instead of using the low level llama.cpp primitives.

Interesting Stuff

Some other interesting things that I encountered in 2024 in no particular order:

Suzume
Land of the Lustrous
Dune 2
The Orbital Children
Sousou no Frieren
RWBY
- RWBY is something I’ve been familiar with since it came out, but I had basically forgotten about it. When I heard that season 9 was out I rewatched all of the series and then season 9. Once I finished that they announced that Rooster teeth was shutting down - this reminded some friends about RWBY so I ended up participating in a complete rewatch with them as well so I watched the entire series twice back to back! Let’s hope for news about Season 10 in 2025.

2023 Retrospective

Sat, 30 Dec 2023 00:00:00 +0000

Ephemeris

In my last retrospective I talked about starting serious work on Ephemeris. Ephemeris is an orbital combat simulator game, with near future technology, attempting to be as realistic as possible.

I started off the year experimenting with an n-body integrator in Unity ECS. Eventually settling on using an RKF45F integrator which allows the timestep to change constantly, doing the least computation required to satisfy your accuracy requirements. You can read more about the integrator in Ephemeris Notes.

After that I started working on… drawing lines! This might sound trivial, but there’s actually a huge amount of complexity in drawing lines and it’s critically important for a game like Ephemeris which will represent almost all game state as lines and labels on those lines! These lines need to be fast (there’s a lot of them), fixed width in screen space (they can’t be in world space, scale is too big in space), not suffering from aliasing (it looks awful), capable of being marked (e.g. points of interest or regions of interest). I experimented with various approaches to line rendering to solve all these problems: GPU rendering from a compute buffer, building meshes on CPU, signed distance fields for grids and pure GPU pixel perfect bezier curve rendering.

Cylon Game Jam

50: Exploration
49: Time
47: Sacrifice
46: Once more, with feeling
45: Navigation
41: Unlikely Combinations
39: Computers/Circuitry
36: Don’t Stop Moving
15: Musical

As usual I massively overdid it and picked the top 4 themes and also decided to make the game a multiplayer game! Ephemeris is a multiplayer game, so this seemed like a good opportunity to build a complete multiplayer game end-to-end.

The game is a twin stick, top down, co-operative, dungeon crawler. The twist on the co-operative dungeon crawler theme is that every time the party dies it rewinds back to the start and you respawn. At the same time copies of yourself replay all of your previous runs at the same time! This means you have to co-operative with yourself to solve various puzzles in the dungeon.

A game like this requires a huge amount of work creating the animation state machine, controlling movement (walk, run, dodge roll, dash), attacks (sword slash, big sword slash, sword jab, fire bow, spear jab etc), defence (shield block) and reactions (hit, big hit, stunned, death) and all of the various transitions between these states. Here’s that in action:

I’ve never used the animation state machine in any of my games before. My games are usually much more like “simulations” that you can interact with, and an animation state machine is not required for that. It was great to learn this tool.

Much of the game ended up becoming driven by this state machine, player inputs would simply trigger animations and if I wanted to adjust the gameplay effect of that (e.g. make a sword slash happen a bit faster) I could just adjust the state machine. I’m sure this is how many games are made, but it’s new for me!

Overall I’m very happy with how the game turned out. The concept is simple, but very fun to play with and people had a lot of fun playing the game in different ways during the demos. For example some people played it solo, some people just explored the dungeon, some people kept killing the same enemies over and over making the fight more and more of a mess, some people tried to speed run it (shortest overall length) and others tried to speed run it (shortest time from final spawn to exiting the dungeon). I think it’s a great sign that people enjoyed the game enough to do all that! I think the concept could be developed into something worth selling, but this kind of game requires a lot of art assets so it’s not something I’m likely to do.

If you would like to try the game out it is available to download here. If you want to play multiplayer you will have to open port 7777 on the host (it is possible to play solo, since you can always co-operate with yourself from another timeline when necessary).

By the way one of the rooms in the game will slow performance to a crawl, if you find it you’ll have to start a new run since performance will be ruined on all future runs too! Gamejam quality software ;)

Obsidian

Last year I started writing the Ephemeris Notes, a set of notes on all things related to Ephemeris. I found this really helpful for organising my thinking on the project, writing down scattered thoughts so they can be refined later or writing detailed articles on complex topics so that I myself can re-read them later when I’ve forgotten the details!

Inspired by that experience I started using Obsidian for general note taking this year. I haven’t fully embraced the note taking, and in fact my use of the notes has fallen off a little recently, but it’s something I’m going to keep working at in 2024.

Placeholder Software

Dissonance

This year we’ve released 4 updates for Dissonance with a number of improvements:

Fixed an issue with resuming audio playback after the app is suspended (e.g. put into background on a phone).
Many smaller fixes related to resuming from backgrounding (e.g. resetting various counters).
Added a new IAudioOutputSubscriber interface, which makes it easy to access (and modify) to the audio Dissonance is playing.
Fixed Amplitude property for the local player so it measures after preprocessing (most importantly, after gain control).
Fixed spelling of SubcribeToVoiceActivation. Only took 6 years for someone to spot that!
Added additional metrics accessible on the server (e.g. packet loss).
Added an “always send” room that forces audio to be sent to the server (even if no one is listening).

Wasmbox

Last year I mentioned that were were developing “Wasmbox”, a WebAssembly runtime for Unity. Wasmbox ended up developing into a very cool project that could do lots of cool tricks. A WASM file could be imported in through the Unity asset pipeline, this import process automatically applied Wizer initialisation, Binaryen optimisations (including asyncify), Wasmtime precompilation (i.e. skipping runtime JIT cost entirely), Brotli compression and auto generated C# wrapper code.

The C# wrapper code was particularly useful, it enabled us to make sure that a loaded WASM instance is not “misused”. For example a call can be made inside a job and the entire instance is unusable while the job is running (i.e. an extension of the Unity “safety system” to arbitrary WASM calls).

You might notice that this section is written in the past tense, well that’s because of the Unity shaped elephant in the room…

Unity Backstab

Back in September Unity announced a sudden and unexpected pricing change to a totally insane pricing structure based on installs. Everyone in the games industry hated this fee structure and the backlash was incredible to watch. 10 days later they announced a complete replacement of this install fee with a new policy which is much more reasonable. This was a total disaster for Unity, and we still haven’t seen all of the fallout from this decision.

At Placeholder Software we decided to rollback on developing new products (such as Wasmbox, which we have deprecated) and instead to transition to just supporting our existing projects (Dissonance is not going anywhere). This is because if they can decide to just roll out totally insane policies like the runtime fee overnight then Unity no longer seems like a safe foundation to build assets and games on.

Of course this has left me with a big decision to make: do I trust Unity long term and keep developing with Unity (which I do consider to be a good engine)? Or do I migrate to another engine and start learning from scratch again?

At the time this was happening I looked around and found three potential candidate to replace Unity:

As of right now I have not decided which engine to use, or if I should stick with Unity. However, what is clear is that I can’t learn a new engine by working on a project of such complexity as Ephemeris! This has directly led to me starting a new game…

Protologic

Back in 2021 my entry for the Cylon gamejam was Saturn’s Envy. This was a programming where you you used Yolol (the language which Cylon is all about) to program space battleships. When you had a program you could submit it to a Discord bot which would schedule battles and maintain a leaderboard of ships. This was pretty fun, and quite a few people submitted fleets, but it was always held back by the fact that Yolol is a terrible language and no one wants to build large complex software in it!

Protologic re-imagines this idea, the ships now run WASM. That means that the ships can use any language which compiles to WASM, of which there are many! If you’re interested in this the Protologic bot is currently available in Cylon, and the battle replays are available to watch online here.

This is a great project to learn new engines because the sim (running battles & generating replay files) is completely decoupled from the front end (loads replay files and displays them in a game engine). This means I can build a completely new “front end” in another engine without having to worry about the complexity of re-implementing all of the game logic. The current frontend is written in Unity (adapted directly from the gamejam submission) and I plan to write new frontends in Godot and Stride.

Open Source Projects

Wazzy

Wazzy is an implementation of the WASI spec (WebAssembly System Interface). It is written in pure C# and is designed to be easily extensible with custom implementations of parts of the spec. This was originally part of Wasmbox, but it seemed a shame to let it die, so I’ve extracted out everything except for the filesystem implementation into this open source project. This is in active use in Protologic.

Autofocus

Autofocus is a C# API client for the AUTOMATIC1111 API. It attempts to model the API in a type-safe way, preventing possible user errors. This is in active use in *Mute (my Discord bot).

LLamaSharp

LLamaSharp is a C# wrapper around llama.cpp. This is not my project! Back in July I tried to use LLamaSharp in *Mute but I quickly encountered some errors and submitted some PRs to fix them. 295 commits later and I’ve become one of the lead developers on LLamaSharp!

Interesting Stuff

Some other interesting things that I encountered in 2023 in no particular order:

2022 Retrospective

Sun, 01 Jan 2023 00:00:00 +0000

TL;DR

Started working seriously on a new game: Ephemeris.

What Did I Do In 2021?

In my last retrospective I talked about a major redesign of Overcrowded and mentioned starting on a side project (over Christmas) named Ephemeris.

Placeholder Software

Dissonance Voice Chat

This year we’ve released 7 updates for Dissonance with a number of improvements:

Added a new audio synchronisation system which detects extreme packet loss and briefly interrupts audio playback to fully reset the system.
Added a brand new way to do positional audio, based on spatial hashes. This does not scale cost with the numbers of listeners, so it can handle huge crowds.
Fixed some strange MacOS bugs to do with marshalling between C++ and C# components. It seems like Unity used to do it technically wrong and we relied on the old but wrong behaviour, so when they fixed it we ended up with a regression.
Added x86_64 support for Android (this used to be for emulators only, but it seems like some Chromebooks are now x86_64).
Added a way for server side scripts to intercept raw audio packets. We used this to integrate with Toxmod for a client.
Implemented a new way to subscribe to the raw audio stream from the Dissonance microphone.
Added Magic Leap 2 support.
Added Apple M1 support.
Vastly improved audio quality of RNNoise.
Added an “offline comms network” which makes Dissonance think it’s in a chat session even when there’s no network.
Two major new integrations: FMOD Recording and FMOD Playback

New Audio Subscriber

This is an interesting feature not because of what it is, but rather due to the motivations behind it and the lessons learned. Ever since release Dissonance has had a way to subscribe to the raw audio that the Dissonance microphone is recording, some people have used this to show a frequency spectrum as you speak, or to record everything you say to a file.

This subscriber system was always built to be high performance and not particularly easy to use - all the audio is delivered on a background thread (the audio recording thread) which must not be blocked for a long period of time (e.g. doing IO). The documentation explained this and included a very quick-n-dirty script to show how you could transfer audio across to the main thread. We didn’t expect anyone to actually use this script, it was just an example and not something suitable for real world use.

Inevitably every time someone contacted me asking for support about this feature their code would include a copy of the example code, adapted to their use case. I’m definitely not blaming the users here - in fact copying examples from the documentation and adapting them is the right way to do things if you’re unfamiliar with a system! This one was totally on us not understanding that example code should always be production quality!

Toxmod

Toxmod is an automatic content moderation system for voice. It monitors realtime voice streams and flags potential bad behaviour for moderators to review, acting as a filter to cut out the 99% of content which isn’t a problem. When moderating voice this is critically important!

Unfortunately this isn’t available as a plugin for Dissonance since it was developed for a client. But I would encourage anyone else who is serious about moderating voice chat in games to consider Toxmod.

Overcrowded

For the last two years I’ve talked about the struggle to release “Overcrowded”, our asset for steering behaviours and local avoidance. If you check the Placeholder Software store page you’ll see that it has still not released!

As I said last year we had to initiate a total rewrite of the asset due to Unity announcing that ECS would not be available on some versions of Unity 2021. With hindsight this was a mistake because in the time it has taken us to do that rewrite ECS has become fully stabilised and released version 1.0 - we should have just waited. Eventually in mid 2022 we decided to put the project on hold, both of us were getting pretty burned out with the project and initiating another rewrite back to the new ECS 1.0 was definitely not feasible.

We may revisit the project next year, possibly releasing something much smaller than originally planned as a first step in releasing several smaller assets which work together to provide the complete Overcrowded feature set we had originally planned.

WasmBox

Our next project after Overcrowded is a total change of direction: we’re developing a WebAssembly (WASM) integration for Unity. This is based on wasmtime-dotnet and extends it with a number of features such as safe job system support, precompiling WASM at build time etc. We’ll be releasing this very soon, feel free to contact me (martin@placeholder-software.co.uk) if this is something you’d be interested in testing out.

Blogging

As you’ve probably noticed I wrote no new blog posts in 2022, sorry about that. However, I have recently started a new project that’s a bit like a blog: Ephemeris Notes. These are my notes created during the development of Ephemeris on all sorts of relevant topics: games, films, TV, rocketry, weaponry, integrators, floating point accuracy etc. I’m actively adding new content to this all of the time and some of what I write is like a mini article/blog post explaining something to myself. For example this article about floating point precision where I explore the benefits of using different unit scales.

Ephemeris

Ephemeris is a game project that I have been working on as a side project all year. Ephemeris is intended to be a realistic space combat tactics simulator akin to Children Of A Dead Earth. As mentioned above I’m writing development notes on the project, so if this sounds like the kind of game you’d be interested in check those out and maybe join the Discord.

I worked on one version of the project on and off throughout the year, using an asset store package called Gravity Engine to drive the core simulation and Space Graphics Toolkit for most of the graphics. I made some really great progress, putting together a realistic map of the entire solar system with full support for modding in new bodies all driven with an on-rails simulation for the planets and an n-body simulation for the spacecraft.

This was as much a research/learning project as much as it was a serious attempt to make a game. One of the things I learned it that while Gravity Engine is pretty good it really doesn’t fit with how I want things to work - I really want a fully deterministic simulation which runs in the Unity ECS for maximum performance.

As an experiment just before Christmas 2022 I started on a project to write my own gravity system in ECS, just to see how hard it would be. When I started I honestly wasn’t sure if I was going to use it, in fact I expected to discover that it was all vastly more difficult than I expected and to return to using Gravity Engine. Instead I’ve written a hybrid N-Body simulator (all gravity sources such as planets and moons are on rails, all space craft are n-body simulated) which spreads the load across all cores and has a variable timestep to reduce the amount of work necessary. I’m very happy with this and I’m going to be moving across all of the work I did previously to this new project.

This has given a new lease of life to the project - I can now see a path to a high performance multiplayer orbital combat simulation which would never have been possible with Gravity Engine. I’m excited to get working on this in the new year!

Save The Spire

This year we ran another Cylon community gamejam. This year we all suggested a topic and then voted on them to generate a ranked list of themes, this left us with an extremely opened ended theme:

1: Magic 2: Factory 3: Space 4: Indirect Unit Control 5: Zachlike 6: Multiplayer 7: Princess Of Mars 8: Yolol 9: Completely html/CSS/JavaScript game

This year I decided to do a very weird project that I didn’t really expect to work very well as a game. For me that’s the great thing about a gamejam - I can build something weird and wonderful to explore a new bit of game design without having to commit to it or really have any expectations of it.

The game is called “Save The Spire”. You discover a huge magical (1) structure floating through the sky (3) which appears to be some kind of arcane (1) factory (2). There are some units standing on the structure which you cannot directly control (4), instead you can place down new walkways (which the units will walk over to and build). There are various clues hidden around the factory as to how it works and it’s your job to piece them together and restore the factory to working order.

You can download it to try it out here.

As a game this didn’t really work (and that’s fine!). I think with some more careful game design and art direction it could maybe be made to work. There were a few common complaints from all the players:

The start of the game gives no direction at all. The game is meant be to mysterious and encourage you to explore, but it’s a bit too mysterious at the moment.
Golem spawner (little block with a floating Darwinian in it) looks identical to a golem (a normal Darwinian) so it’s unclear it’s something special.
It’s a bit repetitive. There are multiple elements that need connecting up and they are all a bit different, but not enough. I simply didn’t have enough time to create more variations.
It’s too simple. Once you have worked out that connecting things together makes things happen you can win by just connecting everything to everything. There’s never a reason not to connect two things!

Overall I have to say I’m quite happy with this feedback. Everyone agreed the core of the game is something unusual and interesting. The negative feedback is mostly stuff I had anticipated - it just needed a lot more tweaking and maybe one more major mechanic to being it all together (which I didn’t have time to do in a jam).

Projects

Margarine

This is a 3D game for the PicoSystem - and RP2040 based handheld console. It uses a Wolfenstein style raycast renderer to draw 3D (ish) environments. Even at the low resolution of the PicoSystem (240x240) it’s driving the RP2040 CPU as hard as possible to manage this!

SolarBlaze

This year we had a solar system installed (panels and battery). The inverter is obviously a computer system capable of monitoring the system, recording statistics, reporting them back to the manufacturer etc but we’re completely locked out of it! Instead the only way we can monitor the system is to use the app which is provided - that can’t be used on PC, has no option to export the data and is generally quite rubbish.

Fortunately someone has built a project called LXP bridge which interprets the inverter messages and logs them to an MQTT endpoint in a useful format. I’ve used this to build SolarBlaze, which is a Blazor based front end to monitor the data. It’s quite rough at the moment, but once the sun is shining again I’ll get back to working on this and will consider publishing it as a project anyone can use (with proper setup instructions etc).

Interesting Stuff

Some other interesting things that I encountered in 2022 in no particular order:

2021 Retrospective

Thu, 30 Dec 2021 00:00:00 +0000

TL;DR

Overcrowded had to be rebuilt. COVID19 sucks.

What Did I Do In 2020?

In my last retrospective I talked about how Overcrowded was nearly ready for release, learning KiCad and the failure of Bounded Planet project.

Placeholder Software

This year we’ve released 7 updates for Dissonance with a number of improvements:

Add RNNoise support.
Worked around a weird Unity linker bug which broke Dissonance logging in Thailand/Argentina.
Added a low pass filter to attenuate high pitched sounds (above the normal range of human speech).
Modified some power “mobile” platforms to use the desktop AEC, since it’s higher quality.
Showing dBFS in the UI everywhere (See below).
Increased priority of Dissonance AudioSource components (so the Unity audio engine never mutes them).
Not playing silence when not speaking, so Dissonance isn’t using up so many audio engine “voices”.
Added support for capturing and uploading realtime “metrics”.
Various synchronisation fixes (every time we fix this someone finds a new and exciting way to get out of sync).
Improved AGC handling of long periods of silence (no longer amplifying nothing up so that next time you speak it deafens everyone).
Added realtime diagnostics to AEC, accessible through scripts.

As you can see no huge new changes here - Dissonance is quite stable and generally works well in most situations. We’re just adding small new improvements and features (or adding a large ones, such as RNNoise, as optional extras).

dBFS

Previously we’d avoided showing decibels anywhere in the UI. dB is a very misunderstood unit and so we tried to avoid mentioning it anywhere - for example the fader controls were all 0 to 1, which indicated the actual number the signal would be multiplied by. However, this itself is very confusing since the input is linear but the perception is non-linear! Instead we’ve decided to swap to showing dBFS everywhere in the UI - even if you don’t understand what this means (decibels below full scale) it’s more intuitive.

Overcrowded

Last year I mentioned that we’d been working on “Overcrowded” since mid 2019 and it was nearly ready for release. It hasn’t released yet, so what went wrong?

Early in the year we were still working to tweak the old local avoidance system which was based on ORCA (Optimal Reciprocal Collision Avoidance). ORCA is quite a well know algorithm to solve the local avoidance problem, but it seems to have a major problem. In theory it provides optimal solutions to local avoidance, but in practice optimal solutions do not always look natural and when the scenario gets so overcrowded that there is no good solution (or the best option varies wildly from frame to frame, as agents move around) ORCA totally breaks down and looks awful. We had been trying to patch this up for a while with various system to detect poor behaviour and work around it, but we weren’t ever happy with the quality - there was always some new corner case which looked bad.

Eventually Tom invented a completely new local avoidance system which is significantly better than ORCA in practical environments. It’s simpler, faster to evaluate and has the concept of falling back to suboptimal solutions built right in to it. It also handles solving the problem from frame-to-frame better, with agents “sticking” to their previous solution (even if it’s no longer optimal) as long as it’s good enough. Obviously this took a while to design, implement and test so it delayed release quite a bit, but it was definitely worth it!

We were just about ready to release again (around September) when another roadblock occurred. Unity had announced that some bits of DOTS (such as ECS) would not be compatible with Unity 2021 during the beta phase. We had hoped that this would be resolved by the time we came to release, but unfortunately not. We decided that we couldn’t release Overcrowded, which completely depends on the ECS, while ECS was in such an uncertain state. So we’re now rewriting Overcrowded again, with a reduced feature set, this time not depending on ECS at all (but still using the Job system etc). Hopefully we’ll be able to release this in Q1 of 2022!

Blogging

I started a series on building a toy compiler for Yolol, Inspired by my work on Yolol.IL (which compiles Yolol into dotnet bytecode and achieves millions of lines/second). I plan to do a couple more entries in that series on compiling to IL and some optimisation tricks.

Projects

Yolol.IL

I’ve really had fun building Yolol.IL, but I think I’m coming to the end of what I can squeeze out of it in terms of speed. Yolol really isn’t a very good language to optimise - if I wanted to get into incredibly complex analysis and optimisation there’s more I could do but I’ve intentionally tried to keep Yolol.IL very simple (mostly single-pass) for the sake of maintainability.

What has been particularly interesting with this project is it’s seen a huge amount of use in my Discord code-golf challenge (a bot presents a challenge, users can solve it by submitting code), running hundreds of millions of lines of code this year which means that it is one of my most battle tested projects. However, even with all of that when I ran a fuzzer against it I still found a number of bugs. After every change I have run the fuzzer for a couple of hours before updating the code golf challenge and in several cases users have reported bugs. It really goes to show that avoiding bugs in a system like this is nearly impossible without a huge amount of testing!

YololBlazor

This was a fun little project (link) I threw together recently to explore Blazor - a system for building a web applications in C# (compiled to wasm). It’s a debugger for Yolol which saves it’s state in the URL so states can be linked to other people. My code-golf bot uses this for feedback - when a submitted solution fails it links out to the debugger with the state of the program encoded in the URL so it’s as if you hit a breakpoint when it fails!

Blazor is a really impressive project which removes a lot of the pain of building web projects (aka having to touch JavaScript) and I will certainly be using this for future web apps.

Raspberry Pi Pico

Early in 2021 the Raspberry Pi Pico was released. Unlike all other raspberry Pi boards this is not a Linux computer running on a broadcom SoC - instead it’s a custom designed chip (RP2040) and it’s a microcontroller. This is the first time RPi have designed their own chip and I’m very excited to see where they go in the future with this - my expectation is that the RPi5 will be based on a custom SoC.

The Pico particularly caught my imagination because although it’s intended as a microcontroller it has similar power to early games consoles. The Pico can overclock to 250MHz and has 264KB of RAM with 16MB of flash storage, for comparison the Sega Saturn (released 1994) has a 28.6MHz CPU (10x slower) and 2MB of RAM (10x more). Now of course this isn’t a direct comparison - the games consoles usually had custom chips to accelerate graphics and the Pico does not. Nonetheless the similar power really caught my imagination and I started some game development work on the Pico, eventually completing a port of one of my earliest VB6 games (Toast): PicoToast.

I plan to develop more games for the Pico. I have a vague plan of specifying a “console” assembled out of specific parts with free 3D printable housings available which other people can develop for. A bit like a fantasy console (e.g. Pico8) but with some actual hardware!

VBToast

Way back in 2018 I developed Toast in Visual Basic 6 - this was one of my earliest games which was genuinely fun and had other people playing it. For years I thought I had lost the source code when the hard drive in our family PC died but earlier this year I found a copy of it on an ancient backup drive! To ensure it never gets lost again, the source code is now on GitHub along with a downloadable binary.

I found this immediately after finishing PicoToast, so the similar floaty movement in PicoToast is entirely from reverse engineering the compiled VBToast version I still had a copy of.

Saturn’s Envy

This year someone from the Cylon community ran a gamejam with the rules:

Set in space
Somehow related to programming

I created Saturn’s Envy. This is a game in two parts:

The simulator accepts the code (written in Yolol) for two space ships, runs the battle as fast as possible and saves the results to a replay file. The player takes replay files and renders the battle. I integrated this into my Yolol code Discord bot so players can submit fleets through Discord and query for leaderboards etc. Battles are saved on the server and can be viewed with a WebGL compiled version of the player, available here.

This was a great project and I really enjoyed the gamejam format. It was a very slow gamejam (about a month) so it gave time for everyone to talk about their ideas and work on them without having to drop everything else for the duration of the jam. However, it was a short enough time that I had to slash off intended features every day of development just to make the deadline. I learned a lot about managing the scope of games, which is always a major problem for indie games!

Ephemeris

With that experience in mind I have just recently (as my Christmas holiday project) started on a new game called Ephemeris. This is intended as a realistic space combat tactics game/simulator akin to Children of a Dead Earth but less janky/better graphics/multiplayer/wider range of tech etc.

Interesting Stuff

Some other interesting things that I encountered in 2021 in no particular order:

Building An Interpreter

Sat, 27 Feb 2021 00:00:00 +0000

TL;DR

The second stage of compiling a language is interpreting it.

Compiler Series

This article is the third in a series on how to build a compiler for a simple language into CIL.

What Is Interpreting

todo

Building Language Primitives

todo: Number/Rope

Walking The Syntax Tree

todo: ast walking visitor pattern

Yolol Parsing

Fri, 26 Feb 2021 00:00:00 +0000

TL;DR

The first stage of compiling a language is parsing it.

Compiler Series

This article is the second in a series on how to build a compiler for a simple language into CIL.

What Is Parsing

A parser turns a list of characters in a human readable form, such as 1+2*3/4, into a machine readable form (an Abstract Syntax Tree).

The syntax for a language is often defined in Extended Backus Naur Form which is simply a list of rules and which other rules can come after that one. For example an ENBF for a decimal number might look like this:

# 'x' means match the single character "x"
# [x] means `x` is optional
# x,y means `x` followed by `y`
# {x} means `x` can be repeated 0 or more times
number = [ '-' ]
       , digit
       , { digit }
       , [ '.', digit, { digit } ];

# x|y means `x` or `y`
digit = '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9';

This simple grammar defines two rules: number and digit. This grammar matches string like these:

"1" : digit
"-2" : [ '-' ], digit
"12" : digit, { digit }
"21.4" : digit, { digit }, [ '.', digit ]

But does not match string like these:

"" : digit is required
"-" : digit is required
"a" : digit must be 0..9
"9." : digit is required after '.'

When the parser fails to match can be almost as important as when it does match! You’ll want to take these syntax errors and give them to the user in a useful way. For example:

"9."
   ^ Expected `digit` but encountered end-of-file

Abstract Syntax Trees

I mentioned above that a parser usually outputs an Abstract Syntax Tree (AST). An AST is a tree representation of the code.

In this example you can see that the expression 1+2*3/4 has been broken down into a tree of nodes which each represent one part of the expression. Exactly how you represent this depends on your language - in a language like Rust you’d use an enum with a variant for each expression type. Unfortunately C# doesn’t have discriminated unions (yet?) so it’s a little less convenient:

class Number : IExpression {
    public double Value { get; }

    public Number(double value) {
        Value = value;
    }
}

class Divide : BinaryExpression {
	public Divide(IExpression l, IExpression r) : base(l, r) {}
}

abstract class BinaryExpression : IExpression {
	public IExpression Left { get; }
	public IExpression Right { get; }
	
	public BinaryExpression(IExpression l, IExpression r) {	
		Left = l;
		Right = r;
	}
}

interface IExpression { }

This is a very simple AST, a real world one might contain some extra information. For example my Yolol AST classes have an IsConstant property which indicates if the node represents a constant value (3+4 is constant because both 3 and 4 are constants but 3+n is not constant because n is a non-constant variable).

Parsing Options

todo: PEG/LALR/etc

Do You Really Need A Parser?

So now that I’ve spent all this time explaining what parsers are… do you actually need one? If you want to write your own completely original language as a string the answer is simple: yes. So for Yolol I definitely need it.

However, if you’re trying to write a Domain Specific Language (DSL) which is embedded in another language you might not need one at all.

The simplest option is to skip the parser and directly construct the AST. This can be quite ugly, but it completely avoids all of the difficulties associated with parsing and pretty much eliminates the possibility of a “syntax error” (there’s no syntax!).

// a = (17.3 + a) * 4
let ast = Assign(
    Identifier("a"),
    Multiply(
        Add(
            Number(17.3f),
            Identifier("a")
        ),
        Number(4),
    )
)

Another option may be to derive the AST from some other structure that you already have. For example the JIL library accelerates JSON serialisation by compiling a method on-the-fly specifically for the type of JSON you’re handling. It doesn’t need to build an AST, instead it can inspect the AST of the JSON and convert that.

{
    IntVal = 1,
    StringVal = "hello world",
}

Could be walked over and turned into an AST automatically:

let ast = JSON(
    Field<int>("IntVal"),
    Field<string>("StringVal")
)

Finally in C# there’s a fairly unique third option: Expression Trees. Expression trees allow you to write standard C# but instead of that code being compiled into executable code the AST from the C# parser is embedded directly into your program, so you can inspect it at runtime. This basically allows you to directly use the C# parser as your own parser, as long as you want your syntax to look like C#.

Building A Toy Compiler

Thu, 25 Feb 2021 00:00:00 +0000

TL;DR

I’ve been building a compiler for a toy language, it’s fun and surprisingly simple.

Compiler Series

Over the last couple of years I’ve been keeping an eye on Starbase, an in-development game about designing and building spaceships. In particular I’ve been a member of CYLON a server dedicated to exploring the in game programming language “Yolol”. Developing tools for Yolol has been a fun platform for me to explore the world of parsers, optimisers, type checkers and compilers. In this series I’m going to explain how I built a complete Yolol->CIL compiler which runs code about 100x faster than a basic interpreter.

You may wonder: “Why should I read this if I don’t care about Yolol/Starbase”? The point of this series isn’t really Yolol - instead it’s an overview of how to compile a fairly simple (although not trivial) language to CIL so that it runs really fast. Hopefully these techniques could be applied to more useful domain specific languages to vastly speed up certain programs.

An Overview Of Yolol

Although Yolol is not the main focus of this series I’m going to give a quick overview of the language, just to establish a basic understanding of things.

A Yolol program is limited to 20 lines with just 70 characters per line. A line is executed every 0.2 seconds. Any errors in a line (e.g. divide by zero) simply cause execution to fall through to the next line, this means that the layout of code on lines and correct handling of error is critically important to the semantics of the program! Many commonly used tricks in Yolol programs are based around deliberately triggering errors to break out of a tight single-line loop.

i=10                            // Setup loop counter
/* do work */ x/=i-- goto 2     // Keep executing line two until div zero error
                                // Fallthrough to here when loop is complete

A line of Yolol code consists of a list of statements separated by spaces. A statement can be:

goto line (e.g. goto 7
assign (e.g. field=value)
op assign (e.g. field+=value)
modify (e.g. field++)
if (e.g. if a then statements else statements end)

if x then i=10 else i=5 end i*=2 x/=y goto3     // You can use as many statements
d=c-- x/=i-- goto2                              // as you like as long as they
o=d goto1                                       // fit into 70 characters.

Yolol expressions are fairly basic. All of the basic mathematical operators are available (e.g. add/divide etc) as well as some comparison operators (e.g. greater than/equal to etc). There are also some “keyword” operators which take a single value and return a single value (e.g. sin x).

i=10
x=i*sin(12-7) y=x>i

Yolol only has two types: Number and String. Numbers are 64 bit with a fixed precision of just 3 decimal places! Strings are slightly odd - they can be manipulated with many (not all) of the normal mathematical operators. Combining strings and numbers results in the number being converted into a string before the operation happens.

Since Yolol has no indirect addressing at all (no arrays or pointers) strings contain some of the most powerful operations available in the language! Decrementing a string removes the last character and returns the rest of the string. Subtracting a string from another string removes the last instance of the second string from the first.

str="Hello" x=str--                             // x=="Hello", str=="Hello"
left="Hello, World" right="o" sub=left-right    // sub=="Hello, Wrld

All comparison operators simply return 1 for true and 0 for false. When evaluating if a value is “true” (e.g. in an if statement) the number 0 is false and all other values are considered true.

if 0 then this_will_not_happen=1 end
if "hello" then this_will_happen=1 end
if "0" then so_will_this=1 end
x = not "Hello, World"                  // x==0

Simple, But Not Trivial

As you can see Yolol is a fairly simple language with some interesting features. It’s simple enough that I can write all of the tools necessary to work with it, but it’s just complex enough that writing a those tools and optimising them is a fun challenge.

If you’re interested in learning more about the language a member of the community has written an excellent guide to getting started. After that, join CYLON and say hi!

2020 Retrospective

Thu, 31 Dec 2020 00:00:00 +0000

TL;DR

Overcrowded is nearly ready for release. COVID19 sucks.

What Did I Do In 2019?

In my last retrospective I talked about releasing Fancy Folders, beginning development on Overcrowded and getting into the Starbase community.

Placeholder Software

This year we’ve released 11 updates for Dissonance with a number of improvements:

Fixed strange inspector behaviour when editing prefabs (due to a change in how Unity handle prefabs)
Fixed the editor crashing when inspecting the AEC filter (due to a Unity bug)
Fixed muted players not being unmuted instantly
Fixed an issue that could cause voice packets to be dropped or long periods of time!
Dropped support for Unity 2017.4
Improved VU meters in inspector
Removed support for spatializer plugins (due to a change in Unity)
Improved precision of audio sync system
Improved audio quality in poor network conditions
Improved Automatic Gain Control handling of silence
Added better support for custom microphone systems
Released a free integration with NatDevice
Added support for Windows+ARM64 (e.g. Hololens 2)

As you can see there haven’t been any huge changes this year. At this point Dissonance itself is quite stable and most of the changes are small improvements to a specific subsystem or fixes to workaround issues in Unity! I’m overall pretty happy with how Dissonance is at the moment.

We’ve also released 3 updates to Wet Stuff:

Fixed a loading loop that could lock up the editor (due to a change in Unity)
Fixed an ArgumentOutOfRangeException in the instanced rendering system
Removed some (harmless) warnings that were confusing users
Added a warning about using 2D mode in scene view (incompatible with Wet Stuff)
Dropped support for Unity 2017.4
Added support for “Mesh” shaped decals instead of just sphere or box shaped

Again no huge changes here, mostly just small tweaks to improve the overall user experience. Mesh type decals are a pretty major feature from an end user perspective, but actually were quite simple to implement!

Overcrowded

Since mid 2019 we’ve been working on this huge new project. Overcrowded is intended as a full toolkit to handle the movement of agents in the scene. It does pathfinding (currently using the Unity pathfinder, but we’d like to replace that in the future) but that’s not really the main focus of the project. The main focus is what comes after pathfinding: how to get agents to move naturally along paths. This is split into two main systems: global planning and local planning.

Global Planning is where you look at the high level goals of the agent (e.g. follow this path) and then come up with a way to achieve them (e.g. walk towards the next waypoint). This is something you’re familiar with if you’ve used anything like steering behaviours. However, basic steering behaviours have serious problems when you try to compose them, for example they can cancel out and cause the agent to move nowhere. You can add hacks on to try and fix this, but they’re really just fundamentally broken. Overcrowded has a much more advanced system of “Context Aware Steering Behaviours” which aggregates all of the steering impulses into a “context” and then picks the best direction to move from that context - it will never get stuck with two impulses cancelling out. The context also naturally supports negative steering behaviours which cause an agent to not steer in a given direction - that’s not the same as actively steering away from it it’s simply discouraging moving in that direction. This actually turns out to be incredibly powerful and as I’ve used the systems I’ve often found myself mostly using negative steering behaviours combined with just one or two positive behaviours.

Local Avoidance is where the agent looks at the immediate area around it and decides where it should walk for the next one or two steps. This might sound pretty simple but is actually quite complex and is incredibly important to natural looking movement - humans moving in a crowded will tweak their movement direction to avoid collisions with other people, taking into account how those other people will also tweak their movement to avoid collisions. Initially when we started working on Overcrowded we thought that we’d simply be able to implement an algorithm directly from some academic papers on the topic, but it turned out that all of these algorithms had bad behaviour in various different edge cases which made them unacceptable to use in a game. On top of that the performance of many of these algorithms left a lot to be desired! Instead we’ve developed an entirely new local avoidance system that is faster, simpler, cleanly handles most bad cases, is easily extensible and is even built with GPU computation in mind!

“That sounds awesome, where can I buy it?”, you’re thinking. Well… you can’t yet. We’re very close to an initial release, so watch this space. If you’re really interested in trying out this system please send me an email (martin@placeholder-software.co.uk) expressing your interest and I can probably give you an alpha version to try out. We really appreciate the feedback :)

Blogging

Well, I did better than last year at least with my blog post summarising “The Yard Sale”, a game design technique developed by Zac McClendon which I’m a big fan of. I wrote this blog post to explain the technique to a team I wanted to use it with for…

Bounded Planet

Bounded Planet was a game development project with the people from the Cylon Discord server I spend a lot of time hanging out in. The idea grew out of a discussion about an MMORTS I used to play called Boundless Planet. We came up with a plan to write a similar game together in Rust using the Bevy game engine. We made some decent progress with a basic network system, adapting EGUI to render in Bevy, simple landscape generation and rendering and some camera movement.

Unfortunately we could never agree on a good design and the project is mostly dead. All of our design sessions basically degenerated into arguments about how much the game should be a clone of Supreme Commander. The strength of the Yard Sale technique is that it allows you to define some guiding principles and then iteratively create/discard features until you arrive a set of features that satisfy your overall design principles, often that final feature set may not be at all like what you initially imagined which is a good thing - it means you’ve discovered some new ideas! However, if this isn’t kept in mind by everyone involved it becomes an argumentative process as everyone tries to redefine the core principles to support their pet features. I think the “Yard Sale” technique really works better if you’re all physically present and moving the cards around on a pinboard.

KiCad

Lockdown sucks and everyone needs a hobby to get through it. I’ve taken up PCB design and soldering of the completed boards (ordered from JLC) to pass the time. This is a really fun combination of browsing datasheets and electronics stores for interesting parts, learning enough about electronics to design the schematics and then routing the traces on the actual board. Here are a few boards I’ve designed this year:

I have a long term goal of designing something good enough to sell on Tindie. I don’t really expect to sell very many, but it’d be cool to have something good enough to manufacture and sell in (very small) runs. Maybe some kind of hat for the Raspberry Pi400, if I can work out what functionality to put in it!

Interesting Stuff

Some other interesting things that I encountered in 2020 i no particular order:

The Yard Sale

Wed, 16 Sep 2020 00:00:00 +0000

TL;DR

This game design technique is great. I’ve summarised how it works in this blog post.

What Is The Big Board

The Big Board is a practical framework for holistic design iteration detailed by Zak McClendon in this video.

In a highly creative project like a game the project will often start out with a poorly defined game concept and each member of the team will have different ideas about exactly what that concept means. Furthermore there will be a huge number of desirable features that are not critical to the concept but are desirable. Resolving the inevitable creative conflicts between incompatible ideas and low priority ideas is critical to a well designed game. The Big Board is a process to help with this - the most important part of it is not the final artefact (a board full of cards) but is instead the collaborative process of creating the board. The big board helps everyone on the team understand the creative tradeoffs being made, reduces the scope of creative differences and helps tackle scope creep.

The Process

The Big Board process is a simple 6 step process. The steps aren’t strictly ordered and any step can be repeated or done out of order, whatever fits the discussion the best. The main goal of this process is to act as a framework for discussion amongst the team.

Prompt Team with the highest level game design concept. e.g. “An MMORTS”.
Generate ideas.
- Every idea that anyone comes up with should written down on a card.
- Ideas can be very specific technical features such as “the camera should support strategic zoom” or high level concepts such as “the game should promote non violent alternatives to warfare”.
Classify cards into categories (see below for more detail on categories).
- Some cards may be difficulty to classify - break them up into smaller cards and classify all of those. Sometimes it will turn out that all the smaller cards fit into the same category and can be merged back into a single stack of cards that are treated as one card.
Discuss every single card, writing an exact definition of what it means.
- This definition stage is one of the most important parts of the process - it builds a common vocabulary for the team.
- Some cards will be difficult to define - break them up into smaller cards. This reduces the problem to smaller and simpler problems. Sometimes a set of cards can be merged back together into a single stack of cards that are treated as one card.
Rank all cards.
- This stage exposes conflicts within the game design such as Replayability conflicting with highly scripted storyline. Keeping conflicting cards both on the board is fine, this stage is just establishing priorities for different cards.
Repeat 2->5 as many times as necessary.

Benefits

This process and final board full of cards have a number of benefits. The board lives on the wall of the office, creating a visualisation of the scope of the entire project which is visible to everyone. The entire design is no longer living in the head of several designers.

Because the board is created entirely by a collaborative process of collectively generating ideas and then ranking them no one feels left out. If an idea is rejected it was for clear reasons to do with prioritising other things above it. Conversely because there is an easy way to downrank ideas without discarding them out of hand it is natural to drop features than are “nice to have” but aren’t critical to the overall game concept.

Taking It Further

The Big Board isn’t a one off design meeting and shouldn’t become some bits of cardboard on the wall that everyone ignores. Every time someone comes up with a new idea they can collect a few people together to insert it into the correct place on the board and discuss the merits of this idea.

As the game is developed the board should also be frequently reviewed to check that what’s being built is in line with what was expected. For example are creative promises being fulfilled? If not, why not? If the current game is fun without that promise can it be removed? if so, which other features should be removed?

Another potential use of the big board is to generate new ideas by making hypothetical changes. For example if you take your least important creative promise and move it to the top what changes? During the process of investigating you may discover some interesting new things about the design space.

Common Conflicts

While developing the big board you are likely to encounter some of these common problems.encounter.

Inability to order two cards. This likely indicates a core conflict about what different members of the team want from the game. Very important to resolve or else the final game will end up poorly defined between these two things.

Holy Grail cards. These are cards which are just written as trying to fix problems with other games. These aren’t good promises and are quite likely to be impossible to achieve. For example Dwarf Fortress but accessible - DF is only fun because of it’s incredible _depth_m which is in direct conflict with accessibility. Rewrite holy grail cards into positive promises and then discuss them as normal.

Conflicts. Sometimes two cards will directly conflict with each other. For example Excellent Linear Narrative and Replayability. This is fine, one of the cards will be ranked above the other and that makes it clear which features you need to compromise on to achieve both (or which one needs to be sent to the cornfield). Potentially you could add a card to the Foggy Notions category related to combining these two things somehow.

Mixed Messages. Sometimes two cards don’t conflict in the sense that they cannot both be implemented, but simply send mixed messages. For example a creative promise to show the horrors of war is in conflict with a feature card to brutal melee takedowns. Conflicts like this don’t have to be resolved to complete the game, but it’s worth being aware of the Dissonance and trying to remove one of them for a more consistent experience.

2019 Retrospective

Tue, 31 Dec 2019 00:00:00 +0000

TL;DR

I released Fancy Folders, worked on an emulator/optimiser/compiler for an in game language (Yolol) and learned about PCB design (KiCad).

What Did I Do In 2018?

In my last retrospective I talked about working on Wet Stuff For Unity and the death of the Parsec project, a game development project which we worked on for the latter half of the year.

Placeholder Software/Fancy Folders

One of the big changes to Placeholder Software this year came from attending the Unity Unite Europe conference in Copenhagen. This year there was an asset store publisher meetup - it was great to meet up with other people in the same business as us, and to put some faces to some Discord handles! While attending there we heard the same advice from everyone we talked to - open a discord server - so we did. There were lots of smaller tips too, some of which we’ve quietly put into action and some of which will probably be the basis for big changes in the business in the future.

At the start of 2019 we released Fancy Folders onto the asset store. This was an experiment into making a much smaller and cheaper asset than our previous products with a much broader appeal (every single person who uses Unity could benefit from Fancy Folders). As an experiment I have to conclude that it failed. The asset is cheap, which means that we need to drive a lot of sales to make it worthwhile. It’s also highly integrated into the Unity Editor which makes it a lot of work to keep up to date.

Marketing is also not exactly our area of expertise - both of us at Placeholder Software are software engineers and we barely do any marketing for our other projects, ultimately referrals through the Unity Asset Store drives almost all of our sales. This works for quite unique assets with immediately obvious value such as Wet Stuff or Dissonance but doesn’t really work for something like Fancy Folders.

While working on Fancy Folders we’ve continued to provide support and updates for Dissonance. Dissonance is now very stable, with the reported issues generally being quite rare (and hard to track down) or caused by a regression in a version of Unity. We pushed out 7 updates in 2019 with a number of improvements:

Improved Editor Appearance (better VU meters)
Improved Editor Experience (better “dirty” marking of components)
Added VAD sensitivity support
Added assembly definitions
Added runtime permissions checking for Android
Added a workaround for a Unity regression (Dissonance would fail to load about 10% of the time!)
Added better handling for missing/corrupted DLLs (run with disabled functionality instead of crashing)
Added support for Magic Leap (thanks to ML for lending me a device to test on!)
Added network backend for Mirror
Added ultra low latency support for LAN use (a surprisingly frequent request)

As you can see there are all fairly small iterative improvements to a very stable base, which is a situation I’m very happy about!

Overcrowded / Unity DOTS ECS

Since April we have been working on a big new project (working title: crowds) which is for simulating the movement of NPCs in games - a little bit like A* Pathfinding Project but with a focus on much larger groups of NPCs moving freely or in Formation.

We knew right from the start that simply using gameObjects was not going to be viable for the kind of scale we want to achieve. Initially we developed it using gameObjects and components as a kind of “front end” for the developer which accessed data in a simulation object, internally the sim was built as a special purpose ECS which scheduled jobs to process buffers of data. This was a total nightmare - managing ownership of data buffers in a way that satisfied the Unity job system was a constant source of pain and working around the job system safety checks was a constant source of brittle code. We persisted with this approach for about a month but was obvious that it was seriously hurting our pace of development and would never result in code that meets our standards for release.

So we swapped to the Unity DOTS ECS. This was something we were very reluctant to do - part of the reason that we killed Parsec was the technical risk introduced by the Unity ECS being a constant moving target which required us to frequently refactor every single system in the game. Additionally most Unity developers are not using ECS, don’t understand ECS and don’t plan to move to ECS (soon) so we didn’t want to make Crowds be an ECS only asset. This has resulted in a bit of a strange hybrid - the core of crowds is entirely in the ECS and scheduled with jobs (it scales with number of cores very well) but the “frontend” for creating and interacting with agents is still gameObjects and components. This hybrid is an evolution of what we used in Parsec, so it’s nice that something good has come from that work!

Blogging

I’ve been ~~really bad~~ nonexistant at updating the blog this year. I’ve encountered plenty of interesting programming problems with both my work/Unity projects as well as some very interesting side projects. I’ve simply not been in the habit of blogging about these things, I really hope I can get back into writing about things next year.

Starbase/Yolol

Back in May Frozenbyte announced a new game coming soon called Starbase. This is an open world space game with a big focus on detailed engineering of ships (power/data/fuel/structural integrity are all important aspects). This really looks like my kind of thing, so I was very excited.

Another aspect of the game is a language called Yolol which can be used for some limited scripted of devices within the game. They put out a wiki which included a detailed description of the language and so within a couple of weeks of the announcement I released an emulator for the community to use!

I also joined a community that emerged from the Starbase community called CYLON which is primarily intended as a place to learn Yolol at all levels, basic syntax up to designing a full in game interplanetary networking internet using Yolol powered routers. Although CYLON still has a main focuss on StarBase and Yolol it’s also evolved into a fun place to discuss software development with like minded people.

Open Source

I’m a huge fan of open source - I have published 145 GitHub repositories over the last 10 years!

Yolol

This has been my biggest side project of the year. Initially it started off as a basic parser/interpreter for Yolol. After that I started adding utilities for inspecting and transforming the syntax tree which soon evolved into basic optimisation passes which rewrite Yolol into better Yolol. After that I read a bit about how compilers work and added a Control Flow Graph analsysis pass to the system which can be used for far more complex optimisations. Most recently I have been experimenting with converting small parts of the CFG into an SMT solver (Z3) so I can drive optimisations based on the logical deductions of the solver.

I didn’t really know anything about how the internals of compilers worked at the start of this year, so although this might sound like a lot of work for a toy language it’s been worth it as a learning project. Compilers are some of the most complex and important tools I interact with on a daily basis, it’s nice that they’re not quite such a black box any more!

Yolol.IL

This is another interesting learning project masquerading as a useless Yolol project. Yolol.IL compiles Yolol code into dotnet IL code - in theory making it extremely fast to execute. This is a side project to a side project so development on it is pretty slow, I haven’t had time to profile it yet.

Interesting Stuff

Some other interesting things I encountered in 2019 in no partiucular order:

2018 Retrospective

Wed, 02 Jan 2019 00:00:00 +0000

TL;DR

I developed Dissonance Voice Chat, released Wet Stuff and learned about Machine Learning.

What Did I Do In 2017?

In my last retrospective I talked about working on Dissonance Voice Chat For Unity and some interesting open source projects I had worked on.

Placeholder Software/Dissonance/Wet Stuff

In 2018 we released Wet Stuff onto the asset store. Wet Stuff is a graphical asset so it was much easier to advertise than Dissonance - everyone loves eye candy! I spent over a month putting together a minute long video demonstrating what can be done with Wet Stuff. This was the first Unity project I’ve made intended to really look good. Everything else I have done has been related to Dissonance (no graphics at all) or just a little side projects for fun (more concerned about gameplay than visuals). I think it turned out rather well:

This was made using an early version of Wet Stuff, the Cinemachine package (amazing for camera work in Unity), the Viking Village pack and finally the music and sound effects all came from AudioBlocks.

While Tom was working primarily on Wet Stuff I was providing support for Dissonance and pushed out 9 updates over the year. Last year we were pushing hard to add lots of new features and worked our way through several major versions (i.e. breaking changes). This year we’ve really been trying to keep Dissonance stable and I’m happy to say that we haven’t had to put out a single breaking change even while making some pretty major improvements. Some of the more interesting features added this year:

Dark Rift 2 Networking support.
WebRTC Network integration support.
Photon Unity Networking 2 support.
Supporting an unlimited number of players in a Dissonance session (previously limited to ~20).
Enabled Opus Forward Error Correction (FEC). This improves audio quality when there are small amounts of packet loss.
Extra monitoring on the receive system to detect terrible network conditions disrupting voice chat
Increased maximum dynamic sync adjustment, this will improve audio synchronisation when there are skips in time (e.g. low frame rates, overloaded CPU).

Of course over the year we’ve also fixed a lot of bugs, 21 have been mentioned in all the changelogs over the year, but some of those were multiple similar bugs aggregated together so the number is probably closer to ~30. The rate of bugfixes has slowed down a lot in the past few months and I’m really happy with how stable Dissonance is at the moment.

The Parsec Project Is Dead

Placeholder didn’t only do Wet Stuff/Dissonance this year, we also worked on the start of a game project codenamed Parsec. We’ve never mentioned it anywhere before so the first time anyone will hear of it is here, talking about it’s death! This was a game we both really wanted to play and doesn’t really exist anywhere in the market at the moment. We spent the six months up until Christmas building prototypes and learning some of the cool new tech in Unity such as the ECS (which we definitely wanted to use for this game). Unfortunately we killed the project at the end of the six month prototype period, not because we were unhappy with the prototypes but for two other risks: technical risk and business risk.

The business risk is simply because games are a very risky business. We’d probably need to take on debt to finance development of the game and if it didn’t sell well it could very easily destroy the company. It would be better to continue developing tools for developers (much less risky) and using the money from that to expand the company until we can take on bigger projects.

The technical risk is due to ongoing flux in how Unity development happens. Right now there’s a big shift going on to replace the core of Unity with the Entity Component System (ECS). The performance of ECS is incredible and we would definitely want to use it for a game. However, it is still heavily in development and over the 6 month prototype period we had to rewrite systems multiple times to keep up with deprecations. There’s also no editor support for ECS at all, so we had to develop our own hacky system synchronising GameObjects and ECS Entities to even be able to use the Editor. This will all settle down over the next 6 months and become much less painful to use.

Blogging

I’ve been really bad at updating the blog this year. In the past I have written about the projects I am working on (such as Heist), but I don’t want this to become a Placeholder Software blog. Alternatively I have written about interesting programming problems I have encountered and my solutions to them, however simply providing support for Dissonance (and the kind of maintenance updates associated with that) hasn’t introduced me to many new topics that are worth blogging about. I do have a bit of a backlog of topics I’d like to write about at the moment, so maybe I’ll try to do several of those early this year.

I wrote a little about network protocol design in Lessons In Network Protocol Design Learned The Hard Way, inspired by working on two Dissonance updates which I just about managed to make to the network system without breaking compatibility. This wasn’t a particularly well planned post and I’d like to think more deeply about the topic and write a better post in the future.

I also wrote Dithering About Dither which I think is a much better post about the shape and colour of random numbers and how that knowledge can be used to apply dithering. Dithering is a fascinating topic which can be of use to anyone doing any kind of data processing, it’s not just for 8 bit graphics!

Open Source

I’m a huge fan of open source - I have published 132 GitHub repositories over the last 9 years! This year I have three interesting new repositories which are all linked together:

*Mute

*Mute (named after the shipboard AI from Analogue: A Hate Story) is a bot for a personal discord server I hang out in. She’s specifically a side project intended to accept contributions from others in the server, many of whom are programmers but only a couple of them know C#, so code quality is not fantastically high and anything that works is merged even if it’s “ugly”.

Towards the end of the year I started trying to add “natural language” features to *Mute so that people can interact with her conversationally like Alexa/Siri/Cortana. This has lead me down the path of learning all about Machine Learning.

Basic Sentiment

This has been my experimental/learning project for ML. Initially *Mute had a basic sentiment analyser using ML.NET and I used this project to develop a better sentiment analyser using deep learning with Keras. The project has three parts:

The sentiment directory contains the first sentiment analysis model, largely following basic Keras tutorials. This is just an Embedding layer (convert words into vectors), an LSTM (process individual words in turn, converting the entire sentence into a vector) and finally a single Dense layer to classify sentences as Negative, Positive or Neutral.

The w2v directory contains an attempt at building a Word2Vec model. This is a model which converts words into vectors in a relatively low dimension space. In a well trained set of word vectors you can do some pretty interesting things, most famously King - Man + Woman = Queen. The idea is that you can train word vectors and then use those in other models as a pretrained Embedding layer, this can both save training time and significantly improve model performance.

Finally the pretrained_w2v directory contains another sentiment analyser which I built from the ground up, rather than following tutorials. This converts sentences into big matrices using the pretrained word embeddings so the model is much simpler and faster to train because it doesn’t need to train an Embedding layer. The model has a Dropout layer (randomly throws away some dimensions of data during training), a Bidirectional LSTM (process words one by one both forwards and backwards, converting the entire sentence into a vector) and then a set of Dense layers which process the sentence vector before the final Dense layer which outputs the three classes (Negative, Positive and Neutral).

Word2Vec Server

I wanted to deploy my new sentiment analyser into *Mute but had a problem - the pretrained Google word vectors I am using require about 3GB of memory! That’s doable but starting the bot then takes a really long time as it loads the data from disk. This project serves word vectors over a very simple HTTP API, so the word vectors can stay loaded even when the bot is restarted.

Longer term I would like to add indexing to this project so that it doesn’t need to store all of the vectors in memory. Perhaps storing them in a database similar to magnitude. This should also speed up vector -> word lookups which at the moment require searching through all 100M words!

Interesting Stuff

Some other interesting things I encountered in 2018 in no partiucular order:

Lessons In Network Protocol Design Learned The Hard Way

Sat, 23 Jun 2018 00:00:00 +0000

TL;DR

Basic network protocols aren’t too hard to design, but make sure to think about forward compatibility.

Network Protocols

A lot of applications need to communicate over a network, all of these applications will need to decide what protocol they use to communicate. You may think that it’s easy to decide that you’re going to just use JSON/Protobuf/Flatbuffers and call it a done but it’s more complex than that - whichever one you choose you need to choose what data do you put into these containers? In this blog post I’m going to use the low level binary protocol of Dissonance voice chat as my example simply because I’m very familiar with the protocol, but the general lessons learned should apply to any communication protocol.

Protocol Sanity

Weird things happen in networks. At some point you’re going to receive the wrong packet from the wrong address at the wrong time with the wrong encoding and if you’re lucky it’ll just cause a crash because of the faulty encoding. If you’re unlucky it’ll be parsed as a valid message in your protocol and cause some bizarre effect which will leave you scratching your head when you come to debug it. Dissonance includes two sanity checks in the packet to try to make sure this cannot happen:

Magic Number

The very first two bytes in a Dissonance packet are always 0x8bc7. If a packet is received by Dissonance and it does not contain these two bytes it is immediately discarded without anything further being read from it. This keeps the potential damage of receiving the wrong packet to a minimum.

Session Number

The magic number ensures that if a non-Dissonance packet ends up getting given to Dissonance no damage is caused but what if a valid Dissonance packet gets sent to the wrong place? To combat this situation Dissonance chooses a totally random session number when a new server is created - every packet (except handshake packets) has the 4 bytes of the session number immediately after the magic number. Just like the magic number if an incorrect session number packet is received it is immediately discarded.

Protocol Compatibility

The most important lesson I have learned when working on Dissonance is about compatibility - early versions of Dissonance did not provide any guarantees about protocol compatibility and we frequently broke compatibility to add new features or fix small bugs. With hindsight this was a huge mistake, once an application is deployed across a network is becomes very difficult to coordinate upgrading all of the servers and clients at the same time! There are three aspects of compatibility to keep in mind when designing a protocol:

Version Identifiers

Sometimes it’s necessary to make a change which breaks compatibility. In these cases it can become a nightmare to coordinate updating all of the clients and server together if old clients start talking to new servers (or vice versa) and ending up confused by the change in the protocol. It’s always useful to have a version identifier in the protocol so that messages encoded in using the old version of the protocol can be detected and discarded.

Backward Compatibility

Backward Compatibility is when a change to a new version is made in such a way that older versions of the protocol can still understand it. Often this can simply be done by supporting both the old and the new version of the protocol at the same time - when you first establish a new connection you can work out which protocol version to use either by inspecting the version identifier or by performing some kind of negotiation.

Forward Compatibility

Forward compatibility is when the new version is designed in such a way that it can tolerate unknown changes in future (as yet unwritten) new versions. For example a packet decoding system could be written to keep reading a packet until it has parsed everything it “expects” from that packet - newer versions of the protocol can safely append more information to the end of a packet and older clients will simply ignore it. Many low levels serialization protocols will give you this for “free”, for example with JSON the receiver pulls the data out of the fields it knows, any new fields will just be ignored.

Protocol Generality

We have a general principal of trying to ensure compatibility, both forward and backward, but how do we do that? I think this requires a shift in how you think about the individual packets. When designing the Dissonance network protocol my thinking was very simple - whenever the design called for some new interaction between client and server I would design a new packet which encompassed all of the information needed for that transaction. However this kind of design is extremely inflexible.

A concrete example of a problem with this is a bug which was reported on Dissonance - sessions with ~20 people would crash when another person joined. This is because the handshake response from the server to the new clients includes all of the information about the session and all the clients in the session, with ~20 people this information is large enough to exceed the maximum packet size. The solution to this could be quite simple - the handshake response can simply be split up into fragments and the client reassembles the packet by appending together all the fragments until all of them have arrived - after that the handshake is decoded as normal. Of course, this completely breaks compatibility. The solution which I have chosen is to instead return the handshake response with no client information (as if the session is empty) and then to immediately follow it up with a packet per client containing all the information about that client (this ClientState packet already existed in the protocol for another purpose).

In that example I was lucky that the ClientState packet existed and could perform exactly the function I needed. However I realised that it would be possible to design the protocol in such a way that this kind of thing is more likely. Instead of thinking about the exact interactions between client and server and designing a protocol to perform those interactions you should think about the data structures on the client and the server and design a protocol to update them. The client and server in Dissonance both have a set of clients and metadata about them, the ClientState message is a general purpose message which overwrites all of the metadata for a given client.

Dithering About Dither

Tue, 27 Mar 2018 00:00:00 +0000

TL;DR

Dither can be helpful to make a thing appear more accurate than it really is.

Defining Dither

Dither is an interesting, and dead simple, technique which can help improve the perception of a signal - simply add noise! This can be used in audio, video, electronics, mechanical engineering and even high frequency trading!

When you see it in action it’s really quite intuitive how it works. Here’s the same picture of me processed in three different ways:

On the left is the original, a full colour image. In the middle is a rather awful image generated by simply clamping every pixel to either black or white. Finally on the right is another image which is also purely monochrome, however some of the pixels have been moved around.

This works with colour images too. Here’s the same photo again with a limited colour palette, causing terrible banding, and exactly the same colour palette on the right but some of the pixels have been moved around. The image on the right is almost perfect at first glance despite using such an extremely limited colour palette.

What’s happening here is that the quantization error is being diffused.

Quantization error is the error which comes from representing an infinitely variable signal, as usually found in nature, to a signal with a limited resolution, as usually found in computers. For example there are an infinite number of colours in the real world but my camera only captures 16,777,216 million distinct colours, so even the full colour image above isn’t completely accurate!

Diffusing error simply means to spread it out. There are the same number of black pixels in both of the monochrome images above, however the dithered images appear better because the error is spread out.

Anywhere that you suffer from limited precision dither can help you out - image processing, audio rendering, sensor measurements, precision timing just to name a few. In all these cases dither spreads out errors, reducing the impact from one very big error to many small errors. The applications I’m particularly interested in are for realtime rendering for games. For example if colours are only stored with a low precision they will suffer from banding, dithering the colours will reduce the banding (as shown above). In fact even if you have an absolutely perfect representation of the colours you can still benefit from dither because you have to display to an imperfect monitor which will introduce banding - dithering the image before displaying it will reduce this.

The example images above were generated using an algorithm called Floyd-Steinberg Dithering. This is a fairly complex algorithm which is not suitable for realtime rendering. Instead I’m going to go into the details of noise based dithering, which can be used in realtime rendering.

Noise Dither

Before diving into how noise based dither works I’m going to show some dead simple example code. Fundamentally all pixel shaders come down to something like this:

float4 PixelShader(float2 screen_coordinate)
{
  return CalculateColour(screen_coordinate);
}

There are several ways we could dither this, depending on what we’re trying to achieve. For example let’s say we want to output to be monochrome. The naive way to do this is simply to convert it to grayscale and threshold the value to black or white:

bool PixelShader(float2 screen_coordinate)
{
    float4 col = CalculateColour(screen_coordinate);
    float gray = dot(col.rgb, float3(0.3, 0.59, 0.11)); //Magic conversion to grayscale
    return gray > 0.5;
}

This will look terrible, like the second example image above. Instead of that threshold against 0.5 we could do this:

return gray > random_in_range(0.4, 0.5);

Now there will be a softer edge to the black/white border.

An important aspect of dither is that if you’re dithering multiple things random values must be independent. Don’t re-use the same random value twice. For example if you want to add two textures together, you’d dither it like so:

float4 Add_PixelShader(float2 screen_coordinate)
{
  float4 tex1 = SampleTexture(Texture1, screen_coordinate + random_float2());
  float4 tex2 = SampleTexture(Texture2, screen_coordinate + random_float2());
  return tex1 + tex2;
}

The texture coordinates sampled from each texture are dithered slightly, using a different random value for each texture.

The really important thing in the above examples is exactly what kinda of randomness is in use. The type of noise has a large impact upon the quality of the dither.

The Shape Of Noise

When you think of random numbers you probably think of something like a dice roll; something where every value is equally likely i.e. it has a uniform distribution. However not all randomness is like this, you can get random numbers distributed in almost any way you imagine!

A probability density function (PDF) is a way to express how random numbers of distributed. It’s simply a function which returns the probability for a random number being equal to the given value. For a dice it’s pretty simple:

PDF_Dice(value) => 1 / 6

Of course every value on a dice is equally likely, so it’s just a constant one over six. If you were to roll a lot of dice, count up how often each number occurred, then graph the result the shape would be the same as graphing the probability density function.

For dither you probably want your noise not to be uniform. Something like a normal distribution (a.k.a bell curve) is more useful as it will distribute most of the samples over a small area and will fall off smoothly. There’s no hard cutoff to the dither which would introduce more visual artifacts.

Rapidly generating something approximating a normal distribution is surprisingly simple. The Irwin-Hall distribution is approximately the same shape and is generated by simply adding together uniform random values!

Here’s what 100 values of rand(), rand()+rand() and rand()+rand()+rand() look like when graphed on a histogram:

As you can see it’s approaching a bell curve shape even with just three values.

All you need now is to generate some noise and add it together. I’ve talked before about generating noise on the GPU. However the noise I generate in that blog post is (approximately) white noise, which probably isn’t what you want to use for dithering.

The Colour Of Noise

The distribution of the noise isn’t the only aspect to consider, the “colour” of the noise is also important. The shape (distribution) of the noise relates to the likelihood of getting certain numbers, conversely the colour relates to how those values are distributed in a sequence. Here are two random sequences of dice rolls:

1, 6, 2, 4, 1, 1, 3, 1

1, 1, 1, 1, 2, 6, 3, 4

Since these two sequences have exactly the same numbers they obviously have the same shape. However something else about these sequences is different, the order of the numbers within the sequence has changed, they have a different colour.

Sequences of random numbers can be displayed as a simple line graph. Here’s some totally random, uniform distribution, noise:

A simple 1D wave like this could also be though of as an audio signal. Click here to listen to three seconds of the same white noise signal. Now the most important bit, here’s the spectrum analysis which shows what frequencies are in that signal:

The shape of this frequency distribution is the colour - this totally flat distribution is known as White Noise.

So why would we care about the colour of noise? Low frequency signals do not change much from one instant to the next. If a random sequence has a mix of frequencies (such as white noise) sometimes the low frequency component will become the dominant one (purely by chance) causing a run of similarly valued numbers. This is pure randomness but it doesn’t feel random to a person - no doubt you’ve played a board game and been incredulous at someone rolling “too many” sixes in a row. The same problem occurs when rendering with noise. Humans are very good at detecting visual patterns even where they don’t exist. Notice how this texture of uniform white noise looks “clumpy” in places:

That doesn’t look nice, people will spot those clumps in the final render. We need something seemingly more random than pure randomness! Now that we know about the colour it’s pretty easy to understand what that means - we want to reduce or eliminate the low frequency components.

Here are the frequency plots of some other colours of noise, courtesy of wikipedia. White, Pink, Brown (no, this doesn’t have any special biological effects when listened to), Blue, Violet and grey:

Blue noise seems like it does what we want. Here’s what that looks like:

Back To The Dither

Let’s get back to dithering. Here’s a reminder of the pixel shader we’re trying to dither:

float4 Add_PixelShader(float2 screen_coordinate)
{
  float4 tex1 = SampleTexture(Texture1, screen_coordinate + random_float2());
  float4 tex2 = SampleTexture(Texture2, screen_coordinate + random_float2());
  return tex1 + tex2;
}

Adding two textures together, with dithering on the texture coordinates. We now know that we want to use noise with an approximately bell curve distribution, and we want the noise to be blue noise to reduce clumpiness in the final output image.

First we create a random vector:

float2 random_float2()
{
  // Create a vector 2 from 2 random numbers
  return float2(random_float(), random_float());
}

So now we need 2 random numbers, with a normal distribution (bell curve). We can use the Bates Distribution to get approximately the right shape:

float random_float()
{
  return random_blue() + random_blue();
}

Finally we need a random blue noise value. Generating blue noise is actually quite difficult, it’s probably not something you can do on the GPU in a pixel shader! The best approach to this is to keep a texture of blue noise and to sample from that. Check out this fantastic blog post about generating blue noise, there’s also a huge pack of blue noise textures in a variety of formats and sizes!

This does make things a little more complex. We need some kind of texture coordinate so we know where to sample from the blue noise texture - this is essentially the seed to the random number generator. The best way to do this is to use a single blue noise texture with different noise in each channel, that way you get 4 different independent blue noise values from one single “seed”. Our final random function would therefore look like this:

float2 random_float2(float2 seed)
{
  // Get 4 blue noise values. Make sure this is using point sampling!
  float4 blue = sample(blue_noise_texture, seed);

  // Create 2 random numbers, using the bates distribution to make them triangular
  return float2(
    (blue.r + blue.g) / 2,
    (blue.b + blue.a) / 2
  );
}

Practical note: This is not the best way to create a random vector (although it will suffice for dithering). This will generate a box of samples around the center point but you disk of samples. Maybe I’ll do an article on that in the future. For now just Google “Poisson disk sampling”

Conclusion

I’ve covered the basics of dither in this article. It’s a much more complex topic than I first thought! If you’re interested in the topic make sure to read these other things:

Generating blue noise and free blue noise textures: Moments in Graphics

Colours of noise: Wikipedia

Probability Density Function: Wikipedia

Poisson Disk Sampling: Herman Tulleken - Dev.Mag

A purely monochrome game: Return of the Obra Dinn

2017 Retrospective

Sat, 13 Jan 2018 00:00:00 +0000

TL;DR

I developed Dissonance Voice Chat all year.

What Did I Do In 2016?

In my last retrospective I talked about forming Placeholder Software and releasing Dissonance Voice Chat onto the asset store right at the start of the year.

Dissonance Voice Chat

Obviously the bulk of my work in 2017 was on Dissonance voice chat. We released in late January of 2017 and it’s been all go since then - dealing with support requests by email, fixing bugs discovered in the wild, developing new features and pushing all this out in 19 versions.

This has been completely unlike any software development work I’ve done before - previously it’s all been closed source development on Epimetheus/Heist (with no customers influencing my decisions) or it’s open source work where I can happily just answer with “I’ll be happy to accept a PR” if I don’t want to work on whatever is being requested. It can be stressful but in general it’s been great fun!

I think the most interesting aspect is how this has influenced how I write code. New stuff that I write is a lot more robust and ready for release with extra things to help inevitable debugging e.g. live readouts in editor of as much as possible, logging everything but hiding that behind a toggle to not annoy users, verifying as much about the runtime environment as possible (not trusting the user to set it up properly).

Releasing on the asset store

Releasing on a store implies a whole load of other associated work. Making branding images, keeping on top of reviews (making sure to respond to any review that isn’t 5 stars) and collaborating with other developers to cross promote our products. The hardest thing, which I’m still not confident about, is making pricing decisions - in 2017 we raised the price twice as we released major new features or gained confidence about how stable/robust Dissonance is. We also participated in 2 store wide sales (at different discount levels) and were featured once in the 24 hours sale. Measuring the impact of all this and extrapolating it out to make future decisions about the price is very difficult!

Compatibility with Android/Linux/iOS

I have learnt a lot about compiling stuff for other platforms this year. I’ve done a tiny bit of work on a Linux server before (Javascript and Mono) and I’ve written an app for Android before (pure Java), but this was a whole different level. Over the year Dissonance has used 3 native plugins, (Opus, speexdsp and webRTC) and I’ve had to get those building for as many platforms as possible and make sure that they are as backwards compatible with older versions of the platforms.

Learning C++

I’ve always hated C++, it’s such a huge and complex language seemingly packed with as many footguns as possible. With the arrival of Rust I thought there was a good chance I may avoid heaving to learn C++ altogether. Of course this was not to be - I could write all my new code in Rust but the world still runs on C++! I learnt a huge amount about C++ for Dissonance while working on changes to Opus (to make it easier to compile on iOS), reverse engineering how libspeexdsp works (attempting to get AEC to work) and extracting just the bits of webRTC that I need to make preprocessing/VAD/AEC to work.

I must admit C++ isn’t as bad as I thought, as long as you know what legacy parts of the language to avoid and you have decent tooling to back you up it’s actually not a bad way to write relatively cross platform code.

Collaborating With Other Developers

A big feature of Dissonance is that it is specifically designed to abstract over the network layer - allowing us to use the same code for multiple different networking systems. We’ve already got seven network integrations (Forge Remastered, Forge classic, Photon Unity Networking, Photon Bolt, UNet HLAPI, UNet LLAPI and Steamworks p2p) and we plan to write even more. We’ve had a great response from other asset developers collaborating with us to help integrate the two assets together and then cross promoting our assets.

I particularly want to shout out to Darrin at Crazy Minnow Studio. I asked him if he would be interested in bringing together Dissonance and SALSA lip sync, he responded enthusiastically and did a huge amount of work to make it happen - he essentially wrote the entire script, all the documentation and even made a great tutorial video on how to set it up.

The end result is super cool! Especially in VR - it adds a lot of immersion to see character faces animated as they talk.

Open Source

I’m a massive fan of open source and try to release as much of my code as possible, I’m sad to say I’ve slowed down a lot in 2017. Usually I would keep the main project closed source and open source any helper libraries I wrote e.g. the SwizzleMyVectors library which I wrote to help me port Epimetheus over to not use XNA vector types. However, Dissonance is designed to be a single self-contained package so there are no parts we’ve released like that yet - we may break out some parts of it (e.g. logging) into a helper library as we develop more projects.

That’s not to say I’ve not released anything though!

ZedZedZed

This is a C# wrapper for the Z3 theorem prover. Rather than writing constraints like this:

var b = ctx.MkConst("B", ctx.RealSort);
var c = ctx.MkConst("C", ctx.IntSort);
s.Assert(ctx.MkEq(ctx.MkReal(1), ctx.MkDiv((ArithExpr)b, ctx.MkMul(ctx.MkReal(13, 10), (ArithExpr)c))));

You can write it as a C# expression:

var b = ctx.MkConstReal("B");
var c = ctx.MkConstInt("C");
s.Assert(ctx.MkExpr(b, c, (bb, cc) => 1 == b / (c * 1.3)));

This isn’t very well developed at the moment. It works fairly well for booleans and integers but has absolutely no support for reals (they don’t map perfectly into the C# type system).

e3d-scad

This is a library of parts modelled in OpenSCAD. I’m attempting to replicate parts from the e3d product line as I need them for my own designs.

This has been useful for…

Rule Breaker

This is a collaboration with my friend Dan Atkinson to design a CoreXY printer from scratch. We’re both into 3d printing but neither of us is a mechanical engineer so this is a pretty interesting learning experience!

With Rule Breaker 1 we’re just aiming for something which can print - If it can produce a quality benchy I will be very happy! Hopefully once we have built that and learnt all the things we did wrong we will move on to design another printer with a similar design but much much larger.

Rock

Rock is an interpreter for the Nock (the lowest level part of Urbit) written in Rust. Implementing a Nock interpreter has become my goto project for learning a new language: it’s relatively simple to implement the most basic Nock operators, a little complex to implement the complex Nock operators and there’s essentially limitless optimisation opportunity.

3D printing

At the start of the year I got myself a Prusa i3 MK2 and started a year long obsession with 3D printing! As mentioned above I’m working to design my own printer from scratch which has also got me interested in learning about embedded programming (I purchased an STM32F303K8 microcontroller to experiment with over the Christmas holiday).

I also have ambitions to build a hangprinter once the weather warms up a little and I can take over the entire garage to build a printer.

Virtual Reality

Back in July Oculus put on a sale for the headset and touch controllers for just £399! After I got a VR headset I managed to talk a lot of my friends into getting headsets to and it’s caused a pretty big change - we’re all big gamers and we’ve played a big range of fantastic games. But we’ve also spent a lot of time just hanging out in VR, building shared spaces or just chatting in a nice environment.

A lot of people may say VR is overhyped but I honestly haven’t met anyone yet who wasn’t amazed by VR when they tried it - I can’t help but think something with such draw even to non techie people is going to be big!

Interesting Stuff

Some other interesting things I encountered in 2017

Unity Ongui

Mon, 04 Dec 2017 00:00:00 +0000

TL;DR

I’ve encountered a this issue when using Unity GUI several times:

ArgumentException: Getting control 1's position in a group with only 1 controls when doing Repaint

Here’s how to fix it…

Unity IMGUI

Unity has two GUI systems. There’s the component based GUI system which has a hierarchy of control components placed into the scene graph - this is pretty good for building complex in game GUIs.

Then there’s the scripting based Immediate Mode GUI (IMGUI). IMGUI isn’t really intended to be used for game UIs but it’s great for building debugging displays or Unity editor extensions. With IMGUI it’s trivial to build a UI:

void OnGUI() {
  if (GUILayout.Button("Say Hello"))
    Debug.Log("Hello!");
}

I use this a lot for building all of the various editor extensions for Dissonanc requires as well as the UI for the various demo scenes.

The Error

Several times when developing UIs with this system I have encountered the following error:

ArgumentException: Getting control 1’s position in a group with only 1 controls when doing Repaint

This is a little cryptic! Usually this will happen when I’m building a complex UI which has multiple states controlled by the UI itself.

The problem comes from the way the IMGUI works. It does not simply get called once a frame, instead it gets called multiple times and each time it does the Event.current property is different (indicating the reason the call happened). There are two particularly important events, Layout and Repaint. Layout is the first event to happen each frame. Repaint is the last event to happen each frame.

So we can understand part of the error message now - when it refers to “Repaint” it’s referring to the last event to be processed. The rest of the error message is relatively clear to decode to, it’s trying to get the position of a control which doesn’t exist!

Based off this knowledge here’s a fairly contrived example which demonstrates the problem:

private void OnGUI()
{
    if (Event.current.type == EventType.Layout)
    {
        GUILayout.Label("A");
    }
    else
    {
        GUILayout.Label("B");
        GUILayout.Label("C");
    }
}

This is deliberately creating 2 controls during repaint even though only 1 exists during layout. The sequence of events in a single frame will be:

Layout
- Choose position for A
Repaint
- Paint A at position chosen earlier
- Paint B at position chosen earlier - throws exception!

Obviously in my real world UIs I’m changing the state of things in between the Layout and Repaint event and causing this to happen inadvertently.

The Solution

I’ve adopted a solution to this (it’s actually been invented twice in the development of Dissonance, once by Tom and once by me). The idea is to keep major state changes away from the OnGUI method. Usually for a complex GUI you might do something like this:

enum State { A, B, C }

State _state;

void OnGUI()
{
  switch (_state)
  {
    case State.A: DrawGuiA(); break;
    case State.B: DrawGuiB(); break;
    case State.C: DrawGuiC(); break;
  }
}

The problem with this is that if you swap state at any point here you’ll almost certainly cause the error (because there’s probably a very big change in the UI layout when state changes).

Instead, try doing something like this:

interface IState : IDisposable
{
  void Activate();
  IState OnGUI();
}

class A : IState { /** skipped **/ }

class B : IState { /** skipped **/ }

class C : IState { /** skipped **/ }

Here I’ve defined an interface to represent UI states. It’s important to note that the GUI no longer mutates the state, instead it returns another state object which is the next UI state (it can return itself, if it hasn’t changed).

private void OnGUI()
{
  var next = _state.OnGUI();
  
  //Early exit if transitioning to self
  if (ReferenceEquals(next, _state))
    return;
  
  //If we already have a next state, overwrite it
  if (_nextState != null)
    _nextState.Dispose();
    
  //Save next state
  _nextState = next;
}
                
private void Update()
{
  if (_nextState != null)
  {
    //Discard the current state
    _state.Dispose();
    
    //Move to next state
    _state = _nextState;
    _nextState.Activate();
  }
}

Here we’re saving that IState return value and (if it’s different) transitioning to it in the Update method - this keeps the complex changing UI of UI layout away from the OnGUI method where it can cause trouble!

Dissonance Voip Pipeline

Sun, 19 Feb 2017 00:00:00 +0000

TL;DR

I’ve released a voice communications asset on the Unity store!

A Warning To Readers

This blog post was written in February of 2017, just a couple of week after the very first version of Dissonance released onto the asset store. I wrote it because I think it’s a pretty fascinating topic with a lot of subtle complexity that’s not initially apparent.

Almost everything about the pipeline has changed/improved since this was written. If you’re making modifications to Dissonance be cautious about using this post as a reference!

Placeholder Software

In my 2016 retrospective I mentioned that I had started a company called Placeholder software and that we had released Dissonance Voice Chat onto the Unity store. Dissonance has been available for a little over a month now and we’ve been working flat out fixing bugs, fulfilling feature requests, upgrading bits and pieces of the system, and providing support. It’s a part of software development I’ve not done much of before and it’s been a lot of fun!

This blog post is not just an advert for Dissonance - I’m going to break down the long and complex process involved with getting high quality VoIP to work (specifically in Unity, but the concepts obviously are very transferable to any platform).

Voice Pipeline

First off, what is a pipeline and what does it do? For a good conversation over VoIP we need 5 things out of the pipeline:

Low Latency
High Quality Audio
Low Bandwidth Usage
Tolerance To Packet Loss
Tolerance To Clock Skew (transmitter and receiver clocks running slightly out of sync)

Getting all of these things to work at the same time can be quite challenging! A further requirement for Unity is that as little of it runs on the main thread as possible - that runs all of the game logic and we don’t want a low frame rate disrupting voice quality.

Digital Signal Processing Basics

Before we look at the complete Dissonance pipeline it’s important to know a little about how digital signal processing works.

Signal

A signal is just a sound - in physical terms that’s a wave. There are many ways to represent a signal - the most common is to store the displacement at each moment in time. Each of these displacement values is a sample.

Sample

A sample is a single numeric value in the signal. There are a lot of ways to represent numbers in a computer - the most common formats for audio are 16 bit integers, 24 bit integers or 32 bit floating point numbers. Unity uses 32 bit floats for all of it’s audio and so does Dissonance.

Frame

A digital signal processing pipeline does not operate on a stream of individual signals - it instead operates on blocks called frames. Each frame is a short period of time (for example in Dissonance this can be tweaked to 20ms, 40ms, or 60ms). There are some places in the pipeline where samples are added to a buffer and at these points the frame size is generally being converted (hence the buffer, to accumulate the excess samples).

Sample Rate

Recall that a signal is formed of a series of samples which store some data about the signal - the sample rate is how frequently the underlying signal (which has conceptually infinite resolution) is converted into samples. A common audio sample rate is 44100Hz, or 44100 samples every second.

The Dissonance pipeline operates at different sample rates at different places - the resample steps in the pipeline are where the sample rate is changed to whatever we need it to be.

Ok now we have the glossary out of the way let’s take a look at the complete pipeline for Dissonance - this is everything from the microphone on the sender side to the speaker on the receiver side. We’ll break this into bits and look at what they all do:

Click the image to see it in a new window.

Wow that’s larger than I expected when I started making the diagram! First off let’s break this down and look just at the sender.

Capture Pipeline

The sender side of the system is called the “capture” pipeline because it captures the audio from the user.

The work of the capture pipeline is split across two threads. The Mic Capture thread in this diagram is the main Unity thread - we want to do as little work as possible here! The Unity API is completely single threaded so we have to read the data off the mic on the main thread, but we want to move it off to somewhere else as soon as possible.

The Unity API for reading data from the mic looks like this:

public bool GetData(float[] data, int offsetSamples);

Unity keeps a rolling buffer of audio from the mic (in our case up to 1 second long) and then we can read data from it as it becomes available. Once a second has passed it wraps around and begins overwriting the old audio data. In an early version of Dissonance we read from this buffer once enough audio was available to process (one frame, usually 40ms) however this turned out to produce subtle audio artifacts every time it looped around. To fix this we moved to a new technique where we read data from the microphone as soon as it is available and keep our own buffer of data until a frame is ready to go. This is the first two steps in the diagram.

Once a complete frame is in the buffer it is copied out of the buffer and into an array of the correct size - this array is then sent to the encoding thread for the real work to happen. The encoding thread spends it’s entire lift processing frames, when no frames are available it sleeps until a new frame is delivered from the main thread.

Preprocessing

The preprocessing step itself consists of two phases - we first want to run the Voice Activation Detector (VAD) on the plain audio coming from the microphone and then we want to clean up the audio (noise removal, automatic gain control). These steps have to happen in this order because the VAD can get very confused if gain control is done first!

The VAD and the preprocessor are both external open source components - the WebRTC VAD and speexdsp. Both these components work with 16 bit integer audio but the Dissonance pipeline natively operates with 32 bit floating point audio, and in addition the VAD only operates at certain specific sample rates.

The microphone hardware may not supply the sample rate we want so the very first step is to convert the audio to the right sample rate (the capture rate mentioned in the diagram) and then convert it to 16 bit audio. The floating point value will be in the -1 to 1 range so the conversion is very simple:

for (var i = 0; i < count; i++)
{
    var sample = input.Array[i + input.Offset];

    //Clip the sample into the allowable range
    short converted;
    if (sample >= 1.0f)
        converted = short.MaxValue;
    else if (sample <= -1.0f)
        converted = short.MinValue;
    else
        converted = (short)(sample * 0x8000);

    output.Array[i + output.Offset] = converted;
}

And back to float again:

for (var i = 0; i < count; i++)
    output.Array[i + output.Offset] = input.Array[i + input.Offset] / 0x8000;

Once the conversion is done the data is pushed into the VAD which will classify the frame as speech or not-speech. After that the data is pushed through the speex preprocessor - this runs two processes: Automatic Gain Control (AGC) and Noise Removal. AGC automatically tweaks the volume of the input signal so that the output signal is always roughly the same volume - this means that in a group conversation with a collection of different people, speaking different volumes to different microphone hardware everyone will sound roughly the same volume. Noise removal… removes noise.

The next step of the pipeline is the somewhat odd sounding delay buffer. Recall that one of the things this pipeline should do is be low latency - so why is there a deliberate delay buffer? Voice detectors are not perfect and one place they particularly struggle is the first frame of voice - it might only be 25% voice but it still needs to be classified as voice otherwise the start of what someone says is cut off. The delay buffer delays the voice signal by one single frame but the VAD operates before the buffer - this allows the VAD to have ~20-60ms of foreknowledge and almost entirely fixes the cut off problems.

Finally, once all this is done (assuming the VAD is active, or push-to-talk is pressed) we need to transmit the audio to the other players. To do this we need to encode the audio using a codec which will reduce the size of the raw data from the rather ridiculous:

48,000samples/second * 4 bytes = 192,000bytes/second

For this, Dissonance (and almost every other VoIP software) uses Opus. The range of bandwidth Opus can use depends upon quality settings - it’ll be somewhere between 750bytes/second (extremely low quality voice) up to 63,750bytes/second (extremely high quality full orchestral music).

The Network

Once the encoding side has produced a packet of encoded audio we need to send this across to the other people in the session so they can listen to it. Dissonance doesn’t require any particular network architecture (there’s a basic interface which you can implement to provide any network architecture you like). However, there’s a default implementation of that interface for a basic client server architecture. Here’s the pipeline for that architecture:

Pretty basic stuff. The sender transmits the packet to the server on the encoding thread (once again, minimising the work on the main game thread). The server determines which clients need to receive this packet and forwards it on to them.

When the client receives the packet we don’t really know what thread we’re on - the receive method is called by user code which integrates with their network integration. We assume this is probably the main thread and so the packet is copied from the receive buffer and sent over to the decoding thread as soon as possible.

Playback Pipeline

Finally, we have the playback pipeline which once again operates in a separate thread to the main thread. It reads packets from the network “transfer buffer”, parses them and plays them back.

Jitter Buffer

We want our voice to be played back as quickly as possible (low latency). However this can present a problem - if we play back a packet as soon as it arrives then the other packet must arrive exactly on time otherwise we’ll have nothing to play back! The pipeline can handle not having the next packet available - it invokes a part of Opus called Packet Loss Concealment and essentially just makes up some sound to fill the gap - but this doesn’t sound great and we don’t want to use it often.

The jitter buffer fixes this situation by storing enough packets to smooth out the jitter in arrival time. For example if we have 100ms of audio in the jitter buffer then the next packet can be up to 100ms later than normal and it’ll still sound ok. The jitter buffer in Dissonance attempts to size itself dynamically by starting out with a conservative size (100ms) and then shrinking itself (by playing back audio a tiny bit faster) as it measures the actual network jitter.

Volume Ramping

When someone starts or stops talking it can often cause a nasty sounding click because the signal is discontinuous (suddenly skips from silence to speech in one instant). To mitigate this, volume ramping detects if this is a transition frame (first or last) and ramps the volume up or down over the length of the entire frame. This does not totally remove the discontinuity but it does reduce it below the audible threshold.

Convert Frames To Samples

Up to this point (all the way from early in the encoding pipeline) we’ve been handling things one frame at a time. However the Unity audio system does not want frames of the same size - it comes along whenever it wants and demands however many samples it needs.

This stage of the pipeline removes the frames we’ve been handling so far and allows the rest of the pipeline to pull things one sample at a time. Conceptually this is pretty simple - just read a frame and then keep returning samples from it when asked, and when you run out of frames read another one and continue. In practice this turns out to be rather complex - reading one sample at a time is far too inefficient, so instead the converter has to try and read data in the largest blocks possible. Additionally, there is some metadata flowing through the pipeline alongside each frame - the converter has to keep track of the metadata for samples as it reads them and return the correct metadata alongside each block of samples.

Soft Clipping

It’s possible that the encoding pipeline produced a bad audio signal with clipping in it (the signal attempts to go louder than max and just tops out). This could be caused by overeager gain control, a badly configured microphone or just someone shouting loudly! Clipping is one of the worst sounding problems in an audio pipeline - in fact it can be downright painful if you’re using a loud headset.

Soft clipping (which is part of Opus) distorts the signal to remove the horrible clipping artifacts by slightly smoothing out the bits which clip. This isn’t a perfect solution because it introduces slightly incorrect harmonics, but they won’t even be perceptible unless there’s some major clipping going on.

Playback In AudioSource

Finally, we get to the end of the pipeline. This is implemented using the OnAudioFilterRead method of Unity. The unity audio thread comes along whenever it needs new data to play, pulls data out of the decoding pipeline, splits the data out so the same thing is playing on all channels (voice data is totally mono channel), and that’s it! From this point on the Audio passes through the Unity audio system just as if it’s a playing sound effect.

Blatant Self Promotion

2016 Retrospective

Sun, 01 Jan 2017 00:00:00 +0000

TL;DR

I started a company, but not doing what I expected.

What Did I Do In 2016?

In my last retrospective I opened by saying that Epimetheus was a solid engine which I had spent more time (in 2015) developing features of. Towards the end of 2016 and for the first half of 2016 Epimetheus was a very solid platform - I made updates only every few days (fixes and minor API enhancements) while mostly working on stuff on top of the engine.

Floorplan Generation

The main meat of my work was on the Base-CityGeneration library which is the base of all procedural generation for Heist (my game built with Epimetheus). At the end of 2015 I finished off road generation and building shell generation so my work at the start of 2016 was on floor plan generation, this is by far the most complex of any procedural generation I have ever attempted. I think it’s telling that although there are millions of articles on procedural texture generation or landscape generation and there are a fair few on the more complex problems like road generation or building shell generation there is almost nothing in the blogosphere or published articles about floorplan generation. Most systems I could found had annoying restrictions like the floor generates inside an area made entirely of rectangular corners, which is almost useless if you want to generate interesting building shells!

My first work on floorplans was building a low level system for handling the representation of 2D space. This system should be able to handle queries about shapes such as what other shapes neighbour this shape, exactly where along the walls do they overlap and how far away is the neighbour. I also built a system (based on this) which could convert a set of non-overlapping rooms into a list of brushes for a CSG system - this handles complexities such as subdividing the walls into sections, running the facade generators on each wall and slotting little miter brushes in between the walls to join everything up. This results in something like this:

Once this sytem was complete (it took a while, making these kinds of generic data structures for accelerating computational geometry is devilishly complex) I moved onto experimenting with generating actual floor plans. My initial experiments focused on specifying constraints on rooms like min size, max size, must connect to X, at least N must exist etc, then a generic constraint solver ran over these constraints and placed the rooms into appropriate places. Rules were specified in order of importance so less important rules could be violated to satisfy more important ones - for example the bathroom often ends up being squashed below it’s min size or into some awkward shape which is often true of real architecture! This system was incredibly cool and generated some very nice example layouts in my initial tests. Howevever this didn’t scale at all, as soon as I started introducing odd building shapes (e.g. something as simple as a non convex building) the constraint solver would end up following dead ends and never find an appropriate solution (the algorithmic complexity is probably out of this world). This was a shame and I’d really love to pursue this line of research again in the future (maybe using a more powerful constraint solver, they get better all the time).

I moved on to something much simpler. Rather than generating floorplans based off realistic constraints I would generate some acceptable rooms shapes and then tweak them into useful floor plans by merging together rooms which are too small, slightly moving border walls and then assign rooms functions to generated shapes after the fact. This iterative method is basically guatanteed to produce something since if it gets really confused it can just stop tweaking and use whatever it has. I generated my initial floor plans with a growth based system which seeded the outer walls at regular intervals, grew walls inwards and tended to split and merge walls with 90 degree corners. I did some additional work to trace corridors along rooms walls to get the floorplans into the game:

Placeholder Software

Ever since making silly little games in visual basic I’ve always used the moniker “Placeholder Software” as a company name for my work, the plan was always for me to turn this into a real company once Heist was ready for release. In September 2016 Placeholder software did finally become a real company but not in the way I had imagined! My friend Tom Gillen quit his job (software dev, but not game dev) with the intention of doing something related to gamedev for a year to enhance his CV and then looking for a job in the industry. We hatched a plan to devlop Unity assets together (and make some money too, yay).

Our first product Dissonance will release tomorrow (exciting and terrifying all at once). I’ll post another blog post with more information about that in the next few days (I’ll probably do a short series on some of the interesting technology involved in the asset).

Open Source

I’m a massive fan of open source and try to release as much of my code as possible. When developing Epimetheus I would often publish basic packages I developed as part of that as totally separate nuget packages (e.g. SwizzleMyVectors is a set of basic extensions to the dotnet vectors). I’ve slowed down a little on the open source this year because I haven’t been developing Epimetheus and simply haven’t needed to write so many low level packages (Unity is fairly “batteries included”), hopefully as I get to know Unity better I’ll be able to identify more basic things which are missing and release some open source solutions.

Sushi Crash
- A side project to develop a bot capable of playing Dota 2 using the new Lua API added with the 7.0 patch (mid December). I’m hoping for a TI bot tournament in 2017 ;)
- Personal project
- Do not depend upon this for anything
Handy Collections
- A collection of… collections. Various datastructures I have needed over the years have been built into this library. I depended upon this for Epimetheus and some bits of it are in use in Dissonance.
- Actively used and maintained
- Reported bugs will be rapidly fixed
- PRs will be considered (breaking changes likely rejected)
ExternalProcessLauncher-Unity
- A tiny library (single file you can copy into a unity project) which allows you to launch an external process from the Unity launcher. This is intended for when you want to move work out of the Unity editor into another process (e.g. in another language). Handles starting and stopping the process when the editor opens/closes.
- Actively used and maintained
- Reported bugs will be rapidly fixed
- PRs will be considered (including breaking changes)
HashMedly
- Everyone always implements the csharp GetHashCode method wrong, this library is intended to fix that.
- Actively used and maintained
- Reported bugs will be rapidly fixed
- PRs will be considered (breaking changes likely rejected)

Interesting Stuff

I love to learn new things, so every year I encounter a wide variety of interesting things. Here’s a few random things I found interesting this year (in absolutely no order):

Dual Contouring In 2d

Tue, 27 Dec 2016 00:00:00 +0000

TL;DR

Dual contouring is a handy technique for generating a mesh for volume data such as voxels or distance fields which preserves sharp corners, unlike the more famous marching squares.

Constructive Solid Geometry

Constructive solid geometry (CSG) is a powerful technique for editing shapes. A CSG system gives you three fundamental operations to work with:

Using these three basic operations on primitive shapes (cuboids and spheres) you can create a huge variety of shapes.

Volume Data

When thinking about how to implement CSG the first thing to come to mind is probably something working directly with meshes. You can simply walk along the edges, cut them where they intersect another shape and stitch everything together. This can be made to work but is extremely complex, requires that your initial primitives are meshes (limiting their precision) and in my experience frequently suffers from tricky numerical errors.

An alternative way to achieve CSG is to use volume data. With volume data we don’t represent the primitives with a mesh approximating their shape, instead we directly represent the primitives with a mathematical object which perfectly captures the primitive. CSG operations become simple mathematical operations on their objects. Then once all the CSG is done we need a final step to convert into a useful format (i.e. a mesh).

The way we represent the shapes is with a distance field - this is a field which can be sampled at any position to get back a number which represents the distance from the surface of the shape (a negative value indicating that this point is inside the shape). Of course this does not need to be stored as a big 2D array instead it can be generated on the fly to achieve effectively infinite accuracy with zero memory overhead. For example here’s a circle:

class Circle {
  private readonly Vector2 _center;
  private readonly float _radius;

  float Sample(Vector2 position) {
    return Vector2.Distance(_center, position) - _radius;
  }
}

This very simple class implements the contract I mentioned above - when the position is inside the circle the distance will be negative, when it is outside the circle it will be positive. The only other shape I have implemented so far is a halfplane shape which splits the universe into two halves (inside and outside) - any shape with straight edges can be built using a series of CSG operations on half planes.

Constructive Solid Geometry

We’ve established how shapes will be represented, but how does this actually help with CSG? Recall the three fundamental operations of CSG are Union, intersection and difference:

Implementing these operations with distance fields is trivial. To Union two shapes we just take the minimum value from two fields:

public float SampleDistance(Vector2 position)
{
    return Mathf.Min(
      a.SampleDistance(position),
      b.SampleDistance(position)
    );
}

This creates a new distance field which represents the union of the two fields (named a and b in this example).

Intersection is the max value:

public float SampleDistance(Vector2 position)
{
    return Mathf.Max(
      a.SampleDistance(position),
      b.SampleDistance(position)
    );
}

Difference can actually be built out of other operations. Logically difference is taking all the space where A AND Not(B). Since intersection provides us with an “AND” operation, we can implement subtraction with a variation of intersection:

difference = Intersection(a, Negate(b));

Creating A Mesh

Now that we can do CSG on distance fields we need a way to convert a distance field into a mesh. I’m going to use the following example shape:

This is a good test because it combines smooth curves with sharp corners, the meshing system must be able to gracefully handle both these things. For anyone trying to recreate this example the code to produce this shape is:

var e1 = new Circle(v(0.2f, 0.2f), f(0.05f));
var e2 = new Circle(v(0.2f, 0.8f), f(0.05f));
var m = new Difference(
    new Intersection(
        new Rectangle(v(0.6f, 0.15f), v(0.9f, 0.85f)),
        new Circle(v(0.35f, 0.5f), f(0.5f))
    ),
    new Circle(v(0.25f, 0.5f), f(0.5f))
);
var n = new Rectangle(v(0.4f, 0.45f), v(0.5f, 0.55f));

return Union.Create(e1, e2, m, n);

All the values are defined in the 0 to 1 range and the v (for vector) and f (for float) functions simply take those values and scale them into whatever range you’re really using.

Sampling

A distance field has infinite accuracy since it’s just a mathematical model of a shape. Obviously to convert this into a mesh we’re going to need to sample the field at some points - but where?

Grid Sampling

The obvious approach is just a regular sampling on a grid:

This suffers problems with curves and corners. The curves have been rasterized down to the resolution of the grid and all corners have been snapped to 90 degrees. This could be fixed with a denser grid but that doesn’t scale very well because we’d be sampling and storing a load of points in completely empty space! What we need to do is sample the grid with more detail where the distance field is near zero.

Quadtree Sampling

A quadtree is a tree of nodes, each node can contain exactly 4 children which fill in it’s available space. This looks a little bit like this:

Here are have a quadtree with it’s top level in red, this contains four children in green and one of those contains 4 children in blue.

With quadtree sampling we sample the distance field at the corner of each node and then recursively subdivide the nodes which need more detail. This approach is called an “adaptively sampled distance field” and looks a bit like this:

This is much better - the smallest squares here are half the size of the uniform grid sampling (twice as much resolution) but we have less than half the total node count.

This system isn’t without it’s own drawbacks. The most critical question is: how do we decide when a node needs more detail? What we want is to subdivide only nodes which contain the surface of the shape, unfortunately this is (in general) pretty hard to detect just from a distance field. Consider this single quadtree node:

Here we have a very large single quadtree node which contains a small circle in the distance field. Because we only sample the distance field at the corners there’s no way to tell that this node contains the surface of the shape and should be subdivided. To fix this I went back to the source and changed the definition of my distance fields:

public interface IDistanceField
{
    float SampleDistance(Vector2 position);

    bool MayContainIsosurface(Rect rectangle);
}

The “MayContainIsosurface” method take a rectangle and returns a boolean which indicates if the surface of the shape might be within this rectangle. My logic here is that most primitive shapes can trivially determine if a given rectangle contains their surface. Here’s the implementation for my circle class:

public bool MayContainIsosurface(Rect rectangle)
{
    if (!_circle.Overlaps(rectangle))
        return false;

    if (!_circle.Contains(rectangle))
        return false;

    return true;
}

Two very simple checks which exploit the basic geometry primitives I have available in libraries.

With this change to distance fields the problem of which node to subdivide is trivial - just check if the node area may contain the surface of the distance field.

I added an additional optimisation step to the quadtrees after they have finished subdividing. Any particular interior node contains 4 corners points and 5 interior points (the corners of it’s 4 children), if the 5 interior points can all be accurately calculated using a bilinear interpolation of the 4 corner points we can safely discard the leaves because they’re not providing any additional detail.

Making meshes

Now that we have a suitably sampled distance field without horrible memory usage but still preserving detail we need to actually generate some meshes!

Marching Squares

Marching squares is a fairly well known and very simple technique. We simply process each cell completely independently and match it to a lookup table based on the sign at each cell corner. There are sixteen total cases to consider (red indicates inside the shape, i.e. a negative distance):

Here’s the simplest example of marching cubes I can construct:

The red circle is the surface of the shape we’re trying to represent. I’ve tagged all the corners with their distance from the surface. Marching cubes looks at each node one at a time so we’ll do the same:

As you can see all we do is work out how far along the edge the surface is (based on a reverse lerp estimate of where zero is), and then place a straight line across the corner to connect these points. The problem here is immediately obvious - any details which are inside the shape (curves or corners) are completely lost. We could fix the curves by sampling with more resolution but that will never fix corners - they will always be slightly rounded unless they lie exactly on a node edge. You can see this problem (exaggerated by a very low detail quadtree) here:

There’s an additional problem not shown here. Because we consider each node independently the points on each node will not match up perfectly - this will leave tiny cracks (which become obvious with very large or very low detail fields, particularly in 3D).

Dual Contouring

Dual contouring solves both of these problems at once by working with the dual graph of the quadtree. The dual of a graph replaces all the faces with vertices, and all edges with perpendicular edges connecting to these new vertices. Here’s an example quadtree with a dual graph drawn in yellow:

With the dual graph rather than placing vertices along the edges of the nodes we place a single vertex inside each node. If this vertex is placed on top of the sharpest corner in the node then corners will be perfectly preserved (to the limit of the sampling resolution). Placing this vertex is achieved with by minimizing a “Quadratic Error Function” (QEF), this function expresses how far from the surface of the shape a given point is and thus minimizing it naturally places the point on the surface. This is not always perfect because it is possible that there are two corners in a single node and in that case minimizing the QEF will likely place the vertex somewhere between the two.

There are many possible QEFs and techniques for minimizing them. For my simple little test project I used a simple iterative gradient descent however this isn’t perfect as it could get stuck in a local minimum (I hacked a fix for this by subsampling the node to get some extra detail, but this is a very expensive solution).

Finally here’s the result of dual contouring on the smiley (using a very coarse grid to push the system to it’s limits):

Curves are not improved much over marching squares (that can only really be fixed by higher resolution) but the corners are vastly improved! The sharp corners on the mouth and the nose are in exactly the right positions.

Procedural Generation For Dummies: Building Footprints

Sat, 07 May 2016 00:00:00 +0000

Procedural City Generation For Dummies Series

My game, Heist, is a cooperative stealth game set in a procedurally generated city. This series of blog posts is an introduction to my approach for rapidly generating entire cities. If you’re interested in following the series as it’s released you can follow me on Twitter, Reddit or Subscribe to my RSS feed

Building Footprint Generation

After road generation has created blocks (the gaps between roads) and lot generation has split these lots up into spaces for individual buildings we need to decide on the shape of the building to place into the lot - this is the footprint of the building. A simple example of this is your average house. A house does not entirely fill the lot it’s placed in - there could be gardens at the front and back, paths along the sides and space to park a car etc.

Buildings are rarely the same shape all the way up, so once footprint generation has decided on the ground floor we need to run it again for all higher floors to decide how the shape of the building changes. This can simply be the same algorithm used for the ground floor but with the shape of the previous floor input as the starter shape instead of the shape of the lot. My system clips all shapes to the input shape, which ensures that nothing can ever be larger than the lot (which would be an error) or overhang a lower floor (which isn’t strictly an error, but would look strange).

I think there are two fundamentally different approaches to this problem, each of which is applicable to a different type of building. An additive/growth based approach which is good for houses or old buildings (which frequently have extensions added onto the side) and a subtractive approach which is better for very large commercial buildings which often fill most of their lot. I will briefly discuss how both work, however I have only actually implemented the subtractive approach (since commercial buildings are what I really care about for my game).

Additive Footprint Generation

This approach works by starting with a simple seed shape (perhaps a cuboid in the center of the lot) and then adding on new shapes or growing existing shapes until the footprint is sufficiently large. This is best for older buildings which have had a lot of extensions added onto the side over time.

First we start off with a simple “seed shape” - this is some very small shape which fits into the lot space a long way from any of the edges. We will then grow this seed shape by adding new parts on to it until some threshold is reached. Growth could proceed either by literally growing the existing shape (cutting it off when it hits certain reserved areas such as paths and gardens) or by adding new bits onto the shape.

For example here we’ve got the basic seed shape (blue quadrangle), some reserved areas (gardens and paths) and a shape has been added onto the seed shape (circle). When adding parts like this there are two things to consider:

Rotation

A circle doesn’t care about rotation but other shapes do! If a square was added onto the side of the house you wouldn’t want it welded on at some random angle, instead it should intersect the existing house at 90 degrees. As a general rule any part added to the house should intersect whatever parent part it was attached to with it’s internal angle (90 degrees for a square) or half it’s internal angle if it intersects at a corner.

Positioning

If we were to attach new parts completely randomly the house could grow to be a very odd shape. For example attach another circle to the circle in the above example and that’s a bizarre shape! To rectify this we should add two new restrictions. First, different shapes should have different probabilities so a circle has a lower chance of being added than a square. This could quite easily be a tweakable parameter which could result in very different house shapes (e.b. set the chance of all shapes except circles to be zero and you get a pretty interesting architectural style for your city). Second, every new shape added should have a lower probability of a child shape being attached to it. This prevents long chains of shapes being attached to one another. Once again this could be a tweakable parameter leading to different architectural styles with the change of a single value.

Subtractive Footprint Generation

This is the approach that I have actually implemented in the Heist city generator. Subtractive generation works by starting with the lot shape and slicing bits off. This is best for big buildings which almost completely fill their lot. Also it’s a good approach for generating upstairs footprints (since the second floor of a building usually looks a lot of the first floor).

From a technical perspective this is a very simple technique (simpler than the additive technique). For every floor we have a function which takes a shape, and produces a new shape. The ground floor gets passed in the shape of the lot, underground floors gets passed the shape of the floor above and above ground floors get passed the shape of the floor below. My implementation of this idea works by chaining together a series of simple functions:

Becuase each function is simple it’s quite simple to define new functions, which results in a very powerful system for defining new footprints.

Corner Cases

The immediately obvious problem with this system is area. Most of these functions remove some space from the shape which means that if we chain too many together we could end up with a tiny little footprint that can’t have a reasonable floorplan. For this reason there are also conditional steps in the pipeline which perform an action, check if the condition is ok, and execute a fallback if not. For example we can define a part of the pipeline which says something like this:

!MinArea
{
    Area: 100,
    Action: !Shrink { Distance: 10 },
    Fallback: !Bevel { Distance: 1 }
}

Here we’re saying try to shrink the footprint by 10 meters (the Action), if that results in an area < 100 then instead bevel the corners by 1m (the Fallback). This example leads on to another interesting case to consider: sometimes the action you want to take might depend upon the action the previous generator took, this is handled by metadata.

Every generator has a metadata container passed into it which is just a simple in memory key/value store for arbitrary values. Using this you could setup two generators to collaborate. Let’s modify the example above so that it sets a flag indicating which choice it took:

!MinArea
{
    Area: 100,
    Fallback: !MetaSet
        Type: Boolean
        Key: AreaLimited
        Value: True
}

!MetaPredicate
{
    Key: AreaLimited
    Pass: !Bevel { Distance: 1 }
    Fail: !Shrink { Distance: 10 }
}

Now the area check runs and does nothing except set a flag to true. Critically because nothing ever unsets that flag and these generators run in order away from the ground floor once that flag is set all subsequent floors will continue to see it set. The second part then acts on this flag, running a shrink if the flag is unset (i.e. the flag test failed) and running a bevel if the flag is set (i.e. the flag test passed). If we use this same generator for every floor then the building will shrink down to a small footprint and then all subsequent floors will just apply a little bevelling.

Conclusion

Generating building footprints is an important problem which turns out to be fairly simple to solve if you avoid small residential properties. By chaining together a series of very simple generators some very complex behaviours can be achieved. Next time I’ll talk about how the different parts of a building are modelled (in a software engineering sense) by the procedural generation system.

Procedural Generation For Dummies: Half Edge Geometry

Wed, 30 Mar 2016 00:00:00 +0000

Procedural City Generation For Dummies Series

A lot of the code for my game is open source - the code applicable to this article can be found here. Unfortunately it has some closed source dependencies which means you won’t be able to compile it (I hope to fix that soon) but at least you can read along (and criticise my code). I should point out that this code is in a bit of a state of flux at the moment - I’ve just recently started an overhaul of some of its core components - so some of it which is only half overhauled may read a little strangely right now!

Messy Meshes

One thing which I have to deal with all the time in procedural generation is meshes - that is things which can be represented as a set of vertices, edges and faces. They appear in road generation (roads, junctions and blocks), floor plan generation (wall join, walls and rooms) and navmesh generation (faces are walkable areas).

Computational geometry is an infamously tricky subject, so it’s important to have a good set of structures for representing the data you’re working with and a good set of tools to manipulate that data.

Indexed Meshes

Most programmers who have worked with graphics will be familiar with an indexed mesh. This is a very simple structure:

List of vertices
List of ( List of indices )

We have a flat list of vertices (positions) in a completely arbitrary order. Each face is a list of indices, the indices of the vertices which make up this face. Our mesh is then a list of faces - a list of list of indices. If we want to specialise to purely graphical meshes then there is an easy optimisation to make here; since we know every face has three sides simply store the indices as one continuous list (with N, N+1 and N+2 as the three corners). However we want to be a little more general than that so in this system a face is any set with three or more vertices.

Here’s an example:

We’ve got two shapes, A and B. Here’s what that would look like in indexed mesh form:

Vertices: [ (0, 0), (10, 0), (10, 10), (5, 10), (0, 10), (0, 5), (5, 5) ]
Faces: [[4, 3, 6, 5], [0, 5, 6, 3, 2, 1]]

The order of the vertices here is totally arbitrary. The indices in each face are stored in clockwise order around the face.

This is simple to build, and simple to enumerate but it doesn’t really offer any good tools to help you build correct geometry. For examples let’s add another face

[0, 6, 5, 1]

As you can see below this face overlaps the other faces, is incorrectly wound (anticlockwise) and is self intersecting! These are all properties which we want to avoid and, if possible, should be impossible to construct.

Half-Edge Meshes

Indexed meshes are a very topographical way of representing a mesh - they contain the information about positions and not really much else. Checking for the three critical properties (winding order, overlapping and self-intersection) would require a series of costly vector operations. With a Half-Edge mesh we keep a lot of the topological information about the mesh around which means that checking all of these properties becomes just a series of pointer lookups and equality comparisons.

Let’s create the same shape as before, but with a half-edge representation:

I know this looks complex, but that’s because we’re keeping so much more useful information around! Let’s look through the parts one at a time:

Vertices

These are represented by the red dots in the diagram. A vertex contains two pieces of information:

Position
List of edges starting at this vertex

That’s it! You can extend this model to store more per vertex attributes (normals, texture coordinates etc) but I’ve not yet had a use for this.

Half Edges

These are represented by black arrows in the diagram. These are the key part of this structure - every connection between two vertices is a one way edge - hence the name half edge because each complete edge is represented by two one-way halves. A half edge contains 4 pieces of information:

Sibling Edge (opposite direction edge)
End Vertex
Face which this edge borders (possibly null, if there is no such face). This is represented by the blue arrows in the diagram.
Next edge around the face this edge borders (possibly null, if there is no such face). This is represented by the green arrows in the diagram.

Faces

These are represented by the pale green circles (labelled with the same letters as in the previous diagram). The faces are actually very simple, containing just one single piece of information:

An arbitrary edge bordering this face

That’s it! With just this information we can answer a lot of useful queries about the mesh and also ensure that those three critical properties (winding order, overlapping and self-intersection) are maintained at all times.

Critical Properties

It may not be entirely obvious how those critical properties are maintained in this system and in fact they can be violated in certain circumstances - but constructing such an invalid mesh is quite difficult and generally as soon as you try to do anything with it the system is going to notice something is broken and immediately throw an exception.

Winding Order

Here’s the example image from before cut down to just show two edges attached to the two faces. If we tried to reverse A (so it was incorrectly wound) it would be impossible - the edge going up along the right hand side of A is already in use. Since an edge can only ever be assigned to one face if we ever find ourselves trying to attach an edge to another face we know we’re trying to construct invalid geometry.

This property can be violated if you construct two completely independent faces (sharing no vertices) - one clockwise, and one anticlockwise. As soon as you try to connect these faces up in any way it will be impossible.

My implementation actually has an additional check to catch invalid winding much earlier than this. When you construct a new mesh object you specify which winding you want. Whenever a new face is constructed it’s winding order is checked (geometrically, by looking at the area of the face and checking if it is negative).

Self Intersection

The right hand face here is clearly broken and this is another example of how it’s sometimes possible to construct broken geometry - that face does not violate any of the invariants of a half edge structure. However if you have a self intersecting shape at least one of the edges must be incorrectly wound (in this case the top right horizontal edge). This means all self intersecting shapes become a subset of the winding problem and are solved in the same way - as soon as you try to connect that incorrect edge to anything else the error will be detected.

Overlapping

Here we have another clearly broken situation - the larger face overlaps the smaller face. In this case the solution is quite obvious from the image - the grey edges are already in use for the smaller face so we can’t possibly assign them to the larger face!

These Vertices Were Made For Walking

Let’s have a look at the most simple query you can perform: finding all the vertices around a face. With an indexed mesh this is obviously rather trivial:

for (index in face)
    yield vertices[index]

This is pretty much exactly what an indexed mesh is designed for! With a half edge mesh this is slightly more complex. The half edges around a face all point to the next half edge around the same face, in this way they form a circular singly linked list around the face. Additionally a face points to an arbitrary bordering edge. These two pieces of information are all you need:

first = face.edge
edge = first
do
{
    yield edge.EndVertex
    edge = edge.Next
} while (edge != first)

Just keep following those “next” pointers until we get back to where we started!

Many queries can be implemented on a half edge mesh using the same concept - just keep chasing pointers! For example let’s say we want to find all neighbouring faces to a given face. With an indexed mesh this is a nightmare:

# assume we have a variable "face" which is the start of the query
for (neighbour in faces)
    if (neighbour == face)
        continue;
    for (var i = 0; i < neighbour.Length; i++)
        find = face.IndexOf(index)
        if (find != -1 && face[(find + 1) % face.Length)] == neighbour[(i + 1) % neighbour.Length])
            yield neighbour;

I’m not certain that’s correct (it’s just blog pseudocode, gimmie a break) but essentially what I’m doing here is enumerating all vertices of all other faces and trying to find pairs which are in the query face, but reversed. I’ve almost certainly got some off-by-one errors here, and looking back on it I think I’ve got one of the indices the wrong way around. The point is this is annoying code to write and, even worse, the algorithmic complexity is horrible - something like O(#Faces * #AvgVerticesInFace).

Here’s how we do the same thing with a half edge mesh:

first = face.edge
edge = first
do
{
    if (edge.Sibling.Face != null)
        yield edge.Sibling.Face
    edge = edge.Next
} while (edge != first)

I hope you’ll agree that’s much friendlier and far easier to read! All we’re doing here is enumerating the edges around the face (in the exactly same way we did in the previous example) and then jumping across to the sibling edge and getting the attached face. This was really easy to write and is far less algorithmically complex - O(#EdgesInThisFace).

A few more of the queries which can be done easily in a half edge mesh:

Edges around face
Vertices around face
Faces around face
Edges from vertex
Faces around vertex
Pathfinding (and other graph based algorithms)

If you can’t figure out how to implement some of these queries post a comment below and I’ll do another post explaining them in more detail!

How Does This Help?

Here’s a concrete example of how these kinds of queries can help - we can use them to detect even more invalid geometry (winding order, overlapping and self-intersection). Let’s look at the overlapping case, here’s a nasty piece of overlapping geometry which would not be detected normally:

This overlap doesn’t generate any invalid edges and would likely never be detected by the normal checks. However, if we can do intersection checks (just normal 2D collision detection) we can check for this situation quite easily:

for (vertex in face)
    for (neighbour in vertex.Faces())   # <-- .Faces() is the query
        if (face.Intersects(neighbour))
            throw new InvalidMeshException("Oh no!");

Pretty simple - we just query every vertex in the proposed new face for neighbours and then check if we overlap any of those faces. This won’t catch all invalid overlaps but, as with many of these checks, it ratchets up the difficulty of accidentally creating invalid geometry.

Conclusion

Half Edge meshes are a more powerful way to represent meshes than a traditional indexed mesh. I use them as the basis for many of my procedural generators where they help ensure that the generator can only ever generate valid geometry.

Fixed Permalinks

Mon, 21 Mar 2016 00:00:00 +0000

Broken Permalinks

I recently discovered that permalinks to my blog posts have been broken - this means that things like reddit posts and Google search results pointing to my blog were all broken for an unknown amount of time! I think this was caused by Github upgrading their version of Jekyll to 3.0 which has some slightly different rules about how permalinks are configured. Previously my permalinks were:

permalink: /:categories/:year/:month/:day/:title

Jekyll 2.0 interpreted this as meaning an optional trailing slash was allowed. However Jekyll 3.0 requires you to add that trailing slash explicitly if you want it. This means I fixed the entire problem by simply adding a / to the end of the config:

permalink: /:categories/:year/:month/:day/:title/

It’s frustrating this happened just after I published a couple of relatively popular blog posts about procedural generation. If you’ve come to my site and encountered a 404 page I apologise for that :(

Loading Speed Optimisations

While I was fiddling with the configuration for the site I also made some changes to the default layout of pages. All of the Javascript which generates the polygon background is now loaded at the bottom of the body tag, instead of the top of the head tag. If scripts are in the head they block the browser from doing anything else while the script downloads (resulting in a blank page while things download), conversely if scripts are in the body tag they do not block. Of course, the page will render to the screen before the scripts do anything which means you may see the page without it’s polygon background if you’re on a particularly slow connection - but this isn’t really a problem.

Additionally I have also included all of the CSS served from the same domain as my blog (some of it comes from an external CDN) inline in the head tag. This seems ugly but ultimately the browser is going to have to download those bytes - we may as well do it all in one request!

After all this I still don’t get a particularly fantastic score from Google PageSpeed Insights, only 80/100 for desktop, but it’s definitely improved (especially on bad network connections).

Procedural Generation For Dummies?

It’s been a long time since I’ve published an entry in my Procedural Generation For Dummies series. The last entry regarding city generation was published right at the end of December - almost three months ago. This is because I started work on floor plan generation and ultimately decided that the approach I was taking was fundamentally flawed and I needed to go back to the drawing board! Of course this was pretty depressing so I took a break from working on procedural generation altogether for a couple of months to work on character animation.

For the past few weeks I have been working on a new floor plan generation system which seems to be working much better. I will be resuming entries in the series very soon, in the meantime here’s a teaser:

Procedural Generation For Dummies: Galaxy Generation

Thu, 14 Jan 2016 00:00:00 +0000

Procedural City Generation For Dummies Series

My game, Heist, is a cooperative stealth game set in a procedurally generated city. This series of blog posts is an introduction to my approach for rapidly generating entire cities. This post is a fun diversion away from city generation to explore a weekend project from a few months back: Galaxy Generation!

If you’re interested in following the series as it’s released you can follow me on Twitter, Reddit or Subscribe to my RSS feed

The code related to this article is open source and can be found here and here. It’s a complete C# galaxy generation library which you can use in your own projects.

Galaxy Generation

For my city generator the system is hierarchical - a single generator just knows how to generate one level of the city and then invokes more generators to fill in the gaps. For example a city generator just knows how to place roads, which then calls a city block generator to place buildings, which then invokes a floor generator to place rooms etc.

The galaxy generator works in a similar way; the high level galaxy generator invokes separate generators for different parts of the galaxy such as the core and the arms. However, there is a major difference: when a nested generator creates a star the parent generator has a chance to modify it. This allows me to generate the core and the arms and then apply a swirl effect to all the stars later which vastly simplifies the implementation of the sub-generators.

Let’s look a the basic building blocks one by one…

Sphere

This is the simplest of all of the generators; it simply generates a roughly spherical blob of stars. The blob of stars is denser in the middle and slowly fades to nothing as distance increases. This is achieved through a normal distribution - more commonly known as a bell curve.

Standard Deviation Graph - Wikimedia

A normal distribution is a probability distribution with a higher probability of things being in the center. As you can see from the graph above there is a 68.2% chance of something being within 1 deviation of the center, a 27.2% chance of something being within 2 deviations, a 4.2% chance of being within 3 and so on. The deviation is a parameter which you can pick, so a very large value will get a low density blob of stars spread out a long way and a very small value will get a very high density blob of stars clumped tightly together. My implementation allows you to specify the deviation separately for each axis, so you can actual generate oblate spheroids with this.

In the example above is a sphere with the deviation the same on all axes, so it’s roughly the same in all directions but has no other apparent structure - it has a very natural appearance.

Cluster

The cluster generator is similar to the sphere generator. However, instead of placing individual stars according to a normal distribution it places other generators. In the example above I have generated a clusters of spheres, so we have a loose collection of blobs of stars. This is a perfect example of how a generator calls another, simpler, generator to use as a building block.

This generator is inspired by a vaguely scientific basis of how galaxies work - stars are generated in nebulae which will tend to form a clump of stars. Additionally large masses will pull in nearby stars and form even larger groups. This is used most prominently in the core of the galaxy where a large number of clusters are placed into a relatively tight area to simulate the super high density galactic centre we see in real galaxies (around something of immensely high mass such as a supermassive blackhole).

Spiral

This is the generator which really brings the others together to form something that looks like a real galaxy. Near the start I mentioned how higher level generators in this system have a chance to modify the stars generated at lower levels - this capability is used by the spiral galaxy in two ways.

Firstly all generated stars have a swirl effect applied to them to introduce the appearance of rotation. All the parts of the galaxy are generated as if there were no rotation at all (e.g. the arms are dead straight out from the center) and then the spiral galaxy applies a swirl modifier to bend the arms. The swirl method rotates a star around a given axis based on the distance from the center.

Secondly there is a void right at the center of the galaxy (to simulate the location of a supermassive blackhole). All stars generated within this void are simply deleted.

Let’s have a look at the three parts of the spiral generator:

Background

The background is simply a huge sphere with the standard deviation set to the size of the galaxy. This means that the galactic disk will have some stars above and below and there will be a very low density halo of stars scattered around the galaxy in all directions. The swirl effect is not applied to these stars.

Galactic Core

The galactic core is a very tight cluster, with the standard devitation set to be about 5-10% of the size of the galaxy. The swirl effect is five times stronger on these stars (and is naturally stronger at short distances from the origin) so the center is very strongly rotated around - giving the impression of a chaotic galactic center wrapped around a supermassive blackhole. What started off as spheres distributed around the center ends up many tiny little arms wrapped right around the core.

In the example above you can see the galactic core without swirl (on the left) and with swirl (on the right).

Arms

The arms begin life as a series of spheres generated (roughly) along a line. The swirl effect pulls these straight arms around into a nicer shape which gives the entire galaxy the appearance of rotation.

Other galaxy types could be generated by varying how the arms are created. For example a bar galaxy could not apply the swirl effect within a certain distance of the galactic core - which would cause the arms to remain straight near the core and thus create a solid bar. Alternatively some small arms could be generated between the larger ones with lower density of stars, or perhaps starting away from the core.

In the examples above you can see some arms generated without swirl (on the left) and with swirl (on the right).

Colour

You may noticed that these examples all have colours assigned to the stars. These colours are generated by picking the temperature of the star and then converting that temperature into a colour (assuming the star is a black body radiator). This isn’t a particularly scientifically accurate system for two reasons. Firstly the temperature is picked using a normal distribution; however star temperatures are unlikely to be normally distributed in reality. I couldn’t find any information about the real distribution though, so a normal distribution will do. Secondly stars are not true black body radiators due to various elements in their atmosphere absorbing emitted radiation. However a black body estimate is a close enough estimate.

Names

You can’t see this in the renderings above, but all the stars in these galaxies have names assigned to them. This generates names such as:

Superba
Gamma-77
Alpha Alnati II
Ham 44
A12
Many More

There are several different name generators which are randomly chosen between. Having multiple generators ensure there are entirely different styles of names in the pool - this fixes a common problem with procedural generation where things become boring because the patterns become obvious. Most of the credit for these name generators should go to my friend Pi Lanningham, I developed the most generator (the basic markov model) and then he extended this with all the alternative schemes as well as the system for weighted picking between the themes; thanks Pi!

The most basic is simply a markov chain which was taught with a large set of real star names as well as fictional star names. This generates names such as Superba and Alnati - single nonsense words. The implementation of the markov chains is my own open source library Marvellous Markov Models.

A slightly more complex strategy applies random prefixes and suffixes to markov generated names. There is a slight (1%) chance this could chain further prefixes and suffixes. This could generate names like Alpha Alnati II (greek letter prefix and roman numeral suffix), San Gamma (san prefix), Xendi Kappa (greek letter suffix) or San Gamma Gorgon II 44.7 (2 prefixes, 2 suffixes).

Another strategy simply generates names as if from a scientific index; with a letter and an integer. This generates names such as A-21, C-34 or D-07.

Finally there is a strategy which picks names from a pre-set list. This list doesn’t have many names on it at the moment, but can obviously generate completely unique names (perhaps associated with some special gameplay event at that star). An obvious extension which I have not implemented would be for this name generator to remove unique names from the system once used to ensure that they are truly unique.

Different strategies have weights associated with them so the majority of stars have scientifically indexed name. There’s a lower chance of markov names (with prefixes and suffixes). Finally the very lowest chance is for unique names.

2015 Retrospective

Mon, 04 Jan 2016 00:00:00 +0000

TL;DR

2015 was a year of perpetually being close to ready.

What Did I Do In 2015

In my last retrospective I said that I felt Epimetheus was a solid engine and I could spend 2015 working on gameplay instead of engine code. This sort of happened but every time I came to write some game code I discovered something in the engine needed some polish to get it to release quality!

Open Source

I’m a massive fan of open source and release loads of my code publicly - I even hope to open source the entire Epimetheus engine and Heist game I’ve been working on later this year (probably after it goes on sale).

New Open Source Projects

ZSharp A very recent experiment in designing a totally functional language - this is a language which is statically guaranteed to terminate. Turing completeness is overrated!
- Personal project
- Do not depend upon this for anything
SwizzleMyVectors .Net4.6 brings new SIMD enabled vector types. SwizzleMyVectors adds a load of helpful structs and extension methods to them. Things such as rays, line segments, bounding rectangles and boxes are all included.
- Actively used and maintained
- Reported bugs will be rapidly fixed
- PRs will be considered (breaking changes likely rejected)
Beautiful Blueprints A 2D layout system with YAML markup. Specifically designed for building facade layout but intended to be fairly generic for any 2D layout situation.
- Actively used and maintained
- Reported bugs will be fixed, but not rapidly if they do not impact my own usage of the library
- PRs will be considered (breaking changes likely rejected)
Aimless Names A procedural human name generator.
- Used in some of my code, but not something I’m actively working on at the moment
- Reported bugs will probably sit until I come back to working on this code
- PRs will likely be accepted

Interesting Stuff

I love to learn new things, so every year I usually encounter a wide variety of fascinating things to learn about! Here’s a few things I found cool this year (in no particular order):

Game Development

My job is game development. Specifically writing my engine, Epimetheus, and my game, Heist.

Changelogs

I largely stopped doing changelog videos in 2015 - the kind of features I was working on didn’t seem terribly exciting to make a video about. With the city generator shaping up recently I’m hoping to start doing changelog videos again soon - and this time I will advertise them better!

Character Control

I spent some time at the start of the year working on control schemes. Obviously this is incredibly important as everything the player does goes through the control scheme! If it’s bad people will give up playing right away.

This work led me into a few other associated areas such as tools/weapons aiming (closely associated with control schemes, decals for visualising weapons impacts, camera control to ensure the camera doesn’t clip level geometry, sticky cover to make stealthy movement easier and navmesh generation for an experimental NPC aimbot to play against.

Now that this work is complete I have a character controller that makes it easy to quickly move in a stealthy way around an environment. As well as the ability to aim and use weapons and some basic NPC guards (still more work to do here) to play against. I’m pretty happy with how this all turned out, even if it was more work than I initially expected!

Character Animation

A large part of the control system is choosing which animations to play. For example if a character wants to move across a gap from one piece of cover to another I probably want to play a short dive roll animation followed by a repeating idle in cover animation. This is a problem because the character I have been using is just a basic placeholder with a very basic set of animations (walk, run, fall, jump). In May I discovered a service called mixamo which allows you to create characters in their Fuse tool (kind of like a super advanced MMO character creator) and then buy animations to attach to the characters. Here’s some examples which I threw together in no time at all:

I took several weeks trying to extract data from the downloaded mixamo files, rewriting my entire animation content and rendering pipeline in the process (very painful work), but unfortunately I failed to get it working. This turns out to be a known issue in Mixamo (something to do with the way the data is encoded). I will be coming back to character animation soon (it’s critically important, if nothing else I can’t have all the players and every NPC looking exactly the same).

Procedural Generation

With control input done, and character animation on hand I moved on to generating interesting environments. I’ve been using the same test map for over a year (just jamming new features into the map) and it was starting to get pretty boring (and crowded)! In June I began work on an application called NODE/Machine. NODE/Machine is a designer for procedural worlds - it can connect to the game and send across your designs as you design them so you can fly/walk around them and tweak your design.

I last worked on procedural generation in 2014, laying down various base classes for representing the world. This was important work but didn’t really generate anything interesting - I just output the most basic blocky designs to prove everything worked. With NODE/Machine I started building on top of these classes to design far more detailed generators for various parts of the city. The approach I have taken is for each generator to be scriptable using a YAML derived language so anyone can write new scripts describing parts of the world without having to write any C#!

With this technique I have written generators for road layouts, lots, building facades, building internals (order of floors, placement of vertical elements such as stairs) and I am currently working on floorplans. This work has taken most of my time in the last half of the year and I have been terribly bad about writing about it - this has changed just recently with my procedural generation for dummies series.

Mod API overhaul

Right at the end of 2014 I removed mod sandboxing. This was for a variety of reasons which you can read about here. Sandboxing the mods put a lot of limits on how I could write the API (this was one of the reasons for removing it) and so as I have worked on all these other things I have refactored and redesigned almost the entire API. The end result is a far nicer modding API which makes writing mods far more pleasant.

Epimetheus

Epimetheus is my C# game engine which I am building alongside my game, Heist. The engine is designed to have excellent support for plugins which can add new functionality. Ultimately the Heist gamemode is simply a set of interdependent plugins loaded into the engine. Currently the engine is closed source, but hopefully that will change next year. I’ve skimmed my changelogs for the year, largely to remind myself what I achieved this year. Here’s some highlights:

Score service
Primitive sounds engine enhancements
Websocket based UI messaging System
(Many) usability enhancements to entities
Exposed rendering pipeline to plugins
Built in memory index of plugins (speeds up instantiating plugin components)
Implemented Screen Space Deferred Decals
Improved resolution scaling of in game UIs
Implemented 3D text rendering
Fixed navmesh traversal across world chunks
Implemented in world floating/holographic UI displays
Implemented behaviour trees and exposed it to plugin system
Removed custom sound engine and replaced it with SupersonicSound (FMOD)
Implemented loading FMOD sound banks from plugins
Various quality of life improvements to world generator
Removing sources of non-determinism
Added extra sanity checks on parameters
Made deleting the world and creating a new one much easier
Exposed configuration of world generator to ReST API
Added utilities for evaluating bezier/quadratic curves
Experimented with, and eventually gave up on, Fody
Replaced many plugin API return values with IReadonlyList<T> instead of T[]
Massive swapover from using XNA vectors to using System.Numerics vectors
Also upgrade to .net 4.6 at the same time
Verified every single usage of Matrices and Vectors in the entire engine
Built SwizzleMyVectors project with various helper functions and types
Implemented deferred depth peeled transparency
Optimised memory usage
Added support for native DLLs in plugins

Placeholder.*

Placeholder.* is a series of low level game libraries which Epimetheus is built upon. Everything from content management to artificial intelligence. Currently these libraries are closed source, but I hope this will change next year - I’m considering open sourcing the libraries and selling commercial licenses to the unity gamedev crowd.

Added an event with a reference to an entity when it is being disposed. Meaning properties from disposed entites can be inspected.
Upgraded input service to only send events regarding input devices which are in use
Moved polling of input devices to a parallel task
Implemented network encoding of dimensional unit vectors using only 2 bytes
Upgraded navmesh generation (various small improvements)
Implemented “string pulling” algorithm to generate a straight path through a navmesh path
Massive swapover from using XNA vectors to using System.Numerics vectors
Rewritten convex hull generation to be faster, have a better worst-case runtime and be simpler

Procedural Generation For Dummies: Lot Subdivision

Sun, 27 Dec 2015 00:00:00 +0000

Procedural City Generation For Dummies Series

Lot Subdivision

After road generation has finished it will have generated a road map which will look something like this:

The next stage of procedural city generation is to take each area surrounded by roads, called a parcel, and decide where to place buildings in this area. The spaces buildings are placed into are called “lots”.

I have come up with two different lot generation algorithms: OBB Parcelling and Straight Skeleton Subdivision; both based on this paper. So far I have only implemented OBB parcelling.

OBB Parcelling

OBB (Object Aligned Bounding Box) Parcelling is a method for recursively dividing a space into roughly cuboid parcels. It is best when the initial space is nearly cuboid, for example in a Manhattan style city.

The algorithm is quite simple:

function obb_subdivide(space) {
  
  // 1. Fit an Object Aligned Bounding Box around the space
  let obb = fit(space);
  
  // 2. Slice the space along the shorter axis of the OBB
  let parts = slice(obb.shorterAxis, space);
  
  // 3. Check validity of all children, terminate if any are not valid
  // This is the base case
  if (parts.Any(IsNotValid))
    return space;
  
  // 4. Recursively apply this algorithm to all parts
  for (part in parts)
    return obb_subdivide(part);
    
}

The first step is to fit an object aligned bounding box to the space. My approach to this is basic brute force; since the OBB must be aligned to one of the edges of the space, I simply generate every possibility (equal to the number of edges) and then pick the smallest one. Generating a box along an edge simply requires projecting all the points of the shape onto the axis so the total cost ends up being proportional to Edges * Points, which is equivalent to Edges ^ 2. Normally it’s best to avoid algorithms with an exponential cost but in this case it’s ok - the number of edges in a space is unlikely to be high enough for this to become a problem.

Now that we have an OBB surrounding the space the second step is to split the space in half. This is done by cutting the shape along the shorter axis of the bounding box. There are multiple techniques to slice a 2D shape, I decided to come up with my own (based on the code I had available). My technique is based on generating the Delaunay triangulation of the shape (using the Poly2Tri library). Slicing a triangle is trivial - you just need some careful handling for the slice line and triangle edge being perfectly co-linear. Once the triangles are sliced it’s a simple matter of walking all the directed edges and reconstructing the result (I may do a more in depth post on this, if people are interested).

After slicing we have generated two new shapes and we can now apply the same algorithm again, recursively. The only thing left to establish is when to stop, i.e. the base case. My implementation supports four rules, as soon as any rule is violated by a child shape then recursion is stopped.

Area

The most obvious rule is the area rule. This sets a lower limit on the area any lot may be. Recursion stops if any slice line generates a lot below this limit.

Access

This rule governs things parcels have an edge connecting to. The initial edges of a space have “resources” attached to them (e.g. road access), recursion stops if any generated lot does not have access to a required resource.

Aspect Ratio

This rule governs the aspect ratio of a lot (length / width). This sets an upper limit of the aspect ratio of any lot. Recursion stops if any generated lot exceeds this limit.

Frontage

This is a specialisation of the access rule which measures how much access a lot has to a resource. This sets a lower limit on the length of edge next to a given resource, recursion stops is any lot of generated with smaller frontage.

All of these rules have a probability associated with them. This is the chance that recursion will not terminate even if the rule is violated - for example you could have a block setup like this:

- Area { Min: 100, Chance: 0 }
- Frontage: { Min: 5, Chance: 0.25 }
- AspectRatio { Max: 1.5, Chance: 0.25 }
- AspectRatio { Max: 2.5, Chance: 0 }

In this example we have two hard rules which will not be violated (area and aspect ratio 2.5) as well as two rules with a 25% chance of violation (frontage and aspect ratio 1.5). Since the rules are evaluated for every new subdivision the chance of a large rule violation becomes increasingly unlikely with every slice (25%, 6.25%, 1.56%, 0.39% and so on).

Straight Skeleton Subdividing

As mentioned above I have not yet implemented this algorithm so I can’t go into a lot of detail about the implementation.

Straight Skeleton Subdividing (SSS) is an approach to generate lots in one single step. It works only on long thin parcels - quality of the generated lots drops as the initial parcel approaches a square shape - it’s best used for suburbs with long winding roads. The straight skeleton of a shape is a line which is what you’d end up with if you collapsed a shape inwards at an equal rate from all points. This is incredibly complicated to generate, in fact I gave up attempting to implement it myself in C# and ended up writing a C# wrapper around CGAL to access the methods I needed. Wikipedia puts the runtime cost at something like O(N^3 log N), which is pretty scary!

Once you have the straight skeleton, lots can be placed along the edges connecting between the external points and the skeleton. This leaves annoying triangles at the ends which need to be somehow detected and removed. Requiring these kind of heuristics is part of the reason I haven’t implemented this yet.

What’s Next?

In this post I’ve covered two algorithms for how to generate sensibly shaped building lots, as well as implementation details for one of them. Next time we’ll look at the complex relationships which are tracked when deciding what buildings to place in each lot.

Procedural Generation For Dummies

Fri, 11 Dec 2015 00:00:00 +0000

Procedural City Generation For Dummies

My game, Heist, is a cooperative stealth game set in a procedurally generated city. Why put in all the effort to writing a procedural generator when it would be far easier to simply hand build some levels, as most other games do? Fundamentally stealth games are about solving puzzles - with the puzzle constructed in a 3D space out of alarms and NPC guards. With just a few hand constructed levels a player will rapidly learn the solutions to the puzzles and the game has no replay value.

A solution to this is procedural generation of some element of the game which ensures that the puzzle is a little different each time. A great example of this is Left 4 Dead (L4D) which originally had just four maps! It uses procedural narrative to vary the events which happen in response to player actions. This keeps the tension high and makes every play through unique - ensuring that L4D is fun to play again and again.

L4D is a game about shooting zombies, so the procedural aspect of the game varies the timing of zombie attacks. The core of a stealth game is exploring the level to find the locations of guards and alarms so that’s where I need to focus my procedural generation - generating maps.

In this series I will explain the techniques I am using for procedural city generation. This involves the generation of things such as road networks, building shapes and floorplans:

I may also extend into some more things I have planned for the future such as procedural generation of guard patrol paths, alarm system placement and gameplay guided prop placement.

Throughout the series it should be kept in mind that I am trying to create a generator for good looking buildings which are fun to play in - gameplay is far more important than realism! Sometimes a tradeoff between the two has to be made and I will always prefer the choice which enhances gameplay. For example many real buildings will only have one long straight corridor - this is a boring level, it’s far better to have alternative routes through the building to evade guards and alarms.

With that said, let’s dive right in with an overview of how the system is generally structured…

Start At The Top

Fundamentally the entire generator is a massive L-System. Each part of the system only knows how to increase the level of detail by one level and this is recursively applied until we reach the maximum level of detail. For example a typical game level might look like this:

Start With A “City”
“City” Creates roads and blocks
“Road” - Creates road geometry and street furniture
“Block” - Creates lots
“Lot” - Creates building occupying
“Building” - Creates floors and facades
“Facade” - Creates windows and doors
“Floor” - Creates rooms and corridors
“Room” - Creates game props such as furniture

If you start off with a city node and keep subdividing it’s ancestors you eventually end up with an entire city, full of buildings, where you can enter every single room of every single building.

With this structure creating a full procedural world generator becomes a matter of designing a number of separate generators for each individual part. I will cover one generator in each blog post in this series.

Gameplay In An Infinite World

So does this solve the infamous inaccessible area problem of large open world games? i.e. you have a huge explorable world (e.g. watchdogs, GTA) but most of it actually sits behind locked, indestructible doors and cannot ever be explored. In principle yes it does, you just generate everything in the entire city.

In practice I don’t plan to do this for two reasons. Firstly it’s extremely difficult to dynamically generate that much geometry on the fly without bringing powerful computer to it’s knees - and that’s without considering the fact that there is an entire game running at the same time!

Secondly it becomes hard to provide good directed gameplay when the player could be anywhere at any time. I plan to generate large areas, such as one single city block, and then keep the player confined within that area. If you think about games such as GTA and watchdogs they also do this - although you can go anywhere in the city at any time most of the actual gameplay (except car chases) is restricted to a small area of the city such as the single building which you are currently infiltrating. Watchdogs goes as far as to warn you when you’re leaving the “mission area” and you will fail the mission if you do not return - literally keeping you confined to an area of the city.

Follow Along

If you want to follow this blog series there are a number of ways to do so. You can follow me on Twitter or Reddit, Subscribe to my RSS feed, or keep checking back here. This list of posts will update with every new entry in the series:

Procedural Generation For Dummies: Road Generation

Fri, 11 Dec 2015 00:00:00 +0000

Procedural City Generation For Dummies Series

Road Generation

The majority of road generation systems I have looked at tend to be based on a growth system. The general algorithm (see page 15, Section 3.2 of this paper) for pretty much all of them is pretty simple.

Keep a priority queue of candidate road segments
- Initialised with a single seed segment
While priority queue is not empty
- Remove highest priority segment from queue
- Check local constraints on segment
  - This may modify the segment, or even reject it
- Add segment to output
- Generate new segments (connected to this segment) based on global goals

Or in pseudo code:

//Create a priority queue of things to process, add a single seed
let Q : priority queue;
Q.Add( 0, seed );

//Create a list of segments (the result we're building)
let S : segment list;

while !Q.IsEmpty()
{
    //Remove the highest priority item from the priority queue
    let t, segment = Q.RemoveSmallest();
    
    //Check that it is valid, skip to the next segment if it is not
    let modified = CheckLocalConstraints(segment);
    if (modified == null)
        continue;

    //It's valid, so add it to S. It is now part of the final result
    S.Add(segment);
    
    //Now produce potential segments leading off this road according to some global goal
    for (tn, sn) in GlobalGoals(segment)
        Q.Add(t + tn + 1, sn);    
}

Hopefully this is fairly clear. We simply generate a load of candidate points (in the GlobalGoals function), add them to a priority queue and then accept or reject each individual segment (in the CheckLocalConstraints method). The real variety between different algorithms comes in how you generate new constraints and how you express your global goals.

Global Goals

This is our method for producing new segments according to large scale global goals. For example one possible implementation would be to generate a single new segment at the end of the input segment which leads towards the local population centre. This would generate you one long road which leads from your random seed point directly to the population center and then stops.

The critical part of this method is that it generates segments with no concern for if they are possible. This means global goals can be very fast and simple to implement.

Local Constraints

This is our method for accepting, rejecting and modifying individual segments according to small scale local constraints. For example we could come up with a set of rules such as:

If a candidate segment crosses another segment then join the roads together to form a T-Junction.
If a candidate segment stops near to another segment then extend it to join the roads and form a T-Junction.
If a candidate segment stops near an existing T-Junction then extend it to join the junction and form a cross-junction.

As you can see these rules are all about correcting small errors to improve the local consistency of the road network.

Tremendous Tensors

My implementation is based off the paper Interactive Procedural Street Modeling (Chen, Esch, Wonka, Mueller, Zhang, 2008). If you’re not up for reading the paper then there’s a short (5 minute) video here with an overview of the technique.

How this systems works is twofold. First you generate a tensor field (if you don’t know what a tensor is don’t worry about it - in this system the tensors are just 2 dimensional vectors) and then you trace lines through the field (following the tensors).

Every tensor in this system has two eigen vectors (once again, if you don’t know the exact mathematical definition don’t worry about it). The two eigen vectors are always perpendicular and we’re going to refer to them as major and minor. We do the tracing through the field twice - once along the major vectors and once along the minor vectors.

So what use are these tensor fields? Above you can see a tracing of the most basic tensor field - a grid. This is simply the same tensor repeated everywhere and since the major and minor vectors of the tensor are perpendicular we end up with a grid! The paper defines several methods for generating tensor fields for different situations:

Grid
- Major points across grid.
- Minor points along grid.
Radial
- Major points to centre.
- Minor points around centre
Heightmap
- Major points along gradient.
- Minor points across gradient
Polyline
- Major points to nearest point on path.
- Minor points along path

The really cool thing about tensors is that we have ways to easily combine them. If you have multiple tensor fields it turns out that if you simply take a weighted average of all the fields you end up with a sensible result which you can trace. If your weight is set to fall off with distance you can define different tensor fields in different locations and they will slowly blend together as the weight changes!

Here’s a far more complex road network, generated by blending together multiple different fields:

Grid Top Left
Grid Top Right
Grid Bottom Left
Grid Bottom Right
Radial Centre

You can clearly see the five different elements and how the roads smoothly transition from following one to following the next.

Implementation Details

As mentioned at the start of the article the source code for my implementation of this system is here. The specific code for tracing through tensor fields and building a road network is here.

Before I dive into details I should say that this implementation is one of the parts of the city generation system I am most unhappy about. Tracing vectors is extremely sensitive to errors because the errors accumulate along the entire length of the streamline. Additionally the tracing can be quite slow, most of the example images in this post took 5-10 seconds to generate (which isn’t unusably slow, as we only need to do this step once per city).

Design Your Tensor Field

The first step is to build a tensor field. Right now I’m just doing this by hand but ultimately I’d like to do this automatically - for example:

Cover the entire map with a heightmap conforming tensor field
Place gridlines/radials at major population centres
Place a polyline along any major terrain features such as rivers or cliffs

In my implementation I have a base interface for tensor fields which can be sampled at any point. I then have multiple concrete implementations for different tensor field types.

Once the tensor field is created I have a single step which converts the tensor field into an eigen field. An eigen field is simply just a pair of vector fields along the eigen vectors; one along the major eigens and one along the minor eigens. This conversion process samples the tensor field (which a fairly expensive process with a lot of mathematical operations) at set intervals and caches the value. This caching more than tripled the speed of the overall system!

Trace Your Vectors

Now we have two vector fields, we need to trace a line through these fields, a streamline. Conceptually tracing through a vector field is trivial:

let point = start
until (some end condition)
    let direction = sample_vector_field(point)
  point += direction

However this turns out not to work very well. Because this only samples one single point from the field it is extremely sensitive to local noise and does not work at all when the field is highly curved (e.g. around a radial field). Instead I use an RK4 integrator to handle the higher curvature (see this excellent article for more details on integration):

function rk4_sample_vector_field (point)

    let k1 = sample_vector_field(point)
    let k2 = sample_vector_field(point + k1 / 2f)
    let k3 = sample_vector_field(point + k2 / 2f)
    let k4 = sample_vector_field(point + k3f)

    return k1 / 6f + k2 / 3f + k3 / 3f + k4 / 6f

Another problem to handle is that when we’re tracing through the field we don’t care about positive or negative direction - a gridline left to right is the same as a gridline right to left. I handle this by again modifying how I sample the vector field. When a sample is taken it compares the direction of the sample to the direction of the previous vector; if they differ by more than 90 degrees the direction is reversed:

function corrected_sample_vector(point, previous_direction)

    # Sample the vector field, For real this time!
    let sample = sample_vector_field(point)
  
    # If previous is zero that's a degenerate case, just bail out
    # Dot product >= zero indicates angle < 90
    if (previous_direction == Vector2.Zero || Dot(previous_direction, sample) >= 0)
    return v;

    # Since we didn't return one of the cases above, reverse the direction
    return -v;

Back To Basics

How does this all fit into the algorithm I outlined at the top? This vector tracing is effectively the Global Goals function - a single streamline is a collection of candidate segments to add into the map. The global goals in this case are the underlying tensor fields.

How we handle the line segments traced through the vector field is our Local Constraints function.

Rather than keeping a buffer of all the segments of a streamline and handling them one by one, an entire streamline is traced out and stops when a segment is rejected - this is just an optimisation to save a load of bookkeeping work. You can see the method which traces streamlines here and the method which creates each segment (and checks constraints) here.

If any of these conditions is met the streamline is stopped:

Segment is too short
Segment reaches the edge of the map
Segment ends very near an existing vertex
- Connect end of streamline to that vertex
Segment intersects another segment
- Connect on to segment with a T-Junction
Segment connects back to an earlier vertex in the streamline (i.e. forms a loop)

These checks are all implemented with two quadtrees. One keeps track of all vertices in the map and the other keeps track of all segments in the map. This way checking for overlaps and nearby vertices is roughly a log(N) operation (where N is number of vertices/segments in map).

Streamline #2

So far I have described the process of tracing out one single streamline through a vector field until it terminates. However, a road network is obviously formed from more than a single streamline!

As a streamline is being traced seed points are created and added to a priority queue. Each seed is associated with two pieces of information; the position and which field to trace the new streamline through. If we’re currently tracing a major streamline then the new one will be minor and vice versa.

When a streamline ends (due to one of the conditions laid out above) the highest priority seed is selected and becomes the start of a new streamline. Priority of the seed is simply sampled from a scalar field - this could be something like a population density field so that there are more roads created in high population areas.

What’s Next?

In this post I’ve covered a high level overview of how a tensor based road growth system works, as well as details on implementation of such a system. Next time we’ll look at how to generate building lots in the spaces between roads.

Deferred Transparency

Fri, 09 Oct 2015 00:00:00 +0000

TL;DR

I’ve added a form of transparency into my deferred renderer, Myre.

Deferred Renderers Hate Transparency

Deferred rendering is a pretty common technique rendering today, it is separate from the more traditional “forward rendering”. A forward renderer is a pretty basic concept, we just render each object one by one and each object takes into consideration all lights. So fundamentally the rendering algorithm looks like this:

for object in scene
  for light in scene
    render(object, light)

All we’re doing here is looping over the objects we want to be drawn, and drawing them. When an object is drawn it writes the final pixel colour values to the display. This has some pretty serious disadvantages. It should be obvious that the cost of rendering is proportional to the number of objects multiplied by the number of lights. Additionally because we can only pass so many parameters into a pixel shader we have a very small limit on the number of lights we can apply to the scene. Finally we have a lot of overdraw costs here - each item we draw does all the work to calculate pixel lighting and then could be overwritten by another object later.

A deferred renderer is a different approach:

for object in scene
    renderData(object)
    
for light in scene
    renderLighting(light)

What we’re doing here is looping through the objects in the scene and rendering out data about the object into a buffer - things like surface colour, normal and shininess. This looks a bit like this:

We then loop over all the lights in the scene and calculate the final colour for each pixel on the display. This is a huge performance win because now the cost is proportional to number of objects plus number of lights! Also each light is renderer separately so we can have an unlimited number of lights. Finally we completely eliminate overdraw costs because the overdraw is in the (very cheap) renderData phase, not the costly renderLighting phase.

So this is great, deferred renderers save overdraw, decrease lighting costs and support unlimited lights - what’s the catch? The catch is that we cannot do transparencies! Each pixel in our data buffer has the information for one single object and transparency is fundamentally about having two objects contributing to a single pixel.

Making The Impossible Possible

We’ve established that a single deferred rendering pass can only draw opaque things, there are three solutions to this problem:

1. Stippled Transparency

One possible solution is to write both the pixels into the data buffer. Obviously it’s not possible to write two things into exactly the same pixel - but what if we wrote the information into two pixels very close together and then blurred them together later? This can definitely work and has the advantage of requiring no changes to the pipeline.

Unfortunately this technique has some serious practical limitations. Many shaders in the pipeline will sample a few nearby pixels from the data buffer (assuming that those pixels are likely to be from the same object). If you start interleaving objects like this a lot of effects can break. For example SSAO works by sampling the differences in depth and normal direction from pixel to pixel - interleaving like this makes SSAO go nuts. In the end you need to pack an object ID into every pixel, and then modify all your effects to check and handle the object ID.

2. Forward Rendering

Another solution is not to use a deferred renderer at all! All the opaque objects in the scene can be rendered with a deferred renderer as normal and then you can do another pass just for the transparent objects with a forward renderer.

This is a pretty good solution, using each renderer only for what it’s best at. Unfortunately it has a obvious drawback in that you need to build and maintain two completely separate renderers!

3. Multiple Depth-Peeled Deferred Passes

The final solution is to use multiple passes of a deferred renderer. In each pass we rerun the entire deferred renderer for a transparent object and then at the end we collapse the layers back down into a single layer, blending the colours appropriately for each pixel. This has the advantage that you already have the entire pipeline for deferred rendering, any effects that you apply to an opaque layer can be applied just the same to a transparent layer.

The primary disadvantage of this technique is we repeat the entire (costly) lighting stage for each layer.

This is the approach I have decided to use.

Depth Peeling

If we’re going to draw multiple layers of items we need to know what’s in each layer. The simplest way to do this would be to draw every transparent object in it’s own layer and then to merge all the layers back together. However this could end up with hundreds of layers, which would mean hundreds of lighting passes and that would bring even the most powerful GPU to a standstill! We need some better technique for packing items into layers.

What we really want to do here is pack items which do not overlap in screen space into the same layer. Here’s an example scene (top down view):

The camera is a point on the right. The red lines show how the orange sphere overlaps the blue sphere. If we were packing this into as few layers we’d have two layers:

Blue, Green, Pink
Orange

If we render the blue+green+pink spheres first and then render the orange sphere second we’ll get all the transparencies correctly.

Finding overlaps is pretty easy, just project the bounding sphere of each item into screen space then perform a 2D intersection test. Finding the best way to pack the items into the smallest number of layers is still an open problem, my current solution has a limited number of layers (max 5) and then just puts each item into the last layer it can without causing an overlap. If it can’t put it into any layer the system gives up and sticks it in the first layer! A better system would consider how much error (number of overlapping pixels) is caused by putting each item into each layer and the globally minimising this.

Rendering each layer is pretty simple. Just rerun the entire deferred pipeline again, but only drawing items from the current layer. Also re-use the depth buffer from the previous layer/opaque stage and you get pixel perfect depth rejection.

Blending

Once a layer has been rendered we need to blend that layer back into the result of the opaque rendering. At this stage we have two lightbuffers:

To blend them we render the geometry from this layer again. The shader this time can take in the two lightbuffers and blend them. It may seem like a waste to render all the geometry again, but this allows us to store information in the vertices as well as cutting down the copied pixels to only those pixels which are covered by geometry.

A clever little trick here is that for this stage we render the back geometry and flip around the cullmode. This means the vertex shader gets the back geomtry passed in, we’ll see why that’s useful in a second…

//Work out where on screen this pixel in. This tells us where we need to read from in the light buffers
float2 screenPos = input.PositionCS.xy / input.PositionCS.w;
float2 texCoord = float2(
	 (1 + screenPos.x) / 2 + (0.5 / Resolution.x),
	 (1 - screenPos.y) / 2 + (0.5 / Resolution.y)
);

//The normal buffer alpha channel set to zero if nothing was written here. Sample the value as use zero-values as an early exit (a kind of stencil buffer)
float4 sampledNormals = tex2D(normalSampler, texCoord);
if (sampledNormals.a == 0)
    clip(-1);

//This is the clever bit for back-face geometry. the depth of this vertex (input.Depth) is the back of the object...
//...while the value in the depth buffer is the depth from the front of the geometry we rendered before into the gbuffer!
//Thickness is easy to calculate.
float backDepth = input.Depth;
float frontDepth = tex2D(depthSampler, texCoord);
float thickness = (backDepth - frontDepth) * (FarClip - NearClip);

//This is the surface colour of the transparent object
float3 opaque = tex2D(transparencyLightbufferSampler, texCoord).rgb;

//This is the colour of the scene behind the object
float3 background = tex2D(lightbufferSample, texCoord).rgb;

//now blend them. Blending function varies per material
return Blend(opaque, background, thickness);

The great thing is that because we render all the geometry again we can run different shaders for different objects for the blending. By varying the blend function we can get effects like glass, marble, skin or wax.

Drawbacks

This technique, as I’ve implemented it at the moment, has a lot of drawbacks. It’s ok for my purposes because I basically just want it for rendering flat, non-overlapping windows on buildings.

Reflections

Currently there are no reflections. This isn’t really a drawback of the transparency rendering so much as it’s a drawback of the renderer as a whole. However transparent objects often have very pronounced reflections and so this becomes a real problem.

Concave Geometry

If an object overlaps itself we have a problem. This will not render properly at all and it will look like the occluded bit isn’t there at all! This isn’t a problem for me, because I want to draw nice planar sheets of glass which can’t possibly self intersect.

Backside Illumination

Light on the back of a transparent object can be important. For example if you hold your hand up to a bright light you see red illumination through the skin. Because this technique only works out the lightbuffer for the frontside geometry you can’t simulate that kind of effect. A possible solution for this would be to render two light buffers per pass, one special one which renders onto the backside of gbuffer geometry.

How Does This Help Heist?

I’ve kind of been quiet about Heist for a few months, hopefully this post will be the end of that! My “stealth mode” has been because I was talking (particularly in my videos) about fairly mundane features. These features are important but not particularly exciting. I’ve been working hard on the procedural generation so I can generate bigger scenes with greater fidelity - the plan being that if I’m going to demonstrate a mundane feature at least I can demonstrate it in a 2x2Km chunk of procedurally generated city!

Xna In 2015

Mon, 24 Aug 2015 00:00:00 +0000

TL;DR

I’m updating a whole load of my libraries to the latest version of .Net framework.

A Long Overdue Update

Heist is not a standalone project, it is based on a huge collection of libraries - many of which I myself built and maintain. Chief amongst these are the open source Myre framework which I took over maintaining from a friend of mine several years ago and the closed source XNA framework released by Microsoft in 2004!

For the last couple of years Myre has been in a kind of Maintenance mode - changes have only been made to fix bugs or to add critical new features that I needed downstream. Obviously this has left some things to go a little bit rotten. Additionally XNA imposes some fairly annoying requirements on the user - I’ve never been successful in getting it to work with newer framework versions than .Net v4.0, and it refuses to build in anything other than x86 mode. On top of all this XNA is windows only and is long since abandoned by Microsoft.

Because of these problems I would really like to move away from XNA in the long term. Monogame is the obvious choice in the medium term, but longer term I’d like to move to something more powerful such as BGFX.

The most obvious usage of XNA throughout the codebase of all my libraries is Vector types. XNA defines types such as Vector2, Quaternion and Matrix and they’re used absolutely everywhere through all my game development libraries. To add some extra complexity to this problem the XNA vector types are not used in all of my libraries - non XNA libraries such as SupersonicSound (my open source FMOD C# wrapper) and HandyCollections (a set of useful collections) define their own vector types so that they don’t have to take a dependency on XNA.

This all means that by the time you get to the Heist you have:

.Net v4.0 at best (Pretty ancient)
x86 only (No 64 bit version, constrained memory use)
Windows only (No love for Linux/Mac)
4 different (incompatible) vector types

Light At The (Start) Of The Tunnel

The latest version of .Net released recently. This version can be used all the way back to windows 7 and thanks to .Net Core is going to be on other platforms eventually (almost certainly before I put in the work to go cross-platform). This version adds a load of things to the standard library including Vector types! These are types I can happily use in all of my projects without taking an annoying dependency on XNA.

Step 1 - Myre

I’ve just finished updating Myre (it will be pushed up to Github and Nuget soon). There were a couple of kinks I would like to document for the future (largely for my own benefit).

Replace Project Files

Traditionally libraries for XNA projects were not just C# library projects, instead they were a special “XNA Library” project type. This is restrictive in various ways (e.g. cannot change the target framework version at all). Step one is to replace all your XNA library projects with standard C# class libraries. I achieved this by removing the old project, renaming the directory in explorer and then creating a new project with the same name and just duplicating the structure/file of the old project. Finally you need to add in references, for example to XNA.

You may now freely change target framework (as with any other class library). It’s likely still important to build for x86 as not doing so could cause runtime errors within XNA.

My Game Is Not A Library

The game itself is not an XNA library (obviously), instead it is an XNA game project. This can be replaced just like any other XNA project - simply create a C# class library then in the project properties change the “Output Type” to Windows Application and the “Startup Object” to your entry method (most likely Program.Main).

XNA projects expose a property in their project properties page which allows you to specify the framework version - Reach or HiDef. You will always want to use HiDef but it defaults to Reach. Unfortunately now that your game is not an XNA project there is no way to set it in project properties. Instead when you create your GraphicsDeviceManager (usually the first thing you do in your game constructor) you should set the GraphicsProfile proeprty to HiDef:

GraphicsDeviceManager graphics = new GraphicsDeviceManager(this) {
    GraphicsProfile = GraphicsProfile.HiDef,
};

Content Is Complex

XNA has special “Content Projects” which build all of your Content when you build your game. These are not like any other project type and you must keep the content project. You will need to update references in your content project to any content pipeline extensions (which will have been replaced in the last step). XNA projects can have a special reference called a “content reference” which causes the referenced content project to be built, and then copied into the output directory - there is no such reference type for class libraries.

To take advantage of this behaviour simply create a new empty XNA library project (I call mine MyGame.ContentShim) and add the content reference to that. Now your game can add a normal reference to the content shim project and content will be built and copied correctly!

There is one final kink for using Content Pipeline Extensions (loading your own code into the content building process pipeline). Your content pipeline project is now a C# class library, targeting some arbitrary framework version of your choice (v4.6 in my case) but the content project is targeting v4.0 - this will cause runtime failures (content pipeline can’t find or use your alien code from the future). To fix this you need to edit the *.contentproj file. Find the line:

<TargetFrameworkVersion>v4.0</TargetFrameworkVersion>

and replace it with:

<TargetFrameworkVersion>v4.6</TargetFrameworkVersion>

Now your content pipeline will run on the framework of your choice!

Parting Note - How Does This Help Heist?

This is all an effort to reduce my dependency on XNA, with a view to eventually removing XNA altogether. This will allow me to use newer framework versions (better GC), go cross platform (linux and mac), take advantage of SIMD (CPU speedup) and create a 64 bit version of the game (less memory pressure).

Node Machine

Fri, 26 Jun 2015 00:00:00 +0000

TL;DR

I’ve been working on the procedural generation toolkit NODE/Machine.

Why NODE/Machine?

About a month ago I decided that one thing I really needed to be able to show off the game properly was a decent looking level. A huge amount of work has gone into my procedural generation system and I have a lot of really powerful tech lying around unused. There is a good reason this tech has gone unused: writing scripts to define geometric objects is a massive pain in the ass. Now if I, the creator of the system, find this to be a problem no one else is going to stand a chance so something had to be done.

Enter Stage: NODE/Machine

NODE/Machine (NM) is a procedural generation toolkit I have been building to help solve this problem. It is essentially an IDE and a collection of domain specific languages for creating various parts of a procedural world. The first part of this was mentioned in my last post; a 2D layout system for designing procedural facades. I’ve put out a few videos about this on twitter, here’s the latest one on facades:

As you can see from this there’s some markup (inspired by WPF and syntactically based on YAML). This markup has some elements specifically designed for facades. For example the “Fit” element has a single child and clones it multiple times into the available space - this wouldn’t be very useful for a UI layout language but is great for facades where you often want to repeat a common element such as a window several times.

Another key feature of NODE/Machine that you can see here is that it connects to the game and sends designs across instantly. This means you don’t have to visualise the complex geometry that you’re working on in your head, and you don’t have to wait a long time to restart the game and reload the entire level just to see one item.

Building Ex NODE/Machina

The next feature I worked on after facades was the first part of building generation - what floors are placed into a building. This is basically a 1 dimensional problem - just a basic stack of floors. I took the same approach as with facade markup; creating a DSL based on YAML for laying out floors in a single dimension.

Here’s an example of a building:

!Building
Aliases:
  - &residential_floor_count !NormalValue
    Min: 2
    Max: 5
Verticals:
    # First lift from ground->lowest skylobby
    - Tags: { 1: [lift] }
      Bottom: !Num { N: 0 }
      Top: !Id { Id: Skylobby, Search: Up, Filter: First }

    # Set of lifts from skylobby up to next skylobby
    - Tags: { 1: [lift] }
      Bottom: !Id { Id: Skylobby }
      Top: !Id { Id: Skylobby, Search: Up, Filter: First }

    # Express lift for penthouse
    - Tags: { 1: [lift] }
      Bottom: !Num { N: 0 }
      Top: !Id { Id: Penthouse }
Floors:
  - !Floor
    Tags: { 50: [roof, garden], 50: [roof, helipad] }
  - !Floor
    Tags: { 50: [penthouse], 50: null }
  - !Repeat
    Count:
      Min: 1
      Max: 5
    Items:
      - !Floor
        Id: Skylobby
        Tags: { 1: [skylobby] }
      - !Range
        Includes:
          - Count: *residential_floor_count
            Tags: { 1: [apartment] }
  - !Floor
    Tags: { 1: [ground floor, lobby, shops] }
  - !Ground {}

There’s really two main things going on here, floors and verticals. Let’s look at floors first. The lowest items in the floors array is a marker for the ground, anything below this is a basement. Immediately above this is a !Floor tag which means we’re specifying a single floor. The items in the Tags object define what this floor will be, a floorplan script will be found which has all of the specified tags. The number, which is the key, is the probability of this set of tags being selected, so if you look further up you’ll see some floors with two options and a 50/50 chance of choosing each.

The floors also have a !Repeat tag, which means the children of this item are repeated some number of times. Here the count is set to between 1 and 5 repeats in the Count property. The items being repeated are, from the top down, a single floor (a skylobby, in this case) and then a range of floors.

A !Range is a set of includes which be output by the parent, but in no specific order. There’s only one include in this example, so this range isn’t really very useful. An interesting thing to note is that the Count here is set to *residential_floor_count - what does that mean? YAML allows us to define named objects, such as:

- &residential_floor_count !NormalValue
  Min: 5
  Max: 10

We can then reference this named object somewhere later in the markup. If you look right at the top you see the residential_floor_count is defined in the Aliases property. This is important because every time the range gets repeated we always use exactly the same Count object - this means every repeat has the same number of floors included.

Finally at the top we have two more floors. One is a 50/50 chance of a penthouse or null, if null is selected then no floor is created at all. The other is simply a roof with a 50/50 choice between two different types of roof.

An example output from all this is:

roof, garden
penthouse
skylobby
apartment
apartment
apartment
skylobby
apartment
apartment
apartment
skylobby
apartment
apartment
apartment
ground floor, lobby, shops

Exactly what we expected: A ground floor, followed by several exactly duplicated runs of apartments and skylobbies, with a penthouse and a roof garden at the top.

The other part of that example code above is vertical elements, this is things like stairs, lifts, fire escapes and ventilation shafts which are vertically oriented and thus cross several floors. The code for this, duplicated from above, is:

Verticals:
    # First lift from ground->lowest skylobby
    - Tags: { 1: [lift] }
      Bottom: !Num { N: 0 }
      Top: !Id { Id: Skylobby, Search: Up, Filter: First }

    # Set of lifts from skylobby up to next skylobby
    - Tags: { 1: [lift] }
      Bottom: !Id { Id: Skylobby }
      Top: !Id { Id: Skylobby, Search: Up, Filter: First }

    # Express lift for penthouse
    - Tags: { 1: [lift] }
      Bottom: !Num { N: 0 }
      Top: !Id { Id: Penthouse }

There are three vertical elements here. Each one has a set of tags which works in the same way as floors (multiple sets of tags, and relative weighted probabilities). The Bottom and Top properties are “floor selectors” (vaguely inspired by CSS selectors) which matche sets of elements. Vertical elements are created between every pair in the top and bottom sets.

Hopefully it should be fairly self explanatory how these work. The top one is hardcoded to match floor 0, and the finds floors with ID “Skylobby”, the Search direction and filter ensures it only finds at most one floor and that will be the lowest skylobby. The middle one finds all skylobbies as the bottom, and then from each skylobby finds the single next skylobby upwards. Finally the third one matches the ground floor (floor 0) and the penthouse - in some cases the penthouse doesn’t exist in which case this selector simply does nothing because it creates zero pairs of floors.

Here’s my latest video featuring building floor selection:

## What Are All These Embedded Videos?

I haven’t done a full youtube update for a couple of weeks; those videos take several hours to script, film, record, edit and upload (even if they’re only a few minutes long). Instead I have been experimenting with making “micro updates” in webm form and then sharing them on twitter. These videos are entirely unscripted, and I don’t edit them at all - they take as long to make as they do to watch! I’ve really enjoyed making these and I actually plan to start doing one a day, just covering whatever random progress I made that day.

If you want to catch these micro video updates follow me on Twitter.

Procedural Generation Of Facades And Other Miscellania

Fri, 29 May 2015 00:00:00 +0000

TL;DR

FMOD audio. Expanded geometry system. Failing at animations. Mixamo. Procedural Generation Toolkit. Facade Generation.

Oops

It’s been one entire month since I last did a blog post (and my last one even promised a topic coming soon)… oops. I’ve been crunching hard this month and working on loads of different features to try and bring various things up to a quality I would be happy to release. Of course, lots of polishing changes don’t make for interesting blog posts! On top of that, I spent a lot of time working on some stuff which ultimately I failed at so that was a lot of time spent with no results to talk about in a blog posts.

FMOD Audio

The old audio system in game was built on the basic XNA 3D audio system - it could render sounds in 3D and that was it - I have been intending to replace it with something better for months. Just recently I encountered a bug in the old audio system which was (occasionally) crashing the game. Rather than spending a load of time fixing an ultimately useless system I decided to bite the bullet and go ahead with my long planned swap to FMOD.

What is FMOD anyway? To quote myself from a few months back:

FMOD studio is an audio creation tool for your audio artists to use to create complex sound effects - using the studio you can create sounds which depends upon parameters and run complicated state machines and then the game can tweak those parameters at runtime and make your sounds responsive to in game events.

The FMOD programmers API is what you use in your game to playback these complicated sound effects. It handles stuff like loading lots of different file formats, generating sounds and effects in realtime, controlling occlusion and reverberation from geometry and positional sound.

That quote is from a blog post where I announced the release of SuperSonicSound, an open source library which wraps the FMOD C++ style API and presents it in a much more friendly C# style.

FMOD is great because it allows me to author really complex sounds including stuff such as multiple sounds playing, cues from the game, parameters from the game. When I load this sound up in the game I just… play it. No worrying about any of that complexity!

So here’s an example of the reload sound for a magnum:

At the start are 3 overlapping multisounds. Each time a multisound plays it chooses a single sound from a set to play, in this case it chooses randomly. After a small delay there is another multisound which randomly chooses one of several clicking noises for the chamber being closed. On top of all this there is some automatic modulation of pitch and volume - every single time any of these sounds play their pitch and modulation and randomly varied by a small amount.

All of this complexity was really easy to create in the editor, and to use it all in game the code is as simple as:

event = audioService.GetEvent("WeaponSounds/Magnum/Reload");
audioService.Play(event);

Awesome!

Characters And Animations

The next big thing I came to work on was characters and animations. At the moment the game has precisely one character, Zoe:

Zoe isn’t even the right style of character for the game so I need to totally new set of characters, preferably hundreds of them so I can fill up crowds of NPCs on the street. I found this amazing service called Mixamo which allows you to create characters in a tool called Fuse and then you can pick animations from their library of animations (which is huge, and expanding all the time) and then you just download it all and you’re good to go.

I signed up immediately! A few characters I quickly threw together in Fuse look pretty good:

In my experience anything to do with graphics and animations is a total nightmare. Unfortunately this time was no different. The obvious way to use Mixamo is to attach the same animations to every single one of my characters and download that, nice and simple. There are two problems here:

Mixamo charges per animation per character. This means I would need to buy my animations hundreds of times, once per NPC, which would be extremely expensive!
I would have hundreds of copies of the same animation on disk and in memory. Animations are actually surprisingly large and the game would probably be many gigabytes (and require just as much memory).

Luckily the solution to this isn’t too complex. Rather than downloading the same animations hundreds of times, just download them once and then retarget them to whatever NPC you want (at runtime). I spent an entire week working on this and still don’t have all of the bugs ironed out! The problem is that there are so many things that can go wrong:

Maybe the different characters have different skeletons? (I later checked with Mixamo, they don’t).
Maybe I’m an idiot and this approach just won’t work! (I also checked this with Mixamo, the basic idea is fine).
Perhaps the FBX file format sucks, I’ll use Collada instead
Perhaps the collada importer is broken, I’ll use Collada for models and BVH for animations
There’s no BVH importer for XNA, guess I’ll write my own…
Perhaps my retargetting code is wrong (probably)

There are loads of complex little issues here, often to do with some annoying thing like file formats or matrix handedness. After two weeks working on this I put it on the backburner - Mixamo is an awesome service and I’m definitely going to have to do this work in the near future, but for now it’s on hold.

Procedural Generation Toolkit

Procedural generation has always been a core part of what is cool and unique about this game; there’s a limitless number of banks to rob! Sometimes a bank will be insecure because of what is next door and with procedural generation you can explore that possibility. Procedural generation was the earliest part of the engine which I built (a couple of years ago) and although I’ve come back to it a few times to add more complex features (e.g. Floorplan generation) I’ve never really properly demonstrated the full power of this fully armed and operational procedural development engine.

There’s a reason for this lack of complex demos: writing generation scripts is hard, it requires visualising lots of geometry in your head. This is why I’ve been using this same test level for so long, it hasn’t been worth the effort to build a huge new level.

Obviously before I make any kind of release I’m going to need some proper levels and I can’t rely upon building them entirely in code. To this end I’ve started building a tool called NODE/Machine. NODE/Machine is a toolkit for designing and building levels using the procedural generation system - split up into several independent tools for different jobs. All the tools link up to the game, so you can design an element (e.g. a floorplan) and then click a button to see it in game instantly.

Facades

This week I have been working on facade generation. It’s heavily work in progress, but here’s a quick demo of me throwing together a basic facade in NODE/Machine:

There are a couple of bugs here, but hopefully you get the idea - with this simple little markup language you can design walls to go into the game! Next week I’ll (hopefully) have these bugs stamped out and have a much more complicated and impressive looking demo actually rendering in game.

This Path Was Made For Walking

Thu, 23 Apr 2015 00:00:00 +0000

TL;DR

String pulling and steering behaviours are a good combination for walking along a path.

Path Following

NPCs have two stages of work to get to a destination. First they need to perform pathfinding, this is where you use A* (or something similar) on a navmesh which will find a broad path to follow to get to the destination. Secondly, the NPC needs to follow the path.

I’ve talked about pathfinding before, as well as some of my work with the navmeshes which pathfinding works with. These past few weeks I have been working on path following, experimenting with different techniques to see what works best.

Isn’t Path Following Really Simple?

It seems like path following ought to be dead simple, you’ve done the hard bit with your A* search now you just need to walk along the lines connecting the navmesh regions! The problem with this is it can result in dreadful paths with most navmesh geometries. For example, here is a basic floor plan with 2 possible paths drawn onto it:

The red line is the kind of path a human would naturally take, it’s the most direct way to get from A to B because it cuts in close to the corners. This is the kind of behaviour we want from our NPCs.

The blue dots are the center points of each navmesh section and the pink line is what you get if you simply walk along those points - it looks dreadful!

So we need some kind of algorithm which will take in a series of navmesh faces and return a series of points which is the “human like” path to follow.

Steering Behaviours

Absolutely ages ago I forked SharpSteer2 to introduce a load of code quality improvements, use modern C# features, fix bugs and introduce new features. SharpSteer2 is a port of OpenSteer, a library which provides basic primitives that can be used to build complex steering behaviours.

My fork of SharpSteer2 is actively maintained and is available on Github and Nuget.

I experimented with passing the pink path into steering behaviours and then trying to predict ahead along the path to smooth out the rough edges. This is actually what Left4Dead Zombies do (see Slide 14). However I could not get this to work convincingly, smoothing can only look so far ahead and will always be defeated by really long sections. Additionally as the NPC walks along the path she wiggles around as she discovers new path features, this is probably acceptable for a great big mob of zombies charging at the player but it does not look like how humans walk - we take straight lines or big long curves - and would be very obvious in a slower paced stealth game.

Funnel Narrowing

The funnel narrowing algorithm is an algorithm to find a path formed from straight lines along a series of portals. The “portals” in this case are the edges of the navmesh faces along the path, they are perpendicular to the path and indicate gateways that must be passed through.

The algorithm is described in detail here and comes with code to implement it (not C#, but easy enough to transcribe). The essence of funnel narrowing is that we have a triangle (a “funnel”) from the point of the path we’re currently at to the two points on the next portal, this funnel can sometimes be narrows by moving one of the edges to connect to the next portal along. You simply keep narrowing the funnel until changing the edge would widen the funnel. Follow that link above if you’re interested in implementing it.

The funnel narrowing algorithm provides a line something like the red line in the image above which is what we wanted all along… but now what do we do with it?

Steering Behaviours, Part 2

Steering behaviours weren’t any good at solving the problem of finding a good humanlike path through the navmesh but they’re perfectly suited to following the output of the funnel narrowing algorithm. The whole point of steering behaviours is that they can be combined, so we can combine a path following steer with obstacle avoidance (e.g. avoid other people in hallways) to get nice natural path following.

This is dead simple to implement:

//Avoid other NPCs
var steer = SteerToAvoidCloseNeighbors(otherNpcs);

//If we're not avoiding anything, then follow the path
if (steer == Vector3.Zero)
{
    steer = SteerToFollowPath(path);
}

//Apply whatever steering force we calculated above
ApplySteeringForce(steer);

So, Are We Done?

That’s it! We can walk along a path, awesome. But what if:

The NPC leaves the path
The target position moves
Something more important than walking along this path happens

The NPC Leaves The Path

There are events in the game which could knock the NPC far off the path, for example say an explosion blasts the NPC away from the path. This is quite simple to handle; we simply check if the NPC is contained within one of the navmesh faces along the route. If the NPC leaves the navmesh path then we just recalculate the entire path from where they are currently standing.

The Target Position Moves

The target could be moving object, for example the NPC could be pursuing the player. Same as before we simply check if the player is within the navmesh path and if they are not recalculate a new path.

Something More Important Happens

There is more to life than walking along paths. Everything I have described here is just a simple behaviour to get to places; how we make actual decisions is a different topic which I shall cover soon.

Cross Chunk Navmeshes

Fri, 27 Mar 2015 00:00:00 +0000

TL;DR

Navmeshes now connect across chunks. No video because that just isn’t an interesting thing to make a video about!

This Week

I’ve worked on system for automatically connecting navigation meshes across chunk boundaries, allowing AIs to navigate the world just as well as humans can.

The world in the Epimetheus engine is loaded in chunks which are 50x50x50 meters, as the player moves around the world chunks are procedurally generated and loaded/unloaded in the background n theory allowing for very large worlds without any loading screens. Navigation meshes define the area NPCs may walk and are automatically generated for each individual chunk using the amazing recastnavigation system. However, until this week navmeshes did not go between chunks which means that an NPC could not navigate between chunks!

The world generation system is extremely complex because there are three systems involved and all of them run in parallel to the main game thread.

World Generator

The world generator is concerned with generating the data of the world, it creates the geometry and entities in the scene. The world generation system holds a topological map of the world, something like:

Building
- Floor
  - Room
  - Room
- Floor
  - Room
- Floor
  - Room
Building
- Floor
  - Room
- Floor
  - Room
  - Room

And then chooses appropriate nodes in this tree to subdivide and generate the actual physical geometry for. This system attempts to choose nodes near to the player so that as the player moves around the world everything is already generated before they get there.

Chunk Database

The chunk database is, as the name implies, a collection of chunks (50x50x50 meter cubes). Each chunk contains all the data about the world within itself. The chunk database has a very tight memory budget (essentially how many chunks it may keep in memory at any one time) and must write out chunks to disk to stay within budget. Other parts of the game engine can submit their requests for chunks to be loaded and the chunk database will attempt to satisfy them.

There’s some hidden complexity here, chunks store the physical and graphical geometry of the world within their bounds but we might not know what that is yet if the world generator hasn’t created it yet! The world generator is constantly submitting requests to load chunks so that it can modify their geometry, this means the chunk database has to be very fast to because otherwise it would easily become the bottleneck in world generation. Additionally the chunks store their navigation mesh, but once again we don’t know what the navigation mesh looks like if the geometry of this chunk has not been generated. The chunk system has to regenerate navigation meshes once the world generation system indicates that it has completed all it’s work within a chunk.

World Streaming System

The world streaming system is the final part of the puzzle. The other two systems simply generate data and store it appropriately, the world streaming system monitors the player and submits requests to the chunk database to load up chunks which the player needs loaded. For example, when the player moves the world streaming system will unload some chunks behind them and load up some chunks ahead of them.

There is a level of detail progression here, so within 100m of the player the world is fully loaded, rendered and simulated whilst further away the world will be loaded and rendered but is not actually simulated. Finally, very far from the player the chunks will be loaded into memory (reading from disk is slooooow) but will not be rendered.

I’m getting to that! Imagine that the world is completely empty to start with, the first thing that will happen is that the world generator will generate a chunk of geometry and indicate that it is finished, now a navmesh for that chunk will be generated. We can’t yet connect this navmesh to any other chunks because there aren’t any other chunks!

We need to keep track of the fact that this navmesh is generated but not yet connected, and then connect it to other navmeshes when we can. So imagine that we continue playing and the chunk next to this one is finished and generates a navmesh… but there’s no guarantee the first chunk is still loaded, it could be written to disk and we don’t want to load it into memory, which most likely forces something more important out of memory.

We need to keep track of which navmeshes are generated and which chunks they have been connected with. Rather than do all this extra bookkeeping I came up with what I think is quite a neat solution; for every unconnected part of the navmesh I simply put in a connection that links to nowhere (yet). When an NPC is trying to find a path and it asks where this connection leads we go off and search for a connection, if the chunk isn’t loaded we just return null and the AI can’t path that way. This means NPCs will try to path around unloaded chunks which is actually exactly what we want, an NPC which walks into an unloaded chunk will disappear from the game (until the chunk is loaded) and that kinda sucks.

Changelog 27

Fri, 06 Mar 2015 00:00:00 +0000

TL;DR

FXAA, TXAA (maybe), SSAO, Screen Space Decals and a little bit of work on sticky cover.

This Week

Watch The Video!

It’s been two weeks, largely taken up by a load of work on the underlying graphics engine (Myre.Graphics).

FXAA

FXAA was a vast improvement to my anti aliasing system. On the left is an example of my old AA system vs FXAA… if you’re wondering that left hand picture does have AA applied to it, that’s just how bad the old system was! Click the picture for a larger version.

FXAA still only solves half the problem. It makes static images look loads better but doesn’t do anything for flickering in moving images. For example the edges which are drawn around things in Heist flicker badly when they’re in the distance. I’ve been investigating something like TXAA applied just to the edge detection system to help solve this.

Decals

Wow these were complicated! I’ve already done a big in depth blog post on how these work over here, so check that out.

Dynamic Sticky Cover v0.2

I’ve finally got back to cover and as I said in the video a fresh perspective has really helped me stamp out some bugs in the last couple of days. Hopefully I can get this wrapped up next week (famous last words).

What’s Next?

After finishing off the dynamic sticky cover system I’ll probably add in-world control hints. So things like when you look at a surface it displays a little “Q” (or whatever your ‘take cover’ key is) to indicate you can take cover here.

Drawing Stuff On Other Stuff With Deferred Screenspace Decals

Fri, 27 Feb 2015 00:00:00 +0000

TL;DR

I implemented deferred screenspace decals into the Myre renderer - the open source renderer which powers Heist.

What Are Decals For?

Decals are used in games to draw images onto others surfaces. Every time you see a bullet hole or a blood splatter appear on a surface in a game that’s a decal in action. There are a few ways to do decals which all have different tradeoffs.

Traditional Decals

The traditional way to do decals is to build a mesh which conforms to the surface you want to the texture drawn on, and then to draw the decal onto that. The hard part here is extracting the appropriate mesh once you know where you want your decal, after that you can just render the decal however you render any other texture mapped object. There’s a pretty good overview of this technique over at wolfire.

The advantage of this technique is that you don’t need any changes to the rendering pipeline; whatever system you’re currently using can be used to render your decals just the same as anything else. The disadvantage, of course, if that extracting those surface meshes is pretty complex and you have to keep a load of decal geometry around for as long as your decals live. Furthermore surfaces very close together tend to suffer from Z-Fighting and you can’t get much closer together than an exactly conforming mesh in exactly the same place and so you have to carefully tune fiddly depth biases to prevent flickering.

Megatexture

A megatexture is a way to draw one vast texture over your entire world, initially made famous by Rage and John Carmack. There’s an interesting side effect to this; there’s a unique bit of texture for every bit of geometry in your game. So if you want to draw something on top of a piece of geometry you can just edit the megatexture. Planetary Annihilation uses this effect so when there’s an explosion mark drawn on the ground it’s just drawn into the megatexture and then stays there forever.

The advantages here are pretty obvious, you just draw something into your megatexture and you’re done and even better the decal is there forever! The disadvantage, however, is that you have to use megatextures. I’m not saying that megatextures are bad but they’re complicated to implement and rarely used.

Deferred Screen Space Decals

Screen space decals are an approach that only works in a deferred renderer, this would be a pretty big disadvantage if that didn’t describe basically every modern renderer in existence (including the one I use, which is all I really care about)!

With a deferred renderer when you draw an object you don’t draw out the finished image to the screen, instead you draw out information about the object to each pixel. Things like surface texture colour, depth, normal and shininess all get written into a texture called the GBuffer. Once the GBuffer is complete you follow that up with a series of passes which process the GBuffer information into a finished image. Screen space decals fit right into the middle of this pipeline - they modify the GBuffer and change the data written there. Before and after that it’s just a normal deferred rendering pipeline.

So How Do They Work?

Let’s have a look at the start of our pipeline. First we have our gbuffer with normals, this encodes the a vector pointing straight up out of the surface for every pixel.

Then we have our depth buffer, this encodes the distance from the camera to the pixel.

And finally we have our diffuse buffer, this encodes the colour of every pixel (from texturing, no lighting or anything like that).

Our decal obviously wants to modify the diffuse buffer (to change the colour of the surface, that’s the entire point) and the normal buffer (slightly less obvious, but it’s important to have normal information associated with decals). Let’s get started!

The first thing we need to do is get some pixels shaded, this is easy: just draw a box where we want the decal.

Note that depth testing is turned off on this box - even though half of the box is behind the wall we can still see it. This is a workaround for a “feature” of XNA4 managing your depth buffers for you; depth buffers in XNA are bound to a texture and cannot be used again without reusing the same texture which is inconvenient here as we’ll see later. My workaround for this is basically to do the depth test myself in the pixel shader but you don’t need to do this, you could use depth testing (read ON, write OFF) just as you usually do.

This box is obviously too big, some pixels are shaded red here even though they’re not part of the decal. That’s obviously because the 3D box is just an approximation of the 2D decal we really want to draw and sticks out of the wall.

The black lines all the way around are where the wall overlaps the box. The pixels labelled too close are shaded red but should not be - these ones are closer to the camera than the box. The pixels labelled too far are again shaded red but should not be - these ones are further away from the camera than the box.

We’re going to tackle this problem by working out where every pixel on the wall is in three dimensional space and then doing an intersection check between the pixel and the box. One of the inputs to our decal system is the depth buffer, so if we know the direction from the camera to each pixel and we sample the depth buffer it’s easy to work out the location of each pixel (relative to the camera). This is (in theory) pretty simple:

//Position on the screen
float2 screenPos = positionCS.xy / positionCS.w;

//Convert into a texture coordinate
float2 texCoord = float2(
    (1 + screenPos.x) / 2 + (0.5 / Resolution.x),
    (1 - screenPos.y) / 2 + (0.5 / Resolution.y)
);

//Sample a value from the depth buffer
float4 sampledDepth = tex2D(depthSampler, texCoord);

First we work out where we are on the screen. positionCS is the position of the pixel in “Clip Space”, this is simple to get from our vertex shader (just output the position of the vertex itself). Then we need to convert this from screen space to a texture coordinate. Finally we read a value from the depth buffer which is the depth value recorded for the wall when we drew the gbuffer.

Now we’ve got the depth we need to know which direction we’re looking. This is also pretty simple to calculate from some vertex shader outputs:

float3 viewRay = positionVS.xyz * (FarClip / -positionVS.z);

positionVS is the position of the pixel in “View Space”, this is also simple to get from our vertex shader, just emit the position before multiplying in the projection matrix. FarClip is the distance to the camera far plane and is just here to things correctly (it’s a shader parameter set by the CPU).

Finally we can derive our camera relative position:

float3 viewPosition = viewRay * depth;

I did say that bit was simple! To convert from world space to view space uses the ViewMatrix, it stands to reason that to convert from view space to world space one uses the InverseViewMatrix:

return mul(float4(viewPosition, 1), InverseView).xyz;

Great, we have the world position of the wall at each pixel. Now it’s just a simple bounds check - if the position of the pixel is outside the box we can reject this pixel, if it’s inside the box we should draw something onto it. I implemented the bounds check with some more matrix trickery - the box is a unit box centered at the origin, so if I multiply the world position of the pixel by the InverseWorldMatrix for the box I get the position in object space. In object space the bounds check is trivial - if any element is > 0.5 it’s outside the box!

//Convert from world space to object space
float4 objectPosition = mul(float4(worldPosition, 1), InvWorld);

//Perform bounds check
clip(0.5 - abs(objectPosition.xyz));

Now that we’ve done all this the 3D box has been flattened into a 2D sheet which exactly conforms to the wall (and would stick just as perfectly to any geometry no matter how complex). Obviously the next step is to draw a texture onto here instead of just shading everything red.

When I moved everything back into object space to check if the pixel lay inside the box I had an ulterior motive; the object space coordinates are (almost) the texture mapping coordinates we need! Calculating the texture coordinate is as simple as:

//Offset from -0.5 -> 0.5 range into 0 -> 1 range
float2 textureCoordinate = objectPosition.xz + 0.5;

That’s it! Now we can draw a texture onto here.

Normal Mapping

That’s great, we’ve got a texture drawn on a surface with arbitrary geometry so we’re done here… right? No, not quite. We still haven’t done normal mapping. This is a bit complicated because normal maps are defined relative to the surface geometry. This is actually pretty obvious if you think about it - a normal map value pointing straight up indicates a normal pointing straight out from the surface… so by that definition normal maps depend on surface shape! These decals (and thus their normal maps) are applied to arbitrary geometry so we somehow need to transform the input normal map from being surface relative to something more useful (world space or view space normals).

What we need is some information about the shape of the surface. Luckily we have the depth buffer as an input which contains all the information we could ever want about the shape of the surface. First, let’s calculate the normal of the surface according to the depth buffer:

float3 ddxWp = ddx(worldPosition);
float3 ddyWp = ddy(worldPosition);
float3 normal = normalize(cross(ddyWp, ddxWp));

Pretty simple! ddx gets the change in worldPosition between this pixel and the next one over (horizontally) and ddx get the change in worldPosition between this pixel and the next one over (vertically). That gives us two vectors which are perpendicular to the surface, the cross product of those must be the normal of the surface at this point.

Per Pixel Tangent Space

Graphics programmers love technical terms which include the word “space”.

Normal maps are defined in “tangent space”. We need to work out a transform for every pixel from tangent space into view space.

This sounds super complicated but it really isn’t. We have the shape of the surface from the depth buffer which is in view space. If we take the three values determined above (ddxWp, ddyWp and normal) and stuff them into a matrix this defines the basis of tangent space for that pixel and conveniently transforms from tangent space into view space!

//Get values across and along the surface
float3 ddxWp = ddx(worldPosition);
float3 ddyWp = ddy(worldPosition);

//Determine the normal
float3 normal = normalize(cross(ddyWp, ddxWp));

//Normalizing things is cool
binormal = normalize(ddxWp);
tangent = normalize(ddyWp);

//Create a matrix transforming from tangent space to view space
float3x3 tangentToView;
tangentToView[0] = mul(pixelTangent, View);
tangentToView[1] = mul(pixelBinormal, View);
tangentToView[2] = mul(pixelNormal, View);

//Transform normal from tangent space into view space
normal = mul(normal, tangentToView);

Not surprisingly, perhaps, it took me a while to work this out! The value can now be written out into the GBuffer and everything after this point is a perfectly normal deferred renderer.

Are We Done Now?

No, not quite. There’s one really big problem with this system which goes all the way back to how we’re initially getting our shader to run on some pixels. We’re drawing a box… but what happens if the camera touches the box? If the camera is inside the box we have a problem because backface culling stops you seeing any of the box (and thus you can’t see the decal). The solution here is pretty obvious; just use frontface culling if the camera intersects the box, you also need to turn off any depth culling that you’re doing. You could possibly just invert your depth test, I haven’t tried that myself though.

Doing this is dead simple, you can just test if the camera intersects the box on the CPU and set up different render states:

//Create a space sphere for the camera bounds
var cameraSphere = new BoundingSphere(position: cameraPosition, radius: cameraNearClipPlane)

//Transform sphere into object space
    .Transform(inverseDecalTransform);

//Check if the camera bounds overlap the decal bounds (unit cube at the origin)
bool cameraIntersectsDecal = new BoundingBox(new Vector3(-0.5f), new Vector3(0.5f)).Contains(cameraSphere) != ContainmentType.Disjoint;

//Setup render states
if (cameraIntersectsDecal) {
    DepthTestMode(None);
    CullMode(Clockwise);
} else {
    DepthTestMode(LessThan);
    CullMode(CounterClockwise);
}

Can I Use This?

Yes! I built this decal system for my game (Heist) but everything detailed here is part of an underlying game library called Myre which is open source and available here. Myre is strongly tied to XNA and is totally undocumented, so I don’t necessarily recommend you start using it for a new project now (but feel free to if you want). However it’s a great source of reference code and it you want the specific decal shader code to tear apart it can be found here and if you want the CPU side of things it’s split between here and here. Good Luck

Random Gibberish

Sun, 22 Feb 2015 00:00:00 +0000

TL;DR

Random numbers are quite hard to generate, especially in shaders.

Generating Randomness In Shaders

One interesting challenge I had to address whilst developing heist was how to get ~~high quality~~ good looking randomness into my shaders. For one particular effect I was working on I wanted a slight fuzz applied to the entire screen and this needed some randomness.

Just Use A Texture Lookup

The obvious way to get randomness on the GPU is to generate it on the CPU. It’s really easy to generate high quality random numbers of the CPU and you can just stuff them into a texture and pass them to the GPU. You even generate yourself a random texture at compile time and just pass that in, great! Except… I wanted animated fuzz so I needed a different random number for every pixel every frame. Obviously you can extend the texture concept into a volume texture entirely full of random numbers and then move through the volume texture with time but that’s actually a pretty huge texture; 1920x1080x60 and that’s if you don’t mind repeating your randomness every second!

If this appeals to you, it would look something like this:

Sample NoiseVolume;
float Time;

float4 PixelShader(float2 uv)
{
    float randomness = tex3D(NoiseVolume, float3(uv, Time)).r;
}

Why Not Just Use A Smaller Texture?

I could just use a smaller texture and then wrap around my texture coordinates, thus using the same random values on different bits of the screen. The problem is the human eye is really good at spotting patterns, if there is any repetition in the randomess at all it will jump right out at you (because it’s the only pattern in there it sticks out like a sore thumb).

Generating Random Numbers

Ok, so how hard can it be just to generate some random numbers? We just need a function a bit like this available to our pixel shader:

float Random(float2 uv, float seed);

This should take those three floating point numbers and mix them up into a third totally unrelated one. Of course the usage of this shader is quite easy to implement when you have texture coordinates; just pass in the texture coordinates as uv and pass in some value, for example game time, as the seed). I went off to ShaderToy to experiment! ShaderToy is GLSL, but I can port it to HLSL once I’m done.

My first though was that if I just multiply the numbers together and take the fractional part, that might be pretty random:

float Random_1(vec2 uv, float seed)
{
 	 return fract(uv.x * uv.y * seed);
}

As you can see it’s messy but it’s not really random. Not surprising for such a stupid random number generator. Another problem here is that the pattern starts off simple and gets more complex over time. I decided to fix that first since otherwise it would probably break all my future attempts after enough time had passed, which is a pain in the ass to debug.

I tried a variety of things to fix the seed into a small range.

Mod with a prime and then normalize:

float fixedSeed = mod(seed, 7621.0) / 7621.0 + 1.0;

Divide by a prime and then take the fractional part:

float fixedSeed = fract(seed / 491.0) + 1.0;

Just take the fractional part:

float fixedSeed = fract(seed) + 1.0;

None of these really produced the results I was looking for. The last one being the worst because it repeats every second! I did realise, whilst experimenting with seeds that there was one serious problem - the seed being near zero often broke everything. That’s why all the examples above add 1.0 to the output, the seed is never near zero then. In the end I settled for making sure the seed is not near zero and putting up with testing the generator over a wide range of seeds.

I moved on to where all programmers go when not in their area of expertise: Stackoverflow. This answer recommended this for generating pixel randomness:

float random( vec2 p )
{
  // We need irrationals for pseudo randomness.
  // Most (all?) known transcendental numbers will (generally) work.
  const vec2 r = vec2(
    23.1406926327792690,  // e^pi (Gelfond's constant)
     2.6651441426902251); // 2^sqrt(2) (Gelfond–Schneider constant)
  return fract( cos( mod( 123456789., 1e-7 + 256. * dot(p,r) ) ) );  
}

This does look pretty good! The only problem is it doesn’t use the seed and so it’s constant over time. My first instinct was just to try to multiply the seed in right at the end:

return fract(fixedSeed * cos( mod( 123456789., 1e-7 + 256. * dot(p,r) ) ) );

Which sort of worked, but looked very weird over time. It felt like there was some temporal consistency. The particularly clever bit I liked about this generator was using the dot product with some irrational numbers - the problem my first generator had was that it had clear patterns varying over space. A dot product with irrational numbers removes the spatial component and gets us a nice, fairly random, floating point number. I tried multiplying the seed into the irrational vector:

float fixedSeed = abs(seed) + 1.0;

//Mix our seed into the irrational vector
vec2 r = vec2(23.1406926327792690, 2.6651441426902251) * fixedSeed;

//Dot product with the irrationals (add 1 to ensure uv is not near zero)
float x = dot(uv + vec2(1, 1), r);
return fract( cos( mod( 123456789.0, 1e-7 + 256.0 * x ) ) ); 

This produced pretty bad noise, with strange patterns flicking around. Before giving up on this line of enquiry I thought I’d try something else with x, something a little less complex than the StackOverflow method. Something like…

return fract(x);

Not surprisingly, that didn’t quite work. It produced a complicated but regular pattern. Stealing another part from the StockOverflow solution I threw some trigonometry in there:

return fract(sin(x));

This mixed it up a little more, but not quite enough. I thought maybe I should take some even less significant digits from the number, for example:

return fract(sin(x) * 100.0);

This actually looked a lot better, there were still patterns and regularity but they were now a lot less obvious. Continuing with this trend I threw a really massive number in there and it looked pretty good:

After a lot more fiddling, my final shader looks like this:

float Random_Final(vec2 uv, float seed)
{
    float fixedSeed = abs(seed) + 1.0;
    float x = dot(uv, vec2(12.9898,78.233) * fixedSeed);
    return fract(sin(x) * 43758.5453);
}

This has been tested with a wide range of UV values (far beyond what you should be putting in) as well as a huge range of time values (left running most of the day in the background).

Is This Useful?

Your GPU could be bottlenecked in a few places (which are relevant to this discussion):

Total Memory
Texture Lookups (Memory bandwidth)
Arithmetic

Using a lookup texture would cost you memory and bandwidth but save you arithmetic. Modern GPUs are pretty good at arithmetic but tend to be heavily loaded in terms of memory usage, which means saving yourself some memory space/bandwidth at the cost of a few ALU instructions is probably worth it.

Why Aren’t You Using [Insert Well Known PRNG here]?

Random number generation is hardly a new topic! There are endless varieties of random number generators for every possible level of randomness quality, so why am I inventing my own? Almost every random number generator that I could find uses some bitwise operations (which are not available in the HLSL version I am targeting) or worked with integer operations which are slow on all but the most modern GPU architectures. This PRNG uses entirely floating point operations and should be very fast on all levels of GPU hardware.

This Wall Is Sticky

Tue, 17 Feb 2015 00:00:00 +0000

TL;DR

I’m working on a sticky cover system, it’s quite complicated.

What Is Sticky Cover?

Sticky cover is a mechanic where you can move into “cover” and then move around inside the cover. Once you’re in cover your input is handled in a different way; for example rather than A making you move left it instead makes you move left along the cover. Here’s an example of some sticky cover in Splinter Cell: Blacklist:

You can see a few interesting things in the video:

Moving into cover requires just a single keypress
Once in cover moving around will not move you out of cover
There are special animations to move between cover
When in cover the game indicates other pieces of cover you can move to (watch out for little green arrows)

Why Would You Want Sticky Cover?

Let’s look at those three interesting points from the video again…

Moving Into Cover Requires A Single Keypress

Obviously sticky cover does not add anything to the game - you could just require that the player use the normal movement controls to get into cover. This isn’t very nice though, it means that when the player wants to get into cover they have to wrestle with the controls to perform some precise maneuvering. having a Take Cover button means that the player isn’t having immersion broken by wrestling with controls, instead they’re moving around from cover to cover like a badass supersoldier.

Moving Around Will Not Move You Out Of Cover

This is kind of similar to the last point - if we didn’t have sticky cover we could just require that the player move around carefully and not expose themselves. Once again this means the player is having to wrestle with controls to do what they want. With sticky cover you can move around confident you’re not going to hold the D key one second too long and walk right out into the line of fire.

There Are Special Animations To Move Between Cover

Because the movement is context sensitive the game knows what you’re trying to do and can play a more appropriate animations. To achieve this with non context sensitive controls you’d either have to try to detect what the player is trying to do (which will sometimes be wrong and thus very frustrating for the player) or to have an insanely complicated control setup which allows the player to perform all the different actions involved.

When In Cover The Game Indicates Other Cover

It’s a fact that games aren’t perfect simulations of the world, sometimes you’ll look at something and say “yeah that’s definitely good cover” but then it turns out it’s actually 1 centimeter too short and it doesn’t count as cover at all. Or maybe you’re in some cover and you think “yeah that cover is within diving distance” and so you do your dive to move to that cover but it’s 20cm too far away and now you’re dead. With the game showing you where appropriate cover is neither of these are a problem - when you move into some cover you can be absolutely confident that your character will do what you expect and will end up in some valid cover.

Let’s Implement Sticky Cover

We’re convinced that in a tactical game with cover we definitely want sticky cover - let’s implement it! This is where it gets complicated.

Aside: Raycasting

Most of the heavy lifting of sticky cover is done with raycasting (in my implementation, at least) so we’d better understand what raycasting is! Raycasting allows you to pass a ray into the physics engine and get back everything that touches that ray, it’s very cheap and so you can happily perform loads of raycasts each frame.

Step One - Let’s Find Some Cover

This is pretty simple: when the player presses the take cover key you need to find some cover. The first thing to do here is just to cast a ray out from the camera an appropriate distance and check if you can find a surface which could be cover.

Here we’ve got the thick black line indicating the ray coming from the camera and this hits the wall. The red line indicates the normal of the surface at the point where the ray hit. Here’s our first exit condition: if the normal at that point is not roughly horizontal that means the surface is unsuitable to be cover.

We can test if the normal is horizontal with a dot product. The dot product gets us the cosine of the angle between two vectors, so we can do Normal . Up and check that the value lies in some acceptable range.

Step 2 - Is This Point Accessible?

So we have a point next to some roughly vertical surface, that’s not enough to check if this is a cover point or not! We need to check if it’s actually physically possible to get to this point.

Is There A Floor Below This Point?

If there’s no floor, this clearly isn’t a good place to try and stand. We can use the raycast and dot product check again. This time we need a ray down from the point on the wall and then we need to check if the normal at the point found points roughly upwards.

How High Is The Ceiling?

We could have found a perfectly acceptable cover point but the ceiling could be so low we cannot get there even when crouching! This time we don’t care about the direction of the surface normal, instead we just want to perform a raycast upwards and check how far away the first hit is.

There is of course the possibility that the raycast won’t hit anything at all because there is no roof. In this case I just return a sufficiently large value because it doesn’t really matter so long as it’s greater than the player character’s standing height.

Step 3 - Is This Actually Cover?

Let’s say we’ve found a vertical surface with a floor below it and no roof above it, is this definitely cover? No. I could have just described a single brick lying in the middle of a field - it’s accessible but it’s too short; it’s only cover for your toes.

We need to find out how high this vertical surface is. We can achieve this with a binary search along the surface. The lower bound of the search is at the floor point we’ve detected, and the upper bound is at the roof height we’ve detected. All we have to do is project a ray in the opposite direction to the surface normal, if we detect nothing we’ve clearly gone over the top of the cover (lower the upper bound of the search) and if we detect something we’ve hit the cover (raise the lower bound of the search).

Yay Cover

Now that we know how to find a cover point, what do we do with it? First thing we’re likely to want to do is to move to it from a random nearby point. At the moment I achieve this by turning the character towards the cover and making it run, in the future I’m likely to replace this with a specific move into cover animation (a dive or a roll).

Once we’re in cover then we want to handle the player pressing the left and right buttons. This should make the character move left and right perpendicular to the cover normal.

The first thing to do when the player presses a key is to search in that direction for cover points. I achieve this by simply probing to the side of the player, roughly 5cm away, if a cover point is found then the player can move. Since the test is performed every frame whilst the direction key is held down the game has 5cm of warning to stop the character from hitting the end of the cover. Once the game has found a cover point you simply need to start the strafing animation and to turn the player to face the normal of the newly found cover point (which may be different if you’re moving around curved cover).

Changelog 25

Sun, 08 Feb 2015 00:00:00 +0000

TL;DR

Multiplayer testing, new camera system, new character controller and beginning work on a dynamic cover system.

This Week

Watch The Video!

In the last couple of weeks I’ve run a few multiplayer tests and have built up a big list of usability problems people have complained about.

New Camera controller

This new camera controller now use physical casting to check it’s position into the world does not clip through things. This means that the camera will never position itself on the other side of a wall from your character! There’s still more to do here but already this has been a huge win for usability.

In fact, there’s so much to do in third person cameras I think I could probably spend a couple of months on this topic alone… don’t worry, I don’t plan to! Instead I’m going to do the next best thing and write up a big blog post on designing and building 3rd person cameras. Coming Soon (tm).

New Movement System

The old movement system was really just thrown together to be whatever was easy to build. I ended up with a standard first person movement mode - so your character always faced away from the camera and moved in whatever direction you pressed a key.

The new system is a lot more like a classic third person character controller - you face whichever direction key is being pressed and move forwards. So if you press ‘A’ rather than sidestepping, like the old system, you instead turn to face the left (facing across the camera) and walk in that direction. This kind of system is a lot nicer to use because it means you (the player) can be looking in a different direction to your (the character) movement.

Dynamic Sticky Cover v0.1

I’ve begun work on a dynamic sticky cover system. This system uses a set of physics queries to probe the environment when you’re moving into and around cover, depending on the results controls will be changed in context sensitive ways. For example if you press C (whilst looking at some cover) the character will dive into cover and now your control directions are relative to the cover rather than relative to your camera.

What’s Next?

I need to finish off the dynamic cover system. I’ve also been working on a way to render text in 3D so that I can place context sensitive control hints in world right in front of your eyes and that should be done in a couple of weeks.

2014 Retrospective

Tue, 13 Jan 2015 00:00:00 +0000

TL;DR

2014 was a pretty cool year.

What Did I Do In 2014?

In my last retrospective I had just completed removing the Lua API and had high hopes for being on greenlight within the year, obviously this was hopelessly optimistic (let’s face it, you’re not qualified to be a programmer if you can estimate timetables). Despite this failing to happen 2014 wasn’t a disappointing year - quite the opposite! Epimetheus has reached the stage where I feel like it’s a solid engine and I can spend most of my time doing game development instead of engine development, in theory I should be able to very quickly make new and simple gamemodes now.

Japan

The big event of 2014 was, of course, my three and a half week trip to Japan! I took something like 2000 pictures and put together an album here.

Changelogs

The biggest change to how I do development in 2014 was my changelog videos - a quick video update released every week on Sunday telling people about “what’s new and what’s changed in the game”. I’ve skipped over a few of these on weeks when my work wasn’t particularly easy to demo, and I’ve managed to do 24 in total (roughly one every two weeks). What I haven’t done with the changelogs yet is advertise them to try and get a decent number of viewers, this is something I should really start doing once I restart the videos (this week or next week).

Open Source

I continue to be a massive fan of open source and release a lot of my code publicly (MIT License FTW). This year I have made some pretty cool contributions to open source, all of which are available on github.

New Open Source Projects

WebDesktop

A project to render a webpage over your desktop - desktop widgets using a web stack!
Not currently actively developed, but it pretty much works and I’d like to do more work on it sometime.

SupersonicSound

A very lightweight C# wrapper for FMOD
Largely complete, actively developed.

Bastet

A CoAP/HTTP Proxy server for smart home devices
Not currently actively developed, we’re working on other smart home stuff at the moment but Bastet is very core to the whole thing.

LCARS

A CSS framework for Start Trek style LCARS interfaces
Not actively developed, works for a few basic demo interfaces.
This project was just for fun and was a good way to learn various web things (Typescript, LESS, node, HTML5 custom elements).

Sorcery

A library for evaluating Combinator Algebras in C#.
This one was just for fun, I might pick it back up or I might not.

KeybaseFSharp

An F# wrapper around the Keybase API.
Also a WPF desktop client for Keybase.
This project was a way to learn about F#, and then became a way to learn about WPF. Not being developed right now, but I continue to think Keybase is totally awesome and I’ll probably pick this back up at some point.

Cassowary.net

An incremental constraint solver.
This is a fork of Cassowary.net which was a port from Java, which was itself a port from Smalltalk and to be honest… you could tell - the code was not C# style at all!
Improved code quality, implemented unit testing, implemented a new way to express constraints (using C# expression trees), created a nuget package.
This project is largely complete and so does not have any recent commits but it is in use by my city generation so it’s alive and well maintained.

SharpSteer2

This is a fork of SharpSteer2, which was a fork of SharpSteer, which was a port of OpenSteer from C++ to C# for a much older version of C#. As with Cassowary.net the code was functional but wasn’t properly up to the standards of modern C#.
Unit testing, naming conventions, removed dead and useless code, improved usability with extension methods on interfaces which make implementing the interface much easier, bugfixes and new features!
As with Cassowary.net my work on this project is more or less done but the project is in active use by my AI and is well maintained.

PlaceholderHomepage

I made a homepage for my game. It’s a bit out of date now, I guess I should fix that!

Epimetheus-Public

This is a repository for all the public facing parts of Epimetheus (my game engine) - bug reports and documentation for plugin development.

NockFSharp

A basic implementation of Nock in F#.
This was just a learning exercise for F#, as well as an attempt to get my head around Nock (which is part of Urbit, which is bizarre and awesome).

Game Development

My job is game development, so what did I do with that this year? My primary focus is the Epimetheus Engine, my custom game engine. This is supported by Placeholder.*, a (closed source) set of libraries for various game related things (advanced entity management, networking, game audio, AI etc). Finally the whole thing is based on my open source game library Myre. I quickly skimmed the commit notes for the entire of last for for these three core projects and this is the mile high view of what I did (I’m not really mentioning bugfixes in this list, so just mentally drop a couple of hundred bugfixes for each project into this list).

Epimetheus

240 commits
5,840 files changed
12,385 lines written
41,130 lines changed

Implemented animation system for plugins (enables playing animation clips on models)
Typednames change
gbuffer shader in plugin
Steamworks.net
Plugin networking (arbitrary types in messages)
Plugin entry point
Plugin persistent settings
Improved respawning
Improved tool wielding
physics detector volumes
RequireJS UI (massive UI moddability improvement)
Plugin loading events ( etc)
Lots of wrestling with appdomain plugin system
Myre Events across boundaries
Network Pipe messages across boundaries
GC collecting too early
GC never collecting (memory leak!)
Many UI rendering improvements
Improved parallelism of world generator
Improved how the generator chooses which nodes to procedurally generate next
Massively improved world parallelism service with buffering of geometry operations (50x speedup)
Sped up procedural mesh generation (x2 speedup)
Implemented a system for 2D procedural geometry (using CSG)
Improved lighting
Improved physics body scripting
Added new physics shapes
In world user interfaces (using awesomium)
Improved input system (stackable contexts)
Implemented way to “use” items in the world (pressing E)
Mouse input to in world UIs
Exposed broadphase queries to plugins (hugely improves detector volumes)
Implemented collision groups
Physics constraints/joints/motors/limits
Split out UI rendering in rendering pipeline (future UI post processing)
Implemented a way to query the renderer about lighting conditions in a certain position
Scriptable rendering pipeline
Per camera pipeline (means per camera FX, enables e.g. night vision goggles)
Implemented crouching (seriously)
Implemented queries on the state of the character (e.g. HasSupport/HasTraction queries)
Totally rewritten chunking service
Implemented a streaming service to go alongisde the new chunking service
Implemented positional errors in world generator
Implemented navmesh generation
Goal oriented action planning exposed in plugins
Implemented a basic NPC entirely in plugins
Implemented infrastructure for pathfinding
Sped up ReST API
Implemented immediate mode debugging service
String pulling to generate smooth paths
Improved 2D geometry generation
Implemented train level using improved floorplan/2D geometry generation
Removed subtracting from the world is procedural scripts (:O)
Implemented weapon quick switching
Implemented player inventories
Implemented ragdolls (not entirely successfully)
Future proofed all event types
Implemented hitpoints
Implemented 3D audio playback for entities
Implemented a way to play sourceless sounds at arbitrary points in 3D space
Implemented a new system for defining entities (mixins)
Rewritten GOAP AiActions to be easier to use
Added more capabilities to debug drawing service
Implemented a system for NPCs to use tools (e.g. guns)
Removed sandboxing (OMG)
Vastly improved plugin API
Removed a load of hacky types to work around how sandboxing worked
Vast performance improvements
No more memory leaks
No more game crashing when GC collected a vital object on the wrong side of the sandbox
Implemented an aimbot NPC
Improved pathfinding so that queries are cancellable
Implemented a score tracking service
Enabled steam overlay in game

Myre

103 commits
780 files changed
Line count statistics unavailable (they’re thrown off by moving lots of files around)

Implemented animation in the renderer
Implemented animation clip loading in the content pipeline
Implemented parallel evaluation of animation channels
Rewritten data collections to be more strongly typed (TypedName)
Changed EVERYTHING to use TypedName
Rebuilt nuget packages for Myre
Animation bugfixes
Made property initialisation more implicit
Made property initialisation less implicit
Improved renderer pipeline infrastructure
Made point and sun lights (de)activatable
Fixed depth test in particle system
Fixed material content loading
Improved spotlights (angular falloff)
Improved shadow mapping in directional light
Implemented Vector3 swizzling methods
Dropped support for xbox360
Improved animation system to allocate less memory
Implemented infrastructure for procedural animation clip generation (needed for e.g. ragdolls)
Implemented parallel evaluation of animation instances
Improved linear keyframe reduction in content pipeline
Dropped support for windows phone 7
Implemented collection initializers for named box collection
Implemented Writer for console

Placeholder

125 commits
3,103 files changes
9,029 lines added
46,218 lines changed

Implemented system for improved thread safety for steam
Switched to Steamworks.net from my own steamworks wrapper (HotAndSteamy)
Implemented logical assertions for GOAP (AND OR NOT XOR)
Implemented network translators for a load of types
Improved CSG system (error handling)
Made BSP transforms more robust
Implemented 8 way BSP splitting
Implemented navmesh generation (Recast based)
Implemented funnel narrowing path generation
Modified navmesh generation to check ceiling heights
Expression based GOAP condition/goal system
Improved agent plan selection
Improved behaviour tree
Implemented navmesh raycasting
Implemented new nodes for behaviour tree (Condition, Setup)
Implemented rich presence providers
Made CSG operations entirely deterministic
Implemented ADSR amplitude envelopes for audio renderer
Implemented variable length floating point encoding
Implemented a jukebox system to play music (based on XNA MediaPlayer)
Removed Jukebox (XNA MediaPlayer is terrible)
Implemented system for playing sounds at a position (with no associated emitter entity)
Improved expression based GOAP system
Implemented cancelling pathfinding tasks
Begun works on Audio2, a new audio renderer based on FMOD

Plugins

I haven’t got detailed changelogs for these (there are too many) but I have done a lot of development of plugins too. A lot of functionality is tied up in these plugins and they’re a significant part of development. Remember, every plugin I develop is open source, and you can find them all on Bitbucket.

Deathmatch Gamemode

Testing moving, shooting, NPCs etc.

Base-ArtificialIntelligence

Library for building NPCs for Epimetheus.

Electronic-Infrastructure

Library for building circuitry in game (e.g. wiring up logic for controlling automatic doors).

Construct Gamemode

A sandbox test mode.

City1

A set of scripts for the procedural city generator.

Character-Controllers

Library for building player controlled characters for Epimetheus.

Base-Equipment

Library for building wieldable tools for Epimetheus (e.g. guns).

Base-GameUI

A HTML template along with a load of Javascript plugins for constructing UIs (mainly HUDs).

Base-Sneaky

Library for implementing sneaky things (stealth, detection, disguises etc).

Default-Shaders

A set of shaders for the engine (including fundamental things like the GBuffer shader - you can completely change how the renderer works by replacing these).

Base-TrainGeneration

A set of scripts for generating trains. Trains are compact and have lots of little rooms and narrow corridors so this was a great test for the procedural city generator.

Base-CityGeneration

A library for implementing procedural generation scripts.

Base-Achievement

Library for tracking hierarchies of goals.

Things I enjoyed

It turns out that I have spare time to do things that aren’t game development (I know, was surprised too). These are some of the things I enjoyed this year:

That Was Longer Than I Expected

Turns out 2014 was a busy year! let’s hope just as much exciting stuff happens in 2015 (Heist released on greenlight? Could it be!?).

Super Sonic Sound

Thu, 08 Jan 2015 00:00:00 +0000

TL;DR

I have released a C# wrapper for FMOD studio, get it here.

FMOD

FMOD is an audio middleware for games which comes in two parts: FMOD Programmers API and FMOD Studio.

FMOD studio is an audio creation tool for your audio artists to use to create complex sound effects - using the studio you can create sounds which depends upon parameters and run complicated state machines and then the game can tweak those parameters at runtime and make your sounds responsive to in game events.

The FMOD programmers API is what you use in your game to playback these complicated sound effects. It handles stuff like loading lots of different file formats, generating sounds and effects in realtime, controlling occlusion and reverberation from geometry and positional sound.

Way back in March last year I posted about how FMOD had changed their licensing to be very indie friendly. To quote their sales page:

FMOD Studio Free For Indies! License – Budget under $100k USD If you are developing an indie title and your total costs are less than $100k USD at time of shipping, you can now use FMOD for free!

Since then I’ve had a plan to integrate FMOD into the Epimetheus engine and use it in Heist. The sound system I currently have implemented does positional sounds but nothing else and desperately needs replacing.

FMOD API

Unfortunately there’s a slight problem with using FMOD - it’s a C++ API. That’s ok though because it comes with a supplies C# wrapper using PInvoke to directly call the native FMOD code. Of course this wrapper is very thin, it just exposes FMOD into C# in the native style of FMOD and makes no effort to make it look like C# code at all. Here’s an example of some basic FMOD code in C++ and then the same code using the supplied C# wrapper:

// C++
FMOD.Studio.ParameterInstance instance;
ERRCHECK(_eventInstance.getParameter(name, &instance));
return instance;

// C#
FMOD.Studio.ParameterInstance instance;
Util.ERRCHECK(_eventInstance.getParameter(name, out instance));
return instance;

Not the prettiest code I have ever seen. Note that the actual result of this function is passed as an out parameter, and then if there’s an error it returns an error code (which you can easily forget to check). Using this would either drive me mad, make me produce very brittle code, or both. What I really want is something like this:

var instance = _eventInstance.Parameters[name];	//throws exceptions instead of error codes

This is the kind of thing my new open source project, Supersonic Sound, is for.

Supersonic Sound

Supersonic sound (SSS) is built entirely as a wrapper around the C# wrapper supplied with FMOD - I didn’t write a single PInvoke in this project and so you can be confident the foundation of SSS is as solid as if you used the FMOD wrapper (because, in fact, you are). This design means that if the FMOD wrapper is updated it’s very easy to update SSS just by dropping the C# wrapper files into the right place in the project and rebuilding.

Implementation Notes

Exceptions

FMOD has 49 different return codes from functions, one is “OK” every other one is an error code. I decided that the best approach to handling this would be to have a base FmodException class and then create 48 different exceptions derived from it so that every different error code will throw a unique exception. Obviously if you want to catch an exception you don’t need 48 different catch clauses - you just need one to catch FmodException.

try
{
    // Stuff
}
catch (FmodException e)
{
    //Something broke
}

Some of the error codes are in obvious groups, for example:

ERR_INVALID_FLOAT
ERR_INVALID_HANDLE
ERR_INVALID_PARAM
ERR_INVALID_POSITION
ERR_INVALID_SPEAKER
ERR_INVALID_SYNCPOINT
ERR_INVALID_THREAD
ERR_INVALID_VECTOR

All of these are to do with invalid parameters being passed into FMOD. In this case the exceptions all have a common base exception you can catch:

FmodException
- BaseFmodInvalidException
  - FmodInvalidHandleException
  - FmodInvalidParamException
  - FmodInvalidPositionException
  - FmodInvalidSpeakerException
  - FmodInvalidSyncPointException
  - FmodInvalidThreadException
  - FmodInvalidVectorException

Structures

One of the strangest appearing choices I made in Supersonic Sound is to return struct instead of objects for FMOD objects, this is not a very C# style thing to do. I did this because the FMOD wrapper already creates wrapper objects for FMOD objects, if Supersonic Sound created wrapper objects around these objects then the API would allocate two objects for every object and could end up being quite allocation heavy. Instead I have implemented all wrappers as structs which internally have the FMOD wrapper object. All these structs are immutable because all state is held within FMOD, not within the struct itself and there should be no problem passing them around and using them.

One problem with passing structs around instead of objects is that they’re pass-by-value which means passing a large structure around can be very expensive. This is not a large problem for Supersonic Sound because all the wrapper structs just contain a single field (the FMOD wrapper object) and so they’re no more expensive to pass around than an object-by-reference.

Naming

I have tried to maintain the naming of methods in Supersonic Sound as much as possible. Obviously C# naming conventions are different from those of the FMOD library and so I have changed capitalisation of methods and structures etc. e.g.

RESULT getSoundInfo(string key, out SOUND_INFO);

Becomes:

SoundInfo GetSoundInfo(string key);

Here you can see:

The return value is now the sound info, instead of a result code
Method name is capitalised
Structure is SOUND_INFO is called SoundInfo

There are some places in the FMOD API where multiple methods do the same thing with different arguments, e.g.

getBank(string key);
getBankById(GUID id);

These have become:

GetBank(string key);
GetBank(Guid id);

i.e. they are simply overloaded with the same name, instead of having “byId” prefixed on the name.

Finally properties which are getters and setters simply become properties, e.g.

getVolume(out float);
setVolume(float volume);

Instead have become:

float Volume { get; set; }

Naming - Indexers

The most radical name changes between FMOD and Supersonic Sound come in index properties. For example in Sound there are several methods for manipulating channels:

RESULT getMusicNumChannels(out int numchannels);
RESULT setMusicChannelVolume(int channel, float volume);
RESULT getMusicChannelVolume(int channel, out float volume);

The first one could simply become a readonly property MusicNumChannels { get; } and the other two would have to stay as methods because they need the channel index parameter. Instead in cases like these I have decided to use indexed properties, so instead this becomes something like:

thing.MusicChannels.Count;
thing.MusicChannels[123].Volume = 0.5f;

Which is a much more fluent interface than before.

Future Development

There’s little point releasing an open source project without some commitment to future development! As it stands almost everything in Supersonic sound is implemented except a few things which are both not very important and are hard to port. However if you use the library and find that a feature is missing please create an issue and I will try my best to implement the feature - none of the missing features are impossible to implement and so I should be able to get something done for your very quickly.

Similarly if you find a bug, or even an annoying inconsistency, please create an issue and tell me about it.

Nuget package

I plan to get a nuget package up soon^tm but I have no experience with releasing nuget packages with native dependencies and I need to investigate how that’s done.

Sandboxing Is Dead Long Live Sandboxing

Mon, 22 Dec 2014 00:00:00 +0000

TL;DR

Removed sandboxing, it was too slow.

Wait No Video?

No video this week, I spent most of my time chasing down a performance problem and solving it.

This Week

For a long time I’ve had a problem with performance in the Epimetheus Engine - in some rare circumstances performance would absolutely tank (down to less than 1 frame per second sometimes). This was caused by the way sandboxing was done.

What Is Sandboxing?

Epimetheus is an engine built for plugins - there is no game called “Heist”, just a plugin to the Epimetheus Engine. Without plugins there isn’t even a main menu! A problem with plugins is that they could be a security problem - after all anyone can write plugins. To fix this the plugins have always been loaded into a separate “AppDomain”, this is a .Net concept which is a bit like a separate process. The important thing with AppDomains is you can set permissions onto them, for example ban the entire scripting Domain from reading or writing any files on disk!

Sandboxing Is Awesome!

No, it’s not. I mean AppDomains are a nice idea but their implementation turns out to have some big drawbacks.

First up is garbage collection. In C# memory management is done through a garbage collector (GC) - to free up some memory you just get rid of all references to it and the GC will clean up that memory for you - this is a fundamental part of the language. Except in AppDomains. When you have an object in another AppDomain it’s no longer enough to hold a reference to it, instead you have to have some complex code to do with sponsoring the lifetime of the object. Basically this means that I occasionally got an error caused by having a reference to an object in the other AppDomain that has been garbaged collected. This is actually a programmer error, but when you mess around with something as fundamental as changing the entire memory management system that’s what you get!

Second is communicating across the AppDomain boundary. In C# (and basically every other language) there are two types of type: Value Types are passed by value, and Reference Types are passed by reference. This means that if I pass a value type to you then you simply get a copy of the value but if I pass a reference type to you then you actually get a pointer to the same object. Again, this is pretty fundamental to the language. When you pass objects across the AppDomain boundary they are almost always serialized (i.e. Value Type behaviour) even if they’re a reference type. This is confusing as hell to program with! Again, we’re messing around with something fundamental to the language and practically encouraging programmer errors!

Third is performance. It is possible to pass objects across the AppDomain boundary by reference rather then by value. What this actually does is pass a proxy value across the boundary (by value) and then when a method is called on the proxy it calls back to the real object in the other AppDomain. So here’s a simple bit of code:

//This class is instantiated in the engine, and passed by reference into the AppDomain
class MyClass
{
    int value;

    int GetInteger()
    {
        return value++;
    }
}

//This code runs in a plugin
MyClass instance = /* value by reference from engine */
int sum = 0;
for (int i = 0; i < 10000; i++) {
    sum += instance.GetInteger();
}

Pretty simple stuff - this is just looping 10,000 times and summing up the integers it gets. i.e. 10,000 integer additions and 10,000 method calls - this is really basic stuff and computers can do both of these things really fast. In fact this takes just 0.27ms to execute on my PC! Let’s include AppDomains; if I run exactly the same test except this time I run the loop inside another AppDomain this test takes 11.3ms - AppDomains are 37.7 times slower at method calls! This isn’t just breaking something fundamental to the C# programming model, this is breaking something fundamental to programming computers in general, this is begging for programmer errors.

Believe it or not these were just the primary problems with sandboxing, there were more but I don’t want to bore you with a huge list of problems!

Sandboxing Sucks.

There’s only one solution to this: remove AppDomain based sandboxing. Actually removing sandboxing didn’t take me very long, but once sandboxing was gone there were a whole lot of warts in the plugin API that I could fix.

Asynchronous tasks can now return a Task object (previously I could not send Task into the other AppDomain)
Asynchronous tasks can be cancelled (previously cancellation tokens could not be sent into the other AppDomain)
Referencing scripts (internal engine stuff) is more robust
Callbacks now just implemented as normal event handlers (previously I could not send event handlers to the other AppDomain, so I had to invent my own bastardized version)
No more horrible performance pitfalls for plugins!

What’s Next?

I did get a little work done on the aimbot (which I mentioned last week as my goal for this week), in fact it’s more or less done now. I’m going to go full steam ahead for a multiplayer test sometime in the next two weeks - I’d like to get it done this week but I hear there’s some little event going on mid week ;)

Changelog 22 B

Sun, 14 Dec 2014 00:00:00 +0000

TL;DR

Slow week thanks for other commitments. But I still got NPC vision frustums done.

This Week

Watch The Video!

NPC vision frustum
Vision raycasting to determinate entity visibility
NPC tool interface to allow NPCs to use tools (such as guns)

What’s Next?

I’m working towards an NPC which will simply wander around and shoot people on sight, I had hoped to get that done this week but as I said I had a lot of other commitments this week - hopefully next week I can get back on track! After that I’m going to be running a multiplayer deathmatch for some of my friends - this will be a good test for multiplayer games with NPCs in, but will also just be great fun!

Changelog 22

Sun, 07 Dec 2014 00:00:00 +0000

TL;DR

I’ve made a lot of progress towards an NPC which patrols around and shoots players, but there’s nothing worth showing in a video.

This Week

My goal this week has been to get an NPC which patrols around the map and then shoots people that he encounters who are not on the same team. If you think about it this is actually a pretty complex goal with lots of sub goals:

AI steerable characters
Patrol routes
Path finding
Path following
AIs using tools (i.e. guns)
Teams
Detection
Planning

I’ve had a few of these in place for quite a long time, specifically steerable characters, path finding and path following have all been solidly implemented for quite a long time, this narrows my work down to teams, AIs using tools and planning. Planning has also been implemented for a long time (in fact I demonstrated it a while back in Changelog #16) but I’ve not used it for a long time and so my first task was to remind myself how the system worked.

I spent a couple of days neatening up a few sharp corners of the planning system - mostly integrating it better into the plugin API to make it much easier to implement goals and actions. I wasn’t really happy with what I ended up with though, it felt like defining an AI had a lot of annoying “ceremonial” code associated with it. For example, here’s a random like example AI:

//setup graphics.... nothing to do with AI
AddBehaviour<IGraphicsCylinder>();

//setup physics... nothing to do with AI
AddBehaviour<IPhysicsCylinder>();
AddBehaviour<IPhysicsBody>();
AddBehaviour<IHumanoid>();

//Setup sound... nothing to do with AI
AddBehaviour<IAudioEmitter3D>();

//Oh hey looks some AI stuff
AddBehaviour<IPlanningAgent>();
AddBehaviour<AiAction1>();
AddBehaviour<AiAction2>();
AddBehaviour<AiAction3>();
AddBehaviour<AiAction4>();

All this stuff is really important (there’s usually a load of configuration in here, rather than just some behaviours and nothing else but I left that out). The problem is composability: this example AI doesn’t have an animated body it’s just a cylinder which moves around, if I want an animated AI I can use exactly the same AI behaviours (yay, reuse) but I have to repeat everything here except repace IGraphicsCylinder with something else (boo, repetition).

I came with with a system for “Mix-ins” of behaviours. So for example I define a mix-in with the behaviours for AI, physics, audio and graphics and now my definition looks like this:

MixIn<AiStuff>();
MixIn<PhysicsStuff>();
MixIn<GraphicsStuff>();
MixIn<AudioStuff>();

This is great, because I can define a load of default mixins and save everyone from a load of make-work setting up the the same old entities everywhere… but if you do need to setup exactly what’s what you just stop using the mixin and define it yourself! Of course if I want to swap GraphicsStuff from a solid cylinder to a fully animated character well that’s easy, I just mixin a different set of graphical behaviours!

With this done, I spent some time overhauling the entity creation API a little more - again just fixing a few annoying bits of the API and removing unused parts of the system. It required me to go over all of my mods and modify my entity definitions but now they’re all shorter and easier to write which is a massive win for such an important part of the plugin system!

Finally, I spent some time building sensors for the AI to detect things (like other players). I started work on the suspicion and audio detection framework before, but this time I was working on visual detection. The NPC probes the physics engine every frame to find who is in sight and then does some raycasting to determine if they are hidden (behind a wall or in cover or something).

What’s Next?

I’ll continue working on this patrol and shoot AI. This touches most of the parts of the AI system so once this is done I should be able to build much more complex NPCs (for example a cooperating squad of NPCs who work together to encircle and flank players - that would be super cool).

Possible Interruption To Service

Sat, 06 Dec 2014 00:00:00 +0000

TL;DR

Don’t worry if the blog disappears for a while.

URL Fiddling

I’m modifying the configuration for this blog, in theory you shouldn’t notice anything change at all! However, there’s a chance the site may go down for a while as DNS changes propogate etc.

The Game Programmer Awakens (changelog 21b)

Sun, 30 Nov 2014 00:00:00 +0000

TL;DR

Sound!

Last Week

Before I get started I want to apologise for last week - I had a really busy weekend and doing my update totally slipped my mind. I suddenly remember at 23:30 on Sunday evening what is was that had been bugging me all day! By then it was too late and I decided just to leave it to this week instead.

This Week

Watch The Video!

Start of an NPC suspicion framework
Events for NPCs to react to sounds
A sound renderer for humans to hear sounds
Tidied up a large and ugly part of the modding API to insulate plugins from breaking changes within the engine

What’s Next?

The NPC suspicion framework is only a very early start on this stuff - right now I don;t have a single NPC character which actually uses these events. I’m going to do a lot of work next week on neatening up the old AI code and integrating this new event system into it, my aim will be to have a couple of different NPCs who patrol around the test level with different attitudes:

Shoot on sight (simple test of NPC sight)
Pursue and shoot when suspicious (simple test of NPC suspicion)

The Return Of The Programmer (changelog 21)

Sun, 16 Nov 2014 00:00:00 +0000

TL;DR

I’m back from my 3 week holiday in Japan.

This week

There’s no video this week. I did want to get one out but I’d forgotten a particular oddity about my editing software which messed everything up - of course I only realised this when I was just about finished with the 2 hour recording and editing process :(

Most of my time this week was just easing myself back into work and reminding myself how everything in the engine works. I went over the engine with a fresh set of eyes and tied up a few loose ends. I’m quite pleased with how well everything fits together, none of the stereotypical “Oh dear God who wrote this awful code?” moments when looking at my own code!

I did complete one basic task: writing an objective system (you can find the repository on bitbucket as usual). The objective system keeps track of a hierarchical set of goals and calculates things like progress towards completion, or completion of parent goals when all/some/any of the child goals are completed. The system is also written to be serialised easily into a blob of JSON so that the UI can show the objective state.

Here’s what it looks like at the moment, using a set of test goals designed around DoTA2:

What’s Next?

I’ve got the objective system mostly working, although not integrated into any game modes yet. I’ll probably leave that for a while and move onto something gameplay related, probably sneaking mechanics.

The Return Of The Vending Machines

Sun, 26 Oct 2014 00:00:00 +0000

TL;DR

I’m in Japan, Kyoto, the ancient capital of Japan.

Kyoto

Kyoto was the home of the emperor and the capital of Japan from 794 through to 1868. It’s a city absolutely packed with history - there are no less than 17 UNESCO World Heritage sites in the city!

Of course we also visited the two most important cultural sites in Kyoto:

Whilst in Kyoto we stayed in a Japanese style Ryokan - with tatami mats on the floor and futons to sleep on.

We were in Kyoto for an entire week and took loads more pictures of many more places but I only wanted an overview here instead of an entire gallery! I might whip up a full blown gallery once I’m back in the UK and have a bit more time.

Next…

I’m actually writing this in Hiroshima, which we arrived in two days ago. We’ll be departing for Nagoya tomorrow (just for one night) and so I’ll probably do a combined Hiroshima/Nagoya post next.

There Are Vending Machines Everywhere

Thu, 16 Oct 2014 00:00:00 +0000

TL;DR

I’m in Japan, Lake Kawaguchiko, at the foot of Fuji.

The Flight

Getting to where I am took a lot of travelling, and naturally not everything went to plan! We had booked flights with Swiss air - flying to Zurich from Birmingham and then changing onto a long haul flight to Narita. Unfortunately the Zurich flight was delayed (enough that we would miss our connection). I was amazed how well the airline staff handled this, first thing we knew about the delay was when they called us to a desk and gave us tickets for another airline entirely! In the end we flew with Lufthansa to Frankfurt and then with Lufthansa again to Narita - we ended up arriving just 5 minutes later than planned.

Arriving at Narita took about 19 hours:

Drive to Birmingham (1 Hour)
Wait At Airport (2 Hours)
Fly To Frankfurt (1.5 Hours)
Wait At Airport (1 Hour)
Fly To Narita (14 Hours)

And at this point we’re not nearly done yet! Our first hotel is in Kawaguchiko and we’re in Narita. After wandering around the airport for a while we found the station and took the Skyline into Tokyo, at Ueno station. The plan, once in Tokyo, was to take the trains around the city from Ueno to Shinjuku and then take a “limited express” train to Kawaguchiko - this was a little more difficult than expected. The Tokyo stations all have signs in English and Japanese telling you platform numbers and which lines are on which platform, what they don’t have is any maps labelled in English so it’s impossible to find out which line you want! After some Googling (luckily we ordered a WiFi dongle to the airport post office so we’ve had an internet connection the whole time) we worked out which trains we needed and a mere 3 hours train journey and 40 minute walk through the dark and the rain we arrived!

Total Time Travelling/Awake: 25 hours.

So the really long journey was pretty difficult and it was a huge relief to go to bed when we finally got to Kawaguchiko (at 1900). I think we may have dodged a bullet though - we suffered no jet lag! By the time we’d spent so long awake and running around we went to bed as soon as we arrived, slept for 16 hours we’re on Japan time!

Kawaguchiko

Around the base of Fuji there are 5 lakes; Kawaguchiko, Saiko, Yamanakako, Shojiko and Motosuko. We stayed at Kawaguchikiko at the Mizuno Hotel. The hotel description promised great views of Fuji and most definitely did not disappoint! On the very first day we were up at 6am and were lucky enough to see Fuji poking through the fog. We thought that would probably be all that we saw of Fuji, but we turned out to be incredibly lucky with the weather and later in the day we were on top of a nearby hill just as the clouds were clearing and got some beautiful shots. Throughout the day the clouds slowly cleared and we got more incredible shots of Fuji:

Kawaguchiko To Kyoto

I’m actually writing this blog post from my hotel room in Kyoto, yesterday we left Kawaguchiko on the train (exactly on time, of course) and then took the bullet train from Yokohama to Kyoto. I didn’t manage to get a window seat, so I didn’t get any good pictures from the bullet train - but I did get a load from the trains on the way to Yokohama:

And that’s the first couple of days in Japan. Expect my next update within a week or so, covering Kyoto!

Changelog 20

Sun, 12 Oct 2014 00:00:00 +0000

TL;DR

A changelog is out showing off a very basic ragdoll, I’m putting ragdolls on hold for now since they’re not important. No updates for 4 weeks because of a holiday.

This week

Watch The Video!

Ragdolls sort of work
Modelling software is complicated

I planned to spend a couple of days getting manually defined ragdolls working this week and then to move on to some small gameplay feature. Unfortunately after I got manually defined ragdolls working (which took the estimated 2 days) I encountered even more unexpected problems which took the rest of the entire week and still aren’t fully fixed!

What’s Next?

You know what I’m planning to do next? NOTHING.

I’m going to be on holiday in Japan for three and a half weeks and it’s going to be glorious - mostly because I won’t be crunching on game development every day! To be honest, I’m such a terrible workaholic that I won’t be happy if I really do nothing for three weeks so I will be programming and I will probably even do some work on Heist - but I’m just going to mess with some far out ideas I have that are unlikely to succeed. Crazy stuff like procedural conversation tree generation or something like that.

Ragdolls Are Hard

Sun, 05 Oct 2014 00:00:00 +0000

TL;DR

I’ve been working on ragdolls this week but the approach I took didn’t quite work as expected so they’re going to be done early next week instead.

This week

As I said last week I’ve been planning to work on gameplay mechanics from now on. However gameplay mechanics needs a solid foundation and this week I decided to implement ragdolls - which I can then build a few other mechanics on top of. Ragdolls will support things like:

Killing/Incapacitating Someone (body collapses to the floor)
Dragging/Hiding bodies

And although they’re not directly related, bodies are a component of:

Looting incapacitated characters
Stealing Keys
Stealing clothes (for disguises)
Deadly Vs Non-Deadly approaches to play

Groundwork

The first thing I had to do for ragdolls was overhaul the animation system. An animation system has to do a few jobs:

Choose which animation clip is playing
Choose which frame is playing (based on the time)
Blend frames from multiple animation clips (if multiple animations are playing at once)
Interpolate between multiple frames if the time is not exactly on a keyframe
Calculate bone transforms
Calculate world transforms
Calculate skin transforms
Put the transform data into the right place in the rendering engine

Previously I had one big AnimationController class which did most of this work - but that was beginning to be a pain to maintain. So the first thing I did was split this into three classes, AnimationQueue which handles all the animation clip stuff:

Choose which animation is playing
Blend frame from multiple animation clips

Animated which handles most of the calculation:

Calculate world transforms
Calculate skin transforms
Put transform data into the right place in the rendering engine

And finally PlayingClip which handles playing back a single clip:

Choose which frame is playing (based on the time)
Calculating bone transforms (usually just reading them straight from an animation file)
Interpolate between multiple frames if the time is not exactly on a keyframe

The Secret Sauce Is PlayingClip

Ragdolls are effectively procedurally generated animations - objects in the physics engine simulate particular bones and then you just need to supply that data, frame by frame, to the animation system. This new architecture has a class to do exactly that - PlayingClip is responsible for supplying the correct transforms for a particular point time. So all I have to do is place some objects into the physics engine, and create an instance of PlayingClip that copies transforms from them - easy.

So All I Have To Do Is…

Yeah, it’s never that easy is it. The question is - what objects do I have to place in the physics engine and where do I have to place them?

As you can see here the character model already has a skeleton - this is the skeleton that is driven by the animation system whenever an animation is playing from file. It makes sense just to directly put these bones into the physics engine so there’s a very close match up between animations from file and procedural animations from physics engine. Unfortunately the results were less than optimal:

I’m so sorry for your loss.

So what happened? Well if you look closely at that diagram of the graphical skeleton again you’ll see two problems:

1) Hands And Fingers

Physics engines do not like lots of really small and fiddly objects. The graphical skeleton has lots of small and fiddly objects (all in close proximity) on the hands. This leads to a lot of instability which manifests as jittering - that horrible mess above actually looks much worse in game because it’s jittering between various horribly deformed poses!

2) Non Physical Bones

The graphical skeleton has a few bones with no vertices attached to them, for example the “root bone” runs from the floor (in between the two feet) up to the crotch and is the support which the rest of the skeleton is built on top of. The physics engine Really Hates these bones with zero mass/volume (which leads to even more instability and jittering).

Why Don’t We Just…

I tried a few things to fix this. For example the small bones around the fingers are troublesome because of their extremely low mass and tendency to collide with other fingers in troublesome ways - so I tried introducing a minimum mass limit (so fingers are much denser than the rest of the body) as well as making fingers not collide with one another. These kind of hacks changed the nature of the problems, but there were still some pretty nasty instability problems which make the system useless. In the end an amalgamation of hacks and workaround is not going to result is stable skeletons.

Fine Then, You Make The Skeleton

In the end, I decided that the solution to this problem is to not use the graphical skeleton to generate the physical skeleton - they’re used in different ways and there’s just too much of a mismatch between them. Instead I started writing a content pipeline extension which allows you to define a skeleton in an XML and this will be processed into an efficient to load binary skeleton definition when you compile it. Although I wanted to avoid more content authoring workload if possible I don’t think this is too bad - writing the skeleton definitions is pretty simple and then just needs a little bit of tweaking to get a good skeleton.

Next Time

Hopefully I should get hand authored ragdolls done in a couple of days and then I can move onto something else. I’m thinking either dragging bodies or hand to hand combat (although maybe I’ll find something else entirely unrelated to ragdolls instead)!

Changelog 19

Sun, 28 Sep 2014 00:00:00 +0000

TL;DR

A changelog is out showing off a lightning tour of some procedurally generated train carriages.

This week

Watch The Video!

Gameplay, coming soon (tm). A new bit of actual gameplay to show off each week.
Test level for the next couple of months will be a Heist onboard a train.
Each carriage will serve as a test bed for some new piece of gameplay.

What’s Next?

As I said in the video this procedural generation system is a framework for generating arbitrarly large and complex buildings - entire skyscrapers could be generated with this system. However, that’s not what I’m going to do, instead I’m going to focus on making what I have (3 train carriages) more detailed with things like doors (with pickable locks or stealable keys), windows (which can be opened or broken and climbed out of) and chairs (which can be moved, thrown and hidden behind) - basically building an interactive world instead of a huge one. i.e. From here on I’m going to be focussing on gameplay mechanics!

But What Did You Work On This Week?

If you listen carefully to the video you’ll notice I say I finished the procedural floor plan generation system last week - so what did I do this week? I’ve been doing a lot of preparation work for implementing gameplay mechanics over the coming weeks, for example I spent a load of time deciding specifically what mechanics there are in Heist and then the details of how some of them work. I’ve also been laying some groundwork on systems like the inventory system which are quite central to a lot of systems - can’t do pickpocketing without inventories, can’t take items from bodies without an inventory on the body etc.

The Return Of The Game Programmer

Sun, 21 Sep 2014 00:00:00 +0000

TL;DR

I think my work with floor plans is finally over!

The Revenge Of The Floorplans

As I said in my last update, just over a month ago, I’ve been working on floor plan generation and it’s been a lot more difficult than I anticipated! The difficulty comes from what I think is probably overloading too many responsibilities into the system - it tries to ensure that rooms stay inside the bounds of the floor, rooms don’t overlap and to generate facades between rooms with appropriate markup about which facades link which pairs of rooms together. On top of this slightly overloaded design, the entire system is (obviously) about manipulating geometry and those kinds of system tend be to extremely complicated by their nature; subtle bugs to do with numerical accuracy, comparison tolerances and the like start to creep in. Essentially, this is my long winded excuse for no updates for a while: I’ve been making progress but nothing exciting or graphical - just lots of geometry manipulation functions.

However, I think all this is mostly behind me now. I’ve spent the last week using the floor plan system to generate train carriages (nice simple, self contained, single floor buildings - an excellent test case) and then ironing out the last few bugs I’ve found. When designing a floor plan there are now a lot of conveniences available:

Rooms may overlap other rooms, the overlap will be fixed automatically
Rooms may hang outside the building boundary, this will again be fixed automatically
Rooms may have zero wall thickness between them, the rooms will be shrunk to make space for a separating wall
Walls will be placed between all rooms
Walls will be placed around external wall bordering with rooms
Rooms can stamp features (such as doors and windows) into surrounding walls
Rooms are informed of what other rooms border them, and exactly where the borders are

All this make it pretty easy to write a floor layout script, for example here is a excerpt from one of my train carriage test scripts:

    protected override void CreateRooms(FloorPlan plan)
    {
        //Get some style values
        var wallThickness = HierarchicalParameters.InternalWallThickness(Random);

        //Create balconies on either end
        float balconyLength = Math.Min(3, Length / 10f);
        _balcony1 = CreateBalcony(plan, true, balconyLength).Single();
        _balcony2 = CreateBalcony(plan, false, balconyLength).Single();

        //Reference point to create rooms relative to
        var point = plan.ExternalFootprint.First();

        //Add seating room
        _seatingArea = plan.AddRoom(new Vector2[]
        {
            Offset(point, (balconyLength + 0.05f) / Length, 0),
            Offset(point, (Length - balconyLength - 0.05f) / Length, 0),
            Offset(point, (Length - balconyLength - 0.05f) / Length, 1),
            Offset(point, (balconyLength + 0.05f) / Length, 1),
        }, wallThickness, ScriptReference.Find<IClassicRoom, ISecondClassSeating>()).Single();
    }

    protected override void CreatedRooms(FloorPlan plan)
    {
        base.CreatedRooms(plan);

        _balcony1.Node.TryConnect(_seatingArea.Node);
        _balcony2.Node.TryConnect(_seatingArea.Node);
    }

These two methods are all that is needed to create 3 rooms (2 balconies, and an open seating area in the middle) and then to connect the balconies to the seating area with doors punched through the connecting wall. Making sure the rooms don’t overlap, or hang over the side of the carriage is handled for you as well as placing all of the separating walls and determining what rooms are adjacent to what other rooms.

Next!

My plan for the last month or two has been to build a tutorial-like level with little areas to teach/test each new mechanic. This will be very useful for me to develop each mechanic with it’s own little contained environment to test and tweak the mechanic. Train carriages seemed like a good way to do this and so my tutorial level is going to be a train robbery (board the train, fight/dodge guards, move along the train, steal something from somewhere and then jump off the back). My work for the next few months is going to be building out new mechanics each week along test areas for them and probably also building some other small test levels as gameplay tests to see how fun the game is when all the mechanics interact together - exciting stuff!

But Where Is My Video?

Coming Soon(tm). Building floorplan generation wasn’t anything particularly graphical and so I didn’t do any videos, developing gameplay mechanics is the perfect thing for a video to demonstrate the mechanic as well as discuss some of the game design choices I made involving it and so I’m really hoping (and looking forward to) I can get back to making weekly videos - possibly slightly longer or more frequent ones than the brief technical updates that I’ve done so far.

The Ball Is Picked Back Up

Sun, 17 Aug 2014 00:00:00 +0000

TL;DR

I never intended to do a video last week, but forgot to put out a blog post stating this. I’ve been doing work on floor plans, it’s difficult.

Oops!

Last week I was away from home and didn’t intend to put out a video. In these circumstances I usually put out a blog post before I go away, or after I get back explaining the situation but this totally slipped my mind last week! Luckily I’ve been working on the same thing both weeks so I can do it all as one big update this week.

Floorplans

After getting all the AI stuff tied together a couple of weeks ago I decided to move onto something new - generation of building internals. This is something that I’ve worked on before and each time I come back to it I advanced my technique a little bit, usually building out some more supporting libraries to make generation easier. For example earlier this year I built a parcelling system which I use for generating the layout of buildings in a block as well as the layout of rooms on a floor which can generate layouts like this:

Obviously this isn’t a complete layout - it needs corridors and stairwells and other non-room features added, but it’s a good start.

One thing that this floor plan lacks is any information about what room neighbours what other rooms and without this information it’s obviously pretty difficult to generate appropriate doors and windows. This is the problem I’ve been trying to solve the last couple of weeks by building a generic “floor plan” representation which solves several problems:

Overlapping rooms are clipped to no longer overlap
Rooms hanging outside the building are clipped to stay within the allowable bounds
Neighbourhood data is generated between rooms
Walls between rooms have a thickness, and all rooms should be generated with a Wall-Thickness gap between them

My current prototype generates a plan a little like this:

You can see that this plan has:

three rooms (white sections)
Neighbour sections between rooms (red sections)
Walls of appropriate thickness (blue for internal walls, green for external walls)

Aaaargh

Building this system has been a nightmare. The system allows rooms and floors of any shape, and efficiency projecting every room onto every other room and then checking for occlusions by every other room is pretty hard! Honestly, I’m not convinced I’ve gone about this the right way and I’m going to spend some more time next week considering alternative approaches.

Once this floor plan system is done I’m going to be moving onto building a demo/tutorial level which will have distinct areas (rooms/floors/buildings) assigned to individual game mechanics. The plan is that I can add new areas as I add new game mechanics and then I have a single area to test and tune each mechanic.

It's Good To Be Back

Sun, 03 Aug 2014 00:00:00 +0000

TL;DR

I’m back, my wrist is still fragile but I’ve got some great work done on AI.

Triumphant Return

Finally I manage to get a changelog out! This has been a pretty depressing few weeks - always so close to functioning AI but never managing to achieve it because typing is painful. However I’ve finally managed to scrape together enough AI tech to make a video.

Watch The Video!

Completed Integrating GoAP
Integrated and overhauled Behaviour Trees
Polished NPC path generation
NPC path following
Redesigning/Refactoring SharpSteer2, my open source fork of SharpSteer / OpenSteer
Entirely new Base-ArtificialIntelligence project for Epimetheus plugins to use as a base for various AIs in game
Integrating SharpSteer2 into Base-ArtificialIntelligence
Minor refactoring work to tie together AI inputs and the animation controller originally used just for the player character

As you can see, I’ve been quite busy even whilst recovering!

What Next?

As cool as AI is it’s been pretty depressing working on it with RSI - progress was slow and, literally, painful. For now I’m going to distance myself from AI a little and work on something else to get my mind off it!

Serious Injury

Mon, 21 Jul 2014 00:00:00 +0000

TL;DR

I have a pretty fragile wrist (RSI problems) and it flared up very badly last week. No changelog :(

Wai No Changelog?

I really wanted to get a changelog out this week, I’ve been working on a project called Base-ArtificialIntelligence which builds higher level AI abstractions over the very low level things offered directly in the scripting interface. Most of my work has been building a behaviour tree which, when given a target, will generate and follow a path to that target (including regenerating the path when the target moves etc). I’m really excited to get an AI video out because AI is so core to a stealth game - I’m hoping to get a basic AI running around shooting people in a deathmatch within a week or two.

Unfortunately my wrist is in a pretty fragile condition and it really flared up last week. I couldn’t type by the end of the week and I spent most of Thursday and Friday watching anime, reading books and staring longingly at my computer wanting to work on AI!

Not All Hope Is Lost

I have spent the last 4 days avoiding my computer as much as possible (watching a lot of The International and playing a lot of board games) and my I’m feeling like it’s safe to get back to work now. I’ve decided to adopt the Pomodoro Technique since that will force me to take frequent breaks and not overexert my wrist.

Unfortunately this all means that progress is going to be slower than usual as I have to take long breaks to let myself recover.

TL;DR (2)

Don’t let yourself develop RSI, especially if you’re a software developer! It really really sucks and will have a crippling and long term impact on your ability to do your work. Know the warning signs and take action to stop it getting any worse - some wrist exercises (I have some little 1Kg dumbells which are good for wrist exercise) and a correct typing pose can completely prevent problems.

We Marveled At Our Own Magnificence As We Gave Birth To Ai

Mon, 14 Jul 2014 00:00:00 +0000

TL;DR

I’m getting close to an interesting NPC in Heist, this post is the high level view of how AI works.

Wai No Changelog?

There was no changelog published on Sunday 13th July 2014 because I was on holiday for most of the week and thus didn’t have time to work on Heist and get something ready to demo. I did actually make good progress on AI but it just isn’t ready to demo yet (next week, hopefully).

A History Lesson

I’ve previously talked in some depth about different AI techniques and how they work in my Artificial Stupidity series. This series was just a discussion of how these technologies work without talking about if/how I will use them myself in Heist.

I first worked on AI way back in 2012 when I spent some time researching AI for crowds and flocking behaviours. This kind of stuff is useful when you have a large crowd of fairly uniforms NPCs - for example the crowds in Assassins Creed. That was mostly just a research effort, although it did spawn my fork of a C# port of OpenSteer which will come in handy soon.

I eventually came back to AI when I wrote my Artificial Stupidity series in which I investigated different techniques for AI and write about their advantages and disadvantages. When I blogged about these techniques I also tried implementing them so I had a good grasp on exactly how they work and their advantages and disadvantages.

This resulted in me implementing:

Pathfinding (Planning how to get where you want)
G.O.A.P (Planning what to do next)
Behaviour Trees (Hand designed sequences of actions)

These implementations were all independent pieces of code and weren’t really integrated into anything. My work for the past few weeks has been going over these, reminding myself how they work, neatening up the code and then properly integrating them into the systems of the engine.

In fact now that you know this you can see the progression pretty clearly in my changelog videos. The last five changelogs have been:

Pathfinding
G.O.A.P
1. Overhauling G.O.A.P implementation
2. Integrating it into the engine

During my holiday last week, and full time this week, I am working on integrating fully integrating behaviour trees. Once this is done I shall spend some time writing some full NPC characters for Heist - probably a guard who patrols and arrests people, a SWAT squad who move and work in a cohesive unit and a “civilian” NPC who wanders around and flees from danger. Stealth games are ultimately about the player(s) vs the AI and this kind of stuff is absolutely core to Heist.

The View From Ten Thousand Feet

Now that you know what components I’m implementing for AI it might start to become clear how the Heist AI is going to work. G.O.A.P is a technique for deciding what actions to take next but doesn’t have a way to define exactly what actions are. Wheras behaviour trees are a good way to hand define AI behaviours but start to look repetitive when a single big behaviour tree is used for the entire AI behaviour.

I’m going to try and achieve the best of both worlds by combing these two techniques so that I define a single action with a single behaviour tree and then G.O.A.P can string together sequences of these hand programmed behaviour trees into interesting sequences.

AI Extensions

A great feature of this technique is that two actions don’t have to in any way know about each other to work together. This means that if someone writes a new AI behaviour in a plugin it can be used in the NPCs right alongside other behaviours defined in other plugins. In this way NPCs could become very rich as new actions can quickly and easily be added into the mix.

Changelog 17

Sun, 06 Jul 2014 00:00:00 +0000

TL;DR

A changelog is out talking about integrating GOAP and (some of) pathfinding

This (3) week(s)…

Watch The Video!

Integrating GOAP into the scripting system
The first NPC (powered by GOAP)
Pathfinding is half integrated
The Pet Rock NPC finds paths, but doesn’t use them yet
I still need to do more work on making sure paths found are optimal and robust

Changelog 16

Sun, 29 Jun 2014 00:00:00 +0000

TL;DR

A changelog is out showing off the incredibly cool Goal-Oriented-Action-Planning which will power NPCs

This (3) week(s)…

Watch The Video!

Using Goal Oriented Action Planning (GOAP)
Polished GOAP internals
Overhauled entire system for composing GOAP actions and goals
Built basic GOAP support into the game (exposed to scripting)

Under The Hood

GOAP

I’ve talked in some detail about how GOAP works before as part of my Artificial Stupidity series.

Happy Birthday

Sun, 22 Jun 2014 00:00:00 +0000

TL;DR

~~I wrote up and recorded a changelog but unfortunately some complicated technical problems (inexplicable build failures) have popped up. Hopefully I’ll get the changelog out tomorrow!~~

New video is out (one day late) introducing navmesh generation.

Inexplicable Build Failures

Epimetheus/Heist is a complex set of over 40 interdependent projects. Somewhere along the way one of these projects have output some incorrect artefacts and that’s messing everything up which means I can’t record the game for the changelog! These kinds of problems are incredibly difficult to debug and so I can’t rely on getting them fixed in a timely enough manner today to also get a video recorded, edited and uploaded. Hopefully I should get these issues fixed later today/early tomorrow and I can get a video out then.

This Week

Watch The Video!

C# wrapper around Recast
Integrated Recast into my own pathfinding system
Integrated Recast generation into Epimetheus world generation
Minor improvements to world generation error handling

Changelog 14

Sun, 15 Jun 2014 00:00:00 +0000

TL;DR

A changelog is out after a 3 week hiatus. Character controller improvements, first elements of freerunning movement mode, massive world generator speed and quality improvements, beginning navmesh generation.

This (3) week(s)…

Weekly changelog is back after an unplanned break!

Watch The Video!

Character Controller Improvements
Freerunning movement
Tighter integration of animation and player input
World chunking/streaming rewrite
Massive world generation speed improvements
Investigating navmesh generation
Using Recast

Under The Hood

World Chunking

The speed improvements came from two different places in the world generator. Firstly I did a complete rewrite of the “chunking” system - this is the service which divides the world up into (cubic) chunks and does things like splitting up modification commands from world generation scripts into the appropriate pieces to apply to chunks. The new system is much better engineered and does much more efficient (and multithreaded) bookkeeping of what chunks are loaded, this means much less time is wasted organising the data.

Secondly, I modified how chunks work internally. The way chunks are used is that when a modification (any modification) is made the mesh for the entire chunk is regenerated - so if there is a tiny little edit to one cubic meter of a vastly complicated chunk containing an entire building you generate a new mesh for the whole building! The new system has each chunk as an octree - as the chunk gets more detailed it will eventually split up into 8 smaller sub chunks and the sub chunks can split up further too if there is enough detail in them. This means that if an edit is made to that one cubic meter of a complicated chunk you will only regenerate the meshes for a much smaller area.

Losing My Way

Sun, 08 Jun 2014 00:00:00 +0000

TL;DR

No changelog today because I’ve started working on a major new feature (AI) and I can’t get that to a demo-able state in a single week!

This week…

Decided to work on NPCs
Begun overhauling world generation

NPCs and World Generation

At first glance this seems like a terrible lack of focus - why did I decide to work on NPCs and then dive into overhauling world generation? I already have a few AI systems like action planning, decision trees and pathfinding implemented but not integrated into the same and so the sensible thing to do is integrate those.

There is one notable thing I do not have that’s vital for AI - A navigation mesh (navmesh). A navigation mesh defines the bits of floor which can and cannot be walked on, without a one AIs cannot move at all because they don’t know what surfaces may be walked on or what paths are available from here to there. Usually a navmesh is generated as part of the level compilation stage, however since Heist generates levels procedurally that means the navmesh must be generated on-the-fly too.

This brings us to world generation. Navmeshes are inextricably tied to the world and so they must be created as the world is generated and as AIs need them. I had two choices here:

Integrate navmeshes into the current world generation system (a hacky system I have wanted to replace for a while).
Replace the current system and integrate navmeshes into the new and improved system

Obviously I have gone for option two. This new world generation system should lead to massive increases in world generation speed, physics/graphics mesh generation and will have integrated navigation mesh generation too. I hope to get this work finished early next week, and then I can start bringing other AI systems into the engine one by one.

Wildcard

Mon, 02 Jun 2014 00:00:00 +0000

TL;DR

No changelog yesterday because I had a weekend off for a game launch. Overhauled all the movement code.

This week…

Overhauled the movement system (in engine)
Overhauled character controller to support “freerunning”
Added the start of a sticky cover system

Under The Hood

Character Control Systems

The control of characters is split across two separate systems:

In-Engine support for characters basic physical movement (finding support for foot placement, navigating up and down vertical steps, managing the size and shape of the character as pose changes etc)
Mods can use this system to build a variety of character controllers which respond to player input

I have implemented some fairly generic base character controllers in a project which can be used by other mods to implement specific character movement systems. To be honest I’m a little unhappy with this situation - I would prefer the in-engine character controller stuff to be moved out into a plugin so that modders can write entirely new movement modes but the whole character controller is very deeply integrated into the physics engine and it’s not practical to pull it apart right now.

Freerunning

Once I had overhauled in the in-engine controller this was actually quite simple to implement - when the character is in freerun mode the vertical step height (i.e. the height which a character can step up when they encounter a vertical obstacle) is increased to roughly the height of the character. All I need to do to finish this off (which I won’t do for a while due to lack of a suitable animation) is couple it with a climbing animation which triggers when a very large vertical step is encountered.

Sticky Cover

The sticky cover system was slightly harder to implement. When the player presses [Find Cover Key] a ray is projected out from the camera to find what they are looking at, several more rays are then projected out to find out if there is a suitable flat surface to stand on below the point and that the ceiling is high enough above to point. If the position is found to be a suitable cover position then the AI pathfinding system will take over and move the character into the cover just as if it were an NPC. Of course, the part that’s missing from all of this is the pathfinding and NPC bit - I have no support for NPCs yet and so the cover system will remain half implemented until then!

Changelog 13

Sun, 25 May 2014 00:00:00 +0000

TL;DR

Another changelog is out, showing off the new stealth system and the new rendering pipeline/effects system.

This week…

Watch The Video!

Basic Stealth Mechanics
Effects System
Nightvision -Sonar Vision

Under The Hood

Stealth

The light based stealth system was actually surprisingly easy to implement. All lights in the system derive from the same interface (ILightEmitter) and so the first thing was to modify this interface to let me ask it appropriate questions:

 Ray? LightingRayToPoint(Vector3 point, out float length);
 
 float Brightness(Vector3 direction, float distance);

The first method gets the ray from the light to the point in question, and also the length along the ray from the light to the point. This means that if, for example, the player is standing in the sunlight you get a ray pointing from the sky to the player (as well as distance infinity). If the player is standing outside of the cone of a spot light then the method returns null, which indicates that this light cannot possibly be lighting that point. The second method gets how bright the light is at the given distance, this is rather different for different light types:

Sunlight: Always returns sun brightness
Pointligjt: Dimmer with distance from light
Spotlight: Dimmer with distance from light and angle from center of light

The final part of the whole thing is to determine if the point is in shadow. This isn’t actually done with the rendering system at all, instead the ray is passed into the physics engine - if the ray gets all the way from the light to the player then the player must be lit and if they ray hits something else first then the player must be in shadow.

Changelog 12

Fri, 16 May 2014 00:00:00 +0000

TL;DR

Another changelog is out, showing off the new physics constraints

This week…

Watch The Video!

20 Physics Constraints
8 Physics Motors
Electronic Control Of Physics Motors
More Physics Volume Detectors
A Door

Coming Soon

More complicated physics systems. Lifts? Cars? Planes? Helicopters?

WAI NO VIDEO (Again)

Sun, 11 May 2014 00:00:00 +0000

TL;DR

This week I worked on two separate goals and although I finished them both neither are easy to graphically demo in a video format.

This week…

Support For More Complex Electronic Systems
Physics Joints/Constraints

So Why No Video?

The two things I worked on this week are two completely separate systems which I need for what I plan to do next week, unfortunately neither are particularly graphical in nature and so I can’t really demo them in a video!

Complex Electronics

I started off the week working on the electronics for doors. This is a surprisingly complicated set of systems including:

Sensors to detect if the player pressed ‘use’ on the door to open/close it
(pickable) Lock mechanisms
Lock control mechanisms (e.g. a code panel as I showed last week)
Timers (electronic code panel unlocks door for a set time before closing it and relocking)
Motors to open/close the door

None of these is particularly complex on it’s own, but together that’s quite a lot of parts to put together (and of course to debug the underlying electronics framework). It didn’t take me long building these circuits to realise that I had a couple of problems.

1. The Electronics System Was A System Of Devices With One Input And One Output.

This is a bit of a problem! It was possible to work around the limitation by building a device out of multiple interconnected devices and so you could have a load of input devices connected to some logic and a single output device but this was a colossal pain in the ass to work with! It took me a couple of days to refactor the electronic infrastructure system to remove this limitation and then to rebuild a load of devices (which had previously been multiple partial-devices) into single devices. For example Electronic Infrastructure now has a much easier to use boolean logic system (And, Or, Not, Xor etc).

2. Building Circuitry Was Very Repetitive

A timed latch is a pretty simple device consisting of 2 inputs, 1 output and 3 components and 9 connections (i.e. 15 line of code at least). This a fair amount of boilerplate code to have to write every single time you want to use a basic circuit like a timed latch and the same logic extends to any commonly reusable circuitry. To fix this problem I extended the circuitry definition system so that you can use circuits as devices themselves - so now you can define a timed latch circuit and use it in other circuits.

Physics Joints/Constraints

Previously I have built a lot of support for asking the physics engine about the scene - raycasts and volume queries etc. What I haven’t done very well up until now was build support for creating complicated setups within the physics engine - basically I was lacking support for all the joints and constraints you need to build interesting things (like doors, with sliders, motors and hinges). I spent the rest of the week refactoring the (very basic) support I already had in place for constraints (previously used only in the character controller) and expanding it out to support many more types of joints. By the end of Monday I should have support for every single type of joint in the BEPU engine (about 20).

Why Do You Keep Talking About Doors?

My goal for all of this is to build a set of doors with locks, keys, code panels, motors, sensors and alarms (obviously this is pretty important for a game about breaking into buildings). I now have all the separate parts to do this built into the engine and scripting system - it’s just a matter of assembling the parts together in the right way now!

Changelog 10

Sun, 04 May 2014 00:00:00 +0000

TL;DR

Another changelog is out, showing off the new three dimensional input system.

This week…

Watch The Video!

Projecting mouse input into 3D
Rebuilt the entire UI input system to be more flexible for future changes

Under The Hood

It took me a while to get this exactly right because there are a lot of little details to take into consideration for projecting input onto a UI panel. Here’s the things to think of:

The resolution of the webpage (defines coordinate frame for webpage injected mouse)
The resolution of the screen (defines coordinate frame for real mouse)
The position, rotation and scale of the UI display due to model transform
The position and rotation of the UI display due to physics transform

That’s quite a lot to take into account!

Fundamentally, though, this is quite a simple problem. Imagine that we have some plane (the UI display) with some size/rotation/position in the world and a ray which we want to find the 2D coordinates of intersection with the plane. What we really want to do is transform the ray into plane local space. So if we first rotate the everything so that the plane is axis aligned, then transform everything so that the plane is at the origin, and then scale everything so that the plane is 1x1 it’s suddenly dead easy to find the plane coordinates - just apply the same transformation to the 3D intersection and then ignore the Z component!

Changelog 9

Sun, 27 Apr 2014 00:00:00 +0000

TL;DR

Another changelog is out, showing off the electronic infrastructure system for wiring up game logic without any scripting at all.

This week…

Watch The Video!

Transparency support in renderer
Fixed Particles
In World 3D UI

Under The Hood

As I said in the video deferred renderers fundamentally cannot support transparent objects, why is this? Traditional renderers work by rendering all the models in the scene, one by one, and applying lighting to them all individually - this means that the cost of rendering the scene is:

Cost = Number_Of_Lights * Number_Of_Models

This is a problem! You really want to have lots of lights and lots of models in your scene but this cost builds up too fast to allow that. Deferred renderers are a clever technique to reduce the cost to considerably less:

Cost = Number_Of_Lights + Number_Of_Models

How do they do this? Basically instead of rendering out every model with lighting and the full works everything is rendered out as data. When you render a model you don’t output the colour of the pixel you instead output the information about the model at that pixel, thing like:

Surface Colour
Material Specularity (shininess)
Normal Vector

This means that you build up a screen size buffer of information about what data should be rendered for each pixel. After you’ve rendered everything you take these data buffers (called G-Buffers) and calculate lighting which is of course only calculated once for the data in each pixel.

The Transparency Problem

The implicit assumption here is that every pixel has one piece of data which should be used for lighting calculations, obviously this is definitely not the case when you have a transparent thing - now you have 2 bits of data (opaque thing and transparent thing) on the same pixel!

Hack 1 - Stochastic Transparency

One possibility is to render every pixel with either the opaque data, or the transparent data on a probabalistic basis. If the transparent thing is 50% transparent then there’s a 50% chance the pixel will contain that data. Afterwards you render lighting normally and then blur everything together a bit to mix the contributions from opaque and transparent things.

This has the advantage of fitting easily into the deferred rendering pipeline but ends up breaking just about everything else. Any effects which looks at a pixel and the surrounding neighbours will now be sampling randomly from 2 different items and this will break most post processing effects. For example SSAO looks at pixel depths to work out how smooth your surface is, and makes it darker around corners, when you’ve got a transparent item on top of another item it’s as if you’ve got a surface which keeps flickering between 2 depths and EVERYTHING is a corner!

Hack 2 - Deep G-Buffer

Another approach is to store data for multiple layers in a very big G-Buffer, then you can have a more complicated lighting system which evaluates multiple depths of pixels at once to work out what colour the pixel should be.

The problem is that biggest drawback of deferred rendering is the memory consumption of the G-buffer. With deep G-Buffers you’ve now effectively got extra G-Buffers and you’ve probably just tripled (or worse) your memory usage!

Hack 3 - Forward Rendering

The classic approach is to render the opaque bits of the scene using a deferred renderer and then to follow it up by rendering all the transparent bits on top using a non deferred renderer. Again you can get the “perfect” results from this system.

The really obvious problem here is you’ve got 2 COMPLETELY SEPARATE RENDERERS! Maintaining two renderers at once sounds like a complete nightmare.

Hack 4 - Re-entrant Deferred Pipeline

I suggest you read the article (linked in the title) for a more in depth look at how this technique works.

This technique means that you can run the deferred pipeline using the previous pass as an extra input. Then you render your scene several times (once for each layer of transparency) using the output from the rendering of the previous layer as your starting point. In this way you do deferred shading on the transparent things and combine it with the deferred shading of the things from the deeper layer. Everything is deferred shading, yay!

This is actually a pretty neat approach as everything is using deferred rendering and require no special handling to use, you just throw models in the scene (semi transparent or not) and it’ll get rendered. There are, however, quite a few technical drawbacks with this approach:

Shadow maps must be either redrawn or saved between your passes. This either increases memory usage, or render time.
Post-Processing may still be slightly complicated due to only having access to the material information from the last stage (e.g. a motion blue based on velocity information in the buffer would not blur things which were occluded correctly)
Post-Processing techniques cannot use the stencil buffer (because it’s already in use by this rendering system)
Fully HDR pipeline is tricky/impossible.

Heist

I’m not sure exactly what I’ll use in the end. Right now I’m going with the forward rendering technique because it was the simplest to get working - the “forward renderer” that I use to render transparencies totally ignores lights, so that was pretty easy (cheap to render too) but isn’t a great final solution!

Changelog 8

Sun, 20 Apr 2014 00:00:00 +0000

TL;DR

Another changelog is out, showing off the electronic infrastructure system for wiring up game logic without any scripting at all.

This week…

Watch The Video!

Code-less game logic
Rewiring security systems on the fly

Under The Hood

The entire of the Electronic-Infrastructure (EI) plugin is open source, if you’re really interested in how it works under the hood you can grab the source code from here and can even start wiring things up - just make yourself a new uni test to play around in.

The core of EI is the messaging system, everything else is built on top of that. The messaging system is, at heart, just two interfaces:

IMessageConsumer<TMessage>
A thing which receives messages, as well as connection and disconnection messages
IMessageProducer<TMessage>
A thing which can be subscribed to (by a consumer), after subscribing the consumer will receive all messages the producer sends.

This basic layout is pretty much a direct clone of the Reactive Extensions (Rx) but obviously much simpler. In fact internally some of the implementations of these interfaces do use Rx, but that’s hidden away internally.

Once you have this abstraction implementing devices becomes trivial. For example a button (implements IMessageProducer) just has to listen for the engine generated “use” event (triggered when a player presses the use key whilst looking at the button) and when it receives this event it just sends along a value indicating that it was pressed.

Long Overdue Style Upgrade

Fri, 18 Apr 2014 00:00:00 +0000

TL;DR

My old blog wasn’t too bad, but I wanted something with a bit more… colour.

Colourful Is The New Black

I’ve had this blog up for quite a while and most of the time I have been using this theme (called Hooligan). Hooligan isn’t a bad theme structurally, in fact I really like the layout, but it’s a bit boring in terms of style - it’s just too dark. As you can see my new style definitely does not suffer from a lack of colour!

The new system is almost entirely Hooligan under the hood, I simply forked it and changed some small bits of the CSS. I’ve changed over from white text on a black background to more traditional black text on a white background. All the elements of the page (navbar, content area, title bar etc) are all very colour neutral which allows all the colour to come through from the background pattern. The background is generated by Trianglify and uses a random ColorBrewer scale so you get a varied background on every page but the colours are always matching and never clash.

It’s Probably Broken

It’s new, I just wrote it in one day - this means it’s probably broken somewhere! Please don’t hesitate to tell me if you find something broken, you can comment here, tweet at me or log an issue in the issue tracker for this site.

WAI NO VIDEO

Sun, 13 Apr 2014 00:00:00 +0000

TL;DR

This week I completed my task late on Saturday evening so I haven’t got any content for a video.

This week…

Floor Plan Generation

So Why No Video?

This week my task was floor plan generation. I have a system for laying out buildings in a city block, and I have everything in place for representing buildings with multiple floors and cross floor features like elevators and stairwells so the last thing I really need for basic city generation is a way to represent floor plans. However, this turned out to be a lot more difficult than I expected!

Take 1 / Monday

My first approach was to attempt to adapt the city block generation for floor plan generation. A city block is a load of plots of land in a big 2D area (the block) and a floor plan is a load of rooms in a big 2D area (the floor) and so I thought I could generalise what I already had into a system for generating 2D plans. After a huge amount of refactoring work on the parcelling system (and fixing a few bugs in it) I discovered that really there are enough subtle differences in the two systems that a unified plan generator would be extremely complex (and so abstract it would be difficult to use). I undid all this work (except the bugfixes) and went back to the drawing board.

Take 2 / Tuesday + Wednesday

I spent some time on Tuesday sketching out details of a new system based on the topology of floors. By using a doubly connected edge list structure I could represent rooms in a way that makes neighbourhood queries really fast and easy (which rooms neighbour this room, which 2 rooms are either side of this wall, which walls connect to this wall etc).

This system, again, turned out to be too abstract to be of any real use. Building up the rooms in the doubly connected edge list is a bookkeeping nightmare and I couldn’t find a good way to hide all the complexity from the end building designer. The problem is when a doubly connected edge list is still only half built basically none of the queries work, e.g.

Q. What Rooms Are Adjacent To X

A. NullReferenceException

The most important time to make these queries is when you’re still deciding where to put new rooms which is guaranteed to be when the list isn’t complete and thus queries can’t be made!

Take 3 / A.K.A. “This will only take me one day” / Thursday + Friday + Saturday

My third attempt at cracking this problem was to go back to basics - instead of trying to supply a clever system for generating floor plans (parcelling) or a helpful system to make creating floor plans easy (doubly connected edge list) I decided to simply supply all the geometric tools to make working with Concave 2D shapes easy. The new system allows you to place down a room (any 2D shape) in a floor plan, if the room you have placed overlaps with a previous room the system will clip the room to correct the overlap. At any time you can query a room to find all of it’s neighbours and the precise area of wall between two neighbouring rooms.

This system turned out to be incredibly complex (because everything is more complex when you can’t assume shapes are convex) and it took me an extra day of work on Saturday (usually my only full day off) to finally get it working! Usually I aim to get my work done halfway through Friday and then I spend the rest of the afternoon putting together a few showcase bits of the new system for the video on Sunday but that obviously wasn’t a possibility this time and I have no good demos of the new floor plan system which is why there is no video.

Next Week?

My plan for this next week is to finish of some of the last few rough corners in the floor plan system on Monday and then to move onto something new. Hopefully I should have a really big video next week where I can show off both floor plan generation and something new at the same time.

Changelog 7

Mon, 07 Apr 2014 00:00:00 +0000

TL;DR

Another changelog is out, showing off the new facade generation system.

This week…

Watch The Video!

Facade Generation
Constraint Based Design Of Facades

You’re Late!

Yes I am, this blog post is coming in a day late. I was pressed for time when I released the video yesterday and didn’t have time to write a blog post to go with it. In the future I think I’ll start writing the script and the associated blog post on Friday which will give me lots of spare time to get the video out on Saturday or Sunday!

Changelog 6

Sun, 30 Mar 2014 00:00:00 +0000

TL;DR

Another changelog is out, showing off massive improvements to the speed of the procedural generation system.

This week…

Watch The Video!

Procedural Generation Around Player
10-100x Speedup In Generation
Halved Memory Usage

Under The Hood

Procedural Generation Around Player

This one seems like a fairly simple thing to implement, but was actually fairly complex. There is a behaviour which can be attached to an entity which makes it an “Observer”, being an observer makes the procedural generator aware of you and your location and it will pick nodes around observers to subdivide.

Internally each observer finds which node of the world generator it is in which is Subdivided. So for example say we have:

City containing Blocks containing Building containing Rooms

If you’re in a building (and no rooms have yet generated) then the system keeps a pointer to the building you’re in. If you leave the building then it moves up to the city node and then tries to search down for a new building. Wherever an observer is it keeps a list of nodes which are descendants of the node you are in which have not yet subdivided. When the system wants to subdivide a new node it takes the first item from this list.

Once you have this setup prioritisation becomes simple - just sort the list by priority! The sort order is:

    private static float CostFunction(Vector3 positionOfObserver, ISubdivisionContext node)
    {
        var distance = (positionOfObserver - node.BoundingBox.ClosestPoint(position)).Length();
        var volume = (float)Math.Pow(node.BoundingBox.Volume(), 0.333f);
        return distance / volume;
    }

Further nodes are less significant, and larger nodes are more significant. Pretty simple.

10-100x Speedup

The majority of this speedup comes from batching operations. Let’s say we’re generating a staircase in a script, you could do something like this:

for (var i = 0; i < steps; i++)
{
    geometry.Add(GenerateStep());
}

So you just added a load of steps into the world, nice and simple, easy to read code - Great! Except you just issued tens of commands to the engine, and for every single command the engine has to do a load of work to update the world:

Load Chunk (potentially off disk)
Modify Geometry Of Chunk
Generate A New Mesh For Graphics
Generate A New Mesh For Physics
Send Graphics Mesh To Renderer
Send Physics Mesh To Physics Engine

That’s quite a lot. Even worse there are four threads involved in this process (world generation, geometry streaming, rendering, physics) and this code synchronizes with all of them at various points which stalls things.

The obvious solution to this problem: issue less commands. Remember, we’re just adding geometry and addition is commutative, so at the moment we’re doing:

world = world + step
world = world + step
world = world + step
world = world + step
world = world + step

What if we did:

buffer = buffer + step
buffer = buffer + step
buffer = buffer + step
buffer = buffer + step
buffer = buffer + step
world = world + step

Mathematically this is the same thing, but now we’re only interacting with the “world” once which is where the real cost is. This is exactly what the new system does.

Changelog 5

Sun, 23 Mar 2014 00:00:00 +0000

TL;DR

Another changelog is out, showing off how the procedural generation system ties together hand designed content with a procedural generation system.

This week…

Watch The Video!

Getaway Points
Procedural Generation

Aargh!

Something broke with my editing process this week. I was trying out something new with Lightworks and it didn’t really work. Unfortunately fixing this would require a total re-cut of the video which would basically mean no video today :(

I’ll try to make sure this doesn’t happen again!

Under The Hood - Procedural Generation Series

This week there is no under the hood section. Don’t worry - next week I’ll talk about how the procedural generation works internally which is really complicated!

Instead, check out my series on procedural generation from a while back:

Changelog 4

Sun, 16 Mar 2014 00:00:00 +0000

TL;DR

Another changelog is out. I’ve tested and polished the interactivity system, allowing entities to be “used” by the player.

This week…

Watch The Video!

Interactivity System
Plugin Documentation

Under The Hood - Using Objects

The interactivity system is really very simple, actually it’s so simple I almost decided not to write this section! The player character has an ItemUser behaviour attached to it which listens for UseKeyPressed events from the input system. When the behaviour receives an event it projects a short ray into the world (out from the camera position) and finds out what entities it hits. For each entity it does two things:

If the entity has a behaviour which implements the IItemUsable interface then the Use(EntityId user) method is called on that behaviour.
A general event is broadcast indicating that entity A used entity B.

And that’s really it. To make an object interactive you simply need to attach a behaviour implementing IItemUsable to the behaviour and you’re sorted. Or if you want to listen to other entities being interacted with for some reason you can just listen for the event.

This Is Madness!

Tue, 11 Mar 2014 00:00:00 +0000

TL;DR

Several really cool projects have been released by some really big names in gaming.

“If you’re good at something, never do it for free”

There are a lot of tools for building games for free these days, even entire engines like Unity, Unreal and Irrlicht are just given away and this is fantastic - more people making games is good news for the industry. However, some of the best tools are only available for crazy price tags - e.g. Maya is $3,675 - a price tag which makes an Indie developer like me go and cry in a corner. To bring me out from my corner several really cool tools have been released, totally free, in just the last couple of days.

Firelight Technologies: FMOD Audio Middleware

FMOD Studio is an audio content creation tool for games, with a focus on a ‘Pro Audio’ approach. It has an entirely new interface that will be more familiar to those using professional Digital Audio Workstations than existing game audio tools and is loaded with new features.

Firelight Technologies announced yesterday that FMOD is now free for commercial indie projects. Games with a development budget under $100K can use FMOD for free! This is really huge, implementing a full featured Audio engine is a pain in the ass and I expect most indie games just opt to ad-hoc build features as they need them. Someone on Reddit summed this up perfectly:

This is so fucking huge, and I am so excited. Sound designers can find it very difficult to be proud of their work without proper implementation, and it’s been nearly impossible to convince indie devs to shell out for middleware because it’s so goddamn expensive especially with multiple porting becoming commonplace with unity. I am so excited about this. It will save indie devs hundreds of hours of audio programming and exponentially increase the quality of audio in low-budget games.

For me personally this is great. The audio engine in Epimetheus is very basic - it can render 3D audio and only handles attenuation by distance + doppler shifting. I’m not going to replace it any time soon but I expect eventually I’ll drop FMOD in as a straight replacement and then after that doing more advanced audio FX will be much easier.

Valve: DirectX To OpenGL Translator

Direct3D -> OpenGL translation layer.

Taken directly from the DOTA2 source tree; supports: Limited subset of Direct3D 9c. Bytecode-level HLSL to GLSL translator. Some SM3 support: Multiple Render Targets, no Vertex Texture Fetch

TOGL is a translation layer which Valve use in Dota2 to translate the (Windows only) DirectX calls into (Cross platform) OpenGL calls. This isn’t particularly useful to me, personally, but it’s pretty cool and I imagine we might possibly see this code cleaned up a little to become a more general purpose DirectX->OpenGL translator.

Sony: Authoring Tools Framework

Authoring Tools Framework (ATF) is a set of C#/.NET components for making tools on Windows. ATF has been used by most Sony first party game studios to make many custom tools such as Naughty Dog’s level editor and shader editor for The Last of Us, Guerrilla Games’ sequence editor for Killzone games (including the Killzone: Shadow Fall PS4 launch title), an animation blending tool at Santa Monica Studios, a level editor at Bend Studio, a visual state machine editor for Quantic Dream, sound editing tools, and many others. ATF has been in continuous development in Sony’s Worldwide Studios central tools group since early 2006.

This looks like it’s going to be an incredible resource for me when it comes to building a set of modding tools for Epimetheus. ATF includes basically a complete framework for building game development tools (and I guess it would be pretty damn useful for not game development related tools too).

Pretty Cool

This is a pretty cool coincidence that these three sweet tools all came out on the same day. I wonder if this is the start of a trend? I certainly hope so!

Changelog 3

Sun, 09 Mar 2014 00:00:00 +0000

TL;DR

Another changelog is out. The entire User Interface (UI) system has been rewritten so it’s neater, easier to use, more extensible, fully moddable and customisable.

This week…

Watch The Video!

New UI system
Some new UI components
Earliest beginnings of the Heist gamemode

Under The Hood - User Interfaces

User Interfaces are (fairly obviously) all the graphical bits of the game which are there purely for user interaction. This includes buttons and text boxes on the various menu screens as well as reticules and health meters on the in game HUD. Every single UI in the Epimetheus engine is rendered in the same way - they’re all just a website. The game engine actually runs a little local webserver and then the UI system loads menus and HUDs from that, it also makes calls back to the webserver (just like a normal website) when it wants to send a message back to the game.

This may sound like a really weird idea, after all websites aren’t exactly designed for high performance graphics like you expect from a game engine! However, it turns out that modern browsers (Epimetheus uses Webkit via Awesomium) are sufficiently well optimised that they can handle everything I need. Of course the advantage of using web technologies for UI should be fairly obvious, all the basics like text boxes and button are part of HTML and it takes no effort to use them (writing your own text box is a surprisingly difficult job). On top of that there are loads of open source projects out there for Javascript, I suspect that Javascript has the most active open source community of any language and that’s all code I can use in the UI.

How It Used To Be Done

The old way I was doing UI for gamemodes was basically the most obvious way possible. A gamemode includes a .html file, which references a load of CSS and JS files as usual and the game loads that up. So let’s think how this works for the reticule:

Visible elements of the reticule are defined in DeathmatchHUD.html (mixed in with elements from other HUD components)
Code to make the reticule work is in reticule.js
Reticule.js is loaded up in the html file (but in a different place to the visual elements)
Styling for the reticule is in Reticule.css (loaded in yet another location to either the code or the visible elements)

So this one UI element is spread across three different files, one of these files is actually a mix of a load of different UI elements all thrown together. Want to customise this? Tough luck! You have to rewrite the entire HTML file all over again.

What Needs Changing?

So it’s pretty obvious that some stuff needs changing here:

Using a UI component should only require one line of code
CSS should be kept separate from JS, so you can change style without needing to change anything else
You should be able to replace one single UI element without changing everything else

How It’s Done Now

I changed the UI system to entirely use requirejs. Individual UI components are defined as modules, the style is kept in a separate file which the component module depends on. There’s one single script file which sets up your UI and using a new component simply requires depending on a module and then calling:

MyModule.Create(some, options, here);

The real secret sauce is how modules are located when you ask for them. As I mentioned before the game is running a local webserver, so when requirejs wants to find a module it asks the webserver for the module. The webserver then goes off and searches through all your installed mods for a file matching the supplied parameters and certain files can be set to override others.

Want to change the look of the reticule?
Just override HUD/Plugins/Reticule-Plugin.css with your own stylesheet!
Want to entirely replace the deathmatch UI with a different set of UI components?
Just override HUD/js/deathmatch-main.js with your own JS file and setup any UI you like!
Want to add a brand new UI component for your own UI?
Just write it as a requirejs module and use it!

UI Customisation

As I said in the video this all opens up the path to some kind of UI editor in the future which anyone can use to drag/drop elements on the UI and have a totally custom UI. The cool thing would be that this custom UI would just essentially be a plugin, so if you made some really cool UI in the drag/drop editor you could pack it up and share it with people just like any other plugin.

Changelog 2

Sun, 02 Mar 2014 00:00:00 +0000

TL;DR

Another changelog is out. I’ve added a unified inventory system for plugins plus tidied up a lot of code in preparation for starting on the heist-gamemode plugin next week.

This week…

Watch The Video!

New “Under The Hood” section in blog partner blog posts to the video covering technical details
New unified inventory system for plugins
Reworked guns to work with inventory system for ammo
Rebuilt character service to make things like reviving downed teammates, respawning on teammates and spectator cameras for dead/non players much easier to implement.

Under The Hood

As I said in the video it seems like a bit of a waste to have these blog posts where I just re-iterate what I’ve said in the video. Instead I’m going to try and have these “under the hood” sections where I’ll talk about the technical aspects of what I implemented and how I did it.

Event System

The event system is how the engine sends and routes messages about events. Things like:

EntityCreated
EntityDied
PhysicsContactCreated
InventoryUpdated
ToolEquipped

are all events that the engine publishes, along with hundreds of others. Sending events is cheap so almost everything sends out some events about things that other systems may be interested in. Sending an subscribing to events is done with a generically typed system, to get an event sender you call:

 var sender = e.GetEventSender<TypeOfEvent>(scope);

and to subscribe to events:

 e.Subscribe<TypeOfEvent>(subscriber, scope);

So this seems like a pretty good way to do things, event senders and subscribers are all strongly typed so you can never accidentally subscribe the wrong type of listener to the wrong type of event. Additionally there is the scope, which must match up for a message to get from sender to receiver.

However the scripting system had a problem with this: scripting types are never loaded into the engine domain. Which basically means you cannot use a type defined in a plugin within the engine (for sandbox/security purposes). The obvious problem, of course, is if you can’t use the types defined in plugins how can a plugin ever use an interface which depends entirely on generic types?

There are two cases we need to consider:

The plugin is using an engine type
The plugin is using some type defined in the plugin (which the sandbox forbids the engine from understanding)

Plugin Using Engine Types

If the plugin is using a type defined in the engine or some other primitive type then everything is fine, that type is already loaded into the engine and the event system can just be used normally. I determine this by inspecting the assembly of the type and the list of all assemblies loaded into the engine, if the engine has the type assembly loaded then it can understand this type just fine.

Plugin Using It’s Own Type

If the plugin is using some type which it defines then the plugin system exploits the scope system to fake events. An event of type PluginEvent is created, plugin event is a pretty simple type:

struct PluginEvent
{
    byte[] serialized;
}

The scope is set to the full (assembly qualified) name of the type. Any time the plugin sends an event it is serialized and sent as normal. Thanks to the scope system the only receivers which will get the message are ones looking for the same type (whereupon, of course, the type is deserialized back into the correct type).

Character Respawning

The changes to the character system are pretty simple, they’re really just a slight rearrangement of the old system. Under the old system there was a service (in the engine) which monitored the character entity and, when it died, asked the GameMode object to create a new one. The new system does away with this and does the same work within the GameMode object. Since all the work is done within the GameMode this means that more advanced character handling around death and respawning is much easier to implement.

Changelog 1

Sun, 23 Feb 2014 00:00:00 +0000

TL;DR

The first proper changelog is out, I talk about how multiplayer no longer sucks.

This week…

Watch The Video!

Reviewed networking code and fixed a lot small bugs.
Exposed networking system to plugins allowing them to send messages about whatever they like to other peers.
Taken advantage of this in the DM plugin to send player input and sync animations across the network.

What’s Next?

I’m thinking about adding in some of the most basic elements of the actual Heist gamemode next, so I can pull off a (extremely) basic Heist - basically just loot entities to collect, an inventory system to put loot into, an escape point which ends the game and counts up how much loot your team made off with. I’d also like to get a basic bank layout generator going, but I think that’s probably a couple of weeks down the road yet.

Changelog 0

Sat, 15 Feb 2014 00:00:00 +0000

TL;DR

I’m going to be releasing weekly changelog videos, this first one introduces what Heist is and where you can find more information.

Weekly Changelogs Every Saturday

Showing off videos of game engine development is hard because most of the work is really boring technical stuff about network packet encoding, finite state machines and ReSTful APIs. Because of this all the stuff up until now has been blog posts with a lot of technical content.

However, the project is starting to move from entirely being about building the Epimetheus game engine and is starting to pickup some pace with actually building a game on the engine. From now on every Saturday I’m going to be putting out a brief changelog video summarising the changes I made in the preceding week. This first one is a little special, I’m taking this opportunity to really quickly introduce the project and next week (changelog 1) will be the first real changelog.

Changelog #0

Go and watch it, right now!

Where Is The Long Promised Alpha Release?

It’s been an extraordinarily long time since my last release and a lot has changed. Ideally, I would release an alpha build alongside changelog #1. However, right now the game requires steam (for networking) and so I can’t redistribute the game without first going through greenlight. I’ll probably strip out the steam dependency in a month or two and make a single player only release for people to play with.

Payday The Heist 2

Mon, 13 Jan 2014 00:00:00 +0000

TL;DR

A pretty badly broken game.

General Impression

The first payday game was a bit of a disappointment. A co-operative game about breaking into banks is something I want to play so much I’ve spent 2 years building such a game! The game instead turned out to be a series of levels about slaughtering your way through hundreds of police whilst waiting for the worlds slowest drill to finish drilling a lock. Tragically payday 2 is no different - the entire game seems to be designed to degenerate into an endless police slaughterfest as soon as possible.

The Good

To be honest, I’m struggling to come up with good things!

Casing The Joint

Before every mission you have a chance to scout out the level. In multiplayer this is the time when you walk about with your team and discuss how you’re going to pull off the job. I really enjoyed doing this - it’s really fun to walk about discussing how you’re going to knock out these two guards at the exact same moment as the third guy walks in the front and takes everyone hostage to prevent them from escaping, and then how you’ll proceed to bag up all the jewellery without anyone escaping all whilst the drill breaks open a bonus safe in the backroom. It never goes down like this in execution and that’s kind of the cool thing - dealing with the mistakes everyone makes messing up your perfect heist.

Incapacitated Guards Have Pagers

When you incapacitate a guard their pager will go off (I have no idea how the pager magically detects that the guard is unconscious) and you have to answer the pager. This is a pretty neat idea which means it effectively takes a little longer to knock out a guard and makes the whole “sneak around knocking everyone out from behind” approach a little harder.

Branching Missions

To be honest, I haven’t played enough ~~police slaughterfest~~ Payday to really get a feel for how the branching missions work. They were described as each scripted mission having a series of possible follow up scripted missions depending on your performance in a mission. E.g. in mission 1 you steal some art, in mission 2 you take the art to the person who commissioned you to steal it and then either trade it peacefully (mission over) or double cross him and make off with the money and the art which leads to mission 3 (escape). This is pretty cool and it’s an idea I’d like to be possible in Heist - a branching mission generator ought not to be a lot harder to write than a linear mission generator.

The Bad

Casing The Joint

Casing the joint is a very cool idea done very wrong. When you’re walking about scouting things, even if you stick to public areas, guards will detect you if you stand in front of them for more than about 5 seconds. This means that instead of walking around planning out the perfect heist you’re standing outside the building hoping that a random NPC doesn’t walk past and somehow telepathically detect your criminal intent.

On top of this casing the joint is a definite mode - once you want to do anything criminal you have to put on your mask. Once your mask is on then everyone (guards, security cameras, random members of the public) will instantly know you’re a criminal and raise the alarm unless you incapacitate them or take them hostage. This all means you cannot pull off a stealthy heist because as soon as you put on your mask (which you have to do) you alert everyone who sees you.

Incapacitated Guards Have Pagers

Again this is a nice idea done very wrong. When you answer the pager of a guard you obviously have to impersonate the guard and once you’ve done it three times you can’t do it again - on the fourth time the call handler will become suspicious and raise the alarm. This sucks because it means that if you have a 4 player squad (the max) then you can only knock out a total of 12 guards, effectively placing an upper limit on how much you can stealth before you simply have to go in loud.

Failure Amplification

The failure amplification in splinter cell was a bit annoying: sometimes the game would amplify the smallest mistake up to the level of requiring you to reload the latest checkpoint. Payday 2 is much much worse: There are no checkpoints and every single level amplifies the tiniest mistake up to the level of a full on SWAT assault.

You fail to call in on the pager when you incapacitate a guard: FULL SWAT ASSAULT
A single hostage escapes: FULL SWAT ASSAULT
You walk in front of any camera without disabling the security room first: FULL SWAT ASSAULT
You begin drilling a lock without subduing every single person in the building: FULL SWAT ASSAULT
You stand in front of a guard for 0.1s too long, even though you don’t have a mask on and made no suspicious moves: FULL SWAT ASSAULT
I think you get the idea

Naturally once a police assault has started there’s no way to stop it, and no way to resolve it peacefully - you just have to slaughter endless police whilst completing the objective.

730 Days Later

Wed, 01 Jan 2014 00:00:00 +0000

TL;DR

I’ve just uploaded the first version of the Placeholder Software website. Keep an eye on it for a greenlight release of Heist.

Prisoner Zero Has Escaped

I’ve been very busy over the last few months - the change of the engine from Lua to C# scripting has put me closer than ever to being in a state to actually build a game instead of fiddling around with engine tech. Over the last few months I have built most of the scripting new API, built a load of entities in it, constructed a deathmatch gamemode and then rebuilt the underlying network system of the engine to work with Steam. The result of this is that I now have a really simple playable game running on the engine. This has been my aim for first proper release all along, to be honest it’s pretty scary now I’m actually here.

Over the Christmas period I have been working on the things needed for an initial release (on greenlight). The most important part is a New Website, check it out (seriously, go check it out right now). Tomorrow I will be working on brushing up the Steam greenlight page so that it’s ready for me to make public. Once the greenlight page goes public I shall have to look into contacting the gaming press and drumming up some attention.

Cracks In The Wall

There is still a little coding work to do before I can properly call the website complete. Next week I will be working on brushing up the bits of the game needed for a trailer:

A better DM map
A small chunk of city to show off
Some character animation would be nice!

Then I shall have to write a script, film the trailer and edit it. Once that’s all done I can get the greenlight page up and call the website properly done.

The Hard Bit

Heist has always been a difficult idea to communicate. Do I focus on it being a game about breaking into stuff, or do I focus on it being a really flexible engine which can be extended with loads of new gamemodes very quickly and for little cost? My not-really-a-solution at the moment is to have a main trailer which shows off the basics of the game (explorable cities, stealthy theft) and then to have some other trailer/tutorial videos which talk you through modding the engine.

The Rabbit Hole Never Stops

This might sound like the engine is complete and from here on it’s all building the game itself. It’s not. There are still two very big changes to the engine that need doing:

World Generation V3

The current world generator system is an improved version of the very first prototype I made 2 years ago. It’s good at what it does, but I’ve come to realise that the entire way in which it works is pretty flawed. The current generator generates the entire world as fast as possible (prioritising parts of the world close to the observer). A better way to do it would be to instead only generate parts of the world as needed which would effectively reduce the processor load of world generation by spreading it out over more time. This new system will also pave the way for introducing NPCs into the world by generating pathfinding navigation meshes on-the-fly as the world is generated.

Luckily I think this change will probably only take about 2 weeks and is a sufficiently well encapsulated change that nothing about the procedural world generation API for scripting will change (i.e. this isn’t a breaking change).

Artificial Intelligence

I’ve talked about AI a bit before. I have a lot of bits of AI tech mostly complete and tested but not integrated into the actual game engine. Obviously this needs to change and I need to start exposing AI components in the scripting interface and then building some generic NPCs which can be used as “drop-in” NPCs for mods. My immediate goal here would be to build a bot which can play deathmatches - this is a sufficiently well explored area that I’ll have lots of other games to compare my bots to.

This is, obviously, a pretty massive piece of work. A stealth game is very dependent upon good AIs, indeed a stealth game is entirely about playing against NPCs.

But When Can I Play?

Right now! That’s actually a lie, the game is currently only available to my friends who are serving as closed alpha testers, but if you’re one of those people it’s true because the game is actually playable as a very basic deathmatch. It will take about 1 week’s work to properly polish this upon in a decent game (which I’m slowly chipping away at as I complete the publicity work). I’m also going to be adding a few more experimental modes in a similar vein to DM:

Team Deathmatch
Capture the flag
King of the hill
Assassination (Team 1 tries to kill a VIP, team 2 tries to get him to the objective)
Hunted (every person is given a single target to kill)

None of these are particularly groundbreaking, but they should all be fairly simple to implement and will serve as a pretty good testbed.

TL;DR When Can I Play?

Keep an eye out on this blog for an announcement of the game being on Steam greenlight. Once it’s through greenlight (or maybe before) the game will be available for everyone to play.

I Just Needed A Rest

Tue, 12 Nov 2013 00:00:00 +0000

TL;DR

I’ve completed a whole load of really big tasks in the time since my last update.

Scripting

In my last post about the game I talked about how I was starting to replace the old Lua scripting system with a new C# scripting system. I finished up the post by pointing out how this was going to set back any previous timetables I had but that I would hopefully save time in the future. Well I was totally right (on both accounts, so that makes this pretty neutral news).

It took a while to figure out how Appdomains work to properly sandbox scripts but I now have a much better scripting system. The biggest gain is simply having proper types which means I’m no longer constantly consulting documentation on exactly what arguments a function requires (and then in the function itself asserting that the arguments are of the right type and doing lots of silly coercion etc). On top of that C# is simply a faster and more powerful language than Lua, so scripts are much more testable, easier to write and faster to execute. I think the new scripting API is going to be loads more accessible to modders - simply add a reference to EpimetheusPlugins in your C# project and intellisense will tell you the rest!

The downside of course is all the lost work. All the various mods I had implemented like the circular city, Deathmatch, Construct mode, various tools and weapons, BigSkyscraper and others are all gone and must be replaced.

Multiplayer

I have made two attempts at multiplayer before, both times I got a basic multiplayer system working before encountering some annoying difficulties with something like session handling, session initiation, joining in progress, players dropping mid game, entity ownership etc etc and moved on to another task. I didn’t want to talk about multiplayer a third time until I had something solid working.

This time I’ve made a totally new multiplayer session handling system all implemented on top of Steamworks. When a game is in progress all the players are in a steam lobby together, thus if any kind of session management stuff needs to be done (e.g. sending a message to another peer before the session is fully initiated) I don’t have to mess with a half initiated session instead I just send the message via the steam lobby! Obviously the lobby is also used pregame for player chat and choosing game configuration options.

There is one really massive huge problem with this: I don’t have any kind of agreement with steam to distribute the game (I will hopefully be on greenlight before 2013 ends). This obviously means that I cannot distribute the game in its current form. However the whole steam lobby dependency is well abstracted and when I eventually get around to implementing my own lobby system I can swap out steam and distribute that version of the game.

Logging

This is going to sound stupid but it’s true: after nearly two years of development I’ve added proper logging into the engine. It’s not complete but now that I have a logging framework built in it’s really easy to wire up a class with logging messages when I encounter a problem with the class. Sometimes I even just wire up a class with logging when I’m bored to pass the time (very bored).

ReSTful API

One thing that I was unhappy with for a long was how the main menu worked: the menu rendered a webpage (from a file) and also bound a load of C# messages as global javascript functions. I ended up with a horrible ad-hoc mess of bound functions and no good way to interact with them because javascript frameworks all assume (quite reasonably) stuff happens through AJAX and not magic global functions!

There’s an obvious solution here, in fact I already mentioned it: if javascript frameworks are all built around using AJAX then why not use AJAX? This seemed totally crazy since this means the game would have to be run a local webserver in the background! Really though there’s nothing wrong with that - the UI is a website so why not serve it properly?

This has turned out to be the second best (after C# scripting) change I’ve ever made to the engine. It started off as a simple API exposing all the configuration options of the engine:

GET http://localhost:41338/config (list all options)
GET http://localhost:41338/config/{option name} (show specific config option)
PATCH http://localhost:41338/config/{option name} (change a specific config option)

This meant I could get rid of the equivalent methods I was binding before and it seemed to work nicely. It had the added little bonus that you could visit these links in your browser while the game is running and interact with the backend API yourself. Since this experiment worked I went and replaced all of the methods bound in the main menu, this reduced MainMenu.cs from a 500 line beast with lots of complex stuff (like marshalling async callbacks back onto the UI thread to post results back into the javascript context) down to this:

public class MainMenu
        :AwesomiumScreen
{
    public MainMenu(IKernel kernel)
        : base(kernel, Path.Combine(Configuration.UiDirectory,"Screens\\MainMenu2\\MainMenu.html"))
    {
    }
}

That seems like an improvement to me! There was still one thing here that I was unhappy about though and it’s that URL hardcoded in there. The way that works is that whatever path you have configured as your UiDirectory must have a file MainMenu.html at that relative filepath (if it doesn’t then you don’t get a MainMenu shown, bad luck).

The way any other content in the engine is located is you supply a path fragment and it basically finds any file in any mods directory which ends in that fragment. So if you’re looking for “foo/bar.bash” it’ll find things like:

Mod1/foo/bar.bash
Mod2/directory/foo/bar.bash
Mod3/directory/directory/foo/bar.bash

Why should menu locations be any different? The Main menu is making two mistakes here:

Specifying the UiDirectory instead of searching all mods
Being too specific and not just searching for MainMenu.html

Now I already had an API endpoint which would give back the contents of any file in a mod. Using the example file paths from above:

http://localhost:41338/modifications/The_ID_Of_Mod_1/foo/bar.bash

This would return the contents of the file foo/bar.bash. So really the MainMenu should be using a path like this to serve itself. All that was needed was a way to take MainMenu.html and locate which mod and what path to serve. So the new MainMenu.cs looks like this:

public class MainMenu
    :AwesomiumScreen
{
    public MainMenu(IKernel kernel)
        : base(kernel, new Uri("http://localhost:41338/views/MainMenu.html"))
    {
    }
}

“views” is an endpoint that will take the rest of the URL as a search path and search all installed mods for that path. Once it finds something that matches it redirects to that file and renders it out as a html view. This makes the new menu system loads better than the old one. Now you no longer need all your UI elements in one mod (specified by the UiDirectory option). Instead any file in any mod which can be a menu is found as used as appropriate!

Simple Tools Built Quickly

Since this journey of discovery improving the MainMenu.cs I’ve added lots of extra capability to the ReST API:

The entire lobby menu interacts with the steam lobby object via the API (plus a websocket for delivering live messages).
All screen transitions are managed via the API
Various engine performance statistics are published on the API
Details of the render pipeline (with images showing each stage) are published via the API (and the entire render pipeline can be reorganised with a simple PUT)
All entities currently in the scene are shown in the API and can be created, deleted and updated with HTTP requests

The consequence of all this power being in the API is that if I want a tool to do a single job I can build it as a webpage and act through the API. For example, if I want a tool to show all the lights in the scene and manipulate their positions/colours that’s a nice simple webpage to throw together and then I can fiddle with the lighting of the scene in real time from a nice simple purpose build tool.

Ivory Towers/Coming Soon

I’m sure some people who are following the project have thought it (I certainly have): I’m building lots of cool engines tech but no games! For the past two years I’ve been building the engine tech to do some pretty unprecedented stuff (vast procedurally generated worlds) and I’m finally at the point where it’s all mature enough for me to start working on games! I finished work on the entity API today and tied up a lot of little loose ends before moving on to the next big sprint (tomorrow).

The next two things I do are going to be:

Deathmatch
Heist

The deathmatch is the simplest gamemode I can implement. Simply some people in a multiplayer session killing one another with guns. This will be a test of all the basics of the system such as generating small shared worlds, physics, player movement, hit detection etc etc. The Heist mode will start off as a very basic iteration over this. It will basically just be a box (labelled loot) which the players can pick up and carry to a certain point to win the game. Once I have this basic Heist mode I can start working on the features that make Heist fun such as complicated buildings containing the loot, alarm systems guarding the loot and NPCs populating the world around the loot.

Splinter Cell Blacklist

Tue, 27 Aug 2013 00:00:00 +0000

TL;DR

I like a lot of blacklist, it’s let down a by a few bad design choices.

General Impression

I’ve never played any splinter cell games besides blacklist, and from reviews I’ve read it seems like I’m missing out - blacklist does feel a lot like a series of puzzles to solve rather than a simple stealth game. That said, I’ve really been enjoying solving this particular series of puzzles. The game also gets quite a lot of small stuff right, which is great.

The Good

As I said, the game gets a lot of the little details right. This is actually the thing I’m most interested in since I’m reviewing these games as a way to analyse what does and doesn’t work in a stealth game for my own game.

Sticky Cover

The game has a fantastic implementation of sticky cover. In most implementations I’ve seen once you’re in cover you are either stuck in that cover (only able to move left and right), sometimes you’re able to flip across from one piece of cover to another (e.g. across a door). Blacklist extends the cover jumping mechanic to be in the hands of the player - when you’re in one piece of cover and looking at another a single keypress will send you diving across to the cover. This may not sound big but it’s actually a big win for immersion - you’re not busy wrestling with movement controls and manually trying to move from cover to cover, instead you’re diving around cover like a total pro while concentrating on the tactical situation at hand.

In World Hints

The game doesn’t hide objectives in some annoying mission menu (replace “menu” with “PDA”, “Diary”, “Journal”, “Notes” to taste). Missions are instead written into the world - as you enter a new level it will literally have some huge text written onto a nearby building stating your mission, as you complete a mission and round a corner you’ll again be confronted with text stating your mission (while additional details and exposition are given over the radio). This same system of in world hints for missions applies to the entire control scheme - whenever you’re capable of performing some action (e.g. climb over a wall) there will be a little hint painted into the world showing the key to perform the action.

On the surface this sounds terribly immersion breaking - real soldiers don’t often get orders sent to them via mural (or see movement hints painted on walls). I think it’s reasonable to tradeoff the tiny immersion loss of in world hints for the huge confidence gain a player gets by being able to run around a level and know how to do everything - it’s very immersion breaking to run up to a wall and then be caught by a guard because you accidentally vaulted over it instead of taking cover!

Everyone Is Deadly

This is definitely not a game with any bullet sponges, I’ve basically played exclusively using a pistol and a knife (sometimes falling back to a silenced sniper rifle for long range kills when necessary), this makes the player supremely deadly. The enemies, if they ever catch you, are also extremely deadly (not quite as deadly as the player - usually finishing me off with a burst or two of bullets). This obviously all works very well within a stealth focussed game where you know that if you get caught the guards will almost certainly kill you.

Optional Multiplayer

Blacklist has a lot of missions which are not central to the main plot - simple things like “eliminate all the people at this blood diamond mine” or “plant electronic signal taps at 3 locations in this building without being detected”. Most (all?) these missions can be done either solo or cooperatively with one other person, this is really cool. The game doesn’t just drop an extra person into the same mission as call it done either - when you’re working with another person there are new routes available that cannot be done with a single player.

Uneven Multiplayer

Blacklist has the “Spies Vs Mercs” Gamemode which pitches powerful, heavily armed and armoured mercs against sneaky spies, armed with peashooters and paper thin armour. I haven’t played much SvM yet and the bit I did play seemed punishingly difficult to newbies. However, this is a pretty interesting idea to bear in mind for any kind of stealth game - essentially putting one team of players into the shoes of the hapless AI guards getting slaughtered by elite sneaky spies.

The Bad

Failure Amplification

Conversely to the last point the game suffers something I’m going to call Failure Amplification - there are some points where the game deliberately amplifies a very small mistake into having much larger consequences than normal. Usually when you get spotted by an enemy they will shout out (alerting nearby enemies) and start searching for you, after a while they usually give up the search and return to patrolling. However, there are some levels where you are informed that the enemy has reinforcements standing by and that if you are spotted they will summon the reinforcements (who will then arrive instantly, in large numbers, and on a permanent high alert). In these levels the small failure (being spotted by one single enemy) results in a massively amplified response. This doesn’t really increase the standard stealth game tension - after all your sneaking about will be the same as usual - it just introduces an incredibly frustrating event that usually requires a quicksave load.

Dogs

Dogs. Oh God how much do I hate dogs in Blacklist. Humans are predictable and understandable - they react to noises by investigating them and they can only see things in front of them. Not dogs, oh no, Dogs react to small noises by…barking loudly and putting almost every guard on alert (failure amplification to the max). Not only that, I honestly have no clue how dogs detect me half the time, I guess the game gives them some omnidirectional sense of smell that picks up enemies in a radius.

The main problem with dogs is that I simply don’t understand them and the game makes no effort to help. Whenever I’m detected by a human it makes sense, I know that he saw me because I failed to take cover properly. Conversely whenever I’m detected by a dog it feels like total bullshit, I was detected by the dog through two layers of cover, whilst not moving, in a big smelly junk yard… yeah that makes sense.

The Asshole Level

This is one I’ve seen in a lot of games. Most splinter cell levels have a bit of quick storyline justification before dumping you into a building full of bad guys with some objective to complete. If you don’t care about the story it doesn’t matter because it’s just a short interlude between each level which takes a backseat to the game actually being fun. Except the asshole level. The asshole level is the one where the game takes a backseat to the story. Splinter cell has two (that I can think of, I’m pretty sure there’s more).

I’m Taking Away All These Toys

So at one point the protagonist takes a face full of deadly nerve gas (“Sam! Get away from there, just 1 micro-gram of that stuff will kill you”). This extremely deadly stuff then… makes the screen wavy for a bit and makes the controls slightly mushy as the character weaves around before collapsing and being captured in a cutscene. This is pretty annoying, the whole section of the game where you’re playing with a wavy screen was not in the slightest bit fun.

Of course once captured all of your equipment is taken away and when you later escape you have to get away, within a certain time limit (before the nerve gas is untreatable), with a slightly wavy screen and mushy controls, and no equipment. Fun.

Oh I Suddenly Realised This Is The Last Level

The last level of games usually seems to suck, and blacklist is no exception. First control is taken away from the player several times for an important storyline cutscene to do it’s thing, then we have the player captured again, and finally the player character is injured and we have the final boss fight with a character who is injured and can’t do half his usual moves. It really feels like the game was going along full steam when they suddenly realised they needed to finish it, so they tacked a heavily storyline driven level on the end with no consideration for how much it sucks to be railroaded this much in one level.

Gruelling Homework Assignment

Mon, 26 Aug 2013 00:00:00 +0000

TL;DR

I’m giving myself a plausible sounding excuse to slack off and play games.

A Reminder About Heist

I’ve been talking a lot about the development of the engine recently (previously HeistCore, now Epimetheus), talking about how it can be expanded with plugins and also talking about some of the underlying technical details of how it works. However, this stuff is really all besides the point - the ultimate goal of this project is to build a stealth based, co-operative, multiplayer game about breaking into banks.

Standing On The Shoulders Of Giants a.k.a. Talent Borrows, Genius Steals

With the change over to Epimetheus using C# scripting I feel like the engine in nearly in a state when I can start developing full games with it soon. As I have always said I won’t be developing the full Heist game mode immediately but if I want to do that eventually I’d better get doing some serious research! To that end I’m compiling a list of games with similar elements to Heist which I shall play through and will publish a brief post about my thoughts on the game when I finish.

The List

Not all the games on this list are (or will be) stealth games. I’ll probably also add some games on this list which at first have nothing (obviously) in common with Heist but gave me some inspiration while playing them.

The Series (So Far)

Completed Games

Dishonored
Deus Ex: Human Revolution
Assassins Creed
System Shock
Hitman: Sniper Challenge
The Ship
Hotline Miami
Mark Of The Ninja

In Progress

Hitman: Absolution
Monaco
Hitman: Blood Money
Amnesia: The Dark Descent
Thief: Deadly Shadows
Skyrim
Frozen Synapse
Door Kickers
Grand Theft Auto
Splinter Cell: Chaos Theory
Assassins Creed: Black Flag

???

I’m actively looking for suggestions to add to this list. Any suggestions of games with co-operative and/or stealth elements are very welcome.

Scripting Is Dead Long Live Scripting

Wed, 14 Aug 2013 00:00:00 +0000

TL;DR

I’m replacing Lua scripting in the engine with C# scripting.

Scripting Is Dead

Disclaimer

I think Lua is pretty cool, it’s a very minimal and elegant language. for small tasks (configuration files, throwaway scripts, UI mods etc) Lua would probably be my choice of language. This post should most certainly not be read as a criticism purely of Lua!

Let’s Talk About Dynamic Languages

Dynamic languages are languages which defer some of the things traditionally done at compilation time to runtime. The most obvious example is deferring compile time type checking to runtime time checking, another common feature is modifying objects at runtime. Python, Javascript*, Ruby and Perl are all examples of inexplicably popular dynamically typed languages.

Python

I’ve never particularly liked dynamic languages - it always seemed pretty stupid to be to delay useful stuff like type checking until runtime. For example my experience with learning Python** (my first dynamic language) was constantly referring back to the documentation because method signatures are useless:

def __init__(self, *args):

So… how exactly do I call this again? Better refer back to the documentation because I don’t have a type system constraining me to only passing in the correct things! Naturally, if you do anything wrong you don’t know until runtime.

If you’re thinking “It’s ok, unit tests solve this” then you’re an idiot and you need a new Job - writing unit tests (which are essentially hand written compile time checks) to check types is a total waste of my time. Unbelievably there are even lint tools for python - quite why anyone would waste their time writing a heuristic compile time checker for a dynamic language is beyond me.

Javascript

I should mention Javascript, probably one of the worst languages I have ever used. Javascript is not only dynamic it’s also weak (i.e. it tries to completely pretend there’s no such thing as types). For example:

var a = "a";
var b = 1;
print(a - b);

This is a pretty stupid thing to want to do, what does a string minus an int even mean!? In C# (statically and strongly typed) you’d get an error (at compile time, which usually means about 1 second after you’ve typed it):

Operator '-' cannot be applied to operands of type 'int' and 'string'

In Python (dynamically typed, still strong) you’d get basically the same error, just deferred to runtime:

TypeError: unsupported operand type(s) for -: 'str' and 'int'

What about Javascript (delusionally typed)? It says “Sweet, a string minus an int, yeah I can do that!”:

NaN

Fuck you. This happens because Javascript will coerce things to hell and back before it admits there’s such a thing as types! Coercing “a” to a number makes no damn sense and (rather than actually failing) Javascript just says, “yeah, I can totally do that, it’s NaN”. Which of these three responses is the most useful, I wonder?

* To be fair, if you’re doing web development you don’t have much choice to use Javascript (until recently). But that’s another blog post.

** It’s been a really long time since I’ve used python and I’ve done my best to forget the eldritch horrors I encountered when learning it. Therefore my examples are stolen from here

Let’s Talk About Lua

I expect it’s clear by now that I don’t really like dynamic languages, which leads to the natural question: “Why the hell did you spend 1.5 years writing a game engine scriptable in Lua?”. My aim with the Heist engine was (and still is) that it would be very moddable - even games that I myself write for the engine are really just mods. I thought that it would be easier for a newbie/hobbyist programmer to edit a bit of Lua code than to fiddle around with the big scary C# and it’s angry compiler. The thing is the big angry compiler is seriously useful - if you make a little mistake calling one of the many engine methods it will tell you before you can even compile your mod. This is approximately 1000000x better than running a script to generate a city and hoping that your script gets selected to do whatever it does and then hoping you notice the error in the log spam.

There’s another problem with Lua - it’s very different to C#. The Heist Engine is written in C# and the plugins for it (were) written in Lua, to facilitate this I have wasted vast amounts of time writing a huge library of code (both Lua and C#) which exists purely for the purpose of translating between how the two languages want to do things. For example, here’s a script which simply adds a cube into the world:

function Subdivide(node)
    local cube = CSG.CreateCube().Translate(Vector3.New(1, 10, 20));
    node.World.Union(cube);
end

In these two lines of code the boundary between C# and Lua is crossed 6 times. Every single one of those times requires a load of complicated code to coerce types back and forth (which is not only slow and difficult to write, it’s also very slow to execute). I finally got fed up with this the other day and decided to do something about it…

Long Live Scripting

So, I’m getting rid of Lua. I’m replacing it with the only sane solution for scripting a C# game engine: C#.

The Heist engine has been abandoned. I’ve ported all the non scripting related code (most of it) to a new Engine called Epimetheus. This change gives me mods which are:

Strongly typed (more robust)
Faster to execute (faster city generation)
Less memory hungry (bigger cities)
Easier to write (more in varied cities)
Easier for me to design a scripting interface for, which means…
A more comprehensive scripting interface, which means…
More powerful mods, which means…
More game modes!
Better language support (don’t like C#? Use some other CLI language)

Everyone wins!

There’s No Such Thing As A Free Lunch

The down side to all this is that any previous timetable I had has gone out of the window. I’ve got to rewrite the entire scripting interface for C#, of course this shouldn’t be very difficult since all the code I’m trying to expose to scripts is already written in C#! Then I have to port all the old mods over to C#, which will be a little more difficult. That said, I’m certain that any time I spend now will be easily saved in the future which much more robust and rapidly developed mods both by myself and others.

The Reports Of My Death Are Greatly Exaggerated

Mon, 05 Aug 2013 00:00:00 +0000

TL;DR

I suck at keeping this blog up to date.

Where The Hell Have You Been?!

It’s been nearly 2 months since my last update, sorry about that. June is always a busy month for my family and then in July I was crunching hard to try and achieve a goal (multiplayer deathmatch) in time for a LAN party I was running. Unfortunately I didn’t even achieve that goal because I discovered a critical bug in one of my dependencies (I’m still waiting for them to get back to me on that. ~~Protip: Don’t buy the Ludosity Steamworks wrapper~~ See edit below).

What The Hell Have You Been Doing?!

Adding support for save games (not exposed in the demo as it’s not complete yet, but most of the complicated work is done).
Huge improvements to how content is managed (mods can now load content, which is pretty important!)
Added a character and overhauled some of the character controller code to make it more scriptable.
Since I discovered I can’t do multiplayer how I wanted (I’ll come back to that) I’ve been working on building a skyscraper. As I work I’ve been implementing loads of changes to make building design easier (concave footprints, 2D CSG functions, entity creation from world scripts).

Multiplayer

This is the thing that consumed a lot of my time, and unfortunately I have nothing to show for it. Valve recently gave all greenlight developers access to the Steamworks SDK (this is awesome) and so I decided to write multiplayer using steamworks (for now) to free myself of all the horrible complicated bits like NAT traversal and session initiation. I went out and purchased a .Net wrapper around steamworks and got to work. Unfortunately the wrapper seems to have a critical bug in it (it will arbitrarily crash after a garbage collection) and so all my time was wasted. I have contacted ludosity about this (twice, over 1 month ago) but they haven’t got back to me. I can only recommend that you DO NOT BUY THIS WRAPPER (See edit below). I’m considering my options with steamworks at the moment (this will likely be the subject of my next blog post).

But What About AI?

I had intended to finish the AI series with demos of the AI in game (after I got basic multiplayer working), obviously my timetable has been a little thrown off! However, I do intend to come back to the AI series soon (tm).

EDIT: Ludosity - 31st January 2014

I have been contacted by the developers of the Ludosity wrapper apologising for the state of things back when I wrote this blog post. Apparently the critical bug I encountered has been fixed and the wrapper has been shipped in a complete game (indicating it’s probably pretty stable now). I haven’t actually tried the new wrapper though, so don’t take this as an endorsement - just an indication that problems from way back probably don’t apply any more.

27 Gigawatts Of Cake

Tue, 11 Jun 2013 00:00:00 +0000

TL;DR

Goal Oriented Action Planning is the best way for a lazy game designer to build AI.

What Is Goal Oriented Action Planning?

I’ve talked about Finite State Machines (FSM) and Behaviour Trees (BT) as general techniques for deciding what action an AI should take next and I’ve talked about pathfinding as a general AI topic for how to get around the world. What happens when we put all these things together? We get Goal Oriented Action Planning (GOAP)!

With both FSMs and BTs the game designer has to carefully craft the enemy AI so it will act in the way he wants - this design stage is basically the designer laying out the sequence of actions needed to achieve some desired goal. Where else have we talked about a sequence of actions to reach a goal? Pathfinding!

Goal Oriented Action Planning uses pathfinding techniques to search for a valid sequence of actions to satisfy a goal.

So How Does That Work?

Setting up an AI for GOAP is actually pretty simple. The AI simply needs a set of actions it can take, with each action having it’s effect and it’s required conditions carefully described. The system then picks an action that satisfies one of the required conditions of the goal state and applies that, it simply keeps applying actions (picked by a heuristic function, like A* pathfinding) until all the required conditions are complete.

Let’s have an example, this is the one I used while developing the GOAP system.

Actions

Go To Some Destination
Buy Flour (Requires: Location Is Shop)
Buy Eggs (Requires: Location Is Shop)
Buy Sugar (Requires: Location Is Shop)
Bake Cake (Requires: Location Is Kitchen, Inventory Contains: Flour, 2xEgg, Sugar)

Let’s say my current state is:

In Kitchen
0xCakes In Inventory

Having no cake is a terrible situation to be in, we must rectify that! So our goal here is:

1 Cake In Inventory

There’s only 1 action I can take to satisfy this goal: Bake Cake. Let’s apply that. Now our goal is:

Inventory Contains: Flour, 2xEgg, Sugar

We’re already in the kitchen, so that’s not a goal here even though the Bake Cake action requires it. We need eggs, flour and sugar (none of which can be obtained while in the kitchen), the only applicable action here is to go to the shop, which makes our goal:

Inventory Contains: Flour, 2xEgg, Sugar.

We didn’t achieve anything here, what gives? Remember this search is just like A* Pathfinding, sometimes you have to get further away from your destination in order to get there. Now that we’re in the supermarket we can buy eggs, flour or sugar, let’s skip a few steps and do all three. This makes our goal:

… Nothing!

That’s it. We’ve not got a backwards plan for how to make cake. To execute the plan we have to do it in reverse order. An additional gotcha is with goto actions, we need to flip the goto actions around so they goto where they came from during planning, also we need to start the plan with a goto wherever the planning ended up. So the plan becomes:

Goto Shop (Implicitly inserted at start of plan)
Buy Ingredients
Goto Kitchen (Reversal of goto shop during planning)
Bake Cake

Yay, Cake!

My concern when implementing this was that GOAP might be too slow to use practically. My test GOAP implementation (which actually is a lot less efficient because it currently plans forwards not backwards, I’ll change this later) currently generates a plan for this cake scenario in about 0.3 milliseconds. Given that it might take an AI 5 minutes (or more) to execute this plan that’s pretty damn fast!

But Cake Isn’t Everything In Life

Sadly, we can’t get by in life simply by baking 3300 cakes every second. If I wanted to implement the guard AI I’ve been using in my FSM and BT examples I would need two goals:

Do Not Die
Kill Enemies

If the AI had appropriate action available to it then it could satisfy these goals by diving for cover when a grenade is in sight (satisfies goal Do Not Die), by shooting enemies when an enemy is in sight (satisfies goal Kill Enemies) and by patrolling (finding an enemy is a prerequisite to shooting them). Such a simple AI to describe (3 actions, 2 goals) would replace the 8 node Behaviour Tree and the 15+ node Finite State Machine.

The implementation of multiple goals is quite simple. Simply pick the highest priority unsatisfied goal and formulate a plan to satisfy it. If a higher priority goal becomes unsatisfied during plan execution just form a new plan and do that instead. So, for example, if a grenade rolls into sight while patrolling then the AI would interrupt the Kill Enemies goal plan with a new Do Not Die plan (and then presumably dive for cover).

Emergence

The really cool thing about GOAP is emergence. With other techniques the AI will only ever do things that the designer thought of first, so you’re effectively codifying the designer into the game. With GOAP the AI will pick the most efficient plan and do that (not always the most efficient, of course you could add some randomness to the AIs planning), this leads to the AI doing very interesting, unexpected things. For example I also have a test AI which implements the guard example I’ve been using with a few added extras, at one point I removed the Do Not Die goal (essentially made the AI fearless), it formulated a rather interesting plan:

Patrol until you find an enemy
Patrol until you find an enemy
Patrol until you find an enemy (It kept patrolling until all 20 enemies were following it and shooting at it)
Walk into the oncoming hail of bullets to the closest enemy
Punch enemy to death
Take gun from limp corpse
Proceed to perfectly headshot half the remaining enemies
Walk up to nearest dead enemy, take ammo
Reload Gun
Perfectly headshot remaining enemies

This is a pretty stupid plan (obviously), there’s no way the AI is actually going to get to execute all of that plan without dying. But it is fascinating that the AI can come up with such unexpected behaviour all by itself.

This Sounds Like Rubbish, You’re a Lazy Game Designer

Some other titles you may have heard of which use GOAP:

F.E.A.R
S.T.A.L.K.E.R
Fallout 3
Demigod
Just Cause 2
Deus Ex: Human Revolution

Among many others.

Trees Are Well Behaved

Wed, 22 May 2013 00:00:00 +0000

TL;DR

Behaviour trees are better than Finite State Machines for basic AI.

What Is A Behaviour Tree?

A Behaviour Tree (BT) is an improvement on Finite State Machines (FSM) which encapsulates the behaviours an AI undertakes in a more sensible way, making behaviours easier to design individually and then compose together later in a complete AI. Conceptually BTs are pretty simple: every time the AI is updated the tree is traversed from top down, each time execution reaches a node it does what that node says. Let’s have an example, I’m going to make the dead simple guard AI from Last Time using a BT instead of an FSM:

This is slightly less simple than the FSM example, for example what are these “Concurrent Selector” Nodes? How exactly does this get used? Let’s run through an example. Remember, every time the AI is updated, we traverse the entire tree from the top down.

Priority Selector - Traverse child nodes in priority order. In my example left most nodes are a higher priority. So we move to the left most child (the concurrent selector).
Concurrent Selector - A concurrent selector visits all of it’s children each time, and fails if any of them fail. So we move to the first child to find out if it fails.
Condition: Grenade In Sight? - There are no grenades in sight, so this node fails, which means the concurrent selector fails, and we get all the way back up to the priority selector.
Priority Selector - Traverse child nodes in priority order. The left most child is now marked as failed, so moved on to the next child.
Concurrent Selector - Again, visit children in order and fail if any of them fail.
Condition: Enemy In Sight? - Nope, no enemies in sight, so fail and fall all the way back again.
Priority Selector - Both concurrent selectors are now marked as failed, moved on to the next child in priority order.
Action: Patrol - This starts a patrol. Finally we have succeeded in doing something.

The critical thing here is to remember that the entire tree is visited every time. Now that our AI is happily patrolling along everything will work as we might expect - if a grenade appears the first concurrent selector will suddenly succeed and the “Dive For Cover” action will interrupt the “Patrol” action. Similarly, if the AI sees an enemy while patrolling it will start shooting at it.

What else can this thing do?

I mentioned a couple of “selector” nodes in my example above - these are really the language of how behaviour trees express themselves. There are plenty of other nodes:

Priority Selector: Traverses child nodes in priority order. Used for picking among multiple different actions.
Concurrent Selector: Traverse children in order and fail if any of them fail.
Condition: Checks if some Condition is true. Fails if the condition is not true.
Action: Do something.
Loop: Run all children in order, when reaching the last child start again. (e.g. Patrol Point 1, Patrol Point 2, Patrol Point 3).
Sequence: Run all their children in order. Used for executing a sequence of actions (e.g. Get closer to enemy -> punch enemy -> drag unconscious body to prison cell).
Random: Pick some random child.

There are some hidden subtleties here, for example what is the difference between a concurrent selector (visit children in order, fail if any fail) and a sequence selector (visit children in order, fail if any fail)? The different is how they handle long running actions. Some actions (most actions, in fact) will take longer to execute than 1 single update cycle and if we want our AIs to have some consistency to their actions we need to handle this. When an action node in a behaviour tree begins executing it (and all it’s ancestors) are marked as “running”, next time there is an update nodes can handle these running nodes differently. e.g.

Concurrent Selector: Traverse children in order and fail if any of them fail.
Sequence Selector: Traverse children in order (starting from the first “running” node, if there are any).

Now we can see that the difference between the two is how they handle long running sequences of actions. The long running behaviour of most of the nodes is fairly obvious, for example the “random” node always picks a running node, and only picks randomly if there are no running nodes.

So is this how the Heist AI works?

Partly yes, mostly no. Behaviour trees are a good way to design simple AIs but they’re still not a great way to design truly complex AIs which react to player actions in novel ways. I’ll be talking about that next time, when I talk about Goal Oriented Action Planning.

A Brief Project Update

Tue, 14 May 2013 00:00:00 +0000

TL;DR

I’ve been busy.

A Brief Project Update

It’s been a long time since my last update, sorry about that. This isn’t because I’ve been slacking and haven’t had anything interesting to talk about, quite the opposite in fact! So what have I been up to?

Complete User Interface Rewrite
Talking To Valve
Designing The First Game I Intend To Build With The Engine
Beginning Work On Multiplayer Streaming Of Infinite Worlds

Try It Here

First things first, if you want to try the latest version you can get it here. This version has a new main menu which allows things like managing installed mods and configuring game options, also a new HUD for deathmatch mode. Some important caveats:

The main menu is very unfinished, it has only the most basic functionality I personally needed for my testing of the game.
Construct mode is quite broken, it had a lot of custom UI code which I’ve had to disable until I make it compatible with the new UI system.
The Deathmatch mode HUD is incomplete, it doesn’t yet show equipped tools.
Press Y to open up the chat dialog and talk to yourself ;)

User Interface

Just after my last post I started on a major upgrade to the user interface system in Heist, now that’s mostly complete and the new system is significantly more efficient, easier to program and has a much better scripting interface to Lua. If you try out the demo you should instantly notice the new Main Menu which is approximately 1000 times less likely to be a cause of eye cancer than the old one it replaces. There’s also a new in game HUD, but right now that’s still work in progress.

Why did I rebuild the HUD? Well the old one always gave me nightmares - the whole thing was a minefield of picky initialisation logic, complex callback hierarchies and a total aversion to multihtreading. Also, performance sucked. Eventually my hand was forced because I upgraded to a new release of Awesomium and encountered a bug which made the old way of doing things impossible (until they fixed the bug). So I was forced to either wait until they fixed the bug, or find a new way of doing things.

Talking To Valve

Valve recently hosted a chat with all the Greenlught developers which I took part in. To be honest it was a bit of a mess, there were something like 150 developers in a single chat room all yelling out their questions and the meeting ended with half the question never getting answered. Despite that the meeting was very useful - A lot of interesting ideas were thrown back and forth and the Valve people seemed very receptive of new ideas, as well as saying that they were already working on some of the things suggested. For example, I asked about getting access to the Steamworks SDK before getting Greenlighted (e.g. as soon as you become a Greenlight developer) - right now the top thread on the Greenlight developers discussion forum is one created by a Valve person asking about what we’d want early access to Steamworks for and exactly how we’d hope to use it. Overall I’m pretty excited about some of the stuff that came out of this discussion - no one is denying that Greenlight has some major problems right now but everyone has lots of ideas for how to fix it and the people in charge are receptive to these ideas. If you’re interested in a summary of the chat there’s one here or another one here.

Designing The First Game

As I’ve mentioned before my project comes in two bits:

1) The HeistCore engine, which is just a game engine designed to handle huge, procedurally generated, multiplayer environments with lots of scripting support. 2) Multiple games built on the engine.

The “Heist” game (Co-op breaking into banks) was never intended to be the first game - that’s an incredibly complicated game which is going to need a lot of complicated scripting for alarms, complicated scripting for generating banks, complicated programming for realistic AI etc etc. I’ve thrashed out a different design which I’m now working towards and which I’m not going to tell you anything about (yet).

Infinite Streaming

This is really the last core component of the engine which has been missing for all this time. The game currently streams the geometry of the world so it supports very large worlds, but when it unloads part of the world it simply ignores all the entities in that section and they fall into the infinite void below the world. My current work is making it correctly find all the entities in the unloaded section, save their state (including the state of all the scripts) and then remove them from the world. Obviously when a section gets reloaded the game has to go off to it’s directory of saved data and reload all the entities in the section, including correctly reinitializing their state.

There’s a nice side effect to all this. Once I have a system that can unload chunks the player is not in, and then reload them when the player comes back I instantly get save games for free. When the player quits the game I can simply consider all chunks in need of unloading. When the player reloads the games I simply need to load in the chunks around the player as normal. Naturally, multiplayer save games will also be supported (how many FPS games can claim that?).

What’s The Plan?

I think it should take about 2 weeks to get the work on streaming worlds done (including saving and reloading games). After that I’m going to spend some time getting Deathmatch mode up to scratch, with the world streaming in and out as players move about and the entire game state able to be saved and reloaded so that an interrupted match can be continued at a later date.

Finite State Machines (are Boring)

Tue, 16 Apr 2013 00:00:00 +0000

TL;DR

Finite State Machines are the most boring way to do game AI.

What Is A Finite State Machine?

Finite State Machines (FSM) are a very simple concept to understand. You simply work out all the states that your system can be in, and then for every single state work out every single action you can take from that state (and in what circumstance you would take it). This is, I feel, best explained by XKCD. Here’s an example of an extremely basic game AI for a game:

Pretty basic stuff I hope. The AI will patrol around until it finds an enemy. Once it has located the enemy it will shoot it until they are dead, and then it will… have a panic attack because there is no way to get back to the patrol state! The problem is that this AI is really boring – playing against this would get very predictable very fast. You really need a few more states to account for various interesting events. For example:

Oh God, what happened here? This is a slightly more interesting AI, but even this one is missing some pretty vital parts. E.g. If you throw a grenade at an AI making coffee he will totally ignore it – you need to add more connections from basically every node to dive for cover if a grenade is sighted. If the enemy is no longer in sight after dropping snacks the AI will be confused, so you need to add a connection there to “search cautiously”. Once again, there’s no way back from combat to patrolling so we need another connection there. If we wanted to add in little things like AIs meeting each other mid patrol and exchanging some small talk we’d need a whole load of states to handle seeing each other, stopping, talking and all the extra connections for what happens if they see a grenade/enemy while doing all this. Now imagine what would need to be done to add another weapon type into the game (e.g. a flash bang grenade), or a special enemy type (e.g. one who can only be taken down by specialised weaponry).

What I’m trying to get at here is that FSMs might seem like a good way to create AIs (because the game designer retains complete control), but actually they’re terrible because the designer retains complete control of every action the AI will ever make which is loads more control than is really practical.

Why Would You Tell Me About Such A Dreadful Way To Design AI?

Heist doesn’t use any FSMs for AI (it does have a rather complex FSM for network session negotiation, but that’s something we want to be boring and predictable), so why would I bother telling you about this approach? Finite State Machines were the way game AIs were written almost exclusively up until 2000 (as far as I can tell, I’m not exactly a history student). Even today, they are still an important part of game AI. I’ve mostly talked about FSMs because some of the systems I will be discussing were invented almost entirely to solve the problems with them, and discussing these systems without first discussing the basics of FSMs would be very difficult.

Coming Soon

I’m going to discuss Goal Oriented Action Planning and Behaviour Trees in the next couple of entries. These are techniques which improve upon the limitations of Finite State Machines.

Pathfinding

Wed, 10 Apr 2013 00:00:00 +0000

TL;DR

A* pathfinding is pretty conceptually simple, but can be quite complex to implement.

What Is Pathfinding?

Pathfinding is really quite a descriptive name - How does someone get from A to B? Obviously they need to find a path! Of course this is a bit of an oversimplification - you can’t simply dump an AI character into a game and say “Go to this point”, you need to give them some understanding of the shape of the world (e.g. you can’t walk through walls). In games the world is usually represented with a “navigation mesh”, for example:

The grey areas are convex polygons which make up the navigation mesh, AI’s “know” that they can only walk over areas the mesh covers. Hence, an AI getting from A to B would have to follow something like the red line. Let’s simplify the problem a little bit by removing all the geometric aspects of it. This would look a little like this:

Again, the grey blobs are representations of where the AI can walk and the red arrows indicate the solution to the question of how to get from A (contained within “Top Room”) to B (contained within “Left Room”). Formally, the grey blobs are “vertices”, the black arrows are “edges” and a load of vertices and edges together form a “graph”. Pathfinding, in this abstract representation, is simply the question of which edges should be followed to get from one vertex to another.

Brute Force

The obvious way to find a path is with brute force, let’s see how that would work.

Mark current vertex as visited
Add all connected vertices (which are not marked as visited) to a “to do” list
Set current vertex to be a vertex off the “to do” list
If current vertex == destination, we’re done! Exit
Goto 1

Basically, keep exploring every single possible path until you find the destination. I think it should be pretty obvious why this is a dreadful way to solve the problem, to solve even a relatively small problem you’ll end up visiting very large parts of the graph.

Best First

The problem with the previous example was it considered every possibility equal, so it would explore every single possibility on the way to the destination with equal priority. Think about this from a more human perspective - if you were tracing a route on a map you wouldn’t trace every single little side road on the way, instead you’d take the largest roads until you were nearly at the destination and then if that didn’t get you all the way there start backtracking and trying smaller roads. Essentially, a human would try the best possibility first and then backtrack if this fails. This is the essence of A* search - the search will check out the nodes cloest the destination first:

Mark the current vertex as visited
Add all connected vertices (which are not marked as visited) to a “to do” list
Sort the “to do” list by (Distance so far + guess of distance remaining)
Set current vertex to the vertex which is guessed to e closest to the destination
If current vertex == destination, we’re done! Exit
Goto 1

The only difference is that, when picking which vertex to check next, A* picks the one which is probably (according to some guesstimate of distance) closest to the destination. Interestingly it turns out that so long as the guesstimate of remaining distance is always less-than-ore-equal-to the actual distance you’re guaranteed to get the optimal path (even though you didn’t check every possibility). Here’s a handy fact: The distance “as the crow flies” between two points is always less than or equal to the path between those points. So if you’re doing pathfinding in 3D space you can just use A* - with the euclidean distance as your guesstimate - and you’re done.

Fiddly Details

As with any efficient implementation of an algorithm there are fiddly little details that make A* harder to implemented than you might expect. Let’s think about the things that algorithm I described above does:

Mark vertices as visited
Find the vertex on the to do list with the smallest guessed distance
Guess the distance from A to B

Mark Vertices As Visited

The obvious way to do this is for vertices to have a field “visited” which you simply set to true. If you do this then checking if a vertex is visited is a dead simple boolean conditional (can’t get much more optimised than that). The problem is now every pathfind is modifying your vertices, when you complete your pathfind you have to go back over all the vertices and mark them as unvisited and running multiple pathfinds in multiple threads becomes rather difficult. A better approach is to keep a set of visited vertices - checking if a vertex is visited just involves checking if it’s in that set, for a hashset this is an O(1) operation - i.e. pretty cheap.

Pretty cheap wasn’t quite cheap enough for me. Instead of a hashset I use a Bloom Filter which is faster than a hashset (and also uses significantly less memory) but has the possibility of false positives, i.e. sometimes when checking if a vertex is visited a bloom filter will decide it has been visited even though it hasn’t really. If that happens the pathfinder might generate different (less optimal) route. I decided this wasn’t really much of a problem, the chance of false positives is very low and so it will only happen very occasionally, and since we’re trying to simulate human behaviour we don’t want the paths to be perfectly optimal all the time! The one problem with bloom filters than had me worried was that as the number of items in the set grows so does the probability of a false positive - meaning that for a very long pathfind the chance of multiple false positives could grow to a near certainty! To counter this I actually use a Scalable Bloom Filter, which maintains a certain error rate no matter how many items are in it.

Find The Best Vertex

The obvious way to do this is to keep the todo list as a literal list and then simply sort it as items are added. This is a very expensive way to do this because a sort operation sorts the entire list, but all you want to know is the smallest item. A better way to get the smallest item in a set is to use a Heap. A heap allows you to add an item and to remove the smallest item added so far. It does this far more efficiently than sorting the entire thing every time.

Estimate The Distance Remaining

As I mentioned above, the distance estimate must be less than or equal to the actual distance remaining - this means that directly measuring the distance as the crow flies is a good guess for A*. Of course doing so is simple high school maths:

Distance(A, B) = Square Root((A.x - B.x)² + (A.x - B.y)²)

That’s that sorted then. Simple!

Naturally my implementation isn’t really that simple. Although for a perfectly optimal path the heurustic must always be less than or equal to the actual length if the heuristic slightly violates this and overestimates the pathfinder can sometimes find a (less optimal) path slightly more quickly. Wikipedia calls this Weighted A*. The essence is that if, in the worst case, your heuristic is N times as long as the actual distance then, in the worst case, the pathfinder will find a path N times as long as the best path but it will do so more quickly than if you’d forced it to find the best path. I threw this into my implementation as soon as I found out about it, it’s dead simple to implement (just multiply by a number) and if you decide you need an optimal path then you can just use 1 as the multipler!

Path Following

The elephant in the room is that this A* simply tells us which edges in our abstract graph to follow… but what then? How to actually follow the path is a separate problem involving steering behaviours and tying into the animation system to move feet properly (perhaps with inverse kinematics). This is a very different problem which I shall come back to soon^tm.

Artificial Stupidity Series

Mon, 08 Apr 2013 00:00:00 +0000

TL;DR

I’m been hard at work on Artificial Intelligence.

Where Have You Been!?

It’s been a while since my last blog post, sorry about that. I was aiming to get a page up on Steam Greenlight at the end of last month - I decided that I would keep my head down, work extra super hard and then come back with an announcement of my Greenlight page. That didn’t happen because the UI system that Heist uses released a new version which I upgraded to and instantly discovered a major bug in. Rather than wasting my time working around the bug I’ve been working on other stuff until the bug is fixed (soon, according to their support). I have to admit I think this whole thing has worked out rather well - the UI work was really dragging me down and the change to something else has been very refreshing.

Something Else

As you may have guessed from the title, the something else I have been working on is Artificial Intelligence. I first played around with this topic ages ago but I didn’t make any great progress on the AI itself - I was improving the various supporting systems required for AI when RSI put me out of commission for a month. By the time I got back into action I decided to work on other, simpler, tasks to catch up to my timetable a little.

As seems to be becoming tradition on this blog every time I start a significant new phase of development I’m going to do a series of blog posts on Artificial Intelligence for games. Here are all the posts in the series (this will update itself as more are added, so keep checking):

Valve Need To Stop Reading My Mind (and Other Miscellanea)

Wed, 20 Mar 2013 00:00:00 +0000

TL;DR

Valve keep doing stuff which makes my life easier.

Paid Alphas Are Awesome

A couple of months back I was musing on the best way to release Heist. My final conclusion was that Steam is, overwhelmingly, the place to be. However, I have to admit I had some reservations - it wasn’t clear at the time how Valve would handle releasing games before they were finished (i.e. paid alpha). Having a paid alpha is incredibly important to me (and almost every other indie developer) as it means I can get some money sooner to fund development, plus, even more importantly, get critical feedback from players. Heist depends upon this even more than your usual indie game since it is highly moddable - a period of paid alpha is likely to attract the more dedicated gamers who will mod the game and make it even better!

Well, continuing to be exactly the publisher I want them to be, Valve have gone right ahead and solved this problem for me. Today they announced Early Access Games - 12 games (with more to come) which can be purchased right now and accessed as they are developed. This is exactly what I needed - now I know that I can aim to get the game through Greenlight as soon as possible and release it onto steam as a paid Alpha.

Other Miscellanea

I haven’t made any posts for a really long time - this is because I have been insanely busy. I’ve been aiming to get a concept page onto Steam Greenlight by the end of March, which obviously means I need to get the game into some kind of presentable state and then I need to make videos, take pictures and write descriptions of the game.

My main focus for the past month has been the UI for a building designer built into the game. The idea is that a player can start up the building designer, lay out floors on an architect style plan and then export these out to be used in city generation. The technical back-end of this is actually done (with minimal modifications to the engine, powerful modding system FTW), but building the actual user interface has required me to properly learn Javascript and do loads of work on making the system smooth to use. Of course, the cool thing about the whole system is that since the game UI is just a web page rendered on top of the game I can post a demo here really easily, which I’ll do soon (tm).

Elements Of Style

Tue, 26 Feb 2013 00:00:00 +0000

TL;DR

My attempt at analysing what aspects of games I like and dislike, and what design lessons can be learnt from this.

Before We Start

I’ll make references to the Bartle test of gamer psychology several times in this discussion. This is a pretty simple classification of gamers into four categories:

These categories seem fairly self explanatory, but if you’re not quite sure what one means refer back to the wikipedia article that has a nice summary with examples for all four. I also recommend you do the test at gamerDNA.com yourself.

Disclaimer

Naturally the opinions below are just that, opinions. Some of them I think are fairly well justified that most people would agree with (e.g. Quick Time Events Suck) but many of the attributes I’ve selected as good things are largely just things I prefer in a game. Obviously if your results are significantly different you are likely to find yourself disagreeing on a few of these points.

My own Bartle result:

73% Explorer
47% Socializer
47% Killer
33% Achiever

With the disclaimers over, let’s get onto some real content. The points below are in no particular order and each point is prepended with the name of the game I’m using as an example for this particular point, if you haven’t played the example game it doesn’t really matter.

The Good

Rift - Custom Class System

Rift uses a strange skills tree system. Each class allows access to 6 different “souls”, any three of which may be equipped at one time, each soul opens up an entirely new skill tree. This essentially means that for each class there are 120 different sub classes! This system is great, it means that there are many different ways of pulling off each role in the game and players can come up with “custom” classes that suit their method of play. For example I played the rogue class with a soul that unlocks various teleport based dodge skills, a soul which unlocks various tanking skills (including “generate threat every time you teleport”) and a soul which gave me even more teleport based attack skills - suddenly I have a tanking character, built on a rogue class, which tanks by teleporting around the enemy and never getting hit. That class is nothing like how any normal tank classes work, which is what makes it cool, it’s my very own tank class.

I think this kind of thing is great. Players get oodles of choice, and get to build customised classes for exactly how they want to play, which keeps the game interesting for longer.

Guns Of Icarus - Teamwork

TO DO

Hitman/Dishonored/Deus Ex - Stealth

TO DO

Dystopia/Hawken - Large Time To Kill

TO DO

Threadspace:Hyperbol - Sidegrades

Hah! I beat you because I’ve spent more time playing/microtransactions to buy a superior weapon. Isn’t that super fun? Yeah… straight upgrades suck.

I’ve never seen sidegrades done as well as in Threadspace:Hyperbol, every upgrade module gave is an exact sidegrade (i.e. exactly exchange N of attribute 1 for N of attribute 2), this leads to players becoming super specialised as people minmax their attributes to match their playstyle. Critically, there’s no one-true-build that people have to follow for certain roles, all changes are purely to make your attributes enhance your personal playstyle. This kind of thing is great, enhancing individual playstyles really increases the variety of things you see in a match, which keeps the game fresh and fun.

The Bad

Rift/Tera/Guild Wars 2/Knights Of The Old Republic - 5 Man MMO

MMOs are cool because they give you an entire universe to play with - thousands of other people who you’re playing with - it’s a unique experience in games when you know something truly huge, involving thousands of players, is happening. So why is it that most MMORPGs are games for 1-50 people? It seems to me to be entirely backward to go to all the effort of building all the complex infrastructure required to host a server with 10,000 people on it and then to only provide content which ~50 people can be involved in (usually even less people). Even ignoring the waste of effort on the part of the developers MMOs come with certain downsides for the player: overcrowded newbie zones, high levels gankers, lag, subscription fees - why put all these bad things into the game if you don’t really intend to exploit the advantages of an MMO? True MMOs focus on large groups of players working together, emergent gameplay and possibly market economy effects - think Planetside or Eve Online.

Rifts in rift were pretty good GW2 allows lots of people to complete certain missions together

Rift/Tera/Guild Wars 2/Knights Of The Old Republic - Filler Content

TO DO

Fetch Quest Kill 10 Rats Escort Quest Kill 1 Big Bucket Of HP Singlehandedly Wipe Out That Cult Fetch Power Crystals To Improve Your Weapon

Rift/Tera/Guild Wars 2/Knights Of The Old Republic - Crafting Systems

TO DO

Crafting systems are fine Level grinding gathering skills, then crafting skills is not.

ArmA/Day-Z - More Realistic == More Fun!

This makes no sense to me. The real world is a pretty boring place, why would you want to simulate it? Of course the natural suggestion is to have a perfect simulation of everything in the entire world and then you can carry out unreal situations (e.g. zombie outbreak) and see how it would “really” happen. We all know that’s bullshit though, the game isn’t going to be a perfectly realistic simulation. Also players will do things differently to how real people would, because most of the things we do in the real world are really boring. E.g. A realistic zombie outbreak would involve:

7 hours staring at your character sleeping (each person taking shifts of 1 hour to stare idly out of the door window)
1 hour looking for flammable material to build a campfire
3 hours moving across the city at snails pace trying not to be noticed
30 seconds being bitten in the neck by a zombie that jumps out of nowhere

Don’t get me wrong here, I really love super hard games, but making things more “realistic” (and I use the word realistic with some pretty huge reservations) is just lazy game design. There are certain kinds of hard which are fun to deal with and certain kinds of hard which are not - going for realistic is just like saying “Oh we can’t really be bothered to work out what the fun+difficulty+realistic things are, so we’ll just program whatever “realistic” things come to mind and hope it’s fun”.

Battlefield 3 (single player) - Quick Time Events

PRESS ぷ TO CONTINUE READING THIS ARTICLE. OH NO TOO LATE YOU FAILED.

Does anyone actually need me to convince them that quick time events suck? This doesn’t even qualify as lazy game design, at least with the realism laziness it’s the designer thinking “we’ll stumble upon something fun (because it’s difficult) while implementing lots of stuff that’s realistic (and therefore difficult)”. I don’t know what the designer of a quick time event could be thinking, perhaps “players like pressing keys quickly, so we’ll require them to press some arbitrary keys to not fail, yeah that’ll be cool”.

Even worse than quick time events just being really pointless is that they’re immersion breaking, you’re watching a cutscene which is communicating to you the important story which drives the game and makes you care about the characters and then QUICK SUDDENLY PRESS E RIGHT NOW TO STOP YOURSELF FALLING OFF THE TRAIN. If you fail then you will have to rewatch a load of cutscene (which has now gone from important storyline to annoying movie you just wants to skip through to get back to the QTE you failed), if you succeed then you won’t be paying close attention to the rest of the cutscene because you’ll be too busy watching for more arbitrary QTEs.

A personal gripe I have with QTEs (on top of all the previous things) is that the key(s) they ask you to press is usually meaningless. You’re watching yourself falling off a train and suddenly it asks you to press E. Wait… what? We have established mechanics for climbing up things in games, you press W, not E.

Moving To Axmingholmesbury

Sat, 23 Feb 2013 00:00:00 +0000

TL;DR

Markov chains are pretty good at generating likely sounding names.

Where The Hell Is Axmingholmesbury?

Axmingholmesbury is a small English town near to Consfield, Ramsham and Kilbury in the county of beenhamshire. Don’t believe me? I don’t blame you, I just made all that up! These names are all the result of an experimental city name generator I’ve been building for Heist – eventually I want the game to automatically generate a name for a city which is believable but isn’t any real place. I mentioned in my article How Does Procedural Generation Work? that markov chains are a common way of generating names, and that’s exactly what I’m using here.

Markov chains aren’t just useful for generating names – they’re good at generating any kind of sequence of symbols which conforms to some unspoken or intuitive set of rules. For example, in this case I would find it extremely difficult to come up with a specific set of rules for quantifying if a word sounds like an English town name or not – the rules for that are intuitive. Instead of requiring a set of rules, a markov chain works from a set of examples which it then infers rules from. For my city name generator I used this dataset from freebase (semantic web ftw, but that’s another blog post).

How Is Rule Inference Formed?

The analysis of the example set is actually remarkably simple. Each example is split up into individual tokens (i.e. letters) and then the probability of any letter following any other letter is calculated from that. To generate a new item, simply randomly pick tokens based on the probability calculated.

Example set:

Mart
Matt
Mike

Analysis:

(_start_) -> m (100%)
m -> a (66%)
m -> i (33%)
a -> r (50%)
a -> t (50%)
i -> k (100%)
r -> t (100%)
t -> t (50%)
t -> (_end_) (50%)
k -> e (100%)
e -> (_end_) (100%)

Pretty simple, I hope.

Results:

m (must be the start letter, it's the only start letter this system knows about)
a (picked randomly, 66% probability)
t (picked randomly, 50% probability)
(_end_) (picked randomly, 50% probability)

Given such an incredibly limited set of examples Mat is not a bad example name to generate.

This was an order 1 markov chain, and that means it only analyses which single letter follows what other single letter. It’s probably not a surprise that this can generate some very odd words – there are certain sets of two or three letters that should commonly follow other sets of two or three letters in English. This is a simple problem to fix – just analyse what letter often follows what short string of other letters.

Continuing the example above with an order 2 analysis, we would get:

(_start_) -> ma (66%)
(_start_) -> mi (33%)
ma -> r (50%)
ma -> t (50%)
mi -> k (100%)
ar -> t (100%)
at -> t (100%)
ik -> e (100%)
rt -> (_end_) (100%)
tt -> (_end_) (100%)
ke -> (_end_) (100%)

With an order two analysis you get less weird names being generated but that is at the cost of the “inventiveness” of the system: a higher order analysis will lead to less results.

City Names

As I mentioned before for this experiment, I used this dataset from freebase, and I also used an order 3 analysis of the text, which I picked, basically, through trial and error. Before the analysis I removed any examples which were not just single words – for example Greater Something, Something upon Somewhere and Little Borough of Something In Somewhere By Somewhere Else all went. Finally, after the generation phase I removed any suggestions shorter than five letters (ll is not a very good city name and neither is cock, both of which were generated by the system). I also removed any names which were in the example file because generating a real city name is boring.

Finally, here’s what you’ve been all waiting for – a massive list of city names!

What Else Can I Name?

There are a fair number of places I can use this within Heist, just off the top of my head:

City names
Place names (within cities)
Building names
NPC names
Street Names
Layout of buildings along streets
Texture layout
Furniture layout
Room layout along a corridor
Floor layout vertically up a building
Item/weapon/equipment names (á la Borderlands)

Given how successful this city name generation experiment has been, I shall definitely be using markov chains a lot more in the future.

The Future Is Steamy

Fri, 08 Feb 2013 00:00:00 +0000

TL;DR

Valve are destined to take over the world.

Gabe Newell Is Pretty Cool

I’ve recently seen three videos of Gabe Newell discussing his opinions on things, and hinting at the future direction of Steam, valve and the world economy. They’re a pretty fascinating insight into the future of Steam and they’re very exciting to me as both a gamer and a game developer.

If you’re only going to watch one of the videos then watch the D.I.C.E. Keynote. It’s the shortest of the three but still manages to cover all the important points.

I took some notes while watching the videos and decided to summarise them with my opinions (spoiler: I agree with Gabe on nearly everything).

PCs Will Take Over The Living Room

Cheaper than console, because you already have a PC!
PCs are upgradeable, which makes it cheaper to keep pace with technology
Demands of mobile have taken hardware capabilities far beyond what living room needs
PCs can easily be small, quiet, fast and cheap.
A new game console means you need to re-buy everything like controllers and games
New console generations come very rarely and don’t keep up with technology
A new PC keeps your games, controllers, friends lists, save game progress
Apple have the best path into the living room
Smoother upgrade cycle
Lots of existing brand awareness

Personally I don’t agree that Apple have the best path into the living room. Anything Apple makes for the living room would undoubtedly be a very expensive closed platform, which I think would put it on par with games consoles (very expensive, closed platforms).

Games are productivity platforms for gamers creating digital goods and services for other gamers

Community produces 10x more content for TF2 than valve do
Best way for Valve and the community to make money is for Valve to enable community to make even more/better content
Ownership of content should be tracked through the entire system
Someone makes a texture, someone then uses that texture to make a model, and someone else uses the model to make a level.
If someone buys the level, then everyone in the chain should get some reward for the value they created
Enables ad-hoc collaboration
A single game shouldn’t be a closed economy, work put into getting, for example, leather in WoW should be exchangeable for something in another game, e.g. DoTA
Essentially, every game becomes an instanced dungeon of a global game economy
Obvious Example, Steam Trade
Reviews, Replays, Commentary, are all content which valve wants to provide a way to reward
Na’Vi (famous gaming team) provide a service (streaming their games, entertainment), so they should be able to earn money from it
Valve need to provide a framework for people like Na’Vi to earn money from being famous players of DoTA
Na’Vi earn money, and keep on doing what they do
Valve take a cut
Other gamers enjoy watching the show
Everyone benefits!

Engaging with the community is something I consider incredibly important for Heist, because I also think that the community can produce more, better and more varied content than I can. If Valve create a system enabling (and incentivising) content creation for all games, then that will be truly incredibly for gamers (more variety), modders (more money) and game developers (a modded game stays interesting longer).

Valve Are The Bottleneck To Getting Something On Steam

Closed platforms are bad
Create artificial costs to entering the market
Reduces innovation
Less innovation is bad for everyone
Valve can only put so many games onto steam each day, inadvertently creating limited shelf space
Right way to solve the problem is for steam to become a network API for developers to distribute their content, anyone can publish anything on steam, no bottleneck!
A store for games or content should be user generated
Indie devs create a store for their game
Modders create a store for their mods
Reviewers can create a store of good games
This creates more value for Valve (more games == more sales == more money)
This creates value for the players (more games == more variety == more fun)
This creates value for game developers (easier to get a game on steam)
People contributing to the general gaming community (e.g. reviewers) can make money by setting up a store for games they reviewed as good. This rewards the work they do improving the gaming community with reviews

This all sounds… awesome. A few posts back I discussed the different approaches to releasing Heist, I concluded that Steam was the best choice (despite being hard to get onto) because of all the infrastructure (auto updates, massive bandwidth, user statistics, matchmaking etc etc) they provide. If all the Steam infrastructure became available to any indie developer that wanted it that would be incredible.

Greenlight Can Be Done Away With

Greenlight is a bad example of how to do it
Greenlight can eventually be done away with in favour of the more open steam platform already talked about

This makes sense. It now becomes obvious that Greenlight is just a transitional technology while Valve shift their entire distribution system over to user driven stores.

Summary

Although it looks like a new and strange way of doing business Valve are really just taking the next logical step. Over the years games have been increasingly about releasing early and getting lots of community interaction, then constantly updating the game with free or paid DLC. Indies in particular seem to always release their games incomplete (or at least as early as possible) and then develop it onwards in the direction the community likes the most. What Valve are doing looks to be the next step, actually allowing the community to directly interact with and modify the game (and possibly being paid by the rest of the community to do so). This is going to be a very exciting time for gamers, game developers and modders. It’s probably going to be a less exciting time for traditional game publishers.

Mathematical Trickery

Tue, 22 Jan 2013 00:00:00 +0000

TL;DR

Physics guns are probably more complex than you think.

What Is A Physics Gun?

A Physics Gun is just a device to manipulate physical objects in the world. It simply lets the person wielding the gun grab an object and then move, rotate or scale it. Obviously, this is pretty important for the construct mode which is basically an entire gamemode about moving, rotating and scaling stuff!

I’ve spent the last 4 working days building my physics gun – which was a lot longer than expected. This is mostly because I spent 2 days messing around with dot products and cross products and all sorts of nasty geometric mathematics, which got me nowhere useful, before I finally thought of a very cool trick.

Attempt 1: Physics Guns Move Stuff

Ok, so attempt one was based on the (fairly critical) assumption that a physics gun is all about moving stuff. The scripting interface for tools supplies values indicating where the tool is, and which direction it is facing. So all I had to do was move the grabbed object around by however the direction and position values changed – Simple! This turned out to be something of a train wreck:

local entityToolDirection = GetEntity().GetProperty("tool_direction");
local entityToolPosition = GetEntity().GetProperty("tool_position");

function UpdateTargetPosition()
    local relativeGrabDirection = (targetPosition - entityToolPosition.Value).Normalize();  --direction from tool to target grab point
    local requiredDirection = Vector3.New(entityToolDirection.Value).Normalize();           --Vector pointing from tool to where we want target to be
    
    --Now rotate target by angle between tool look direction and grab point
    local axis = relativeGrabDirection.Cross(requiredDirection);
    local angle = math.acos(relativeGrabDirection.Dot(requiredDirection));
    local rotation = Quaternion.CreateFromAxisAngle(axis, angle);
    
    --Apply rotation
    local body = GetBodyFromEntity(target);
    local transform = Matrix4x4.CreateTranslation(-entityToolPosition.Value) * Matrix4x4.CreateFromQuaternion(rotation) * Matrix4x4.CreateTranslation(entityToolPosition.Value);
    body.SetWorldTransform(Matrix4x4.New(body.GetWorldTransform()) * transform);
end

Well… that was painful. Believe it or not, although this is actually a simplified version of the mess I ended up with, it still doesn’t properly handle grabbing objects anywhere other than their dead center, and it doesn’t really handle translation at all (it tries to but it doesn’t work).

So how is this meant to work? Let’s pretend that all we want to handle is rotation – so that means all we need is to determine an axis (to rotate around) and an angle (the amount to rotate). Taking the cross product of the vector to the grabbed object and the tool look direction gets us an axis, then the dot product of the same two gets us an angle. These things can then be used to create a quaternion (a clever mathematical thing which describes rotations) which we apply to the body to rotate it relative to the physics gun.

This is, to say the least, a little bit ugly. There are several special cases I skipped over in the example above, and it doesn’t even do the entire job! Here’s a video of the gun not working.

Attempt 2: Physics Guns Stop Stuff Moving

Wait, what? That title makes no sense! Physics guns are all about moving stuff, aren’t they? Another way to think of a physics gun is that it makes stuff not move relative to the gun. As the gun moves, the grabbed object will stay in exactly the same position relative to the gun no matter how you wave the gun around. It seems pedantic to restate it like that, but it’s actually very important because there are established ways to make one thing move relative to another which are nice and simple to use. Let’s have a look (this is the complete thing, handling rotation and movement):

local inverse = Matrix4x4.Invert(previousToolTransform);    --Invert frame of reference from last time
local transform = CalculateToolTransform();                 --Calculate the frame of reference defined by the tool
previousToolTransform = transform;

local body = GetBodyFromEntity(target);
body.SetWorldTransform(body.GetWorldTransform() * inverse * transform);

This is conceptually quite simple – all we do is transform the tool into the “frame of reference” defined by the tool. Before we do that though, we have to translate it out of the frame of reference from the last time – this is what the inverse does.

Why Do I Care?

I spent several days working this out, and it was a non obvious trick to get something which seems almost trivially simple to implement. Also, when I inevitably fall behind schedule I can point to this as an excuse ;)

Latest Release

There’s another release out in the usual place. When you create the game, you should select construct mode and then you can create shapes, move them about the world and mess around with constructive solid geometry to create stuff.

Coming Soon

I’m moving into mixing construct mode in with the city generation now. The plan is to have this done in a month or two. If everything goes to plan then in March I’ll be releasing a playable construct mode. I’ll also have a much more impressive city (since I will use my own tools to add lots more detail) being prepared, ready for a Greenlight release.

Thinking Aloud About Release

Wed, 09 Jan 2013 00:00:00 +0000

TL;DR

Greenlight seems like the way to go.

A Quick Recap

Last week I mentioned that my current mid term goal is to get some kind of release out for Heist within a few months. I considered my options last week to be:

Working towards a full Steam release (1+ Year(s))
Working towards a Greenlight release (within 6 months)
Putting myself on Greenlight as a preview/concept (now?)
Hosting my own paid alpha (now?)
Kickstarter (???)

This is obviously a really big decision and I’m going to use this blog post as a place to think out loud about the relative advantages and disadvantages of the different options.

Steam

Publishing on Steam has some huge advantages. The most obvious is that Steam is ridiculously popular - simply getting onto Steam gets you exposure to ~10,000,000 gamers! Additionally, Valve are masters at selling games – getting their advice on pricing and marketing schemes is worth paying for. If you publish on Steam you also get access to the Steamworks SDK which looks like it would save me from implementing a whole load of backend infrastructure myself. Here are just a few things Steamworks does for you:

Multiplayer matchmaking
Multiplayer session initiation (NAT holepunching!
Leaderboards
Valve anti cheat
Cloud save files
Steam workshop for mods (huge feature for a game designed for modding like Heist)
Voice chat
In game DLC/micro transactions

Every single one of these things could take (optimistically) 2 to 4 weeks for me to implement, and I likely wouldn’t do it as well as Valve. This means hosting on Steam and using Steamworks instantly saves me about 6 months work building infrastructure.

Of course, Valve take a cut of my sales through Steam. However, I think saving myself at least sixth months work as well as getting onto the most popular games distribution platform in the world is worth paying for.

Full Steam Release

A release on Steam requires the game to be in some state I can call “complete” with a straight face, and I estimate it would take me nearly a year to get Heist to this point. This is longer than I would like before getting at least some kind of release out. Practically, this means that if I go for a full Steam release I would probably also need to self host a paid alpha, which is a lot of extra effort. On top of this Valve are obviously moving Steam away from a closed submission system and more towards Greenlight.

I don’t really see this as a viable option any more. It would take too long, require a lot of extra effort and may not even be a viable option given the way Valve could decide to change Steam over the next year.

Greenlight

Greenlight is the new, community curated, system for getting a game onto Steam – It’s obviously the route meant for indies like myself. Critically, getting a game accepted onto Steam still gets you access to the Steamworks SDK, so I still have that massive advantage. Greenlight has two different sections: concept and full game. The concept section of Greenlight is for (obviously) semi complete game concepts. It says:

If you have an early game or software concept that you’d like to get in front of potential customers, you can post them here [in the concept section] to get feedback and reaction. Posting in the concept section allows you to create a page that supports the same features as normal Greenlight items such as discussions and comments. The voting here serves only to give the developer data and reactions and doesn’t work toward getting the game distributed on Steam.

So I can get a concept page up as soon as I feel I have something (anything) worth showing.

After concepts a game can be moved into being a full Greenlight submission at any time. Full submissions are for when the game is (nearly) ready for a release on Steam. I’ve skimmed Greenlight and found a few projects which have been greenlit which are promising alpha/beta/full release dates, so it looks like I could put Heist on Greenlight fairly early in development (e.g. as soon as I have one single completed gamemode), and promise many more updates once the game was released.

Self Hosted

Self hosting is what I am currently doing, with a self updating version of the game available for free from here. However, if I really want to self host a proper paid release of Heist I would need (at the very least):

A Website
A Store (with proper setup for payment providers)
An account system
Better modding support
Better multiplayer setup (NAT, matchmaking)
Discussion forums

I have no real experience setting up infrastructure stuff like this and I think it could easily take me a couple of months to get just these things setup. Given that Steam provides every single one of these things (although I would probably want to have a separate website still), I can’t consider self hosting a realistic alternative.

Kickstarter

Kickstarter is not exactly an alternative to the above – if I ran a Kickstarter campaign I’d still need to select a release system. A few people have mentioned Kickstarter to me though, so I thought I should consider it out loud. Realistically, the only way a load of funding from Kickstarter could help is if I hired another developer to work on Heist with me. This would require about $30,000 (software developer wages for 1 year). This is, of course, assuming that while I hire this other person I still get nothing myself. Hiring another developer would require me to setup a company, keep proper accounts, pay an accountant, hunt for a developer, interview developers, bring them up to speed on how Heist works and then manage them. Realistically, this could take a couple of months to get right and would land a lot of extra work (but no extra money) on me. Kickstarter is still an option I’m considering, but right now I don’t feel it’s the right fit for Heist.

Conclusion

I think it should be fairly obvious that Greenlight is the only viable option here. Steam is such a fantastic marketing opportunity and Steamworks is such a huge amount of infrastructure work I don’t have to do that it’s easily worth paying the percentage fees to Valve.

I think the way forward from here is going to be to put Heist up on Greenlight concepts within the month, then perhaps move over to a full Greenlight game as soon as I have a single gamemode done (most likely construct mode). Exactly when I move from concepts will depend upon the feedback I get from people on Greenlight, of course.

Game Developers Don't Make Timetables

Thu, 03 Jan 2013 00:00:00 +0000

TL;DR

I want a timetable to work to, and I don’t see any reason to keep it secret.

An Explosion Of Buildings

The coolest thing about the way cities work in Heist is that lots of people can all contribute a small bits and the system will place them together even though no one is collaborating with anyone else. Of course for this to happen it is vitally important that contributing something small is actually a small amount of effort - even better it should be fun!

At the moment contributing new parts of a city is not fun. First you need to learn lua, and then you need to learn the Heist procedural generation API, and then you need to manipulate a load of CSG brushes in your head until you’ve worked out how to construct a parametric building just by writing down lines of code. Needless to say this is why the city currently isn’t terribly detailed, I’m the only contributor and I have better things to do with my time (like writing the actual game).

But It Doesn’t Have To Be This Way!

Games like Minecraft, Garrys mod and Kerbal Space Program have proved that people really love to build things. The problem isn’t that people don’t want to build cities. The problem is that right now, building cities is really boring and requires not insignificant programming skills and effort.

The Big Reveal

The next few months of development on Heist will be working towards solving this problem by developing an in game construction mode which allows players to create/edit/save and load shapes and then to use these shapes to build procedural cities which can then be loaded in any of the other Heist gamemodes as places to play in. This is quite exciting for me because most of the work for this mode will not be engine work, instead I will be doing most of my work in writing Lua scripts to create the new gamemode and associated scripted tools/entities which means this will be the first real test of how flexible the game is for modding.

The other thing I’m going to be working on in the same period is working out how exactly I’m going to start making money off all this hard work. To start with I’m simply going to be posting on Screenshot Saturday over at r/gamedev ~~every~~ most weeks to get a little more attention on the project. Beyond that point I see my options as:

Working towards a full steam release
Working towards a Greenlight release
Putting myself on Greenlight as a preview/concept
Hosting my own paid alpha
Kickstarter

Of these options I think hosting my own paid alpha is the most likely while getting onto steam is the most desirable. There’s a lot to talk about here so I’ll talk about this in more detail in another blog post.

Timetable

I promised a timetable, so here it is:

January

(COMPLETE) Basic Construct Mode (tools to create and manipulate CSG brushes)
(COMPLETE) Material Selection for CSG Brushes
(ON HOLD) Shape Library (a place to save and load custom shapes from, persistant)

February

Parametric Construct Mode (allow players to create nodes containing shapes with sizes like “X is 75% of the width of Y”)
Parametric Saving (save parametric shapes as scripts for city generator)
Generate Cities (allow players to generate an regenerate small chunks of a city on the fly, including using the paramtetric shapes they just saved)
Improve Demo City (craft some interesting buildings using Construct Parametric Mode)

March

Implement Building Topology (Wiring, water pipes, computer networks, AI navmesh)
Integrate Topology Into Construct Mode
Put Heist on Steam Greenlight (Concept or Alpha Release?)

What Is Heist?

Thu, 20 Dec 2012 00:00:00 +0000

TL;DR

Heist is a game engine, and a series of games built using the engine.

Why This Post?

I’m planning to start being a bit more public about Heist soon, initially by posting on r/gamedev. I don’t want to do a full explanation of exactly what Heist is every time I introduce someone new to it so that’s what this post is for.

What Is Heist?

What Was Heist To Start With?

Heist was originally conceived about 6 years ago (2007). First there was this crazy idea I got into my head that entire cities could be generated by a computer, I was totally obsessed with this idea for a couple of years while in college. The second part was that I love Heist movies, and many of them consist of creatively misusing the environment around the target of the Heist to somehow create a flaw within the bank. The simplest example would be The Bank Job where they simply tunnel into the bank. Once you have these two thoughts bouncing around your head the idea becomes pretty obvious: Generate a city around a bank for the ultimate Heist game!

Once I started work on Heist (in January of 2012) it became clear that the project, in some ways, wasn’t ambitious enough (not something I get accused of very often). Building an entire city generator and then just setting one single game in it felt like such a massive waste of potential. The biggest demonstration of this was that every time I mentioned the city generator to people they would mention other games they would love to see in such a system.

A few suggestions people have made:

Break Into Banks
Multiplayer Deathmatch
Procedural Racing Game
Procedural Mirrors Edge
Spiderman/Worms 3D mashup
Detective Game (explore city, talk to NPCs)
Assassination
Zombie Survival (The most common suggestion by far)
World Building (Minecraft) / Contraption Building (Gmod) Mashup

Luckily this became clear while I was experimenting with procedural generation technology and hadn’t even started on building the game yet. Once I began thinking about building the game itself I expanded the scope a little…

So What Is Heist?

A Game Engine

Heist is a game engine, which is a platform for building many other games with. The engine supports procedural generation of massive worlds and is designed to be controlled by lua scripts to define new gamemodes and entities.

A Series Of Games

While the engine is in development I will also be building test games using the engine to push it’s capabilities. If these games turn out to be any good then I shall keep them around, polish them up and release them.

A Game About Breaking Into Banks

Once the engine has been developed enough that it has all the features I need to build this game mode, I will do so. There’s nothing special about the Heist game mode any more, it’s just another game in the series.

How Does Procedural Generation Work?

Fri, 14 Dec 2012 00:00:00 +0000

TL;DR

L-Systems are about replacing lines of meaningless text with cool stuff… like most programming, really.

But There’s A Hole In My Bucket…

So we’ve talked about what procedural generation is for, and what it isn’t but how does it really work?

Ultimately procedural generation is just about building algorithms to create art – of course, there’s no one magic algorithm to do that! However, there are a few basic elements that often appear in relation to procedural generation algorithms. Big disclaimer: these are the things that often seem to appear in relation to me thinking about how to solve my procedural generation problems; I’m certain that other things could be added to this list. If you think of anything missing tell me about it, and I’ll probably learn something new. Ordered roughly by how complex they are to use, from least to most complex:

Noise
Voronoi diagrams
L-Systems
Markov chains

Noise

Noise is actually quite hard to define. Everyone understands noise from their day to day experiences as “an unwanted random addition to a signal” – think annoying traffic noises added to the “signal” coming from your car radio, or unwanted background chatter added to the “signal” of the conversation you’re trying to have. Another definition that wikipedia gives, which I think is more relevant, is: “noise can be considered random unwanted data without meaning”. Why is this more relevant? Well, sometimes we want random data without meaning. Imagine the patterns in a piece of marble. They have no meaning and are randomly laid out, so if we wanted to procedurally generate a marble pattern it stands to reason we’d need some noise.

The most well known kind of noise (in the field of computer graphics anyway) is Perlin Noise

The point of perlin noise is that it has pseudo random features which are all the same size. This is really useful for many things – if you know roughly how large the noise features are going to be, you can more carefully control the way in which you use the noise to create certain effects. For example if you’re generating landscapes you could add together many layers of perlin noise with different scales. You use the layer with features a few feet wide to control little bumps in the landscape and you use the layer with features a few kilometres wide to control the overall geography of the landscape. This is actually how Minecraft generates a lot of the landscape: biomes are selected with overlaid heat and moisture maps, mountains are generated with large scale simplex noise added to small scale simplex noise for hills and finally caves are multiple levels of simplex noise subtracted off the main landscape.

Since inventing perlin noise Ken Perlin has since created simplex noise, which has several advantages over perlin noise. Simplex noise has the same features as perlin noise but is far faster to compute. If you’re interested in how simplex noise works (or how it differs from perlin noise) check out Simplex Noise Demystified.

Voronoi Diagrams

Voronoi diagrams are dead simple to explain and have many applications, both in science and art. Imagine a load of points scattered across a plane, which we’ll call seed points. Now we want to classify every single point on the plane into a region. To do this, we simply put each point into the same region as the closest seed point. If you inspect the demo above you’ll see that’s what is going on – each coloured region shows the set of points which are closest to the (white dot) seed point in that region. Naturally, this definition extends into as many dimensions as you like, most usefully 2D and 3D of course.

If noise is good for generating continuous things, voronoi noise is good for generating discontinuous things. For example, you could use voronoi noise as the basis for generating city blocks by placing roads along the edges of the regions and filling in the regions with buildings and maybe some minor roads for large blocks.

If you’re interested in how voronoi diagrams and noise can be used together check out this fantastic blog post about procedurally generating an island.

L-Systems

Click For New Trees:

Noise and Voronoi regions are good ways to generate largely meaningless data, but what about when we want data which is in some way meaningful? L-Systems were originally invented by a biologist to simulate the growth of mould and it has turned out that they’re good at generating many other patterns. An L-System can be imagined as a string rewriting system – you start off with some string and then, according to a set of rules, you replace all the letters in the string with other sequences of letters and you keep doing this as many times as you like.

That’s a bit abstract, so here’s an example. The trees above are generated with this starting string:

and these rules:

Replace “X” with “F-[[X]+X]+F[+FX]-X”
Replace “F” with “FF”
If not “X” or “F”, don’t change it

That’s it! Let’s step through a few iterations of this.

X
F-[[X]+X]+F[+FX]-X
FF-[[F-[[X]+X]+F[+FX]-X]+F-[[X]+X]+F[+FX]-X]+FF[+FFF-[[X]+X]+F[+FX]-X]-F-[[X]+X]+F[+FX]-X

Now these strings aren’t really looking terribly useful – however does that string describe that tree? In this case the letters are taken as commands to a turtle graphics system:

“F” means draw a short line forward
”-“ means turn left some small randomised amount
”+” means turn right some small randomised amount
”[” means save the current position and angle on a stack
”]” means restore the previous position and angle off the stack

L-Systems don’t have to just be string rewriting systems either – that’s just a convenient way to talk about them. For example Heist uses what amounts to an L-System in it’s city generation, but is implemented very differently. Each script is a node which places a bit of geometry or a few child nodes. For example, a “City” is a node, which places down a load of building and road nodes. A “Road” is a node which places a load of pavement and streetlamp nodes. A “Building” is a node which places a load of window, door and room nodes inside itself. If you think about it, this is an L-System – we have symbols:

City
Road
Building
Pavement
Street lamp
Window
Door
Room

and we have rules:

City => Road, Building
Road => Street lamp, pavement
Building => Door, Window, Room

Markov Chains

data but as a thing that type of areas has several stages of the name and then another small game projects of the game but has IP address pairs it to put up Sold So here are procedurally generated by pasting the name General Chat Peer Assigns Pipe General Chat Time

L-Systems are a good way to generate data which is structured and meaningful in a way based on solid rules, but a lot of the world doesn’t have solid rules which can be simply expressed to an L-System. Instead, a lot of the world is based on fuzzy half rules which no one really fully understands. This is where Markov chains shine.

Markov chains are for generating strings of values which are in some way coherent, or conform to unspoken or intuitive rules. To express the rules to the system you simply given it some existing data which conforms to these rules, and it will generate more data in the same trend. The paragraph of nonsense above was generated by using my entire blog as the definition of how text is written and then asking for more text. Markov chains have been used in games to generate names of people or places. This is a perfect example of where the rules for the system are complex, difficult to express to a computer and are intuitively understood by the reader. Simply give the Markov generator a load of place names and now you have an almost infinite supply of new place names – perfect!

The way in which this works is actually very simple. From the example data the Markov chain generator works out the probability of any given symbol being followed by other particular symbols. To generate new data the system picks a random start symbol and then just randomly picks subsequent symbols weighted by how common they are. There are loads of applications of this:

Generate Character Names
Generate Road Names
Generate Strings of Building Types (Shops often go next to other shops, but sometimes offices and occasionally houses)
Generate Music (This is totally awesome, I didn’t think it would sound so convincing!)

Putting It All Together

None of these techniques are good enough to generate complicated things on their own. The point is that these four techniques (and others) can act as basic building blocks for much larger procedural generation algorithms. It becomes the job of the artist not to create art, but to pick out the building blocks and the parameters for algorithms to create the kind of art that they want.

I mentioned it before, but check out this blog post for an example of putting bits together. He uses perlin noise and voronoi diagrams to generate a landscape with roads and biomes.

Why Does Heist Keep Crashing?

Fri, 07 Dec 2012 00:00:00 +0000

TL;DR

Heist crashes lots because it’s in development!

A Nice Problem To Have

I have a very nice problem with Heist – I keep getting bug reports being emailed to me! This is nice because it means that people are playing my game even though it’s really early in development, doesn’t have any gameplay, is totally unstable and I’ve not advertised it at all! Almost all bug reports share the same root problem…

Persistence

A critical point to remember is that Heist is designed to be highly modifiable – without any installed mods Heist doesn’t even get as far as showing a main menu, let alone any gameplay! Of course, not showing a main menu and having no gameplay would kind of suck, so the game currently has a default mod baked into it which shows a menu and downloads some default gameplay.

Mods aren’t the only thing that can be configured about Heist either. During development I’ve made an effort to have no as few hardcoded values as possible; if I feel I need a value then it gets put into the configuration file. These configuration values can be anything from simple stuff (key bindings) to complex stuff (network NAT negotiation period) to inexplicable stuff (synchronicity of the world generator).

My point here is that Heist has a lot of stuff that gets saved out to disk, and it is loaded back up next time you play. It would be a waste to download and unpack the default mods every time you played. It would also be incredibly annoying if Heist forgot changes to your keybinds every time you played!

Version Management

This is why Heist keeps breaking in almost the same way with every single version. I’m developing so rapidly at the moment that every time you play you’re loading up a new and incompatible version of the game. When the game starts it tries to load your configuration and mods from the last time and BOOM something somewhere chokes on the old data.

To be clear, the incompatibility isn’t because I’ve changed the syntax. If that were the case I could easily just parse the file in the old way and the new way and use whichever one worked, and then I could write out the data in the new format so that it would only ever be a problem once. The problem is that I’m frequently changing the semantics of files. For example, I occasionally change the name of arguments in the scripting interface – there’s no good way to handle this.

But You Could…

I could handle this much better. The mod system could ignore incompatible mods and inform the user that they need to fix something. The configuration file could be ignored if it’s badly formatted. I could publish a list of compatible versions and make the game delete all old configuration if it’s from an incompatible version. For now though, these things require more effort than I think is really worth it.

Of course, once the game is properly released these problems will start to go away from “both ends”. Development will almost never be making breaking changes to the on disk format, which means data will be compatible between most versions and these problems will occur much less frequently. Also, handling of incompatible data or broken mods will be implemented properly by then, so if there is something wrong the game will handle it gracefully and inform the user rather than exploding in your face and sending me an email.

What Can I Do About It?

All the configuration data and mod data that Heist stores is in one folder:

C:\Users\YOUR USERNAME HERE\AppData\Roaming\Heist

If your game breaks, delete that folder and try again. Of course, if it still breaks make sure to send me a bug report.

What Isn't Procedural Generation?

Fri, 23 Nov 2012 00:00:00 +0000

TL;DR

An overview of what Procedural generation is not and common misconceptions.

It Isn’t What It Ain’t Not

So we’ve covered what procedural generation is, but what is it not? Procedural generation is one of those phrases that has been around long enough for most gamers to have heard of it… and to misunderstand what it is. I’m just going to cover a few myths about procedural generation here – I expect to keep coming back to edit this post as I bump into more misconceptions.

Procedural Generation Is Random Generation

No! Well, sort of. Random generation implies unpredictability; you ask for some content and you get some random thing back which looks a little like what you asked for. Computers are naturally deterministic and procedural generation is just about running an algorithm to get some content, so it must be deterministic. Procedural content can be completely predictable and can just be about storing things in a more efficient form (don’t send them an entire tree, just send the parameters to feed into a generator to create that exact tree).

Procedural generation can be random if you want it to be. The input parameters to the generator can be randomised and then using those random parameters you go off and generate some content. This is what, for example, Minecraft does with world generation. The entire world generator is totally deterministic and the only randomness in the entire world comes from the seed you can input when you create the level.

Procedural Generation Is All About Replacing Artists

If you were paying attention to the last myth then this one ought to be obvious. If you’re not supplying random parameters to the procedural generation then where are you getting them from? I guess you’re going to need a human to carefully hand tweak the parameters of the generator to achieve the right look (and we could call this person an artist).

In many cases procedural generation is about helping artists do a better job. Why force an artist to hand make a building façade by pasting the same window in 5 times for each of the 10 floors when a procedural generator could just be told Generate a 5×10 façade using this window? If you can free up artists from the drudgery of making art by giving them better tools then it’s only natural they can make better content.

An interesting point to make is that I think this may explain why procedural generation has never yet become mainstream. One year people might talk about a really cool procedural tech to generate cities, the next year artists are using a tool which makes building cities easier and the quality of content goes up (as people expect). By the time procedural generation gets into use it’s just a useful artist tool and not procedural generation any more!

Procedural Generation Is Done On The Client

Again, this one ought to be pretty obvious. It’s fairly rare for games to come with an artist supplied in the box to tweak the values on the client computer while the game is being played. Naturally, this means that if the generator is being hand tweaked then it must be run as part of the production process of the game.

Procedural generation is quite regularly used in major games as part of the content creation pipeline. Games like the Elder Scrolls series, Eve Online and Just Cause have all had their (massive) maps generated by a computer before humans dived in to tweak the world and apply some interesting features.

Procedural Generation Is Hard Work

That all depends on your definition of hard work doesn’t it? Personally I find spending a week or two writing an algorithm and then being supplied with trillions of bits of content easier than spending a week or two hand modelling 10 bits of content. A general rule of thumb that I have is that if I have to do a bit of extra work to make my future life 12 orders of magnitude easier it’s probably worth it ;)

Procedural generation is basically the gift that keeps on giving. If you’re making a small, very focused game, which requires carefully designed things then maybe procedural generation isn’t so useful. For example, using random level generation like X-Com to generate puzzles for portal would probably be completely disastrous. N.b. I totally want to see a portal procedural level generator – that would be awesome if it worked.

Procedural Generation Is For [Levels/Textures/Meshes/Sounds/Music/Stuff] Only And Cannot Be Used For [Levels/Textures/Meshes/Sounds/Music/Stuff]

Procedural generation has been used for Levels, textures, Meshes, Sounds, Music and Other Stuff.

I’m pretty sure Procedural Generation can be applied to almost any area of content generation. Whatever it is that artists are doing there is some drudgery that could be automated for them.

How Does This Apply To Heist?

One of the big features of Heist is procedurally generated cities. Now that we know that procedural generation can be very heavily human directed (an artist tool) or totally automated (pure random generation) where does Heist lie on the scale? Heist lies closer to the human directed end of the scale than you might expect.

Cities are generated with human written scripts which individually “describe” a small feature of the city. Of course, the description can use (seeded) randomness to add a little variation – for example a skyscraper could be generated with anything between 10 and 1000 floors. The important bit is that the skyscraper script would mark a load of areas as floors in a skyscraper and then the system would go and find some scripts which describe a skyscraper floor – the procedural generator doesn’t generate new content, it just pieces together descriptions of little bits of content written by humans to create a large and unique piece of content.

What Is Procedural Generation?

Sun, 18 Nov 2012 00:00:00 +0000

TL;DR

An overview of what Procedural generation is and why it is useful.

What’s All The Fuss About?

Game development budgets are exploding. Driven largely by demands for super realism or for ever larger worlds, dev teams are growing to encompass more and more people to produce the ever increasing amount of content required. This is a problem for a couple of reasons:

Big Budgets Kill Creativity

If a company spends $50,000,000 making a game they want to be very sure that it will sell! That means that no one is willing to invest that huge budget in a new idea because no one knows if it will flop. Instead, it makes business sense to invest in new iterations of existing sequels for a safe return.

Hand Made Content Kills Replayability

You buy a big budget game. Play the hand-crafted, highly linear, single player campaign. Then have a blast killing people on the 5 multiplayer maps available.

Well, that kinda sounds like a bad way to spend $60 doesn’t it? There’s no point playing the single player again – you already know how it ends! And those built in multiplayer maps start to get a bit old after a while.

Fundamentally…

The problem is that hand crafted content costs lots and it just isn’t viable to spend all that extra time and money on a more replayable experience. A studio could dedicate the resources to building a branching storyline with completely different choices at every opportunity and hundreds of totally different endings, but they’re going to massively increase development costs. What’s more, most players will still only play once or twice anyway so that definitely isn’t a good investment.

What’s really needed is another entire approach to developing content. An approach where, after an initial cost, you can build lots and lots of content very quickly and for almost no extra cost.

Surprise Reveal

That’s what procedural generation is (no one saw that coming, right?). Procedural content generation (PCG) is simply the idea of getting the computer to create content for you instead of an artist. The key ideas within PCG are variety, size and replayability.

Infinite Variety

The quintessential example of an infinite variety PCG system is SpeedTree. Ever wondered how games have such a vast number of trees, none looking exactly the same? It’s a safe bet that SpeedTree was used.

SpeedTree is an “imperfect factory”. Essentially, an artist spends some time describing the rules that define the kind of tree they want (parameters like branch length and angle, tree height, leaf density etc.), and once that initial effort has been put in SpeedTree can generate a literally infinite number of trees which meet these parameters. The incredible thing here is that the initial work by the artist is quite probably less than the work required to hand model a single tree, and the total development cost of SpeedTree is undoubtedly less than how much it would cost to hand model an entire forest!

Some other examples:

Borderlands has 87 bazillion unique guns
Strategy games usually come with the option to generate a random map (implying there are billions of maps)
Rhythm games procedurally generate their levels from a seed file (the music). Implying as many levels as there are possible music tracks

Infinite Size

A natural consequence of infinite variety is that it becomes easy to generate really big worlds. Got an algorithm to create varied tiles of landscape? Keep applying it until you have a world of any size!

Elite has 2048 systems (a huge number for the 1984 release date)
The huge worlds in the Elder Scrolls series were all procedurally generated during development
Minecraft generates a world significantly larger than the surface area of the entire Earth

Infinite Replayability

Procedural generation of gameplay rather than content is starting to become popular and I think we’re going to see a lot of development in this area over the next few years.

The best example I have of this is in Left 4 Dead. In L4D the “AI director” coordinates the game behind the scenes. If a team is playing well together they will encounter more zombies, more frequently, with more specials and less medkits scattered around the world. Whereas if a team is playing badly they will encounter less enemies, less frequently and will frequently find medkits and better guns scattered around the world. This all adds up to L4D having huge replay value, even though it only has a few hand-crafted maps which do not change between play sessions.

This kind of generation can be stretched further to create highly branching storylines, with every action the player takes generating new events, stories and missions.

How Does This Apply To Heist?

Heist will take advantage of all three benefits of procedural generation. Right now the most obvious use is the procedurally generated cities which reap the benefits of size and variety. In the future I plan to implement an “AI director”, just like L4D, which will coordinate the AI of the entire city, partly for realism and partly for responsive gameplay.

Procedural Generation Series

Thu, 15 Nov 2012 00:00:00 +0000

TL;DR

I’m going to write a series of blog posts about procedural generation.

That Was Fun!

I’ve just finished writing the networking series and I’m moving right on to a series about procedural generation. The networking series was fun to write – hopefully it was interesting to read! While writing it I noticed a few optimisations I could make to Heist, so it was useful to at least one person.

The networking series was written while I was working on a big internal milestone (more details on that in a week or two when I have it all polished up). Obviously this super secret milestone involved lots of networking, so it made sense to write a networking blog series while working on it. Now that this milestone is done I’m going to move on to the next one soon, which will involve building a new gamemode for Heist and all the technology and tools needed to make it possible.

Minecraft Is So 2011

This might seem unrelated, but do you ever see cool constructions in real life or other games and think to yourself, “That’s really cool… I wonder if I could build that in Minecraft?” I certainly do, and it’s lead to an incredible number of wasted hours building cool things. Recently I’ve been playing Guns of Icarus (basically a game about flying steampowered airships blowing each other up), and I also saw some videos, all of which set my imagination going about how well I could build a steam powered airship thing in Minecraft. Alas, I decided that I didn’t have time to dedicate to a new Minecraft mega project and that was that.

And That Was That

A big part of Heist is procedural generation of worlds. Most of the time the world would be a city to carry out a bank job in, but there’s nothing stopping the world generator from being used to generate other megastructures… floating steam powered airships for example. So, this is my next major goal for Heist – I’m going to polish up the world generator and the scripting interface and then build all the tools I need to create the scripts for a procedurally generated airship into the game. The critical part here is that the tools I use will be built into the game as a new gamemode. Building new parts for a world will not involve writing boring lua scripts (unless you want to), and it won’t involve opening up a separate world builder tool (although I might build such a thing with enhanced debugging capabilities for hand written scripts). Instead, you simply start up the game and start crafting a world.

The Series

This series is going to have 2 parallel sets of entries.

Procedural Generation

This is the main meat of the series – a set of posts on the subject of procedural generation with overviews of different ways of doing it, how other games do it, drawbacks of these techniques and some of the technical underpinnings of implementing procedural generation.

Steampunk Airships

These will be a series of posts about the new gamemode in Heist, with details on how it works, how to play it, where to get preview releases and what progress I am making on the steampunk airship world scripts.

Procedural Generation Series

Packet Encoding(3)

Mon, 12 Nov 2012 00:00:00 +0000

TL;DR

Sometimes the best way to make something smaller is to throw half of it away.

Topics

In this series I’m covering 5 topics:

Topology
Session/Connection Initiation
Delivery Guarantees
Named Data Pipes
Packet Encoding (Architecture)
Packet Encoding (Lossless Compression/Variable Length Encoding)
Packet Encoding (Lossy Compression)

Don’t Throw That Away!

Last time I talked about the variable length encoding Heist uses for integers which are likely to be near zero. There were a few other lossless operations I listed which I should explain before moving on:

WriteVariableUint64
WriteRotationQuaternion64
WriteVector3Float32
WriteVector2Float32

WriteRotationQuaternion64

A rotation quaternion is a way of storing rotations in 3D space. It’s the way Heist uses because the BEPU Physics engine prefers to represent rotations like this. A rotation Quaternion has 2 parts, an axis vector and an angle of rotation. As axis vector is a 3 dimensional vector which represents an axis – this means its values will all be between 1 and -1 and that the total length of the vector will be 1. An angle of rotation is a rotation about this axis and will lie between -1 and 1. An interesting property of rotation quaternions is that if we have a quaternion which has SOME_AXIS and SOME_ROTATION it is equivalent to the quaternion with -SOME_AXIS and -SOME_ROTATION.

WriteRotationQuaternion takes advantage of all but one of these constraints to pack a quaternion down. The first thing to do is to make sure the rotation component is positive, and once it is we know it will definitely lie within the range of 0 to 1. XNA includes a system for encoding 16 bit floats which are in the 0 to 1 range, so the angle is written out using this. The axis is pretty similar – the best encoding method to use is the same one used for the rotation but we can’t do that directly (remember the value is in the -1 to 1 range). Instead we simply divide the number by 2 (now it’s -0.5 to 0.5) and then add 0.5 (now it’s 0 to 1). Problem solved!

if (ANGLE < 0)
{
    ANGLE = -ANGLE;
    AXIS = -AXIS;
}

WriteNormalizedFloat16(ANGLE);          //Must be in 0 to 1 range
WriteNormalizedFloat16(AXIS.X / 2 + 1); //Remap into 0 to 1 range, then write
WriteNormalizedFloat16(AXIS.Y / 2 + 1);
WriteNormalizedFloat16(AXIS.Z / 2 + 1);

Did you notice which property isn’t exploited here? We know that the axis values must represent a vector of length 1, so this property could be exploited to increase the precision by encoding the signs of the axis elements and 2 of the 3 elements (less elements means we can take more space per element which means more precision). To decode this it’s simple maths to calculate the value of the remaining element (and then give it the correct sign, which is stored explicitly). I don’t do this because it’s significantly more complicated than simply writing out 4 16 bit floats after some mathematical trickery!

Write VectorNFloat32

These methods are pretty boring – they just directly write out the elements of a vector as full 32 bit floating point values. Used for things where large range and precision is important, e.g. position.

Out With the Less Important Bits

The lossy encoding options aren’t actually very exciting, they are all just variations on the theme of writing floating point numbers with less bits of precision. I am quite likely to add more methods in the future which will be much more interesting (e.g. I have some thoughts on variable length encoding floating point numbers). For now though, here’s a summary of the lossy methods for completeness’ sake:

WriteFloat16. Lose some precision and write out a float using 16 instead of 32 bits
WriteRotationQuaternion32. Same as write quaternion 64, except use half as many bits for everything
WriteVector3Float16. Use WriteFloat16 for all elements of a vector3
WriteVector2Float16. Use WriteFloat16 for all elements of a vector2
WriteNormalizedFloat8. Write a floating point value using 8 bits assuming it is in the 0 to 1 range
WriteRangeLimitedFloat8. Remap a number into the 0 to 1 range, then write a normalized float

And We’re Done

That’s that, I’m fed up with talking about encoding numbers into bytes! I’m thinking about doing another series soon (tm) on how the procedural generation works in Heist, which is a far more interesting topic (with much more screenshot potential).

Packet Encoding(2)

Thu, 08 Nov 2012 00:00:00 +0000

TL;DR

Some things can be smaller than other things in some circumstances.

Topics

In this series I’m covering 5 topics:

Topology
Session/Connection Initiation
Delivery Guarantees
Named Data Pipes
Packet Encoding (Architecture)
Packet Encoding (Lossless Compression/Variable Length Encoding)
Packet Encoding (Lossy Compression)

Compression

It’s a basic fact of data compression that you can’t compress everything, if you make one thing take less space then you must be making something else take more space to store. Given this fact how does anyone ever compress any data? Image compression, music compression and general compression (e.g. zip) all do a pretty good job of it. All data compression simply works in one of two ways.

Lossy Compression

As the title implies, lossy compression loses some data when it compresses it. It’s obvious how you can make a file smaller if you throw some of the data away! This might sound pretty stupid but it’s probably the kind of compression you use most often, every time you view and image on a website or listen to an mp3 file you’re using lossy compression. The trick to lossy compression is knowing which data to throw away.

Lossless Compression

You may have guessed that the other type of compression does not lose data when it is compressed. It’s less obvious how this works, after all if you’ve got exactly the same data it must take the same space… right? Lossless compression works by making more common patterns take less space, and less common patterns take more space. For example, music files are not just random noise (usually) instead they are smooth curves which means that any value is going to be (normally) only a small difference from the previous value, so we can use a compression system that writes out the differences between values and uses less bits to encode small numbers and suddenly our music file is smaller (this is roughly how the lossless flac music format works).

Glossary

A floating point number stores a huge range of numbers, both very large and very small. Using less bits to store a float does not reduce the range, instead it reduces the precision.

An Unsigned Integer stores (obviously) whole numbers. Using less bits to store an integer does not reduce the precision instead it reduces the range of possible values that can be represented. i.e. a 16 bit integer simply cannot represent values larger than 65535, but a 32 bit one can represent all the way up to 4,294,967,295.

A Signed Integer can represent negative (whole) values. A signed integer simply shifts the range of possible values down by half, if an unsigned 16 bit integer can represent 0 -> 65535 then a signed 16 bit integer can represent -(65535/2) -> (65535/2).

Binary number are represented using Bits, A bit is either a 1 or a 0. In the same way as normal numbers the rightmost number means it’s own value, the next number to the left means it’s value * 10, and so on. A more in depth explanation is pretty far out of scope here, read wikipedia if you’re unfamiliar with binary.

A byte is the smallest value we can represent, it takes 8 bits. The packet encoding system never writes values less than 1 byte because it’s just a pain to handle.

Network Encoding

When writing a packet you’ll usually use a combination of both of these methods. Last time I included a list of all the methods Heist offers for writing to packets, here they are again classified as lossless or lossy:

Lossless

WriteVariableUint64
WriteRotationQuaternion64
WriteVector3Float32
WriteVector2Float32

Lossy

WriteFloat16
WriteVariableFloat16
WriteNormalizedFloat8
WriteRangeLimitedFloat8
WriteRotationQuaternion32
WriteVector3Float16
WriteVector2Float16

WriteVariableUint64

This is a lossless compression method for writing out any unsigned integer (64 bits is the largest any C# integer, hence UInt64). The key to how this method works is that it takes less space to write out smaller values, it’s used in Heist to write out things like the length of any array (which is quite likely to be small) or the ID of an entity (which is assigned in ascending order, and so get larger with time).

How do we achieve this compression? It’s quite simple really:

for (int i = 0; i < sizeof(UInt64); i++)        //Keep looping over all the bits in this number
{
    byte writeByte = (byte)(value & 127);       //Take the 7 least significant bits
    value >>= 7;                                //shift the next 7 bits of the value into the least significant position

    if (value != 0)                             //If there are any useful bits left...
        writeByte |= 128;                       //...Then set the most significant bit of the byte to 1

    Write(writeByte);                           //Write out this single byte

    if (value == 0)                             //If there's no more data
        return;                                 //...Then stop trying to encode more data
}

I lied, that’s not simple! Here’s an example using binary to represent the numbers:

value = 0001010110100101                        //A 16 bit integer, the value 5541

writeByte = value & 01111111                    //This is value & 127 expressed in binary. Basically this takes the 7 least significant (right most) bits.
                                                //This means writeByte is now 0100101 (37)
                                                
value >>= 7                                     //Move all the bits in value 7 to the right
                                                //This means value is now 0000000000101011
                                                
if (value != 0)                                 //True! Value is still 43
    writeByte |= 10000000                       //This means set the bits in writeByte to true if they are already set *or* they're set in the given value
                                                //00100101 or 10000000 = 10100101
                                                
Write(writeByte)                                //Put this byte in the packet to be sent

if (value == 0)                                 //False! Value is still 43

/* Next Loop Iteration */

writeByte = value & 01111111                    //This means writeByte is now 0101011 (43)

value >>= 7                                     //This means value is now 0000000000000000

if (value != 0)                                 //False! Value is 0

Write(writeByte)                                //Put this byte in the packet to be sent

if (value == 0)                                 //True!
    return;                                     //We're done

So in this case we took a 16 bit integer (2 bytes) and saved ourselves… nothing, we still used two bytes. However if we had used a 64 bit integer (8 bytes) it would still have only taken 2 bytes, that’s a 75% compression ratio (which is awesome).

Hopefully it should be relatively obvious how this is decoded. You read byte by byte, and if the 8th bit in the byte is set it means there are more bytes to come, if it’s unset it means we’re done.

As I mentioned earlier, all compression makes some values bigger to make others smaller. Of course, the things that get here are the big numbers… the really really big numbers. A 64 bit integer can represent values up to 1.8446744x10^19. I can’t think of any reason Heist is likely to send values this big, so we’re good!

That Was Quite Long

Explaining all that took more space than I expected! I’m going to stop there and continue explaining there other encoding systems (the lossy ones) at a later date.

Packet Encoding

Wed, 07 Nov 2012 00:00:00 +0000

TL;DR

I have a pretty neat system based on C# generics and reflection for encoding of packets.

Topics

In this series I’m covering 5 topics:

Topology
Session/Connection Initiation
Delivery Guarantees
Named Data Pipes
Packet Encoding (Architecture)
Packet Encoding (Lossless Compression/Variable Length Encoding)
Packet Encoding (Lossy Compression)

The Question Remains

Last time I talked about how packets get routed to the right place through pipes, the question remains what exactly gets sent? The obvious solution is just to say that all pipes send and receive byte arrays. That’s a pretty terrible solution though, it means that every sender and receiver needs a load of logic for translating binary blobs into some type of data. It’s obviously going to be better in the long term to encapsulate the translation somewhere else, so that’s what Heist does. A pipe directly sends and receives types (in C# parlance, pipes are generically typed), the pipe system finds a translator which knows how to turn Type(T) into a binary blob and then back again.

Translators in Heist are just a class which extends the BasePipeTranslator<T> class, when the pipe system wants to find a translator it simply scans all of the loaded types available to it and finds the one where T is the type it wants. Of course, writing a translator is usually really trivial:

class Vector3Translator
    :BaseTranslator<Vector3>
{
    public override Vector3 FromBytes(BasePacketReader stream)
    {
        return new Vector3(
            stream.ReadSingle(),
            stream.ReadSingle(),
            stream.ReadSingle()
        );
    }

    public override void ToBytes(Vector3 data, BasePacketWriter stream)
    {
        stream.WriteSingle(data.X);
        stream.WriteSingle(data.Y);
        stream.WriteSingle(data.Z);
    }
}

What’s going on here is fairly simple. The BaseTranslator<Vector3> class is abstract and requires that ToBytes and FromBytes are implemented, what each one does is (I hope) fairly obvious. The BasePacketWriter and BasePacketReader are convenience classes which implement reading and writing of all the basic primitive types (int, uint, short etc) which I’ll talk about in more detail in a minute.

So how is this used? Here’s a basic example of a pipe which sends and receives Vector3’s:

var pipe = GetPipe<Vector3>("Name Of This Pipe");
pipe.Send(new Vector3(1, 2, 3));

That’s it! The system automatically finds a translator for Vector3, and every time Send is called it passes the value through the ToBytes method and sends the resulting blob of bytes across the network.

Reusability is cool

There’s another cool use of this translator system which makes writing translators quite simple in many cases. The BasePacketWriter type has a generically typed method Write<T> which (similarly to pipes) finds a translator which knows how to translator type(T) into binary and back and then uses it. This means that translators can easily use other translators inside themselves, for example:

class PhysicsStateTranslator
    :BaseTranslator<PhysicsState>
{
    public override void ToBytes(PhysicsState data, BasePacketWriter stream)
    {
        stream.Write<Vector3>(data.Position);
        stream.Write<Vector3>(data.Velocity);
        stream.Write<Vector3>(data.Orientation);
        stream.Write<Vector3>(data.AngularVelocity);
    }

    public override PhysicsState FromBytes(BasePacketReader stream)
    {
        return new PhysicsState(
            stream.Read<Vector3>(),
            stream.Read<Vector3>(),
            stream.Read<Vector3>(),
            stream.Read<Vector3>()
        );
    }
}

As you can see, PhysicsState wants to send 4 Vector3s, so instead of writing out all 12 floats one by one it uses the Write<Vector3> method to invoke the Vector3Translator. I think that’s pretty cool!

Clever Bit Twiddling Hacks

I’m going to do another set of blog posts of some of the clever bit twiddling used within Heist but here’s a bit of a taster ;)

As I mentioned earlier the BasePacketReader/Writer classes handle reading and writing of the primitive types. This is not as simple as it sounds, every bit counts when it comes to packing data into a network packet! Most values can be encoded in such a way that more common values use a smaller amount of space than less common ones. For example Heist has an encoder that writes out arrays of items, it’s obvious that most arrays are going to be quite small but I don’t want to enforce a hard limit by encoding the length as a byte. So in this case the length of the array is encoded using “variable length encoding”, this writes out the bytes of the value in 7 bit chunks, each byte uses 7 bits to store actual data and then a single bit to indicate if there are more bytes of data coming up.

This principle applies to a load of different values and so the packet reader/writers have many different helper methods for encoding data in different ways. Here’s a list of all the most interesting encoding methods, I’ll talk about these soon (tm):

WriteVariableUint64
WriteFloat16
WriteVariableFloat16
WriteNormalizedFloat8
WriteRangeLimitedFloat8
WriteRotationQuaternion32
WriteRotationQuaternion64
WriteVector3Float32
WriteVector3Float16
WriteVector2Float32
WriteVector2Float16

Deployment Hax

Sun, 28 Oct 2012 00:00:00 +0000

TL;DR

My current deployment is pretty horrible.

Idealistic Drivel

I think it’s pretty widely accepted that the ideal deployment process is the one a developer barely even realises exists. What I would really love for Heist is something like this:

Develop feature in a git branch
Merge branch into “Release” branch
Push to github
Build Server picks up change to release branch, builds code and (if it passes all tests), deploys new version of game
When a player starts the game, it auto updates

That would be pretty cool.

Tests

Here’s the biggest problem. I don’t even nearly have test coverage like this needs. The base libraries Heist is built upon are fairly well tested (although even they need more tests), but Heist itself is very poorly covered. This is partly an artifact of the kind of code that goes into Heist, game specific stuff usually to do with user interactions or UI - this kind of stuff is hard to test.

This is kind of an excuse though, in my opinion code which is hard to test is code which could be better written. So perhaps sometime I need to go through Heist refactoring stuff to be more testable, I certainly need to if I ever want any kind of automated deploy system like this.

Build Server

I don’t have one, but that’s easily fixable.

Auto Update

I do have this! Heist is deployed and installed using the “Microsoft One Click Installer”, which literally requires one click for a user to download, install and run the game. Whenever the application is started OneClick does a quick check for a new version.

All I have to do to deploy a new version of the game is upload certain files to the correct place on my server.

A Non Ideal World

So if I don’t have a shiny auto deployment and testing system, what do i have? For a while I’ve simply been manually building the game, and then copying the folder up to my server over SCP. Finally, once the upload was done, I just renamed the directories to swap the old and new versions. This works, but on my home internet connection it takes about 20 minutes to upload a new build of Heist. Every. Damn. Time. That was kinda getting annoying.

Now, obviously the change from one build to another is generally pretty small. In fact, most of the space in the upload is content which most of the time is exactly the same between builds. Delta encoding to the rescue! The difference between version 1.0.0.35 and 1.0.0.35 (generated by bsDiff) is 3Kb, which is a 99.9% compression ratio.

With bsDiff I’ve made upload time infinitely faster (approximately), but now my deploy process has gone from:

Copy Folder to server
Do something else for a while
Rename folders

To:

Download rar of previous version
Rar compress folder of new version
Generate diff
Upload diff
Execute a remote script to apply diff
unrar the patched file
Copy result into place

That’s a bit of a pain.

All Problems In The World Can Be Saved By One More Script

A boring, repetative task, which requires no human input (except credentials to upload) - sounds like a job for a script!

Here it is in all it’s ugly, hacky, glory:

"C:\Program Files\WinRAR\rar.exe" a "heistgamesetup.rar" "heistgamesetup" -m5
rmdir /S /Q "heistgamesetup"

"bin\DownloadFile.exe" "http://www.placeholder-software.co.uk/static-files/setup/.heistgamesetup.rar"

"bin\bsdiff" ".heistgamesetup.rar" "heistgamesetup.rar" "patch.diff"
DEL ".heistgamesetup.rar"
DEL "heistgamesetup.rar"

"C:\Program Files (x86)\WinSCP\WinSCP.com" /script=upload.script

DEL patch.diff

When this is run it takes the contents of heistgamesetup (which is where Visual Studio is set to publish new builds to), and rars them with the best compression mode. Then it downloads the rar of the previous version from the server and generates a diff between old and new. Finally it logs in to my server with WinSCP and runs upload.script, which uploads the patch file and triggers the remote deployment script.

Wibbly Wobbly Pipey Wipey

Wed, 24 Oct 2012 00:00:00 +0000

TL;DR

Network messages are sent in named pipes.

Topics

In this series I’m covering 5 topics:

My Worst Nightmare

So far we’ve covered how Heist gets into a session, and then how it guarantees (or doesn’t) delivery of packets. The question remains what do we actually put into packets now that we have the ability to send them? One Possible way to organise things is like this:

Function PacketDelivered(Packet)
{
    flag = Packet.ReadByte();
    if (flag == 0)
        HandleFooPacket(Packet);
    else if (flag == 1)
        HandleBar(Packet);
    else if (flag == 2)
        HandleBash(Packet);
    else if
        //And so on

That’s one way of doing things, it’s also horrible. Adding any new packet type requires manually updating this if statement, which would very rapidly grow out of control as it gained references to every networked part of your project so that it could route packets as necessary.

Of course the above can be made a little better, we could have something like this:

Handlers = {};

Function RegisterPacketHandler(Flag, Handler)
{
    Handlers[flag] = Handler;
}

Function PacketDelivered(Packet)
{
    flag = Packet.ReadByte();
    Handlers[flag].Handle(Packet);
}

That’s a bit better, now you don’t have a massive if statement to keep updating and the network system no longer has to reference every part of the entire project in an effort to route packets. It’s still not great though, each handler has to specify a flag which isn’t something it really cares about (and if two handlers specify the same flag it’s game over). The instant thought that comes to mind here is to do this:

NextHandler = 0;

Function RegisterPacketHandler(Handler)
{
    id = NextHandler;
    NextHandler = NextHandler + 1;
    Handlers[id] = Handler;
}

But that doesn’t work at all! Remember, this is a distributed system, we need the same number allocated to the same handler on all machines in the session otherwise this is all pointless. So, the RegisterPacketHandler function is going to need to specify some kind of unique ID, and preferably it shouldn’t be something arbitrary like a number.

Named Pipes

This is where named pipes come in. When something wants to register a new type of packet it creates a named pipe like so:

Pipe = Pipes.GetPipe("General Chat", true, true);

This example creates a pipe with the name “General Chat”, which is Reliable and Ordered. Now it can subscribe to the pipe:

Pipe.Subscribe(function(msg, sender)
    //received msg from sender
end);

Once subscribed, any time any peer sends a message it will get sent across the network to all suscribing peers. Nice and simple.

Strings are big, integers are not

This is where it gets complicated, strings are really big so we can’t just go sticking a string at the front of every packet in place of the single byte we were using as a flag before. A single character of a string is at best one byte long, and at worst could be 4 bytes long - for a single character!

Heist solves this problem by turning string names into unique integer IDs which it negotiates as pipes are created, an integer is 4 bytes long (and with clever tricks Heist only ever needs to send 3 bytes of it). Each pipe really has several names, the string name which everything externally refers to it as and one or more integer ID(s) which the internal piping system refers to pipes as, when a peer sends out a pipe packet it can use the integer ID as the flag instead of the string name which is an awful lot smaller. So, the basic execution of this is:

Peer 1 Creates Pipe "General Chat"
Peer 1 Assigns Pipe "General Chat" the ID 1
Peer 1 Sends a message to all peers assigning ID 1 to Pipe "General Chat"

--Time passes--

All other Peers send back message confirming ID of "General Chat" is 1

--Time passes--

Peer N Creates Pipe "General Chat", which already has ID 1 assigned
Peer N Sends message with flag 1

--Time passes--

Peer M Receives a message with flag 1, it knows this is shorthand for "General Chat"

We seem to have everything here, no matter which peer initially creates the pipe all peers use the correct ID and can talk to any other peer with the ID. What we’re missing is a subtle race condition, what if this happens:

Peer 1 Creates Pipe "General Chat"
Peer 1 Assigns Pipe "General Chat" the ID 1
Peer 1 Sends a message to all peers assigning ID 1 to Pipe "General Chat"

--Time passes--

Peer 2 Creates Pipe "General Chat"
Peer 2 Assigns Pipe "General Chat" the ID 258
Peer 2 Sends a message to all peers assigning ID 258 to Pipe "General Chat"

--Time passes--

Other Peers explode

After some investigation it turned out that every peer in the network session simultaneously exploding was not the optimal handling of this case. Luckily, I’ve already handled this! Remember I said that every pipe has a single string name and one or more integer ID(s), this is why. When a peer receives a message with an ID for a pipe it stores that ID as shorthand and sends back a confirmation to the peer which sent it, if it already has a shorthand stored for this pipe name well that’s fine, now it has two. When a peer wants to send out a message it will use whichever shorthand it happened to receive first, since all other peers know about this shorthand that’s fine and the second shorthand which got sent out is still stored and valid if a peer happens to send out a message using that shorthand.

New Release(2)

Wed, 24 Oct 2012 00:00:00 +0000

TL;DR

An new release of Heist.

Where Is It?

If you’ve never played Heist before you can get it from here. If you’ve played Heist before then just run it and it should offer to auto update (if it doesn’t, just get it again and it’ll write over the old version. Then send me an email and tell me you had this problem).

Major Features

Bugfixes in Behaviour Scripting Interface
Bugfixes in Entity Scripting Interface
New Network Protocol (saving 1 byte in every single packet sent)

How to play

It’s still not entirely obvious how to play, I’ll be working on fixing that for the next release. A few of the least intuitive things:

When you first spawn you’ll drop out of the sky and die to fall damage then instantly respawn (long standing bug)
Number keys do not toggle weapons (work in progress)
Sounds created by tools all emanate from the world origin (work in progress)

What are my guns!?

Since there’s no indicator what you have equipped yet, these are the tools:

Box Spawn (click to spawn a box on top of your head)
Sniper Rifle (click to fire, this will knock back boxes an impressive distance)
Laser Rifle (click to fire, this will fire a slow moving yellow laser bolt which looks cool at night)
Torch (Pretty useful at night)

Work In Progress

I’m currently working on a weapon indicator, which will automatically appear in your game once I have it done.

New Release

Sat, 20 Oct 2012 00:00:00 +0000

TL;DR

An new release of Heist.

Where Is It?

Major Features

Better multiplayer implementation
In Game HUD
New Scripting API for entities
New Scripting API for behaviours
Internally reimplemented tools (still a work in progress)
New Scripting API for tools
A sun with a day night cycle (entire cycle is 3 minutes)
A Laser Rifle

How to play

It’s still not entirely obvious how to play, I’ll be working on fixing that for the next release. A few of the least intuitive things:

When you first spawn you’ll drop out of the sky and die to fall damage then instantly respawn (long standing bug)
There’s no indicator which weapons you have equipped (work in progress)
When you spawn you have weapon 0 equipped, but the only valid weapons are 1, 2, 3 and 4. Rolling the mouse wheel will move you off this invalid weapon to a valid one (Fixed)
Number keys do not toggle weapons (work in progress)
Sounds created by tools all emanate from the world origin (work in progress)

What are my guns!?

Since there’s no indicator what you have equipped yet, these are the tools:

Box Spawn (click to spawn a box on top of your head)
Sniper Rifle (click to fire, this will knock back boxes an impressive distance)
Laser Rifle (click to fire, this will fire a slow moving yellow laser bolt which looks cool at night)
Torch (Pretty useful at night)

Full Changelog

This isn’t really a full changelog, it’s just a summary of the most important things I noticed while skimming over the past 2 months of git logs.

Fixed bugs in world subdivision
Faster texture atlas loading
Texture atlas will no longer consume 1Gb of memory while loading
Written some vague code exploring how to implement AI
Added in game HUD
Added a Lua to Javascript bridge for in game HUD scripting
Written multiplayer chat entirely in lua
Added a score board to HUD
Added a kill log to HUD
Supported BSON data in networking
Silently translate Lua tables to BSON for network transport
Better abstracted script sources (now called data sources)
Improved UI resource loading
Rebindable game controls (still a work in progress)
Added Script resource loading (currently only text and audio, still a work in progress)
Refactored network packet reading
Rewritten entire packet encoding/decoding system
Rewritten peer connection initialisation
Endlessly bug tested networking
Implemented a new system for synchronising network peers to the correct world state
Cut out half the code from Heist and moved it into a separate library
Achieved 52% test coverage on entire networking library (100% on some parts like encoding)
Written a load of clever bit twiddling trickery for packet encoding
Modified entity factory to handle default values, making scripting a load simpler in the default case
Refactored Configuration system to be much more robust
Rewritten Sound Renderer
Rewritten backend metaserver infrastructure
Migrated backend to a new server host
Neatened up physics body implementation (still a work in progress)
Neatened up API for entities
Neatened up API for behaviours
Rewritten every script to be in line with new APIs
Added many new input events (weapons, road vehicles, air vehicles, naval vehicles)
Rewritten texture pack format
Modified all data sources to support versioning
Changed format of data source configuration file

Say What?

Wed, 17 Oct 2012 00:00:00 +0000

TL;DR

Sending a network message doesn’t guarantee that it will arrive.

Topics

In this series I’m covering 5 topics:

Really Brief Glossary

TCP

TCP is the Transmission Control Protocol. TCP makes a socket look like a stream of bytes, bytes which are written to the TCP socket are guaranteed to arrive with no corruption and in the order they were sent.

UDP

UDP is the User Datagram Protocol. UDP allows you to send packets of binary, and the only guarantee it provides is that if a packet arrives, it won’t be corrupted. If the packet was corrupted, it will simply be thrown away before you ever receive it.

IP

IP is the Internet Protocol. IP is the basic protocol which UDP and TCP are built on top of. IP makes no guarantees, your packet might arrive, and if it does it might be (partially or completely) corrupted.

UDP > TCP

Wait, what? That title makes no sense. TCP guarantees that your data will arrive and everything, it sounds great! Why would you ever use UDP?

The problem is how TCP makes those guarantees. At the lowest level TCP is still suffering from the same problems that IP has - packet loss and corruption, some of the bytes you send may not arrive or they may be corrupted (in which case the receiver will throw them away, so corruption is the same as not arriving). The problem is that when some data doesn’t arrive, it will (sort of, find more detail at wikipedia) send back a message asking for that data to be resent and will buffer all other arriving information until it gets a copy of the data which never arrived, so TCP introduces fake latency here by holding perfectly good information in memory. It has to, of course, the guarantee of in order delivery means that it can’t release that data to the receiver until the previous data has been resent.

Hopefully it’s obvious why TCP is totally useless for games. Think about the data about player position, for example, I want it to arrive in order (so that I don’t update to an old position) but I don’t need it to be reliable, if some data is lost then just drop it and give me newer data.

Now, you may be thinking to yourself that TCP isn’t totally useless, after all some data in the game still needs to be reliable, ordered and isn’t time critical (e.g. player text chat, so long as it arrives within 300ms of sending it’s probably ok). It’s still not ok to use TCP even here, the fantastic Glen Fielder in his Networking For Game Programmers series references a paper which essentially says that using TCP and UDP side by side can actually increase packet loss for UDP.

##UDP < TCP

So UDP isn’t quite enough, and TCP is way too much. What we really want is several different transmission modes…

Reliable+Ordered

Packets will arrive, and they will arrive in order. This is exactly what TCP offers.

Reliable+Unordered

Packets will arrive, but may arrive out of order.

Unreliable+Ordered

Packets may get lost, but the packets that do arrive are in order (in practice this simply means dropping packets which arrive late).

Unreliable+Unordered

Packets might not arrive, and they might not arrive in order. This is what plain UDP offers.

To implement these in TCP is impossible, we’d have to remove some of what it does for you (which it doesn’t allow). Implementing them over UDP is perfectly possible though, so that’s what Heist does.

When Should I Use X?

The general policy I have for working out which transmission mode to use is to work out the minimal requirements for what you really have to have and use that. Using the “higher” guarantees also comes with higher costs, and should be avoided as much as possible.

How Heist Does It

Heist actually does not implement all of these new transmission modes itself, instead it uses the fantastic (lidgren networking library)[https://code.google.com/p/lidgren-network-gen3/] to abstract away a lot of the details of how sockets work. Lidgren internally has a single UDP socket, and over the top of that it pretends to maintain connections to other lidgren sockets, keeps up to date ping information and implements some very basic flow control. Of course, these fake connections offer the four transmission modes outlined above.

Get Up And Initiate That Session

Mon, 15 Oct 2012 00:00:00 +0000

TL;DR

Part #2 in my series on how Heist does networking stuff. This time about setting up a session and initiating connections.

Topics

In this series I’m covering 5 topics:

Topology
Session/Connection Initiation
Delivery Guarantees
Named Data Pipes
Packet Encoding (Architecture)
Packet Encoding (Lossless Compression/Variable Length Encoding)
Packet Encoding (Lossy Compression)

Making The First Move

There are two parts to session initiation:

How to host a new session and advertise it to the world
How to find an existing session and join it

Hosting A Session

Global Server List

When a peer starts a new game session it submits some information to a central server list and gets back a JSON response which indicates that this server will be listed for the next 2 minutes, this response includes a special control key. While the game is still running the hosting peer can use the control key to “ping” the game and reset the timer, or it can specifically delete the game from the list.

NAT Negotiation

NAT (Network Address Translation) is a technique used by routers to translate one network address space into another. For example your home router has one single IP which the general internet can talk to, and yet you (probably) have multiple computers attached to that router all with different addresses - this translation from a single public IP to multiple private IP addresses is NAT. NAT is evil.

NAT is evil because it makes establishing an inward connection really hard impossible. Here’s an example:

You have a home network, with 2 computers, one of which is hosting a Heist game
I try to join your game, so a packet gets sent to your router
Router receives a packet and has no idea which of the two computers to send it to, so it ignores it.

Fail.

So, we need some way to convince the router that it needs to send packets on a certain port to a certain machine inside the network. For this purpose the hosting peer listens to a (pubnub)[http://www.pubnub.com/] channel. Pubnub is a cloud messaging service which basically allows you to create a named pipe and then anyone anywhere in the world (with the correct details) can send a receive messages on this pipe. Critically pubnub doesn’t care about NAT because the internal computer establishes an outward connection to the pubnub servers. Once an outward connection is established breaking through NAT is possible, the connecting peer can talk to the hosting peer and they can negotiate some way through the NAT (more on this later).

Joining A Session

Global Server List

The hosting peer, as mentioned, lists it’s game session on a global server list. Logically, the place to start to find a session to join is this list. If you submitted a server at that link earlier try visiting this link now - if you read all of the above in under 2 minutes then your fake server entry should still be listed. The server list returns 7 pieces of information:

Name. The name of this session.
Game. The game which this session is part of, e.g. Heist
Port. The network port to talk to this server on.
IP. The IP of the session host as perceived by the session host, this will usually be a local IP from behind NAT.
RemoteIP. The IP of the host as perceived by the server list, this will usually be the public IP.
Tag. Any arbitrary information.
Id. A unique ID for this server.

NAT Negotiation

As I mentioned before, at this point the host server is sitting around listening to a pubnub channel waiting for something to happen. The connecting peer now connects to the same pubnub channel and sends a message to the host containing telling the host the public IP of the connecting host. Now is when the NAT punching happens, the host now knows who is trying to connect to it and where from so it also starts trying to connect back to them - the router(s) sitting in the middle performing NAT will see an incoming stream of packets on Port X and an outgoing stream of packets on Port X, it will reasonably assum ehtye have something to do with one another and will route the incming packets to the computer which sent the outgoing packets.

To be honest this is a pretty nasty hack, it works in most circumstances with your normal NAT configuration but won’t work in many others. Such is the nature of NAT negotiation though.

Why 2 IP Addresses

I’m sure you’re asking yourself right now why the server list has 2 IP addresses for the game host. Interesting question, here’s a subtle case for you to consider… What if the host and the peer are both behind the same NAT?

In this case, if the connecting peer tried to connect to the public IP of the host it would be connecting to the public IP of its own NAT. The router would get very confused because it would see an outward stream of packets from the connecting peer to… itself? And then another outwards stream of packets from the host, trying to initiate a connection back to… itself again?

Instead, Heist handles this case itself. Before trying the whole NAT punching pubnub negotiating thing Heist just tries to connect to the local IP of the server, if the connecting peer just happens to be on the sharing the same NAT as the hosting peer then the connection will be established successfully and the connecting peer will never progress to trying out all the clever NAT tricks.

Everything Is More Fun With Three People

This is all very nice, but Heist supports more than 2 peers (technically it supports 256, but only if the peers have an awesome upload rate). So, let’s pretend we have a session set up with two peers and a third peer wants to join, what does it do?

First peer3 connects to the host peer as usual. When any peer connects the host peer responds with a blob of important session information, part of which is a list of other peers in the session (with each peer having 2 IP addresses listed, same as before, so that NAT can be negotiated if necessary). After this point everything is the same as before, peer3 now has a load of IP address pairs it has to connect to if it wants to join the session and it can simply work through them one by one negotiating NAT as necessary over the pubnub channel.

http://metaserver.placeholder-software.co.uk/serverlist/Heist_Debug.json

Topological My Dear Watson

Sun, 14 Oct 2012 00:00:00 +0000

TL;DR

In depth talk about the technical details of how networking in Heist works #1

Fair Warning

Networking is a huge topic, and I’m going to be talking about a lot of technical details. This is not going to be a blog post for the faint of heart!

Where To Start?

There’s a lot to talk about with networking, I’m going to try and cover all these topics:

That’s a lot of topics though, so I’m going to cover each one in a separate blog post rather than writing one colossal post right now!

Topology

Heist is built to be a peer to peer game where every peer in the session should have a connection to every other peer in the session. This has a few advantages and a few disadvantages:

The Good:

Less Latency
No Central Point Of Failure

The Bad:

More Complex Implementation (Sort of)
More Connections

Let’s go over those one by one.

Less Latency

This one should be pretty obvious. If everyone is talking via a server then the time for a message to get from peer A to peer B is:

Time(A->Server) + Time(Server->B)

instead of:

Time(A->B)

No Central Point Of Failure

Again this should be obvious. If there’s a single central server then if the server goes down the entire game goes down, which sucks.

Now, saying Heist does not have a central server is slightly misleading. There is a “special” peer in the session which is the “host” peer, this is the first peer that new peers connect to when they join the session, it’s also the peer that lists this game session on the global server list and keeps it up to date. This is hardly a central point of failure though, migrating the responsibilities of this peer would not be terribly difficult to implement (it’s planned). Even without migrating the responsibilities when this peer drops the session will continue on for the rest of the players.

More Connections

Again very obvious, every peer has to maintain (N-1) connections to all the other peers, instead of just 1 connection to the server. Establishing a connection between two peer is hard work, there’s a whole laod of different ways in which it can fail and establishing N-1 connections is a lot more hard work. Fortunately I don’t really care, it’s hard to code but it’s something I only have to do once and then I can reuse this logic every time I want to establish a new connection.

More Complex Implementation

This is the biggie, the really really big biggie. With a peer to peer implementation everything is more complex. There’s no single place you can point to and say “That’s the state of the world, synchronise everything to it”, instead the state of the world is spread across all of the peers, they all “own” entities arbitrarily spread out across the world and they all need to interact these with other entities they don’t necessarily own. If this sounds complicated that’s good, it is.

It’s complicated in a way that I think is good though, the solution I have adopted splits actions up into three parts:

Check State, and decide what to do (this runs on the peer which owns the entity in question)
Receive a message of what to do, and do it (this runs on all peers with a copy of this entity)
Apply error correction algorithms over time to make sure things stay in step

This is really highly decoupled, and it’s been proven a few times in my experiences programming Heist that this kind of decoupling can come in useful for other things. For example, I can get in there and trigger an action in other ways, just by sending out the correct type of message and pretending it was from the state checking code.

I Lied To You

The topology of Heist isn’t actually fixed. The game is built in such a way that peers understand the fact that some entities are owned locally, and some are owned remotely. Well, in Client/Server no entities are owned locally, and all are owned remotely (by the peer), this is a situation which Heist understands. This is the real reason that Heist is Peer to Peer, it could be turned into a Client/Server setup with a fairly small amount of work in session initiation and entity ownership code. I already have some ideas in mind for long running Heist servers, where a very low power machine can host the world and basically just act as a packet switch and a database which sends packets to the appropriate peers and will send out the current state of an entity when a peer requests it - with these very simple changes Heist suddenly has persistant world servers!

Omg Wtf Multiplayer

Sat, 13 Oct 2012 00:00:00 +0000

TL;DR

The next major feature of multiplayer is mostly done.

OMG!

I have recovered from the RSI I spoke about last time, which is good because I was actually starting to go crazy due to lack of coding! Since then I haven’t said anything on my blog, because I’ve been in a crazy coding frenzy to make up for lost time. My focus has been split between two tasks…

The Other Game

I mentioned last time that I was working on another game project with some friends using a lot of the technology I’ve built for Heist to rapidly build another small game. That sort of fell through, turns out not everyone is as into making games as I am (who would have guessed), the project is still around but now I’m working on it with a more experienced programmer who is as in to making games as I am.

This project is proving to be incredibly useful for Heist, which is excellent. Thanks to the smaller scope of this project some things which are not clear within the complexity of Heist become much clearer, this leads to improvements in the base code library both games depend on and thus leads to improvements in Heist. The big improvements have come with multiplayer…

Multiplayer

Multiplayer is something that I have been working on for Heist on and off for months, it seems like every time I complete a major feature in Heist my next major task is more work on multiplayer! A vast amount of work for the new project was picking the multiplayer code out of Heist and making it into something more generally usable in lots of games, of course this is a good thing because it means I only have to solve these hard problems once and then I can use the code for multiplayer in all my games.

I’m going to do a blog post later/tomorrow detailing a whole load of stuff about the networking system I’ve built, overall I’m quite happy with the abstraction I’ve built although there are a few improvements I’d like to make which I might talk about too.

WTF?

Getting an abstraction for multiplayer is half the battle, the other half is getting it integrated with Heist. When you start a game in Heist you will have a load of active data sources, these sources supply scripts for various things in the game (e.g. the World Scripts which define the buildings in the world), of course if you join a multiplayer game it is critical that you have the same set of active data sources as all the other people in the game otherwise crazy-weird-stuff (tm) will happen. So what Heist does is when you join a game the host sends an XML document a bit like this:

<sources>
  <source type="World" protocol="hg" name="WorldScripts">
    <path><![CDATA[https://martindevans@bitbucket.org/martindevans/worldscripts]]></path>
  </source>
<sources>

This document simply lists all the sources the host has enabled and then the joining client can simply go and fetch all those sources (in this case using “hg”, the Mercurial version control system). There is an interesting hidden subtlety here, what happens if you start a server and then the data source is updated? For example you start a server using the bitbucket worldscripts data source, and then I push a new load of scripts to bitbucket. The host won’t have these new scripts since data sources are only pulled when a game starts but the client will since it joined later. To solve this problem Heist also sends along the “version” of each script source, for Mercurial this is simple (it is a version control system, after all), I then make sure the client has the same version as the host and all will be good :D

BBQ!

The end result of all this work is this:

This screenshot is taken from my laptop, down there is a player running on my PC (waving a torch about). Top left you can see the chatbox, added entirely through scripts, working perfectly.

Coming Soon

Some details of the technical internals of exactly what the networking system does and how it does it.

Full Steam Ahead

Mon, 24 Sep 2012 00:00:00 +0000

TL;DR

Don’t get RSI, it sucks.

Remorseless Self Pity

As I briefly mentioned a couple of weeks ago, I’ve had RSI these last couple of weeks. It seems that years of gaming and coding finally caught up with me with a vengeance and forcibly put me out of action. This was some incredibly bad timing, since I’ve had RSI we’ve had:

An Expensive New Gaming Mouse
Lots of Coding Time
Guild Wars 2
Lots of Coding Time
Battlefield 3: Armoured Kill
Lots of Coding Time
Borderlands 2

You get the idea, I would like to do some of the above. Preferably I would like to do all of the above! Anyway if you’ve been feeling some slight twinges in your wrist, arms or back while gaming/coding recently then for your own sake go and do something else! My solution (once unable to use the computer) was to watch anime while doing pressups and weightlifting to strengthen my arm muscles, multitasking FTW!

Remorse

I haven’t been completely idle these last three weeks, people who know me probably realise it is physically impossible for me to not write code for three entire weeks. Instead of writing new code, I’ve been reading code, in two ways.

Refuctoring

I mentioned last time that I was going over the networking code, writing a few unit tests. I’ve gone a little bit beyond that now, in the last week I’ve pulled out 6000 lines of code from Heist and moved them into a separate library (Called Placeholder.*) for building games - that’s about 20% of the entire of Heist modified in a single week. During this time I have:

Redesigned the networking system
Redesigned the packet serialisation system for networking
Achieved 100% test coverage on packet serialisation
Completely rewritten the audio renderer
Completely rewritten the input system
Lots of polising of existing code
Fixed endless bugs
Written a library which is easily usable in other game projects of mine

The new library is better tested, better designed, has neater abstractions and most importantly is designed for re-use in more games! I’m certain I would not have done this if it were not for the RSI, which honestly doesn’t in the slightest make up for weeks of pain, but it’s a nice thought.

I Can Barely Type, What I Need Is Another Game Project!

Logic! No, honestly, that really is logical. I started a new project with a few friends of mine, all programmers of varying levels of experience who were interested in making games. We’re remaking a game I made ages ago in Java, this time using C#. The important thing about this project is I’m not writing much code for it, I’m just the project leader (see, I told you it was logical). This is intended to be a short project, I’ve set a goal of getting the game complete enough to get it onto steam greenlight by christmas, which actually shouldn’t be a terribly difficult goal. I’m sure sometime around new year I shall write a summary of the project, and what I learnt about project management doing it. I’m also sure I’ll mention the game long before then, when we have something demoable in a month or so I’ll put up some pictures here.

The hardest part of the project for me so far is simply bringing the team up to speed, the problem is we’re using a lot of technology I’m very familiar with:

C#/XNA (My primary language for 5 years)
Myre (I’ve been involved with this for over 3 years)
Placeholder.* (Basically all of my time coding over the past 8 months)

A lot of the basic setup of the project (the stuff we’re doing now, at the start) is plugging together components that are already written into Myre or Placeholder.* which isn’t what anyone is really interested in when making games. It’s a fun challenge.

MORE GAMES

The coolest thing that’s come out of all this is that I realised that I have a whole load of really awesome technology for making games really quickly - I reckon I could throw together a simple game in two or three days, even with network multiplayer included, and have it be fairly polished. My mind is abuzz with ideas for games, it’s a fun time to be a game designer! Once I properly get recovered from RSI and back to games I suspect I shall do something like a monthly game jam - build whatever game is on my mind at the moment, with a time limit of 36 hours coding time over three days, and just see how it plays and if it’s worth developing into something more.

You Should Check Out Greenlight

Sat, 01 Sep 2012 00:00:00 +0000

TL;DR

Greenlight needs a lot of work, but you should check it out anyway. Yes, you.

WTF is Greenlight?

Greenlight is a new service released on steam which empowers the gamers to decide which games get onto steam! Obviously valve still control the big releases on steam - Greenlight is all about indies gaining community support, getting feedback from fans and then getting enough support to get the game onto steam. It’s a pretty simple concept, gamers upvote games, comment on them and once a game has enough votes valve put it into steam.

For someone like me this is really cool. No longer do I have to gain community support myself, and then extract feedback from them, and then submit my game to an opaque steam submission process. Instead of all that I can put my games onto Greenlight early in their development, get instant exposure to the steam userbase (pretty much more exposure than I could ever hope to get otherwise), this will get me lots of feedback and the progress to getting enough votes to get into the steam store is right there on the front page for my game.

What’s Not Totally Awesome About This?

Right now greenlight is not perfect, far from it! The only filtering available is by a limited selection of genres and there is no sorting available at all! The store is getting pretty crowded (right now there are 658 items waiting for my rating and it’s only been open about 48 hours), combine such an incredible rate of games being added with no sorting and filtering and discoverability through the store is basically nil. That sucks but it’s not the end of the world - I can view items my friends have rated and anyone can create a new collection to pick out a related set of games they feel people might like - it’s not fantastic but it’s a start on discoverability and valve have more to come for greenlight.

A Little Something To Get You Started

Since discoverability is the big problem with greenlight right now, here’s a few games I discovered for you (I think you need to be logged in to steam to see these):

[The Impossible Game] (http://steamcommunity.com/sharedfiles/filedetails/?id=93339572) I like a challenge
[NEOTOKYO] (http://steamcommunity.com/sharedfiles/filedetails/?id=92931910) First person Sci-Fi shooter
[Receiver] (http://steamcommunity.com/sharedfiles/filedetails/?id=92927535) First person shooter puzzler thing
[No Time To Explain!] (http://steamcommunity.com/sharedfiles/filedetails/?id=92922559) Platformer puzzler with time travel
[Project Zomboid] (http://steamcommunity.com/sharedfiles/filedetails/?id=92916900) Zombies survival RPG
[The Intruder] (http://steamcommunity.com/sharedfiles/filedetails/?id=92910126) First person horror game
[Project Giana] (http://steamcommunity.com/sharedfiles/filedetails/?id=92852223) Beautiful looking platformer puzzler
[Dream] (http://steamcommunity.com/sharedfiles/filedetails/?id=92647949) Atmospheric Exploration

Why aren’t you on Greenlight then?

I had originally planned to put heist on greenlight on the release day, with some kind of playable game mode available for download (probably multiplayer deathmatch). The FAQ says:

Can I post my game concept or early builds? Absolutely! We encourage you to post information about your game as early in the development process as you are comfortable with. Greenlight will let you define whether you are posting your game as a concept/early build or as a playable game that is nearing completion.

However, when I came to submit Heist the submission form told me that during the start up period of greenlight the game needs to be almost complete.

Although I was a bit disappointed I think this is actually a good thing. Greenlight is currently getting about 10 games submitted every hour and the store discoverability systems can barely handle that, imagine the complete madness that would have ensued from every developer with a half realised concept adding their games to greenlight on release day!

The other aspect of this is that I missed my deadline, everything was going well until I developed RSI and my work rate hit the floor. I spent the last week doing some long needed refactoring of the networking and writing nice short unit tests, I suspect this blog post makes up for a significant percentage of all the text I have typed this week! To be honest it has been a long term win because code quality in some areas has improved quite significantly, but no real progress has been made towards a playable game this week.

It's Full Of Entities

Mon, 20 Aug 2012 00:00:00 +0000

TL;DR

I talk about how Heist holds its view of the world in memory.

Background

I was planning to talk about my progress on the in game User Interface this week, but as I planned out the post in my mind I hit a bit of a wall - my solution to loading the UI revolves around loading certain things into the game in a certain way which turns out to be really neat - but if you don’t understand how Heist manages the world it’ll all go over your head! So this week I’m going to explain some of the internals of how Heist manages the world.

The Scene

The Scene is the entire world. Literally everything goes into the scene, characters are in the scene, lights are in the scene, even cameras are in there. The scene is essentially the single store of state in the entire game, if there is a piece of information about the game that changes over time it’s probably stored in the scene. This means that all the parts of code which make up the world (e.g. the rendering engine) simply grab the bits of the scene they need every frame and do their thing (e.g. Render the next frame), which is conceptually nice and simple.

Entities

Of course, the scene isn’t just a random collection of data thrown together, that would get very messy very quickly! Rather, data is stored in objects called entities. Each entity has a load of properties and a load of behaviours.

Properties

Properties are the only place to store state in an entity, everything which changes about an entity is stored in a property. For example, if you have a player entity it would have properties for position, velocity, rotation, remaining health, total health, health regen rate etc etc. Properties have a name, so various parts of the system can all retrieve the same property from the same entity and will all be reading the same value. Conversely, if something gets the property and changes it, then everything else which has retrieved the same property will now get the updated value.

Behaviours

Properties just store data, of course we need something to read and write that data data - that’s where behaviours come in, they describe how an entity acts in the world. So let’s say, for example, we have an entity with 3 properties:

Position
Velocity
Mesh

This entity has all that we need for a bouncing ball, but as it is it won’t do anything, in fact it won’t even be rendered! But if we add some behaviours to it:

Renderable
GraphicsMesh
PhysicsMesh
PhysicsObject

Suddenly this ball is visible (Renderable+Graphics Mesh), it will collide with things (PhysicsObject+PhysicsMesh), and it will move (PhysicsObject)! Magic!

Isn’t this all a bit complex?

Some people might be thinking this sounds a bit complex, but actually it simplifies the system. Look at the last example, the Physics system and the renderable system don’t know anything about the existance of the other, but they work together to render a moving ball because they’re both working on the same set of properties and as physics updates the position then the rendering system updates what it draws.

Other stuff

In truth there are a few other parts to the scene. For example, there are “Behaviour Managers”, one for exach type of behaviour, which holds a list of all behaviours of that type - this is use for things like the renderer, which needs a list of all Renderables to draw. That kind of stuff is an implementation detail though, and so far doesn’t expose itself to the scripting system at all because I don’t think it’s needed.

Open Source

If you’re interested in the details of this system then you’re in luck, the entire entity system used by heist is an open source project which you can find here on github. It’s a bit light on documentation at the moment, which is something I plan to fix, but in the meantime if you have any question feel free to drop me an email or leave a comment below :)

Persistence

Mon, 06 Aug 2012 00:00:00 +0000

TL;DR

I had an interesting discussion about heist multiplayer with a friend of mine and we realised that Heist could actually largely be played in asychronous multiplayer.

An Important Note

In previous blog posts I’ve talked about Heist in the general sense, which has been slightly misleading. Heist is two things:

A game engine which supports scripted gamemodes
A specific gamemode

In this post, I’m talking specifically about the gamemode called Heist which is obviously something to do with stealing stuff. I’m not certain on how the Heist mode is going to play, So don’t take anything here as set in stone

Persistent Characters

One thing that I really like in games is long term persistence, having a long term character profile can add a lot to the experience of a game. For example, Left 4 Dead is a really good game, but DayZ communicates the experience of surviving a zombie apocalypse a lot better because you have a persistant character and death really becomes important instead of a minor inconvenience. Of course persistant characters don’t suit every game, they only work when the experience you’re trying to communicate actally involves long term importance and significance.

Asynchronous Multiplayer

This is the idea that two people don’t have to be playing a game at the same time to play together, for example postal chess, “draw something” and “(Frozen Synapse)[http://www.frozensynapse.com/]” all allow two people who are never playing at the same time to play together. Any turn based game can be made into an asynchronous game trivially… Heist isn’t turn based, but I’m not one to let the impossible stop me.

Let’s Play

Conventionally a Heist has several stages:

Pick the target
Scout out the target
Come up with a plan
Get equipment
Pull off Heist

Only one of these stages (#5) requires everyone to be online together, all the others can be done whenever people want. Picking a target and scouting it out require only the same city (which is easy, cities are procedurally generated off a seed). Coming up with a plan doesn’t even require you to be in the game. Getting equipment could also be done out of the game, through some kind of web portal which allows you to spend your hard earned cash on new equipment, alternatively you could go play in single player to get some equipment, and thanks to persistant characters you’ll still have that equipment in multiplayer.

I Thought You Were Working on AI?

I am, but it’s really hard, I have a lot of choices to make about:

How to modify the city system to generate AI data
How to write the scripting system for AI behaviours
How to synchronise hundreds of AI characters over the network
How to simulate hundreds of AI characters without melting your CPU

I am making progress but I don’t want to blog about it until I have made some solid dicisions.

I'll Be Back

Wed, 01 Aug 2012 00:00:00 +0000

TL;DR

I’m working on big stuff for AI right now, which means I just can’t keep up with a release every two weeks

No More Fortinghtly Fun :(

I started doing these releases every two weeks when the city engine was approaching some form of completion, it’s been very useful having a target to motivate me through the stages of polishing up a few bits of the city generation. However, for now, the city generation is done (there is more to be done on it, but it’s not a priority) and releases every two weeks are basically impossible right now. I need a few weeks to sort out exactly how AI is going to work and then lay out a lot of groundwork before anything changes in the game.

Progress

It’s not all bad news though, I’m making good progress on AI. I’ve been working on a prototype for crowd AI and I think I’ll have that finished later today, once that’s done I’ll begin work on the lua scripting system for AI. My aim is to have a top down 2D world with AIs running lua scripts by next week, the system should run in a way that new lua scripts can be dropped in and they automatically start getting used by the AIs as they’re needed. The idea is that scripts find others scripts to do work for them, so my example last week was:

Get Lunch

Run a behaviour to search the generated city for places to get food
Run a behaviour to walk to the place 2a. Run a pathfind behaviour 2b. Run a follow path behaviour 2bI. Constantly keep running navigate crowd behaviours
Arrive at place to eat, query the building for suitable scripts to run

This system allows a lot of flexibility. In stage one, it searches the city for places which publish themselves as somewhere suitable to eat, so someone could write a new script to generate a new type of place to eat and suddenly AIs would start using it. Stage three retrieves a behaviour from the place itself, this means that AIs in different places will act differently and appropriately, so for example in a coffee shop they’ll queue up and collect an order from the counter but in a restaurant they’ll sit down and wait for someone to take their order.

Topology

Implied above is some kind of markup on the buildings of the city, certain buildings can “publish” themselves as suitable for certain actions (e.g. get food) and then AIs can query the building for suitable behaviours. Essentially this is another part of city generation, at the moment the topography of the city is generated but not the topology. Implementing this will be the next big task after the AI scripting system is done, changing the city generation system to add some concept of topology to the city.

Next Release

I do still intend to make extremely alpha releases whenever I can, but the next one will take a while! I imagine it will be a fairly boring hand build environment with a few markers scattered around (“work”, “get food”, “home”) which the AIs will navigate between as they wish, obviously navigating around each other in a realistic looking crowd.

Artificial Stupidity

Tue, 24 Jul 2012 00:00:00 +0000

TL;DR

I’m working on Artificial Intelligence for Heist at the moment.

The Soul Of A City Is In Its People

I’ve spent the last few months putting in a lot of effort to building the city in Heist, building the systems required for a nice scripting system which allows you to construct buildings with geometric brushes and then building all the systems behind the scene to manage all that geometry, and generating it as fast as possible so the player never encounters non generated world. That’s been fun (I should probably blog about how it all works sometime), but as the title says The Soul Of A City Is In Its People - my virtual cities don’t feel like cities at all if there are no people there!

Aspects Of AI

There are two aspects of AI to consider for Heist:

The first aspect is the quantity of the AI - I need a good way to simulate large crowds wandering around. This doesn’t actually need as many AIs as you might think, when you’re down inside a crowd in a first person view you can’t see very far thanks to all the people in the way - so long as there is always a crowd immediately around the players they’ll never know the difference. So the system will be constantly spawning characters out of the players view and then making them walk past the player and around a corner where they disappear.

The second aspect to consider is the quality of the AI - a single character needs to be realistic and contribute to the feel of the city. If the player were to pick a person out of the crowd and stalk them for a while then that person should act realistically, e.g. Go to a local café at lunch, head home at the end of the day, fetch their car from the car park, go to a house somewhere in the city and hang around in it for a while, go to bed at night etc. This is pretty much completely opposed to the number of AIs, it’s not possible to simulate an entire crowd of AIs with that much depth and complexity! That’s ok though, as I mentioned before the system will constantly spawn AIs around the player and despawn them when they’re far enough away from the player. This means that no single AI will ever be near the player for very long, if they are near the player for a long time then the system can dynamically “upgrade” the AI to add more complex behaviours to it.

Quantity

There are various systems for simulating large crowds of entities and handling all the jostling for position that goes on in such a system. One is [Continuum Crowds] (http://grail.cs.washington.edu/projects/crowd-flows/continuum-crowds.pdf) which simulate the crowd as a flow field, sort of as if it were a kind of liquid. Variations of this system can simulate huge crowds. These are the kinds of crowd used in assassins creed, which is one of the few games to simulate an entire city like Heist. Supreme Commander 2 also uses the same system for controlling vast armies of robots of varying sizes and speeds moving around one another.

Another system I looked at is called (OpenSteer)[http://opensteer.sourceforge.net/], this system is a library which helps programmers rapidly build AI “vehicles” for games. OpenSteer can’t handle crowds of tens of thousands like the Continuum Crowds, but it’s significantly simpler to understand and there is already a (port to C#)[http://sharpsteer.codeplex.com/], which means it’s a lot less work for me to use. The port was unfortunately somewhat out of date, and had a few little issues I wanted to fix (bringing it better in line with .net conventions, and fixing some bugs in the process), so it looks like (I’m now maintinaing the C# port)[https://github.com/martindevans/SharpSteer2] of OpenSteer! See a video of it in action here, simulating a crowd of ~150 people.

Quality

Quality of AI is a bit harder to handle than quantity. Right now I don’t have a complete plan for how to do AI, I do have some ideas of doing it like the city generation so people write small self contained behaviours for AIs in lua, which use other behaviours so that the system works together cooperatively. E.g. A “Get Lunch” behaviour could run the “Get Lunch From Cafe” behaviour, which in turn runs the “Find Cafe” and “Walk to Objective” behaviours, which could be using the “Find Path To Objective” and “Navigate Through Crowd” etc etc. Coupled with a hierarchy of needs the AI could just keep invoking scripts like this to satisfy needs and that may be enough to get a convincing AI. That’s next weeks task!

Overindulgence

Mon, 16 Jul 2012 00:00:00 +0000

TL;DR

I fixed up graphics and have made a new release. Download it from [the usual place] (http://placeholder-software.co.uk/setup/heistgame/publish.htm).

Graphics!

That’s right, Heist finally has some graphics which aren’t black and white! At the moment the graphics don’t actually look that great, but that’s mostly because I’m a rubbish artist and only made 2 materials - concrete and brick. Once you get into a city you’ll see everything is mostly greyish white, that’s concrete (look closely and you can see some swirly patterns in it which look kinda like poured concrete). A few buildings are a pale blue colour, which is brick. I’m not to sure why I made bricks blue to be honest… it’s been a long week ;)

Graphics are not quite finished right now, and I’m going to leave them unfinished for a while - there are more important tasks to get Heist to something playable right now.

Ditch The Diffuse

The way in which city materials are drawn is inspired by [this essay] (http://artisaverb.info/DitchingDiffuse.html). Normally games texture things by an artist painting a diffuse map, which contains all the surface details in one texture including little bits of damage and grime applied to the surface. The problem with this approach is that surfaces cannot share a texture unless they also share exactly the same patterns of grime and damage. My approach is that the game loads in several maps, a height map defines the shape of the surface, a damage map defines how the surface might be damage (and this can be blended in differently for each surface), and then a gradient map defines how to turn the damaged heightmap into a coloured diffuse map. In theory this all means that an artist only has to define each surface once, they simply need to split damage, height and colour out into three textures instead of cramming them all into one as usual.

Texture Packs

Heist now actually supports custom texture packs, which is kinda cool (maybe a better artist than me could make some non blue bricks).

Loading A Custom Texture Pack

Right now there’s no menu for selecting your custom texture pack, so if you want to use one you’ll have to drop the pack into C:\Users\USERNAME\AppData\Roaming\Heist\TexturePacks then go into C:\Users\USERNAME\AppData\Roaming\Heist\Configuration.ini and change TexturePack= to TexturePack=PATH-TO-PACK.zip, now the game will load in your texture pack (If the game can’t find your texture pack, it will load the default one).

Creating A Custom Texture Pack

A large part of my work this week has been making a tool to create texture packs from a set of source images. This tool takes a load of 512x512 images and packs them up into a single texture atlas for the game to use, it also optionally uses the incredible [pngcrush] (http://pmt.sourceforge.net/pngcrush/) program to crush the images down to a much smaller size so that texture packs don’t get too big. The tool also generates normal maps off the input heightmaps, so you don’t have to generate them yourself. You can download the TexturePackr tool [here] (http://placeholder-software.co.uk/setup/TexturePackr.zip), there’s an included readme with detailed instructions. If you make a texture pack, I’d love to see what you do! Send me pictures, or even just the pack itself so I can have a play :D

Greenlight

I might write a full blog post about this sometime in the week, but Greenlight looks brilliant, I’m aiming to get something playable by the time that’s out so I can be on there at launch with something interesting for people to try.

Some Time

Now that I have a texture atlas system in place it actually wouldn’t be that much more work to implement a Sparse Virtual Texturing system (otherwise known as MegaTexturing). I’m not certain a SVT system would actually be terribly useful, but I will look into that sometime. Not soon though, a SVT system wouldn’t get me a signficant way towards anything playable, which is the most important thing right now.

Next Time

I’m going to be looking into Artificial Intelligence for the next couple of weeks. AI is incredibly important to Heist - lots of buildings is only half of a city, the other half is hundreds of NPCs wandering around doing their own thing and reacting to the player in believable ways. Obviously the AI system is going to be scriptable in Lua, I’m thinking some kind of modular system which allows you to define a single behaviour and some triggers, so the NPC keeps transitioning between a load of different behaviours based on environmental triggers. My aim for the release in two weeks is currently a load of plain old NPCs wandering around, and a few zombies who wander around threatening and infecting other NPCs - which sounds like a good AI test and something vaguely playable all rolled up into one.

Not My Cup Of Tea

Mon, 09 Jul 2012 00:00:00 +0000

TL;DR

I made terrible progress this last week thanks to graphics programming being horrible.

What Went Right?

A few things did go right this week. I updated the scripting interface to handle coercing of types between lua and C# in a more consistent way and it has turned out to be fantastic to work with, hopefully no one will ever notice this because it just does what you expect! Another thing was adding a torch tool to the player, this turned out to be trivially simple to create the tool and add it to the player inventory. Basically I now have more confidence in my scripting system, which is nice, but it wasn’t what I was really working on.

What Went Wrong?

Lights

They are still way too dim, and I have no idea why! This isn’t really a bug so much as an artifact of bad documentation for the renderer and I haven’t spent very much time looking into it.

Texture Atlas

This is the biggie. A texture atlas is basically all the textures to apply to the city squashed together into one single texture, this means I’m only really appplying one texture to the city, I just apply different bits of it to different parts of the city and it looks like lots of textures. The obvious problem with such a system is leaking of data between the textures, just around the edge of one texture I might accidentally sample pixels from the next. There are standard fixes to this kind of problem (make all textures sized to a power of two, have a single pixel border around the egde of each texture, don’t use linear filtering)… none of which have worked! I’m currently totally at a loss as to what might cause these borders.

The Light At The End Of The Tunnel

Luckily, whatever is causing these borders is probably my fundamental ignorance of anything to do with programming proper graphical things. Once I discover what the problem is I expect it will be trivially easy to fix.

Artsy Stuff

Mon, 02 Jul 2012 00:00:00 +0000

TL;DR

No release this week, I’ve been experimenting with art styles for the game and haven’t quite settled on anything yet.

Painfully Minimalistic

One thing a lot of people have said to me when trying Heist is that they loved the art style. To be honest, this came as quite a surprise to me, the style I had in Heist wasn’t a deliberate choice so much as throwing the first thing that looked reasonable on screen! If you’ve played the game, you’ll recognise this kind of thing:

This week I’ve been playing with establishing a rendering system for the game, all the time I was working I tried to keep in mind that the end style should be something minmal. Why not just keep what I already had? Even though people like it, it just won’t scale up to a full game, long play sessions would start to get very boring in a plain white world like this.

An Unpleasant Texture

My first task with establishing a new rendering system was texturing, the old world was rendered like that because I didn’t know how to apply textures to my procedurally generated world. As it happens I haven’t finished this task completely yet and that’s party why I’m not making any release this week.

Textures are applied to a model by supply “texture coordinates” to each vertex in the model, these coordinates tell the graphics card which part of the texture to draw for each pixel on screen. Normally an artist will flatten out a model and calculate a texture coordinate map as part of the content creation process, the problem with this is my models are all procedurally generated! Luckily Heist doesn’t actually need such a detailed texture map, because of the minimal style I’m going to be going for textures will be quite subtle and suggestive of surface detail rather than containing a lot of actual detail - this allows me to sort of splat textures over models in any way and it’ll look ok. In the end I decided to use a trick inspired by the way quake generated texture coordinates for levels…

Texture Coordinates

A quick aside for a load of badly explained maths:

This trick from quake is actually pretty simple. Every vertex has a 3D position in space, and I want to convert this to a 2D position on a texture, essentially I have to decide which coordinate axis is the “least important” and drop that. First I take the Normal vector of the plane of the face this vertex is part of, since the normal points out from a flat plane clearly I want to paint the texture across the normal, but which orientation do I want? To decide on orientation I walk through all the edges of the face and pick the edge which is most closely aligned to a vector pointing directly west, this gives me an arbitrary alignment for textures, and textures on similarly aligned surfaces will tend to be roughly aligned with one another. Now that I have 2 vectors, one defining alignment and one defining orientation of the texture I can simply project the vertex positions on this plane and the coordinates drop right out! Using my test texture (rainbow coloured bricks), this got me something like this:

That Looks Terrible

Yes, it does. The texture fades into a horrible mess of random noise in the distance, this is texture aliasing caused by trying to draw a high resolution texture in just a few pixels (basically Nyquist says no). The solution to this problem is just to turn on mip maps, so as distance increases and we try to pack the texture into ever smaller areas of screen we actually draw a smaller (lower detail) version of the texture. This is a visualisation of the kind of effect this has:

I’ve split the texture up into 3 parts: red is the high detail part, green is the medium detail part and blue is the low detail part. Multiplying by the original minimal style to get edges back again, we end up with something like this:

Much better, the textures fadeout to a flat colour in the distance instead of crowding in too much detail and looking ugly.

I’m Beginning To Hate Rainbows

I have to admit, I lied a bit earlier when I said that I was just painting a raindbow brick texture over the scene - this is actually two textures! The first texture is a grayscale heightmap which defines the shape of the bricks, the second is a rainbow. The texture mapping shader actually picks the colour from the second texture based on the height of the first (this is apparently called gradient mapping. This two step procedure is useful, because it allows various colour effects to be applied to the tiny little colour map and suddenly they’re applied to the entire scene, also if you want to change your bricks from rainbow coloured to some other colour you just swap out the colour map and keep the brick map the same. This means you can get something like this:

Brown, Now we’re talking Next Gen! Here you can see that I’ve swapped out the rainbow for a grey/brown colour map and got a totally different look without changing the brick texture at all. You can also see a couple of other cool effects in here. Near the camera at the top right are some cracks, these are dynamically blended in according to a per vertex damage amount, which helps break up the uniformity of things a little. Also in the distance I’ve blended in the original effect, just to see how things have changed.

One thing that I haven’t done yet is multiple textures, at the moment everything is brick! That’s my next task.

A Heavenly Light

Once I had textures, the next thing to do was lighting. This is actually pretty trivial, the graphics engine I’m using includes various different types of lights so all I have to do is create a light entity and attach it to something. Let’s try attaching it to the player:

The really terrible colour banding in this picture is because the torch is absurdly dim at the moment, and I had to shift the brightness way up to see anything at all. I’ll be looking into why that is later.

Shaken, Not Stirred

The problem I have here is the game is starting to look like a conventional game but with crappier art. Actually right now that’s a pretty accurate summary of the looks! Remember back at the start I wanted to keep the minimalist look. One thing that would help with that is a more minimal texture, the brick texture is quite noisy (and incredibly ugly). Another thing that would help is a shader which automatically makes the scene a bit more stylised and minimal.

A Kuwahara filter is an effect which makes a picture look “painterly”. It does this by looking around the current pixel in little boxes, it then picks the average colour of the box with the least variance. This means you pick the colour which is defining a solid region of colour nearby, which means that you don’t lose edges - which are the details that really matter! The other great thing about this shader is it’s a constant effect in screen space, which ameans in world space it has more of an affect on far away things - it acts as a kind of depth blur which pulls the player focus in to nearby things.

So here’s my test image:

Here’s the image with the plain Kuwahara filter applied:

And here’s how I imagine using it in game, with an edge detect applied:

I haven’t shown you in game images of this filter because it wouldn’t be very useful right now, the game doesn’t have enough detail for the filter to actually change very much. Hopefully that will change once I have various different textures painted onto the world.

Next Time

I’ve still got quite a lot of work to do with graphics. Hopefully I can get a release out next week with a variety of better looking textures, a torch which puts out a bit more light than a geriatric firefly and a Kuwahara shader making it look all painterly.

This Was A Triumph

Mon, 25 Jun 2012 00:00:00 +0000

TL;DR

I improved rendering performance a lot.

It’s Hard To Overstate My Satisfaction

Heist, obviously, has a huge amount of geometry it needs to draw every frame - ideally the game should be drawing an entity city full of buildings and rooms every 16 milliseconds!

But There’s No Sense Crying Over Every Mistake.

So the first approach I took with Heist was a very carefully considered and well designed approach to rendering I like to call, “RENDER ALL THE THINGS”:

Essentially, I dispatch the geometry of the entire city as one massive render call every frame. This actually isn’t as stupid as it may sound, multiple render calls can get expensive and the GPU is good at pushing large amounts of geometry, especially in a deferred renderer like Heist uses. The other advantage of this approach is that it’s really simple to implement; there is one massive vertex buffer which data is thrown into, and then I just render that buffer.

And Tore Me To Pieces.

It’s common sense when rendering stuff not to render things which aren’t on screen! This was the flaw with the old system, it’s simple and only requires one single draw call but it draws everything all the time - even things which are behind you. So this improvement of mine introduces culling to the city, the system only draws things which are within the camera viewport. This was actually a fairly simple change, the Myre.Graphics engine already does culling if you present the data to it in the right way (which I wasn’t before), once I’d split the city model up into lots of entities (still sharing the same buffer behind the scenes) suddenly performance increased as the engine automagically took advantage of this and started culling away things out of view!

I Think I Prefer To Stay Inside.

~~I still have some tidying up to do on this code, so I haven’t released it yet. I will be pushing out this update as soon as possible (hopefully within a few hours).~~

Turns out I didn’t have as much work to do on this as I thought I did, it is now uploaded and ready for you to try.

Hopefully once I release it you’ll see the general performance of the game increase a huge amount - to me it feels a lot smoother on my high performance gaming PC, and it even runs smoothly on my laptop which before only managed ~10fps!

Multiplayer Release

Mon, 18 Jun 2012 00:00:00 +0000

TL;DR

I’m releasing multiplayer in a very early form, it should work for many players but has only been tested with 2. Download it from [the usual place] (http://placeholder-software.co.uk/setup/heistgame/publish.htm).

Release Often, Break Things

As promised last week I’m releasing the first early version of multiplayer for Heist this week. In this release are a few other things too:

Multiplayer Sessions
Tools And Guns
New Menu System
Fixes For Bugs Reported Last Time
Performance Enhancements
Overhauled Server List
A New Character Model
More Reliable Handling Of Script Updating

The top three are the most interesting ones of course.

Multiplayer Sessions

Setting up or joining a multiplayer session should be relatively simple. To host simply:

Click Multiplayer > Click Host > Enter a session Name > Click Host > Enjoy

To join, click Multiplayer and there is a list of all currently running sessions, select one and click the name to join. When you’re selecting a server, you can filter by name and location just by typing into the search boxes at the top.

Ports And Stuff

If you want to play a multiplayer session of Heist you almost certainly need to forward port 56451, this applies if you are hosting or joining! If you manage to join without forwarding this port then please contact me and tell me what router you’re using!

Tools And Guns

Once you’re in a game (singleplayer ot multiplayer) you’ll have access to a tool and a gun, to swap between them roll your mouse wheel (there’s no indication which one you have equipped yet). If you have the tool equipped left clicking will spawn a box above your head which will disappear after 30 seconds. If you have the gun equipped left clicking will fire the gun, shooting players doesn’t kill them (yet) but it will hit the box around.

I’m pretty proud of the new menu system, it’s basically just rendering a locally hosted website in the game client so I can use the power of HTML5, CSS3 and Javascript to build cool menus - I expect I’ll be swapping the menu designs around quite a lot when I need a bit more of an easygoing task to work on!

Next Time

I’m currently deciding what to work on for the next release; I have a choice between writing a scriptable AI system or working on graphics. Personally I think AI would the sensible choice, but for some reason I really wanted to work on some kind of procedural texturing system and animation rigging for characters!

Fortnightly Fun

Sun, 10 Jun 2012 00:00:00 +0000

TL;DR

I’m going to make a release every two weeks, on Mondays, starting from next Monday (18th June).

Regular Releases

Two weeks ago I made the first release of Heist. Everyone who tried the game seemed to think it was pretty cool, and was interested in the direction I would be taking it in. It’s been really encouraging that other people think this is a cool idea, and it’s not just me being weird!

I’ve decided that this was all so fun that I’m going to try and do regular releases. Every two weeks, on Mondays, I will upload a new version of Heist ([here] (http://placeholder-software.co.uk/setup/heistgame/publish.htm)) and will blog about the changes I’ve made here. The installer should auto update the game when you run it, so you don’t even need to check if there’s a new version you’ll just automatically get it.

I’m Not Good With Timetables

I’m sure that you’re now thinking to yourself that it’s been two weeks since the initial launch and that I should be releasing a new build tomorrow. If you were then award yourself a cookie, I should be uploading a new build tomorrow… but I won’t be. The next feature I really wanted to get out the door is multiplayer, it’s something that really needs a lot of testing by a lot of players across as many networks as possible - not something I can possibly do myself. Of course, because multiplayer needs so much testing, it’s really hard to develop it and I’ve not been able to get it stable enough for me to release it tomorrow.

I’ve decided I will release a build next Monday instead, 1 week late. It will be the multiplayer build and I will try my best to make multiplayer as reliable as possible by then!

New Blog

Thu, 31 May 2012 00:00:00 +0000

TL;DR

My old blog sucked, this is my new one.

Bookmark this: http://martindevans.me

Or this, if you prefer: http://martindevans.github.com

You Spoke, I Listened

Earlier this week (on my old blog) I launched the demo version of heist, I posted about it on some forums I frequent and got some very positive feedback. Great!

There was just one piece of very negative feedback which everyone agreed upon: Your blog SUCKS.

I built the site way back in the first year of university, a few of the problems I encountered:

Appengine was new
I’d never used python before
I had no idea about client web development
I had no idea about server side web development

Oddly enough this combination didn’t come together to produce the fantastic, cutting edge, site I expected. Over the years since then I’ve had a few attempts at restructuring the site a little but it really needs a total rebuild - the backend is a mess of spaghetti code and the front end is eye bleedingly ugly. It had to go.

It Had To Go

Luckily just as I got pushed into constructing a new blog by the complaints of the forumites I was experimenting with github pages for hosting documentation for Heist. Github pages are free hosting, don’t required any backend coding, allow me to write my blog posts in markdown and require essentially no work to set up. Sold!

So, here were are. My old blog will stay up until I transfer across all the content, if that ever happens. But new content will appear here from now on.