Deployment Hax In Which A Horrible Hack Is Uncovered

Written By Martin Evans On 28 October 2012 Filed In Heist-Game

TL;DR

My current deployment is pretty horrible.

Idealistic Drivel

I think it’s pretty widely accepted that the ideal deployment process is the one a developer barely even realises exists. What I would really love for Heist is something like this:

  • Develop feature in a git branch
  • Merge branch into “Release” branch
  • Push to github
  • Build Server picks up change to release branch, builds code and (if it passes all tests), deploys new version of game
  • When a player starts the game, it auto updates

That would be pretty cool.

Tests

Here’s the biggest problem. I don’t even nearly have test coverage like this needs. The base libraries Heist is built upon are fairly well tested (although even they need more tests), but Heist itself is very poorly covered. This is partly an artifact of the kind of code that goes into Heist, game specific stuff usually to do with user interactions or UI - this kind of stuff is hard to test.

This is kind of an excuse though, in my opinion code which is hard to test is code which could be better written. So perhaps sometime I need to go through Heist refactoring stuff to be more testable, I certainly need to if I ever want any kind of automated deploy system like this.

Build Server

I don’t have one, but that’s easily fixable.

Auto Update

I do have this! Heist is deployed and installed using the “Microsoft One Click Installer”, which literally requires one click for a user to download, install and run the game. Whenever the application is started OneClick does a quick check for a new version.

All I have to do to deploy a new version of the game is upload certain files to the correct place on my server.

A Non Ideal World

So if I don’t have a shiny auto deployment and testing system, what do i have? For a while I’ve simply been manually building the game, and then copying the folder up to my server over SCP. Finally, once the upload was done, I just renamed the directories to swap the old and new versions. This works, but on my home internet connection it takes about 20 minutes to upload a new build of Heist. Every. Damn. Time. That was kinda getting annoying.

Now, obviously the change from one build to another is generally pretty small. In fact, most of the space in the upload is content which most of the time is exactly the same between builds. Delta encoding to the rescue! The difference between version 1.0.0.35 and 1.0.0.35 (generated by bsDiff) is 3Kb, which is a 99.9% compression ratio.

With bsDiff I’ve made upload time infinitely faster (approximately), but now my deploy process has gone from:

  1. Copy Folder to server
  2. Do something else for a while
  3. Rename folders

To:

  1. Download rar of previous version
  2. Rar compress folder of new version
  3. Generate diff
  4. Upload diff
  5. Execute a remote script to apply diff
  6. unrar the patched file
  7. Copy result into place

That’s a bit of a pain.

All Problems In The World Can Be Saved By One More Script

A boring, repetative task, which requires no human input (except credentials to upload) - sounds like a job for a script!

Here it is in all it’s ugly, hacky, glory:

"C:\Program Files\WinRAR\rar.exe" a "heistgamesetup.rar" "heistgamesetup" -m5
rmdir /S /Q "heistgamesetup"

"bin\DownloadFile.exe" "http://www.placeholder-software.co.uk/static-files/setup/.heistgamesetup.rar"

"bin\bsdiff" ".heistgamesetup.rar" "heistgamesetup.rar" "patch.diff"
DEL ".heistgamesetup.rar"
DEL "heistgamesetup.rar"

"C:\Program Files (x86)\WinSCP\WinSCP.com" /script=upload.script

DEL patch.diff

When this is run it takes the contents of heistgamesetup (which is where Visual Studio is set to publish new builds to), and rars them with the best compression mode. Then it downloads the rar of the previous version from the server and generates a diff between old and new. Finally it logs in to my server with WinSCP and runs upload.script, which uploads the patch file and triggers the remote deployment script.