Aurifi – a postmortem

Posted on September 4, 2010

Hello!

Thanks for all the great feedback on last weeks post about QuizQuizQuiz it was really appreciated.

Today I’m going to talk a little about a project called Aurifi, an audio-only game for iPhone/iPod which was launched at the end of May.

We performed all the programming work and were also involved in the design process with the great bunch of people at Punk Pie Ltd.

What is Aurifi? (Or-If-Eye)

The game is made up of a set of challenges which the player must successfully completeĀ  to advance and ‘grow’ the music. Each challenge only requires you to listen and to touch, tap, tilt and shake as indicated by the beat. The game features no meaningful visuals and can be played with your eyes closed (in fact it’s almost recommended!)

The music in the game is mixed real-time from a large amount of professionally created tune segments, because of the way these segments are selected and mixed together the music you hear will never be the same. After completing each piece of music the player will be asked to ‘steer’ towards the style of music they would like for the next set of challenges.

Be sure to check out the teaser trailer that Punk Pie had produced before release and even better buy the game


Scripting / Fast iteration

The design process was really interesting as not being able to use visual cues is a handicap when you’ve become used to certain techniques to communicate with the player – especially with any slightly complex mechanic!

Due to the amount of tweaking and testing we’d need to do during the development before the first design document appeared I started putting together a prototyping app based on the initial list of possible challenges. I did this using our own multi-platform middleware (‘Lemon engine’) at the backend, FMOD for the audio capabilities (at this stage we expected to use a lot of DSP effects) and Lua.

Previously we’ve written our own C++ to Lua glue code there are however quite a few options for this nowadays, I picked luabridge (despite – I believe – it no longer being maintained) simply because I preferred the syntax after scanning through a couple one evening. It was pretty complete in terms of our needs.

The prototyping app was designed for quickly testing out the ideas we had for the challenges, we worked on PC initially quickly tweaking the challenges in self contained Lua files with calls for playing samples, applying DSP effects to active channels, adjusting pitch, frequency, volume, looping mode, ‘3d’ position and velocity. Executing one of the challenge lua scripts overwrote the previous definition for that challenge so reloads could be done incredibly quickly.

My next step was to move all our prototyping across to the device, this never quite happened as we got too involved on the design and challenge side of things for me to implement the simple network update system I had in mind for the Lua files. The idea was that a HTTP server on the Mac/PC I was editing the challenge Lua files on would get requests from the iPhone to check for updates to any files and would push down the update.

This would have worked perfectly I feel but we stuck with PC due to time. We paid for this as we later encountered issues on device that we’d have found sooner (Early SDK versions of FMOD iPhone reacted a little differently to FMOD PC).

While(1) { script_some_more(); }

We ended up with a lot of script in the game by the end (I’ll include a few codebase stats later), initially the plan was just to script the challenges and have the rest of the game be implemented in C++ (mainly as it wouldn’t be tweaked much).

I found though that I was working so quickly in Lua for implementing challenges that the song selection and playback code (plus the data definitions for the pieces of music) just ended up getting implemented before I could think about what language I was working in!

The only code that I planned to script and ended up in C++ was the menu code, which was added fairly late in the project and I was coming back to the project having been on a non-Lua project for a little bit so perhaps wasn’t in the groove as much!

Terms & Conditions?!!

During development and as we came up to release there was trouble rumbling on the net over section 3.3.1 of Apple’s developer agreement stating

Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

As a company we felt this was mainly aimed at Flash however we were bracing ourselves to re-code the Lua scripts into C++ should we be rejected by Apple. We knew of plenty of other apps using Lua on the store so felt pretty happy, we also precompiled our scripts into bytecode which made the scripting a little less obvious!

Sound plan?

So how did things work out?

Well the client was (they tell me!) very happy with the way things went during development which is always a nice result.

The scripting system worked great for quick adjustments to the challenges as we tweaked near the end of development (this was stuff like tolerances for acceptance, applying different filtering to the inputs and in some cases a complete adjustment of how the scoring system worked).

Choosing to use Lua wasn’t an accident as we’ve got a fair bit of experience with it (we started using it around 8 years ago on a Xbox 1 title followed by heavy use on a MMO system, plus various smaller projects and some personal WoW mod writing!). It is possible to make quite a few syntax errors with Lua especially if you’re coming back to use it after a while, there are quite a lot of code modifications and tricks you can use to help with this and it’s still seems to be as popular as ever as a scripting language.

The main issues encountered with the project were down to the FMOD wrapper I wrote and not completely understanding FMODs channel management when initially writing it (and due to time not fixing it quite as well as we should have). This was further complicated as our requirements in terms of sound channel grouping changed later on as we rewrote parts of the song playback and randomisation system.

When moving over from PC to device we found had a lot of sync issues due to the update rate on device being slower than it was on PC (we were working with a dynamic timestep so the iPhone when running slower due to the amount of mixing / DSP work being done in FMOD ran at a slower rate). These were fixed by implementing callbacks for touches so that input code could run immediately and also by reducing the size of FMODs mix buffer (which was a fixed size in an earlier version of the FMOD iPhone libs) to reduce the latency between a call in the script and the sound being heard.

Another issue was memory usage, we had some crashes on older devices (jailbroken in particular) which we couldn’t replicate but do plan to re-investigate soon, I think this was purely down to available RAM on certain devices and our startup RAM usage being quite high with all the tutorial assets pre-cached. The best solution for this will be to stream more of the assets on lower RAM devices and pay attention to Apples low memory warnings at the expense of reponsiveness!

The main reason for our high memory usage was that lower end devices couldn’t decode the number of compressed samples we needed at once as well as FMOD mixing and applying DSP effects so we streamed and decoded to PCM data on load and then passed uncompressed buffers in (plus FMOD doesn’t support on the fly OGG decoding – we did implement our own but again it was just a little too slow and wasn’t worth the time optimising).

To control all this we had a fairly awkward pre-cache system which allowed us to tell the background stream system what we knew we’d need soon (for example during song transitions or in tutorials) and we could force these to be freed when we no longer needed them.

Stat attack!

A few stats on the data / source code involved with this game

The game data contained

  • 24minutes 48seconds of music audio (compressed with OGG Vorbis to 20.3mb)
  • 12minute 16seconds of game samples and tutorial speech (compressed with OGG Vorbis to 11.6mb)
  • 722,762 bytes of PNG images! Kind of obvious it’s an audio game looking at how little this is of the total build size!

The game source code was made up of

  • Lua – 11021 lines of code (28% comments)
  • C++ / ObjectiveC – 7679 lines of code (21% comments)
    • This is of actual game code we wrote not including PNG loader, fmod headers, lua, ogg or vorbis. This also doesn’t include the GL rendering code.

Note I’m considering writing an alternative loc (lines of code) tool which will calculate the number of cans of Pepsi (or Dr Pepper, Sprite, Fanta or tea based on the individual developer) involved in development of each SVN commit! A very important statistic I feel!

Unique set of gamers

The game had a great initial following despite it being a really hard concept to communicate to people, I believe it’s still selling well today.

Specifically one market of really strong supporters of the game are blind gamers, through the tilt controllable menu and having no requirement to see the screen we of course hoped it would really appeal. I think this is really great and I’d love to think about how our future games could be modified to cater for this passionate group of gamers.

Thanks!

Hopefully this was a useful review of Aurifi, we’ve been really lucky to be involved in some very interesting projects over the years and this was certainly up there in terms of the challenges faced especially in design terms.

Things we’ve been enjoying this week

Be Sociable, Share!

Tags: ,

4 Responses

  1. Simeon
    September 4, 2010

    Great blog post. šŸ™‚


  2. Nicolas
    September 5, 2010

    Very nice article!

    For my lua bridging needs I prefer luabind, but there are SO MANY options out there, that I feel is more of a personal thing.


  3. admin
    September 5, 2010

    Simeon : Thanks!

    Nicolas : Thanks for the comment, lots of different options as you say. OOLua is one I’ll certainly be looking at next, also I believe one of the Game Programming Gems reviewed a few different binding methods in terms of speed / usability.


  4. […] A final brief mention for another quality compression format from xiph.org this time Vorbis which is a very decent competitor for mp3 and doesn’t come with any associated licensing overhead or concerns. Again we use this for decompressing on the fly on PC and newer console hardware plus have used on iPhone on Aurifi […]


Leave a Reply