A Metal Renderer in Swift Playgrounds (Part 5)

With the planet, spaceship, skybox and trails in place, the scene was starting to look pretty cool. But it was far from a game. Although I was not planning to get sound working at this stage, I allocated some time to thinking about how it could work. Some time ago I used a macOS app called bfxr which generated 8/16 bit sound effects such as lasers, jumps, and so on. I wanted to have a similar tool available on the iPad, however I could not find an appropriate one, so I opted to investigate how to generate sounds.

Audio Explorer

To accomplish this I created a new app which I called Audio Explorer. I found a way to play music as well as a generated sine wave. I will need the ability to do both of these in the game. I created a primitive interface to start and stop the sounds with SwiftUI. SwiftUI is fantastic and so easy to use for an amateur like me! In addition to the buttons I wanted to visualise the wave so I experimented with the Path API and was able to reuse the same function for generating the sound wave as for the ends of the line segment. It hadn’t occurred to me that both are just sampling operations on a function until I spent some time thinking about it. I expanded the wave generation to include square, triangle, and sawtooth waves.

I don’t know much about sound effect generation yet but I do know that an important part is the ability to “shape” waves using an envelope. I implemented an attack, release, sustain, decay envelope so that the sounds would have clear boundaries rather than being continuous tones. I removed the music playing functionality for now. Incidentally the music track I added to the app was a short demo song written in GarageBand on the iPad, staying within the constraint that everything must be made on the iPad.

Icons

Playgrounds offers a set of placeholder icons for apps. None really suited Audio Explorer so I decided to make one. I have been using image editors casually for a couple of decades by now, mostly for making silly banter with friends and family, or for hobby projects that need design work. For vector graphics on iPad and macOS I like to use Affinity Designer. I used Designer to composite an icon reflecting the current scene for Kobold, as well as a sine wave with a funky gradient for Audio Explorer. These were easily applied to the apps in playgrounds and makes working with them feel a bit more like they’re really apps!

Low Poly/Voxel Clash

As the scene has evolved, an issue that I’ve observed is that the planet is clearly a polygonal mesh, while the spaceship is clearly made of voxels. This combination is a bit jarring. I’ve always liked the visible triangles of the icosahedron based planet and can’t see myself changing this. That means I need to either find a way to do low poly modelling on the iPad or find some other approach.

Probably the highest quality non-subscription 3d modelling app I’ve come across is Nomad Sculpt. However it’s an app for sculpting and hard surface modelling is not its forte. Nonetheless I decided to try to make a hard surface sculpt, and then look at whether it’s feasible to get something significantly low poly out of it. I watched some excellent tutorials by Glen Southern aka SouthernGFX who has a video describing an approach to hard surface modelling in Nomad, and another for low poly modelling. These videos are heavily caveated with the fact that Nomad is not intended for those use cases, but are still excellent resources. I experimented with the low poly approach, but the lack of convenient alignment and batch vertex tooling made it unworkable for me.

I made a better attempt with the hard surface modelling approach and ended up with a passable sketch of a spaceship model. By the nature of how Nomad represents meshes for its primary purpose of sculpting, the polygon count stretched into the hundreds of thousands which is orders of magnitude larger than what I want. I toyed with the idea of trying to write a tool to optimise the mesh, but felt this wasn’t a good use of my time. None of this is to disparage Nomad Sculpt which is an excellent tool, rather that it is not appropriate for my use case and low skill level. It was great fun working in it too!

This entry is part of a series on writing a Metal Renderer in Swift Playgrounds.