Programming Resources
Over the past 8 years in academia and industry, I’ve spent the vast majority of my working time writing code. Along the way, I’ve developed some fairly strong opinions about good and bad ways to write code, and I’ve collected a reasonably large supply of educational resources. The purpose of this post is to collect those resources in one organized place.
System Architecture
Casey Muratori has been massively influential on my understanding of system architecture, and many of my formative opinions on the subject have come from his “Handmade Hero” and “Performance-Aware Programming” series.
“The Big OOPs” describes the history of object-oriented architecture, and along the way uncovers a 35-year gap between the first usage of an “entity-component system” in 1963(!) and it’s re-discovery in 1998.
I could have easily picked 7 or 8 clips from “Handmade Hero”, but this clip is arguably my favorite. Casey points out the complexity that comes with common system architectures for things like memory allocation and error handling, and encourages the programmer to think more in terms of “groups” of things rather than “individual” things.
Data-Oriented Design
Mike Acton’s 2014 CppCon keynote has achieved legendary status amongst some classes of programmers, and for good reason. Along with laying out the groundwork and implications of successful data-oriented programming, Mike has a great philosophy of treating computer science as a proper “science” where claims can be tested and verified on the target hardware.
Nic Barker gives a somewhat friendlier introduction to data-oriented programming. In particular, I love his analogy for the various layers of data caches and how he builds an intuition for how long memory access takes across the different layers.
Memory Allocation
Ryan Fleury’s article about memory arenas was massively influential for me, and changed the way that I do memory management in my day-to-day work. In particular, I like how he breaks down the barriers of what language constructs make available (“the stack”, “the heap”, etc) in order to find a memory interface works best for the specific problems that he’s working on.
Ginger Bill's series on memory allocation strategies, covering arenas, pools, and other approaches to manual memory management.
Ginger Bill (creator of the Odin programming language) has a wonderful series of blogs outlining the construction of many basic memory allocators, including arenas, pools, buddy allocators, and more.
I gave a talk for The Audio Programmer a couple years ago, describing how I use memory arenas specifically for real-time audio applications.
Multi-Threaded Programming
Multi-threaded programming is a massive topic in its own right. This list is a bit heavier on things that matter a lot for the types of software that I write, notably lock-free and wait-free programming.
Jon’s talk for Jane Street breaks down the relationship between CPU cache lines and cross-core communication that even “atomic” operations typically require. He also introduces a clever “left-right” algorithm that allows wait-free “read” operations to parallelize at scale.
Dennis Gustafsson presents a specific parallel programming problem, and gives great insight into profiling the bottlenecks and arriving at an efficient solution.
Ryan Fleury talks through parallelism more at the architecture level. Incidentally, the design that he arrives at is quite similar to the “thread pool” mechanism present in the CLAP plugin standard.
Timur Doumler is really the person to listen to when it comes to real-time programming in C++ for audio applications. I could have picked a number of his blog posts or conference talks, but I really like the RCU algorithm thta he presents here, and it’s been quite influential on my own work.
Speaking of… This is a talk that I gave for the Northwest C++ Users Group discussing the development of a wait-free algorithm from first principles. This talk was heavily influenced by Timur and Jon’s talks linked above.
Coding Style
When I was learning to program “Clean Code” was all the rage. It took a while to de-program myself from that mindset.
Casey Muratori did a lot of that work in just 20 minutes, demonstrating how commonly cited “clean coding” practices could easily wipe out a decade or more of hardware performance improvements.
In this essay I want to talk about how I write code. I am going to call my approach “codin’ dirty” because I often go against the recommendations of Clean Code, a popular approach to writing code.
Carson Gross makes two great points here:
- Breaking “Clean Code” principles can (often) make your code more understandable and maintainable.
- There are lots of ways to write code successfully… find what works for you, and don’t be intimidated by seemingly strict sets of rules.
Philosophy
Why and how I sometimes write stuff from scratch.
Programmers are often told to “not reinvent the wheel”. Of course, people who actually make wheels re-invent them all the time, otherwise we might be driving cars that look like this:
don't reinvent the wheel pic.twitter.com/xWuoXWl2qn
— Tyler Glaiel (@TylerGlaiel) April 3, 2020
I spend a lot of time thinking about software performance, and perhaps even more time trying to get other people to care. Grace Hopper presents an obvious physical reminder of “what programmers are throwing away, when they throw away microseconds”.
Technology occupies a precarious position in our current civilization. While our society depends ever more on technology, there are massive problems with supply chains, overly centralized knowledge, and especially complexity that could lead to a civilization-level collapse, as seen during the Bronze Age and other times throughout history.
Building up fundamental understanding and cultures of engineering excellence can go a long way towards stabilizing our relationship with technology… Even in seemingly non-serious areas like audio and games programming.
A layman's guide to thinking like the self-aware smol brained.
Speaking of complexity… The Grug Brained Developer is here to remind us that you don’t need to be smart or clever to be effective. In fact, programmers trying to be smart or clever are often the ones who get into trouble with complexity. Simplify. Say no. Say: complexity very, very bad.