During my early days as a backend developer, I built my first parallel feature. It was a small algorithm that ran on multiple threads to generate data to be written on a report. I was inexperienced and didn’t fully understand what I was doing. It was way before concurrency and cloud computing became a thing, and despite all that, the application worked amazingly.
Years went by, I got a lot more experience using parallel computing both in backend development and gaming development and now I could understand the concepts, challenges, and practical applications behind these techniques. In this article, I want to share the lessons from my experience and help others who might be going through the same journey. I decided to make some comparisons on techniques that could be used on both backend and game development, using examples from my own experience to showcase where they work, common pitfalls, and solutions that can mitigate them.
They are starving!
Let’s imagine we are in a restaurant and the waiter should not take more than one minute to run all the tables taking orders. If there are few customers at the tables, the waiter has a good amount of time to listen to the current customer before moving to the next table, and eventually taking the order of all customers in one minute, for example. As time passes, more customers arrive, resulting in the waiter having less and less time to listen to each request, forcing him to make more rounds around the room to attend to all customers. That’s what thread starvation looks like! Your CPU does not run everything at the same time, it runs parts of the code in a time slice. If you have many processes to be executed they’ll process less instructions because the slices of time will be smaller!
Race conditions
Race conditions are the second worst problem on concurrency, right behind the race conditions! Data corruption, system locks, and syncing are the main issues here. These problems happen when two or more processes try to manipulate the same resources at the same time! They are unpredictable, which makes them hard to debug, there is no silver bullet solution for this problem and, to make it worse, the solutions might come with severe drawbacks!
Concurrency
Concurrency happens when you have two or more processes sharing the same resources. The processes may or may not be related but you know at some point you need to sync everything together. Data corruption, locks, sync latency, and race conditions are the most challenging issues in concurrency. The synchronization may cause unexpected delays that may be worse than non-concurrent processes, badly defined critical sections or misuse of locking techniques may cause the applications to freeze. Thread pools and message queues are well-known techniques used in backend development to mitigate the concurrency issues but in game development, it’s common to work at a much lower level where these techniques aren’t always applicable.
Immutability
The cool kids on functional programming are already familiar with this and as the name suggests, it’s just about not changing the values you use during the process. If you need to change some value, create a copy of that variable and then change it!
It can prevent data corruption, since no data will be changed and no lock or sync is necessary. One good use case is when multiple AI threads operate on the same target. If the target does not change until the end of the processes, any AI thread can safely query it without fear of data corruption or the need for any kind of lock. However, you have to ensure there is no change on target until all the AI threads have finished their operations.
Critical sections
The definition of critical sections using locking techniques (like mutexes, spin locks, semaphores, etc.) helps to prevent data corruption and synchronizes variables used by different threads. But if poorly implemented, they can drastically reduce game performance or, in the worst cases, make the game freeze at random moments. Mutex can be used to prevent a variable from being written while it is being read. Semaphores may block different sections of the code, without completely stopping execution, and spin locks may prevent processes from staying locked, waiting on each other. For example, consider a physics thread running, there is no need for locks, as it is executing non-critical operations. But the moment the thread must write the results for the other threads to read, a critical section occurs, ensuring that the physics thread does not write while another thread is reading, or causes the reading thread to wait until the physics thread finishes writing.
Actor models
Actor models operate at a higher level. They consist of entities with self-centered states that use communication channels to trade information between themselves. This approach is beneficial in game development since preventing mutable states from being shared, ensures that no data corruption will happen. This strategy is excellent for a large number of entities. A good use case is in crowd manager systems, where you can create crowd manager actors that could run independently and control the crowd members, which also run independently. Using communication channels to pass information between them, this setup can make the system flexible enough to allow the crowd manager to be integrated with AI systems and delegate this communication with AI APIs without compromising the crowd member itself.
Parallelism
Parallelism is at the opposite end of the spectrum from concurrency, where two or more processes run without sharing resources. At first, they seem to be safe but we need to pay attention to various points, such as synchronization overheads, thread starvation, which are the most common problems here. It is crucial to plan ahead how the execution should be started and how these results should be collected, if applicable. As previously mentioned I’ll present a few issues and techniques along with some use cases.
Fire and forget
When you have a process that you need to execute, and its result can be ignored, fire-and-forget is a good strategy. If unmanaged, fired-and-forgotten tasks that have potential risk to be stuck and have never terminated may cause thread starvation. That’s why using a manager, a thread pool system, or an observer is advised to avoid this issue. On the backend, this technique can be used for logging, generating reports, condense data, and any process that you don’t need the result of the process itself directly. On games, Fire-and-Forget can be used for non-vital visual effects, achievement registering, or any other process where a side effect is expected but the result of the operation isn’t needed.
Worker queue
A worker queue is a technique that consists of creating the data needed for the tasks to be run and storing it in a queue. At a given moment, a worker will pull that data and use it to run a thread. This strategy has many advantages, such as allowing you to control how many workers you can have to process the queue, avoiding starvation since it will create a limit on the resource usage, based on the workers. You can also define how many machine resources a worker can use. Having many small workers to process many small tasks quickly, or a few heavy workers to handle more demanding tasks, will allow the program to have a more efficient usage of the machine resources. Worker queues can be used for asynchronous loading or preparatory tasks for other processes (like rendering or physics).
