Space Engineers Parallel Modding Guide

Greetings modders!

This week we've added mod profiling to the performance warning system. This will show a warning to players when mods are running slowly and causing the game to lag. The goal of this tool is to help players understand the problems which are causing poor performance in their game. Sometimes modders aren't aware of performance issues, sometimes they are. Either way, this tool can help you make your mods run better.

The version of the mod profiler included in the main release only reports the total time your mod spends in the game thread. We're working on an improved version which will profile each and every method in your mod, however the profiler can actually slow down the game itself when handling that much data, so we can't include it in the main release. We're currently investigating methods to get this profiler to you, and we hope to release it soon!

To help your mods have less impact on the game, I'm here to explain how multithreading works in Space Engineers. This is a pretty complex subject, and I'm going to assume you're vaguely familiar with the concept of threads, Actions, and how they work. If you aren't familiar with these concepts, I suggest you read up on multithreading in C#. Or keep reading this post anyway, I'll try to keep it as simple as I can

There is a LOT of information in this post. Too much, in fact, and there's ten times more that I could talk about. I've tried to keep it to the bare minimum and break it up into sections. If you need help with your modding, our Discord server is the best place to do it. We have many great modders in there who love to help new people, and I'm almost always in there as well. Join us at https://discord.gg/KeenSWH
Now, grab a snack and strap in for by far the longest and most involved modding guide I've ever written.

Background

The majority of Space Engineers runs on one thread. We generally call it the game thread, main thread, or update thread. Most of the game logic happens in here, as does each update. The game wants to update 60 times per second, this means each update has at most 16.6 milliseconds to complete. Keep in mind that your mod's UpdateBeforeSimulation, UpdateAfterSimulation, etc. methods all execute inside the main thread. If your mods take too much time, the update will run over its 16ms slot, simspeed drops, the game lags, and players are unhappy.

Remember, the entire game has to run in that single 16ms slot. That means it is not safe for your mod to take 10ms each tick. You're leaving a measly 6ms for the game, physics, input, and any other mods in the game. You must try to keep your time in the main thread to an absolute minimum.

Some mods simply must run large calculations, access disk, or otherwise just do stuff that takes a long time. How can you calculate the 17000th digit of pi and not tie up the game thread? Threads. ModAPI has access to SE's ParallelTasks library, which allows you to shove work onto a worker thread, to be processed in parallel with the main thread. You also have access to some System threading tools, but we'll get to that in a minute.

ParallelTasks

All of the parallel tools live at MyAPIGateway.Parallel. There's three basic groups of methods from the ParallelTasks library in here; Do, Start, and StartBackground. There's a bit more detail on this at the end of the post, as comments inside IMyParallelTasks.

• Do group contains various methods which allow you to set up a bunch of Action or IWork objects and execute them all together, possibly in parallel with each other, and it will block (wait) until all your tasks are finished.
  
• Start will queue your work for SE's internal pool of worker threads, and returns a Task object which you can monitor to check when your work is done. These methods do not block until completion, they will return immediately. However, some have a callback, which is an Action that will be run when the task is finished. (See the Callbacks section)
  
• StartBackground works exactly like Start, except it uses a separate pool of worker threads which is not shared with the game.

SE maintains a pool of threads that are used internally by the game and the ParallelTasks library. Please keep this fact in mind while designing your mulithreading; if you have a huge task that takes a very long time to run, it will be better to use StartBackground instead of Start or Do. A task started with StartBackground is run in a thread that is completely dedicated to that task, and will never be shared with the game. However, you have to be careful with this, and only use it when it's absolutely necessary. If you start too many Background tasks, you could potentially spawn too many threads, which will decrease overall system performance.

WorkData

One of the nicer ways to deal with tasks that need to be run asynchronously is the WorkData pattern. Here's an example from one of my mods:

Code:

public static class ConveyorWork
{
   public static void DoWork(WorkData workData)
   {
     var data = workData as ConveyorWorkData;
     if (data == null)
       return;

     IMyTerminalBlock workBlock;
     if (!data.Blocks.TryDequeue(out workBlock))
       return;

     IMyInventory compareInventory = data.CompareBlock.GetInventory();
     IMyInventory workInventory = workBlock.GetInventory();

     if (compareInventory == null || workInventory == null)
       return;

     if (compareInventory.IsConnectedTo(workInventory))
       data.ConnectedInventories.Add(workBlock);
     else
       data.DisconnectedInventories.Add(workBlock);
   }

   public class ConveyorWorkData : WorkData
   {
     public IMyTerminalBlock CompareBlock;
     public MyConcurrentQueue<IMyTerminalBlock> Blocks;
     public MyConcurrentHashSet<IMyTerminalBlock> ConnectedInventories;
     public MyConcurrentHashSet<IMyTerminalBlock> DisconnectedInventories;

     public ConveyorWorkData(IMyTerminalBlock compare, MyConcurrentQueue<IMyTerminalBlock> blocks)
     {
       CompareBlock = compare;
       Blocks = blocks;
       ConnectedInventories = new MyConcurrentHashSet<IMyTerminalBlock>();
       DisconnectedInventories = new MyConcurrentHashSet<IMyTerminalBlock>();
     }
   }
}

Imagine you have a huge list of inventory blocks and you want to find all blocks in your list which are connected to another block you're interested in. You can queue up your blocks to check like so:

Code:

private List<IMyTerminalBlock> _inventoryList;
private IMyTerminalBlock _compareInventory;

private void ProcessConveyors()
{
   var blocks = new MyConcurrentQueue<IMyTerminalBlock>(_inventoryList.Count);
   foreach (IMyTerminalBlock b in _inventoryList)
     blocks.Enqueue(b);
   var data = new ConveyorWork.ConveyorWorkData(_compareInventory, blocks);
   for (var i = 0; i < _inventoryList.Count; i++)
     MyAPIGateway.Parallel.Start(ConveyorWork.DoWork, Callback, data);
}

private void Callback(WorkData workData)
{
   var data = workData as ConveyorWork.ConveyorWorkData;
   if (data == null)
     return;

   if (data.Blocks.Count > 0)
     return; //work isn't done

   MyConcurrentHashSet<IMyTerminalBlock> connected = data.ConnectedInventories;
   MyConcurrentHashSet<IMyTerminalBlock> disconnected = data.DisconnectedInventories;
   //process results
}

The pool of worker threads will chew through your tasks, and each time it comes back with a result, the callback action will be run, and your results will be put into a list for you to look through later.

Callbacks

Put simply, a Callback is a method which will be called after your asynchronous task is finished. This is one of the easiest ways to consume results which your asynchronous worker has produced.

An important note about callbacks: they aren't run as soon as the task finishes. Instead, the thread which spawned the task is responsible for checking if there are callbacks to be run, and running them. The main thread handles this automatically at the beginning of every tick, but the worker threads do NOT track callbacks. If you start a Task with a callback from a worker thread, the callback WILL NOT RUN. However, all the callbacks are run synchronously -- that is, one after the other. So it's safe for each of the callbacks to add an item to your result list.

Thread Safety

Now we have to talk about thread safety. If you aren't careful and know exactly what you're doing, you will get a lot of exceptions. Mostly CollectionModifiedExceptions, but those are easy enough to track down. The worst errors happen when one thread steps on another's memory, which can do some really nasty stuff. So my general rule of thumb is this: If you're changing something in the world, be it inventory, block settings, adding/removing entities, you have to do it in the game thread. The reason for this is pretty straightforward; the main thread is always working, doing its own thing. If your thread and the main thread try to modify the same thing at the same time, you can get corrupt data.

Here's a few things that are thread safe, even though it breaks my rule:

• Adding Physics forces
• Raycasting
• Inventory functions not involving items
• Voxel Storage functions

If you want to run some code on the main thread, you can use MyAPIGateway.Utilities.InvokeOnGameThread. This adds an Action into a queue which is processed at the beginning of each tick. That means the call will return immediately, it will not wait for your action to complete. And again, you need to make sure that any Actions you use here are as light as possible so you don't tie up the game thread unnecessarily.

In general, reading values is thread safe, and writing is not. In particular, collections need special care in multithreaded environments. If you are iterating a collection with foreach while another thread modifies the same collection, you will see a CollectionModifiedException. There's several solutions to this problem.

Parallel.ForEach

A simpler way to process things in parallel is to use MyAPIGateway.Parallel.For and ForEach. These function exactly like the System versions. Here's a version of the previous example, but using Parallel.ForEach:

Code:

private List<IMyTerminalBlock> _inventoryList;
private IMyTerminalBlock _compareInventory;
private MyConcurrentHashSet<IMyTerminalBlock> connected = new MyConcurrentHashSet<IMyTerminalBlock>();
private MyConcurrentHashSet<IMyTerminalBlock> disconnected = new MyConcurrentHashSet<IMyTerminalBlock>();

private void ProcessConveyors()
{
   var compare = _compareInventory?.GetInventory();
   if (compare == null)
     return;

   MyAPIGateway.Parallel.ForEach(_inventoryList, block =>
                          {
                            var inv = block.GetInventory();
                            if (inv == null)
                              return; //same as 'continue' in normal foreach

                            if(compare.IsConnectedTo(inv))
                              connected.Add(block);
                            else
                              disconnected.Add(block);
                          });
   //process results
}

The main difference here is that Parallel.ForEach will block until it's done, whereas the first version will return as soon as all the tasks have been scheduled. A big part of asynchronous programming is deciding when you should wait for your results, and when you should use a callback. Using a ForEach like this makes sense when you have a small number of items, or when each evaluation takes a very short time. If you expect your task to take a very long time, you're better off using a callback, or some other design pattern.

It's important to note here that there's no guarantee that either of these examples will run multiple tasks in parallel. They will always be in parallel with the game thread, but you have no way to know if one, two, or eighty of your task are running at once. This is down to the way that tasks are assigned to threads. The worker threads can each only do one thing at a time. When one task is finished, the thread will ask the task scheduler for more work, and the scheduler decides what to give it next based on several factors. If there is only one thread available for the lifetime of your work, then your work will only happen on one thread. 99% of the time this doesn't really matter, but it's worth knowing.

Concurrent Collections

Concurrent collections are collections (like List, HashSet, Dictionary, etc.) which are designed to be used in multithreaded environments. Mods have access to all System defined types, as well as some defined in VRage. If you look at the earlier example using Parallel.ForEach you can see instead of locking my result lists, I simply used a MyConcurrentHashSet. Internally, the Add method has locks, which means it's safe for many threads to call .Add all at once.

Concurrent collections often do not behave the same as the regular collection types you may be familiar with. Most of them have a completely separate set of methods that make them safe. You should read up on the documentation for all the System types: https://msdn.microsoft.com/en-us/library/system.collections.concurrent(v=vs.110).aspx

Locks

An extremely important concept in multithreaded programming is that of locking. Put simply, when you lock an object, no other thread can obtain a lock on it. This is the most fundamental way to synchronize thread access to an object. There's a few ways to use locks, but for now I'll focus on the system lock keyword.

Code:

List<char> _charList = new List<char>();
public void RunThreadA()
{
   lock (_charList)
   {
     _charList.AddRange(new [] {'a', 'b', 'd', 'g', 'o'});
   }
}

public void RunThreadB()
{
   lock (_charList)
   {
     _charList.RemoveAll(c => c == 'a' || c == 'e' || c == 'i' || c == 'o' || c == 'u');
   }
}

Here, thread A adds a bunch of letters to the list, then thread B removes all vowels from the list. Let's assume for a moment that AddRange takes several seconds to execute. If you start ThreadA, then immediately after start ThreadB, what happens? Because ThreadA obtains a lock first, ThreadB will just sit and wait until the lock is released, then continue with its operation.

For more information on locks, you should read this article on MSDN: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/lock-statement

You need to be EXTREMELY careful when using locks inside the game thread. If you aren't, you could accidentally create a deadlock, and freeze the entire game. The only way to recover from this is to force close SE from the task manager. Deadlocks in the worker threads are also bad, they can prevent the game from saving, or exiting properly. Also waiting for a lock in the game thread is simply wasted time. The thread is waiting 6ms for a lock when it could be calculating physics or doing something more useful.

Now I'll post the IMyParallelTasks interface and a list of System and game Types which are whitelisted. These are all the multithreaded support we have right now. If something is missing or not working correctly, please contact me and I'll see about fixing it

Whitelist

Code:

public interface IMyParallelTask
{
  /// <summary>
  /// Default WorkOptions.
  /// DetachFromParent = false, MaximumThreads = 1
  /// </summary>
  WorkOptions DefaultOptions { get; }

  /// <summary>
  /// Starts a task in a secondary worker thread. Intended for long running, blocking, work
  /// such as I/O.
  /// </summary>
  /// <param name="work">The work to execute.</param>
  /// <param name="completionCallback">A method which will be called in Parallel.RunCallbacks() once this task has completed.</param>
  /// <returns>A task which represents one execution of the work.</returns>
  /// <exception cref="ArgumentNullException">
  /// <paramref name="work"/> is <see langword="null"/>.
  /// </exception>
  /// <exception cref="ArgumentException">
  /// Invalid number of maximum threads set in <see cref="IWork.Options"/>.
  /// </exception>
  Task StartBackground(IWork work, Action completionCallback);

  /// <summary>
  /// Starts a task in a secondary worker thread. Intended for long running, blocking, work
  /// such as I/O.
  /// </summary>
  /// <param name="work">The work to execute.</param>
  /// <returns>A task which represents one execution of the work.</returns>
  Task StartBackground(IWork work);

  /// <summary>
  /// Starts a task in a secondary worker thread. Intended for long running, blocking, work
  /// such as I/O.
  /// </summary>
  /// <param name="action">The work to execute.</param>
  /// <returns>A task which represents one execution of the action.</returns>
  Task StartBackground(Action action);

  /// <summary>
  /// Starts a task in a secondary worker thread. Intended for long running, blocking, work
  /// such as I/O.
  /// </summary>
  /// <param name="action">The work to execute.</param>
  /// <param name="completionCallback">A method which will be called in Parallel.RunCallbacks() once this task has completed.</param>
  /// <returns>A task which represents one execution of the action.</returns>
  /// <exception cref="ArgumentNullException">
  /// <paramref name="action"/> is <see langword="null"/>.
  /// </exception>
  Task StartBackground(Action action, Action completionCallback);

  /// <summary>
  /// Starts a task in a secondary worker thread. Intended for long running, blocking work such as I/O.
  /// </summary>
  /// <param name="action">The work to execute.</param>
  /// <param name="completionCallback">A method which will be called in Parallel.RunCallbacks() once this task has completed.</param>
  /// <param name="workData">Data to be passed along both the work and the completion callback.</param>
  /// <returns>A task which represents one execution of the action.</returns>
  /// <exception cref="ArgumentNullException">
  /// <paramref name="action"/> is <see langword="null"/>.
  /// </exception>
  Task StartBackground(Action<WorkData> action, Action<WorkData> completionCallback, WorkData workData);

  /// <summary>
  /// Executes the given work items potentially in parallel with each other.
  /// This method will block until all work is completed.
  /// </summary>
  /// <param name="a">Work to execute.</param>
  /// <param name="b">Work to execute.</param>
  void Do(IWork a, IWork b);

  /// <summary>
  /// Executes the given work items potentially in parallel with each other.
  /// This method will block until all work is completed.
  /// </summary>
  /// <param name="work">The work to execute.</param>
  void Do(params IWork[] work);

  /// <summary>
  /// Executes the given work items potentially in parallel with each other.
  /// This method will block until all work is completed.
  /// </summary>
  /// <param name="action1">The work to execute.</param>
  /// <param name="action2">The work to execute.</param>
  void Do(Action action1, Action action2);

  /// <summary>
  /// Executes the given work items potentially in parallel with each other.
  /// This method will block until all work is completed.
  /// </summary>
  /// <param name="actions">The work to execute.</param>
  void Do(params Action[] actions);

  /// <summary>
  /// Executes a for loop, where each iteration can potentially occur in parallel with others.
  /// </summary>
  /// <param name="startInclusive">The index (inclusive) at which to start iterating.</param>
  /// <param name="endExclusive">The index (exclusive) at which to end iterating.</param>
  /// <param name="body">The method to execute at each iteration. The current index is supplied as the parameter.</param>
  void For(int startInclusive, int endExclusive, Action<int> body);

  /// <summary>
  /// Executes a for loop, where each iteration can potentially occur in parallel with others.
  /// </summary>
  /// <param name="startInclusive">The index (inclusive) at which to start iterating.</param>
  /// <param name="endExclusive">The index (exclusive) at which to end iterating.</param>
  /// <param name="body">The method to execute at each iteration. The current index is supplied as the parameter.</param>
  /// <param name="stride">The number of iterations that each processor takes at a time.</param>
  void For(int startInclusive, int endExclusive, Action<int> body, int stride);

  /// <summary>
  /// Executes a foreach loop, where each iteration can potentially occur in parallel with others.
  /// </summary>
  /// <typeparam name="T">The type of item to iterate over.</typeparam>
  /// <param name="collection">The enumerable data source.</param>
  /// <param name="action">The method to execute at each iteration. The item to process is supplied as the parameter.</param>
  void ForEach<T>(IEnumerable<T> collection, Action<T> action);


  /// <summary>
  /// Creates and starts a task to execute the given work.
  /// </summary>
  /// <param name="action">The work to execute in parallel.</param>
  /// <param name="options">The work options to use with this action.</param>
  /// <param name="completionCallback">A method which will be called in Parallel.RunCallbacks() once this task has completed.</param>
  /// <returns>A task which represents one execution of the work.</returns>
  Task Start(Action action, WorkOptions options, Action completionCallback);

  /// <summary>
  /// Creates and starts a task to execute the given work.
  /// </summary>
  /// <param name="action">The work to execute in parallel.</param>
  /// <param name="options">The work options to use with this action.</param>
  /// <returns>A task which represents one execution of the work.</returns>
  Task Start(Action action, WorkOptions options);

  /// <summary>
  /// Creates and starts a task to execute the given work.
  /// </summary>
  /// <param name="action">The work to execute in parallel.</param>
  /// <param name="completionCallback">A method which will be called in Parallel.RunCallbacks() once this task has completed.</param>
  /// <returns>A task which represents one execution of the work.</returns>
  Task Start(Action action, Action completionCallback);

  /// <summary>
  /// Creates and starts a task to execute the given work.
  /// </summary>
  /// <param name="action">The work to execute in parallel.</param>
  /// <returns>A task which represents one execution of the work.</returns>
  Task Start(Action action);

  /// <summary>
  /// Creates and schedules a task to execute the given work with the given work data.
  /// </summary>
  /// <param name="action">The work to execute in parallel.</param>
  /// <param name="completionCallback">A method which will be called in Parallel.RunCallbacks() once this task has completed.</param>
  /// <param name="workData">Data to be passed along both the work and the completion callback.</param>
  /// <returns>A task which represents one execution of the action.</returns>
  Task Start(Action<WorkData> action, Action<WorkData> completionCallback, WorkData workData);

  /// <summary>
  /// Creates and starts a task to execute the given work.
  /// </summary>
  /// <param name="work">The work to execute in parallel.</param>
  /// <param name="completionCallback">A method which will be called in Parallel.RunCallbacks() once this task has completed.</param>
  /// <returns>A task which represents one execution of the work.</returns>
  /// <exception cref="ArgumentNullException">
  /// <paramref name="work"/> is <see langword="null"/>.
  /// </exception>
  /// <exception cref="ArgumentException">
  /// Invalid number of maximum threads set in <see cref="IWork.Options"/>.
  /// </exception>
  Task Start(IWork work, Action completionCallback);

  /// <summary>
  /// Creates and starts a task to execute the given work.
  /// </summary>
  /// <param name="work">The work to execute in parallel.</param>
  /// <returns>A task which represents one execution of the work.</returns>
  Task Start(IWork work);

  /// <summary>
  /// Suspends the current thread for the specified number of milliseconds.
  /// </summary>
  /// <param name="millisecondsTimeout"></param>
  void Sleep(int millisecondsTimeout);

  /// <summary>
  /// Suspends the current thread for the specified amount of time.
  /// </summary>
  /// <param name="timeout"></param>
  void Sleep(TimeSpan timeout);
}

System:

• lock keyword
• System.Threading.Monitor
• System.Threading.AutoResetEvent
• System.Threading.ManualResetEvent
• System.Threading.Interlocked
• System.Collections.Concurrent namespace

Game:

• ParallelTasks.IWork
• ParallelTasks.Task
• ParallelTasks.WorkOptions
• ParallelTasks.WorkData
• ParallelTasks.SpinLock
• ParallelTasks.SpinLockRef
• VRage.FastResourceLock
• VRage.Collections namespace

 

Please do not stop to do these tutorials, they are extremely useful! thanks!