Parallelism

Multithreading = Use of multiple threads Concurrency = Order in which multiple tasks execute is not determined Parallelism = Simultaneous execution (e.g. on multiple cores)

class Printer {
  char ch;
  int sleepTime
  public Printer(char c, int t) {
    ch = c;
    sleepTime = t;
  }
  public void Print() {
    for (var i = 0; i < 100; i++) {
      Console.Write(ch);
      Thread.Sleep(sleepTime)
    }
  }
}
class Test {
  static void Main() {
    var a = new Printer('.', 60);
    var b = new Printer('*', 70);
    new Thread(() => a.Print()).Start();
    new Thread(() => b.Print()).Start();
  }
}

csharpThreadStatus

Thread types

Foreground thread = Program will not terminate as long as at least one foreground thread is running

Background thread = Background threads do not prevent the program from terminating

var bgThread = new Thread(…);
bgThread.IsBackground = true;
bgThread.Start();

Threadpooling

Thread creation requires quite some resources ~1’000’000 clock cycles, about 1MB of memory and also requires kernel interaction. ThreadPool offers automatic thread management and recycling Number of threads is limited, additional requests are queued Used for short running tasks Don’t change thread priority or thread state Background threads only Control over the thread only inside the method given.

Number of Cores: Environment.ProcessorCount

Starting a new thread: var t = new Thread (() => ); t.Start();

Wait for another thread to finish: t.Join()

Use a thread from the ThreadPool: ThreadPool.QueueUserWorkItem((o) => );

Mutual exclusion: lock(someObject)

Using semaphores: var sem = new Semaphore(0,3); sem.WaitOne(); sem.Release();

Using barriers: var b = new Barrier(7); b.SignalAndWait();

Parallel Library

Task Parallel Library (TPL) offers abstractions and reuse approach over threads

PLINQ

PLINQ allows us to parallelize LINQ statements which is based on the TPL.

Partitioning Strategies

Data parallelism = The simultaneous execution of the same function across split data of a data set. For example processing 100 elements, two cores work on 500 each.

Task parallelism = The simultaneous execution of multiple and different functions across the same or different data sets. For example sharpen and resize 100 pictures. First task sharpens, second task resizes.

Parallel.For

//sequential execution
for(var i = 0; i < 10; i++)
{
 Console.WriteLine(i);
}
// parallel execution
Parallel.For(0, 10, i =>
{
 Console.WriteLine(i); // order is unspecified!
});

Parallel.ForEach

string[] capitals = {"London", "Paris", ...}
//sequential execution
foreach(var city in capitals)
{
 Console.WriteLine(city);
}
// parallel execution
Parallel.ForEach(capitals, city =>
{
 Console.WriteLine(city); // order is unspecified!
});

Parallel.Invoke

Parallel.Invoke(()=>Function1(),
    ()=>Functions2()
    );

Invoke returns when all actions are finished and order is unspecified. Not necessarly in parallel but will try.

Task.Run

Task < double > [] tasks = {
  Task.Run(() => DoComputation1()),
  Task.Run(() => DoComputation2())
};
var results = new double[tasks.Length];
for (var i = 0; i < tasks.Length; i++)
  results[i] = await tasks[i];

Using PLINQ

var parallelQuery = Enumerable.Range(3, 30)
  .Where(n => SomePredicate(n))
  .Sum(n => n * n);
// becomes
var parallelQuery = Enumerable.Range(3, 30).AsParallel()
  .Where(n => SomePredicate(n))
  .Sum(n => n * n);

AsParallel() = Chunk Partitioning: threads grab chunk by chunk itself ParallelEnumerable.Range(3,30) = Range Partitioning: range of work preassigned

Limit the number of threads with .WithDegreeOfParallelism(4)

PLINQ only parallelizes work, if it suspects benefits. You can force parallelization with .WithExecutionMode(ParallelExecutionMode.ForceParallelism)

Ordering can be forced with .AsOrdered() Lift ordering requirement with .AsUnordered()

A function has “side effects” when it modifies something outside the function make sure this never happens!

Concurrency Interoperability