Performance tests of Event, ActionList, Observer, InterfaceObserver

Sometimes we need to call multiple callbacks on one event - what we should to use for this case for less performance penalty? Answer was not obvious.

Sometimes we need to call multiple callbacks on one event - what we should to use for this case for less performance penalty? Answer was not obvious.

Test environment

We will run 10000 iterations and will repeat this process 10 times, then will take average result time. We will measure time for calling our callbacks for one event at each implementation. As we will do this inside Unity we should knows that:

  • We can’t use Time.realtimeSinceStartup for performance measurements - accuracy is very low. Instead we will use standard System.Diagnostics.Stopwatch class.
  • We can’t start measure right on game start - Unity needs time to full initialization, we should wait few seconds when hardware resources will be freed. 3 seconds - enough for this test.
  • Editor always compiles and runs code in DEBUG mode. For proper measure we need RELEASE version - we should creates standalone build and makes measurements with this build outside Unity editor. We will use Debug.Log method for save results to external logs.

Event

First case - built-in event processing behaviour in C#:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
class EventTest : MonoBehaviour {
const int TESTS = 10;
const int T = 10000;
public static int Counter;

event Action _event;

void TestCB () {
EventTest.Counter = (EventTest.Counter + 1) % 10000;
}

IEnumerator Start () {
yield return new WaitForSeconds (3f);
var result = 0;
var sw = new System.Diagnostics.Stopwatch ();
for (var i = 0; i < T; i++) {
_events += TestCB;
}
for (var test = 0; test < TESTS; test++) {
sw.Reset ();
sw.Start ();
for (var i = 0; i < T; i++) {
_events ();
}
sw.Stop ();
result += _sw.Elapsed.Milliseconds;
}
Debug.LogFormat ("event call: {0:F2}", result / (float) TESTS);
}
}

ActionList

Second case - list of System.Action:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
class ActionListTest : MonoBehaviour {
const int TESTS = 10;
const int T = 10000;
public static int Counter;

List<Action> _actions = new List<Action> ();

void TestCB () {
EventTest.Counter = (EventTest.Counter + 1) % 10000;
}

IEnumerator Start () {
yield return new WaitForSeconds (3f);
var result = 0;
var sw = new System.Diagnostics.Stopwatch ();
for (var i = 0; i < T; i++) {
_actions.Add (TestCB);
}
for (var test = 0; test < TESTS; test++) {
sw.Reset ();
sw.Start ();
for (var i = 0; i < T; i++) {
for (int j = 0; j < _actions.Count; j++) {
_actions[j] ();
}
}
sw.Stop ();
result += _sw.Elapsed.Milliseconds;
}
Debug.LogFormat ("action call: {0:F2}", result / (float) TESTS);
}
}

Observer

Third case - use Observer-pattern over list of instances with our method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class ObserverTest : MonoBehaviour {
const int TESTS = 10;
const int T = 10000;
public static int Counter;

List<Observer> _observers = new List<Observer> ();

IEnumerator Start () {
yield return new WaitForSeconds (3f);
var result = 0;
var sw = new System.Diagnostics.Stopwatch ();
for (var i = 0; i < T; i++) {
_observers.Add (new Observer ());
}
for (var test = 0; test < TESTS; test++) {
sw.Reset ();
sw.Start ();
for (var i = 0; i < T; i++) {
for (int j = 0; j < _observers.Count; j++) {
_observers[j].Action ();
}
}
sw.Stop ();
result += _sw.Elapsed.Milliseconds;
}
Debug.LogFormat ("observer call: {0:F2}", result / (float) TESTS);
}

class Observer {
public void Action () {
ObserverTest.Counter = (ObserverTest.Counter + 1) % 10000;
}
}
}

InterfaceObserver

Fourth case - as previous case, but through interface:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
class InterfaceObserverTest : MonoBehaviour {
const int TESTS = 10;
const int T = 10000;
public static int Counter;

List<IObserver> _interfaces = new List<IObserver> ();

IEnumerator Start () {
yield return new WaitForSeconds (3f);
var result = 0;
var sw = new System.Diagnostics.Stopwatch ();
for (var i = 0; i < T; i++) {
_interfaces.Add (new VirtualObserver ());
}
for (var test = 0; test < TESTS; test++) {
sw.Reset ();
sw.Start ();
for (var i = 0; i < T; i++) {
for (int j = 0; j < _interfaces.Count; j++) {
_interfaces[j].Action ();
}
}
sw.Stop ();
result += _sw.Elapsed.Milliseconds;
}
Debug.LogFormat ("interface-observer call: {0:F2}", result / (float) TESTS);
}

interface IObserver {
void Action ();
}

class VirtualObserver : IObserver {
public void Action () {
InterfaceObserverTest.Counter = (InterfaceObserverTest.Counter + 1) % 10000;
}
}
}

Results

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// unity 2017.3.0f3, fw3.5 profile, 10k iterations, average of 10 tests.

// standalone macos 10.13.2, mono.
event call: 752.10
action call: 720.10
observer call: 716.40
interface-observer call: 724.50

// samsung N9005, mono.
event call: 599.60
action call: 271.30
observer call: 628.90
interface-observer call: 427.60

// samsung N9005, il2cpp.
event call: 815.60
action call: 313.70
observer call: 767.50
interface-observer call: 405.70

// iPhone6s iOS 11.2.2, il2cpp, fast-no-exceptions.
event call: 804.10
action call: 626.60
observer call: 62.00
interface-observer call: 490.10

// iPadPro 13" iOS 11.2.2, il2cpp, fast-no-exceptions.
event call: 465.80
action call: 316.00
observer call: 862.60
interface-observer call: 186.10

Very strange results for iPhone6s / iPadPro13”: Observer-case on first device much faster than InterfaceObserver, on second - vice versa.

Other results are stable enough: ActionList-case faster or equals than other cases.

Event-case slowest in all cases, but has one important feature - we can modify subscriber’s list during call, other cases will be crashed without additional isolation like copy to temp list for each call.