Introduction

touchstone is a metadata-driven benchmarking framework, built on top of benchmark.js

Declarative Benchmarks#

With touchstone you define your benchmark like this:

import { Suite, Case } from '@pebula/touchstone';

@Suite({ name: 'My First Benchmark Suite' })
class MyFirstBenchmarkSuite {
 @Case({ name: 'my-first-benchmark' })
  firstBenchmark() {
    /* Benchmarking... */
  }

  @Case()
  async secondBenchmark() {
     // Will automatically detect that it's async. Name is taken from method name.
    /* Benchmarking... */
  }
}

Life Cycle Events#

All benchmark.js event are wrapped and delivered in a normalized manner:

@Suite()
class TestSuite {
  @OnStart() start(event: SuiteStartEvent) { }

  @OnCaseComplete() caseComplete(event: CaseCompleteEvent) { }

  @OnComplete() complete(event: SuiteCompleteEvent) { }

  // @OnReset, @OnAbort, @OnError
}

info

There are additional, touchstone specific, events...

Composition & Re-use#

touchstone is fully extensible through inheritance or composition (mixins):

import { Suite, Case, Mixin, VegaLiteReporter, SimpleConsoleReporter } from '@pebula/touchstone';

@Suite({ name: 'My First Benchmark Suite' })
class MyFirstBenchmarkSuite extends Mixin(SimpleConsoleReporter, VegaLiteReporter) {
 @Case({ name: 'my-first-benchmark' })
  firstBenchmark() {
    /* Benchmarking... */
  }
}

In the example above we Mixin reporting behavior from 2 built-in reporters:

SimpleConsoleReporter - Will log progress to the console
VegaLiteReporter - Will output HTML, SVG and PNG charts using vega-lite

TouchStone Events#

There are 2 touchstone events:

@OnTouchStoneStart() - Fired with the TouchStoneStartEvent event context parameter
@OnTouchStoneEnd() - Fired with the TouchStoneEndEvent event context parameter (which contains the SuiteResult[] property)

Both events can be registered on any suite.

Multiple Suites#

You can declare multiple suite's, touchstone will execute them one after the other.

tip

Because multiple suites can be used for a single run it might not make sense to register mixins on the suite. For suite events this might be ok but the touchstone start/end events will trigger multiple times, once for every suite.

For this scenario, and in general, we recommend using a container to manage all mixins, configuration, etc... The container events will invoke once for all of the events in the system. (i.e. A case complete event, from any suite, will fire once on the container)

Execute#

To execute the suite/s and start benchmarking you need to invoke the touchStone() function.

import { Suite, Case, touchStone } from '@pebula/touchstone';

@Suite({ name: 'My First Benchmark Suite' })
class MyFirstBenchmarkSuite {
 @Case({ name: 'my-first-benchmark' })
  firstBenchmark() {
    /* Benchmarking... */
  }

  @Case()
  async secondBenchmark() {
     // Will automatically detect that it's async. Name is taken from method name.
    /* Benchmarking... */
  }
}

await touchStone();

tip

When using a touchstone configuration container you don't need to call touchStone(), the benchmark will automatically execute.

Demo#

The following is the output of the a demo benchmark application: benchmark-chart