覚えたら書く

IT関係のデベロッパとして日々覚えたことを書き残したいです。twitter: @yyoshikaw

JMH - fork数の指定

JMHでは、fork数というものを指定することができます。

fork数は、Benchmarkの取得(ウォームアップ開始から計測終了までの)処理の実行回数に対応します。
fork数が1であれば、1回だけベンチマークの取得を行い、fork数が2であればベンチマークの取得を2回行います。

fork数はコード上は以下で指定可能です

  • @Forkアノテーションのパラメータ
  • OptionsBuilder#forksメソッド

ちなみに明示的なfork数の指定をしない場合は、fork数=10で処理されます

以下、fork数=1の場合と2の場合の動きの確認を行ったサンプルとなります


fork数=1の場合

@Frokアノテーションでパラメータに1を指定した場合の動きの確認です

■サンプルコード

import java.util.concurrent.TimeUnit;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;

@Warmup(iterations=5)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@BenchmarkMode(Mode.Throughput)
@Fork(1)
public class ExampleBenchmark1 {

    @Benchmark
    public void date() {
        java.util.Date d = new java.util.Date();
    }

    @Benchmark
    public void calndar() {
        java.util.Calendar c = java.util.Calendar.getInstance();
    }

    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                 .include(ExampleBenchmark1.class.getName())
                .build();
        new Runner(opt).run();
    }
}

■実行結果

(一部省略)

# Warmup: 5 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.jmh.sample.fork.ExampleBenchmark1.calndar

# Run progress: 0.00% complete, ETA 00:00:50
# Fork: 1 of 1
# Warmup Iteration   1: 831.441 ops/ms
# Warmup Iteration   2: 1703.648 ops/ms
# Warmup Iteration   3: 1996.411 ops/ms
# Warmup Iteration   4: 2221.892 ops/ms
# Warmup Iteration   5: 2221.753 ops/ms
Iteration   1: 2234.066 ops/ms
Iteration   2: 2228.459 ops/ms
Iteration   3: 2233.434 ops/ms
Iteration   4: 2189.125 ops/ms
Iteration   5: 2185.146 ops/ms
Iteration   6: 2176.646 ops/ms
Iteration   7: 2223.254 ops/ms
Iteration   8: 2235.134 ops/ms
Iteration   9: 2206.835 ops/ms
Iteration  10: 2184.213 ops/ms
Iteration  11: 2223.862 ops/ms
Iteration  12: 2204.172 ops/ms
Iteration  13: 2209.429 ops/ms
Iteration  14: 2189.011 ops/ms
Iteration  15: 2212.771 ops/ms
Iteration  16: 2220.309 ops/ms
Iteration  17: 2094.173 ops/ms
Iteration  18: 2237.235 ops/ms
Iteration  19: 2239.695 ops/ms
Iteration  20: 2171.550 ops/ms


Result "calndar":
  2204.926 ±(99.9%) 29.482 ops/ms [Average]
  (min, avg, max) = (2094.173, 2204.926, 2239.695), stdev = 33.951
  CI (99.9%): [2175.444, 2234.408] (assumes normal distribution)


# Warmup: 5 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.jmh.sample.fork.ExampleBenchmark1.date

# Run progress: 50.00% complete, ETA 00:00:26
# Fork: 1 of 1
# Warmup Iteration   1: 35242.744 ops/ms
# Warmup Iteration   2: 39343.880 ops/ms
# Warmup Iteration   3: 41861.348 ops/ms
# Warmup Iteration   4: 43479.735 ops/ms
# Warmup Iteration   5: 43060.873 ops/ms
Iteration   1: 40059.448 ops/ms
Iteration   2: 42027.314 ops/ms
Iteration   3: 42882.505 ops/ms
Iteration   4: 43190.173 ops/ms
Iteration   5: 43106.067 ops/ms
Iteration   6: 38970.701 ops/ms
Iteration   7: 31057.854 ops/ms
Iteration   8: 38502.386 ops/ms
Iteration   9: 41849.571 ops/ms
Iteration  10: 39960.991 ops/ms
Iteration  11: 42030.462 ops/ms
Iteration  12: 39700.306 ops/ms
Iteration  13: 33005.496 ops/ms
Iteration  14: 32551.353 ops/ms
Iteration  15: 40351.816 ops/ms
Iteration  16: 42844.689 ops/ms
Iteration  17: 39530.398 ops/ms
Iteration  18: 41194.335 ops/ms
Iteration  19: 38503.230 ops/ms
Iteration  20: 42519.444 ops/ms


Result "date":
  39691.927 ±(99.9%) 3112.498 ops/ms [Average]
  (min, avg, max) = (31057.854, 39691.927, 43190.173), stdev = 3584.357
  CI (99.9%): [36579.429, 42804.425] (assumes normal distribution)


# Run complete. Total time: 00:00:53

Benchmark                   Mode  Cnt      Score      Error   Units
ExampleBenchmark1.calndar  thrpt   20   2204.926 ±   29.482  ops/ms
ExampleBenchmark1.date     thrpt   20  39691.927 ± 3112.498  ops/ms

各ベンチマークの取得が1回ずつ行われています。
(最終的な結果の「Cnt」の値が、1回のベンチマーク取得のイテレーション数(=20)と一致しています)


fork数=2の場合

@Frokアノテーションでパラメータに2を指定した場合の動きの確認です

■サンプルコード

import java.util.concurrent.TimeUnit;

import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Warmup;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;

@Warmup(iterations=5)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@BenchmarkMode(Mode.Throughput)
@Fork(2)
public class ExampleBenchmark2 {

    @Benchmark
    public void date() {
        java.util.Date d = new java.util.Date();
    }

    @Benchmark
    public void calndar() {
        java.util.Calendar c = java.util.Calendar.getInstance();
    }

    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                 .include(ExampleBenchmark2.class.getName())
                .build();
        new Runner(opt).run();
    }
}

■実行結果

(一部省略)

# Warmup: 5 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.jmh.sample.fork.ExampleBenchmark2.calndar

# Run progress: 0.00% complete, ETA 00:01:40
# Fork: 1 of 2
# Warmup Iteration   1: 380.554 ops/ms
# Warmup Iteration   2: 1404.574 ops/ms
# Warmup Iteration   3: 1775.697 ops/ms
# Warmup Iteration   4: 2199.900 ops/ms
# Warmup Iteration   5: 2174.488 ops/ms
Iteration   1: 2197.306 ops/ms
Iteration   2: 2155.286 ops/ms
Iteration   3: 1672.854 ops/ms
Iteration   4: 1956.515 ops/ms
Iteration   5: 2166.409 ops/ms
Iteration   6: 2177.722 ops/ms
Iteration   7: 2202.844 ops/ms
Iteration   8: 2193.363 ops/ms
Iteration   9: 2145.254 ops/ms
Iteration  10: 1984.780 ops/ms
Iteration  11: 2074.642 ops/ms
Iteration  12: 2176.908 ops/ms
Iteration  13: 2235.651 ops/ms
Iteration  14: 2157.439 ops/ms
Iteration  15: 2106.411 ops/ms
Iteration  16: 2237.605 ops/ms
Iteration  17: 2222.141 ops/ms
Iteration  18: 2257.170 ops/ms
Iteration  19: 2068.010 ops/ms
Iteration  20: 2172.682 ops/ms

# Run progress: 25.00% complete, ETA 00:01:19
# Fork: 2 of 2
# Warmup Iteration   1: 790.197 ops/ms
# Warmup Iteration   2: 1689.466 ops/ms
# Warmup Iteration   3: 1954.762 ops/ms
# Warmup Iteration   4: 2167.059 ops/ms
# Warmup Iteration   5: 2138.363 ops/ms
Iteration   1: 2185.719 ops/ms
Iteration   2: 2191.517 ops/ms
Iteration   3: 2198.738 ops/ms
Iteration   4: 2171.285 ops/ms
Iteration   5: 2156.772 ops/ms
Iteration   6: 2177.691 ops/ms
Iteration   7: 1860.556 ops/ms
Iteration   8: 1778.019 ops/ms
Iteration   9: 2095.871 ops/ms
Iteration  10: 2131.721 ops/ms
Iteration  11: 2172.790 ops/ms
Iteration  12: 2155.680 ops/ms
Iteration  13: 1750.421 ops/ms
Iteration  14: 2020.798 ops/ms
Iteration  15: 2188.226 ops/ms
Iteration  16: 2192.676 ops/ms
Iteration  17: 2123.054 ops/ms
Iteration  18: 1838.596 ops/ms
Iteration  19: 1941.434 ops/ms
Iteration  20: 1854.807 ops/ms


Result "calndar":
  2093.684 ±(99.9%) 83.611 ops/ms [Average]
  (min, avg, max) = (1672.854, 2093.684, 2257.170), stdev = 148.619
  CI (99.9%): [2010.073, 2177.295] (assumes normal distribution)


# Warmup: 5 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.jmh.sample.fork.ExampleBenchmark2.date

# Run progress: 50.00% complete, ETA 00:00:52
# Fork: 1 of 2
# Warmup Iteration   1: 36140.792 ops/ms
# Warmup Iteration   2: 34133.137 ops/ms
# Warmup Iteration   3: 41963.870 ops/ms
# Warmup Iteration   4: 42815.723 ops/ms
# Warmup Iteration   5: 43747.612 ops/ms
Iteration   1: 42782.092 ops/ms
Iteration   2: 43386.928 ops/ms
Iteration   3: 43478.123 ops/ms
Iteration   4: 43853.401 ops/ms
Iteration   5: 43457.458 ops/ms
Iteration   6: 41413.993 ops/ms
Iteration   7: 42727.939 ops/ms
Iteration   8: 41937.952 ops/ms
Iteration   9: 41972.959 ops/ms
Iteration  10: 38810.244 ops/ms
Iteration  11: 33062.208 ops/ms
Iteration  12: 39069.110 ops/ms
Iteration  13: 42127.344 ops/ms
Iteration  14: 41802.879 ops/ms
Iteration  15: 41666.472 ops/ms
Iteration  16: 41320.208 ops/ms
Iteration  17: 38895.911 ops/ms
Iteration  18: 39079.369 ops/ms
Iteration  19: 41867.575 ops/ms
Iteration  20: 38397.728 ops/ms

# Run progress: 75.00% complete, ETA 00:00:26
# Fork: 2 of 2
# Warmup Iteration   1: 35586.457 ops/ms
# Warmup Iteration   2: 37043.212 ops/ms
# Warmup Iteration   3: 43079.716 ops/ms
# Warmup Iteration   4: 43520.875 ops/ms
# Warmup Iteration   5: 43306.178 ops/ms
Iteration   1: 43471.973 ops/ms
Iteration   2: 43707.980 ops/ms
Iteration   3: 43700.414 ops/ms
Iteration   4: 43308.918 ops/ms
Iteration   5: 42410.655 ops/ms
Iteration   6: 43507.472 ops/ms
Iteration   7: 43828.828 ops/ms
Iteration   8: 43538.341 ops/ms
Iteration   9: 43601.270 ops/ms
Iteration  10: 43396.345 ops/ms
Iteration  11: 43404.096 ops/ms
Iteration  12: 43608.037 ops/ms
Iteration  13: 43146.327 ops/ms
Iteration  14: 43485.104 ops/ms
Iteration  15: 43512.130 ops/ms
Iteration  16: 43681.587 ops/ms
Iteration  17: 43497.131 ops/ms
Iteration  18: 43712.018 ops/ms
Iteration  19: 41678.344 ops/ms
Iteration  20: 43141.722 ops/ms


Result "date":
  42211.215 ±(99.9%) 1215.111 ops/ms [Average]
  (min, avg, max) = (33062.208, 42211.215, 43853.401), stdev = 2159.858
  CI (99.9%): [40996.104, 43426.325] (assumes normal distribution)


# Run complete. Total time: 00:01:45

Benchmark                   Mode  Cnt      Score      Error   Units
ExampleBenchmark2.calndar  thrpt   40   2093.684 ±   83.611  ops/ms
ExampleBenchmark2.date     thrpt   40  42211.215 ± 1215.111  ops/ms

各ベンチマークの取得が2回ずつ行われています。
(最終的な結果の「Cnt」の値が、1回のベンチマーク取得のイテレーション数×2(=40)と一致しています)



関連エントリ