开发笔记:BenchmarkDotNet性能测试

BenchmarkDotNet是一款开源的性能测试工具&＃xff0c;使用方式非常简单&＃xff0c;特别是对实现同一功能的两种方式&＃xff0c;犹豫不决时&＃xff0c;可以使用它进行个对比。

比如我们比较ADO.NET方式查询数据库表&＃xff0c;和用Dapper方式获取数据为表&＃xff0c;只需要在两个方法上增加 [Benchmark]就可以了&＃xff0c;然后调用var summary &＃61; BenchmarkRunner.Run();就实现了性能测试。

using System;
using System.ComponentModel;
using System.Data;
using System.Data.SqlClient;
using System.Security.Cryptography;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using Dapper;
namespace MyBenchmarks
{
public class Product
{
public string Name { get; set; }
public string ProductNumber { get; set; }
public bool MakeFlag { get; set; }
public bool FinishedGoodsFlag { get; set; }
public string Color { get; set; }
public short SafetyStockLevel { get; set; }
public short ReorderPoint { get; set; }
public decimal StandardCost { get; set; }
public decimal ListPrice { get; set; }
public string Size { get; set; }
public string SizeUnitMeasureCode { get; set; }
public string WeightUnitMeasureCode { get; set; }
public double Weight { get; set; }
public int DaysToManufacture { get; set; }
public string ProductLine { get; set; }
public string Class { get; set; }
public string Style { get; set; }
public int ProductSubcategoryID { get; set; }
public int ProductModelID { get; set; }
public DateTime SellStartDate { get; set; }
public DateTime SellEndDate { get; set; }
public DateTime DiscontinuedDate { get; set; }
public Guid rowguid { get; set; }
public DateTime ModifiedDate { get; set; }

}
public class Cycle
{
[Benchmark]
public void DapperTest()
{
var conn &＃61; new SqlConnection("Data Source&＃61;.;Initial Catalog&＃61;AdventureWorks2016;Persist Security Info&＃61;True;User ID&＃61;sa;password&＃61;sa;");
var table &＃61; conn.Query("select top 100 * from production.product");
}
[Benchmark]
public void CommandTest()
{
var conn &＃61; "Data Source&＃61;.;Initial Catalog&＃61;AdventureWorks2016;Persist Security Info&＃61;True;User ID&＃61;sa;password&＃61;sa;";
var sql &＃61; "select top 100 * from production.product";
var dapp &＃61; new SqlDataAdapter(sql, conn);
var table &＃61; new DataTable();
dapp.Fill(table);
}
}
public class Program
{
public static void Main(string[] args)
{
var summary &＃61; BenchmarkRunner.Run();
}
}
}


运行一下看一下结果&＃xff0c;为了减少篇幅&＃xff0c;删除了一些结果&＃xff0c;从结果看出&＃xff0c;为了得到较为客观的性能测试结果&＃xff0c;BenchmarkDotNet还是做了很多工作的&＃xff1a;

OverheadJitting&＃xff0c;WorkloadJitting&＃xff0c;WorkloadPilot&＃xff0c;OverheadWarmup&＃xff0c;OverheadActual&＃xff0c;WorkloadWarmup&＃xff0c;WorkloadActual&＃xff0c;WorkloadResult。

并且通过最后的一个表格&＃xff0c;清楚的给出了结果。同时还给出了Warnings &＃xff0c;以供参考。

// Validating benchmarks:
// ***** BenchmarkRunner: Start *****
// ***** Found 2 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start *****
// start dotnet restore /p:UseSharedCompilation&＃61;false /p:BuildInParallel&＃61;false /m:1 /p:Deterministic&＃61;true /p:Optimize&＃61;true in C:\\MyFile\\Source\\Repos\\Asp.NetCoreExperiment\\Asp.NetCoreExperiment\\BenchmarkDotNet\\Demo01\\bin\\Release\\net6.0\\03bf0209-ac26-4de1-b763-4ba03e46cf4d
// command took 2.83s and exited with 0
// start dotnet build -c Release --no-restore /p:UseSharedCompilation&＃61;false /p:BuildInParallel&＃61;false /m:1 /p:Deterministic&＃61;true /p:Optimize&＃61;true in C:\\MyFile\\Source\\Repos\\Asp.NetCoreExperiment\\Asp.NetCoreExperiment\\BenchmarkDotNet\\Demo01\\bin\\Release\\net6.0\\03bf0209-ac26-4de1-b763-4ba03e46cf4d
// command took 4.29s and exited with 0
// ***** Done, took 00:00:07 (7.52 sec) *****
// Found 2 benchmarks:
// Cycle.DapperTest: DefaultJob
// Cycle.CommandTest: DefaultJob
// **************************
// Benchmark: Cycle.DapperTest: DefaultJob
// *** Execute ***
// Launch: 1 / 1
// Execute: dotnet "03bf0209-ac26-4de1-b763-4ba03e46cf4d.dll" --benchmarkName "MyBenchmarks.Cycle.DapperTest" --job "Default" --benchmarkId 0 in C:\\MyFile\\Source\\Repos\\Asp.NetCoreExperiment\\Asp.NetCoreExperiment\\BenchmarkDotNet\\Demo01\\bin\\Release\\net6.0\\03bf0209-ac26-4de1-b763-4ba03e46cf4d\\bin\\Release\\net6.0
// BeforeAnythingElse
// Benchmark Process Environment Information:
// Runtime&＃61;.NET 6.0.0 (6.0.21.25307), X64 RyuJIT
// GC&＃61;Concurrent Workstation
// Job: DefaultJob
OverheadJitting 1: 1 op, 229700.00 ns, 229.7000 us/op
WorkloadJitting 1: 1 op, 410021000.00 ns, 410.0210 ms/op
WorkloadWarmup 1: 1 op, 4581600.00 ns, 4.5816 ms/op
WorkloadWarmup 2: 1 op, 746500.00 ns, 746.5000 us/op
WorkloadWarmup 3: 1 op, 830000.00 ns, 830.0000 us/op
WorkloadWarmup 4: 1 op, 822900.00 ns, 822.9000 us/op
WorkloadWarmup 5: 1 op, 741800.00 ns, 741.8000 us/op
WorkloadWarmup 6: 1 op, 746900.00 ns, 746.9000 us/op
WorkloadWarmup 7: 1 op, 826300.00 ns, 826.3000 us/op
WorkloadWarmup 8: 1 op, 698300.00 ns, 698.3000 us/op
// BeforeActualRun
WorkloadActual 1: 1 op, 755100.00 ns, 755.1000 us/op
……
WorkloadActual 100: 1 op, 712700.00 ns, 712.7000 us/op
// AfterActualRun
WorkloadResult 1: 1 op, 755100.00 ns, 755.1000 us/op
……
WorkloadResult 95: 1 op, 712700.00 ns, 712.7000 us/op
// AfterAll
// Benchmark Process 26724 has exited with code 0
Mean &＃61; 753.054 us, StdErr &＃61; 11.687 us (1.55%), N &＃61; 95, StdDev &＃61; 113.909 us
Min &＃61; 578.400 us, Q1 &＃61; 668.600 us, Median &＃61; 734.900 us, Q3 &＃61; 798.800 us, Max &＃61; 1,052.600 us
IQR &＃61; 130.200 us, LowerFence &＃61; 473.300 us, UpperFence &＃61; 994.100 us
ConfidenceInterval &＃61; [713.353 us; 792.754 us] (CI 99.9%), Margin &＃61; 39.701 us (5.27% of Mean)
Skewness &＃61; 0.87, Kurtosis &＃61; 2.93, MValue &＃61; 3.63
// **************************
// Benchmark: Cycle.CommandTest: DefaultJob
// *** Execute ***
// Launch: 1 / 1
// Execute: dotnet "03bf0209-ac26-4de1-b763-4ba03e46cf4d.dll" --benchmarkName "MyBenchmarks.Cycle.CommandTest" --job "Default" --benchmarkId 1 in C:\\MyFile\\Source\\Repos\\Asp.NetCoreExperiment\\Asp.NetCoreExperiment\\BenchmarkDotNet\\Demo01\\bin\\Release\\net6.0\\03bf0209-ac26-4de1-b763-4ba03e46cf4d\\bin\\Release\\net6.0
// BeforeAnythingElse
// Benchmark Process Environment Information:
// Runtime&＃61;.NET 6.0.0 (6.0.21.25307), X64 RyuJIT
// GC&＃61;Concurrent Workstation
// Job: DefaultJob
OverheadJitting 1: 1 op, 215400.00 ns, 215.4000 us/op
WorkloadJitting 1: 1 op, 339247500.00 ns, 339.2475 ms/op
WorkloadWarmup 1: 1 op, 3680100.00 ns, 3.6801 ms/op
WorkloadWarmup 2: 1 op, 943000.00 ns, 943.0000 us/op
WorkloadWarmup 3: 1 op, 875200.00 ns, 875.2000 us/op
WorkloadWarmup 4: 1 op, 982500.00 ns, 982.5000 us/op
WorkloadWarmup 5: 1 op, 956900.00 ns, 956.9000 us/op
WorkloadWarmup 6: 1 op, 1030800.00 ns, 1.0308 ms/op
WorkloadWarmup 7: 1 op, 1170900.00 ns, 1.1709 ms/op
WorkloadWarmup 8: 1 op, 1169400.00 ns, 1.1694 ms/op
// BeforeActualRun
WorkloadActual 1: 1 op, 975400.00 ns, 975.4000 us/op
……
WorkloadActual 100: 1 op, 897400.00 ns, 897.4000 us/op
// AfterActualRun
WorkloadResult 1: 1 op, 975400.00 ns, 975.4000 us/op
……
WorkloadResult 91: 1 op, 897400.00 ns, 897.4000 us/op
// AfterAll
// Benchmark Process 17300 has exited with code 0
Mean &＃61; 951.263 us, StdErr &＃61; 13.676 us (1.44%), N &＃61; 91, StdDev &＃61; 130.465 us
Min &＃61; 759.300 us, Q1 &＃61; 864.000 us, Median &＃61; 912.900 us, Q3 &＃61; 1,004.600 us, Max &＃61; 1,316.000 us
IQR &＃61; 140.600 us, LowerFence &＃61; 653.100 us, UpperFence &＃61; 1,215.500 us
ConfidenceInterval &＃61; [904.736 us; 997.789 us] (CI 99.9%), Margin &＃61; 46.526 us (4.89% of Mean)
Skewness &＃61; 0.95, Kurtosis &＃61; 3.05, MValue &＃61; 2.44
// ***** BenchmarkRunner: Finish *****
// * Export *
BenchmarkDotNet.Artifacts\\results\\MyBenchmarks.Cycle-report.csv
BenchmarkDotNet.Artifacts\\results\\MyBenchmarks.Cycle-report-github.md
BenchmarkDotNet.Artifacts\\results\\MyBenchmarks.Cycle-report.html
// * Detailed results *
Cycle.DapperTest: DefaultJob
Runtime &＃61; .NET 6.0.0 (6.0.21.25307), X64 RyuJIT; GC &＃61; Concurrent Workstation
Mean &＃61; 753.054 us, StdErr &＃61; 11.687 us (1.55%), N &＃61; 95, StdDev &＃61; 113.909 us
Min &＃61; 578.400 us, Q1 &＃61; 668.600 us, Median &＃61; 734.900 us, Q3 &＃61; 798.800 us, Max &＃61; 1,052.600 us
IQR &＃61; 130.200 us, LowerFence &＃61; 473.300 us, UpperFence &＃61; 994.100 us
ConfidenceInterval &＃61; [713.353 us; 792.754 us] (CI 99.9%), Margin &＃61; 39.701 us (5.27% of Mean)
Skewness &＃61; 0.87, Kurtosis &＃61; 2.93, MValue &＃61; 3.63
-------------------- Histogram --------------------
[ 545.635 us ; 582.035 us) | &＃64;
[ 582.035 us ; 650.935 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[ 650.935 us ; 726.135 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[ 726.135 us ; 791.665 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[ 791.665 us ; 852.365 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[ 852.365 us ; 924.365 us) | &＃64;&＃64;&＃64;
[ 924.365 us ; 1,009.685 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[1,009.685 us ; 1,085.365 us) | &＃64;&＃64;
---------------------------------------------------
Cycle.CommandTest: DefaultJob
Runtime &＃61; .NET 6.0.0 (6.0.21.25307), X64 RyuJIT; GC &＃61; Concurrent Workstation
Mean &＃61; 951.263 us, StdErr &＃61; 13.676 us (1.44%), N &＃61; 91, StdDev &＃61; 130.465 us
Min &＃61; 759.300 us, Q1 &＃61; 864.000 us, Median &＃61; 912.900 us, Q3 &＃61; 1,004.600 us, Max &＃61; 1,316.000 us
IQR &＃61; 140.600 us, LowerFence &＃61; 653.100 us, UpperFence &＃61; 1,215.500 us
ConfidenceInterval &＃61; [904.736 us; 997.789 us] (CI 99.9%), Margin &＃61; 46.526 us (4.89% of Mean)
Skewness &＃61; 0.95, Kurtosis &＃61; 3.05, MValue &＃61; 2.44
-------------------- Histogram --------------------
[ 721.230 us ; 780.280 us) | &＃64;&＃64;
[ 780.280 us ; 860.730 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[ 860.730 us ; 936.870 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[ 936.870 us ; 1,020.120 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[1,020.120 us ; 1,080.630 us) | &＃64;&＃64;&＃64;&＃64;&＃64;
[1,080.630 us ; 1,170.980 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[1,170.980 us ; 1,247.120 us) | &＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;&＃64;
[1,247.120 us ; 1,277.930 us) |
[1,277.930 us ; 1,354.070 us) | &＃64;
---------------------------------------------------
// * Summary *
BenchmarkDotNet&＃61;v0.13.0, OS&＃61;Windows 10.0.19043.1052 (21H1/May2021Update)
Intel Core i7-7700HQ CPU 2.80GHz (Kaby Lake), 1 CPU, 8 logical and 4 physical cores
.NET SDK&＃61;6.0.100-preview.4.21255.9
[Host] : .NET 6.0.0 (6.0.21.25307), X64 RyuJIT [AttachedDebugger]
DefaultJob : .NET 6.0.0 (6.0.21.25307), X64 RyuJIT

| Method | Mean | Error | StdDev | Median |
|------------ |---------:|---------:|---------:|---------:|
| DapperTest | 753.1 us | 39.70 us | 113.9 us | 734.9 us |
| CommandTest | 951.3 us | 46.53 us | 130.5 us | 912.9 us |
// * Warnings *
MultimodalDistribution
Cycle.DapperTest: Default -> It seems that the distribution is bimodal (mValue &＃61; 3.63)
Environment
Summary -> Benchmark was executed with attached debugger
MinIterationTime
Cycle.DapperTest: Default -> The minimum observed iteration time is 578.4000 us which is very small. It&＃39;s recommended to increase it to at least 100.0000 ms using more operations.
Cycle.CommandTest: Default -> The minimum observed iteration time is 759.3000 us which is very small. It&＃39;s recommended to increase it to at least 100.0000 ms using more operations.
// * Hints *
Outliers
Cycle.DapperTest: Default -> 5 outliers were removed (1.06 ms..2.17 ms)
Cycle.CommandTest: Default -> 9 outliers were removed (1.37 ms..2.51 ms)
// * Legends *
Mean : Arithmetic mean of all measurements
Error : Half of 99.9% confidence interval
StdDev : Standard deviation of all measurements
Median : Value separating the higher half of all measurements (50th percentile)
1 us : 1 Microsecond (0.000001 sec)
// ***** BenchmarkRunner: End *****
// ** Remained 0 benchmark(s) to run **
Run time: 00:00:03 (3.12 sec), executed benchmarks: 2
Global total time: 00:00:10 (10.65 sec), executed benchmarks: 2
// * Artifacts cleanup *