Wasm tests use `Microsoft.NET.Sdk.WebAssembly` and support CoreClr runtime by ilonatommy · Pull Request #2998 · dotnet/BenchmarkDotNet

ilonatommy · 2026-02-11T17:20:34Z

Problem

The autogenerated benchmark project uses Microsoft.NET.Sdk, which does not set UseMonoRuntime=true. This causes ProcessFrameworkReferences to resolve the CoreCLR runtime pack (Microsoft.NETCore.App.Runtime.browser-wasm) instead of the Mono one (Microsoft.NETCore.App.Runtime.Mono.browser-wasm). The CoreCLR pack does not exist for browser-wasm, so the build fails.

PR #2994 worked around this by adding -r browser-wasm to the restore command, but this only defers the error — the runtime pack is still not resolved correctly during build.

Fix

Switch the autogenerated WASM project SDK from Microsoft.NET.Sdk to Microsoft.NET.Sdk.WebAssembly. This SDK auto-defaults UseMonoRuntime=true, ensuring the correct Mono runtime pack is selected, and auto-defaults PublishTrimmed=true, so we can drop that property too.

Additional cleanup

Rename test-main.js → benchmark-main.mjs: BDN's 9-line entry point was confusingly named the same as the runtime's 379-line test-main.js. Using .mjs also tells JS engines the file is an ES module, eliminating the need for --module flags.
Embed benchmark-main.mjs as a template resource: Instead of shipping it as a test asset in AppBundle/, it's now an embedded resource placed into wwwroot/ during project generation — matching the WebAssembly SDK's output layout.
Remove WasmGenerateRunV8Script: This generated an unused run-v8.sh script. BDN invokes the JS engine directly.
Remove redundant PublishTrimmed: The WebAssembly SDK defaults this to true.
Remove -r browser-wasm from restore: The WebAssembly SDK sets RuntimeIdentifier in the project, so the explicit restore flag is no longer needed.
Update working directory from AppBundle/ to wwwroot/: Matches the WebAssembly SDK's publish output layout.

Testing

Integration tests: 4/4 WasmTests pass (net8.0, AOT + Interpreter × V8 + JSC)
benchmarks_ci.py: 5/5 Perf_Boolean.Parse benchmarks pass on Wasm AOT

maraf · 2026-02-12T13:11:19Z

The failure is caused by the fact that autogenerated project uses Microsoft.NET.Sdk and thus, keeps UseMonoRuntime=false by default. As a result, restore tries to download clr runtime, not mono runtime.

Instead of explicitly passing UseMonoRuntime, we should disable runtime/targeting pack download, similar to dotnet/runtime#124288

ilonatommy · 2026-02-12T13:14:00Z

Instead of explicitly passing UseMonoRuntime, we should disable runtime/targeting pack download, similar to dotnet/runtime#124288

We don't pass it explicitly. We decided to change the sdk type instead, to change the value UseMonoRuntime defaults to.

…RunV8Script and redundant PublishTrimmed.

src/BenchmarkDotNet/Templates/WasmCsProj.txt

tests/BenchmarkDotNet.IntegrationTests/WasmTests.cs

src/BenchmarkDotNet/Toolchains/MonoWasm/WasmGenerator.cs

Require --output (BuildPartition.cs) — WASM needs explicit output paths because the WebAssembly SDK publishes to wwwroot/ inside the output directory. Without --output, build artifacts land in the default bin/ tree where the executor doesn't look. excludeArtifactsPath (DotNetCliCommand.cs) — BDN normally passes /p:ArtifactsPath to redirect output. But the SDK auto-sets UseArtifactsOutput=true when ArtifactsPath is non-empty, which adds $(ArtifactsPath)/** to DefaultItemExcludes — excluding every file in the project directory (including wwwroot/) from default Content globs. For WASM, we pass /p:UseArtifactsOutput=false instead, since OutDir/OutputPath/--output already control where output goes. Content Update (WasmCsProj.txt) — With UseArtifactsOutput=false, the SDK's default <Content Include="wwwroot\**" ...> glob works, so Content Update can upgrade those items with CopyToOutputDirectory=PreserveNewest.

timcassell · 2026-02-14T09:24:52Z

src/BenchmarkDotNet/Toolchains/DotNetCli/DotNetCliCommand.cs

        internal static string GetBuildCommand(ArtifactsPaths artifactsPaths, BuildPartition buildPartition, string filePath, string tfm, string? extraArguments = null, string? binLogSuffix = null, bool excludeOutput = false)
-            => new StringBuilder()
+        {
+            bool excludeArtifactsPath = buildPartition.RepresentativeBenchmarkCase.Job.GetToolchain() is WasmToolchain;


Better to pass it in as an optional argument.

timcassell · 2026-02-14T09:28:41Z

src/BenchmarkDotNet/Toolchains/DotNetCli/DotNetCliCommand.cs

+                    // exclude all project files (including wwwroot/) from default Content globs.
+                    .AppendArgument(excludeArtifactsPath
+                        ? "/p:UseArtifactsOutput=false"
+                        : $"/p:ArtifactsPath=\"{artifactsPaths.BuildArtifactsDirectoryPath}{Path.AltDirectorySeparatorChar}\"")


We use artifacts path so that builds are safe to run in parallel. Without it, the build must be ran sequentially. If this is truly required, you'll need to update

BenchmarkDotNet/src/BenchmarkDotNet/Running/BenchmarkRunnerClean.cs

Lines 90 to 97 in c342c49

var sequentialBuildPartitions = buildPartitions.Where(partition =>

partition.Benchmarks.Any(x => x.Config.Options.IsSet(ConfigOptions.DisableParallelBuild))

// .Net SDK 8+ supports ArtifactsPath for proper parallel builds.

// Older SDKs may produce builds with incorrect bindings if more than 1 partition is built in parallel.

|| (partition.RepresentativeBenchmarkCase.GetToolchain().Generator is DotNetCliGenerator

&& partition.RepresentativeBenchmarkCase.GetRuntime().RuntimeMoniker.GetRuntimeVersion().Major < 8)

)

.ToArray();

timcassell · 2026-02-14T11:57:30Z

src/BenchmarkDotNet/Toolchains/MonoWasm/WasmGenerator.cs

        {
            CustomRuntimePack = customRuntimePack;
-            MainJS = (targetFrameworkMoniker == "net5.0" || targetFrameworkMoniker == "net6.0") ? "main.js" : "test-main.js";
+            MainJS = (targetFrameworkMoniker == "net5.0" || targetFrameworkMoniker == "net6.0") ? "main.js" : "benchmark-main.mjs";


It looks like the net5.0 and net6.0 path can be removed now, and the MainJS field can be made a constant.

ilonatommy added 2 commits February 11, 2026 18:13

Fix.

131269b

Move test-main.js to Templates as embedded resource for WasmGenerator,

c642f31

ilonatommy added a commit to dotnet/performance that referenced this pull request Feb 12, 2026

Correct WasmMainJSPath to match changes in dotnet/BenchmarkDotNet#2998.

1e0203a

ilonatommy mentioned this pull request Feb 12, 2026

WASM benchmark uses WebAssembly SDK and supports CoreClr runtime dotnet/performance#5114

Open

Add missing namespace.

431e5de

ilonatommy mentioned this pull request Feb 12, 2026

Migrate to WebAssembly SDK dotnet/performance#5101

Closed

ilonatommy added 3 commits February 12, 2026 15:41

Rename test-main.js to benchmark-main.mjs; remove unused WasmGenerate…

758df2b

…RunV8Script and redundant PublishTrimmed.

Comment was out-of-date.

708faaf

Missing changes.

bd7110c

ilonatommy marked this pull request as ready for review February 13, 2026 07:19

maraf reviewed Feb 13, 2026

View reviewed changes

src/BenchmarkDotNet/Templates/WasmCsProj.txt Outdated Show resolved Hide resolved

tests/BenchmarkDotNet.IntegrationTests/WasmTests.cs Outdated Show resolved Hide resolved

pavelsavara reviewed Feb 13, 2026

View reviewed changes

src/BenchmarkDotNet/Toolchains/MonoWasm/WasmGenerator.cs Show resolved Hide resolved

ilonatommy added 2 commits February 13, 2026 15:20

Feedback: enable coreclr microbenchmarks.

a42ec61

Add runtime flavor switch for CLI.

62df686

ilonatommy changed the title ~~Wasm tests use Microsoft.NET.Sdk.WebAssembly~~ Wasm tests use Microsoft.NET.Sdk.WebAssembly and support CoreClr runtime Feb 13, 2026

timcassell reviewed Feb 14, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Wasm tests use `Microsoft.NET.Sdk.WebAssembly` and support CoreClr runtime#2998

Wasm tests use `Microsoft.NET.Sdk.WebAssembly` and support CoreClr runtime#2998
ilonatommy wants to merge 9 commits intodotnet:masterfrom
ilonatommy:wasm-tests-use-wasm-sdk

ilonatommy commented Feb 11, 2026 •

edited

Loading

Uh oh!

maraf commented Feb 12, 2026

Uh oh!

ilonatommy commented Feb 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

timcassell Feb 14, 2026

Uh oh!

timcassell Feb 14, 2026

Uh oh!

timcassell Feb 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	var sequentialBuildPartitions = buildPartitions.Where(partition =>
	partition.Benchmarks.Any(x => x.Config.Options.IsSet(ConfigOptions.DisableParallelBuild))
	// .Net SDK 8+ supports ArtifactsPath for proper parallel builds.
	// Older SDKs may produce builds with incorrect bindings if more than 1 partition is built in parallel.
	\|\| (partition.RepresentativeBenchmarkCase.GetToolchain().Generator is DotNetCliGenerator
	&& partition.RepresentativeBenchmarkCase.GetRuntime().RuntimeMoniker.GetRuntimeVersion().Major < 8)
	)
	.ToArray();

Uh oh!

Conversation

ilonatommy commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Fix

Additional cleanup

Testing

Uh oh!

maraf commented Feb 12, 2026

Uh oh!

ilonatommy commented Feb 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

timcassell Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

timcassell Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

timcassell Feb 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ilonatommy commented Feb 11, 2026 •

edited

Loading