Browse Source

Add some notes on testing trimming (#48333)

* Add some notes on testing trimming

* Fix slashes in links

Co-authored-by: Eric Erhardt <[email protected]>

---------

Co-authored-by: Eric Erhardt <[email protected]>
Andrew Casey 2 years ago
parent
commit
8aa4a527c5
1 changed files with 62 additions and 0 deletions
  1. 62 0
      docs/Trimming.md

+ 62 - 0
docs/Trimming.md

@@ -48,3 +48,65 @@ This should update the `WarningSuppressions.xml` files associated with projects.
 - LegacyRouteTableFactory.&lt;&gt;c.{Create}b__2_1(System.Reflection.Assembly)
 + LegacyRouteTableFactory.&lt;&gt;c.&lt;Create&gt;b__2_1(System.Reflection.Assembly)
 ```
+
+## Validate trimming behavior
+
+We have infrastructure for writing tests to validate trimming behavior.
+The two most common tasks are confirming that functionality still works after trimming and confirming that a particular type or member was not preserved.
+
+### Infrastructure
+
+Because trimming is only performed during publish, the tests can't be normal xunit tests.
+Instead, we have a system that wraps individual source files in appropriately configured projects and publishes them as standalone executables.
+These executables are then run and the test outcome is based on the exit code.
+The tests are powered by [trimmingTests.targets](../eng/testing/linker/trimmingTests.targets), which is based on corresponding functionality in https://github.com/dotnet/runtime.
+
+### Adding a new test project
+
+Adding a new test project is very simple - just call it *.TrimmingTests.proj and give it a body like
+```xml
+<Project Sdk="Microsoft.NET.Sdk">
+  <ItemGroup>
+    <TestConsoleAppSourceFiles Include="MyTest.cs">
+  </ItemGroup>
+</Project>
+```
+
+Unfortunately, there's no good way to specify which aspnetcore projects the tests depend upon so, to avoid dependency ordering problems, it's easiest to specify that they should be built after the main build.
+Do this by adding an entry to [RequiresDelayedBuildProjects.props](../eng/RequiresDelayedBuildProjects.props).
+
+### Adding a new test
+
+A test is just a C# file with a main method (or top-level statements, if you prefer).
+Write a program that either should keep working after trimming or that validates a particular type/member has been trimmed (e.g. using `Type.GetType`).
+If things behave as expected exit with code `100` (a magic number intended to prevent accidental passes).
+Any other result - a different exit code or a crash - will count as a failure.
+
+Now that you have a test program, add a `TestConsoleAppSourceFiles` item to the corresponding test project.
+
+If you have additional files, typically because you want to share code between tests, you can use `AdditionalSourceFiles`.
+If you want to en/disable a particular feature switch, you can use `EnabledFeatureSwitches`/`DisabledFeatureSwitches`.
+
+### Running tests
+
+To run the tests locally, build the projects it depends on (it's usually easiest to just build the whole repo) and then run
+```
+.dotnet\dotnet.exe test path\to\project\folder\
+```
+
+Alternatively, you can use `Activate.ps1` to ensure you're using the right dotnet binary.
+
+### Native AOT
+
+Native AOT testing is substantially the same - just name your project *.NativeAotTests.proj, rather than *.TrimmingTests.proj.
+
+### Tips
+
+- It's easier to author your test as a standalone Native AOT app and then copy the final code into the test (don't forget to add the copyright header).
+  The two main advantages are that building is much faster and you get console output that you can use to help debug your test.
+- If you are using AdditionalSourceFiles and you need to make a change to one of the additional files, you will also need to update the test file itself - just editing the additional file _won't trigger a rebuild_ on the next run.
+- `Type.GetType` is known to the trimmer, so you'll have to (a) make it impossible to statically determine what argument you're passing (so that it isn't preserved) and (b) suppress the resulting warning: `[UnconditionalSuppressMessage("ReflectionAnalysis", "IL2057:UnrecognizedReflectionPattern", Justification = "Using GetType to test trimming")]`
+- Pass an assembly-qualified name (`"ns.type, assembly.name"`) to `Type.GetType` as it searches only the executing assembly and corlib by default.
+- There's no output indicating that a test has passed, so it's important to check that it's actually running (e.g. by inserting an artificial failure).
+- If your test depends on any JSON serialization functionality, you'll probably need to disable the `System.Text.Json.JsonSerializer.IsReflectionEnabledByDefault` feature switch.
+- If your test validates that a type is being trimmed, it's good practice to write a dual test (sharing the same helpers) that validates that it is preserved (in a different scenario, of course) so that the trimming test doesn't accidentally pass because of (e.g.) a typo in the type name passed to `GetType`.