Enhancements to Helix testing (#10007)

* Add Windows 7 and 8.1 testing on Helix
* Install SQL Server on-demand in Helix test queues
* Only install mssql on Windows runs
* Use exit /b
* Add targets to better support running helix locally
* Set maxretrycount to 2
* Handle IIS issues on win7/win8
* Make HelixPreCommand's fail the workitem
* Add a pre-generated test cert of IIS Express
* Update helix doc and ignore netsh script failures
* Fix bug in detecting Windows queues and disable Win 7 until we have queues ready
* Fix HttpSys functional tests on Helix

docs/Helix.md

@@ -2,12 +2,30 @@ Helix testing in ASP.NET Core
 ==============================
 
 Helix is the distributed test platform that we use to run tests.  We build a helix payload that contains the publish directory of every test project that we want to test
-send a job with with this payload to a set of queues for the various combinations of OS that we want to test 
+send a job with with this payload to a set of queues for the various combinations of OS that we want to test
 for example: `Windows.10.Amd64.ClientRS4.VS2017.Open`, `OSX.1012.Amd64.Open`, `Ubuntu.1810.Amd64.Open`. Helix takes care of unzipping, running the job, and reporting results.
 
-For more info about helix see: [SDK](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.Helix/Sdk/Readme.md), [JobSender](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.Helix/Sdk/Readme.md) 
+For more info about helix see: [SDK](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.Helix/Sdk/Readme.md), [JobSender](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.Helix/Sdk/Readme.md)
 
-## How do I look at the results of a helix run?
+## Running helix tests locally
+
+To run Helix tests for one particular test project:
+
+```
+cd src/MyCode/test
+dotnet build /t:Helix
+```
+
+To run tests for the entire repo, run:
+
+```
+.\build.cmd /t:Helix
+```
+
+This will restore, and then publish all of the test projects including some bootstrapping scripts that will install the correct dotnet runtime/sdk before running the test assemblies on the helix machine, and upload the job to helix, it won't wait for the jobs to complete, but you can go to https://mc.dot.net/#/user/$(your user name)/builds.
+
+
+## How do I look at the results of a helix run on Azure Pipelines?
 There's a link embedded in the build.cmd log of the helix target on Azure Pipelines, near the bottom right that will look something like this:
 ```
 2019-02-07T21:55:48.1516089Z   Results will be available from https://mc.dot.net/#/user/aspnetcore/pr~2Faspnet~2Faspnetcore/ci/20190207.34
@@ -23,7 +41,7 @@ There's a link embedded in the build.cmd log of the helix target on Azure Pipeli
 2019-02-07T22:06:33.6898567Z   Job 82f27d4c-9099-4f0e-b383-870c24d8dc2c is completed with 108 finished work items.
 ```
 
-The link will take you to an overview of all the tests with clickable links to the logs and each run broken down by queue. 
+The link will take you to an overview of all the tests with clickable links to the logs and each run broken down by queue.
 
 All of the helix runs for aspnetcore can be found here https://mc.dot.net/#/user/aspnetcore/builds
 
@@ -40,11 +58,6 @@ If that doesn't help, you can try the Get Repro environment link from mission co
 ## Differences from running tests locally
 Most tests that don't just work on helix automatically are ones that depend on the source code being accessible. The helix payloads only contain whatever is in the publish directories, so any thing else that test depends on will need to be included to the payload (TBD how to do this).
 
-## Running helix tests locally
-`.\build.cmd /t:Helix /p:IsHelixJob=true`
-
-This will restore, and then publish all of the test projects including some bootstrapping scripts that will install the correct dotnet runtime/sdk before running the test assemblies on the helix machine, and upload the job to helix, it won't wait for the jobs to complete, but you can go to https://mc.dot.net/#/user/aspnetcore/builds and look for a source that matches private-yourusername
-
 ## How to skip tests on helix
 There are two main ways to opt out of helix
 - Skipping the entire test project via `<BuildHelixPayload>false</BuildHelixPayload>` in csproj (the default value for this is IsTestProject).

docs/ProjectProperties.md

@@ -7,3 +7,4 @@ Property name      | Meaning
 -------------------|--------------------------------------------------------------------------------------------
 IsShippingPackage  | When set to `true`, the package produced by from project is intended for use by customers. Defaults to  `false`, which means the package is intended for internal use only by Microsoft teams.
 IsAspNetCoreApp    | Set to `true` when the assembly is part of the [Microsoft.AspNetCore.App shared framework](./SharedFramework.md) and is not available as a NuGet package (unless IsShippingPackage is also set to `true`).
+TestDependsOnMssql | Set to `true` when your tests depends on SQL Server. This will ensure distribute tests on Helix install LocalDB ([more information on Helix](./Helix.md)).

eng/helix/content/README.md

@@ -0,0 +1,3 @@
+# Helix content
+
+The content of this folder is included in the test payload for each Helix work item. The code here is mean to be used alongside test binaries.

eng/scripts/RunPowershell.cmd → eng/helix/content/RunPowershell.cmd

@@ -7,5 +7,5 @@ SET POWERSHELL=%windir%\System32\WindowsPowerShell\v1.0\powershell.exe
 
 rem Force 64bit powershell
 if /i "%PROCESSOR_ARCHITEW6432%" EQU "AMD64" SET POWERSHELL=%windir%\sysnative\WindowsPowerShell\v1.0\powershell.exe
-
+echo "PS: Running '%~dp0%1' %_TAIL%"
 %POWERSHELL% -NoProfile -NoLogo -ExecutionPolicy unrestricted -Command "[System.Threading.Thread]::CurrentThread.CurrentCulture = ''; [System.Threading.Thread]::CurrentThread.CurrentUICulture = ''; try { & '%~dp0%1' %_TAIL%; exit $LASTEXITCODE } catch { write-host $_; exit 1 }"

eng/helix/content/mssql/InstallSqlServerLocalDB.ps1

@@ -0,0 +1,34 @@
+<#
+.SYNOPSIS
+    Installs SQL Server 2016 Express LocalDB on a machine.
+.DESCRIPTION
+    This script installs Microsoft SQL Server 2016 Express LocalDB on a machine.
+.LINK
+    https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/sql-server-2016-express-localdb?view=sql-server-2016
+    https://docs.microsoft.com/en-us/sql/database-engine/install-windows/install-sql-server-from-the-command-prompt?view=sql-server-2016
+#>
+
+$ErrorActionPreference = 'Stop'
+$ProgressPreference = 'SilentlyContinue' # Workaround PowerShell/PowerShell#2138
+Set-StrictMode -Version 1
+
+$intermedateDir = "$PSScriptRoot\obj"
+mkdir $intermedateDir -ErrorAction Ignore | Out-Null
+
+Write-Host "Installing SQL Server 2016 Express LocalDB" -f Magenta
+
+# Download SqlLocalDB.msi.
+$installerFilename = "SqlLocalDB.msi"
+$installerPath = "$intermedateDir\$installerFilename"
+Write-Host ""
+Write-Host "Downloading '$installerFilename' to '$installerPath'."
+Invoke-WebRequest -OutFile $installerPath -UseBasicParsing `
+    -Uri 'https://download.microsoft.com/download/9/0/7/907AD35F-9F9C-43A5-9789-52470555DB90/ENU/SqlLocalDB.msi'
+
+# Install LocalDB.
+$arguments = '/package', "`"$installerPath`"", '/NoRestart', '/Passive', `
+  'IACCEPTSQLLOCALDBLICENSETERMS=YES', 'HIDEPROGRESSBAR=YES'
+Write-Host ""
+Write-Host "Running 'msiexec $arguments'."
+$process = Start-Process msiexec.exe -ArgumentList $arguments -NoNewWindow -PassThru -Verbose -Wait
+exit $process.ExitCode

eng/helix/vstest/runtests.cmd → eng/helix/content/runtests.cmd

@@ -1,10 +1,12 @@
+@echo off
 REM Disable "!Foo!" expansions because they break the filter syntax
 setlocal disableextensions
 
 set target=%1
-set sdkVersion=%2
-set runtimeVersion=%3
-set helixQueue=%4
+set targetFrameworkIdentifier=%2
+set sdkVersion=%3
+set runtimeVersion=%4
+set helixQueue=%5
 
 set DOTNET_HOME=%HELIX_CORRELATION_PAYLOAD%\sdk
 set DOTNET_ROOT=%DOTNET_HOME%\x64
@@ -19,13 +21,18 @@ powershell.exe -NoProfile -ExecutionPolicy unrestricted -Command "[Net.ServicePo
 
 set HELIX=%helixQueue%
 
+if (%targetFrameworkIdentifier%==.NETFramework) (
+    xunit.console.exe %target% -xml testResults.xml
+    exit /b %ERRORLEVEL%
+)
+
 %DOTNET_ROOT%\dotnet vstest %target% -lt >discovered.txt
 find /c "Exception thrown" discovered.txt
 REM "ERRORLEVEL is not %ERRORLEVEL%" https://blogs.msdn.microsoft.com/oldnewthing/20080926-00/?p=20743/
 if not errorlevel 1 (
     echo Exception thrown during test discovery. 1>&2
     type discovered.txt 1>&2
-    exit 1
+    exit /b 1
 )
 
 set exit_code=0
@@ -51,5 +58,5 @@ if errorlevel 1 (
     REM DO NOT EXIT and DO NOT SET EXIT_CODE to 1
 )
 
-exit %exit_code%
+exit /b %exit_code%
 

eng/helix/helix.proj

@@ -1,7 +1,7 @@
 <Project Sdk="Microsoft.DotNet.Helix.Sdk" DefaultTargets="Test">
 
-  <Import Project="$(RepositoryRoot)\eng\targets\Helix.Common.props" />
-  <Import Project="$(RepositoryRoot)\build\repo.props" />
+  <Import Project="$(MSBuildThisFileDirectory)..\targets\Helix.Common.props" />
+  <Import Project="$(MSBuildThisFileDirectory)..\..\build\repo.props" />
 
   <ItemGroup>
     <HelixTargetQueue Include="@(HelixAvailableTargetQueue)" />
@@ -9,17 +9,27 @@
 
   <PropertyGroup>
     <HelixSource>pr/aspnet/aspnetcore</HelixSource>
-    <HelixType>ci</HelixType>
     <HelixBuild>private-$(USERNAME)</HelixBuild>
     <HelixBuild Condition=" '$(USERNAME)' == '' ">private-$(USER)</HelixBuild>
-    <HelixBuild Condition=" '$(CI)' == 'true' ">$(BUILD_BUILDNUMBER)</HelixBuild>
-    <WaitForWorkItemCompletion Condition=" '$(CI)' == 'true' ">true</WaitForWorkItemCompletion>
-    <FailOnMissionControlTestFailure>true</FailOnMissionControlTestFailure>
-    <EnableAzurePipelinesReporter>true</EnableAzurePipelinesReporter>
     <IsExternal>true</IsExternal>
-    <Creator>aspnetcore</Creator>
     <SkipInvalidConfigurations>true</SkipInvalidConfigurations>
-    <MaxRetryCount Condition="'$(MaxRetryCount)' == ''">4</MaxRetryCount>
+    <MaxRetryCount Condition="'$(MaxRetryCount)' == ''">2</MaxRetryCount>
+  </PropertyGroup>
+
+  <PropertyGroup Condition=" '$(CI)' == 'true' ">
+    <HelixType>ci</HelixType>
+    <Creator>aspnetcore</Creator>
+    <HelixBuild>$(BUILD_BUILDNUMBER)</HelixBuild>
+    <WaitForWorkItemCompletion>true</WaitForWorkItemCompletion>
+    <EnableAzurePipelinesReporter>true</EnableAzurePipelinesReporter>
+    <FailOnMissionControlTestFailure>true</FailOnMissionControlTestFailure>
+  </PropertyGroup>
+
+  <PropertyGroup Condition=" '$(CI)' != 'true' ">
+    <HelixType>dev</HelixType>
+    <Creator>$(USERNAME)</Creator>
+    <Creator Condition="'$(USERNAME)' == ''">$(USER)</Creator>
+    <HelixBuild>$([System.DateTime]::Now.ToString('yyyyMMdd HH:mm'))</HelixBuild>
   </PropertyGroup>
 
   <Target Name="Gather" BeforeTargets="Build">

eng/helix/xunit/runtests.cmd

@@ -1,3 +0,0 @@
-set target=%1
-set helix=%4
-xunit.console.exe %target% -xml testResults.xml

eng/targets/Helix.Common.props

@@ -12,6 +12,8 @@
     <HelixAvailablePlatform Include="Linux" />
 
     <HelixAvailableTargetQueue Include="Windows.10.Amd64.ClientRS4.VS2017.Open" Platform="Windows" />
+    <HelixAvailableTargetQueue Include="Windows.81.Amd64.Open" Platform="Windows" />
+    <HelixAvailableTargetQueue Include="Windows.7.Amd64.Open" Platform="Windows" />
     <HelixAvailableTargetQueue Include="Windows.10.Amd64.EnterpriseRS3.ASPNET.Open" Platform="Windows" EnableByDefault="false" />
     <HelixAvailableTargetQueue Include="OSX.1012.Amd64.Open" Platform="OSX" />
     <HelixAvailableTargetQueue Include="OSX.1014.Amd64.Open" Platform="OSX" />

eng/targets/Helix.props

@@ -1,38 +1,38 @@
 <Project>
 
-    <Import Project="Helix.Common.props" />
-
-    <ItemDefinitionGroup>
-        <HelixContent>
-            <CopyToBuildDirectory>Never</CopyToBuildDirectory>
-            <CopyToPublishDirectory>Always</CopyToPublishDirectory>
-        </HelixContent>
-    </ItemDefinitionGroup>
-
-    <PropertyGroup>
-        <CreateHelixPayload>true</CreateHelixPayload>
-        <HelixTimeout>00:30:00</HelixTimeout>
-        <IsWindowsHelixQueue>$(HelixTargetQueue.Contains('Windows'))</IsWindowsHelixQueue>
-        <HelixTestName>$(MSBuildProjectName)-$(TargetFramework)</HelixTestName>
-        <HelixUseArchive>false</HelixUseArchive>
-        <LoggingTestingDisableFileLogging Condition="'$(IsHelixJob)' == 'true'">true</LoggingTestingDisableFileLogging>
-    </PropertyGroup>
-
-    <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
-        <HelixProjectPlatform Include="Windows" />
-
-        <HelixContent Include="$(RepositoryRoot)eng\helix\xunit\runtests.cmd" />
-    </ItemGroup>
-
-    <ItemGroup Condition="'$(TargetFrameworkIdentifier)' != '.NETFramework'">
-        <HelixProjectPlatform Include="@(HelixAvailablePlatform)" />
-
-        <HelixContent Include="$(RepositoryRoot)eng\helix\vstest\runtests.sh" />
-        <HelixContent Include="$(RepositoryRoot)eng\helix\vstest\runtests.cmd" />
-    </ItemGroup>
-
-    <ItemGroup>
-        <HelixContent Include="$(RepositoryRoot)eng\scripts\RunPowershell.cmd" />
-    </ItemGroup>
+  <Import Project="Helix.Common.props" />
+
+  <ItemDefinitionGroup>
+    <HelixContent>
+      <CopyToBuildDirectory>Never</CopyToBuildDirectory>
+      <CopyToPublishDirectory>Always</CopyToPublishDirectory>
+    </HelixContent>
+  </ItemDefinitionGroup>
+
+  <PropertyGroup>
+    <CreateHelixPayload>true</CreateHelixPayload>
+    <HelixTimeout>00:30:00</HelixTimeout>
+    <IsWindowsHelixQueue>false</IsWindowsHelixQueue>
+    <IsWindowsHelixQueue Condition="$(HelixTargetQueue.Contains('Windows')) or $(HelixTargetQueue.Contains('windows'))">true</IsWindowsHelixQueue>
+    <HelixTestName>$(MSBuildProjectName)/$(TargetFramework)</HelixTestName>
+    <HelixUseArchive>false</HelixUseArchive>
+    <LoggingTestingDisableFileLogging Condition="'$(IsHelixJob)' == 'true'">true</LoggingTestingDisableFileLogging>
+  </PropertyGroup>
+
+  <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
+    <HelixProjectPlatform Include="Windows" />
+  </ItemGroup>
+
+  <ItemGroup Condition="'$(TargetFrameworkIdentifier)' != '.NETFramework'">
+    <HelixProjectPlatform Include="@(HelixAvailablePlatform)" />
+  </ItemGroup>
+
+  <ItemGroup>
+    <HelixContent Include="$(RepositoryRoot)eng\helix\content\**\*" />
+  </ItemGroup>
+
+  <ItemGroup Condition="'$(TestDependsOnMssql)' == 'true' AND '$(IsWindowsHelixQueue)' == 'true'">
+    <HelixPreCommand Include="call RunPowershell.cmd mssql\InstallSqlServerLocalDB.ps1 || exit /b 1" />
+  </ItemGroup>
 
 </Project>

eng/targets/Helix.targets

@@ -2,13 +2,25 @@
 
   <!-- Item group has to be defined here becasue Helix.props is evaluated before xunit.runner.console.props  -->
   <ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
-      <HelixContent Condition="'$(XunitConsole472Path)' != ''" Include="$([System.IO.Path]::GetDirectoryName('$(XunitConsole472Path)'))/**/*" />
+    <HelixContent Condition="'$(XunitConsole472Path)' != ''" Include="$([System.IO.Path]::GetDirectoryName('$(XunitConsole472Path)'))/**/*" />
   </ItemGroup>
 
   <ItemGroup>
     <Content Include="@(HelixContent)" />
   </ItemGroup>
 
+<!--
+
+This target is meant to be used when invoking helix tests on one project at a time.
+
+Usage: dotnet build /t:Helix src/MyTestProject.csproj
+
+-->
+  <Target Name="Helix">
+    <MSBuild Projects="$(MSBuildThisFileDirectory)..\helix\helix.proj"
+             Properties="Projects=$(MSBuildProjectFullPath)" />
+  </Target>
+
   <!-- Build the actual helix work items to send to helix queues -->
   <Target Name="CreateHelixPayload" Returns="@(HelixWorkItem)">
     <ItemGroup>
@@ -44,10 +56,8 @@
   </Target>
 
   <Target Name="_PublishHelixArchive" DependsOnTargets="Publish" >
-
     <ZipDirectory Condition="'$(HelixUseArchive)' == 'true'" SourceDirectory="$(PublishDir)" DestinationFile="$(PublishDir)../$(HelixTestName).zip" Overwrite="true" />
     <RemoveDir Condition="'$(HelixUseArchive)' == 'true'" Directories="$(PublishDir)" />
-
   </Target>
 
   <Target Name="_CreateHelixWorkItem" Condition="$(BuildHelixPayload)">
@@ -64,7 +74,7 @@
         <TestAssembly>$(TargetFileName)</TestAssembly>
         <PreCommands>@(HelixPreCommand)</PreCommands>
         <PostCommands>@(HelixPostCommand)</PostCommands>
-        <Command Condition="$(IsWindowsHelixQueue)">call runtests.cmd $(TargetFileName) $(NETCoreSdkVersion) $(MicrosoftNETCoreAppPackageVersion) $(HelixTargetQueue)</Command>
+        <Command Condition="$(IsWindowsHelixQueue)">call runtests.cmd $(TargetFileName) $(TargetFrameworkIdentifier) $(NETCoreSdkVersion) $(MicrosoftNETCoreAppPackageVersion) $(HelixTargetQueue)</Command>
         <Command Condition="!$(IsWindowsHelixQueue)">./runtests.sh $(TargetFileName) $(NETCoreSdkVersion) $(MicrosoftNETCoreAppPackageVersion) $(HelixTargetQueue)</Command>
         <Timeout>$(HelixTimeout)</Timeout>
       </HelixWorkItem>

src/Identity/EntityFrameworkCore/test/EF.Test/Microsoft.AspNetCore.Identity.EntityFrameworkCore.Test.csproj

@@ -2,6 +2,7 @@
 
   <PropertyGroup>
     <TargetFramework>netcoreapp3.0</TargetFramework>
+    <TestDependsOnMssql>true</TestDependsOnMssql>
 
     <!-- These tests are broken in Azure Pipelines right now due to deadlocks in EFCore on machines with few CPU cores. -->
     <SkipTests Condition=" '$(BUILD_REASON)' == 'PullRequest' ">true</SkipTests>

src/Middleware/Diagnostics.EntityFrameworkCore/test/FunctionalTests/Diagnostics.EFCore.FunctionalTests.csproj

@@ -4,6 +4,7 @@
     <TargetFramework>netcoreapp3.0</TargetFramework>
     <!-- Mitigation for long path issues -->
     <AssemblyName>Diagnostics.EFCore.FunctionalTests</AssemblyName>
+    <TestDependsOnMssql>true</TestDependsOnMssql>
   </PropertyGroup>
 
   <ItemGroup>

src/Middleware/Diagnostics/test/FunctionalTests/Diagnostics.FunctionalTests.csproj

@@ -4,6 +4,7 @@
     <TargetFramework>netcoreapp3.0</TargetFramework>
     <SignAssembly>false</SignAssembly>
     <AssemblyName>Diagnostics.FunctionalTests</AssemblyName>
+    <TestDependsOnMssql>true</TestDependsOnMssql>
   </PropertyGroup>
 
   <ItemGroup>

src/Servers/HttpSys/test/Directory.Build.props

@@ -0,0 +1,9 @@
+<Project>
+  <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory)..\, Directory.Build.props))\Directory.Build.props" />
+
+  <ItemGroup>
+    <!-- HttpSys tests are Windows-only -->
+    <HelixProjectPlatform Remove="Linux;OSX" />
+  </ItemGroup>
+
+</Project>

src/Servers/HttpSys/test/FunctionalTests/Microsoft.AspNetCore.Server.HttpSys.FunctionalTests.csproj

@@ -8,7 +8,6 @@
   <ItemGroup>
     <Reference Include="Microsoft.AspNetCore.Server.HttpSys" />
     <Reference Include="System.Net.Http.WinHttpHandler" />
-    <Compile Include="$(SharedSourceRoot)test\SkipOnHelixAttribute.cs" />
   </ItemGroup>
 
   <PropertyGroup>
@@ -16,4 +15,10 @@
     <PackageTags>214124cd-d05b-4309-9af9-9caa44b2b74a</PackageTags>
   </PropertyGroup>
 
+  <ItemGroup>
+    <HelixContent Include="$(RepositoryRoot)src\Servers\IIS\tools\TestCert.pfx" />
+    <HelixContent Include="$(RepositoryRoot)src\Servers\IIS\tools\UpdateIISExpressCertificate.ps1" />
+    <HelixPreCommand Include="call RunPowershell.cmd UpdateIISExpressCertificate.ps1 || exit /b 1" />
+  </ItemGroup>
+
 </Project>

src/Servers/IIS/IIS/test/Common.FunctionalTests/ClientCertificateTests.cs

@@ -35,6 +35,7 @@ namespace Microsoft.AspNetCore.Server.IIS.FunctionalTests
 
         [ConditionalTheory]
         [MemberData(nameof(TestVariants))]
+        [OSSkipCondition(OperatingSystems.Windows, WindowsVersions.Win7, WindowsVersions.Win2008R2)]
         public Task HttpsNoClientCert_NoClientCert(TestVariant variant)
         {
             return ClientCertTest(variant, sendClientCert: false);
@@ -42,6 +43,7 @@ namespace Microsoft.AspNetCore.Server.IIS.FunctionalTests
 
         [ConditionalTheory]
         [MemberData(nameof(TestVariants))]
+        [OSSkipCondition(OperatingSystems.Windows, WindowsVersions.Win7, WindowsVersions.Win2008R2)]
         public Task HttpsClientCert_GetCertInformation(TestVariant variant)
         {
             return ClientCertTest(variant, sendClientCert: true);

src/Servers/IIS/IIS/test/Common.FunctionalTests/HttpsTests.cs

@@ -31,6 +31,7 @@ namespace Microsoft.AspNetCore.Server.IISIntegration.FunctionalTests
 
         [ConditionalTheory]
         [MemberData(nameof(TestVariants))]
+        [OSSkipCondition(OperatingSystems.Windows, WindowsVersions.Win7, WindowsVersions.Win2008R2)]
         public async Task HttpsHelloWorld(TestVariant variant)
         {
             var port = TestPortHelper.GetNextSSLPort();

src/Servers/IIS/IIS/test/Common.FunctionalTests/Inprocess/StartupTests.cs

@@ -113,6 +113,7 @@ namespace Microsoft.AspNetCore.Server.IISIntegration.FunctionalTests
         [SkipIfNotAdmin]
         [RequiresNewShim]
         [RequiresIIS(IISCapability.PoolEnvironmentVariables)]
+        [SkipOnHelix]
         [Flaky("https://github.com/aspnet/AspNetCore-Internal/issues/2221", FlakyOn.Helix.All)]
         public async Task StartsWithDotnetInstallLocation(RuntimeArchitecture runtimeArchitecture)
         {

src/Servers/IIS/IIS/test/FunctionalTest.props

@@ -16,15 +16,17 @@
     <HelixProjectPlatform Condition="'$(IsIISTest)' == 'true'" Remove="@(HelixProjectPlatform)" />
 
     <HelixProjectTargetQueue Condition="'$(IsIISTest)' == 'true'" Include="Windows.10.Amd64.EnterpriseRS3.ASPNET.Open" />
+    <HelixProjectTargetQueue Condition="'$(IsIISTest)' == 'true'" Include="Windows.81.Amd64.Open" />
+    <!-- Disabled until https://github.com/dotnet/core-eng/issues/5506 is resolved -->
+    <!-- <HelixProjectTargetQueue Condition="'$(IsIISTest)' == 'true'" Include="Windows.7.Amd64.Open" /> -->
 
     <HelixContent Include="..\..\..\tools\update_schema.ps1" />
-    <HelixContent Include="..\..\..\tools\InstallIISFeatures.ps1" />
     <HelixContent Include="..\..\..\tools\UpdateIISExpressCertificate.ps1" />
+    <HelixContent Include="..\..\..\tools\TestCert.pfx" />
     <HelixContent Include="..\..\..\AspNetCoreModuleV2\AspNetCore\aspnetcore_schema_v2.xml" />
 
-    <HelixPreCommand Condition="'$(IsIISTest)' == 'true'" Include="call RunPowershell.cmd InstallIISFeatures.ps1" />
-    <HelixPreCommand Include="call RunPowershell.cmd update_schema.ps1" />
-    <HelixPreCommand Include="call RunPowershell.cmd UpdateIISExpressCertificate.ps1" />
+    <HelixPreCommand Include="call RunPowershell.cmd update_schema.ps1 || exit /b 1" />
+    <HelixPreCommand Include="call RunPowershell.cmd UpdateIISExpressCertificate.ps1 || exit /b 1" />
   </ItemGroup>
 
   <Target Name="BuildAssets" AfterTargets="Build">

src/Servers/IIS/IIS/test/IIS.Shared.FunctionalTests/MofFileTests.cs

@@ -16,6 +16,7 @@ namespace IIS.FunctionalTests
         [OSSkipCondition(OperatingSystems.Linux | OperatingSystems.MacOSX)]
         [RequiresIIS(IISCapability.TracingModule)]
         [Flaky("https://github.com/aspnet/AspNetCore-Internal/issues/2222", FlakyOn.Helix.All)]
+        [SkipOnHelix]
         public void CheckMofFile()
         {
             var path = Path.Combine(TestPathUtilities.GetSolutionRootDirectory("IISIntegration"), "aspnetcoremodulev2", "aspnetcore", "ancm.mof");

src/Servers/IIS/tools/UpdateIISExpressCertificate.ps1

@@ -1,20 +1,24 @@
-$cert = New-SelfSignedCertificate -DnsName "localhost", "localhost" -CertStoreLocation "cert:\LocalMachine\My" -NotAfter (Get-Date).AddYears(5)
-$thumb = $cert.GetCertHashString()
+$ErrorActionPreference = 'Stop'
 
-$Store = New-Object  -TypeName System.Security.Cryptography.X509Certificates.X509Store -ArgumentList 'root', 'LocalMachine'
-$Store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite)
-$Store.Add($cert)
-$Store.Close()
+if (-not $PSScriptRoot) {
+    # Older versions of Powershell do not define this variable
+    $PSScriptRoot = Split-Path $MyInvocation.MyCommand.Path -Parent
+}
 
-$tempFile = [System.IO.Path]::GetTempFileName();
-$content = "";
+# The certificate thumbprint of TestCert.pfx
+$thumb = 'CBA8EF428446072286B8201C2877F5EF0EA3B804'
 
-for ($i=44300; $i -le 44399; $i++) {
-    $content += "http delete sslcert ipport=0.0.0.0:$i`n";
-    $content += "http add sslcert ipport=0.0.0.0:$i certhash=$thumb appid=`{214124cd-d05b-4309-9af9-9caa44b2b74a`}`n";
+& certutil -f -v -p testpassword -importpfx "$PSScriptRoot\TestCert.pfx"
+if ($lastexitcode -ne 0) {
+    throw 'Failed to import test certificate into machine root store. This is required for IIS Express tests.'
 }
 
-[IO.File]::WriteAllLines($tempFile, $content)
+$tempFile = [System.IO.Path]::GetTempFileName()
+
+for ($i=44300; $i -le 44399; $i++) {
+    Add-Content -Path $tempFile "http delete sslcert ipport=0.0.0.0:$i"
+    Add-Content -Path $tempFile "http add sslcert ipport=0.0.0.0:$i certhash=$thumb appid=`{214124cd-d05b-4309-9af9-9caa44b2b74a`}"
+}
 
-netsh -f $tempFile
-Remove-Item $tempFile;
+& netsh -f $tempFile
+Remove-Item $tempFile -ErrorAction Ignore