This is a continuation of the sample .NET MAUI – UI testing with Appium and NUnit created as part of Gerald’s blog Getting started with UI testing .NET MAUI apps using Appium. In this blog, we will see how to use BrowserStack App Automate to run the previously written tests in the cloud on real devices! This blog provides a guide on setting up BrowserStack with your existing Appium user-interface (UI) tests of your .NET MAUI apps, and we will also see how to setup a continuous integration (CI)/continuous delivery (CD) pipeline to run these tests in an automated fashion. What is BrowserStack App Automate? BrowserStack’s App Automate lets you test your native and hybrid apps on a variety of mobile and tablet devices. The devices you access are all real, physical devices housed in their data centers. Run tests on 2000+ real iOS and Android devicesApp Automate gives you instant access to 2000+ real iOS and Android devices, with varying OS versions and form factors in the cloud, thereby reducing the cost of building and managing an in-house lab. You can test your app under real-world conditions and identify issues that might not appear in emulators or simulators. Run test suites on BrowserStack in minutesUse the BrowserStack SDK that allows you to integrate your test suites with App Automate in minutes. Simply install the SDK, set up a YAML file, and trigger your tests to get started. You can also leverage features like parallel orchestration and local testing with ease using the SDK. All this, with no code changes! Test apps in real-world user conditionsBrowserStack lets you test the behavior of your app on different devices under different network conditions. App Automate offers several presets also lets you define your own custom network conditions to check your app’s behavior. You can also change the network conditions mid-way during the test run, just like in real world where the end-user’s network varies. Steps to Add BrowserStack to Existing Appium UITests If you already have Appium UI tests written for your .NET MAUI App, the following steps show you how to run these tests on the devices in the cloud provided by BrowserStack App Automate. To follow along with this blog, you can refer to this repository with sample code. In order to be able to run your .NET MAUI iOS and Android UI tests with BrowserStack, follow these steps: Sign Up for BrowserStack: Create an account on BrowserStack, a free trial to get started is available. Find pricing details for BrowserStack Subscriptions here. Create BrowserStack Credentials: Obtain your BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY from the BrowserStack account settings. Android and iOS BrowserStack Configuration Files: The repository includes BrowserStack configuration files for both Android and iOS projects. These files define the specific settings and capabilities required to run the tests on BrowserStack. The browserstack.yml file for the Android project can be found at BasicAppiumNunitSample/UITests.Android/browserstack.yml and the browserstack.yml file for the iOS project can be found at BasicAppiumNunitSample/UITests.iOS/browserstack.yml. The configuration in these files should mostly be self-explanatory, but here are a few of the key parts in this configuration file: userName & accessKey: These are your BrowserStack credentials. They can be hardcoded or set as environment variables/GitHub Action secrets. automationName: Specifies the automation engine to be used. For iOS, use XCUITest and for Android use UIAutomator2. appiumVersion: Specifies the version of Appium to use, this should match the version of Appium used in the UITests. app: Path to the iOS app (IPA) or Android app (APK) to be tested. For .NET MAUI Apps, if the app is built part of an automated pipeline, you can add the path to the publish folder here. For example: ./MauiApp/bin/Release/net8.0-android/publish/com.companyname.basicappiumsample-Signed.apk browserstackLocal: Set to false to run on BrowserStack App Automate, which runs the tests on the selected devices on the Device Cloud. This setting is mostly for web-based solutions and therefore should always be false for .NET MAUI apps. The configuration of the test devices to be used is determined by so-called capabilities. This ranges from the operating system version to be used, to what logs should be captured, and many more device specific features like enabling Apple Pay, simulating a geolocation coordinate, etc. To help you configure these capabilities, BrowserStack also has a really great Capability Generator Tool that walks you through generating the browserstack.yml file. After that, you can simply copy the generated file from the tool into your repo. Update UITest Projects to Use BrowserStack: Add the browserstack.yml files that you have just created/generated for each platform to the respective UITests.Android and UITests.iOS folders. You will need to use separate browserstack.yml files for each platform. Just place the file in the root folder for each test project. To the respective UITests.Android and UITests.iOS projects, add the BrowserStack.TestAdapter nuget package. <PackageReference Include="BrowserStack.TestAdapter" Version="0.13.3" /> Run BrowserStack App Automate Tests from Local Machine: Follow the informative Documentation provided by BrowserStack to run the tests from your local machine. The tests can be run from Visual Studio on Windows or by BrowserStack CLI for Mac. Run BrowserStack App Automate Tests in CI/CD Workflow: For Azure DevOps Pipelines, the steps can be found in the BrowserStack Documentation. To setup the GitHub Actions Workflow to run the BrowserStack App Automate Test as part of your CI/CD Automation, follow the next section of this blog. GitHub Actions Workflow The GitHub Actions workflow file .github/workflows/browserStackTests.yml is set up to run the UI Tests on BrowserStack for both iOS and Android platforms. For details on the steps that build the .NET MAUI App, you can read this blog Getting Started with DevOps and .NET MAUI, this section will focus specifically on the steps to run the tests on BrowserStack App Automate. Prerequisites for GitHub Actions You should set your BrowserStack Username and Access Key as GitHub Action Secrets, i.e. BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY respectively. Store Credentials as GitHub Secrets: Go to your GitHub repository. Navigate to Settings > Secrets and variables > Actions. Add two new secrets: BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY. Let’s look at some of the important steps in the GitHub Actions pipeline, which sets up the workflow to run the tests in BrowserStack App Automate. Install Appium: Installs Appium and the XCUITest or UIAutomator2 driver respectively per platform. These are needed to perform the interactions with the running application. The driver knows how to perform a click, or scroll or switch the device to dark theme, for instance. npm install -g appium # For Android appium driver install uiautomator2 # For iOS appium driver install xcuitest For Mac Runners with Apple Silicon Chips Only: When using a Mac Runner with Apple Silicon processor, we need the following extra step that installs the BrowserStack SDK .NET tool and sets it up. dotnet tool install browserstack-sdk --version 1.16.3 --create-manifest-if-needed dotnet browserstack-sdk setup-dotnet --dotnet-path "." --dotnet-version "8.0.403" --yes BrowserStack CLI Docs Step 3: [Only for Macs with Apple Silicon] Install dotnet x64 on MacOS Build Appium BrowserStack Tests: Builds the Appium tests for Android or iOS project. # For Android dotnet build BasicAppiumNunitSample/UITests.Android/UITests.Android.csproj # For iOS dotnet build BasicAppiumNunitSample/UITests.iOS/UITests.iOS.csproj Run Appium BrowserStack Tests: Runs the Appium tests on BrowserStack. # For Android dotnet test BasicAppiumNunitSample/UITests.Android/UITests.Android.csproj # For iOS ./dotnet test BasicAppiumNunitSample/UITests.iOS/UITests.iOS.csproj BrowserStack Test Reports and Dashboard When you run your tests on BrowserStack, detailed test reports are generated. These reports include information such as test execution logs, screenshots, and even videos of the test runs and more. You can access these reports through the BrowserStack Dashboard. BrowserStack App Automate Dashboard The BrowserStack App Automate Dashboard provides detailed and comprehensive overview of the test execution. Alternatively, if you need to integrate with your own custom dashboard, you can use the REST API provided by BrowserStack. Some highlights of the App Automate test report includes: Session Video: Captures the recording of the test as it happens in the session. Use this recording to go at a precise point in time when an error occurred and debug. https://devblogs.microsoft.com/dotnet/wp-content/uploads/sites/10/2025/03/TestRunVideo.mp4 Logs tab: Select Text Logs, Console Logs, or Screenshots tab to view detailed logs. The Logs also include Appium Logs and Network Logs! Summary This blog demonstrates how to integrate BrowserStack App Automate with Appium NUnit tests for .NET MAUI applications. It provides this sample code to show how to run BrowserStack App Automate with your existing Appium UI Tests and explains the GitHub Actions workflow used in this repository. UITesting is crucial for ensuring that your application behaves as expected from the user’s perspective. Running tests on real devices, as opposed to emulators or simulators, helps identify issues that might only appear under real-world conditions, such as different network environments, device-specific quirks, and actual user interactions. BrowserStack App Automate is a service that makes it easy to run your existing Appium NUnit tests on their App Automate Device Cloud. Check out the more samples at dotnet/maui-samples. Please let us know if anything is unclear or what you would like to see in follow up posts. The post Use BrowserStack App Automate with Appium UI Tests for .NET MAUI Apps appeared first on .NET Blog. View the full article

Announcing the general availability of custom instructions for VS Code. Read the full article View the full article

The .NET team has just released preview2 of .NET 10, and it’s got a bundle of new features and improvements. You might want to give these a try but don’t want to mess with your local development environment. A great way to try out a .NET preview is by using dev containers. In this blog post, we’ll walk you through the steps to set up and use dev containers for experimenting with a new .NET release. What are Dev Containers? Dev containers are pre-configured, isolated environments that allow developers to work on projects without worrying about dependencies and configurations. They are particularly useful for trying out new technologies, as they provide a consistent and reproducible setup. Many development environments, including Visual Studio Code, support dev containers. This allows you to easily create and manage these environments. You can also use dev containers in GitHub Codespaces, which provide a cloud-based development environment. Types of .NET Container Images There are many types of .NET container images available, each designed for different scenarios. .NET container images are published to the Microsoft Artifact Registry. These images are regularly updated to include the latest patches and features, ensuring that you have access to the most secure and up-to-date versions. If you want to get more information about a specific .NET container image, such as mcr.microsoft.com/dotnet/nightly/sdk:9.0, you can use one of the following methods: From the Microsoft Artifact Registry: The Microsoft Artifact Registry documentation provides comprehensive information about the .NET container images, including how they are tagged and updated. Use the docker inspect Command: If you have already pulled the image, you can use the docker inspect command to get detailed information about the image. For example: docker inspect mcr.microsoft.com/dotnet/nightly/sdk:9.0 Some of these images are designed specifically for running .NET applications in production. For exploring a new .NET release, you’ll want to use a dev container image that includes the .NET SDK and runtime. For exploring a preview release, you’ll probably want a dev container for a current GA release augmented with the preview version of .NET 10 you want to try out. Container Type Best For Example Tag Notes SDK Development mcr.microsoft.com/dotnet/sdk:9.0 Includes full SDK, runtime, and development tools Runtime Production mcr.microsoft.com/dotnet/runtime:9.0 Smaller image with just the runtime ASP.NET Web apps mcr.microsoft.com/dotnet/aspnet:9.0 Includes ASP.NET Core runtime Nightly Testing previews mcr.microsoft.com/dotnet/nightly/sdk:10.0 Latest preview builds Dev Container Local development mcr.microsoft.com/devcontainers/dotnet:1-8.0 Pre-configured development environment with additional tools Setting Up Your Dev Container To get started with dev containers, you’ll need Docker and Visual Studio Code with the Dev Containers extension installed. Follow these steps to set up your dev container for exploring a new .NET release. Create a Dev Container Configuration In your project directory, create a .devcontainer folder and add a devcontainer.json file. The easiest way to do this is with the Dev Container extension in Visual Studio Code. Open the Command Palette (Ctrl+Shift+P) and select “Dev Containers: Add Development Container Configuration Files…”. You can store the configuration files in the workspace or in the user data folder, outside the workspace — I generally choose the workspace option. Choose the “C#(.NET)” template, and it will create a .devcontainer folder with a devcontainer.json file that you can then customize as needed. For GA versions of .NET, there are prebuilt dev containers available. For preview versions, you can create a custom dev container configuration. Add a Dockerfile In my devcontainer configuration, I use a Dockerfile to pull in all the versions of .NET I need. I typically want the most recent LTS version and the most recent STS version as a base image — currently that means .NET 8 and .NET 9. These are useful for running dotnet based tools that depend on one of these GA versions. On top of this I install the preview version of .NET 10 I want. The Dockerfile lives in the same directory as the devcontainer.json file. You point to from the devcontainer.json file using the “dockerFile” property of the “build” property, as follows: "build": { "dockerfile": "./Dockerfile", "context": "." }, This replaces the “image” property in the stock devcontainer.json file. I use the devcontainer of the most recent .NET LTS as the base image for my dev container. FROM mcr.microsoft.com/devcontainers/dotnet:1-8.0 Then I “install” the most recent STS version of .NET, which is currently .NET 9, and the preview version of .NET 10 I want to try out. To install these SDK versions, I copy the SDK from the corresponding SDK image, using the Docker COPY command, as follows: # Install the current .NET STS release on top of that COPY --from=mcr.microsoft.com/dotnet/sdk:9.0 /usr/share/dotnet /usr/share/dotnet # Finally install the most recent .NET 10.0 preview using the dotnet-install script COPY --from=mcr.microsoft.com/dotnet/nightly/sdk:10.0.100-preview.2 /usr/share/dotnet /usr/share/dotnet You can see my complete devcontainer.json and Dockerfile for .NET 10 preview2 in my aspnet-whats-new repo. Other dev container configuration options You can customize your dev container further by adding additional configuration options in the devcontainer.json file. What’s really great about dev containers is that you can tailor them to your specific needs, meaning that you only install the specific tools and dependencies you need for your project. Here are a few common options you might want to consider: Extensions: You can specify any Visual Studio Code extensions you want to install in your dev container. For example, to install the C# DevKit extension, add the following line to your devcontainer.json file: "extensions": [ "ms-dotnettools.csdevkit", ] Features: You can specify additional features to include in your dev container. For example, to include the Azure CLI, add the following line to your devcontainer.json file: "features": { "azure-cli": "latest" } You can see the full list of available features in the dev container features documentation. Post-Create Command: You can specify a command to run after the dev container is created. This is useful for installing additional dependencies or running setup scripts. For example, to install the latest version of the .NET CLI, add the following line to your devcontainer.json file: "postCreateCommand": "dotnet tool install -g dotnet-ef" Be aware that the postCreateCommand runs every time you start the dev container, in contrast to commands in the Dockerfile, which are only run when the dev container is built. Build and Start Your Dev Container Once you have your dev container configuration set up, you can build and start your dev container. The easiest way to do this is with the “Dev Containers: Open Folder in Container” command. This will build and start your dev container based on the configuration files you created. You can check that the .NET versions are installed correctly with the dotnet –list-sdks command: Note that once the container image is built, it is cached locally, so that you don’t have to rebuild it every time you start the dev container. However, this means that if there are new service releases of .NET 8 or .NET 9, you will need to rebuild the dev container to pick up those changes. You can do this with the “Dev Containers: Rebuild Container Without Cache” command. This will force a rebuild of the dev container image, and it will pull the latest version of the base images. Conclusion Using dev containers is a fantastic way to try out new .NET releases without affecting your local development environment. With a consistent and isolated setup, you can explore new features and enhancements with ease. Give it a try and let us know what you think! The post Exploring new .NET releases with Dev Containers appeared first on .NET Blog. View the full article

Xbox services powers many of the core experiences in Xbox Gaming. Our services are primarily HTTP based microservices and enable experiences that range from telling our users who are online and what games are being played, to the ability to log in, to chat services. These services run on Azure compute and are primarily .NET based. Given that our service history spans all the way from Xbox 360 to the current generation of consoles, and that we must maintain compatibility across the multiple device types as well as individual games we support, it is key that any migrations or updates are performed carefully. Streamlining Innovation with .NET Aspire For the past couple of years, we have been modernizing our codebase to adopt the latest patterns and versions of .NET as well as a focus on the latest security best practices. This includes upgrading from .NET framework to the latest versions of .NET, and moving to modern orchestration platforms like Azure Kubernetes Service (AKS). As we pushed more on our move to AKS and started to iterate, we realized that doing a full validation in a ‘true’ environment is quite slow! After making our changes, and deploying the code, we’d often find out we missed something subtle in our telemetry with how we send them or a naming issue that required yet another iteration. As we heard more about .NET Aspire, we investigated and found that .NET Aspire was a perfect fit! It lets us find all of those minor issues locally, and removes much of the need for full deployment to do our basic hookup validation. With the ability to do this development locally, hit breakpoints, and make quick changes, .NET Aspire quickly became a key tool for Xbox service. Now, the team leverages .NET Aspire’s capabilities to streamline the transition. .NET Aspire lets us flight locally the newly transitioned service, including seeing the critical metrics and logs so that we may debug locally prior to deployment. This reduces the need of deploying to a ‘real’ environment to detect many issues. .NET Aspire also automates emulator usage for Azure dependencies out of the box, saving developers time to focus on writing their own code. No more keeping emulators up to date and writing scripts to wire it all together. .NET Aspire enables our goal of ‘clone + F5’, which will ultimately ease developer onboarding and debugging. Setting up .NET Aspire .NET Aspire app host is a new type of project in Visual Studio — I think of the .NET Aspire app host project as a superpowered startup script, and it comes with deep integration with Azure and .NET out of the box. And you get to use C# to wire things up! Using .NET Aspire’s app host, you can fire up Azure emulators, connect them to your service, providing local environmental overrides as well as generate test data to inject at startup directly into the emulated resource. It’s a familiar programming model and formalizes the fact that we are all cloud devs first, and that it’s a fact of life that we would need to integrate many processes + Azure on our own anyway. Having any reliance on real Azure resources during local development comes with a ton of risk, not much repeatability, and a challenge for developers to share those types of resources — so having an easily hooked up local Azure emulator is great. If you are on the latest .NET, using ASP.NET and integrating with Azure, it’s a no-brainer to kick off your emulators. For our example, we have a worker role consume from an Azure Event Hub and publish to a Cosmos DB, while a microservice frontdoor reads the processed event from the same Cosmos DB. So, we call AddAzureCosmosDB and AddAzureEventHubs and run them as emulators. IResourceBuilder<AzureCosmosDBResource> cosmosdb = builder.AddAzureCosmosDB(cosmosDbResourceName) .RunAsEmulator(c => c.WithLifetime(containerLifetime)); var eventHub = builder.AddAzureEventHubs(eventHubsConnectionName) .RunAsEmulator() .AddEventHub(eventHubResourceName) Next, we can set up our front door microservice, making sure our frontdoor will wait for the emulators to finish initializing and then go about injecting test data and off we go. Be sure to also set up your OTEL exporters to get deep telemetry. var fd = builder.AddProject<FrontDoor>("frontdoor") .WithEnvironment("ASPNETCORE_ENVIRONMENT", "Development") .WithReference(cosmosdb) .WithOtlpExporter() .WaitFor(cosmosdb) Below, we are outputting the ‘browser Cosmos DB’ explorer endpoint so we can manually browse the emulator, and then we create relevant databases and inject our test data. private static async Task<CosmosClient> CreateCosmosCollections(IResourceBuilder<AzureCosmosDBResource> cosmosDbResource) { CosmosClient client = await TestDataGenerator.GetCosmosClient(cosmosDbResource.Resource); Console.WriteLine($"https://localhost:{client.Endpoint.Port}/_explorer/index.html"); using CancellationTokenSource cts = new CancellationTokenSource(CosmosDbInitLoopTimeout); // Set up the database/containers DatabaseResponse dbRef = await client.CreateDatabaseIfNotExistsAsync(DatabaseId, cancellationToken: cts.Token); // Set up container for documents. _ = await dbRef.Database.CreateContainerIfNotExistsAsync( new ContainerProperties { // Exercise for the reader }, ThroughputProperties.CreateAutoscaleThroughput(10000), cancellationToken: cts.Token); return client; } private static async Task LoadCosmosCollections(CosmosClient client) { var documents = TestDataGenerator.GenerateDocs([ ]); foreach (var d in documents) { await client.GetContainer(DatabaseId, ContainerLocator.Get<Document>()) .CreateItemAsync<Document>(d, new PartitionKey(d.PartitionKey)); } } From there, we have a full end-to-end for anyone who clones and runs the repo! Once you kick off your solution, you get a nice view of all the emulators and services running. Below, shows the Cosmos DB emulator, Event Hub emulator, and then our processor/frontdoor roles. This drastically speeds up the onboarding process for developers and eases our local debugging experience. Want to test a specific type of test data? You can manually explore to CosmosDB to inject or change the raw data, then perform your experiment. Later, you could formalize your data and turn that into a test. Aspire Dashboard and Observability Key to Xbox service’s high availability is our logs, metrics, and traces. Since we are aligning on OTEL, which .NET Aspire supports out of the box, we can see all our data right away, and in real-time. While we are migrating our services to adopt new frameworks, we use .NET Aspire to ensure that we are emitting the right metrics locally (And named properly! This is key!!!) and fully exercise all our call paths to see the output data. From the initial .NET Aspire dashboard, we can then view all the relevant console output, structured logs, tracing, and metrics that our app produces. We get to see our custom XAC (Xbox Availability Counter) metrics as well! By providing a unified view of all telemetry data, .NET Aspire reduces the need for custom local monitoring tools and makes for a more efficient local development environment all up. Ultimately, this means we can fine-tune our service and telemetry locally! Streamlining Xbox Services: The Role of .NET Aspire in Development and Testing Once setup, .NET Aspire has helped us with multiple development scenarios — Seemingly simple things like just making sure the service starts properly, not even taking traffic yet. For example, we are moving our legacy services to a Dependency Injection (DI) pattern, and there are a lot of subtle issues that can crop up, especially for devs new to the pattern. Just having .NET Aspire get our project up and running, and connecting to emulators, and having the service respond to a simple health check is enough to force a lot of code to initialize and startup. Thanks to .NET Aspire, we can find and iterate on a lot of issues with this simple type of test! Metrics, tracing, and logs are so important for running our high scale services. We also have downstream consumers we need to ensure that the names and formats align, so having small errors can trigger alerts to not fire and dashboards to not work. Using .NET Aspire, we get a chance to double check all of this locally, before any costly (in time) deployments to a ‘real’ environment The ability to spin up our own .NET Aspire hosting integrations has been valuable in sharing some of our core services. This capability allows other services to depend on integrations we are creating for our common services for their local development, providing early integrations and testing. Since the .NET Aspire integrations also output their own telemetry, we can start to see how the services interact and both output metrics/trace/logs. This also gives us a fantastic baseline prior to migration to more modern platforms! The third-party integration capabilities of .NET Aspire are also impressive, particularly with tools like Wiremock to simulate REST services that we don’t otherwise own. This allows us to test our services in a controlled environment, ensuring that they can handle various scenarios and integrations seamlessly. It has made our testing process more comprehensive and reliable! Conclusion By providing a comprehensive and unified view of all telemetry data, and allowing us to easily hook our services all up together, .NET Aspire allows for the seamless integration and observation of logs, metrics, and traces. This has streamlined our local development environment — Clone the repo, hit F5 and call your service! Another major advantage is the elimination of disparate tools. With .NET Aspire, we can programmatically wire up our services/emulators, and put all relevant data in one place, reducing complexity and saving time. This has significantly improved our local dev loop debugging experience! The post Xbox + .NET Aspire: Transforming Local Development Practices appeared first on .NET Blog. View the full article

MSTest, also known as Microsoft Testing Framework, simplifies the testing experience for .NET applications. This test framework allows users to write and execute tests while providing test suites integrated into the Test Explorer’s of Visual Studio and Visual Studio Code, as well as many CI pipelines. Fully supported, open-source, and cross-platform, MSTest works with all supported .NET targets while maintaining support for VSTest as well as adding support for the improved experience with Microsoft.Testing.Platform (MTP). MSTest 3.8 highlights We continually work to improve MSTest, address your feedback and offer smoother testing experience. MSTest 3.8 is a big release, with lots of great new features: Filtering: The VSTestBridge extension of Microsoft.Testing.Platform (MTP) 1.6 and later now supports filtering while discovering tests (–list-tests). This feature has been highly requested by users to help verify the tests applicable to a given filter. Running tests with MSBuild: Extended support for the -t:Test or /t:Test target to support MSBuild.exe. Improved iterating experience: Updated the Microsoft.Testing.Extensions.TrxReport extension behavior to overwrite the content instead of failing when the TRX file already exists. Enriched metapackage: Simplified using TRX report and Code Coverage by removing the requirement to manually declare these MTP extensions. Support of modern UWP: MSTest now supports .NET 9 in UWP. Improved assertions: Introduced new assertion APIs like Assert.Throws, Assert.ThrowsAsync, Assert.ThrowsExactly, and Assert.ThrowsExactlyAsync. Better data-driven tests: Parameterized generic test methods are now supported. Retrying flaky tests: Introduced RetryAttribute to automatically retry failing test methods. Introducing conditional tests: Added OSConditionAttribute for fine-grained control over where your tests run. Analyzers: The MSTestAnalysisMode feature allows you to control the severity levels of code analyzers within your test projects. Read on for more, and check out the full changelog on GitHub for details. Filtering The VSTestBridge extension of Microsoft.Testing.Platform (MTP) 1.6 and later now supports filtering while discovering tests (--list-tests). This feature has been highly requested by users to help verify the tests applicable to a given filter. This gives more confidence that you have the correct filter when you run tests. Because MSTest with MTP uses the VSTestBridge, that feature is available in MSTest 3.8 and later. Based on your requests, we also added support for the runsettings TestCaseFilter element in MTP. Note that this is also available for any test framework relying on Microsoft.Testing.Extensions.VSTestBridge extension. Running tests with MSBuild Another of your requests was to extend the support of the -t:Test or /t:Test target that was a dotnet build only feature to support MSBuild.exe. This is now available for MSTest 3.8, and any test framework based on MTP 1.6+. Improved iterating experience We updated the Microsoft.Testing.Extensions.TrxReport extension behavior when the TRX file already exists to overwrite the content instead of failing. The extension will display a warning message to notify that the file already exists and will be overwritten. For example, when running the following command twice: dotnet run -- --report-trx --report-trx-filename test.trx The second run will not fail, but instead show the following warning: Warning: Trx file 'path/to/TestResults/test.trx' already exists and will be overwritten. Enriched metapackage Since most MSTest projects use both TRX report and Code Coverage, we’ve simplified using them by removing the requirement to manually declare these MTP extensions. With MSTest 3.8 and MTP, you can simplify your projects from: <ItemGroup> <PackageReference Include="MSTest" /> <PackageReference Include="Microsoft.Testing.Extensions.CodeCoverage" /> <PackageReference Include="Microsoft.Testing.Extensions.TrxReport" /> </ItemGroup> to: <ItemGroup> <PackageReference Include="MSTest" /> </ItemGroup> Support of modern UWP UWP recently announced support for .NET 9 in Modernize your UWP app with preview UWP support for .NET 9 and Native AOT blog post and through close collaboration, we are happy to announce that MSTest supports this new mode. For that to work, you need to be using Microsoft.NET.Test.Sdk version 17.14.0-preview-25107-01 or above. Stay tuned for a dedicated blog post with all the information and examples coming soon. Improved assertions Exception asserts Historically, only Assert.ThrowsException and Assert.ThrowsExceptionAsync were available, requiring an exact match between the specified exception type and the one thrown. Many users requested a simpler naming convention and support for derived exceptions. In response, this release introduces: Assert.Throws Assert.ThrowsAsync Assert.ThrowsExactly Assert.ThrowsExactlyAsync To facilitate a smooth upgrade to MSTest 3.8 without causing disruptions to your projects, the old APIs remain available. However, when creating new tests, they will not appear in IntelliSense, encouraging the use of simpler names. The following is an example of usage of the new APIs: [TestClass] public sealed class Test1 { [TestMethod] public void TestMethod1() { // Will pass because the exceptions have the exact same type Assert.Throws<OperationCanceledException>(() => throw new OperationCanceledException()); // Will pass because TaskCanceledException derives from OperationCanceledException Assert.Throws<OperationCanceledException>(() => throw new TaskCanceledException()); // Will fail because the exceptions do not have exactly the same type Assert.ThrowsExactly<OperationCanceledException>(() => throw new TaskCanceledException()); } } We also include MSTEST0039 analyzer along with a codefix to help you with transitioning away from the old APIs to the new ones. In addition, we added new overloads to the new APIs that accepts a messageBuilder parameter, which is a Func<Exception?, string>. This allows you to customize the failure message based on information from the exception thrown (if any). For example: [TestMethod] public void TestMethod1() { Assert.ThrowsExactly<InvalidOperationException>( () => throw new ArgumentException("Message of ArgumentException"), messageBuilder: actualException => { if (actualException is null) { return "Expected InvalidOperationException but no exception was thrown."; } return $"Expected InvalidOperationException, but found {actualException.GetType().Name} with message {actualException.Message}"; }); } The above test will fail as follows: Assert.ThrowsExactly failed. Expected exception type:<System.InvalidOperationException>. Actual exception type:<System.ArgumentException>. Expected InvalidOperationException, but found ArgumentException with message Message of ArgumentException Collection asserts We introduced new collection-related assertion APIs in the Assert class, designed to work with IEnumerable<T>, expanding the assertion capabilities for collections: Assert.HasCount: Asserts that an IEnumerable<T> has a given number of elements. Assert.IsEmpty: Asserts that an IEnumerable<T> doesn’t have any elements. Assert.IsNotEmpty: Asserts that an IEnumerable<T> has at least one element. Assert.ContainsSingle: Asserts that an IEnumerable<T> has exactly a single element and returns that element. These new APIs have been added to the Assert class instead of CollectionAssert to improve discoverability. We also plan to make all CollectionAssert APIs available on Assert. Interpolated string handler overloads for message parameter Finally, we added new interpolated string handler overloads for many existing assertion APIs. If you heavily use the Assert methods overload with message parameter and use an interpolated string as argument to the message parameter, this will unnecessarily allocate a string that will end up being unused in most cases, as this message is only used in case the assertion failed. Starting with 3.8, new overloads that use interpolated string handler will ensure that the string isn’t allocated unless the assertion fails. Better data-driven tests Parameterized generic test methods are now supported, allowing you to write tests like: [TestClass] public sealed class TestClass1 { [TestMethod] [DataRow(2, 5, 10)] [DataRow(2.0d, 5.0d, 10.0d)] [DataRow(5, 5, 25)] [DataRow(5.0d, 5.0d, 25.0d)] public void TestMethod1<T>(T number1, T number2, T expectedResult) where T : IMultiplyOperators<T, T, T> { Assert.AreEqual(expectedResult, number1 * number2); } } Additionally, we’ve enhanced the behavior of DynamicDataAttribute to automatically detect the type of the referenced member used as the data source as the default. With this improvement, you can simplify your tests from: [DynamicData(nameof(MyProperty), DynamicDataSourceType.Property)] [DynamicData(nameof(MyMethod), DynamicDataSourceType.Method)] to: [DynamicData(nameof(MyProperty))] [DynamicData(nameof(MyMethod))] Another highly requested feature was the ability to ignore specific data sources from parameterized tests. Starting in 3.8, DataRowAttribute and DynamicDataAttribute have an IgnoreMessage property. When it’s set to non-null string, the source will be ignored. We are also allowing any custom ITestDataSource to have a similar behavior by simply implementing the ITestDataSourceIgnoreCapability. With this change, you can now do: [TestMethod] [DataRow(0)] [DataRow(1, IgnoreMessage = "This row is ignored")] [DataRow(2)] public void TestMethod1(int x) { Assert.AreNotEqual(1, x); } or [TestMethod] [DynamicData(nameof(Data1))] [DynamicData(nameof(Data2), IgnoreMessage = "This dynamic data source is ignored")] [DynamicData(nameof(Data3))] public void TestMethod1(int x) { Assert.IsTrue(x < 10); } // 0 to 3 - passes public static IEnumerable<int> Data1 => Enumerable.Range(0, 4); // 50 to 59 - ignored, and will fail if not ignored public static IEnumerable<int> Data2 => Enumerable.Range(50, 10); // 4 to 9 - passes public static IEnumerable<int> Data3 => Enumerable.Range(4, 6); Finally, we have introduced TestDataRow<T> allowing users to wrap their data in a customizable container where you can set the display name, ignore the entry and much more to come. Any ITestDataSource can now return IEnumerable<TestDataRow<T>>. For example: [TestMethod] [DynamicData(nameof(GetData))] public void TestMethod1(int num1, int num2, int expectedSum) { Assert.AreEqual(expectedSum, num1 + num2); } private static IEnumerable<TestDataRow<(int Num1, int Num2, int ExpectedSum)>> GetData() { yield return new((1, 2, 3)); yield return new((2, 3, 5)); yield return new((3, 4, 7)) { DisplayName = "This test case has a special display name" }; yield return new((3, 3, 7)) { IgnoreMessage = "Ignored because of ..." }; } Retrying flaky tests We’re excited to introduce RetryAttribute, a powerful new feature that lets you automatically retry failing test methods! With just one attribute, you can configure MaxRetryAttempts, MillisecondsDelayBetweenRetries, and BackoffType (choosing between Constant and Exponential strategies for smarter retries). For example: [TestMethod] [Retry(3, MillisecondsDelayBetweenRetries = 100, BackoffType = DelayBackoffType.Constant)] public void TestMethod() { Assert.Fail("Failing test"); } This test method will run four times (the original run and 3 retries), with a time delay of 100 milliseconds between each run. Need more flexibility? You can create your own custom retry logic by inheriting from RetryBaseAttribute, the same foundation as the RetryAttribute. If you are using MTP, enabling retries across your entire test suite is even easier, just leverage the retry extension and let your tests recover automatically! Introducing conditional tests MSTest now includes OSConditionAttribute, giving you fine-grained control over where your tests run. This attribute lets you specify which operating systems a test should or should not execute on, making it easier to handle OS-specific behavior in your test suite. Here are some examples of this attribute: [TestClass] public sealed class MyTestClass { [TestMethod] [OSCondition(OperatingSystems.Windows)] public void TestMethodRunningOnlyOnWindows() { } [TestMethod] [OSCondition(OperatingSystems.Linux | OperatingSystems.OSX)] public void TestMethodRunningOnlyOnLinuxAndMac() { } [OSCondition(ConditionMode.Exclude, OperatingSystems.Windows)] public void TestMethodRunningOnAllOSesButWindows() { } } As part of the development of this feature, we are providing the ConditionBaseAttribute abstract class as parent of OSConditionAttribute and IgnoreAttribute. You can use this new attribute as the root of any conditional feature you would like to implement. We have many ideas for extra conditional features we could provide, and we would really appreciate your input in prioritizing the ones that would be the most useful to you! Analyzers The MSTestAnalysisMode feature allows you to control the severity levels of code analyzers within your test projects. By setting the MSTestAnalysisMode MSBuild property, you can determine which analyzers are enabled and their corresponding severity levels. Available Modes: None: Disables all analyzers. You can enable individual analyzers using .editorconfig or .globalconfig files. Default: Follows the default behavior for each rule. Rules enabled by default will use their default severity, while those disabled by default will have a severity of none. Recommended: Elevates rules enabled by default with an Info (suggestion) severity to warnings. Certain rules may be escalated to errors in both Recommended and All modes.For example, MSTEST0003: Test methods should have valid layout is escalated to an error in these modes. All: More aggressive than Recommended. All rules are enabled as warnings, with certain rules escalating to errors. Usage Example: To set the MSTestAnalysisMode to Recommended, include the following in your project file or in Directory.Build.props: <PropertyGroup> <MSTestAnalysisMode>Recommended</MSTestAnalysisMode> </PropertyGroup> This configuration ensures that your test projects adhere to best practices by appropriately adjusting the severity levels of various analyzers. Note that using Directory.Build.props is more recommended than project file, as it ensures the property applies to all your test projects and you add it in one place instead of multiple project files. It’s also safe if MSTestAnalysisMode is defined for non-test projects via Directory.Build.props. It only affects MSTest projects. For more detailed information, refer to the MSTest code analysis documentation. Let’s now focus on 2 newly added analyzers that have shown promising results during internal dogfood: MSTEST0038: Don’t use ‘Assert.AreSame’ or ‘Assert.AreNotSame’ with value types MSTEST0040: Do not assert inside ‘async void’ contexts MSTEST0038 warns when the methods Assert.AreSame and Assert.AreNotSame are called with a value type argument. These methods work by comparing references, making its usage with value types incorrect as it yields unexpected results. The following use of Assert.AreSame will always fail: Assert.AreSame(0, 0); This is because we pass them to ReferenceEquals, which accepts object. So, each argument will be boxed into a different object, and they will not have equal references. The same applies to Assert.AreNotSame, but is more critical. Assert.AreNotSame(0, 0); By applying the same logic, the above assert will always pass. This can hide real production bugs as you may not be asserting what you want. Note that the above examples are for demonstration purposes only and are not real-world examples. A more realistic example can be: // This will still pass even if the array has a single element. Assert.AreNotSame(1, myArray.Length); For more information, see MSTEST0038: Don’t use ‘Assert.AreSame’ or ‘Assert.AreNotSame’ with value types. MSTEST0040 warns for usage of assertion APIs in async void. Depending on whether you use a custom SynchronizationContext and whether you use VSTest under .NET Framework, an exception thrown in async void context can either be swallowed completely or can crash the test host. For more information, see MSTEST0040: Do not assert inside ‘async void’ contexts. Wrapping up With MSTest 3.8, we’re continuing our mission to make testing in .NET easier, more powerful, and more enjoyable. This release brings highly requested features, reduces tedious tasks, and enhances the overall developer experience — allowing you to focus on writing great tests instead of wrestling with infrastructure. We hope these improvements help streamline your workflow and showcase our commitment to listening to your feedback and evolving MSTest to meet your needs. As always, we’d love to hear your thoughts — try out MSTest 3.8 and let us know what you think by opening an issue or a discussion on microsoft/testfx GitHub repository! Try MSTest 3.8 Today A huge shoutout to @SimonCropp for his incredible contributions to MTP and MSTest! His dedication and expertise continue to make a lasting impact, and this release is no exception. We’re grateful for his ongoing support and invaluable efforts in helping shape a better testing experience for the entire .NET community. Thank you, Simon! The post MSTest 3.8: Top 10 features to supercharge your .NET tests! appeared first on .NET Blog. View the full article