PlayNicely.Executor 1.0.2

Play Nicely Executor

The Play Nicely Executor project depends on the PlayNicely.Projects package, extending it to support execution of programs, against test case projects, within a pre-defined environment. Executor then collects program output so that test assertion can be made.

The Executor package, provides a fluent interface to define an ITestEnvironment, within which, some programs can be defined as unavailable, other programs as required. A test environment can also define what NuGet package sources are available, so that BDD projects can test new functionality. If a test case project is specified, the file system of that project will be 'materialized' and the working directory for the executed process.

Getting Started

The main artefacts are the ITestEnvironment, TestEnvironmentBuilder, and ITestEnvironmentRunner. An ITestEnvironment specifies the target project and ensures a side effect free environment in which any testing can run. You create an ITestEnvironment using the fluent interface of TestEnvironmentBuilder. With this class you define the desired context for your ITestEnvironmentRunner.

Tests run using the ITestEnvironmentRunner, this package defines a basic ProcessRunner which runs an executable with arguments, much like a command line, and returns a basic ExecutionResult, which includes whether the command succeeded and the stdout and stderr streams. The DotNetRunner class, defined in PlayNicely.Executor.DotNet executes a dotnet subcommand, with arguments, and returns a ExecutionResult<DotNetExecutionContext>. This includes detailed information about the dotnet command results, build context, targets ran, projects built, lists of errors, etc.

If you have a specific process you'd like to support, you can define a concrete implementation derived from ITestEnvironmentRunner.

Getting Started

Here we are using the generic ProcessRunner to execute a dotnet build process on a pre-defined test case project, within a side effect free test environment. The purpose of this getting started is to demonstrate how to set up a test environment, execute a runner and assert the result.

ℹ️ I'm using ProcessRunner here with dotnet to illustrate getting started. The PlayNicely.Executor.DotNet package provides a specific implementation for dotnet that includes important context information after execution. If you are running dotnet tests, we recommend using that runner instead.

This getting started is code-first, in a typical configuration you would likely use the IDE to define test case projects and SpecFlow (or some other BDD framework) to define the environment.

Define the test case project

Let's build a scenario to test for a failing build, because the .NET target framework is invalid. First, define the test case project and file system.

var testCaseProject = new TestCaseProject("my-failing-project");
var projectFile = testCaseProject.AddFile("proj.csproj");

testCaseProject.ProjectFile = projectFile;

using(var writer = new StreamWriter(projectFile.OpenWriteStream())
{
    writer.WriteLine("<Project Sdk=\"Microsoft.NET.Sdk\">");
    writer.WriteLine("  <PropertyGroup>");
    // Note invalid target framework...
    writer.WriteLine("    <TargetFramework>my-net9.0</TargetFramework>");
    writer.WriteLine("  </PropertyGroup>");
    writer.WriteLine("</Project>");
}

Define the environment

Use the TestEnvironmentBuilder to specify:

• RequiredCommands - any command that must be available in the test environment. Using this method ensures commands can always be executed, by finding them in the current $PATH, and creating a temporary bin directory with symbolic links to the actual executables. This bin directory is prepended to the $PATH in the subsequent environment. On Windows this is less likely to be required, but on Linux, most commands are in shared directories like /usr/bin or /usr/local/bin, if something is excluded (see next bullet point), by removing one of these paths, it is likely to exclude a lot of other commands, RequiredCommands ensures the essential commands can still be executed from the $PATH.
• ExcludeCommandFromPath - it is often neccesary, in a negative test case, to assume a program is not installed. This method locates a command on the current $PATH and removes any directories where it is found. The result of this, when running in the ITestEnvironment is that any attempt to run an excluded command will fail (because it isn't found on the $PATH).
• AddPackageSource - often, the usage of this package is to test another NuGet package project. So that the test environment can run release tests using the under development version of a package, this method allows overriding NuGet.Config in the test environment. When attempting to locate packages, dotnet will only look in these locations.
• SetProject - the target project for this test environment, the project's file system will be materialized and set as the working directory for the process.

Example for clarity

Continuing context from here. We need to ensure dotnet is always available, and specify the test case target project.

var builder = new TestEnvironmentBuilder();
var testEnv = await builder.RequiredCommands("dotnet")
                           .SetProject(testCaseProject)
                           .BuildAsync();

Run the test

With the environment built, simply construct the runner and assert the result (which we expect to fail).

var runner = new ProcessRunner("dotnet", "build"); // Don't need to specify project path,
                                                   // process working dir is root of project
var testResult = await runner.ExecuteAsync(testEnv);

Assert.That(testResult.Succeeded, Is.False);

Why?

This project came about to support the use of NodeJS packages within .NET projects in a .NET first way. You can achieve the integration of NodeJS tools using plugins or other tooling. The problem with this (using plugins) is, these can often become out of date or stale. Most of the NodeJS packages are developed by an actively community, so accessing the latest npm packages directly makes the most sense (we get latest features and security updates).