The Friendly Coder

On software development and technology

MSBuild Gotchas: Traversed Static Item Evaluation

If you know anything about MSBuild, you know that knowing the order of evaluation for properties, items and such is critical. But how does the order of evaluation work when chaining control to other scripts using the task (otherwise known as script traversal)? The answer may surprise you.

Silly Assumptions

Before going too deep lets lay some groundwork. I assume you are familiar with the basic order or evaluation of MSBuild elements including properties and items ( particularly dynamic and static item evaluation ), and are familiar with the task predefined by the standard build tools. If not, then you may want to come back after you nail down the basics.

Simple Traversal

Now, lets start with a simple example. Lets suppose you have a primary build script that looks something like this:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <Target Name="DoWork">
        <MSBuild Projects="helper.proj" Targets="Clean" />
        <MSBuild Projects="helper.proj" Targets="DoMoreWork" />
    </Target>
</Project>

Looks harmless enough, right? Before getting ahead of ourselves, lets take a look at our sample helper.proj script.

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <ItemGroup>
        <StaticFiles Include="code\*.*" />
    </ItemGroup>

    <Target Name="Clean">
        <Message Text="Cleaning files '@(StaticFiles)'" />
    </Target>

    <Target Name="DoMoreWork">
        <Message Text="Working on '@(StaticFiles)'" />
    </Target>
</Project>

So, now for the trick question: what will be the output of these two helper targets when called from the main project? You may intuitively say that the output from both targets will be the list of files in the code sub-folder, which is typically correct. So now you may ask, why is this a trick question? Well, what happens if the content of the code folder changes between the call to “Clean” and the call to “DoMoreWork” as shown below?

<Target Name="DoWork">
    <MSBuild Projects="helper.proj" Targets="Clean" />

    <WriteLinesToFile File="code\new.txt" Lines="Hello World" />

    <MSBuild Projects="helper.proj" Targets="DoMoreWork" />
</Target>

Now, I bet you would intuitively say that DoMoreWork would output the same thing as the Clean target but with the newly generated file added to the output, and therein lies the catch!

The Catch

If you try running these sample projects you will see that the output shown from both helper targets is the same! For some reason the second MSBuild task doesn’t seem to detect the new file created just before. What is going on here?

So lets think this through. We know that static items (those declared in the global namespace) are evaluated as the MSBuild engine loads and parses the file (MSBuild 101). So it seems that MSBuild only loads and parses the helper.proj script one time during the execution of this script, likely when executing the first task. Luckily, we can test this easily simply by removing the call to the “Clean” target, as in:

<Target Name="DoWork">
    <!--MSBuild Projects="helper.proj" Targets="Clean" /-->

    <WriteLinesToFile File="code\new.txt" Lines="Hello World" />

    <MSBuild Projects="helper.proj" Targets="DoMoreWork" />
</Target>

Now the call to “DoMoreWork” detects the newly created file!

The Explanation

So why does MSBuild behave like this? I believe the answer can be deduced from the detailed description of the task on MSDN, which states:

Unlike using the Exec Task to start MSBuild.exe, this task uses the same MSBuild process to build the child projects. The list of already-built targets that can be skipped is shared between the parent and child builds.

I believe that the behavior we are experiencing in this example is a side effect of the optimization logic in the MSBuild engine that caches the results of each executed target to prevent redundant executions of the same logic. Although it doesn’t say it explicitly, I suspect that the first time each script file is loaded and parsed by the current instance of MSBuild, all elements and conditions are evaluated and their states cached for future reference.

The real hideousness with all of this is that in a complex production MSBuild script, traversals to different targets in the same file may be spread across multiple build targets in multiple files, making it difficult to isolate which one executed first thus making bugs related to it difficult to isolate.

On a final, more positive note I want to mention that dynamic items (those defined inside targets) do not suffer from this evaluation problem because they are evaluated at run-time when their containing target gets executed. Just keep in mind that MSBuild prevents the same target from executing more than once per instance of the engine and you should be fine.

Summary

Unfortunately I have yet to find an easy or obvious way to avoid this behavior or problems that may arise from it. You must simply be diligent in keeping the design and implementation of your scripts as simple and clean as possible so as to avoid having to make multiple attempts to traverse between scripts.

On the other hand, perhaps there is a solution out there for this problem that I have not considered. If anyone out there has found some way to disable this caching (e.g.: via some command line parameter or MSBuild setting or something) please post a comment below. The time you save may be your own!

Leave a Reply