Drop Builds’ primary output in a “Latest” folder

As briefly mentioned in another post, we have a specific “Reference Assemblies” folder on the Build Machines containing the latest version of each assembly issued from a successful Build. Assemblies in that folder, know as the “Latest” folder, can be used for “Continuous Integration” Builds. This post is about the pragmatic solution implemented to drop only the primary output of the Builds in that folder.

Click to Read MoreClick to Close

The output of a Build contains not only the assemblies and satellite assemblies issued from the compilation of the Visual Studio Projects (I.e.: assemblies part of the “primary output” as named in the Microsoft Setup Projects). It contains also a copy of all the referenced assemblies (file references) with the property “CopyLocal”=”True”.

It’s important for our purpose to only drop the primary output into the “Latest” folder, otherwise we could override the latest version of some referenced assemblies with a copy of the specific version found for the Build (e.g.: when targeting the Integration Environment, we use the promoted version on, the assemblies which are possibly not the latest).

We may not set “CopyLocal”=”False” on all the “file references” because MSTest needs a copy of those in the bin folder to be able to run the unit Tests (That would not be the case if we could find for MSTest an equivalent of the “ReferencePath” parameter of MSBuild).

We don’t have access to methods (or well described “algorithms”) to retrieve the exact list of assemblies part of the “primary output”. Such methods are only implemented in the Microsoft Setup Projects (Projects not supported, by the way, by MSBuild).

We don’t want all our developers to add MSBuild Scripts in their Visual Studio Projects to drop the “Targets” in the “Latest” folder.

Ex. copy “$(TargetDir)$(TargetName).???” “C:\RefAssemblies\Latest”

This is not only too error prone (like everything you request to a human developer), but it’s also not robust enough. It’s in our opinion impossible to maintain this with enough guarantee taking into account that developers can for example add support for new languages at any time. For each Project, in addition to the assembly $(TargetDir)$(TargetName).dll, we also have to drop the satellite assemblies (e.g.: <culture>\<assemblyname>.resources.dll”), etc…

The most generic solution we found to identify the assemblies part of the primary output, although quick and dirty, consists in parsing the FileListAbsolute.txt file generated by MSBuild itself and available in the “obj” folder. We “rely” so on direct output of Microsoft (the content of that file) and avoid to implement (and maintain) our own generic (and most probably complex) algorithm (“complex” because I have no idea how to detect satellite assemblies in a Visual Studio Project).

Notice: We may only drop the primary output of Builds in the “Latest” folder for “Continuous Integration” Builds or Builds targeting the Integration Environment (both using the latest version of the sources). Builds targeting Qualification, Pre-Production or Production use always a specific version of the sources instead of the latest version. Their purpose is indeed to fix a bug in the assemblies deployed in the targeted environment and build with older sources.

Click to Read MoreClick to Close

[csharp] using System;
using System.IO;
using System.Linq;
using System.Activities;
using System.Reflection;
using System.Diagnostics;
using System.ComponentModel;
using System.Collections.Generic;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.Build.Evaluation;

namespace AG.SCRM.TeamBuild.Activity
{
/// <summary>
/// Collect Build’s Primary Output Items
/// </summary>
[BuildActivity(HostEnvironmentOption.All)] public sealed class GetPrimayOutput : CodeActivity
{
/// <summary>
/// List of Build’s Primary Output Items
/// </summary>
[RequiredArgument] public InOutArgument<List<FileInfo>> PrimaryOutputItems { get; set; }

/// <summary>
/// Build’s Configuration Parameter
/// </summary>
[RequiredArgument] public InArgument<string> Configuration { get; set; }

/// <summary>
/// Build’s Platform Parameter
/// </summary>
[RequiredArgument] public InArgument<string> Platform { get; set; }

/// <summary>
/// Local path of Build’s project file
/// </summary>
[RequiredArgument] public InArgument<string> LocalProject { get; set; }

[RequiredArgument] public InArgument<string> OutDir { get; set; }

/// <summary>
/// Collect Build’s Primary Output Items
/// </summary>
/// <param name="context"></param>
protected override void Execute(CodeActivityContext context)
{
var fileListAbsolute = PrimaryOutputItems.Get(context);
var localProject = LocalProject.Get(context);
var configuration = Configuration.Get(context);
var platform = Platform.Get(context);

// Default Configuration is "Debug"
if (string.IsNullOrEmpty(configuration)) configuration = "Debug";

// Initialize the list of Build’s Primary Output Items if required.
// Otherwise, add Build’s Primary Output Items to the provided list
// to possibly support a loop on all Visual Studio Solution files in
// Build’s
if (fileListAbsolute == null) fileListAbsolute = new List<FileInfo>();

if (Path.GetExtension(localProject).Equals(".sln", StringComparison.InvariantCultureIgnoreCase))
{
// Parse the Visual Studio Solution file
SolutionParser sln = new SolutionParser(localProject);
string root = Path.GetDirectoryName(localProject);
foreach (SolutionProject project in sln.MSBuildProjects)
{
localProject = Path.Combine(root, project.RelativePath);
CollectOutputItems(context, fileListAbsolute, configuration, platform, localProject);
}
}
else
{
// Validate that the file is a Visual Studio Project
var projectCollection = new ProjectCollection();
try
{
Project project = projectCollection.LoadProject(localProject);
string projectDirectoryPath = Path.GetDirectoryName(localProject);
CollectOutputItems(context, fileListAbsolute, configuration, platform, localProject);
}
catch (Exception ex)
{
throw new Exception(string.Format("Project file ‘{0}’ is not a valid Visual Studio project.", localProject), ex);
}
projectCollection.UnloadAllProjects();

}

context.SetValue(PrimaryOutputItems, fileListAbsolute);
}

private static void CollectOutputItems(CodeActivityContext context, List<FileInfo> items, string configuration, string platform, string project)
{
string projectFileName = Path.GetFileName(project);
string projectName = Path.GetFileNameWithoutExtension(project);
string projectPath = Path.GetDirectoryName(project);
string fileListAbsolute = GetFileListAbsolute(configuration, platform, projectFileName, projectPath);

if (!string.IsNullOrEmpty(fileListAbsolute))
{
System.IO.StreamReader file = null;
string line;

string parent = Path.GetDirectoryName(fileListAbsolute);

// Read the file and parse it line by line.
using (file = new System.IO.StreamReader(fileListAbsolute))
{
while ((line = file.ReadLine()) != null)
{
// Ignore obj folder’s local items.
if (!line.StartsWith(parent, StringComparison.OrdinalIgnoreCase))
{
FileInfo item = new FileInfo(line);
if (item.Exists)
{
// We are actually only interested in .dll and .exe + their .pdb, .xml and .config
if (item.Extension.Equals(".dll", StringComparison.OrdinalIgnoreCase) || item.Extension.Equals(".exe", StringComparison.OrdinalIgnoreCase))
{
//MessageHelper.DisplayInformation(context, string.Format("Build output of {0} contains: {1}.", projectName, item.Name));
items.Add(item);

var pdb = new FileInfo(Path.Combine(item.DirectoryName, Path.GetFileNameWithoutExtension(item.FullName) + ".pdb"));
if (pdb.Exists)
{
items.Add(item);
}

/// <summary>
///
/// </summary>
/// <param name="configuration">Build’s Configuration: Debug or Release</param>
/// <param name="platform">Build’s Target Platform: Any CPU or x86</param>
/// <param name="projectFileName">Visual Studio Project filename</param>
/// <param name="projectPath">Visual StudioProject path</param>
/// <returns>The path of the FileListAbsolute.txt file.</returns>
/// <remarks>This file should be located under /obj/{platform}/{configuration/}.
/// But we also check the parent folders if the file is not found where expected.
/// We did indeed experienced problem to locate this file for Builds with "Mixed Plateform" Target.</remarks>
private static string GetFileListAbsolute(string configuration, string platform, string projectFileName, string projectPath)
{
var fileListAbsoluteName = projectFileName + ".FileListAbsolute.txt";
var fileListAbsolutePath = Path.Combine(projectPath, "obj", platform, configuration, fileListAbsoluteName);
if (!File.Exists(fileListAbsolutePath))
{
fileListAbsolutePath = Path.Combine(projectPath, "obj", configuration, fileListAbsoluteName);
if (!File.Exists(fileListAbsolutePath))
{
fileListAbsolutePath = Path.Combine(projectPath, "obj", fileListAbsoluteName);
if (!File.Exists(fileListAbsolutePath))
{
fileListAbsolutePath = null;
}
}
}

return fileListAbsolutePath;
}
}

/// <summary>
/// This Visual Studio Solution file Parser can be used to retrieve lists of projects in a Solution.
/// The first list contains all projects that can be built with MSBuild.
/// The second list contains all the other projects.
/// </summary>
/// <remarks>
/// Based on http://stackoverflow.com/questions/707107/library-for-parsing-visual-studio-solution-files.
/// It’s a wrapper on Microsoft Build’s internal class "SolutionParser"
/// </remarks>
public class SolutionParser
{
static readonly Type s_SolutionParser;
static readonly PropertyInfo s_SolutionParser_solutionReader;
static readonly MethodInfo s_SolutionParser_parseSolution;
static readonly PropertyInfo s_SolutionParser_projects;

public List<SolutionProject> MSBuildProjects { get; private set; }
public List<SolutionProject> OtherProjects { get; private set; }

static SolutionParser()
{
s_SolutionParser = Type.GetType("Microsoft.Build.Construction.SolutionParser, Microsoft.Build, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a", false, false);
if (s_SolutionParser != null)
{
s_SolutionParser_solutionReader = s_SolutionParser.GetProperty("SolutionReader", BindingFlags.NonPublic | BindingFlags.Instance);
s_SolutionParser_projects = s_SolutionParser.GetProperty("Projects", BindingFlags.NonPublic | BindingFlags.Instance);
s_SolutionParser_parseSolution = s_SolutionParser.GetMethod("ParseSolution", BindingFlags.NonPublic | BindingFlags.Instance);
}
}

public SolutionParser(string solutionFileName)
{
if (s_SolutionParser == null)
{
throw new InvalidOperationException("Cannot find type ‘Microsoft.Build.Construction.SolutionParser’ are you missing a assembly reference to ‘Microsoft.Build.dll’?");
}
var solutionParser = s_SolutionParser.GetConstructors(BindingFlags.Instance | BindingFlags.NonPublic).First().Invoke(null);
using (var streamReader = new StreamReader(solutionFileName))
{
s_SolutionParser_solutionReader.SetValue(solutionParser, streamReader, null);
s_SolutionParser_parseSolution.Invoke(solutionParser, null);
}
MSBuildProjects = new List<SolutionProject>();
OtherProjects = new List<SolutionProject>();
var array = (Array)s_SolutionParser_projects.GetValue(solutionParser, null);
for (int i = 0; i < array.Length; i++)
{
SolutionProject project = new SolutionProject(array.GetValue(i));

if (project.ProjectType == "KnownToBeMSBuildFormat")
MSBuildProjects.Add(project);
else
OtherProjects.Add(project);
}
}
}

/// <summary>
/// This class represent a Visual Studio Project member of a Solution.
/// </summary>
/// <remarks>
/// It’s a wrapper on Microsoft Build’s internal class "ProjectInSolution"
/// </remarks>
[DebuggerDisplay("{ProjectName}, {RelativePath}, {ProjectGuid}, {ProjectType}")] public class SolutionProject
{
static readonly Type s_ProjectInSolution;
static readonly PropertyInfo s_ProjectInSolution_ProjectName;
static readonly PropertyInfo s_ProjectInSolution_RelativePath;
static readonly PropertyInfo s_ProjectInSolution_ProjectGuid;
static readonly PropertyInfo s_ProjectInSolution_ProjectType;

static SolutionProject()
{
s_ProjectInSolution = Type.GetType("Microsoft.Build.Construction.ProjectInSolution, Microsoft.Build, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a", false, false);
if (s_ProjectInSolution != null)
{
s_ProjectInSolution_ProjectName = s_ProjectInSolution.GetProperty("ProjectName", BindingFlags.NonPublic | BindingFlags.Instance);
s_ProjectInSolution_RelativePath = s_ProjectInSolution.GetProperty("RelativePath", BindingFlags.NonPublic | BindingFlags.Instance);
s_ProjectInSolution_ProjectGuid = s_ProjectInSolution.GetProperty("ProjectGuid", BindingFlags.NonPublic | BindingFlags.Instance);
s_ProjectInSolution_ProjectType = s_ProjectInSolution.GetProperty("ProjectType", BindingFlags.NonPublic | BindingFlags.Instance);
}
}

public string ProjectName { get; private set; }
public string RelativePath { get; private set; }
public string ProjectGuid { get; private set; }
public string ProjectType { get; private set; }

public SolutionProject(object solutionProject)
{
this.ProjectName = s_ProjectInSolution_ProjectName.GetValue(solutionProject, null) as string;
this.RelativePath = s_ProjectInSolution_RelativePath.GetValue(solutionProject, null) as string;
this.ProjectGuid = s_ProjectInSolution_ProjectGuid.GetValue(solutionProject, null) as string;
this.ProjectType = s_ProjectInSolution_ProjectType.GetValue(solutionProject, null).ToString();
}
}

/// <summary>
/// List of known project type Guids from http://www.mztools.com/articles/2008/mz2008017.aspx
/// + BizTalk: http://winterdom.com/2008/12/biztalkserver2009msbuildtasks
/// + Workflow 4.0
/// </summary>
public enum ProjectType
{
[Description("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}")] Windows_CSharp,
[Description("{F184B08F-C81C-45F6-A57F-5ABD9991F28F}")] Windows_VBNET,
[Description("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}")] Windows_VisualCpp,
[Description("{349C5851-65DF-11DA-9384-00065B846F21}")] Web_Application,
[Description("{E24C65DC-7377-472B-9ABA-BC803B73C61A}")] Web_Site,
[Description("{F135691A-BF7E-435D-8960-F99683D2D49C}")] Distributed_System,
[Description("{3D9AD99F-2412-4246-B90B-4EAA41C64699}")] Windows_Communication_Foundation_WCF,
[Description("{60DC8134-EBA5-43B8-BCC9-BB4BC16C2548}")] Windows_Presentation_Foundation_WPF,
[Description("{C252FEB5-A946-4202-B1D4-9916A0590387}")] Visual_Database_Tools,
[Description("{A9ACE9BB-CECE-4E62-9AA4-C7E7C5BD2124}")] Database,
[Description("{4F174C21-8C12-11D0-8340-0000F80270F8}")] Database_other_project_types,
[Description("{3AC096D0-A1C2-E12C-1390-A8335801FDAB}")] Test,
[Description("{20D4826A-C6FA-45DB-90F4-C717570B9F32}")] Legacy_2003_Smart_Device_CSharp,
[Description("{CB4CE8C6-1BDB-4DC7-A4D3-65A1999772F8}")] Legacy_2003_Smart_Device_VBNET,
[Description("{4D628B5B-2FBC-4AA6-8C16-197242AEB884}")] Smart_Device_CSharp,
[Description("{68B1623D-7FB9-47D8-8664-7ECEA3297D4F}")] Smart_Device_VBNET,
[Description("{14822709-B5A1-4724-98CA-57A101D1B079}")] Workflow_30_CSharp,
[Description("{D59BE175-2ED0-4C54-BE3D-CDAA9F3214C8}")] Workflow_30_VBNET,
[Description("{06A35CCD-C46D-44D5-987B-CF40FF872267}")] Deployment_Merge_Module,
[Description("{3EA9E505-35AC-4774-B492-AD1749C4943A}")] Deployment_Cab,
[Description("{978C614F-708E-4E1A-B201-565925725DBA}")] Deployment_Setup,
[Description("{AB322303-2255-48EF-A496-5904EB18DA55}")] Deployment_Smart_Device_Cab,
[Description("{A860303F-1F3F-4691-B57E-529FC101A107}")] Visual_Studio_Tools_for_Applications_VSTA,
[Description("{BAA0C2D2-18E2-41B9-852F-F413020CAA33}")] Visual_Studio_Tools_for_Office_VSTO,
[Description("{F8810EC1-6754-47FC-A15F-DFABD2E3FA90}")] SharePoint_Workflow,
[Description("{6D335F3A-9D43-41b4-9D22-F6F17C4BE596}")] XNA_Windows,
[Description("{2DF5C3F4-5A5F-47a9-8E94-23B4456F55E2}")] XNA_XBox,
[Description("{D399B71A-8929-442a-A9AC-8BEC78BB2433}")] XNA_Zune,
[Description("{EC05E597-79D4-47f3-ADA0-324C4F7C7484}")] SharePoint_VBNET,
[Description("{593B0543-81F6-4436-BA1E-4747859CAAE2}")] SharePoint_CSharp,
[Description("{A1591282-1198-4647-A2B1-27E5FF5F6F3B}")] Silverlight,
[Description("EF7E3281-CD33-11D4-8326-00C04FA0CE8D")] BizTalk,
[Description("32f31d43-81cc-4c15-9de6-3fc5453562b6")] Workflow_40
};

/// <summary>
/// Helper Class to manage Visual Studio Project’s types
/// </summary>
public static class ProjectTypeExtensions
{
public static Guid ToGuid(this ProjectType val)
{
DescriptionAttribute[] attributes = (DescriptionAttribute[])val.GetType().GetField(val.ToString()).GetCustomAttributes(typeof(DescriptionAttribute), false);
return attributes.Length > 0 ? Guid.Parse(attributes[0].Description) : Guid.Empty;
}

public static ProjectType Parse(string val)
{
return Parse(Guid.Parse(val));
}

public static ProjectType Parse(Guid val)
{
ProjectType? type = null;
FieldInfo[] fis = typeof(ProjectType).GetFields();
foreach (FieldInfo fi in fis)
{
DescriptionAttribute[] attributes = (DescriptionAttribute[])fi.GetCustomAttributes(typeof(DescriptionAttribute), false);
if (attributes.Length > 0)
{
if (Guid.Parse(attributes[0].Description) == val)
{
if (Enum.IsDefined(typeof(ProjectType), fi.Name))
type = (ProjectType)Enum.Parse(typeof(ProjectType), fi.Name);
break;
}
}
}
if (type.HasValue)
return type.Value;
else
throw new FormatException(string.Format("'{0}’ is not a valid Project Type’s Guid", val.ToString()));
}

public static List<ProjectType> GetMSBuildProjectTypes(string localProject)
{
var projectCollection = new ProjectCollection();
Project project;
try
{
project = projectCollection.LoadProject(localProject);
}
catch (Exception ex)
{
throw new Exception(string.Format("Project Type cannot be determined as ‘{0}’ is not a valid VS project.", localProject), ex);
}

var projectTypes = GetMSBuildProjectTypes(project);

projectCollection.UnloadAllProjects();

return projectTypes;
}

public static List<ProjectType> GetMSBuildProjectTypes(Project project)
{
try
{
var projectTypeGuids = (from property in project.Properties
where property.Name == "ProjectTypeGuids"
select property.EvaluatedValue).FirstOrDefault();

List<ProjectType> projectTypes;
if (string.IsNullOrEmpty(projectTypeGuids))
projectTypes = new List<ProjectType>();
else
projectTypes = (from guid in projectTypeGuids.Split(‘;’) select Parse(guid)).ToList();

return projectTypes;
}
catch (Exception ex)
{
throw new Exception(string.Format("Unable to determine the project type of ‘{0}’ due to: {1}", Path.GetFileNameWithoutExtension(project.FullPath), ex.Message));
}
}
}
}
[/csharp]

BeatificaBytes

Drop Builds’ primary output in a “Latest” folder

Comments

Leave a Reply Cancel reply