nov. 23

If you, like me, are used to run your tests outside of Visual Studio directly in command line via MsTest, you may encounter some stange and unexpected behavior while migrating to Visual Studio 2008.
Note that I'm mainly using the command line for customizing my continuous integration build and not for the day to day life.

These last months, I have been working a lot using Visual Studio 2008 and .NET framework 3.5.
But in these last weeks, to prepare the migration of a big project to Visual Studio 2008, I have decided to work with Visual Studio 2008 but targeting .NET framework 2.0 (which will be the default migration behavior when migrating a project from VS 2005 to VS 2008).
(Note I have done this test using Visual Studio 2008 Beta 2 and not with the RTM version yet)

What is the result ? We will simply compile with VS 2008 and it will generate a .NET 2.0 DLL.
So what was my thought ? That the VS 2005 tools will work on it. To check this assumption, I have launched MsTest (2005) on my DLL. (Remember it's a .NET 2.0 DLL generated by VS 2008)
And the result ? MsTest give me a simple message "No test to run".

This is very surprising for me as I was sure MsTest was loading and executing the tests via reflexion.
So why on hell it doesn't find any tests ?
Note that "of course", if you launch your test using VS 2008's MsTest, all the tests will be correctly launched.

What is the reason?

  • Reflexion limitation?
  • MsTest limitation?
  • VS 2008 beta 2 compilation problem?

So far, I do not have any explanation to this phenomenon, but I'm currently searching in the MsTest DLL itself and in the MSDN forum to find an answer. I hope I will be able to come back soon to you !
If you have any clue or idea, leave a comment !

Edit: 2007-11-26 : Correction of typo

nov. 14

As you will be writing code, you will probably be writing tests. And you will probably come to writing internal methods to limit their visibility. And it will be necessary in lots of cases to test also these internal methods. How to do that ? You won't be able to access them from your test project.

It's not a big problem as Visual Studio 2005 (and 2008) will generate accessors for you using reflection. VS 2005 will generate a code file (that will have the big disadvantage that when you will change the name of a method, the file won't gets generated and your project will continue to compile. VS 2008 will generate an accessor DLL that will gets compiled (and so re-generated) at the same time you build your project.

So do we really need something else ? Definitely yes.

  1. If you work in TDD, you will write your test when your method do not exist yet (and so neither your accessor) and writing without any mistake the name of the future accessor may be a bit tricky. Ok, it's not the main problem
  2. You may - and you will - write code that cannot be tested via the generated accessors
    • VS 2005 will, in some cases, not generate a method accessor
    • VS 2008 will generate accessors for all the methods, but some won't work (at least in the Beta 2 version, and I have read that the accessor support will be increased in the final version, but I haven't been able to test it yet)

What happens so ? Let's imagine you write code like :

   internal static IEnumerable<T> Distinct<T>(IEnumerable<T> list)
   {
      List<T> result = new List<T>();
 
      foreach ( T item in list )
         if ( !result.Contains(item) )
            result.Add(item);
 
      return result;
   }

Which solution so we have so ? Either write your own accessor (we can call such a method using reflection) or you use another solution called "InternalsVisibleToAttribute". This attribute will make all your internal types visible to one or more specific assemblies and you will be able to call them as if they were public.

Note that when you ask Visual Studio to generate unit tests, it will propose to "honor the InternalsVisibleToAttribute", meaning that by default, it won't generate an accessor, but will add the tag to the assembly file.

To have more information about this attribute, refer to the online MSDN : http://msdn2.microsoft.com/en-us/library/system.runtime.compilerservices.internalsvisibletoattribute.aspx

Note also that if you are testing a strong named assembly you will need to

  1. Add a strong name to your test assembly
  2. Update the InternalsVisibleToAttribute (from your code project) to add the public key of the test project

How can we extract a public key from a DLL ?

Simply by using the "SN" tool. It is available in the .NET framework SDK, ie by default in the folder "%ProgramFiles%\Microsoft Visual Studio 8\SDK\v2.0\Bin".

You can use it as follow :

C:\>sn -Tp c:\WINDOWS\assembly\GAC_MSIL\Accessibility\2.0.0.0__b03f5f7f11d50a3a\Accessibility.dll

Microsoft (R) .NET Framework Strong Name Utility  Version 2.0.50727.42
Copyright (c) Microsoft Corporation.  All rights reserved.

Public key is
002400000480000094000000060200000024000052534131000400000100010007d1fa57c4aed9
f0a32e84aa0faefd0de9e8fd6aec8f87fb03766c834c99921eb23be79ad9d5dcc1dd9ad2361321
02900b723cf980957fc4e177108fc607774f29e8320e92ea05ece4e821c0a5efe8f1645c4c0c93
c1ab99285d622caa652c1dfad63d745d6f2de5f17e5eaf0fc4963d261c8a12436518206dc09334
4d5ad293

Public key token is b03f5f7f11d50a3a

Note that you will need to provide the full public key (and not the public key token) in the attribute to make it work correctly. So for example:

[assembly: InternalsVisibleTo("Jarod.Sdc.Extensions.Task.Test", "0024000004...518206dc093344d5ad293")]

if I was using the public key of the .NET framework.

Tags: | |
nov. 07

Tonight I was working to see how it was possible to wrap the MsTest execution to avoid returning an error code when a test is inconclusive.

In fact the "problem" comes from the DLL "Microsoft.VisualStudio.QualityTools.Common" from the path "c:\program files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies".

This DLL defines an enum TestOutcome and a method TestOutcomeHelper.IsFail

How to overcome this? The simplest way is probably to detail more your MsTest command line to see this outcome result (which is stored in the .TRX file) to be able to analyse it.

To do that, here is typically what you could use :

  mstest /testcontainer:myTestLibrary.dll /detail:outcometext

What is this detail switch used for ? Simply to extract automatically some values from the TRX file. And I have found this link tonight on the MSDN that give a first list of the possible properties : http://msdn2.microsoft.com/en-GB/library/ms182489(VS.80).aspx#detail