ValidationAttribute Test Harness updated on Codeplex

Ahead of tomorrow’s 12:00-1:00 Central Brown Bag / Show and Tell over at C4MVC, I made a few changes for the ValidationAttribute Test Harness based on feedback. The changes focus on two areas:

1. Discoverability: Do we really need to identify all the custom validators on startup? No – MEF handles this very well thank you.
2. Runtime Statistics: Can we get a picture of what was done on a given run?

These changes were added to software, and the mechanism of calling the Analyze method (1) gives the ability to extend other scanning options and (2) provides for a platform where the MEF discovery results can be reused over multiple calls.

   1: var analyzer = new ValidationAnalyzer();

   2: var results = analyzer.AnalyzeAssembly(typeof (ChangePasswordModel).Assembly, true);

   3:  

   4: Console.WriteLine(results.AnalysisSummary());

   5:             

   6: Assert.That(results.Valid, Is.EqualTo(true), results.CompileErrorMessage());

This is going to be a fun call. We’ll focus on the latest tidbits about how ValidationAttributes are integrated with MVC, a set of ‘gotchas’ that the test harness keeps you clear of and more.

Introducing the ValidationAttribute Test Harness on Codeplex

***************************

*** Update: for the latest on how to use the ValidationAttribute Test Harness, check out my latest post in this series...

***************************

This article is an introduction to the ValidationAttribute Test Harness project on Codeplex. This project exists to fill a testing void and maintain quality on ValidationAttributes. Our of the box – the ValidationAttribute Test Harness monitors the declaration of error messages and validates their runtime state. This article briefly discusses the nature of ValidationAttributes, the testing issues around the resource approach and how this new CodePlex project addresses the shortcomings.

The standard thinking around testing ValidationAttribute derived classes is that these are not individually testable – but rather relegated to automated ui testing or exploratory testing. While a certain portion of that is true – we cannot fully recreate the framework that interacts with these properties - we can provide a contextual harness that allows us to test the declarations to ensure that the declarations map to their intended targets (and that intended targets are who they say they are).

Purpose 1: Verifying Properly Localized ValidationAttributes at their Attribution Site

When looking at the ValidationAttribute, you quickly notice three properties that give access to declaring the error message raised when the rule is violated. These three are:

• string ErrorMessage
• string ErrorMessageResourceName
• Type ErrorMessageResourceType

It becomes apparent that there are quality issues, especially when it comes to localization.

1. Setting a string on ErrorMessage means the error message is not localizable.
2. The ErrorMessageResourceName is meant to point to a static public property on a resource class – but this is a magic string.
3. Both the Type and Name must be set for resourced error messages to be identified.
4. There are a number of combination-style errors that can arise.
5. The errors that are thrown are obscure and difficult to locate – with details buried deep in the exceptions.

We need a mechanism to test for these scenarios and provide a list of readable errors that lead us directly to the source and allow us to triage them quickly.

Purpose 2: Verifying Custom Attribute Annotations

Custom-defined Attributes, aside from the standard set, are as wide open as we like them to be. They are especially useful when declaring relationships between data. The ASPNET MVC framework generates one out of the box in it’s application template called PropertiesMustMatchAttribute. The attribute is placed at the class level where it declares two properties (via strings) to declare that the two properties are equal which is useful on the model when resetting a password.

This has all the same problems we see with magic strings -

1. What if the declaration is wrong
2. What if one of the property names change. The stringified name won’t be changed with it automatically.
3. What if only one or none of the property names is declared?

Waiting for runtime to discover these kinds of errors is unacceptable.

Using the ValidationAttribute Test Harness to Raise the Quality Bar

Out of the box, the ValidationAttribute Test Harness will take a look at all the ValidationAttribute attributions in your project and verify the configuration of the Error Message related attributes. It will also pull together error information into one concise list that can be easily triaged.

   1: var analyzer = new ValidationAssemblyAnalyzer(typeof(ChangePasswordModel).Assembly, true);

   2: Assert.That(analyzer.ValidationErrorInfoList.Count, Is.EqualTo(0), analyzer.CompileErrorMessage());

The analyzer can also take lists of custom validators so we can provide customized tests based on the attribute type like so:

   1: var analyzer = new ValidationAssemblyAnalyzer(typeof (ResourcedValidationAssemblyAnalyzerTests).Assembly, true, 

   2:                           null,

   3:                           new[] {new PropertiesMustMatchValidator()});

   4:  

   5: Assert.That(analyzer.ValidationErrorInfoList.Count, Is.EqualTo(0), analyzer.CompileErrorMessage());

… this is a great place for MEF, but I’ve held off to avoid dependencies on this initial release. If people just copy the code into their projects, great. If they tend to grab the assembly as a dependency in their unit test project, then it makes more sense. For net4.0, MEF is a no-brainer but we’ll cross that bridge when we get to it. …

Note that the Analyzer takes two enumerable lists – one for member-specific attribute validators and one for class-specific attribute validators. These are denoted by the interface they are derived from: ICustomValidationByTypeAnalyzer or ICustomValidationByMemberAnalyzer.

Understanding How To Write Custom Validators

Writing the custom validators is straightforward and simple. Just implement the ICustomValidator interface on your validator according to the attribute site target. From there, you declare:

1. The type of the attribute that you are testing.
2. The actual test using the contextual information about the attribution site.

And there you have it. Check out the code below to see the custom validator in detail. It’s not fully fleshed out, but provides a good boilerplate as such. Now each time you run your unit tests, your models will be validated and you won’t have to

   1: // since the matching attribute only applies at the class level, we'll use the ByTypeAnalyzer variant

   2: public class PropertiesMustMatchValidator : ICustomValidationByTypeAnalyzer 

   3: {

   4:     #region ICustomValidationByTypeAnalyzer Members

   5:  

   6:     /// <summary>

   7:     /// Here we'll test that the property exists on the object according to our criteria

   8:     /// </summary>

   9:     /// <param name="candidateType">attributed class</param>

  10:     /// <param name="propertyName">property name to look for</param>

  11:     /// <returns></returns>

  12:     private bool PublicPropertyExists(Type candidateType, string propertyName)

  13:     {

  14:         return candidateType.GetProperty(propertyName, BindingFlags.GetProperty | BindingFlags.Public | BindingFlags.Instance) != null;

  15:     }

  16:  

  17:     /// <summary>

  18:     /// Responsible for raising errors according to the tests being performed on

  19:     /// the PropertiesMustMatch validator

  20:     /// </summary>

  21:     /// <param name="candidateType">candidate class type information</param>

  22:     /// <param name="validationAttribute">the validation attribute instance that is being tested</param>

  23:     /// <param name="enforceResourceMode">whether to enforce valid resourcing</param>

  24:     /// <returns>list of error error information</returns>

  25:     public IEnumerable<CustomValidationInfo> Validate(Type candidateType, ValidationAttribute validationAttribute, bool enforceResourceMode)

  26:     {

  27:         var list = new List<CustomValidationInfo>();

  28:         var matchAttribute = validationAttribute as PropertiesMustMatchAttribute;

  29:  

  30:         if (!String.IsNullOrEmpty(matchAttribute.OriginalProperty) &&

  31:             !String.IsNullOrEmpty(matchAttribute.ConfirmProperty))

  32:         {

  33:             if( ! PublicPropertyExists(candidateType, matchAttribute.OriginalProperty))

  34:                 list.Add(new CustomValidationInfo(ValidationStatus.DeclarationError, string.Format(CultureInfo.InvariantCulture, "OriginalProperty named '{0}' does not exist on the object", matchAttribute.OriginalProperty)));

  35:  

  36:             if( ! PublicPropertyExists(candidateType, matchAttribute.ConfirmProperty))

  37:                 list.Add(new CustomValidationInfo(ValidationStatus.DeclarationError, string.Format(CultureInfo.InvariantCulture, "ConfirmProperty named '{0}' does not exist on the object", matchAttribute.ConfirmProperty)));

  38:  

  39:             // more tests could be added here - validating type, other required attribute declarations, etc

  40:         }

  41:         else

  42:             list.Add(new CustomValidationInfo(ValidationStatus.DeclarationError,

  43:                                               "Original Property and/or ConfirmPropert is not set"));

  44:  

  45:         return list;

  46:  

  47:     }

  48:  

  49:     /// <summary>

  50:     /// the attribute type that this validator is testing

  51:     /// </summary>

  52:     /// <returns>Type info on the targeted attribute</returns>

  53:     public Type TargetType()

  54:     {

  55:         return typeof (PropertiesMustMatchAttribute);

  56:     }

  57:  

  58:     #endregion

  59: }

Summary

And there you have it. Check out the code above to see the custom validator in detail. It’s not fully fleshed out, but provides a good boilerplate as such. Now each time you run your unit tests, your models will be validated and you won’t have to ‘wait until runtime’ to discover the quality of your model attributions.

Localizing LabelFor in ASP.NET MVC 2

I started the weekend spelunking ModelBinder with Reflector and started toying with the front-end html helpers. The ASP.NET MVC 2 builds on the MVCContrib FluentHtml implementation and includes some fantastic features for eliminating magic strings in your html. If you are not familiar with the implementation, this piece of code summarizes the Html.LabelFor implementation. For a broader view of these modifications, check out Scott Gu's blog series on the broader functionality.

   1: <div class="editor-label">

   2:    <%=Html.LabelFor(m => m.LastName)%

   3: </div>

If you refactor your model implementation, the front-end html gets refactored along with it. The default implementation relies on marking up the model using DisplayNameAttribute to resolve a user-readable name.

   1: [Required]

   2: [DisplayName("Last Name")]

   3: [StringLength(64)]

   4: public string LastName { get; set; }

The problem with Attributes is that there is no reasonable way I’ve seen to create a DisplayName attribute style variant that is resource-aware. This is primarily due to constraints on Attributes in the form of generic and runtime type constraints. In short, there is no way to create a flexible approach with (1) no magic strings involved and (2) a format convention that can vary across different MVC Areas.

After looking over the first approach, I looked at creating a new LabelFor extension method. I had two driving criteria in creating the solution - (1) avoid any magic strings in the views and (2) allow for easy and flexible variation of naming rules associated with resource classes.

I like the result as the core workings are very testable and the flexibility is straight-forward.

The first part of the solution includes the LabelFor extension. You’ll notice it mirrors the LabelFor(Expression<…>) implementation by adding the ResourcePropertyResolver<T> implementation. In order to grab the associated metadata, I am relying on the same ModelMetadata calls that the existing LabelFor implementation uses. It then relies on the existing Label(string) implementation to generate the output.

   1: /// <summary>

   2: /// Our own set of custom label extensions

   3: /// </summary>

   4: public static class CustomLabelExtensions

   5: {

   6:     /// <summary>

   7:     /// Returns an HMTL label element with the content resolved for the given propety according to

   8:     /// the format specified in the parameters

   9:     /// </summary>

  10:     /// <typeparam name="TModel">The model typically inferred from the prage</typeparam>

  11:     /// <typeparam name="TValue">Value type inferred from expression</typeparam>

  12:     /// <typeparam name="TResourceType">Resource type used as the string resolution target</typeparam>

  13:     /// <param name="html">extension method parameter</param>

  14:     /// <param name="resourcePropertyResolver">resource resolver that indicates the resolution target 

  15:     /// and format strings</param>

  16:     /// <param name="expression">expression tree intended to point at the property being named</param>

  17:     /// <returns>localized, resolved label for the property</returns>

  18:     public static MvcHtmlString LabelFor<TModel, TValue, TResourceType>(this HtmlHelper<TModel> html, ResourcePropertyResolver<TResourceType> resourcePropertyResolver, Expression<Func<TModel, TValue>> expression)

  19:     {

  20:         var metaData = ModelMetadata.FromLambdaExpression(expression, html.ViewData);

  21:  

  22:         var resourcePropertyName = string.Format(CultureInfo.InvariantCulture, resourcePropertyResolver.ResourcePropertyFormatPolicy,

  23:                                                  metaData.ContainerType.Name, metaData.PropertyName);

  24:  

  25:         return html.Label(resourcePropertyResolver.GetResourceValue(resourcePropertyName));

  26:     }

  27: }

The next part is a default implementation of ResourcePropertyResolver. Starting at the top, this is a generic class requires the type of your resource class. A resource class is effectively any class that has static properties – the code-gen component of Visual Studio takes the resource table and implements it as a plain old class. Remember that the code generator property defaults to internal types – called out on the Custom Tool property. You can make these all public types by changing the Custom Tool property to PublicResXFileCodeGenerator.

First off, a static readonly type is resolved once for the class to avoid multipel typeof(…) resolutions – an efficiency move here. Next, a couple of string formatting properties. The first calls out the naming policy for properties on the resource class. The default value is ‘[class name]_[property name]’. The second format calls out the value that should be returned when a matching value is not found. Note that both of these are overridable which I will demonstrate in a moment.

Finally, the GetResourceValue method does the heavy lifting of locating the property in the table and formatting output accordingly.

   1: /// <summary>

   2: /// Implements access of the actual resource class and the associated

   3: /// formatting rules for property names and unresolved resource values

   4: /// </summary>

   5: /// <typeparam name="TResourceType">resource class</typeparam>

   6: public class ResourcePropertyResolver<TResourceType>

   7: {

   8:     private static readonly Type _resourceType = typeof(TResourceType);

   9:  

  10:     public virtual string ResourcePropertyFormatPolicy

  11:     { get { return "{0}_{1}"; } }

  12:  

  13:     public virtual string UnresolvedValueFormatString

  14:     { get { return "{0}.{1}"; } }

  15:  

  16:     internal string GetResourceValue(string resourcePropertyName)

  17:     {

  18:         // both public and non-public are members are being searched because resource properties

  19:         // will either be generated as internal or public. Optimize this by making it specific to

  20:         // your case.

  21:         var propertyInfo =

  22:             _resourceType.GetProperty(resourcePropertyName,

  23:                                        BindingFlags.Static  BindingFlags.Public  BindingFlags.NonPublic);

  24:  

  25:         if (propertyInfo == null)

  26:             return String.Format(CultureInfo.InvariantCulture, UnresolvedValueFormatString, _resourceType.Name, resourcePropertyName);

  27:  

  28:         return (string)propertyInfo.GetValue(null, null);

  29:     }

  30: }

Next, let’s create a custom rule that outputs a patterned, more identifiable output with a little helper that simplifies the View markup. In this case, we simply override the rules we want to and then create a single live, static instance that is quickly referencable.

   1: public class WebStringResolver : ResourcePropertyResolver<WebStringTable>

   2: {

   3:     public override string UnresolvedValueFormatString

   4:     {

   5:         get

   6:         {

   7:             return "*** {0}.{1} ***";

   8:         }

   9:     }

  10:  

  11:     private static WebStringResolver _instance = new WebStringResolver();

  12:     public static WebStringResolver Instance 

  13:     {

  14:         get

  15:         {

  16:             return _instance;

  17:         }

  18:     }

  19: }

So finally, this leads us to the modified markup that is quite explicit to which resource table it is tied and no magic strings in place.

   1: <div class="editor-label">

   2:     <%=Html.LabelFor(WebStringResolver.Instance, m => m.LastName)%>

   3: </div>

So there you have it- my diversion from reflector-spellunking into the ModelBinder.