*** Update: for the latest on how to use the ValidationAttribute Test Harness, check out my latest post in this series...
This article is an introduction to the ValidationAttribute Test Harness project on Codeplex. This project exists to fill a testing void and maintain quality on ValidationAttributes. Our of the box – the ValidationAttribute Test Harness monitors the declaration of error messages and validates their runtime state. This article briefly discusses the nature of ValidationAttributes, the testing issues around the resource approach and how this new CodePlex project addresses the shortcomings.
The standard thinking around testing ValidationAttribute derived classes is that these are not individually testable – but rather relegated to automated ui testing or exploratory testing. While a certain portion of that is true – we cannot fully recreate the framework that interacts with these properties - we can provide a contextual harness that allows us to test the declarations to ensure that the declarations map to their intended targets (and that intended targets are who they say they are).
Purpose 1: Verifying Properly Localized ValidationAttributes at their Attribution Site
When looking at the ValidationAttribute, you quickly notice three properties that give access to declaring the error message raised when the rule is violated. These three are:
- string ErrorMessage
- string ErrorMessageResourceName
- Type ErrorMessageResourceType
It becomes apparent that there are quality issues, especially when it comes to localization.
- Setting a string on ErrorMessage means the error message is not localizable.
- The ErrorMessageResourceName is meant to point to a static public property on a resource class – but this is a magic string.
- Both the Type and Name must be set for resourced error messages to be identified.
- There are a number of combination-style errors that can arise.
- The errors that are thrown are obscure and difficult to locate – with details buried deep in the exceptions.
We need a mechanism to test for these scenarios and provide a list of readable errors that lead us directly to the source and allow us to triage them quickly.
Purpose 2: Verifying Custom Attribute Annotations
Custom-defined Attributes, aside from the standard set, are as wide open as we like them to be. They are especially useful when declaring relationships between data. The ASPNET MVC framework generates one out of the box in it’s application template called PropertiesMustMatchAttribute. The attribute is placed at the class level where it declares two properties (via strings) to declare that the two properties are equal which is useful on the model when resetting a password.
This has all the same problems we see with magic strings -
- What if the declaration is wrong
- What if one of the property names change. The stringified name won’t be changed with it automatically.
- What if only one or none of the property names is declared?
Waiting for runtime to discover these kinds of errors is unacceptable.
Using the ValidationAttribute Test Harness to Raise the Quality Bar
The analyzer can also take lists of custom validators so we can provide customized tests based on the attribute type like so:
… this is a great place for MEF, but I’ve held off to avoid dependencies on this initial release. If people just copy the code into their projects, great. If they tend to grab the assembly as a dependency in their unit test project, then it makes more sense. For net4.0, MEF is a no-brainer but we’ll cross that bridge when we get to it. …
Note that the Analyzer takes two enumerable lists – one for member-specific attribute validators and one for class-specific attribute validators. These are denoted by the interface they are derived from: ICustomValidationByTypeAnalyzer or ICustomValidationByMemberAnalyzer.
Understanding How To Write Custom Validators
Writing the custom validators is straightforward and simple. Just implement the ICustomValidator interface on your validator according to the attribute site target. From there, you declare:
- The type of the attribute that you are testing.
- The actual test using the contextual information about the attribution site.
And there you have it. Check out the code above to see the custom validator in detail. It’s not fully fleshed out, but provides a good boilerplate as such. Now each time you run your unit tests, your models will be validated and you won’t have to ‘wait until runtime’ to discover the quality of your model attributions.