Sunday, 14 December 2008

Assertive tests

Continuing my review of unit testing frameworks, I have been looking at how you might evaluate the features. Some of the issues that I have looked at include:

  1. the way that tests are identified

  2. the type of standard asserts or comparisons provided

  3. the ease with which the assert framework can be extended

  4. the coding style of the asserts

  5. the ability to run the tests in different environments (i.e. the GUI, Ant tasks, and Maven builds)

I am sure there are other comparisons that could be done but I wanted a quick ability to compare the different frameworks. The focus of this blog is on the asserts supported by each of the frameworks. My assumption when I started working with these frameworks is that they would all support the common set since most are built on the JUnit framework. It seemed logical that all would provide the same asserts unless the language made one meaningless or there were new conditions that needed special handling.

So what assert conditions are supported by each suite?

Condition

FlexUnit

Fluint

FUnit

FluxUnit

Equals

4

4

4

4

Not equal

4

4

Object equals

4

4

Object not equal

Same object

4

Not same object

4

String match (1)

4

4

4

String no match

4

4

String contained

4

4

Not contained

4

String empty / not empty

4

4

Starts with

4

Ends with

4

Equal ignoring case

4

True

4

4

4

4

False

4

4

4

4

Null

4

4

4

4

Not null

4

4

4

4

Undefined

4

4

4

Not undefined

4

4

4

Fail

4

4

Instance of

4

Not instance of

4

Is finite

4

Is NaN

4

Is >

4

Is >=

4

Is <

4

Is <=

4

Collections

4

Asynchronous tests

4

(1) Regular Expression string match

It is possible to extend the asserts in all of the frameworks.

FlexUnit defines its asserts in a static class, Assert, that is extended by the TestCase class. This is simply so that the user doesn't have to type Assert.assertEquals().

Flunit defines its asserts as protected methods within the TestCase class. This has a similar effect to FlexUnit with respect to how the asserts are referenced.

FUnit uses static methods in an Assert class that is used by typing Assert.areEqual().

FluxUnit provides a mechanism for defining your own matchers. This is described in the documentation. These are referenced trough the expect().to(equal(), value) structure. The equal is a matcher. As well as the .to method, there is a to_not method. The biggest difficulty with defining matchers is understanding the dynamic function and class mechanisms used in Flux units programming.

If you are interested in maximum flexibility without having to extend the framework then FUint has the greatest range of predefined asserts. In this respect, it provides a good base for doing unit testing.

No comments: