What unit testing frameworks are available for F#

Question

I am looking specifically for frameworks that allow me to take advantage of unique features of the language. I am aware of FsUnit. Would you recommend something else, and why?

Answer 1

My own unit testing library, Unquote, takes advantage of F# quotations to allow you to write test assertions as plain, statically checked F# boolean expressions and automatically produces nice step-by-step test failure messages. For example, the following failing xUnit test

[<Fact>]
let ``demo Unquote xUnit support`` () =
    test <@ ([3; 2; 1; 0] |> List.map ((+) 1)) = [1 + 3..1 + 0] @>

produces the following failure message

Test 'Module.demo Unquote xUnit support' failed: 

([3; 2; 1; 0] |> List.map ((+) 1)) = [1 + 3..1 + 0]
[4; 3; 2; 1] = [4..1]
[4; 3; 2; 1] = []
false

        C:File.fs(28,0): at Module.demo Unquote xUnit support()

FsUnit and Unquote have similar missions: to allow you to write tests in an idiomatic way, and to produce informative failure messages. But FsUnit is really just a small wrapper around NUnit Constraints, creating a DSL which hides object construction behind composable function calls. But it comes at a cost: you lose static type checking in your assertions. For example, the following is valid in FsUnit

[<Test>]
let test1 () =
    1 |> should not (equal "2")

But with Unquote, you get all of F#'s static type-checking features so the equivalent assertion would not even compile, preventing us from introducing a bug in our test code

[<Test>] //yes, Unquote supports both xUnit and NUnit automatically
let test2 () =
    test <@ 1 <> "2" @> //simple assertions may be written more concisely, e.g. 1 <>! "2"
    //           ^^^
    //Error 22 This expression was expected to have type int but here has type string

Also, since quotations are able to capture more information at compile time about an assertion expression, failure messages are a lot richer too. For example the failing FsUnit assertion 1 |> should not (equal 1) produces the message

Test 'Test.Swensen.Unquote.VerifyNunitSupport.test1' failed: 
  Expected: not 1
  But was:  1
    C:UsersStephenDocumentsVisual Studio 2010ProjectsUnquoteVerifyNunitSupportFsUnit.fs(11,0): at FsUnit.should[a,a](FSharpFunc`2 f, a x, Object y)
    C:UsersStephenDocumentsVisual Studio 2010ProjectsUnquoteVerifyNunitSupportVerifyNunitSupport.fs(29,0): at Test.Swensen.Unquote.VerifyNunitSupport.test1()

Whereas the failing Unquote assertion 1 <>! 1 produces the following failure message (notice the cleaner stack trace too)

Test 'Test.Swensen.Unquote.VerifyNunitSupport.test1' failed: 

1 <> 1
false

    C:UsersStephenDocumentsVisual Studio 2010ProjectsUnquoteVerifyNunitSupportVerifyNunitSupport.fs(29,0): at Test.Swensen.Unquote.VerifyNunitSupport.test1()

And of course from my first example at the beginning of this answer, you can see just how rich and complex Unquote expressions and failure messages can get.

Another major benefit of using plain F# expressions as test assertions over the FsUnit DSL, is that it fits very well with the F# process of developing unit tests. I think a lot of F# developers start by developing and testing code with the assistance of FSI. Hence, it is very easy to go from ad-hoc FSI tests to formal tests. In fact, in addition to special support for xUnit and NUnit (though any exception-based unit testing framework is supported as well), all Unquote operators work within FSI sessions too.