Leveraging Property Based Testing
Property-based testing is one of the powerful technique to unit test the code. Unlike the example-based testing (where we Arrange a set of example inputs to Assert the system under test), Property based testing enable us to focus on what we wanted to test and liberate us from doing mundane work on coming up with sample inputs.
Recently, I got a chance to use Property based testing in an open source library that I am developing to use Suave in Azure Functions. I felt very productive and it was a such a joy to write the unit tests.
In this blog post, you are going to learn how Property-based testing has helped me and why you need to consider using(or learning) it
HttpStatusCode is an enum type and HttpCode is a discriminated union type. Refer this blog post to know more about this difference.
Identifying the Property
The first step is Property-based testing is identifying the property. In other words, it forces to think about the relationship between the input and the output.
I believe this thinking will make a huge difference in the quality of the software that we deliver.
In the first task that we are about to implement, the relationship is the integer representation of the HTTP status code is both HttpStatusCode and HttpCode.
Programmatically, this can be asserted like
1 2 3
The EnumToValue returns the integer value associated with the enum, which is nothing but the integer representation of the HTTP status code it represents. The
codeis a member of HttpCode that represents the same integer.
If this property holds true for given set of inputs, then we can assert that the function that does the transformation is working correctly.
Implementing the mapping function
The initial implementation that I had in my mind was
1 2 3 4 5 6 7
If I haven’t choosen to use Property based testing, I might have ended up with this line by line mapping for all the HTTP status codes. As a matter of fact, in my last blog post on using Suave in Azure Functions, I’ve used this same approach.
While thinking regarding properties to assert the transformation, for the first time I came to know about the Language Primitives module in F# and the EnumToValue function.
There should be an another function EnumOfValue right?
Let’s use this in the
httpStatusCode function implementation
Short and Sweet!
Property-based testing made my day and helped me to save a lot of keystrokes!
Writing Our First Property based Unit Test
1 2 3 4 5 6 7 8 9 10 11
The magic here is the
Property attribute. It makes our life easier by auto-populating the parameter
httpCode with the all the possible values and runs the unit test against each value.
As a convention, we need to return boolean by validating the property that we wanted to assert.
There is no Arrange step to setup the input parameter and FsCheck does that for you with 100% test coverage :-)
Mapping HttpCode to HttpStatusCode
The next task in the use case is doing the reverse, mapping HttpStatusCode to HttpCode.
Let’s start with the test first.
The integer representation of HTTP status code property that we used on the previous task holds true for this mapping also.
1 2 3 4 5 6 7
The implementation of
suaveHttpCode function is
1 2 3 4 5 6 7
Now if we run the unit tests, You will be getting the following error (if you are using Suave NuGet package version <= 1.1.3)
1 2 3 4 5
The reason for this failure is Suave (up to v1.1.3) doesn’t have the HTTP status code 306 (UnUsed). The unit test that we wrote was exercised by the FsCheck for the all possible values of the
HttpStatusCode enum till it encountered this failure case.
Let’s patch our unit test to fix this failure.
1 2 3 4 5 6 7 8
If you run the unit test after this, you will get an another failure
1 2 3 4 5
Again, The reason is HTTP Status Code
426 (UpgradeRequired) is not supported is Suave at this point of writing. Let’s patch this too by adding this to the list
1 2 3
After this FsCheck is happy and we too.
If I haven’t used Property based testing, for sure, I might have missed these two unsupported HTTP status codes.
The Spirit of Open Source
While looking at
Suave.Http codebase to fix the unit test failures, this is what I saw in the code
This is the beauty of open source, and I am glad to do it
Writing tests first forces you to think about the problem you're solving. Writing property-based tests forces you to think way harder.— Jessica Kerr (@jessitron) April 25, 2013