Comments

For the last one year, I was working with the development team of a leading media entertainment company in India and developed a cloud platform to distribute the DCPs across the globe in a uniform, scalable and cost-effective manner.

We built the entire platform using golang and event-driven microservices architecture. It was an exciting journey, and this blog post summarizes my experiences of using golang in this project.

The Learning Curve

Golang is a simple language to learn. Any developer who has some good experience in any programming language can pick it in a month. It took me two weeks to understand the language.

I’ve also learned a lot from the code review comments from the other developers of my team who are already familiar with the language.

The official golang site has done an excellent job in coming up two important learning resources for the beginners.

  • A Tour of Go - Through a series of in-browser hands-on tutorials you can get a good feel of the language in few hours.
  • Effective Go - It is one the best introduction as well as best practices documentation of a programming language that I have read. Right from package naming convention to how to do state sharing, this documentation will provide all it required to write an idiomatic golang code.

In addition to these resources, I recommend the Golang FAQs and the GoByExample for the beginners.

Go Routines, Channels and Select

In my perspective, go routines, channels and select are the sweet spots of golang. These abstractions are elegant and enable you to write concurrent programs without any hassles.

We have made substantial use of golang concurrency whenever it made sense, and the benefits were incredible. The blog post on Go Concurrency Patterns is an excellent read to get a feel of it.

Interfaces in golang

My favorite feature in golang is its support for interfaces.

Unlike C# or Java, you don’t need to specify the name of the interface when a type implements an interface. Because of this, the package in which the type present, doesn’t need to refer the package in which the interface has been declared.

Yan Cui has written a great blog post about its elegance.

The design for Go’s interface stems from the observation that patterns and abstractions only become apparent after we’ve seen it a few times. So rather than locking us in with abstractions at the start of a project when we’re at the point of our greatest ignorance, we can define these abstractions as and when they become apparent to us. - Yan Cui

The Go Proverbs

The Go Proverbs provides a set of principles and guidelines while developing software in golang.

Though certain principles (A little copying is better than a little dependency.) look weird, they have profound meaning behind it.

Like the saying, “You need to spend time crawling alone through shadows to truly appreciate what it is to stand in the sun.”, to appreciate these proverbs you need to spend a decent amount time in developing software in go.

Golang Standard Library

The golang standard library is futuristic. It has pretty much all that is required for solving the general problems efficiently.

The surprising factor for me is treating JSON as a first class citizen in the standard library.

Most of the languages rely on an open source library to deal with JSON. Having JSON handling as part of the standard library shows how much thought and care has been put in.

Declared and Not used Compiler Error

Another good thing that golang got it right is showing compiler errors for unused variables and packages.

The software which we write evolve along with its business requirements. Continuous refactoring, feature addition or deletion, changing the architecture to meet the new demands are some the stuff that we do during the evolution of the software. During these activities often we miss removing the unused code and the packages that we were using before.

Through this compiler error, golang forces the developer to write better code. The biggest advantage is while reading the code written by someone, you can be sure that the variables and the packages are always being used.

Also removing unused packages results in smaller binary size.

go Command (Command Line Interface) and go tools

The go command is an another well thought out feature in golang which comes by default with the installation.

Using go command, you can compile & build the code, download and install external packages, run the tests and lot more without relying on any other external tools. The go test command is incredibly fast and it made our job easier while doing Test-Driven development.

The gofmt command enabled our team to follow a consistent formatting of golang source files. Thanks to this awesome tool we hardly had code review comments on the format of the code.

The golang tools, especially goimports and gorename has been extremely useful

go install and Single executable binary

The command which I loved the most in golang is the go install command. It takes care of compiling the code and produces a stand alone executable binary. Due to caching of intermediary packages, it builds the executable binary incredibly fast.

To run this resulted binary file, we don’t have to install anything on the target machine (Like JVM, .NET).

It enabled us to share the microservice as an executable file with the other team members to create spikes, develop and locally test other dependent microservices.

Just like bundling the front-end assets into single javascript file and serving them via CDNs, we can even do continuous deployment of golang codebase by serving the binaries from a content provider like Amazon S3, if it makes sense!

We can also do cross-compilation of your golang code for different OSes and Architectures using go build command.

Open Source Libraries in golang

Golang has a thriving open source community, and it houses a vast collection of open source libraries and frameworks. Here is the list of some of the things that we have used heavily

  • AMQP - To interact with RabbitMQ
  • GORM - To perform database operations in PostgreSQL
  • go-uuid - To generate and use UUIDs
  • migrate - For database migrations
  • envconfig - For managing configuration data from environment variables
  • negroni - To write HTTP middlewares
  • httprouter - For handling HTTP routes. We have used this along with golang’s standard HTTP handlers
  • logrus - Simple and powerful library for logging
  • testify - For unit test assertions

For most of the common things that you’d like to do in your project, you can always find an off the shelf open source library

Private methods, functions and struct fields

I was admired when I came to know that just by using a naming convention, I can specify public and private methods, functions and struct fields.

It is a smart feature in the language. If you want to expose them as public, start with an upper case letter and start with a lower case letter if you want to them to be used only inside a package or a struct.

Not So Good Features in Golang

So far in this blog post, I’ve shared the features that I liked very much in golang. In the rest of the post, I am about to share the not so good features in golang. The heading was intentionally started with “Not So Good” as the things that I am about to share are not bad ones according to the golang language design principles and the go proverbs that we have seen earlier.

IMHO there is no perfect programming language, and there is no silver bullet. All languages shine in certain areas and not so good in some other areas. At the end of the day, a programming language is just a tool which we use to solve the business problems. We need to pick an appropriate language that solves the problem in hand.

Disclaimer: The intention of this blog post is to share my experiences and provide honest feedback to the people who is evaluating golang.

The Golang way of handling errors

The imperative way of handling errors is one of the common complaint raised by many people.

1
2
3
4
5
6
7
8
9
10
11
12
13
_, err = foobar1()
if err != nil {
 return err
}
_, err = foobar2()
if err != nil {
 return err
}
_, err = foobar3()
if err != nil {
 return err
}
// and so on

Rob Pike addresses this concern in a detailed blog post, Errors are values and quotes a pattern using errWriter to avoid the repetition. It is a good design and elegantly wraps the nil check on the error. The downside of this approach is we need to write an extra wrapper for every new type to get rid of the repetitiveness. The wrapper approach may get complex if the foobar1, foobar2, and foobar3 are addressing different concerns.

For people who is coming from typed functional programming languages (F#, Haskell, Scala) background, like me, this may look little awkward to write it in this way. There is a proposal to include Result type in golang but I feel it may not be incorporated.

Treating nil as value

While other languages are moving away from dealing with null values and replacing it with option types, golang is taking a backward step by treating nil (equivalent for null in Go) as a valid value. So we need to be cautious while using the nil value. Lack of discipline may result in runtime exceptions (panic in go vocabulary)

This problem can be avoided to some degree by having a function/method to return two values, the actual value to be returned and the error, and ensuring that the actual value will never be nil, if the error is nil and vice versa. Unfortunately, this approach leads to the repetition concern that we just saw.

Absence of generics

This also an another typical critic in golang. There is an answer of for this question in golang FAQ saying, it is in the pipeline but my opinion is it may not be in the near future.

There are some workarounds like using an empty interface and casting it to appropriate types using type assertion.

The real concern is trying to do map, filter, reduce operations over collections. Again there are workarounds by using templating and reflection, but it has its own cost (performance and boilerplate code). After fiddling around with different options, We went back to the classic way of iterating through for loop.

Mutable variables and imperative coding

For developers who is coming from the functional programming background, the mutable variables and imperative way of writing code are speed breakers in the golang journey.

Though golang has some good support for doing functional programming, the presence of mutable variables and imperative way doing certain things, hinder the progress.

Summary

Apart from these few concerns, I’ve liked golang and enjoyed using it.

The language design principles of golang are my important takeaways. I strongly recommend watching the talks of Rob Pike on golang design; even you are not going to use golang.

I’d consider using golang in an another project in future for sure if it is an appropriate language for the problem at hand.

Comments

Two-factor authentication is a type of Multi-factor authentication which adds an extra layer of security to the applications.

Google Authenticator is one of the popular application that implements two-factor authentication services. In this blog post, we are going to learn how to implement Two-factor authentication in web applications developed using suave

The idea presented here is a naive implementation of Two-factor authentication. The objective here is to demonstrate how to implement it in a functional programming language, F#. Things like TLS/HTTPS, preventing CSRF and other attacks are ignored for brevity.

This blog post is a part of fsharp advent calendar 2016.

Prerequisite

This blog post assumes that you are familiar with the concept of Two-factor authentication and Google Authenticator.

If you would like to know more about these, check out the below resources to get a picture of what it is all about.

We are going to use Time-based One-time Password(TOTP) algorithm in this blog post

What we will be building

We are going to build a tiny web appliaction that has an inbuilt user account with the username foo and the password bar

After successful login, the user redirected to the Profile page where the user sees his name with a couple of buttons. One to enable Two-factor authentication and another one to log out

Upon clicking the Enable Two Factor Authentication button, the user redirected to the Enable Two Factor Authentication page where the user has to scan the QR Code with the Google Authenticator App (For Android or iPhone). Then he needs to enter the verification code to enable Two-factor authentication for his account.

Google Authenticator App

If the verification code matches, the updated Profile page will look like

Now if the user logout and login again, he will be prompted to enter the verification code

After entering the verification code from the Google Authenticator, the user will be redirected to his Profile page.

Getting Started

Create a new F# Console Project with the name Suave.TwoFactorAuth and use Paket to install the following dependencies.

paket.dependencies

1
2
3
4
5
nuget FSharp.Core
nuget Suave
nuget DotLiquid
nuget Suave.DotLiquid
nuget OtpSharp

Then reference them in the Suave.TwoFactorAuth project.

Suave.TwoFactorAuth/paket.references

1
2
3
4
5
FSharp.Core
Suave
DotLiquid
Suave.DotLiquid
OtpSharp

The OtpSharp is an .NET library that we will be using to generate keys and to verify the verification code from Google Authenticator app using the TOTP algorithm.

The reference to DotLiquid library is required to render the templates using Suave.DotLiquid

Initializing DotLiquid

To use DotLiquid to render the views in Suave, we need to set the templates directory explicitly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
module Suave.TwoFactorAuth.Main

open Suave
open System.IO
open System.Reflection
open Suave.DotLiquid

let initializeDotLiquid () =
  let currentDirectory =
    let mainExeFileInfo =
      new FileInfo(Assembly.GetEntryAssembly().Location)
    mainExeFileInfo.Directory
  Path.Combine(currentDirectory.FullName, "views")
  |> setTemplatesDir

[<EntryPoint>]
let main argv =
  initializeDotLiquid ()
  0

In this sample application we are going to create a directory views. This views directory will contain the liquid templates of our appliaction

Serving the Login Page

Let’s start by serving the Login page.

Create a new directory with the name views in the Suave.TwoFactorAuth project and add a new liquid template page.liquid. This page.liquid is the master template for our application

After creating, change the ‘Copy to output’ property of the page.liquid file to ‘Copy if newer’ so that the view files copied to the build output directory.

This step is applicable for all the other view templates that we will be creating later

If you are using VS Code or atom editor, you need to do this property change manually by opening the Suave.TwoFactorAuth.fsproj file

1
2
3
4
5
6
7
8
9
10
<!-- .... -->
<ItemGroup>
  <Folder Include="views\" />
</ItemGroup>
<ItemGroup>
  <None Include="views\page.liquid">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </None>
</ItemGroup>
<!-- .... -->

Then create a new template file login.liquid in the views directory

The login.liquid view extends the page.liquid view and fill the placeholders for head and content.

To display the error messages like Password didn’t match, login.liquid view bounded to the model of type string.

Now we have the view template for the login page ready and the next step is to render it upon receiving an HTTP request.

Create a new fsharp source file Login.fs and update it as below

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
module Suave.TwoFactorAuth.Login

open Suave
open Suave.DotLiquid
open Suave.Filters
open Suave.Operators

let loginPath = "/login"

let renderLoginView (request : HttpRequest) =
  let errMsg =
    match request.["err"] with
    | Some msg -> msg
    | _ -> ""
  page "login.liquid" errMsg

let loginWebPart =
  path loginPath >=> choose [
      GET >=> request renderLoginView]

As a good practice let’s create a new module Web which will be containing all the WebParts of the application

1
2
3
4
5
6
7
// Suave.TwoFactorAuth/Suave.Web.fs
open Suave
open Login

let app =
  choose [
    loginWebPart]

Then start the Web Server

1
2
3
4
5
6
7
8
// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
// ...
open Web
// ...
let main argv =
  // ...
  startWebServer defaultConfig app
  0

Keeping all the Suave WebParts in a single place like we did in the Web.fs file, enable us to host Suave in Azure Functions or Asp.Net Core without doing any significant changes.

Handling User Login

To handle the login request from the user, we need to have some users in the application. Let’s hardcode a user account.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
module Suave.TwoFactorAuth.User

open System.Collections.Generic

type User = {
  Username : string
  Password : string
}

let private users = new Dictionary<string, User>()

users.Add("foo", {Username = "foo"; Password = "bar"})

let getUser username =
  match users.TryGetValue username with
  | true, user -> Some user
  | _ -> None

Post successful login, to serve the subsequent requests we need to identify the user who logged in. We can achieve it Suave using statefulForSession, which initializes a user state for a browsing session.

Let’s create some helper functions to do this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// Suave.TwoFactorAuth/Combinators.fs
module Suave.TwoFactorAuth.Combinators

open Suave.State.CookieStateStore
open Suave.Cookie
open Suave
open Suave.Operators
open Suave.Authentication

let sessionSet failureF key value =
  statefulForSession
  >=> context (fun ctx ->
                match HttpContext.state ctx with
                | Some state -> state.set key value
                | _ -> failureF
              )

let sessionGet failureF key successF =
  statefulForSession
  >=> context (fun ctx ->
                match HttpContext.state ctx with
                | Some store ->
                  match store.get key with
                  | Some value -> successF value
                  | _ -> failureF
                | _ -> failureF
  )

let clearSession =
  unsetPair SessionAuthCookie
    >=> unsetPair StateCookie

The sessionSet function takes a WebPart and a key value pair and tries to persist the value in the session state with the given key. If it fails, it calls the WebPart.

The sessionGet function takes a success WebPart Combinator, a failure WebPart, and a key. If retrieving the value from session state is successful it calls the success WebPart combinator with the retrieved value. In case of retrieval failure it calls the failure WebPart

The clearSession function clears the state. We will be using it while implementing log out

Now we have all the building blocks for handling user login request, and it’s time to start its implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
// Suave.TwoFactorAuth/Login.fs
// ...
open Suave.Redirection
open Suave.Authentication
open Suave.Cookie

open Combinators
open User
// ...
let userSessionKey = "loggedUser"

let redirectToLogin = function
  | Some errMsg -> FOUND (sprintf "%s?err=%s" loginPath errMsg)
  | None -> FOUND loginPath

let loginSucess failureW redirectPath username =
  authenticated Cookie.CookieLife.Session false
    >=> sessionSet failureW userSessionKey username
    >=> FOUND redirectPath

let onLogin redirectPath (request : HttpRequest) =
  match request.["Username"], request.["Password"] with
  | Some username, Some password ->
    match getUser username with
    | Some user ->
      match user.Password = password with
      | true -> loginSucess never redirectPath username
      | _ -> redirectToLogin (Some "Password didn't match")
    | _ -> redirectToLogin (Some "Invalid username")
  | _ -> redirectToLogin (Some "Invalid request")

let secured webpart =
  let onFail = redirectToLogin (Some "sign-in to access")
  sessionGet onFail userSessionKey webpart

let loginWebPart redirectPath =
  path loginPath >=> choose [
      // ...
      POST >=> request (onLogin redirectPath)]

The key function to note here is secured that takes a WebPart. It calls this WebPart only if the user has logged in. If he didn’t the user will be redirected to the Login page

After successful login, we need to redirect the user to his profile page. Let’s create a profile.liquid a view template for the Profile page

To render this profile page let’s add some code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// Suave.TwoFactorAuth/Profile.fs
module Suave.TwoFactorAuth.Profile

open Suave.DotLiquid
open Suave.Redirection
open Suave.Filters
open Suave.Operators

open User
open Login

type ProfileViewModel = {
  Username: string
  SecretKey : string
  IsTwoFactorAuthEnabled : bool
}
with static member FromUser user =
        {
          SecretKey = ""
          IsTwoFactorAuthEnabled = false
          Username = user.Username
        }

let profilePath = "/profile"

let renderProfile notFoundPath username =
  match getUser username with
  | Some user ->
    user
    |> ProfileViewModel.FromUser
    |> page "profile.liquid"
  | _ -> FOUND notFoundPath

let profileWebPart notFoundPath =
  path profilePath >=> secured (renderProfile notFoundPath)

The labels IsTwoFactorAuthEnabled, SecretKey are just blank right now, and we will be seeing them in action while adding two-factor authentication

The notfound.liquid page that is going to our fancy 404 page

1
2
<!-- Suave.TwoFactorAuth/views/not_found.liquid -->
Not Found :(

The final step is to put these WebParts together

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Suave.TwoFactorAuth/Web.fs
// ...
open Profile
open Suave.DotLiquid
open Suave.Filters
open Suave.Operators

let notFoundPath = "/notfound"

let app =
  choose [
    loginWebPart profilePath
    profileWebPart notFoundPath
    path notFoundPath >=> page "not_found.liquid" ""
  ]

Handling Logout

Handling logout is a simpler task as we have all the infrastructure already in place.

1
2
3
4
5
6
7
8
9
10
11
// Suave.TwoFactorAuth/Web.fs
// ...
open Combinators
open Login
// ...

let app =
  choose [
    // ...
    path "/logout" >=> clearSession >=> redirectToLogin None
  ]

Enabling Two-factor Authentication

To enable Two-factor authentication, we need to change our User domain model first.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// Suave.TwoFactorAuth/User.fs
// ...
type TwoFactorAuthentication =
| Enabled of SecretKey:string
| Disabled

type User = {
  // ...
  TwoFactorAuthentication : TwoFactorAuthentication
}
// ... 
// Let's assume TwoFactorAuthentication is disabled by default 
users.Add("foo", {Username = "foo"; Password = "bar"; TwoFactorAuthentication = Disabled})
// ...
let enableTwoFactorAuth username key =
  match getUser username with
  | Some user ->
    users.[username] <- {user with TwoFactorAuthentication = Enabled key}
  | _ -> ()

The next step is to define a liquid view template for the enable_two_factor page.

While enabling the Two-factor authentication, we need to generate a secret key for the user that will be required for both, one-time verification code generation as well as its verification.

So, in the enable_two_factor.liquid template we pass the generated SecretKey as a hidden input which will then be used for the verification of the code.

1
<input type="hidden" name="SecretKey" value="">

Now we need to render the enable_two_factor page in response to the HTTP GET request /enable_two_factor

Let’s create a new module GoogleAuthenticator to put the Two-factor authentication related functions together

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
// Suave.TwoFactorAuth/GoogleAuthenticator.fs
module Suave.TwoFactorAuth.GoogleAuthenticator

open Suave
open Suave.Filters
open Suave.Operators
open Suave.DotLiquid
open Suave.Redirection
open OtpSharp
open Base32

open Login
open User

let enableTwoFactorAuthPath = "/enable_two_factor"

type EnableTwoFactorViewModel = {
  Key : string
  Url : string
  Err : string
}
with static member From username err =
      let secretKey = KeyGeneration.GenerateRandomKey(20)
      let appName = "SuaveRocks"
      let label = sprintf "%s (%s.com/%s)" appName appName username
      let keyUrl = KeyUrl.GetTotpUrl(secretKey, label)
      { Url = sprintf "https://qrcode.kaywa.com/img.php?s=4&d=%s&issuer=%s" keyUrl appName
        Key = Base32Encoder.Encode(secretKey)
        Err = err}

let renderEnableTwoFactorAuthView notFoundPath username ctx = async {
  match getUser username with
  | Some user ->
    let err =
      match ctx.request.["err"] with
      | Some err -> err
      | _ -> ""
    let vm = EnableTwoFactorViewModel.From username err
    return! page "enable_two_factor.liquid" vm ctx
  | _ -> return! redirect notFoundPath ctx
}

let googleAuthenticatorWebPart notFoundPath =
  choose [
    path enableTwoFactorAuthPath >=> choose [
      GET >=> secured (renderEnableTwoFactorAuthView notFoundPath)
    ]]

As we did in the login page, we are using the err query string in the request to pass the verification code mismatch errors.

We are leveraging the OtpSharp library to generate the URL that is in turn represented as a QR Code.

If you would like to how Google Authenticator interprets the generated key and the issuer name from the URL embedded in the QR Code, check out the UriFormat documentation.

The last step in rendering this page is adding the googleAuthenticatorWebPart in the Web module where we are putting all the WebParts together

1
2
3
4
5
6
7
8
9
// Suave.TwoFactorAuth/Web.fs
// ...
open GoogleAuthenticator

let app =
  choose [
    // ...
    googleAuthenticatorWebPart notFoundPath
  ]

While enabling Two-factor authentication, the user has to scan the QR-Code from his Google Authenticator app and he will be getting a one-time verification code like this upon adding

Then he will be entering this in the enable_two_factor page and click Enable.

Let’s handle this POST request.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Suave.TwoFactorAuth/GoogleAuthenticator.fs
// ...
let verifyOtp secretKey code =
  let otp = new Totp(Base32Encoder.Decode secretKey)
  otp.VerifyTotp(code, ref 0L, new VerificationWindow(2, 2))

let enableTwoFactorAuth redirectPath notFoundPath username ctx = async {
  match ctx.request.["SecretKey"], ctx.request.["Code"] with
  | Some secretKey, Some code ->
    match verifyOtp secretKey code with
    | true ->
      enableTwoFactorAuth username secretKey
      return! redirect redirectPath ctx
    | _ ->
      let redirectTo =
        sprintf "%s?err=code validation failed" enableTwoFactorAuthPath
      return! redirect redirectTo ctx
  | _ -> return! redirect notFoundPath ctx
}

let googleAuthenticatorWebPart redirectPath notFoundPath =
  choose [
    path enableTwoFactorAuthPath >=> choose [
      // ...
      POST >=> secured (enableTwoFactorAuth redirectPath notFoundPath)
    ]]

Thanks to the OtpSharp library for making our job simpler here. We just need to get the SecretKey, and the Code from the POST request and get it verified using OtpSharp’s VerifyTotp function.

If the verification is successful, we will be enabling the Two-factor authentication for the user in our in-memory backend using the enableTwoFactorAuth function and then redirect to the redirect path which in this case the Profile page.

1
2
3
4
5
6
7
8
// Suave.TwoFactorAuth/Web.fs
// ...

let app =
  choose [
    // ...
    googleAuthenticatorWebPart profilePath notFoundPath
  ]

Login With Two-factor Authentication

The last step is to prompt for the verification code whenever the Two-factor Authentication enabled user tries to log in and verify the one-time verification code before granting the access.

Let’s start by defining the liquid view template auth_code.liquid for getting the one-time verification code.

1
2
3
4
5
6
7
8
// Suave.TwoFactorAuth/GoogleAuthenticator.fs
// ...
let authCodePath = "/authcode"

let googleAuthenticatorWebPart redirectPath notFoundPath =
  choose [
    // ...
    path authCodePath >=> page "auth_code.liquid" ""]

After username and password verification, the user has to be redirected the auth_code page

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
// Suave.TwoFactorAuth/Login.fs
// ...
let authCodeSessionKey = "loginUser"

let onLogin redirectPath authCodePath (request : HttpRequest) =
  match request.["Username"], request.["Password"] with
  | Some username, Some password ->
    match getUser username with
    | Some user ->
      match user.Password = password, user.TwoFactorAuthentication with
      | true, Disabled -> loginSucess never redirectPath username
      | true, Enabled _ ->
          sessionSet never authCodeSessionKey username
            >=> FOUND authCodePath
      // ...

// ...

let loginWebPart redirectPath authCodePath =
  path loginPath >=> choose [
      // ...
      POST >=> request (onLogin redirectPath authCodePath)]
1
2
3
4
5
6
7
// Suave.TwoFactorAuth/Web.fs
// ...
let app =
  choose [
    loginWebPart profilePath authCodePath
    // ...
  ]

We are using a separate session key authCodeSessionKey to hold the username of the user.

The final step is verifying the one-time verification code (from the Google Authenticator app) entered by the user.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// Suave.TwoFactorAuth/GoogleAuthenticator.fs
// ...
open Combinators
// ...
let onAuthCodeVerification redirectPath (request : HttpRequest) username =
  match request.["Code"], getUser username with
  | Some code, Some user ->
    match user.TwoFactorAuthentication with
    | Enabled secretKey ->
      match verifyOtp secretKey code with
      | true -> loginSucess never redirectPath user.Username
      | _ -> redirectToLogin (Some "invalid otp")
    | _ -> redirectToLogin (Some "invalid request")
  | _ -> redirectToLogin (Some "invalid request")

let onVerifyAuthCode redirectPath httpRequest =
  let onFail = redirectToLogin (Some "invalid request")
  let onAuthCodeVerification =
    onAuthCodeVerification redirectPath httpRequest
  sessionGet onFail authCodeSessionKey onAuthCodeVerification

let googleAuthenticatorWebPart redirectPath notFoundPath =
  choose [
    // ...
    path "/verify_auth_code" >=> request (onVerifyAuthCode redirectPath)]

That’s it! We have successfully implemented Two-factor authentication.

The complete source code is available in my GitHub repository

Property-based testing is one of the powerful technique to unit test the code. Unlike the example-based testing (where we Arrange a set of example inputs to Assert the system under test), Property based testing enable us to focus on what we wanted to test and liberate us from doing mundane work on coming up with sample inputs.

Recently, I got a chance to use Property based testing in an open source library that I am developing to use Suave in Azure Functions. I felt very productive and it was a such a joy to write the unit tests.

In this blog post, you are going to learn how Property-based testing has helped me and why you need to consider using(or learning) it

Use Case

Let me start with the use case that I wanted to test. The library that I am working is an adapter between two HTTP protocol abstractions, System.Net.Http and Suave.Http

The first task is to map HttpStatusCode of System.Net to the HttpCode of Suave.Http and the next task is doing the vice-versa.

HttpStatusCode is an enum type and HttpCode is a discriminated union type. Refer this blog post to know more about this difference.

Identifying the Property

The first step is Property-based testing is identifying the property. In other words, it forces to think about the relationship between the input and the output.

I believe this thinking will make a huge difference in the quality of the software that we deliver.

In the first task that we are about to implement, the relationship is the integer representation of the HTTP status code is both HttpStatusCode and HttpCode.

Programmatically, this can be asserted like

1
2
3
// boolean condition
// In F# (=) operator represents the equality checking
LanguagePrimitives.EnumToValue httpStatusCode = httpCode.code

The EnumToValue returns the integer value associated with the enum, which is nothing but the integer representation of the HTTP status code it represents. The code is a member of HttpCode that represents the same integer.

If this property holds true for given set of inputs, then we can assert that the function that does the transformation is working correctly.

Implementing the mapping function httpStatusCode

The initial implementation that I had in my mind was

1
2
3
4
5
6
7
let NetStatusCode = function
| HttpCode.HTTP_200 -> HttpStatusCode.OK
| HttpCode.HTTP_201 -> HttpStatusCode.Created
| HttpCode.HTTP_400 -> HttpStatusCode.BadRequest
| HttpCode.HTTP_404 -> HttpStatusCode.NotFound
| HttpCode.HTTP_202 -> HttpStatusCode.Accepted
| // ...

If I haven’t choosen to use Property based testing, I might have ended up with this line by line mapping for all the HTTP status codes. As a matter of fact, in my last blog post on using Suave in Azure Functions, I’ve used this same approach.

While thinking regarding properties to assert the transformation, for the first time I came to know about the Language Primitives module in F# and the EnumToValue function.

There should be an another function EnumOfValue right?

Yes!

Let’s use this in the httpStatusCode function implementation

1
2
let httpStatusCode (httpCode : HttpCode) : HttpStatusCode =
  LanguagePrimitives.EnumOfValue httpCode.code

Short and Sweet!

Property-based testing made my day and helped me to save a lot of keystrokes!

Writing Our First Property based Unit Test

In F# we can use the FsCheck library to write the Property based unit tests. For this implementation, we will be using FsCheck.Xunit to run Property tests in XUnit

1
2
3
4
5
6
7
8
9
10
11
open Suave.Http
open System.Net
open FsCheck
open FsCheck.Xunit

[<Property>]
let ``httpStatusCode maps Suave's HttpCode to System.Net's HttpStatusCode ``
  (httpCode : HttpCode) =

    let httpStatusCode = Response.httpStatusCode httpCode
    LanguagePrimitives.EnumToValue httpStatusCode = httpCode.code

The magic here is the Property attribute. It makes our life easier by auto-populating the parameter httpCode with the all the possible values and runs the unit test against each value.

As a convention, we need to return boolean by validating the property that we wanted to assert.

There is no Arrange step to setup the input parameter and FsCheck does that for you with 100% test coverage :-)

Mapping HttpCode to HttpStatusCode

The next task in the use case is doing the reverse, mapping HttpStatusCode to HttpCode.

Let’s start with the test first.

The integer representation of HTTP status code property that we used on the previous task holds true for this mapping also.

1
2
3
4
5
6
7
[<Property>]
let ``suaveHttpCode maps System.Net's HttpStatusCode to Suave's HttpCode if exists``
  (httpStatusCode : HttpStatusCode) =
    let httpCode = suaveHttpCode httpStatusCode
    let code = LanguagePrimitives.EnumToValue httpStatusCode

    Option.isSome httpCode && httpCode.Value.code = code

The suaveHttpCode function returns an Option type as Suave.Http.HttpCode’s tryParse function parses the integer HTTP status code and returns the corresponding HttpCode if the parsing succeeds.

The implementation of suaveHttpCode function is

1
2
3
4
5
6
7
let suaveHttpCode (httpStatusCode : HttpStatusCode) =
  let code =
    LanguagePrimitives.EnumToValue httpStatusCode
    |> HttpCode.tryParse
  match code with
  | Choice1Of2 httpCode -> Some httpCode
  | _ -> None

Now if we run the unit tests, You will be getting the following error (if you are using Suave NuGet package version <= 1.1.3)

1
2
3
4
5
Suave.Azure.Functions.Tests.suaveHttpCode maps System.Net's HttpStatusCode to Suave's HttpCode [FAIL]
FsCheck.Xunit.PropertyFailedException :
     Falsifiable, after 7 tests (0 shrinks) (StdGen (1629668181,296211181)):
     Original:
     Unused

The reason for this failure is Suave (up to v1.1.3) doesn’t have the HTTP status code 306 (UnUsed). The unit test that we wrote was exercised by the FsCheck for the all possible values of the HttpStatusCode enum till it encountered this failure case.

Let’s patch our unit test to fix this failure.

1
2
3
4
5
6
7
8
let httpCode = Response.suaveHttpCode httpStatusCode
  let code = LanguagePrimitives.EnumToValue httpStatusCode
  let unSupportedSuaveHttpCodes = [306]

  if List.contains code unSupportedSuaveHttpCodes then
    Option.isNone httpCode
  else
    Option.isSome httpCode && httpCode.Value.code = code

If you run the unit test after this, you will get an another failure

1
2
3
4
5
Suave.Azure.Functions.Tests.suaveHttpCode maps System.Net's HttpStatusCode to Suave's HttpCode [FAIL]
FsCheck.Xunit.PropertyFailedException :
  Falsifiable, after 10 tests (0 shrinks) (StdGen (507658935,296211184)):
  Original:
  UpgradeRequired

Again, The reason is HTTP Status Code 426 (UpgradeRequired) is not supported is Suave at this point of writing. Let’s patch this too by adding this to the list unSupportedSuaveHttpCodes.

1
2
3
// ...
let unSupportedSuaveHttpCodes = [306;426]
// ...

After this FsCheck is happy and we too.

If I haven’t used Property based testing, for sure, I might have missed these two unsupported HTTP status codes.

The Spirit of Open Source

While looking at Suave.Http codebase to fix the unit test failures, this is what I saw in the code

This is the beauty of open source, and I am glad to do it

Summary