Comments

Two-factor authentication is a type of Multi-factor authentication which adds an extra layer of security to the applications.

Google Authenticator is one of the popular application that implements two-factor authentication services. In this blog post, we are going to learn how to implement Two-factor authentication in web applications developed using suave

The idea presented here is a naive implementation of Two-factor authentication. The objective here is to demonstrate how to implement it in a functional programming language, F#. Things like TLS/HTTPS, preventing CSRF and other attacks are ignored for brevity.

This blog post is a part of fsharp advent calendar 2016.

Prerequisite

This blog post assumes that you are familiar with the concept of Two-factor authentication and Google Authenticator.

If you would like to know more about these, check out the below resources to get a picture of what it is all about.

We are going to use Time-based One-time Password(TOTP) algorithm in this blog post

What we will be building

We are going to build a tiny web appliaction that has an inbuilt user account with the username foo and the password bar

After successful login, the user redirected to the Profile page where the user sees his name with a couple of buttons. One to enable Two-factor authentication and another one to log out

Upon clicking the Enable Two Factor Authentication button, the user redirected to the Enable Two Factor Authentication page where the user has to scan the QR Code with the Google Authenticator App (For Android or iPhone). Then he needs to enter the verification code to enable Two-factor authentication for his account.

Google Authenticator App

If the verification code matches, the updated Profile page will look like

Now if the user logout and login again, he will be prompted to enter the verification code

After entering the verification code from the Google Authenticator, the user will be redirected to his Profile page.

Getting Started

Create a new F# Console Project with the name Suave.TwoFactorAuth and use Paket to install the following dependencies.

paket.dependencies

1
2
3
4
5
nuget FSharp.Core
nuget Suave
nuget DotLiquid
nuget Suave.DotLiquid
nuget OtpSharp

Then reference them in the Suave.TwoFactorAuth project.

Suave.TwoFactorAuth/paket.references

1
2
3
4
5
FSharp.Core
Suave
DotLiquid
Suave.DotLiquid
OtpSharp

The OtpSharp is an .NET library that we will be using to generate keys and to verify the verification code from Google Authenticator app using the TOTP algorithm.

The reference to DotLiquid library is required to render the templates using Suave.DotLiquid

Initializing DotLiquid

To use DotLiquid to render the views in Suave, we need to set the templates directory explicitly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
module Suave.TwoFactorAuth.Main

open Suave
open System.IO
open System.Reflection
open Suave.DotLiquid

let initializeDotLiquid () =
  let currentDirectory =
    let mainExeFileInfo =
      new FileInfo(Assembly.GetEntryAssembly().Location)
    mainExeFileInfo.Directory
  Path.Combine(currentDirectory.FullName, "views")
  |> setTemplatesDir

[<EntryPoint>]
let main argv =
  initializeDotLiquid ()
  0

In this sample application we are going to create a directory views. This views directory will contain the liquid templates of our appliaction

Serving the Login Page

Let’s start by serving the Login page.

Create a new directory with the name views in the Suave.TwoFactorAuth project and add a new liquid template page.liquid. This page.liquid is the master template for our application

After creating, change the ‘Copy to output’ property of the page.liquid file to ‘Copy if newer’ so that the view files copied to the build output directory.

This step is applicable for all the other view templates that we will be creating later

If you are using VS Code or atom editor, you need to do this property change manually by opening the Suave.TwoFactorAuth.fsproj file

1
2
3
4
5
6
7
8
9
10
<!-- .... -->
<ItemGroup>
  <Folder Include="views\" />
</ItemGroup>
<ItemGroup>
  <None Include="views\page.liquid">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </None>
</ItemGroup>
<!-- .... -->

Then create a new template file login.liquid in the views directory

The login.liquid view extends the page.liquid view and fill the placeholders for head and content.

To display the error messages like Password didn’t match, login.liquid view bounded to the model of type string.

Now we have the view template for the login page ready and the next step is to render it upon receiving an HTTP request.

Create a new fsharp source file Login.fs and update it as below

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
module Suave.TwoFactorAuth.Login

open Suave
open Suave.DotLiquid
open Suave.Filters
open Suave.Operators

let loginPath = "/login"

let renderLoginView (request : HttpRequest) =
  let errMsg =
    match request.["err"] with
    | Some msg -> msg
    | _ -> ""
  page "login.liquid" errMsg

let loginWebPart =
  path loginPath >=> choose [
      GET >=> request renderLoginView]

As a good practice let’s create a new module Web which will be containing all the WebParts of the application

1
2
3
4
5
6
7
// Suave.TwoFactorAuth/Suave.Web.fs
open Suave
open Login

let app =
  choose [
    loginWebPart]

Then start the Web Server

1
2
3
4
5
6
7
8
// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
// ...
open Web
// ...
let main argv =
  // ...
  startWebServer defaultConfig app
  0

Keeping all the Suave WebParts in a single place like we did in the Web.fs file, enable us to host Suave in Azure Functions or Asp.Net Core without doing any significant changes.

Handling User Login

To handle the login request from the user, we need to have some users in the application. Let’s hardcode a user account.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
module Suave.TwoFactorAuth.User

open System.Collections.Generic

type User = {
  Username : string
  Password : string
}

let private users = new Dictionary<string, User>()

users.Add("foo", {Username = "foo"; Password = "bar"})

let getUser username =
  match users.TryGetValue username with
  | true, user -> Some user
  | _ -> None

Post successful login, to serve the subsequent requests we need to identify the user who logged in. We can achieve it Suave using statefulForSession, which initializes a user state for a browsing session.

Let’s create some helper functions to do this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// Suave.TwoFactorAuth/Combinators.fs
module Suave.TwoFactorAuth.Combinators

open Suave.State.CookieStateStore
open Suave.Cookie
open Suave
open Suave.Operators
open Suave.Authentication

let sessionSet failureF key value =
  statefulForSession
  >=> context (fun ctx ->
                match HttpContext.state ctx with
                | Some state -> state.set key value
                | _ -> failureF
              )

let sessionGet failureF key successF =
  statefulForSession
  >=> context (fun ctx ->
                match HttpContext.state ctx with
                | Some store ->
                  match store.get key with
                  | Some value -> successF value
                  | _ -> failureF
                | _ -> failureF
  )

let clearSession =
  unsetPair SessionAuthCookie
    >=> unsetPair StateCookie

The sessionSet function takes a WebPart and a key value pair and tries to persist the value in the session state with the given key. If it fails, it calls the WebPart.

The sessionGet function takes a success WebPart Combinator, a failure WebPart, and a key. If retrieving the value from session state is successful it calls the success WebPart combinator with the retrieved value. In case of retrieval failure it calls the failure WebPart

The clearSession function clears the state. We will be using it while implementing log out

Now we have all the building blocks for handling user login request, and it’s time to start its implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
// Suave.TwoFactorAuth/Login.fs
// ...
open Suave.Redirection
open Suave.Authentication
open Suave.Cookie

open Combinators
open User
// ...
let userSessionKey = "loggedUser"

let redirectToLogin = function
  | Some errMsg -> FOUND (sprintf "%s?err=%s" loginPath errMsg)
  | None -> FOUND loginPath

let loginSucess failureW redirectPath username =
  authenticated Cookie.CookieLife.Session false
    >=> sessionSet failureW userSessionKey username
    >=> FOUND redirectPath

let onLogin redirectPath (request : HttpRequest) =
  match request.["Username"], request.["Password"] with
  | Some username, Some password ->
    match getUser username with
    | Some user ->
      match user.Password = password with
      | true -> loginSucess never redirectPath username
      | _ -> redirectToLogin (Some "Password didn't match")
    | _ -> redirectToLogin (Some "Invalid username")
  | _ -> redirectToLogin (Some "Invalid request")

let secured webpart =
  let onFail = redirectToLogin (Some "sign-in to access")
  sessionGet onFail userSessionKey webpart

let loginWebPart redirectPath =
  path loginPath >=> choose [
      // ...
      POST >=> request (onLogin redirectPath)]

The key function to note here is secured that takes a WebPart. It calls this WebPart only if the user has logged in. If he didn’t the user will be redirected to the Login page

After successful login, we need to redirect the user to his profile page. Let’s create a profile.liquid a view template for the Profile page

To render this profile page let’s add some code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
// Suave.TwoFactorAuth/Profile.fs
module Suave.TwoFactorAuth.Profile

open Suave.DotLiquid
open Suave.Redirection
open Suave.Filters
open Suave.Operators

open User
open Login

type ProfileViewModel = {
  Username: string
  SecretKey : string
  IsTwoFactorAuthEnabled : bool
}
with static member FromUser user =
        {
          SecretKey = ""
          IsTwoFactorAuthEnabled = false
          Username = user.Username
        }

let profilePath = "/profile"

let renderProfile notFoundPath username =
  match getUser username with
  | Some user ->
    user
    |> ProfileViewModel.FromUser
    |> page "profile.liquid"
  | _ -> FOUND notFoundPath

let profileWebPart notFoundPath =
  path profilePath >=> secured (renderProfile notFoundPath)

The labels IsTwoFactorAuthEnabled, SecretKey are just blank right now, and we will be seeing them in action while adding two-factor authentication

The notfound.liquid page that is going to our fancy 404 page

1
2
<!-- Suave.TwoFactorAuth/views/not_found.liquid -->
Not Found :(

The final step is to put these WebParts together

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Suave.TwoFactorAuth/Web.fs
// ...
open Profile
open Suave.DotLiquid
open Suave.Filters
open Suave.Operators

let notFoundPath = "/notfound"

let app =
  choose [
    loginWebPart profilePath
    profileWebPart notFoundPath
    path notFoundPath >=> page "not_found.liquid" ""
  ]

Handling Logout

Handling logout is a simpler task as we have all the infrastructure already in place.

1
2
3
4
5
6
7
8
9
10
11
// Suave.TwoFactorAuth/Web.fs
// ...
open Combinators
open Login
// ...

let app =
  choose [
    // ...
    path "/logout" >=> clearSession >=> redirectToLogin None
  ]

Enabling Two-factor Authentication

To enable Two-factor authentication, we need to change our User domain model first.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// Suave.TwoFactorAuth/User.fs
// ...
type TwoFactorAuthentication =
| Enabled of SecretKey:string
| Disabled

type User = {
  // ...
  TwoFactorAuthentication : TwoFactorAuthentication
}
// ... 
// Let's assume TwoFactorAuthentication is disabled by default 
users.Add("foo", {Username = "foo"; Password = "bar"; TwoFactorAuthentication = Disabled})
// ...
let enableTwoFactorAuth username key =
  match getUser username with
  | Some user ->
    users.[username] <- {user with TwoFactorAuthentication = Enabled key}
  | _ -> ()

The next step is to define a liquid view template for the enable_two_factor page.

While enabling the Two-factor authentication, we need to generate a secret key for the user that will be required for both, one-time verification code generation as well as its verification.

So, in the enable_two_factor.liquid template we pass the generated SecretKey as a hidden input which will then be used for the verification of the code.

1
<input type="hidden" name="SecretKey" value="">

Now we need to render the enable_two_factor page in response to the HTTP GET request /enable_two_factor

Let’s create a new module GoogleAuthenticator to put the Two-factor authentication related functions together

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
// Suave.TwoFactorAuth/GoogleAuthenticator.fs
module Suave.TwoFactorAuth.GoogleAuthenticator

open Suave
open Suave.Filters
open Suave.Operators
open Suave.DotLiquid
open Suave.Redirection
open OtpSharp
open Base32

open Login
open User

let enableTwoFactorAuthPath = "/enable_two_factor"

type EnableTwoFactorViewModel = {
  Key : string
  Url : string
  Err : string
}
with static member From username err =
      let secretKey = KeyGeneration.GenerateRandomKey(20)
      let appName = "SuaveRocks"
      let label = sprintf "%s (%s.com/%s)" appName appName username
      let keyUrl = KeyUrl.GetTotpUrl(secretKey, label)
      { Url = sprintf "https://qrcode.kaywa.com/img.php?s=4&d=%s&issuer=%s" keyUrl appName
        Key = Base32Encoder.Encode(secretKey)
        Err = err}

let renderEnableTwoFactorAuthView notFoundPath username ctx = async {
  match getUser username with
  | Some user ->
    let err =
      match ctx.request.["err"] with
      | Some err -> err
      | _ -> ""
    let vm = EnableTwoFactorViewModel.From username err
    return! page "enable_two_factor.liquid" vm ctx
  | _ -> return! redirect notFoundPath ctx
}

let googleAuthenticatorWebPart notFoundPath =
  choose [
    path enableTwoFactorAuthPath >=> choose [
      GET >=> secured (renderEnableTwoFactorAuthView notFoundPath)
    ]]

As we did in the login page, we are using the err query string in the request to pass the verification code mismatch errors.

We are leveraging the OtpSharp library to generate the URL that is in turn represented as a QR Code.

If you would like to how Google Authenticator interprets the generated key and the issuer name from the URL embedded in the QR Code, check out the UriFormat documentation.

The last step in rendering this page is adding the googleAuthenticatorWebPart in the Web module where we are putting all the WebParts together

1
2
3
4
5
6
7
8
9
// Suave.TwoFactorAuth/Web.fs
// ...
open GoogleAuthenticator

let app =
  choose [
    // ...
    googleAuthenticatorWebPart notFoundPath
  ]

While enabling Two-factor authentication, the user has to scan the QR-Code from his Google Authenticator app and he will be getting a one-time verification code like this upon adding

Then he will be entering this in the enable_two_factor page and click Enable.

Let’s handle this POST request.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Suave.TwoFactorAuth/GoogleAuthenticator.fs
// ...
let verifyOtp secretKey code =
  let otp = new Totp(Base32Encoder.Decode secretKey)
  otp.VerifyTotp(code, ref 0L, new VerificationWindow(2, 2))

let enableTwoFactorAuth redirectPath notFoundPath username ctx = async {
  match ctx.request.["SecretKey"], ctx.request.["Code"] with
  | Some secretKey, Some code ->
    match verifyOtp secretKey code with
    | true ->
      enableTwoFactorAuth username secretKey
      return! redirect redirectPath ctx
    | _ ->
      let redirectTo =
        sprintf "%s?err=code validation failed" enableTwoFactorAuthPath
      return! redirect redirectTo ctx
  | _ -> return! redirect notFoundPath ctx
}

let googleAuthenticatorWebPart redirectPath notFoundPath =
  choose [
    path enableTwoFactorAuthPath >=> choose [
      // ...
      POST >=> secured (enableTwoFactorAuth redirectPath notFoundPath)
    ]]

Thanks to the OtpSharp library for making our job simpler here. We just need to get the SecretKey, and the Code from the POST request and get it verified using OtpSharp’s VerifyTotp function.

If the verification is successful, we will be enabling the Two-factor authentication for the user in our in-memory backend using the enableTwoFactorAuth function and then redirect to the redirect path which in this case the Profile page.

1
2
3
4
5
6
7
8
// Suave.TwoFactorAuth/Web.fs
// ...

let app =
  choose [
    // ...
    googleAuthenticatorWebPart profilePath notFoundPath
  ]

Login With Two-factor Authentication

The last step is to prompt for the verification code whenever the Two-factor Authentication enabled user tries to log in and verify the one-time verification code before granting the access.

Let’s start by defining the liquid view template auth_code.liquid for getting the one-time verification code.

1
2
3
4
5
6
7
8
// Suave.TwoFactorAuth/GoogleAuthenticator.fs
// ...
let authCodePath = "/authcode"

let googleAuthenticatorWebPart redirectPath notFoundPath =
  choose [
    // ...
    path authCodePath >=> page "auth_code.liquid" ""]

After username and password verification, the user has to be redirected the auth_code page

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
// Suave.TwoFactorAuth/Login.fs
// ...
let authCodeSessionKey = "loginUser"

let onLogin redirectPath authCodePath (request : HttpRequest) =
  match request.["Username"], request.["Password"] with
  | Some username, Some password ->
    match getUser username with
    | Some user ->
      match user.Password = password, user.TwoFactorAuthentication with
      | true, Disabled -> loginSucess never redirectPath username
      | true, Enabled _ ->
          sessionSet never authCodeSessionKey username
            >=> FOUND authCodePath
      // ...

// ...

let loginWebPart redirectPath authCodePath =
  path loginPath >=> choose [
      // ...
      POST >=> request (onLogin redirectPath authCodePath)]
1
2
3
4
5
6
7
// Suave.TwoFactorAuth/Web.fs
// ...
let app =
  choose [
    loginWebPart profilePath authCodePath
    // ...
  ]

We are using a separate session key authCodeSessionKey to hold the username of the user.

The final step is verifying the one-time verification code (from the Google Authenticator app) entered by the user.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// Suave.TwoFactorAuth/GoogleAuthenticator.fs
// ...
open Combinators
// ...
let onAuthCodeVerification redirectPath (request : HttpRequest) username =
  match request.["Code"], getUser username with
  | Some code, Some user ->
    match user.TwoFactorAuthentication with
    | Enabled secretKey ->
      match verifyOtp secretKey code with
      | true -> loginSucess never redirectPath user.Username
      | _ -> redirectToLogin (Some "invalid otp")
    | _ -> redirectToLogin (Some "invalid request")
  | _ -> redirectToLogin (Some "invalid request")

let onVerifyAuthCode redirectPath httpRequest =
  let onFail = redirectToLogin (Some "invalid request")
  let onAuthCodeVerification =
    onAuthCodeVerification redirectPath httpRequest
  sessionGet onFail authCodeSessionKey onAuthCodeVerification

let googleAuthenticatorWebPart redirectPath notFoundPath =
  choose [
    // ...
    path "/verify_auth_code" >=> request (onVerifyAuthCode redirectPath)]

That’s it! We have successfully implemented Two-factor authentication.

The complete source code is available in my GitHub repository

Property-based testing is one of the powerful technique to unit test the code. Unlike the example-based testing (where we Arrange a set of example inputs to Assert the system under test), Property based testing enable us to focus on what we wanted to test and liberate us from doing mundane work on coming up with sample inputs.

Recently, I got a chance to use Property based testing in an open source library that I am developing to use Suave in Azure Functions. I felt very productive and it was a such a joy to write the unit tests.

In this blog post, you are going to learn how Property-based testing has helped me and why you need to consider using(or learning) it

Use Case

Let me start with the use case that I wanted to test. The library that I am working is an adapter between two HTTP protocol abstractions, System.Net.Http and Suave.Http

The first task is to map HttpStatusCode of System.Net to the HttpCode of Suave.Http and the next task is doing the vice-versa.

HttpStatusCode is an enum type and HttpCode is a discriminated union type. Refer this blog post to know more about this difference.

Identifying the Property

The first step is Property-based testing is identifying the property. In other words, it forces to think about the relationship between the input and the output.

I believe this thinking will make a huge difference in the quality of the software that we deliver.

In the first task that we are about to implement, the relationship is the integer representation of the HTTP status code is both HttpStatusCode and HttpCode.

Programmatically, this can be asserted like

1
2
3
// boolean condition
// In F# (=) operator represents the equality checking
LanguagePrimitives.EnumToValue httpStatusCode = httpCode.code

The EnumToValue returns the integer value associated with the enum, which is nothing but the integer representation of the HTTP status code it represents. The code is a member of HttpCode that represents the same integer.

If this property holds true for given set of inputs, then we can assert that the function that does the transformation is working correctly.

Implementing the mapping function httpStatusCode

The initial implementation that I had in my mind was

1
2
3
4
5
6
7
let NetStatusCode = function
| HttpCode.HTTP_200 -> HttpStatusCode.OK
| HttpCode.HTTP_201 -> HttpStatusCode.Created
| HttpCode.HTTP_400 -> HttpStatusCode.BadRequest
| HttpCode.HTTP_404 -> HttpStatusCode.NotFound
| HttpCode.HTTP_202 -> HttpStatusCode.Accepted
| // ...

If I haven’t choosen to use Property based testing, I might have ended up with this line by line mapping for all the HTTP status codes. As a matter of fact, in my last blog post on using Suave in Azure Functions, I’ve used this same approach.

While thinking regarding properties to assert the transformation, for the first time I came to know about the Language Primitives module in F# and the EnumToValue function.

There should be an another function EnumOfValue right?

Yes!

Let’s use this in the httpStatusCode function implementation

1
2
let httpStatusCode (httpCode : HttpCode) : HttpStatusCode =
  LanguagePrimitives.EnumOfValue httpCode.code

Short and Sweet!

Property-based testing made my day and helped me to save a lot of keystrokes!

Writing Our First Property based Unit Test

In F# we can use the FsCheck library to write the Property based unit tests. For this implementation, we will be using FsCheck.Xunit to run Property tests in XUnit

1
2
3
4
5
6
7
8
9
10
11
open Suave.Http
open System.Net
open FsCheck
open FsCheck.Xunit

[<Property>]
let ``httpStatusCode maps Suave's HttpCode to System.Net's HttpStatusCode ``
  (httpCode : HttpCode) =

    let httpStatusCode = Response.httpStatusCode httpCode
    LanguagePrimitives.EnumToValue httpStatusCode = httpCode.code

The magic here is the Property attribute. It makes our life easier by auto-populating the parameter httpCode with the all the possible values and runs the unit test against each value.

As a convention, we need to return boolean by validating the property that we wanted to assert.

There is no Arrange step to setup the input parameter and FsCheck does that for you with 100% test coverage :-)

Mapping HttpCode to HttpStatusCode

The next task in the use case is doing the reverse, mapping HttpStatusCode to HttpCode.

Let’s start with the test first.

The integer representation of HTTP status code property that we used on the previous task holds true for this mapping also.

1
2
3
4
5
6
7
[<Property>]
let ``suaveHttpCode maps System.Net's HttpStatusCode to Suave's HttpCode if exists``
  (httpStatusCode : HttpStatusCode) =
    let httpCode = suaveHttpCode httpStatusCode
    let code = LanguagePrimitives.EnumToValue httpStatusCode

    Option.isSome httpCode && httpCode.Value.code = code

The suaveHttpCode function returns an Option type as Suave.Http.HttpCode’s tryParse function parses the integer HTTP status code and returns the corresponding HttpCode if the parsing succeeds.

The implementation of suaveHttpCode function is

1
2
3
4
5
6
7
let suaveHttpCode (httpStatusCode : HttpStatusCode) =
  let code =
    LanguagePrimitives.EnumToValue httpStatusCode
    |> HttpCode.tryParse
  match code with
  | Choice1Of2 httpCode -> Some httpCode
  | _ -> None

Now if we run the unit tests, You will be getting the following error (if you are using Suave NuGet package version <= 1.1.3)

1
2
3
4
5
Suave.Azure.Functions.Tests.suaveHttpCode maps System.Net's HttpStatusCode to Suave's HttpCode [FAIL]
FsCheck.Xunit.PropertyFailedException :
     Falsifiable, after 7 tests (0 shrinks) (StdGen (1629668181,296211181)):
     Original:
     Unused

The reason for this failure is Suave (up to v1.1.3) doesn’t have the HTTP status code 306 (UnUsed). The unit test that we wrote was exercised by the FsCheck for the all possible values of the HttpStatusCode enum till it encountered this failure case.

Let’s patch our unit test to fix this failure.

1
2
3
4
5
6
7
8
let httpCode = Response.suaveHttpCode httpStatusCode
  let code = LanguagePrimitives.EnumToValue httpStatusCode
  let unSupportedSuaveHttpCodes = [306]

  if List.contains code unSupportedSuaveHttpCodes then
    Option.isNone httpCode
  else
    Option.isSome httpCode && httpCode.Value.code = code

If you run the unit test after this, you will get an another failure

1
2
3
4
5
Suave.Azure.Functions.Tests.suaveHttpCode maps System.Net's HttpStatusCode to Suave's HttpCode [FAIL]
FsCheck.Xunit.PropertyFailedException :
  Falsifiable, after 10 tests (0 shrinks) (StdGen (507658935,296211184)):
  Original:
  UpgradeRequired

Again, The reason is HTTP Status Code 426 (UpgradeRequired) is not supported is Suave at this point of writing. Let’s patch this too by adding this to the list unSupportedSuaveHttpCodes.

1
2
3
// ...
let unSupportedSuaveHttpCodes = [306;426]
// ...

After this FsCheck is happy and we too.

If I haven’t used Property based testing, for sure, I might have missed these two unsupported HTTP status codes.

The Spirit of Open Source

While looking at Suave.Http codebase to fix the unit test failures, this is what I saw in the code

This is the beauty of open source, and I am glad to do it

Summary

Recently Microsoft Azure has made F# as a first-class citizen to write Azure Functions. As F# is a functional-first programming language, I feel Azure Functions and F# would be a match made in heaven.

In this blog post, you are going to experience a scaled up version of Azure Functions in F# using Suave

What’s in the Function Signatures?

In a functional programming language, we define small functions that do one thing well and then we compose them together to represent the solution. To compose functions, we need to be thoughtful while designing the signature of a function.

Let’s see the signature of an Azure Function in F#

1
2
3
// HttpRequestMessage -> HttpResponseMessage
let Run(req: HttpRequestMessage) =
  new HttpResponseMessage(HttpStatusCode.OK)

The Run function takes a HttpRequestMessage and returns the HttpResponseMessage. This signature is simple, but it has a limitation. The limitation has been showcased in the templates directory of Azure Webjobs SDK

My each C, R, U, D are in different functions. Well, there is nothing wrong here. These templates are suitable for getting started in Azure Functions. But what will you do if you have a requirement to expose CRUD of a resource as an Azure Functions?

One option is to define each part of the CRUD as separate Azure Functions (as defined by the templates). If you choose to go by this, you will have four different endpoints and I am sure your client code will have a hard time to consume these endpoints. In addition to this, you will also need to manage four things to satisfy your one requirement.

The other option is putting the CRUD inside a single function.

1
2
3
4
5
6
7
8
9
10
11
let Run (req:HttpRequestMessage) =
    if req.Method = HttpMethod.Get then
        // ...
    else if req.Method = HttpMethod.Post then
        // ...
    else if req.Method = HttpMethod.Put then
        // ...
    else if req.Method = HttpMethod.Delete then
        // ...
    else
        // ...

Though this approach solves the problem, it comes with another set of challenges. In Object Oriented Programming, we typically use Polymorphism to replace the conditional logic.

Revisiting Function Signature

A request handler looks for some condition to be meet in the incoming HTTP request, and if the predicate succeeds, it modifies the HTTP response.

The signature of the Run function, HttpRequestMessage -> HttpResponseMessage is not completely reflecting the above specification.

Let’s have a look at the limitations of this signature

  • The Run function doesn’t return the HttpRequestMessage. So if we have multiple handlers we are constrained to use either if else if or Polymorphism.

  • It doesn’t represent a handler that doesn’t handle the HTTP request. If the HTTP request is GET, the handler for HTTP POST will not modify the HttpResponseMessage

The better signature would have the following to describe a handler in a better way

  • The handler has to be pure function that takes both Request and Response as it’s parameters

  • If the handler is not handling the HTTP request, it has to return the unmodified Request and Response along with an indicator saying that it didn’t handle the request.

It’s where the Suave library shines. Suave defines a type called WebPart with the signature to model the handler with the above-said expectations.

1
2
3
4
5
6
7
type HttpContext = {
  request    : HttpRequest
  response   : HttpResult
  // ...
}

type WebPart = HttpContext -> Async<HttpContext option>

The Async represents that the WebPart function is a non-blocking asynchronous function and option type models the WebPart which doesn’t handle the HTTP request

The real power of Suave is its set of combinators to manipulate route flow and task composition. You can define an API in Suave that only handles HTTP POST requests and returns Hello as text without typing too much.

1
2
// HttpContext -> Async<HttpContext option>
let app = POST >=> OK "HelloSystem.Net.Http

To learn more about the Suave combinators refer my blog post on Building REST API in suave

If you notice the binding app itself is a WebPart (which in turn a function) with the signature HttpContext -> Async<HttpContext option>. So, you can call this function in your application code and project the output of the function to any output medium that you wish.

The Difference

The Azure Functions do an incredible job in helping you to define a part of your system as a function. Suave takes it to the next level by helping you to define your system as function.

In nutshell, Suave complements Azure Functions and helps you to define your system as a Serverless Function

Creating a Suave Adapter

So, to scale up Azure Functions using Suave, all we need is an adapter.

The adapter does the following

  • Transforms HttpRequestMessage from System.Net.Http to HttpRequest of Suave.Http

  • Then create an empty Suave’s HttpContext with the above HttpRequest and call the WebPart (that represents your system).

  • The final step is converting the HttpResult of Suave.Http to HttpResponseMessage of System.Net.Http.

Let’s start from HttpRequestMessage

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
// SuaveAdapter.fsx
let SuaveHttpMethod (httpMethod : System.Net.Http.HttpMethod) =
  match httpMethod.Method with
  | "GET" -> HttpMethod.GET
  | "POST" -> HttpMethod.POST
  | "PUT" -> HttpMethod.PUT
  | "DELETE" -> HttpMethod.DELETE
  | x -> HttpMethod.OTHER x

let SuaveHeaders (headers : HttpRequestHeaders) =
  headers
  |> Seq.map (fun h -> (h.Key, h.Value |> Seq.head))
  |> Seq.toList

let SuaveRawForm (content : System.Net.Http.HttpContent) = async {
  let! content = content.ReadAsByteArrayAsync() |> Async.AwaitTask
  return content
}

let SuaveRawQuery (requestUri : System.Uri) =
  if requestUri.Query.Length > 1 then
    requestUri.Query.Substring(1)
  else
      ""

let NetHeaderValue (headers : HttpRequestHeaders) key =
    headers
    |> Seq.tryFind (fun h -> h.Key = key)
    |> Option.map (fun h -> h.Value |> Seq.head)

let SuaveRequest (req : HttpRequestMessage) = async {
  let! content = SuaveRawForm req.Content
  let host = defaultArg (NetHeaderValue req.Headers "Host") ""
  return {HttpRequest.empty with
            url = req.RequestUri
            ``method`` = SuaveHttpMethod req.Method
            headers = SuaveHeaders req.Headers
            rawForm = content
            rawQuery = SuaveRawQuery req.RequestUri
            host = host}
}

As a convention, I’ve used Net and Suave prefixes in the function name to represent the returning type of System.Net.Http and Suave.Http respectively.

I hope that these functions are self-explanatory, so let’s move on the next step.

To keep it simple, I’ve ignored other HTTP Methods like PATCH, HEAD, etc.

The next step is creating Suave HttpContext

1
2
3
4
let SuaveContext httpRequest = async {
  let! suaveReq = SuaveRequest httpRequest
  return { HttpContext.empty with request = suaveReq}
}

Then we need to convert HttpResult to HttpResponseMessage

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
let NetStatusCode = function
| HttpCode.HTTP_200 -> HttpStatusCode.OK
| HttpCode.HTTP_201 -> HttpStatusCode.Created
| HttpCode.HTTP_400 -> HttpStatusCode.BadRequest
| HttpCode.HTTP_404 -> HttpStatusCode.NotFound
| HttpCode.HTTP_202 -> HttpStatusCode.Accepted
| _ -> HttpStatusCode.Ambiguous

let NetHttpResponseMessage httpResult =
  let content = function
  | Bytes c -> c
  | _ -> Array.empty
  let res = new HttpResponseMessage()
  let content = new ByteArrayContent(content httpResult.content)
  httpResult.headers |> List.iter content.Headers.Add
  res.Content <- content
  res.StatusCode <- NetStatusCode httpResult.status
  res

To keep it simple, I’ve ignored other HTTP StatusCodes

The final step is putting these functions together and run the WebPart function with the translated HttpContext.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
let SuaveRunAsync app suaveContext = async {
  let! res = app suaveContext
  match res with
  | Some ctx ->
    return (NetHttpResponseMessage ctx.response, ctx)
  | _ ->
    let res = new HttpResponseMessage()
    res.Content <- new ByteArrayContent(Array.empty)
    res.StatusCode <- HttpStatusCode.NotFound
    return res,suaveContext
}

let RunWebPartAsync app httpRequest = async {
  let! suaveContext = SuaveContext httpRequest
  return! SuaveRunAsync app suaveContext
}

Suave Adapter In Action

Let’s see the Suave Adapter that we created in action.

As already there are two great blog posts by Greg Shackles and Michał Niegrzybowski, I am diving directly into Azure functions in F#.

Let me create a new Azure Function application in Azure with the name “TamAzureFun” and then define the first function HelloSuave.

The function.json of HelloSuave has to be updated with the methods property to support different HTTP request methods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
    "disabled": false,
    "bindings": [{
      "type": "httpTrigger",
      "name": "req",
      "methods": ["get","put","post","delete"],
      "authLevel": "anonymous",
      "direction": "in"
    },{
      "type": "http",
      "name": "res",
      "direction": "out"
    }]
}

Then add the Suave dependency in project.json

1
2
3
4
5
6
7
8
9
{
  "frameworks": {
    "net46": {
      "dependencies": {
        "Suave": "1.1.3"
      }
    }
  }
}

Let’s start simply by defining small API (system) that handles different types of HTTP methods.

1
2
3
4
5
6
7
8
9
10
11
12
// app.fsx
open Suave
open Suave.Successful
open Suave.Operators
open Suave.Filters

let app =
  choose [
      GET >=> OK "GET test"
      POST >=> OK "POST test"
      PUT >=> OK "PUT test"
      DELETE >=> OK "DELETE test"]

The final step is referring the SuaveAdapter.fsx & app.fsx files in the run.fsx and have fun!

1
2
3
4
5
6
7
8
9
10
11
// run.fsx
#load "SuaveAdapter.fsx"
#load "app.fsx"
open SuaveAdapter
open App
open System.Net.Http
open Suave

let Run (req : HttpRequestMessage) =
  let res, _ = RunWebPartAsync app req |> Async.RunSynchronously
  res

Let’s make some HTTP requests to test our implementation.

Suave is rocking!

Creating a REST API in Azure Functions

We can extend the above example to expose a REST end point!

In Suave a REST API is a function.

Create a new Azure Function HelloREST and add NewtonSoft.Json & Suave dependencies in project.json

1
2
3
4
5
6
7
8
9
10
{
  "frameworks": {
    "net46": {
      "dependencies": {
        "Suave": "1.1.3",
        "NewtonSoft.Json" : "9.0.1"
      }
    }
  }
}

To handle JSON requests and responses, let’s add some combinators

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
// Suave.Newtonsoft.Json.fsx
open Newtonsoft.Json
open Newtonsoft.Json.Serialization
open System.Text
open Suave.Json
open Suave.Http
open Suave
open Suave.Operators

let toJson<'T> (o: 'T) =
  let settings = new JsonSerializerSettings()
  settings.ContractResolver <- new CamelCasePropertyNamesContractResolver()
  JsonConvert.SerializeObject(o, settings)
  |> Encoding.UTF8.GetBytes

let fromJson<'T> (bytes : byte []) =
  let json = Encoding.UTF8.GetString bytes
  JsonConvert.DeserializeObject(json, typeof<'T>) :?> 'T

let mapJsonWith<'TIn, 'TOut>
  (deserializer:byte[] -> 'TIn) (serializer:'TOut->byte[]) webpart f =
  request(fun r ->
    f (deserializer r.rawForm)
    |> serializer
    |> webpart
    >=> Writers.setMimeType "application/json")

let MapJson<'T1,'T2> webpart =
  mapJsonWith<'T1,'T2> fromJson toJson webpart

let ToJson webpart x  =
  toJson x |> webpart >=> Writers.setMimeType "application/json"

Then define the REST api in app.fsx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
#load "Suave.Newtonsoft.Json.fsx"
open Suave
open Suave.Successful
open Suave.Operators
open Suave.Filters
open System
open Suave.Newtonsoft.Json

type Person = {
  Id : Guid
  Name : string
  Email : string
}

let createPerson person =
  let newPerson = {person with Id = Guid.NewGuid()}
  newPerson

let getPeople () = [
  {Id = Guid.NewGuid(); Name = "john"; Email = "j@g.co"}
  {Id = Guid.NewGuid(); Name = "mark"; Email = "m@g.co"}]

let getPersonById id =
  {Id = Guid.Parse(id); Name = "john"; Email = "j@g.co"}
  |> ToJson ok

let deletePersonById id =
  sprintf "person %s deleted" id |> OK

let app =
  choose [
    path "/people" >=> choose [
      POST >=> MapJson created createPerson
      GET >=> ToJson ok (getPeople ())
      PUT >=> MapJson accepted id
    ]
    GET >=> pathScan "/people/%s" getPersonById
    DELETE >=> pathScan "/people/%s" deletePersonById
  ]

To keep things simple, I am hard coding the values here. It can easily be extended to talk to any data source

Our SuaveAdapter has capable of handling different HTTP methods and but it hasn’t been programmed to deal with different paths.

Here in this example we need to support two separate paths

1
2
GET /people
GET /people/feafa5b5-304d-455e-b7e7-13a5b3293f77

The HTTP endpoint to call an Azure function has the format

1
https://{azure-function-app-name}.azurewebsites.net/api/{function-name}

At this point of writing it doesn’t support multiple paths. So, we need to find a workaround to do it.

One way achieving this is to pass the paths as a Header. Let’s name the Header key as X-Suave-URL. Upon receiving the request we can rewrite the URL as

1
https://{azure-function-app-name}.azurewebsites.net/{header-value-of-X-Suave-URL}

Let’s update SuaveAdapter.fsx to do this

1
2
3
4
5
6
7
8
9
10
11
12
13
let RunWebPartWithPathAsync app httpRequest = async {
  let! suaveContext = SuaveContext httpRequest
  let netHeaderValue = NetHeaderValue httpRequest.Headers
  match netHeaderValue "X-Suave-URL", netHeaderValue "Host"  with
  | Some suaveUrl, Some host ->
    let url = sprintf "https://%s%s" host suaveUrl |> System.Uri
    let ctx = {suaveContext with
                request = {suaveContext.request with
                            url = url
                            rawQuery = SuaveRawQuery url}}
    return! SuaveRunAsync app ctx
  | _ -> return! SuaveRunAsync app suaveContext
}

The final step is updating the run.fsx file to use this new function

1
2
3
4
5
6
7
8
9
10
#load "SuaveAdapter.fsx"
#load "app.fsx"
open SuaveAdapter
open App
open System.Net.Http
open Suave

let Run (req : HttpRequestMessage) =
  let res, _ = RunWebPartWithPathAsync app req |> Async.RunSynchronously
  res

Serverless REST API in Action

This blog post is a proof of concept to use Suave in Azure Functions. There are a lot of improvements to be made to make it production ready. I am planning to publish this as a NuGet package based on the feedback from the community.

Update : Suave.Azure.Functions is available now as a Nuget Package

Summary

The complete source code is available in my GitHub repository.

Are you interested in learning more about F#?

I’m delighted to share that I’m running a tutorial at Progressive F# Tutorials 2016, London on Dec 5, 2016. I’m excited to share my experiences with Suave and help developers to understand this wonderful F# library.

The Progressive F# Tutorials offer hands-on learning for every skill set and is led by some of the best experts in F# and functional programming