In my previous blog post, we have seen how to use the create and find function in gorm along with use-case driven approach. In this blog post, we will be extending our blogging platform gomidway by implementing an another interesting use case.

Use Case #3 - Publishing a new blog post

Publishing a new blog post use case involves the following

• A user can publish a new blog post by providing a title, body of the post and a set of tags.

• If the title already exists we need to let the user know about it.

• We also need to let him know if publish went well.

Though the requirement looks simple on paper, there are some complexities regarding organizing the code and orchestrating the entire operation.

Let’s dig up and see how we can solve it!

The Database Schema

As a first step, let’s add some new tables to our existing schema which has only the users table.

The first one is the posts table

1
2
3
4
5
6
7

CREATE TABLE posts(
  id SERIAL PRIMARY KEY,
  title VARCHAR(50) UNIQUE NOT NULL,
  body TEXT NOT NULL,
  published_at TIMESTAMP NOT NULL,
  author_id INTEGER NOT NULL REFERENCES users(id)
);

Then we need to have an another table for tags

1
2
3
4

CREATE TABLE tags(
  id SERIAL PRIMARY KEY,
  name VARCHAR(50) NOT NULL
);

And finally, a bridge table to associate posts and tags

1
2
3
4

CREATE TABLE posts_tags(
  tag_id INTEGER REFERENCES tags(id),
  post_id INTEGER REFERENCES posts(id)
);

Note: a blog post can have multiple tags, and a tag can have multiple blog posts associated with it

Model Definitions - Post and Tag

The next step is defining equivalent models for Post and Tag in golang.

Let’s create a new folder called tag and define the Tag model.

1
2
3
4
5
6
7

// tag/model.go
package tag

type Tag struct {
  ID   uint
  Name string
}

Then define the Post model in an another new folder post

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// post/model.go
package post

import (
  "time"

  "github.com/tamizhvendan/gomidway/tag"
)

const (
  UniqueConstraintTitle = "posts_title_key"
)

type Post struct {
  ID          uint
  Title       string
  Body        string
  AuthorID    uint
  Tags        []tag.Tag `gorm:"many2many:posts_tags;"`
  PublishedAt time.Time
}

type TitleDuplicateError struct{}

func (e *TitleDuplicateError) Error() string {
  return "title already exists"
}

As we have seen in the previous blog post, the constant UniqueConstraintTitle and the TitleDuplicateError are required for doing unique constraint violation error check on the title column and to communicate it to the application respectively.

The important thing to notice in this model definition is the Tags field

1
2
3
4
5

type Post struct {
  // ...
  Tags        []tag.Tag `gorm:"many2many:posts_tags;"`
  // ..
}

The Tags field has a golang struct tag gorm defining the association type many2many and the name of the bridge table posts_tags.

Implementing new blog post use case

Now we have all the models required to enable publishing a new blog post, and it’s time to have to go at its implementation.

Let’s start by creating a new folder publish under post and add the scaffolding for the handler

As we have seen earlier, the folder structure represent the use case, and the handler orchestrate the use case

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

// post/publish/handler.go
package publish

import (
  "github.com/jinzhu/gorm"
)

type Request struct {
  Title    string
  Body     string
  AuthorID uint
  Tags     []string
}

type Response struct {
  PostId uint
}

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // TODO
}

The implementation of publishing a new post involves the following steps

1. For all the Tags that are part of the request, we need to create an entry in the tags table. If it is already there, we don’t need to create.

2. The next step is creating a new entry in the posts table with the given details.

3. Finally, we need to associate the Tags with the newly created Post via the bridge table.

Since it involves multiple inserts on the database side, all the three steps should happen inside a transaction.

The first two steps can be executed in any order as they are independent of each other

Code Organization (aka Responsibility Separation)

We discussed a little bit about code organization in the last blog post. One important thing which can help us, in the long run is having proper separation of concern in the code base.

There are multiple ways we can separate the concern. In our case, we are organizing by use cases with handler driving the implementation. The handler may access the data layer if the use case requires.

To keep things simple, we are not discussing dependency injection in handler and data layer interaction here. I am planning to cover this in my future blog posts.

Back to our business, the data access logic of the three steps will be in their respective packages and the publish handler coordinate the entire use case logic.

Step 1: Create a Tag if not exists

The first step is creating a tag if it is not there in the database. For the both new tags and the existing tags we need to get its id from the database to associate it with the posts.

Gorm has a method called FirstOrCreate to help us to implement this step.

1
2
3
4
5
6
7
8
9
10
11
12
13

// tag/create.go
package tag

import "github.com/jinzhu/gorm"

func CreateIfNotExists(db *gorm.DB, tagName string) (*Tag, error) {
  var tag Tag
  res := db.FirstOrCreate(&tag, Tag{Name: tagName})
  if res.Error != nil {
    return nil, res.Error
  }
  return &tag, nil
}

The FirstOrCreate method populates the Id field of the tag.

Step 2: Creating a new Post

It is similar to creating a new user that we saw in the last blog post

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

// post/create.go
package post

import (
  "github.com/jinzhu/gorm"
  "github.com/tamizhvendan/gomidway/postgres"
)

func Create(db *gorm.DB, post *Post) (uint, error) {
  res := db.Create(post)
  if res.Error != nil {
    if postgres.IsUniqueConstraintError(res.Error, UniqueConstraintTitle) {
      return 0, &TitleDuplicateError{}
    }
    return 0, res.Error
  }
  return post.ID, nil
}

Step 3: Associating Tag with Post

The final step is associating the tag with the post in the database. Gorm has a decent support for Associations. The one that we needed from gorm to carry out the current step is its Append method.

Let’s define a constant in Post model which holds the AssociationTag

1
2
3
4
5
6
7

// post/model.go
// ...
const (
  // ...
  AssociationTags = "Tags"
)
// ...

Then add a new file tag.go in the post folder and implement the third step as below

1
2
3
4
5
6
7
8
9
10
11
12

// post/tag.go
package post

import (
	"github.com/jinzhu/gorm"
	"github.com/tamizhvendan/gomidway/tag"
)

func AddTag(db *gorm.DB, post *Post, tag *tag.Tag) error {
  res := db.Model(post).Association(AssociationTags).Append(tag)
  return res.Error
}

Publishing New Blog Post

Now we have all the individual database layer functions ready for all the three steps, and it’s time to focus on the implementation of publishing a new blog post.

We already have the scaffolding in place

1
2
3
4
5

// publish/handler.go
// ...
func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // TODO
}

As a first step, let’s begin a new transaction using gorm’s Begin method.

1
2
3
4
5
6
7

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  tx := db.Begin()
  if tx.Error != nil {
    return nil, tx.Error
  }
  // TODO
}

Then call the Create function in post package to create a new blog post

1
2
3
4
5
6
7
8
9
10
11
12
13
14

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  newPost := &post.Post{
    AuthorID:    req.AuthorId,
    Title:       req.Title,
    Body:        req.Body,
    PublishedAt: time.Now().UTC(),
  }
  _, err := post.Create(tx, newPost)
  if err != nil {
    return nil, err
  }
  // TODO
}

Then for all the tags in the request, call the CreateIfNotExists function in tag package to get its respective Ids and associate it with the newly created post using the AddTag function in the post package.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  for _, tagName := range req.Tags {
    t, err := tag.CreateIfNotExists(tx, tagName)
    if err != nil {
      tx.Rollback()
      return nil, err
    }
    err = post.AddTag(tx, newPost, t)
    if err != nil {
      tx.Rollback()
      return nil, err
    }
  }
  // TODO
}

A thing to note here is we are rolling back the transaction using the RollBack method in case of error.

The final step is committing the transaction and returning the newly created post id as response

1
2
3
4
5
6
7
8

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  res := tx.Commit()
  if res.Error != nil {
    return nil, res.Error
  }
  return &Response{PostId: newPost.ID}, nil
}

Test Driving Publish New Blog Post

Let’s test drive our implementation from the main function with some hard coded value

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

// main.go
package main

import (
  // ...
  "github.com/tamizhvendan/gomidway/post"
  "github.com/tamizhvendan/gomidway/post/publish"
)
// ...
func main() {
  // ...
  publishPost(db)
}


func publishPost(db *gorm.DB) {
  res, err := publish.NewPost(db, &publish.Request{
    AuthorId: 1,
    Body:     "Golang rocks!",
    Title:    "My first gomidway post",
    Tags:     []string{"intro", "golang"},
  })
  if err != nil {
    if _, ok := err.(*post.TitleDuplicateError); ok {
      fmt.Println("Bad Request: ", err.Error())
      return
    }
    fmt.Println("Internal Server Error: ", err.Error())
    return
  }
  fmt.Println("Created: ", res.PostId)
}

if we run the program with these hard coded values, we will get the following output

1

Created: 1

if we rerun the program without changing anything, we will get the bad request error as expected

1

Bad Request: title already exists

Summary

In this blog post, we have seen how to perform, create operation of a model having many to many relationship using gorm. The source code is available in my GitHub repository.

We have been using gorm as a primary tool to interact with PostgreSQL from golang for almost a year now.

Gorm does an excellent job as an ORM library, and we enjoyed using it in our projects.

Through this blog post series, I will be sharing our experiences on how we leveraged gorm to solve our client’s needs.

The Domain

In this blog post series, we will be implementing the data persistence side of a blogging platform similar to medium, called gomidway.

Part-1 Introduction

In this blog post, we will be working on defining the backend for signing up a new user and providing a provision for user login.

The User Model

Let’s start our modeling from the User table.

1
2
3
4
5
6

CREATE TABLE users(
  id SERIAL PRIMARY KEY,
  username VARCHAR(50) UNIQUE NOT NULL,
  email VARCHAR(255) UNIQUE NOT NULL,
  password_hash TEXT NOT NULL
);

Then define an equivalent model in golang.

1
2
3
4
5
6

type User struct {
  ID           uint `gorm:"primary_key"`
  Username     string
  Email        string
  PasswordHash string
}

Before taking the next steps, let me spend some time on explaining where to put this User struct

The Folder Structure

There are two common ways to organize the models in golang.

One approach is defining a folder called models and put all the models here

The another approach is an invert of this structure. In this design, we will have a separate folder for each model.

Both the approaches have pros and cons. Choosing one over the other is entirely opinionated, and my preference is the second one.

IMHO, the folder structure has to represent the domain and the code associated a domain model should coexist with proper separation of concern.

Software architectures are structures that support the use cases of the system - Ivar Jacobson

In this gorm blog post series, I will be following the domain based folder structure.

Use Case #1 - User Signup

The Signup use case of a user is defined as

• A user should sign up himself by providing his email, username, and password
• If the username or the email already exists, we need to let him now
• We also need to let him know if there is any error while persisting his signup details
• If signup succeeds, he should be getting a unique identifier in the system

To keep things simple and the focus of this series is on the data persistence side, we are not going to discuss/implement the HTTP portion of the application. Instead, we will be driving our implementation with some hard code values during the application bootstrap.

Defining the Signup handler

Like many terms in software engineering, the term handler has different meanings. So, let me start by explaining what I mean by a handler here.

A handler is a function that represents an use-case. It takes its dependency(ies) and its input(s) as parameters and returns the outcome of the use case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

// user/signup/handler.go
package signup

type Request struct {
  Username string
  Email    string
  Password string
}

type Response struct {
  Id uint
}

func Signup(db *gorm.DB, req *Request) (*Response, error) {
  // ...
}

One important thing to notice here is, the Request and the Response are golang structs. How the request is being populated from user’s request (JSON Post / HTML form Post / command from a message queue) and how the response communicated to the user (JSON response / HTML view / event in the message queue) are left up to the application boundary.

In the Signup function, as a first step, we need to create the hash for the password using bcrypt and then create the new user.

1
2
3
4
5
6
7
8
9
10
11
12

func Signup(db *gorm.DB, req *Request) (*Response, error) {
  passwordHash, err := bcrypt.GenerateFromPassword([]byte(req.Password), bcrypt.DefaultCost)
  if err != nil {
    return nil, err
  }
  newUser := &user.User{
    Username:     req.Username,
    Email:        req.Email,
    PasswordHash: string(passwordHash),
  }
  // ????
}

The next step is persisting this newly created user

Adding User Create function

The create user function is straight forward. We just need to call the Create method in gorm

1
2
3
4
5
6
7
8
9
10

// user/create.go
package user

func Create(db *gorm.DB, user *User) (uint, error) {
  err := db.Create(user).Error
  if err != nil {
    return 0, err
  }
  return user.ID, nil
}

But the work is not done yet!

As per our use case, We need to let the handler to know if the username or the email already exists.

We already have unique constraints in place in the users table.

1
2
3
4
5
6
7
8
9
10
11
12

gomidway=# \d users
                                    Table "public.users"
    Column     |          Type          |                     Modifiers
---------------+------------------------+----------------------------------------------------
 id            | integer                | not null default nextval('users_id_seq'::regclass)
 username      | character varying(50)  | not null
 email         | character varying(255) | not null
 password_hash | text                   | not null
Indexes:
    "users_pkey" PRIMARY KEY, btree (id)
    "users_email_key" UNIQUE CONSTRAINT, btree (email)
    "users_username_key" UNIQUE CONSTRAINT, btree (username)

So, the Create method in gorm would return an error of type pq Error with the ErrorCode as "23505" for unique_violation, and the Constraint field will be having the unique constraint key name users_email_key and users_username_key for email and username duplicate error respectively.

Though this error does communicate what we wanted, it is very generic and what we want is something concrete to our use case.

To make it happen, let’s create a new folder postgres (aka package) and write a utility function IsUniqueConstraintError which checks whether the given error is a unique constraint error or not.

1
2
3
4
5
6
7

// postgres/pq.go
func IsUniqueConstraintError(err error, constraintName string) bool {
  if pqErr, ok := err.(*pq.Error); ok {
    return pqErr.Code == "23505" && pqErr.Constraint == constraintName
  }
  return false
}

and then in the model.go, where we have the User model, add the constraint names as constants.

1
2
3
4
5
6
7

// user/model.go
// ...
const (
  UniqueConstraintUsername = "users_username_key"
  UniqueConstraintEmail    = "users_email_key"
)
// ...

Finally, define the custom error types

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

// user/model.go
// ...
type UsernameDuplicateError struct {
  Username string
}

func (e *UsernameDuplicateError) Error() string {
  return fmt.Sprintf("Username '%s' already exists", e.Username)
}

type EmailDuplicateError struct {
  Email string
}

func (e *EmailDuplicateError) Error() string {
  return fmt.Sprintf("Email '%s' already exists", e.Email)
}

With this new helper function, constants and types in places, we can complete the create user function as follows.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

// user/create.go
func Create(db *gorm.DB, user *User) (uint, error) {
  err := db.Create(user).Error
  if err != nil {
    if postgres.IsUniqueConstraintError(err, UniqueConstraintUsername) {
      return 0, &UsernameDuplicateError{Username: user.Username}
    }
    if postgres.IsUniqueConstraintError(err, UniqueConstraintEmail) {
      return 0, &EmailDuplicateError{Email: user.Email}
    }
    return 0, err
  }
  return user.ID, nil
}

On the handler side, we just pass the outcome of this create function to the outside(application boundary) layer

1
2
3
4
5
6
7
8
9
10

// user/signup/handler.go
// ...
func Signup(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  id, err := user.Create(db, newUser)
  if err != nil {
    return nil, err
  }
  return &Response{Id: id}, err
}

Test Driving User Signup

As we discussed earlier, let’s test drive the implementation from the application bootstrap.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

// main.go
func panicOnError(err error) {
  if err != nil {
    panic(err)
  }
}

func main() {
  db, err := gorm.Open("postgres",
    `host=localhost 
      user=postgres password=test
      dbname=gomidway 
      sslmode=disable`)
  panicOnError(err)
  defer db.Close()

  signupUser(db)
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

// main.go
// ...
func signupUser(db *gorm.DB) {
  res, err := signup.Signup(db, &signup.Request{
    Email:    "foo@bar.com",
    Username: "foo",
    Password: "foobar",
  })
  if err != nil {
    switch err.(type) {
    case *user.UsernameDuplicateError:
      fmt.Println("Bad Request: ", err.Error())
      return
    case *user.EmailDuplicateError:
      fmt.Println("Bad Request: ", err.Error())
      return
    default:
      fmt.Println("Internal Server Error: ", err.Error())
      return
    }
  }
  fmt.Println("Created: ", res.Id)
}

if we run the program for the first time after setting up the PostgreSQL database gomidway with the users table, we will get the following output

1

Created:  1

if we rerun the program again, we’ll see the bad request error

1

Bad Request:  Username 'foo' already exists

if we modify the username from foo to bar and run the program, we’ll again get a bad request error for the email address

1

Bad Request:  Email 'foo@bar.com' already exists

That’s it! We have completed the first use case of gomidway!!

Use Case #2 - User Login

The next use case that we are going to implement is user login. The requirement for login has been defined as

• if the user logs in with a different email address or with a different password, we need to show him appropriate errors

• if the email and the password matches, let the user know that he has logged in

Defining the Login handler

As we did for the signup, let’s start our implementation from defining the handler for login

1
2
3
4
5
6
7
8
9
10
11
12
13

// user/login/handler.go
type Request struct {
  Email    string
  Password string
}

type Response struct {
  User *user.User
}

func Login(db *gorm.DB, req *Request) (*Response, error) {
  // ...
}

To implement login, we need some help from the persistence. In other words, we have to find whether the user with the given email address exists in our application.

Adding User FindByEmail function

Let’s create a new file find.go in the user folder and define the FindByEmail function

1
2
3
4
5
6

// user/find.go
func FindByEmail(db *gorm.DB, email string) (*User, error) {
  var user User
  res := db.Find(&user, &User{Email: email})
  return &user, res.Error
}

That’s great. But how are we going to find if the email didn’t exist in the first place?

Thankfully, We don’t need to anything extra other than calling the RecordNotFound to figure this out!

Let’s define a custom error type EmailNotExistsError and return it if no records found.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// user/find.go
type EmailNotExistsError struct{}

func (*EmailNotExistsError) Error() string {
  return "email not exists"
}

func FindByEmail(db *gorm.DB, email string) (*User, error) {
  var user User
  res := db.Find(&user, &User{Email: email})
  if res.RecordNotFound() {
    return nil, &EmailNotExistsError{}
  }
  return &user, nil
}

Now its time to turn our attention to Login handler to wire up the login functionality.

1
2
3
4
5
6
7
8
9

// user/login/handler.go
// ...
func Login(db *gorm.DB, req *Request) (*Response, error) {
  user, err := user.FindByEmail(db, req.Email)
  if err != nil {
    return nil, err
  }
  // ...
}

The next scenario that we need to handle is, compare the password in the request with the password hash. As we need to let the user know in case of password mismatch, let’s create a PasswordMismatchError in the login handler and return it during the mismatch.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// user/login/handler.go
// ...
type PasswordMismatchError struct{}
func (e *PasswordMismatchError) Error() string {
  return "password didn't match"
}

func Login(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  err = bcrypt.CompareHashAndPassword([]byte(user.PasswordHash), []byte(req.Password))
  if err != nil {
    return nil, &PasswordMismatchError{}
  }
  return &Response{User: user}, nil
}

With this, we are done with login handler implementation. Let’s test drive it!

Test Driving User Login

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// main.go
// ...
func main() {
  // ...
  loginUser(db)
}
// ...
func loginUser(db *gorm.DB) {
  res, err := login.Login(db, &login.Request{
    Email: "foo@bar.com",
    Password: "foobar"})
  if err != nil {
    switch err.(type) {
    case *user.EmailNotExistsError:
      fmt.Println("Bad Request: ", err.Error())
      return
    case *login.PasswordMismatchError:
      fmt.Println("Bad Request: ", err.Error())
      return
    default:
      fmt.Println("Internal Server Error: ", err.Error())
      return
    }
  }
  fmt.Printf("Ok: User '%s' logged in", res.User.Username)
}

Summary

In this blog post, we have seen how we can use the create and find function in gorm along with the use case driven approach. The source code can be found in my GitHub repository.

In my previous blog post, we have seen how interfaces in golang can help us to come up with a cleaner design. In this blog post, we are going to see an another interesting use case of applying golang’s interfaces in creating adapters!

Some Context

In my current project, we are using Postgres for persisting the application data. To make our life easier, we are using gorm to talk to Postgres from our golang code. Things were going well and we started rolling out new features without any challenges. One beautiful day, we came across an interesting requirement which gave us a run for the money.

The requirement is to store and retrieve an array of strings from Postgres!

It sounds simple on paper but while implementing it we found that it is not straightforward. Let me explain what the challenge was and how we solved it through a Task list example

The Database Side

Let’s assume that we have database mytasks with a table tasks to keep track of the tasks.

The tasks table has the following schema

1
2
3
4
5
6

CREATE TABLE tasks (
  id SERIAL PRIMARY KEY,
  name TEXT NOT NULL,
  is_completed BOOL NOT NULL,
  tags VARCHAR(10)[]
)

An important thing to note over here is that each task has an array of tags of type varchar(10).

The Golang Side

The equivalent model definition of the tasks table would look like the following in Golang

1
2
3
4
5
6

type Task struct {
  Id          uint
  Name        string
  IsCompleted bool
  Tags        []string
}

The Challenge

Everything is set to test drive the task creation.

Let’s see what happens when we try to create a new task!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

package main
import (
  "fmt"

  "github.com/jinzhu/gorm"
  _ "github.com/jinzhu/gorm/dialects/postgres"
)

func panicOnError(err error) {
  if err != nil {
    panic(err)
  }
}

func CreateTask(db *gorm.DB, name string, tags []string) (uint, error) {
  newTask := &Task{Name: name, Tags: tags}
  result := db.Create(newTask)
  if result.Error != nil {
    return 0, result.Error
  }
  return newTask.Id, nil
}

func main() {
  db, err := gorm.Open("postgres",
    `host=localhost 
      user=postgres password=test
      dbname=mytasks 
      sslmode=disable`)
  panicOnError(err)
  defer db.Close()

  id, err := CreateTask(db, "test 123", []string{"personal", "test"})
  panicOnError(err)
  fmt.Printf("Task %d has been created\n", id)
}

When we run this program, it will panic with the following error message

1

panic: sql: converting Exec argument $3 type: unsupported type []string, a slice of string

As the error message says, the SQL driver doesn’t support []string. From the documentation, we can found that the SQL drivers only support the following values

1
2
3
4
5
6

int64
float64
bool
[]byte
string
time.Time

So, we can’t persist the task with the tags using this approach.

Golang’s interface in Action

As a first step towards the solution, let’s see how the plain SQL insert query provides the value for arrays in Postgres

1

INSERT INTO tasks(name,is_completed,tags) VALUES('buy milk',false,'{"home","delegate"}');

The clue here is the plain SQL expects the value for array as a string with the following format

1

'{ val1 delim val2 delim ... }'

double quotes around element values if they are empty strings, contain curly braces, delimiter characters, double quotes, backslashes, or white space, or match the word NULL. Double quotes and backslashes embedded in element values will be backslash-escaped. - Postgres Documentation

That’s great! All we need to do is convert the []string to string which follows the format specified above.

An easier approach would be changing the Tags field of the Task struct to string and do this conversion somewhere in the application code before persisting the task.

But it’s not a cleaner approach as the resulting code is not semantically correct!

Golang provides a neat solution to this problem through the Valuer interface

Types implementing Valuer interface are able to convert themselves to a driver Value.

That is we need to have a type representing the []string type and implement this interface to do the type conversion.

Like we did in the part-1 of this series, let’s make use of named types by creating a new type called StringSlice

1

type StringSlice []string

Then we need to do the type conversion in the Value method

1
2
3
4
5
6
7
8

func (stringSlice StringSlice) Value() (driver.Value, error) {
  var quotedStrings []string
  for _, str := range stringSlice {
    quotedStrings = append(quotedStrings, strconv.Quote(str))
  }
  value := fmt.Sprintf("{ %s }", strings.Join(quotedStrings, ","))
  return value, nil
}

Great!

With this new type in place, we can change the datatype of Tags field from []string to StringSlice in the Task struct.

If we rerun the program, it will work as expected!!

1

Task 1 has been created

Filter by tag

Let’s move to the query side of the problem.

We would like to get a list of tasks associated with a particular tag.

It’d be a straightforward function that uses the find method in gorm.

1
2
3
4
5
6
7
8

func GetTasksByTag(db *gorm.DB, tag string) ([]Task, error) {
  tasks := []Task{}
  result := db.Find(&tasks, "? = any(tags)", tag)
  if result.Error != nil {
    return nil, result.Error
  }
  return tasks, nil
}

Then we need to call it from our main function

1
2
3
4
5
6
7

// ...
func main() {
  // ...
  tasks, err := GetTasksByTag(db, "project-x")
  panicOnError(err)
  fmt.Println(tasks)
}

Unfortunately, if we run the program, it will panic with the following error message

1
2
3

panic: sql: Scan error on column index 3: unsupported Scan, storing driver.Value type []uint8 into
type *main.StringSlice; sql: Scan error on column index 3: unsupported Scan,
storing driver.Value type []uint8 into type *main.StringSlice

As the error message says, the SQL driver unable to scan (unmarshal) the data type byte slice ([]uint8) into our custom type StringSlice.

To fix this, we need to provide a mechanism to convert []uint8 to StringSlice which in turn will be used by the SQL driver while scanning.

Like the Valuer interface, Golang provides Scanner interface to do the data type conversion while scanning.

The signature of the Scanner interface returns an error and not the converted value.

1
2
3

type Scanner interface {
  Scan(src interface{}) error
}

So, it implies the implementor of this interface should have a pointer receiver (*StringSlice) which will mutate its value upon successful conversion.

1
2
3

func (stringSlice *StringSlice) Scan(src interface{}) error {
  // ...
}

In the implementation of this interface, we just need to convert the byte slice into a string slice by converting it to a string (Postgres representation of array value) first, and then to StringSlice

1

[]uint8 --> {home,delegate} --> []string{"home", "delegate"}

After successful conversion, we need to assign the converted value to the receiver (*stringSlice)

1
2
3
4
5
6
7
8
9
10
11
12

func (stringSlice *StringSlice) Scan(src interface{}) error {
  val, ok := src.([]byte)
  if !ok {
    return fmt.Errorf("unable to scan")
  }
  value := strings.TrimPrefix(string(val), "{")
  value = strings.TrimSuffix(value, "}")

  *stringSlice = strings.Split(value, ",")

  return nil
}

That’s it. If we run the program now, we can see the output as expected.

1
2

[{2 schedule meeting with the team false [project-x]}
  {3 prepare for client demo false [slides project-x]}]

Summary

In this blog post, we have seen how we can make use of Valuer and Scanner interfaces in golang to marshal and unmarshal our custom data type from the database.

The source code can be found in my GitHub repository