Modern API design with Golang, PostgreSQL and Docker
organize your containers with Docker Compose and use CURL to make requests
Intro
Go is a powerful language that's highly performant and it has concurrency built in the language itself making it perfect for micro services. Here we will make a small API focusing on understanding HTTP and overall API design.
If you want to know more about Go concurrency look into how go routines and channels work.
Github repo & more
You can find all the code in my repository. I've been writing the article alongside the code so you can follow the commits. If you find a bug feel free to submit a PR.
You can also find me on LinkedIn, Twitter and Mastodon.
Target audience
This article is aimed at anyone who wants to build a CRUD micro service or a simple RESTful API using Go (or Golang for SEO purposes).
Additional resources
However if you are a junior dev or just starting out I'd recommend you check my Node.js (MERN) or Spring Boot series and if you are looking for more Go content I have a small example of a mini-twitter clone you can check. Additionally if you are after video tutorials I highly recommend Matt KØDVB's channel.
Tech
I'm on Go 1.18, for database we will use PostgreSQL and we will containerize our app with Docker and then use Docker Compose to link our app with the database. We will also include a small check for production environments in case you want to deploy it somewhere.
Libraries
We will be using Chi as our router and Go-PG in order to communicate with our PostgreSQL database.
Additional implementation
We will be focusing on the CRUD part in this article however in the real world you will also need to add authentication, authorization, logging and testing.
The Code
We will build a comments micro service, this means that you will have a CRUD API through which you will be able to create, read, update and delete comments.
Docker and Docker Compose
In order for the project to be able to run on any machine we will be using Docker and Docker Compose to help us set up the database, PostgreSQL in our case.
Start your project with
go mod init go-microservice-example
this way we can track all the packages we are going to be using.
Then add a folder called cmd, inside a folder called server and inside that folder add main.go, call the package main as well. This will be where we build our binary and start the server.
Inside main.go paste the following:
package main
import (
"log"
)
func main() {
log.Print("server has started")
}
Right now we are just making sure our program works with Docker so we are just printing one line.
Now add the Dockerfile (remember it doesn't have an extension):
FROM golang
RUN mkdir /app
ADD . /app
WORKDIR /app
RUN go build -o main ./cmd/server/main.go
EXPOSE 8080
CMD [ "/app/main" ]
What we are doing here is basically creating an ./app folder inside of our container, making it the work directory, building the binary, exposing the port and then calling the binary we just built so it can run our code.
Now since we are using PostgreSQL we will add our database using the Docker Compose file, so add docker-compose.yml with the following inside:
version: '3.8'
services:
db:
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: admin
api:
build: .
ports:
- 8080:8080
environment:
- PORT=8080
- DATABASE_URL=db
depends_on:
- db
Here we are declaring that we will be running 2 services, db and api, with api depending on db and making sure the ports are properly exposed: we will be working with the 8080 port. Additionally I've also added two environment variables that we will use later: PORT and DATABASE_URL.
With everything set up you can start your app with docker compose by typing:
docker compose up --build
You might see that the api service starts before the database, this isn't a bug. Just shut it down with CTRL+C and restart it with:
docker compose down; docker compose up --build
If everything went correctly you should see the message from the log:
Migrations
Before diving into the code we need to set up our database. Make a migrations folder.
This is the first migration to set up users, 1_users.up.sql:
CREATE TABLE IF NOT EXISTS users (
id SERIAL PRIMARY KEY,
name VARCHAR NOT NULL
);
And this is the second for comments, 2_comments.up.sql:
CREATE TABLE IF NOT EXISTS comments (
id SERIAL PRIMARY KEY,
comment VARCHAR NOT NULL,
comment_date DATE DEFAULT CURRENT_DATE,
user_id BIGINT REFERENCES users(id)
);
INSERT INTO users(name) VALUES('dev_test_user');
INSERT INTO comments (comment) VALUES('first test comment');
Connecting to the DB
Let's get Go-PG with
go get github.com/go-pg/pg/v10
and since we are using migrations let's get the migrations package as well
go get github.com/go-pg/migrations/v8
Create a db.go file now inside pkg/db folder
This should go inside your db.go file:
package db
import (
"log"
"os"
"github.com/go-pg/migrations/v8"
"github.com/go-pg/pg/v10"
)
func StartDB() (*pg.DB, error) {
var (
opts *pg.Options
err error
)
//check if we are in prod
//then use the db url from the env
if os.Getenv("ENV") == "PROD" {
opts, err = pg.ParseURL(os.Getenv("DATABASE_URL"))
if err != nil {
return nil, err
}
} else {
opts = &pg.Options{
//default port
//depends on the db service from docker compose
Addr: "db:5432",
User: "postgres",
Password: "admin",
}
}
//connect db
db := pg.Connect(opts)
//run migrations
collection := migrations.NewCollection()
err = collection.DiscoverSQLMigrations("migrations")
if err != nil {
return nil, err
}
//start the migrations
_, _, err = collection.Run(db, "init")
if err != nil {
return nil, err
}
oldVersion, newVersion, err := collection.Run(db, "up")
if err != nil {
return nil, err
}
if newVersion != oldVersion {
log.Printf("migrated from version %d to %d\n", oldVersion, newVersion)
} else {
log.Printf("version is %d\n", oldVersion)
}
//return the db connection
return db, err
}
Both libraries have docs and examples, so check them out, for example this is for migrations.
Setting up your API
Now that we have our database set up let's start working on the API, get Chi with the following command:
go get -u github.com/go-chi/chi/v5
and create a folder called api inside the pkg directory, then add an api.go file:
To get started let's just make sure our Chi router works so let's make two routes, "/" and "/comments/" inside api.go:
package api
import (
"net/http"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/go-pg/pg/v10"
)
//start api with the pgdb and return a chi router
func StartAPI(pgdb *pg.DB) *chi.Mux {
//get the router
r := chi.NewRouter()
//add middleware
//in this case we will store our DB to use it later
r.Use(middleware.Logger, middleware.WithValue("DB", pgdb))
r.Route("/comments", func(r chi.Router) {
r.Get("/", getComments)
})
r.Get("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("up and running"))
})
return r
}
func getComments(w http.ResponseWriter, r *http.Request){
w.Write([]byte("comments"))
}
Afterwards go back to cmd/server/main.go:
package main
import (
"fmt"
"go-microservice-example/pkg/api"
"go-microservice-example/pkg/db"
"log"
"net/http"
"os"
)
func main() {
log.Print("server has started")
//start the db
pgdb, err := db.StartDB()
if err != nil {
log.Printf("error starting the database %v", err)
}
//get the router of the API by passing the db
router := api.StartAPI(pgdb)
//get the port from the environment variable
port := os.Getenv("PORT")
//pass the router and start listening with the server
err = http.ListenAndServe(fmt.Sprintf(":%s", port), router)
if err != nil {
log.Printf("error from router %v\n", err)
}
}
Note how we are using the functions we have made from our DB and API packages: StartDB and StartAPI.
Now start the app with docker by doing
docker compose up --build
If you navigate to localhost:8080/ you should see "up and running" and if you navigate to localhost:8080/comments you should see "comments":
On the terminal you should be seeing the GET requests as well:
Now that we have set up everything let's dig into the logic, remember that there's no right answer for the structure of a project so if you don't like what I'm doing feel free to change it.
Create and Read operations
Let's write the basic POST (Create) and GET (Read) operations for our API. I recommend you go through every line and check what's going on, as well as the PG and CHI libraries.
First need to define how our app is going to communicate with the database, if you are coming from working with other frameworks or libraries you might think about this as doing our models (schemas) and making a repository - this basically means we will make a data struct (think of a POJO if you are coming from Java) that we will use to communicate with our database.
Make a models folder inside db then create two files:
This is inside user.go:
package models
type User struct {
ID int64 `json:"id"`
Name string `json:"name"`
}
Since we are not modifying the user we don't have any logic inside.
This is our comment.go:
package models
import "github.com/go-pg/pg/v10"
type Comment struct {
ID int64 `json:"id"`
Comment string `json:"comment"`
UserID int64 `json:"user_id"`
User *User `pg:"rel:has-one" json:"user"`
}
func CreateComment(db *pg.DB, req *Comment) (*Comment, error) {
_, err := db.Model(req).Insert()
if err != nil {
return nil, err
}
comment := &Comment{}
err = db.Model(comment).
Relation("User").
Where("comment.id = ?", req.ID).
Select()
return comment, err
}
func GetComment(db *pg.DB, commentID string) (*Comment, error) {
comment := &Comment{}
err := db.Model(comment).
Relation("User").
Where("comment.id = ?", commentID).
Select()
return comment, err
}
func GetComments(db *pg.DB) ([]*Comment, error) {
comments := make([]*Comment, 0)
err := db.Model(&comments).
Relation("User").
Select()
return comments, err
}
The logic is fairly simple, just make sure you understand the relation and how the library works. The ORM is making the SQL queries for us using the values we pass.
Now let's go back to our API folder and use all the functions we have made. Remember that we need to import the models package first. Then, in order to get the request or response from the server let's capture it through a struct (CommentResponse and CommentRequest) however be careful because first we need to decode and encode the data every time we send it back or receive it.
This is our current api.go file:
package api
import (
"encoding/json"
"go-microservice-example/pkg/db/models"
"log"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/go-pg/pg/v10"
)
//start api with the pgdb and return a chi router
func StartAPI(pgdb *pg.DB) *chi.Mux {
//get the router
r := chi.NewRouter()
//add middleware
//in this case we will store our DB to use it later
r.Use(middleware.Logger, middleware.WithValue("DB", pgdb))
//routes for our service
r.Route("/comments", func(r chi.Router) {
r.Post("/", createComment)
r.Get("/", getComments)
})
//test route to make sure everything works
r.Get("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("up and running"))
})
return r
}
type CreateCommentRequest struct {
Comment string `json:"comment"`
UserID int64 `json:"user_id"`
}
type CommentResponse struct {
Success bool `json:"success"`
Error string `json:"error"`
Comment *models.Comment `json:"comment"`
}
func createComment(w http.ResponseWriter, r *http.Request) {
//get the request body and decode it
req := &CreateCommentRequest{}
err := json.NewDecoder(r.Body).Decode(req)
//if there's an error with decoding the information
//send a response with an error
if err != nil {
res := &CommentResponse{
Success: false,
Error: err.Error(),
Comment: nil,
}
err = json.NewEncoder(w).Encode(res)
//if there's an error with encoding handle it
if err != nil {
log.Printf("error sending response %v\n", err)
}
//return a bad request and exist the function
w.WriteHeader(http.StatusBadRequest)
return
}
//get the db from context
pgdb, ok := r.Context().Value("DB").(*pg.DB)
//if we can't get the db let's handle the error
//and send an adequate response
if !ok {
res := &CommentResponse{
Success: false,
Error: "could not get the DB from context",
Comment: nil,
}
err = json.NewEncoder(w).Encode(res)
//if there's an error with encoding handle it
if err != nil {
log.Printf("error sending response %v\n", err)
}
//return a bad request and exist the function
w.WriteHeader(http.StatusBadRequest)
return
}
//if we can get the db then
comment, err := models.CreateComment(pgdb, &models.Comment{
Comment: req.Comment,
UserID: req.UserID,
})
if err != nil {
res := &CommentResponse{
Success: false,
Error: err.Error(),
Comment: nil,
}
err = json.NewEncoder(w).Encode(res)
//if there's an error with encoding handle it
if err != nil {
log.Printf("error sending response %v\n", err)
}
//return a bad request and exist the function
w.WriteHeader(http.StatusBadRequest)
return
}
//everything is good
//let's return a positive response
res := &CommentResponse{
Success: true,
Error: "",
Comment: comment,
}
err = json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error encoding after creating comment %v\n", err)
w.WriteHeader(http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusOK)
}
type CommentsResponse struct {
Success bool `json:"success"`
Error string `json:"error"`
Comments []*models.Comment `json:"comments"`
}
func getComments(w http.ResponseWriter, r *http.Request) {
//get db from ctx
pgdb, ok := r.Context().Value("DB").(*pg.DB)
if !ok {
res := &CommentsResponse{
Success: false,
Error: "could not get DB from context",
Comments: nil,
}
err := json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error sending response %v\n", err)
}
w.WriteHeader(http.StatusBadRequest)
return
}
//call models package to access the database and return the comments
comments, err := models.GetComments(pgdb)
if err != nil {
res := &CommentsResponse{
Success: false,
Error: err.Error(),
Comments: nil,
}
err := json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error sending response %v\n", err)
}
w.WriteHeader(http.StatusBadRequest)
return
}
//positive response
res := &CommentsResponse{
Success: true,
Error: "",
Comments: comments,
}
//encode the positive response to json and send it back
err = json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error encoding comments: %v\n", err)
w.WriteHeader(http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusOK)
}
I've left all the error handlers so you know exactly what's going on. Rebuild the Docker containers and start the app using
docker compose up --build
CURL
We will be using CURL to make sure everything works, so include this request.json in the root of your project:
{
"comment": "test comment",
"user_id": 1
}
Note that if you are on Windows and having some issues with the command just do:
Remove-item alias:curl
Now open the terminal inside your project and type:
curl -X POST localhost:8080/comments -d "@request.json" | jq
This will make a POST request to the /comments route and the function createComment will handle the rest. You should get this response if everything went correctly:
Now to make sure the comment is stored let's make a GET request via:
curl -X GET localhost:8080/comments
And you should get back the comments (or in this case just one) stored in your database:
If you are using VSCode you can separate the terminal which comes in handy when you are running a server, this is what my current terminal looks like:
Now that we know everything works go to request.json and delete the "comment" line then try to POST it again, you should see the following error:
DRY
DRY stands for "don't repeat yourself" so let's refactor a bit our api.go and introduce the error handling functions:
package api
import (
"encoding/json"
"go-microservice-example/pkg/db/models"
"log"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/go-pg/pg/v10"
)
//start api with the pgdb and return a chi router
func StartAPI(pgdb *pg.DB) *chi.Mux {
//get the router
r := chi.NewRouter()
//add middleware
//in this case we will store our DB to use it later
r.Use(middleware.Logger, middleware.WithValue("DB", pgdb))
//routes for our service
r.Route("/comments", func(r chi.Router) {
r.Post("/", createComment)
r.Get("/", getComments)
})
//test route to make sure everything works
r.Get("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("up and running"))
})
return r
}
// -- Responses
type CreateCommentRequest struct {
Comment string `json:"comment"`
UserID int64 `json:"user_id"`
}
type CommentResponse struct {
Success bool `json:"success"`
Error string `json:"error"`
Comment *models.Comment `json:"comment"`
}
type CommentsResponse struct {
Success bool `json:"success"`
Error string `json:"error"`
Comments []*models.Comment `json:"comments"`
}
//-- UTILS --
func handleErr(w http.ResponseWriter, err error) {
res := &CommentResponse{
Success: false,
Error: err.Error(),
Comment: nil,
}
err = json.NewEncoder(w).Encode(res)
//if there's an error with encoding handle it
if err != nil {
log.Printf("error sending response %v\n", err)
}
//return a bad request and exist the function
w.WriteHeader(http.StatusBadRequest)
}
func handleDBFromContextErr(w http.ResponseWriter) {
res := &CommentResponse{
Success: false,
Error: "could not get the DB from context",
Comment: nil,
}
err := json.NewEncoder(w).Encode(res)
//if there's an error with encoding handle it
if err != nil {
log.Printf("error sending response %v\n", err)
}
//return a bad request and exist the function
w.WriteHeader(http.StatusBadRequest)
}
// -- handle routes
func createComment(w http.ResponseWriter, r *http.Request) {
//get the request body and decode it
req := &CreateCommentRequest{}
err := json.NewDecoder(r.Body).Decode(req)
//if there's an error with decoding the information
//send a response with an error
if err != nil {
handleErr(w, err)
return
}
//get the db from context
pgdb, ok := r.Context().Value("DB").(*pg.DB)
//if we can't get the db let's handle the error
//and send an adequate response
if !ok {
handleDBFromContextErr(w)
return
}
//if we can get the db then
comment, err := models.CreateComment(pgdb, &models.Comment{
Comment: req.Comment,
UserID: req.UserID,
})
if err != nil {
handleErr(w, err)
return
}
//everything is good
//let's return a positive response
res := &CommentResponse{
Success: true,
Error: "",
Comment: comment,
}
err = json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error encoding after creating comment %v\n", err)
w.WriteHeader(http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusOK)
}
func getComments(w http.ResponseWriter, r *http.Request) {
//get db from ctx
pgdb, ok := r.Context().Value("DB").(*pg.DB)
if !ok {
handleDBFromContextErr(w)
return
}
//call models package to access the database and return the comments
comments, err := models.GetComments(pgdb)
if err != nil {
handleErr(w, err)
return
}
//positive response
res := &CommentsResponse{
Success: true,
Error: "",
Comments: comments,
}
//encode the positive response to json and send it back
err = json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error encoding comments: %v\n", err)
w.WriteHeader(http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusOK)
}
Get Comment By Id
We already have the function to get a comment by id in our models package, let's add a route and a function to handle that route in our API package now:
// here's the route
r.Get("/{commentID}", getCommentByID)
And here's the function to handle it:
func getCommentByID(w http.ResponseWriter, r *http.Request) {
//get the id from the URL parameter
//alternatively you could use a URL query
commentID := chi.URLParam(r, "commentID")
//get the db from ctx
pgdb, ok := r.Context().Value("DB").(*pg.DB)
if !ok {
handleDBFromContextErr(w)
return
}
//get the comment from the DB
comment, err := models.GetComment(pgdb, commentID)
if err != nil {
handleErr(w, err)
return
}
//if the retrieval from the db was successful send the data
res := &CommentResponse{
Success: true,
Error: "",
Comment: comment,
}
//encode the positive response to json and send it back
err = json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error encoding comments: %v\n", err)
w.WriteHeader(http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusOK)
}
PUT
Now that we have READ and POST let's focus on updating our data by handling a PUT request. First let's edit our models package (comment.go) by adding this function:
func UpdateComment(db *pg.DB, req *Comment) (*Comment, error) {
_, err := db.Model(req).
WherePK().
Update()
if err != nil {
return nil, err
}
comment := &Comment{}
err = db.Model(comment).
Relation("User").
Where("comment.id = ?", req.ID).
Select()
return comment, err
}
Then let's add the route inside the API package (api.go):
r.Put("/{commentID}", updateCommentByID)
And finally let's write the function that handles the request, gets it from the user then interacts with the database by updating the data:
func updateCommentByID(w http.ResponseWriter, r *http.Request) {
//get the data from the request
req := &CommentRequest{}
//decode the data
err := json.NewDecoder(r.Body).Decode(req)
if err != nil {
handleErr(w, err)
return
}
pgdb, ok := r.Context().Value("DB").(*pg.DB)
if !ok {
handleDBFromContextErr(w)
return
}
//get the commentID to know what comment to modify
commentID := chi.URLParam(r, "commentID")
//we get a string but we need to send an int so we convert it
intCommentID, err := strconv.ParseInt(commentID, 10, 64)
if err != nil {
handleErr(w, err)
return
}
//update the comment
comment, err := models.UpdateComment(pgdb, &models.Comment{
ID: intCommentID,
Comment: req.Comment,
UserID: req.UserID,
})
if err != nil {
handleErr(w, err)
}
//return successful response
res := &CommentResponse{
Success: true,
Error: "",
Comment: comment,
}
//send the encoded response to responsewriter
err = json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error encoding comments: %v\n", err)
w.WriteHeader(http.StatusBadRequest)
return
}
//send a 200 response
w.WriteHeader(http.StatusOK)
}
Note that we have to convert the string commentID into the int ID, there's a difference between receiving data as json and how we handle the data inside structs, that's why inside structs you have things like
type Comment struct {
ID int64 `json:"id"`
Comment string `json:"comment"`
UserID int64 `json:"user_id"`
User *User `pg:"rel:has-one" json:"user"`
}
For example the ID is how we name the commentID in our codebase but if we send or receive that ID through JSON then we will just name it "id".
Now modify your request.json (change the comment data) and use this command to send a PUT request:
curl -X PUT localhost:8080/comments/1 -d "@request.json" | jq
If everything went well you should see this in your terminal:
SuccResponse DRY
Since our success response is the same everywhere let's refactor it to follow the DRY principle, we will call this function succ because I'm childish, so refactor the code in our API package:
func succCommentResponse(comment *models.Comment, w http.ResponseWriter) {
//return successful response
res := &CommentResponse{
Success: true,
Error: "",
Comment: comment,
}
//send the encoded response to responsewriter
err := json.NewEncoder(w).Encode(res)
if err != nil {
log.Printf("error encoding comment: %v\n", err)
w.WriteHeader(http.StatusBadRequest)
return
}
//send a 200 response
w.WriteHeader(http.StatusOK)
}
Delete
Let's finish this, go to your models package and add the Delete function:
func DeleteComment(db *pg.DB, commentID int64) error {
comment := &Comment{}
err := db.Model(comment).
Relation("User").
Where("comment.id = ?", commentID).
Select()
if err != nil {
return err
}
_, err = db.Model(comment).WherePK().Delete()
return err
}
And inside the API package add the route and the function to handle that route:
r.Delete("/{commentID}", deleteCommentByID)
func deleteCommentByID(w http.ResponseWriter, r *http.Request) {
//parse in the req body
req := &CommentRequest{}
err := json.NewDecoder(r.Body).Decode(req)
if err != nil {
handleErr(w, err)
return
}
//get the db from ctx
pgdb, ok := r.Context().Value("DB").(*pg.DB)
if !ok {
handleDBFromContextErr(w)
return
}
//get the commentID
commentID := chi.URLParam(r, "commentID")
intCommentID, err := strconv.ParseInt(commentID, 10, 64)
if err != nil {
handleErr(w, err)
return
}
//delete comment
err = models.DeleteComment(pgdb, intCommentID)
if err != nil {
handleErr(w, err)
}
//send successful response
succCommentResponse(nil, w)
}