The goal of this blog post is to refresh concepts for senior engineers and introduce topics of concurrency, parallelism, race conditions, deadlock, mutex and more to anyone who hasn’t had to deal with concurrent programming directly.

Concurrency

We can consider concurrency as parts of the program that execute independently in a non-deterministic order.

Concurrency is NOT Parallelism: depending on the hardware we can run parts of the program at the same time (in parallel) but just because we have concurrency it doesn’t mean we have parallelism.

What does it mean, “non-deterministic order”? It’s different behaviour on different runs but same result. For example processes(*) have a different trace of execution: ​ {1, 2a, 3a, 2b, 3b, 4}
​{1, 2a, 2b, 3a, 3b, 4}​
{1, 3a, 3b, 2a, 2b, 4}​ ​

*process: a series of actions or events to achieve a goal

We can look at a concurrent program with a non-deterministic trace of execution like this:

Concurrency vs Parallelism​

Concurrency: dealing with things happening out of order.

Parallelism: dealing with things happening at the same time.

Parallelism will make your program faster while concurrency doesn’t have to. Concurrency can happen in a single core processor​. Parallelism can only happen in a multi-core processor.

Concurrency can make your program slower, if you are on a single core machine or your logic doesn't need concurrency​. But it can make your program faster when you have to wait for something – by letting other parts of the program execute.

Why do we need concurrency?

• Enable parallelism

• Let parts of the program execute independently

• Remove the need to wait for an event

If we look at our previous concurrent example but this time we have access to parallelism as well, we can see that the program will be much faster:

Another picture to illustrate concurrency, parallelism and everything put together:

Concurrency Problems

The main issue with concurrency comes from its non-deterministic nature that can produce race conditions.

For example:
{1, 2a, 3a, 2b, 3b, 4}​
{1, 2a, 2b, 3a, 3b, 4}​
{1, 3a, 3b, 2a, 2b, 4}​
{1, 3a, 2a, 2b, 3b, 4} <- Race condition

Race Condition

What is it and when does it happen?
- When two or more processes try to alter a shared resource simultaneously​.
- It happens when there's a fault in the timing or order of events​.
- Data Race is caused by a Race Condition

How to avoid Race Conditions?

• Make the operations atomic for example make READ, MODIFY and STORE one operation​

• Ensure that the shared resource, if being modified, is used by one process at a time – one atomic operation​

• Atomicity adds more sequential order to our operations​

Looking at the previous example we can make atomic operations and add a more sequential order that will prevent our program into running into race conditions, the drawback is that by adding deterministic outcome you reduce speed of execution.

Mutex (Lock)

A great way to prevent race conditions is to use a Mutex: you can think of it as a lock that controls who has access on shared resources. We use a mutex for mutual exclusion to control concurrency.
Wikipedia has a great definition: a synchronization primitive that prevents state from being modified or accessed by multiple threads of execution at once. Locks enforce mutual exclusion concurrency control policies

I also like this response from Stack Overflow about using a rubber chicken to allow people talk in a heated discussion while others wait.

Deadlock

A possible issue with concurrency is having a deadlock, which happens when
- Two processes are waiting for each other
- Mutual exclusion is not present: we have no control over using the shared resource

Key Concurrency Concepts

• Use concurrency to enable parallelism and/or avoid waiting for events​

• Make sure your shared resources don't produce race conditions by using mutual exclusion and atomic operations

Golang

It’s a language designed to use for servers on the cloud. Go concurrency model is based on: Hoare's CSP (Communicating Sequential Processes) in 1978 and even Dijkstra's guarded commands (1975).
Golang's main difference from other similar languages is channels. Erlang is closer to the original CSP, where you communicate to a process by name rather than over a channel.

Github Repo with examples

I will be using examples that are available on my Github and you can also find the links to go dev playground if you don’t want to run the code locally in there.

Goroutines

• Goroutine is a lightweight "thread" we use for concurrent operations​

• Use goroutines for async programming: spin a goroutine to do something else while your program continues by using the go keyword before the function​

• You can use anonymous functions with goroutines​

• Goroutine is a wordplay related to Coroutine​

package main

import (
    "fmt"
    "time"
)

// print the ints in the loop
func printFrom(from string) {
    for i := 0; i < 3; i++ {
        fmt.Println(from, ":", i)
    }
}

func main() {
    printFrom("direct")

    // if you keep launching the program sometimes you will see goroutine2 first
    go printFrom("goroutine1: runs concurrently")
    go printFrom("goroutine2: runs concurrently")

    go func(msg string) {
        fmt.Println(msg)
    }("anonymous goroutine")

    // wait a second for goroutines to complete
    // otherwise the program finishes without displaying results from our goroutines
    time.Sleep(time.Second)
    fmt.Println("done")
}

Channels

• A channel is like a (UNIX) pipe: data goes in one end and comes out of another in the same order until the channel is closed​

• Channels act as first-in-first-out queues​

• Channels are bi-directional: you can restrict a channel's direction​

• You can have multiple writers and readers on the same channel

package main

import "fmt"

func main() {
    fmt.Println("starting the program")
    // make a new channel
    messages := make(chan string)

    // use an anymous  goroutine to send the string "ping" to the channel
    go func() { messages <- "first msg" }()
    go func() { messages <- "second msg" }()

    // we only loop twice becase we know we sent 2 messages
    for i := 0; i < 2; i++ {
        fmt.Println(<-messages)
    }
}

In this example we can see how to send 2 messages concurrently to the same channel and read from the channel using a for loop.

Key concepts about channels:

• Channel will block until someone reads the data, unless it's buffered​

• You can have receive only or send only channels​

• You can pass channels in between functions​

• You can NOT send data to a closed channel

Deadlock example with channels

package main

import "fmt"

func main() {
    fmt.Println("starting the program")
    // make a new unbuffered channel
    messages := make(chan string)

    // we pass the channel in the function
    // share memory by communicating
    deadlockExample(messages)
    // normalLoop(messages)
}

func deadlockExample(messages chan string) {
    // use an anymous  goroutine to send the string to the channel
    go func() { messages <- "first msg" }()
    go func() { messages <- "second msg" }()
    // Deadlock!
    // this loop is waiting forever on the channel
    // because the channel is never closed
    for msg := range messages {
        // display the data we got
        fmt.Println(msg)
    }
}

func normalLoop(messages chan string) {
    go func() {
        // send the data
        messages <- "first msg"
        messages <- "second msg"
        // close the channel
        close(messages)
    }()

    // now that the channel is closed this for loop won't wait forever
    for msg := range messages {
        // get the data from the channel
        // display the data we got
        fmt.Println(msg)
    }
}

Key Considerations:

• Range and For loops will wait to get data indefinitely from the channel until they are closed​

• The sender should CLOSE the channel

Buffered channels

For better performance we can use buffered channels if we know the capacity we will use.

package main

import "fmt"

func main() {
    messages := []string{"first msg", "second msg", "third msg"}

    // this channel only has capacity for 3
    // but we will use concurrency to share memory and send the latest message
    // we can use buffered channels so we never have to wait for the receiver to store the data
    // we store the data in the buffer, if the buffer is full we block
    bufferedChannel := make(chan string, 3)

    messages = addMessageToSlice("last msg", messages)
    // by enabling concurrency by using "go" keyword in front of this function
    // we send the latest message to the channel when the first one is out
    // if we remove the "go" keyword we will have a deadlock!
    go sendMessagesToChannel(messages, bufferedChannel)

    for m := range bufferedChannel {
        fmt.Println(m)
    }
}

func sendMessagesToChannel(messages []string, bufchan chan string) {
    for _, m := range messages {
        bufchan <- m
    }
    // remember to always close the channel
    close(bufchan)
}

func addMessageToSlice(msg string, messages []string) []string {
    messages = append(messages, msg)
    return messages
}

If we remove the go keyword from go sendMessagesToChannel(messages, bufferedChannel) we will have bufchan <- m waiting to send a 4th message but capacity is full and the for loop in main is waiting for the data to start to flow, creating a deadlock!

WaitGroup

WaitGroup is an object from the sync package that will let you control the flow in a more sequential way, check the example below.

package main

import (
    "fmt"
    "sync"
)

func main() {
    messages := []string{"first msg", "second msg", "third msg"}
    moreMessages := []string{"fourth msg", "fifth msg", "last msg"}

    bufferedChannel := make(chan string, 3)

    // we make an object to control the flow of our execution
    // it contains a counter for each executed function
    wg := sync.WaitGroup{}
    for _, m := range moreMessages {
        // every time we spin a new goroutine we increase the counter
        // on how much we have to wait
        wg.Add(1)
        // keep in mind the messages will not arrive in order because go routines spawn concurrently (non-deterministic behaviour)
        go func(m string) {
            messages = addMessageToSlice(m, messages)
            // once the execution is done we let our WaitGroup know (this reduces the counter)
            wg.Done()
        }(m)

    }

    // we make sure we wait for the previous block to finish
    wg.Wait()

    go sendMessagesToChannel(messages, bufferedChannel)

    for m := range bufferedChannel {
        fmt.Println(m)
    }
}

func sendMessagesToChannel(messages []string, bufchan chan string) {
    for _, m := range messages {
        bufchan <- m
    }
    // remember to always close the channel
    close(bufchan)
}

func addMessageToSlice(msg string, messages []string) []string {
    messages = append(messages, msg)
    return messages
}

Don’t use WaitGroup just because you can

Here’s an example of the same code as above but refactored to work as expected without using WaitGroup.

package main

import (
    "fmt"
)

func main() {
    messages := []string{"first msg", "second msg", "third msg"}
    moreMessages := []string{"fourth msg", "fifth msg", "last msg"}

    bufferedChannel := make(chan string, 3)

    // due to the concurrent nature of our program we can just keep sending messages to the channel
    for _, m := range moreMessages {
        messages = addMessageToSlice(m, messages)
    }

    go sendMessagesToChannel(messages, bufferedChannel)

    for m := range bufferedChannel {
        fmt.Println(m)
    }
}

func sendMessagesToChannel(messages []string, bufchan chan string) {
    for _, m := range messages {
        bufchan <- m
    }
    close(bufchan)
}

func addMessageToSlice(msg string, messages []string) []string {
    return append(messages, msg)
}

Channel Key Points

• Used to communicate by sharing memory​

• Remember to close the channel (sometimes GC will take care of it)​

• If you already know your channel's capacity use a buffered channel

Select

• The select statement lets a goroutine wait on multiple communication operations​

• A select blocks until one of its cases can run, then it executes that case. It chooses one at random if multiple are ready

package main

import (
    "fmt"
    "time"
)

func main() {
    // Create two channels
    channel1 := make(chan string)
    channel2 := make(chan string)

    // msg will arrive after 2 seconds
    go func() {
        time.Sleep(2 * time.Second)
        channel1 <- "Message from Channel 1"
    }()

    // msg will arrive after 1 second
    go func() {
        time.Sleep(1 * time.Second)
        channel2 <- "Message from Channel 2"
    }()

    // we will execute this block twice and check every
    // if we get a msg back
    // then print whichever is ready
    for i := 0; i < 2; i++ {
        // select block waits for one of the channels to receive a msg
        // it waits for at least one channel to be ready
        select {
        case msg1 := <-channel1:
            fmt.Println(msg1)
        case msg2 := <-channel2:
            fmt.Println(msg2)
            // we can add a default case for when we don't want to wait for a msg
            // default:
            //    fmt.Println("default")
        }
    }
}

If you are unsure about the data arriving but want to proceed anyway have a default case in your select

package main

import (
    "fmt"
    "time"
)

func main() {
    channel := make(chan string)

    // wait 2s and send data to the channel
    go func() {
        time.Sleep(2 * time.Second)
        channel <- "Data from Goroutine"
    }()

    // infinite loop
    for {
        select {
        case msg := <-channel:
            fmt.Println(msg)
            return // break infinite loop

        // default case simply simulates some work
        // we call it every time  until we get the other case
        default:
            fmt.Println("No data yet, doing other work...")
            time.Sleep(500 * time.Millisecond) // Simulate some work
        }
    }
}

Mutex

• Use a mutex to access your shared resources​

• Think if you really want or need to have a shared state

Mutex ensures only one process is modifying the shared data at a time​.

package main

import (
    "fmt"
    "sync"
)

type Container struct {
    counters map[string]int // shared resource that doesn't support concurrent writes
    mu       sync.Mutex     // protect the shared resource with a lock
}

func (c *Container) inc(name string) {
    // try removing the lock here and see what happens
    // the program will panic with an error
    // fatal error: concurrent map writes
    c.mu.Lock()
    defer c.mu.Unlock()
    c.counters[name]++
}

func (c *Container) dec(name string) {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.counters[name]--
}

func main() {
    c := Container{
        counters: map[string]int{"a": 0, "b": 0}, // map doesn't support concurrent access
    }

    var wg sync.WaitGroup

    increment := func(name string, n int) {
        for i := 0; i < n; i++ {
            c.inc(name) // increment counter
        }
        wg.Done()
    }

    decrement := func(name string, n int) {
        for i := 0; i < n; i++ {
            c.dec(name) // decrement counter
        }
        wg.Done()
    }

    // if you WaitGroup counter is lower than needed your program will panic
    // if it's higher it will be blocked in a deadlock
    wg.Add(3)
    go decrement("a", 1000)
    go increment("a", 1000)
    go increment("b", 1000)

    wg.Wait()

    fmt.Println("Final Counters:", c.counters)
}

There’s more to concurrency in Go!

In this article you have things to get started and to understand the concurrency model but to master it going forward I’d recommend looking into patterns like Fan-In Fan-Out (Multiplexing) and Pipelines to get started.

Additionally once you start developing services it’s a good practice to rely con context for cancellation and more.

This great comic is from False Knees and it illustrates that you can find patterns everywhere, you can also check out this National Graphic article. Your code base is no different.

The objective of this article is to understand or refresh the memory of the different patterns and then jump into Dependency Injection to do something cool.

Resources

You can find all of the code in this repository on my Github.

If you are kind of new or not sure what's all this about I recommend you read my previous blog post: Software Engineering OOP principles and good practices to avoid spaghetti code.

Additionally, I highly recommend the site Refactor Guru to learn about design patterns and refactoring.
As well as Christopher Okhravi's Youtube Channel for more in-depth videos and examples, he's a great teacher.

Tip

I recommend exploring all the resources above to have a better understanding of each pattern as my intention is to provide and simplify examples. Wikipedia is another great resource but it might be a bit too difficult for beginners.

Strategy

The strategy pattern allows you to select an algorithm at run time. The name is "strategy" because given a certain context you can select a strategy appropriate for that context.

This is directly related to one of the SOLID principles, the open-closed principle that states software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.

Let's look at an example with TypeScript, and if you are using any other language the most important part is you understand how the classes and interfaces work together.

//define interface we will use for different strategies
interface Strategy {
    execute(str: string): string
}

//define different strategies we could use
class upperCaseStrategy implements Strategy {

    public execute(str: string): string {
        return str.toUpperCase()
    }
}

class lowerCaseStrategy implements Strategy {

    public execute(str: string): string {
        return str.toLowerCase()
    }
}

class Context {

    //we will be using a different strategy for every situation
    //however here we just define a parameter of type Strategy
    //that will be changed as we wish
    private strategy: Strategy

    //while making the object we will include the strategy to use
    constructor(strategy: Strategy) {
        this.strategy = strategy
    }

    //in case we want to change out strategy we have a method for that
    public setStrategy(strategy: Strategy) {
        this.strategy = strategy
    }

    //we will get back the modified string according to the strategy we use
    public executeStrategy(str: string): string{
        return this.strategy.execute(str)
    }
}


//define a string we will be using for testing
const randomString = 'rAnDoM sTrInG'
console.log('inittial string:', randomString)
//make a new context with the upper case strategy
const context = new Context(new upperCaseStrategy())
//use the current strategy (which is upperCase) to get an upper case String
const upperCaseString = context.executeStrategy(randomString)
//check it's upper case
console.log('upper case:', upperCaseString)
//change context's strategy
context.setStrategy(new lowerCaseStrategy())
//get the lower case string with a different strategy
const lowerCaseString = context.executeStrategy(upperCaseString)
//check it's lower case
console.log('lower case:', lowerCaseString)

In this example you can see how we will be able to use the same Context class throughout our entire program but by changing strategies we will be able to get different results when we call the executeStrategy method.

If you want more complex examples Refactoring Guru has a cache implementation in Go and a payment method in Java.

Dependency Injection vs Strategy

If you are somehow familiar with DI and not sure why the strategy pattern is used, I'd recommend reading this Stack Overflow Answer.

Observer

The observer pattern relies on one principle: the observable (which often is called subject or event manager) needs to notify/update the observers (which can be called subscriber) it contains about an event.

The most common case would be a notification of some kind, you want to notify your users (or a specific set of users) about something that has happened so you take the observers and push their update method.

Note that for the observers to be notified they need to be somehow connected to the observable and the observable also needs to have a way to call the notify/update methods of the observables.

Let's look at the code, but keep in mind that I'm using the words observable, subject and event manager/dispatcher interchangeably. The same goes for observer and subscriber.

//subject or event manager/dispatcher
interface Observable {
    //add the observers we want to track out observable
    add(observer: Observer): void
    //same but delete
    remove(observer: Observer): void
    //notify all of the observers
    notify(): void
}

//event listener
interface Observer {

    name: string
    //the update method will be triggered when the observable uses it
    //you can pass the observable to get more context or not
    update(observable: Observable): void
}

//since a lot of the examples use Subject instead of Observable let's call our class Subject
//it will be the one notifying the different observers
class Subject implements Observable {

    name: string

    constructor(name: string) {
        this.name = name
    }

    //using a Map but this can just be an array
    private observersMap: Map<Observer['name'], Observer> = new Map<Observer['name'], Observer>

    //add the observer/subscriber to our observable/subject
    public add(observer: Observer): void {

        this.observersMap.set(observer.name, observer)
        console.log(`added observer ${observer.name} to ${this.name}`)
    }

    public remove(observer: Observer): void {

        this.observersMap.delete(observer.name)
        console.log(`removed observer ${observer.name} to ${this.name}`)
    }

    //we do a forEach on our map so we can press the update method in all of them and pass the current observable/subject
    public notify(): void {

        //note that we allow to pass the instance of our observable to every observer, check the interface
        this.observersMap.forEach((v,k,m) => v.update(this)) 
    }

    public getName(): string {
        return this.name
    }

    //example of a possible method will have the notify method inside, note how we call it at the end to notify our observers
    //this is a simple change of name but it could be anything
    public changeName(name: string): void {

        console.log(`changing name from ${this.name} to ${name}..`)
        this.name = name

        console.log(`notifications being sent..`)
        this.notify()
    }
}

class webObserver implements Observer {

    name: string

    constructor(name: string) {
        this.name = name
    }
    update(observable: Subject): void {
        console.log(`${this.name} notification from ${observable.getName()}`)
    }
}

class phoneObserver implements Observer {

    name: string

    constructor(name: string) {
        this.name = name
    }
    update(observable: Subject): void {
        console.log(`${this.name} notification from ${observable.getName()}`)
    }
}

//init all our objects
const subject = new Subject('THANOS SERVICE')
const webObs = new webObserver('WEB')
const phoneObs = new phoneObserver('PHONE')

//make sure to subscribe/add the observers the observable
subject.add(webObs)
subject.add(phoneObs)

//notify all of our observers
subject.notify()

//now let's change the name
//this is just an example of a possible change of state inside the class, it could be a new item arrival for example or something else
subject.changeName('IRONMAN')

This is the output:

If you want more examples feel free to check Refactoring Guru, here's Go and Java.

Observer vs Pub/Sub

You might have noticed that the observer pattern could end up being too tightly coupled and it's very similar to another architecture: pub/sub.

Pub/Sub is a bit different and adds complexity, however it's great if you decide to work with events as it provides a separate channel for subscribers.

Singleton

If you want to ensure there's only one instance of the class then use the singleton pattern. Although some people might argue against it this pattern is widely used, especially with database connections.

The beauty of this design is its use of the private initializer.

class Singleton {
    //here's the best part, we use a static variable to store our singleton instance
    //and make it private so we can only get it through the getsingleton() method
    private static singleton: Singleton

    private constructor() {
        console.log('starting singleton..')
    }

    //call this method every time we want a singleton
    public static getSingleton(): Singleton {

        console.log('getting singleton..')
        //if the singleton instance (static singleton) is null we will initialize it
        if(!Singleton.singleton) {
            console.log('no singleton available')
            Singleton.singleton = new Singleton()
        }

        return Singleton.singleton
    }
}

function mainProgram() {
    //... code
    const singleton1 = Singleton.getSingleton()
    const singleton2 = Singleton.getSingleton()

    if(singleton1 === singleton2) {
        console.log('same instance')
    }

}
mainProgram()

This is the output:

As always Refactoring Guru has examples in Java and Go as well.

Factory

As you can see in the UML diagram the whole point of the factory pattern is to have a specific place (interface of the factory) where you can create the needed objects. The factory is called "creator" and the objects created are called "products".

Basically what we are doing is encapsulating the business logic (a particular way of constructing objects). The principle is very simple but you can make it as complex as you want, the idea is to always have a factory method available that you can move around your codebase and get the concrete objects you want or need.

I really like this definition from Refactoring Guru:

Factory Method is a creational design pattern that provides an interface for creating objects in a superclass, but allows subclasses to alter the type of objects that will be created.

Let's look at a code example with TypeScript:

//pizza is going to be the product that a factory can make
interface Pizza {
    name: string
    getName(): string
    cookingTime(): void
    eat(): void
}

//this is just an example, NY pizza is different from Chicago pizza
class NewYorkPizza implements Pizza {

    name: string

    constructor(name: string) {
        this.name = name
    }

    getName(): string {
        return this.name
    }

    cookingTime(): void{
       console.log('15min') 
    }

    eat(): void {
        console.log('eating NY pizza')
    }
}

class ChicagoPizza implements Pizza {

    name: string

    constructor(name: string) {
        this.name = name
    }

    getName(): string {
        return this.name
    }
    cookingTime(): void {
        console.log('30min')
    }
    eat(): void {
        console.log('eating Chicago pizza')
    }
}

//this is our creator, we can use this interfece anywhere in order to make our products(pizzas)
interface PizzaFactory {
    makePizza(pizza: Pizza): Pizza
}

//lets think of different restaurants that want to make pizzas
class NewYorkRestaurant implements PizzaFactory {

    makePizza(): Pizza {
        return new NewYorkPizza('best NY pizza')
    }
}

class ChicagoRestaurant implements PizzaFactory {

    makePizza(): Pizza {
        return new ChicagoPizza('best Chicago pizza')
    }
}

//create our factories
const newYorkRestaurant: NewYorkRestaurant = new NewYorkRestaurant()
const chicagoRestaurant: ChicagoRestaurant = new ChicagoRestaurant() 

//get the product
const myNewYorkPizza = newYorkRestaurant.makePizza()
const myChicagoPizza = chicagoRestaurant.makePizza()

//here we use our products
//this is a simple example but remember you can add a lot more logic and extend the examples
//the point of the pattern is so that you could use the factories anywhere you need
//and modify the logic inside
//try adding a delivery method for example to one of the restaurants or something concrete to the pizzas
const newYorkPizzaName = myNewYorkPizza.getName()
console.log(`NY pizza name is: ${newYorkPizzaName}`)
myNewYorkPizza.cookingTime()

const chicagoPizzaName = myChicagoPizza.getName()
console.log(`NY pizza name is: ${chicagoPizzaName}`)
myChicagoPizza.cookingTime()

This is the output:

Check out additional examples from RG: Java and Go.

Abstract Factory

The Factory method is usually used, or well, it evolves into an Abstract Factory. As the name says, what you do is abstract the factory so you can make more concrete factories that mix different implementations of the concrete products.

This UML from Wikipedia sums it up fairly well:

Refactoring Guru uses a GUI example and I think is the best one for this pattern, so check it out:

They also have pseudocode that illustrates everything going on. This Python example from Wikipedia is also pretty good.

Dependency Injection

DI is one of those concepts that can be either too easy to understand or too hard, and by this I mean even people who use it constantly might not be able to fully explain the cycle of what's going on or how it works.

My intention is to give you a general understanding so you could dive further in, or at least be able to use DI in your projects.

The picture above is from this article. You don't have to read it, I just want to point out there's a lot going on. Additionally for a more in depth post on Inversion of Control I recommend this article from Martin Fowler (if you don't know who he is then you should definitely check out his blog).

Dependency Injection relies on the Inversion of Control principle.

IoC

As always one of the main problems when writing software is tight coupling, in OOP you get classes entangled with each other which makes it impossible to extend or modify. This issue is mainly because the Open-Closed Principle isn't followed.

We apply Inversion of Control to loosen and decouple our classes. Invert the control of dependencies. Instead of our current class controlling the creation of a new class we want to have something in between, a layer of abstraction that will have that control instead.

IoC isn't only applied to classes but I want to make this as simple as possible.

You can think about IoC as removing the dependency for further extension.

The picture above is from this article and has some examples.

DI Example

The theory is the same as with IoC, we want to avoid hard-coded dependencies so we can maintain our code in the future. Interfaces and design patterns exist, let's use them.

Knowing all of the above let's think about how to implement Dependency Injection: the main thing we have to do is to relinquish the control of creating an object we need somewhere else. We are still going to use the object of course so we need to store it, but let's not worry about how it gets made.

The best examples I could find are from Digital Ocean (Java) and Android (Kotlin, Java) so feel free to dig in, I liked the example from Android so I'm going to simplify it with TypeScript.

Constructor Injection: You pass the dependencies of a class to its constructor.

//dependency of car that we have extracted
class Engine {

    ignite(): void {
        console.log('vroom vroom')
    }
}

//car has the engine dependency
//but it's not in charge of constructing that dependency
//only of using it
class Car {
    //we still need a variable to store the engine
    engine: Engine

    constructor(engine: Engine) {
        this.engine = engine
    }

    //use of engine after getting its dependency
    start(): void {
        this.engine.ignite()
    }
}

//make a new engine
const engine = new Engine()
//INJECT the DEPENDENCY
const car = new Car(engine)

//do your business logic
car.start()

You should get a 'vroom vroom' when you run it!

Tekbeans

If you made it all the way here congratulations!

I've started working on a TS package called tekbeans, it's on npm and of course you can check the source code on Github. It's on very early stages, however if you want to dig deeper into IoC and DI with TS then feel free to play with it. The package will be heavily inspired on Spring as it's the most famous framework with DI that uses explicit beans I believe.

Fin

As a reminder, you can find the code to all examples on my Github in the patterns repository, I will probably keep adding interesting stuff there.

You can also follow me here or reach out on Twitter or maybe even Mastodon.

It's 4 AM right now as I'm finishing this article, so if it has helped you in any way let me know, insults are welcome as well since this is the internet.

There's an xkcd for everything!

If you are preparing for interviews or just studying DSA you will eventually reach graph problems and then dynamic programming. This post is the step before recursion, we are going to see how BFS and DFS algorithms work iteratively with examples in JavaScript since most people have touched it in one way or another.

Note that if this is the first time you hear about BFS and DFS I'd recommend you go through binary trees first before starting with graphs.

To understand how these algorithms work first we need to have a proper understanding of stack and queue data structures.

I will be using examples from Programiz to illustrate some of my points. I'd recommend you check their content if this post isn't clear enough. Structy is another great page with plenty of free resources to get started.

Pre-requirements

Stack

Stack follows the LIFO principle: Last In First Out. You can think of it as a pile of plates (or anything you have to stack), every new item you put on top of others, or better put: every new item is put on the previous item and in order to get the last item you just take it from the pile. However, only the last item is available to take out.

In this illustration from Programiz you can see that you keep pushing elements/items in the stack and you keep pushing new elements on top of each other and if you want to retrieve them then you need to retrieve the elements starting from the Last In because it's the First Out.

If you are not doing OOP or you need to implement the data structure itself a simple JavaScript use would be to just make an array:

const stack = []
//push element
stack.push(1)
//get last element
const lastElement = stack.pop()

Additionally stack usually has a method/function that allows you to peek at the last element:

//peek
const getLast = stack[stack.length-1]

Queue

Queue follows the FIFO principle: First In First Out. It follows the same principle as any queue, you get in line and wait until it's your turn, so if there are 5 people ahead of you and you are the number 6 then you need to wait for those 5 people to get in before you can go.

In this illustration from Programiz you can see two elements that get queued, the first to get in line (queue) is the first to get out.

The JavaScript implementation would be something like this:

const queue = []
//queue elements
queue.push(1) //queue: [1]
queue.push(2) //queue: [1,2]
queue.push(3) //queue: [1,2,3]
//dequeue elements
const firstEle = queue.shift() //this gets 1 and queue: [2,3]
const secondEle = queue.shift() //this gets 2nd and queue: [3]
const lastEle = queue.shift() //gets 3rd and queue: []

The main difference between stack and queue is that we don't pop any elements from the queue, the pop method/function will give us the last element but since we are working with queues we want to get the First In element which will go First Out.

You could also implement a queue in reverse order:

const queue = []
//queue elements
queue.unshift(1)
queue.unshift(2)
queue.unshift(3)
//dequeue elements
const first = queue.pop()
const second = queue.pop()
const third = queue.pop()

This is still a queue, the only difference is that we are inserting from the beginning (bottom) and popping the elements from the end (top).

Time complexity

Just a quick pause to remind you that if you use any methods/functions that have to move all array elements (unshift for example) this becomes an additional cost on your algorithm since moving all elements it's O(n) so generally speaking, if you have to choose you might want to try the stack data structure first since your elements inside of the array don't have to move.

Graphs

Preparing the data

When you are working with graphs the data/input for your function might be all over the place so you will need to organize it better or make it into something where you can easily apply the algorithms and data structures you are familiar with.

Before attempting to solve the problem you might want to first take a look at the data or input you are provided, and depending on what you need to do perhaps you should transform that data into something else, be it to avoid exponential runtime or just to see the problem better.

Just check if you might need a set, a hashmap or something else. With graph problems what we usually need is an adjacent matrix.

So for example, if we are given this LeetCode problem (1971. Find if Path Exists in Graph) where the input is as follows:

edges = [[0,1],[1,2],[2,0]]

We need to use a function to transform that array of edges into a matrix (I usually just call it getGraph since we want to get the graph from the array):

  const getGraph = (edges) => {
    const graph = {}

    for(let i = 0; i < edges.length; i++){
      let [a, b] = edges[i]
      if(!graph[a]) graph[a] = []
      if(!graph[b]) graph[b] = []
      graph[a].push(b)
      graph[b].push(a)
    }

    console.log(graph)
    return graph
  }

If you run that function you should get this output which corresponds to the picture of the graph above:

{ '0': [ 1, 2 ], '1': [ 0, 2 ], '2': [ 1, 0 ] }

Graph adjacent nodes (neighbours)

In order to understand how BFS and DFS work one of the keys is to realise that we need to visit all the adjacent nodes of your current node, aka the neighbours. It might sound obvious and simple but you can miss it.

We will use a stack or a queue to keep track of our current node and then push the neighbour nodes into our stack or queue. The most important concept here is that we are working on our current node, aka the node which we took off from our stack or queue. This will make sense in a bit.

And why do we care about the neighbours so much? To avoid running over the same nodes, what is the best data structure if we want to think about unique values? A set! You get to store unique values and then check very fast if you already interacted with a value/element/node before.

In JavaScript if you want to make a set you do:

//name our set visited because we will track our visited nodes here
const visited = new Set()

Now that we have our set if we are interacting over our current node to check the neighbours we just do a for loop:

for(let neighbour of graph[currentNode]){
    //check if our Set contains an adjacent node which means we already visited it
    if(!visited.has(neighbour) //do something
}

This would be the same loop:

for(let neighbourIndex = 0; neighbour < graph[currentNode].length; neighbour++){
    if(!visited.has(graph[neighbourIndex]) //do something
}

Now that we have the basic concepts down it should be easier to understand the implementation of both DFS and BFS. However, first I'd recommend you look up videos of what the algorithm does (without looking at the code). Here's a video from FreeCodeCamp and the creator of Structy, you don't have to look at the entire video just until you understand how DFS and BFS work visually:

DFS vs BFS

The best way to understand how DFS works is to think about its name, we are going to search in-depth first. This means that we will take our search in a specific direction until we hit a wall (or a null). If we were talking about trees DFS would just always go to the left or right node first, so basically we are just going to always go deep into one direction until we can't find anything.

On the other hand, if we are using BFS then before going in deeper we will look around us and check for any adjacent nodes and if they exist then we will visit those first. So instead of going deeper in one direction as with DFS here, we'd go more in a circular manner to find the neighbours first before committing to a specific direction.

I can't recommend this video enough to understand the core concepts. You can also visit the Structy page directly.

DFS implementation

Remember the previous LeetCode problem: (1971. Find if Path Exists in Graph)? We will solve it using DFS and everything we have learned so far.

Let's write a DFS algorithm to find a solution and check if a path exists between two nodes. I've included a lot of console.logs so you can see what's happening and how the algorithm is working, you can run it and check the different test cases for it.

I recommend that you edit the test cases and just go step by step, the most important part here is to understand how we are using the stack data structure and adding all the neighbours.

/**
 * @param {number} n
 * @param {number[][]} edges
 * @param {number} source
 * @param {number} destination
 * @return {boolean}
 */
var validPath = function(n, edges, source, destination) {
    const adj = getGraph(edges)
    console.log(adj)
    return hasPath(adj, source, destination)
};

const getGraph = (edges) => {
    console.log('inside getGraph')
    const graph = {}
    for(let i = 0; i < edges.length; i++){
        let [a, b] = edges[i]
        if(!graph[a]) graph[a] = []
        if(!graph[b]) graph[b] = []
        graph[a].push(b)
        graph[b].push(a)
    }
    console.log(graph)
    return graph
}

const hasPath = (graph, src, dst) => {
    console.log('inside haspath')
    let visited = new Set()
    let stack = [src]
    while(stack.length > 0){
        let curr = stack.pop()
        if(curr === dst) return true

        visited.add(curr)

        console.log(visited)
        console.log('stack: ')
        console.log(stack)
        console.log('curr:')
        console.log(curr)
        console.log('graph curr')
        console.log(graph[curr])
        for(let n of graph[curr]){
        console.log(n)
        if(!visited.has(n))stack.push(n) 
        }
    }
    return false
}

I recommend you go through the results on the console, in case you can't here's the console output for test case 2:

inside getGraph
{
  '0': [ 1, 2 ],
  '1': [ 0 ],
  '2': [ 0 ],
  '3': [ 5, 4 ],
  '4': [ 5, 3 ],
  '5': [ 3, 4 ]
}
{
  '0': [ 1, 2 ],
  '1': [ 0 ],
  '2': [ 0 ],
  '3': [ 5, 4 ],
  '4': [ 5, 3 ],
  '5': [ 3, 4 ]
}
inside haspath
Set(1) { 0 }
stack: 
[]
curr:
0
graph curr
[ 1, 2 ]
1
2
Set(2) { 0, 2 }
stack: 
[ 1 ]
curr:
2
graph curr
[ 0 ]
0
Set(3) { 0, 2, 1 }
stack: 
[]
curr:
1
graph curr
[ 0 ]
0

DFS algorithm

Here's the function without all the clutter and more comments, it's used to determine if a path exists between a source and destination node.

const hasPath = (graph, src, dst) => {
    //init set to check if we have been here already
    let visited = new Set()
    //put the source node in our stack since we are starting here
    let stack = [src]
    //as long as the stack isn't empty it means we are still checking nodes
    while(stack.length > 0){
        //we get our current node from the stack
        let curr = stack.pop()
        //check if the current node is our destination
        if(curr === dst) return true
        //since we already visited this node we add it to our set of visited nodes
        visited.add(curr)
        //if we didn't visit the neighbours of our current node then add them to the stack
        //where we check if the current node is the destination
        for(let n of graph[curr]){
            if(!visited.has(n)) stack.push(n) 
        }
    }
    //if we can't find the destination then return false
    return false
}

JavaScript shenanigans

function doSomething(a,b,c) {
}

is the same as

const doSomething = (a,b,c) => {
}

BFS algorithm

The main difference between DFS and BFS is how we get our current node (and therefore the way we store and iterate over the neighbours). With a stack you will always go in the same direction until the end of the graph while with a queue you will visit the nodes that are on the same "level" first.

Here's the modification of our current DFS algorithm into BFS:

const hasPath = (graph, src, dst) => {
    //init set to check if we have been here already
    let visited = new Set()
    //init queue with the source node in
    let queue = [src]
    //as long as the stack isn't empty it means we are still checking nodes
    while(queue.length > 0){
        //we get our current node from the queue
        let curr = queue.shift()
        //check if the current node is our destination
        if(curr === dst) return true
        //since we already visited this node we add it to our set of visited nodes
        visited.add(curr)
        //if we didn't visit the neighbours of our current node then add them to the queue
        //where we check if the current node is the destination
        for(let n of graph[curr]){
            if(!visited.has(n)) queue.push(n) 
        }
    }
    //if we can't find the destination then return false
    return false
}

Note that if we try to run it we will pass almost all the test cases except the ones with a large input, this is due to BFS being more costly since it's using a queue instead of a stack.

A common case for BFS use in graph problems is to find the shortest path from one node to another, you could use something like this BFS helper function to get the distance:

const bfsHelper = (graph, node, target, visited) => {
  //define queue which will store our current node and the distance count
  let queue = [[node, 0]]
  //iterate until we don't have any current nodes left
  while(queue.length > 0){
    //extract the current nude and current distance
    let [curr, distance] = queue.shift()
    //if our current node equals target then we are done with our algorithm and we return how many steps we had to take to find it 
    if(curr === target) return distance
    //if the current node isn't our target then we add the current node to a visited set
    visited.add(curr)
    //iterate over our adjacent neighbours to add them to the queue if we haven't visited them yet
    for(let neighbor of graph[curr]) {
      if(!visited.has(neighbor)) {
        queue.push([neighbor, distance+1])
      }
    }
  }
  //if we can't find the target then we return -1
  return -1
}

This is just the beginning

The objective of this post was to establish some key concepts and see how DFS and BFS behave as well as approach the graph problems, the next steps would be to dive into recursion or perhaps go back to binary tree problems.

It might take a while to understand the best approaches or how things work, just keep practising and it will click eventually. And if you are tired of LeetCode or HackerRank I highly recommend CodeWars.

Go is a powerful language that's highly performant and it has concurrency built in the language itself making it perfect for micro services. Here we will make a small API focusing on understanding HTTP and overall API design.
If you want to know more about Go concurrency look into how go routines and channels work.

Github repo & more

You can find all the code in my repository. I've been writing the article alongside the code so you can follow the commits. If you find a bug feel free to submit a PR.

You can also find me on LinkedIn, Twitter and Mastodon.

Target audience

This article is aimed at anyone who wants to build a CRUD micro service or a simple RESTful API using Go (or Golang for SEO purposes).

Additional resources

However if you are a junior dev or just starting out I'd recommend you check my Node.js (MERN) or Spring Boot series and if you are looking for more Go content I have a small example of a mini-twitter clone you can check. Additionally if you are after video tutorials I highly recommend Matt KØDVB's channel.

Tech

I'm on Go 1.18, for database we will use PostgreSQL and we will containerize our app with Docker and then use Docker Compose to link our app with the database. We will also include a small check for production environments in case you want to deploy it somewhere.

Libraries

We will be using Chi as our router and Go-PG in order to communicate with our PostgreSQL database.

Additional implementation

We will be focusing on the CRUD part in this article however in the real world you will also need to add authentication, authorization, logging and testing.

The Code

We will build a comments micro service, this means that you will have a CRUD API through which you will be able to create, read, update and delete comments.

Docker and Docker Compose

In order for the project to be able to run on any machine we will be using Docker and Docker Compose to help us set up the database, PostgreSQL in our case.

Start your project with

go mod init go-microservice-example

this way we can track all the packages we are going to be using.

Then add a folder called cmd, inside a folder called server and inside that folder add main.go, call the package main as well. This will be where we build our binary and start the server.

Inside main.go paste the following:

package main

import (
    "log"
)

func main() {
    log.Print("server has started")
}

Right now we are just making sure our program works with Docker so we are just printing one line.

Now add the Dockerfile (remember it doesn't have an extension):

FROM golang

RUN mkdir /app

ADD . /app

WORKDIR /app

RUN go build -o main ./cmd/server/main.go

EXPOSE 8080
CMD [ "/app/main" ]

What we are doing here is basically creating an ./app folder inside of our container, making it the work directory, building the binary, exposing the port and then calling the binary we just built so it can run our code.

Now since we are using PostgreSQL we will add our database using the Docker Compose file, so add docker-compose.yml with the following inside:

version: '3.8'

services:
  db:
    image: postgres
    restart: always
    environment:
      POSTGRES_PASSWORD: admin
  api:
    build: .
    ports:
      - 8080:8080
    environment:
      - PORT=8080
      - DATABASE_URL=db
    depends_on:
      - db

Here we are declaring that we will be running 2 services, db and api, with api depending on db and making sure the ports are properly exposed: we will be working with the 8080 port. Additionally I've also added two environment variables that we will use later: PORT and DATABASE_URL.

With everything set up you can start your app with docker compose by typing:

docker compose up --build

You might see that the api service starts before the database, this isn't a bug. Just shut it down with CTRL+C and restart it with:

docker compose down; docker compose up --build

If everything went correctly you should see the message from the log:

Migrations

Before diving into the code we need to set up our database. Make a migrations folder.

This is the first migration to set up users, 1_users.up.sql:

CREATE TABLE IF NOT EXISTS users (
    id SERIAL PRIMARY KEY,
    name VARCHAR NOT NULL
);

And this is the second for comments, 2_comments.up.sql:

CREATE TABLE IF NOT EXISTS comments (
    id SERIAL PRIMARY KEY,
    comment VARCHAR NOT NULL,
    comment_date DATE DEFAULT CURRENT_DATE,
    user_id BIGINT REFERENCES users(id)
);

INSERT INTO users(name) VALUES('dev_test_user');
INSERT INTO comments (comment) VALUES('first test comment');

Connecting to the DB

Let's get Go-PG with

go get github.com/go-pg/pg/v10

and since we are using migrations let's get the migrations package as well

go get github.com/go-pg/migrations/v8

Create a db.go file now inside pkg/db folder

This should go inside your db.go file:

package db

import (
    "log"
    "os"

    "github.com/go-pg/migrations/v8"
    "github.com/go-pg/pg/v10"
)

func StartDB() (*pg.DB, error) {
    var (
        opts *pg.Options
        err  error
    )

    //check if we are in prod
    //then use the db url from the env
    if os.Getenv("ENV") == "PROD" {
        opts, err = pg.ParseURL(os.Getenv("DATABASE_URL"))
        if err != nil {
            return nil, err
        }
    } else {
        opts = &pg.Options{
            //default port
            //depends on the db service from docker compose
            Addr:     "db:5432",
            User:     "postgres",
            Password: "admin",
        }
    }

    //connect db
    db := pg.Connect(opts)
    //run migrations
    collection := migrations.NewCollection()
    err = collection.DiscoverSQLMigrations("migrations")
    if err != nil {
        return nil, err
    }

    //start the migrations
    _, _, err = collection.Run(db, "init")
    if err != nil {
        return nil, err
    }

    oldVersion, newVersion, err := collection.Run(db, "up")
    if err != nil {
        return nil, err
    }
    if newVersion != oldVersion {
        log.Printf("migrated from version %d to %d\n", oldVersion, newVersion)
    } else {
        log.Printf("version is %d\n", oldVersion)
    }

    //return the db connection
    return db, err
}

Both libraries have docs and examples, so check them out, for example this is for migrations.

Setting up your API

Now that we have our database set up let's start working on the API, get Chi with the following command:

go get -u github.com/go-chi/chi/v5

and create a folder called api inside the pkg directory, then add an api.go file:

To get started let's just make sure our Chi router works so let's make two routes, "/" and "/comments/" inside api.go:

package api

import (
    "net/http"

    "github.com/go-chi/chi/v5"
    "github.com/go-chi/chi/v5/middleware"
    "github.com/go-pg/pg/v10"
)

//start api with the pgdb and return a chi router
func StartAPI(pgdb *pg.DB) *chi.Mux {
    //get the router
    r := chi.NewRouter()
    //add middleware
    //in this case we will store our DB to use it later
    r.Use(middleware.Logger, middleware.WithValue("DB", pgdb))

    r.Route("/comments", func(r chi.Router) {
        r.Get("/", getComments)
    })

    r.Get("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("up and running"))
    })

    return r
}

func getComments(w http.ResponseWriter, r *http.Request){
    w.Write([]byte("comments"))
}

Afterwards go back to cmd/server/main.go:

package main

import (
    "fmt"
    "go-microservice-example/pkg/api"
    "go-microservice-example/pkg/db"
    "log"
    "net/http"
    "os"
)

func main() {
    log.Print("server has started")
    //start the db
    pgdb, err := db.StartDB()
    if err != nil {
        log.Printf("error starting the database %v", err)
    }
    //get the router of the API by passing the db
    router := api.StartAPI(pgdb)
    //get the port from the environment variable
    port := os.Getenv("PORT")
    //pass the router and start listening with the server
    err = http.ListenAndServe(fmt.Sprintf(":%s", port), router)
    if err != nil {
        log.Printf("error from router %v\n", err)
    }
}

Note how we are using the functions we have made from our DB and API packages: StartDB and StartAPI.

Now start the app with docker by doing

docker compose up --build

If you navigate to localhost:8080/ you should see "up and running" and if you navigate to localhost:8080/comments you should see "comments":

On the terminal you should be seeing the GET requests as well:

Now that we have set up everything let's dig into the logic, remember that there's no right answer for the structure of a project so if you don't like what I'm doing feel free to change it.

Create and Read operations

Let's write the basic POST (Create) and GET (Read) operations for our API. I recommend you go through every line and check what's going on, as well as the PG and CHI libraries.

First need to define how our app is going to communicate with the database, if you are coming from working with other frameworks or libraries you might think about this as doing our models (schemas) and making a repository - this basically means we will make a data struct (think of a POJO if you are coming from Java) that we will use to communicate with our database.

Make a models folder inside db then create two files:

This is inside user.go:

package models

type User struct {
    ID   int64  `json:"id"`
    Name string `json:"name"`
}

Since we are not modifying the user we don't have any logic inside.

This is our comment.go:

package models

import "github.com/go-pg/pg/v10"

type Comment struct {
    ID      int64  `json:"id"`
    Comment string `json:"comment"`
    UserID  int64  `json:"user_id"`
    User    *User  `pg:"rel:has-one" json:"user"`
}

func CreateComment(db *pg.DB, req *Comment) (*Comment, error) {
    _, err := db.Model(req).Insert()
    if err != nil {
        return nil, err
    }

    comment := &Comment{}

    err = db.Model(comment).
        Relation("User").
        Where("comment.id = ?", req.ID).
        Select()

    return comment, err
}

func GetComment(db *pg.DB, commentID string) (*Comment, error) {
    comment := &Comment{}

    err := db.Model(comment).
        Relation("User").
        Where("comment.id = ?", commentID).
        Select()

    return comment, err
}

func GetComments(db *pg.DB) ([]*Comment, error) {
    comments := make([]*Comment, 0)

    err := db.Model(&comments).
        Relation("User").
        Select()

    return comments, err
}

The logic is fairly simple, just make sure you understand the relation and how the library works. The ORM is making the SQL queries for us using the values we pass.

Now let's go back to our API folder and use all the functions we have made. Remember that we need to import the models package first. Then, in order to get the request or response from the server let's capture it through a struct (CommentResponse and CommentRequest) however be careful because first we need to decode and encode the data every time we send it back or receive it.

This is our current api.go file:

package api

import (
    "encoding/json"
    "go-microservice-example/pkg/db/models"
    "log"
    "net/http"

    "github.com/go-chi/chi/v5"
    "github.com/go-chi/chi/v5/middleware"
    "github.com/go-pg/pg/v10"
)

//start api with the pgdb and return a chi router
func StartAPI(pgdb *pg.DB) *chi.Mux {
    //get the router
    r := chi.NewRouter()
    //add middleware
    //in this case we will store our DB to use it later
    r.Use(middleware.Logger, middleware.WithValue("DB", pgdb))

    //routes for our service
    r.Route("/comments", func(r chi.Router) {
        r.Post("/", createComment)
        r.Get("/", getComments)
    })

    //test route to make sure everything works
    r.Get("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("up and running"))
    })

    return r
}

type CreateCommentRequest struct {
    Comment string `json:"comment"`
    UserID  int64  `json:"user_id"`
}

type CommentResponse struct {
    Success bool            `json:"success"`
    Error   string          `json:"error"`
    Comment *models.Comment `json:"comment"`
}

func createComment(w http.ResponseWriter, r *http.Request) {
    //get the request body and decode it
    req := &CreateCommentRequest{}
    err := json.NewDecoder(r.Body).Decode(req)
    //if there's an error with decoding the information
    //send a response with an error
    if err != nil {
        res := &CommentResponse{
            Success: false,
            Error:   err.Error(),
            Comment: nil,
        }
        err = json.NewEncoder(w).Encode(res)
        //if there's an error with encoding handle it
        if err != nil {
            log.Printf("error sending response %v\n", err)
        }
        //return a bad request and exist the function
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    //get the db from context
    pgdb, ok := r.Context().Value("DB").(*pg.DB)
    //if we can't get the db let's handle the error
    //and send an adequate response
    if !ok {
        res := &CommentResponse{
            Success: false,
            Error:   "could not get the DB from context",
            Comment: nil,
        }
        err = json.NewEncoder(w).Encode(res)
        //if there's an error with encoding handle it
        if err != nil {
            log.Printf("error sending response %v\n", err)
        }
        //return a bad request and exist the function
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    //if we can get the db then
    comment, err := models.CreateComment(pgdb, &models.Comment{
        Comment: req.Comment,
        UserID:  req.UserID,
    })
    if err != nil {
        res := &CommentResponse{
            Success: false,
            Error:   err.Error(),
            Comment: nil,
        }
        err = json.NewEncoder(w).Encode(res)
        //if there's an error with encoding handle it
        if err != nil {
            log.Printf("error sending response %v\n", err)
        }
        //return a bad request and exist the function
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    //everything is good
    //let's return a positive response
    res := &CommentResponse{
        Success: true,
        Error:   "",
        Comment: comment,
    }
    err = json.NewEncoder(w).Encode(res)
    if err != nil {
        log.Printf("error encoding after creating comment %v\n", err)
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    w.WriteHeader(http.StatusOK)
}

type CommentsResponse struct {
    Success  bool              `json:"success"`
    Error    string            `json:"error"`
    Comments []*models.Comment `json:"comments"`
}

func getComments(w http.ResponseWriter, r *http.Request) {
    //get db from ctx
    pgdb, ok := r.Context().Value("DB").(*pg.DB)
    if !ok {
        res := &CommentsResponse{
            Success:  false,
            Error:    "could not get DB from context",
            Comments: nil,
        }
        err := json.NewEncoder(w).Encode(res)
        if err != nil {
            log.Printf("error sending response %v\n", err)
        }
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    //call models package to access the database and return the comments
    comments, err := models.GetComments(pgdb)
    if err != nil {
        res := &CommentsResponse{
            Success:  false,
            Error:    err.Error(),
            Comments: nil,
        }
        err := json.NewEncoder(w).Encode(res)
        if err != nil {
            log.Printf("error sending response %v\n", err)
        }
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    //positive response
    res := &CommentsResponse{
        Success:  true,
        Error:    "",
        Comments: comments,
    }
    //encode the positive response to json and send it back
    err = json.NewEncoder(w).Encode(res)
    if err != nil {
        log.Printf("error encoding comments: %v\n", err)
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    w.WriteHeader(http.StatusOK)
}

I've left all the error handlers so you know exactly what's going on. Rebuild the Docker containers and start the app using

docker compose up --build

CURL

We will be using CURL to make sure everything works, so include this request.json in the root of your project:

{
    "comment": "test comment",
    "user_id": 1
}

Note that if you are on Windows and having some issues with the command just do:

Remove-item alias:curl

Now open the terminal inside your project and type:

curl -X POST localhost:8080/comments -d "@request.json" | jq

This will make a POST request to the /comments route and the function createComment will handle the rest. You should get this response if everything went correctly:

Now to make sure the comment is stored let's make a GET request via:

curl -X GET localhost:8080/comments

And you should get back the comments (or in this case just one) stored in your database:

If you are using VSCode you can separate the terminal which comes in handy when you are running a server, this is what my current terminal looks like:

Now that we know everything works go to request.json and delete the "comment" line then try to POST it again, you should see the following error:

DRY

DRY stands for "don't repeat yourself" so let's refactor a bit our api.go and introduce the error handling functions:

package api

import (
    "encoding/json"
    "go-microservice-example/pkg/db/models"
    "log"
    "net/http"

    "github.com/go-chi/chi/v5"
    "github.com/go-chi/chi/v5/middleware"
    "github.com/go-pg/pg/v10"
)

//start api with the pgdb and return a chi router
func StartAPI(pgdb *pg.DB) *chi.Mux {
    //get the router
    r := chi.NewRouter()
    //add middleware
    //in this case we will store our DB to use it later
    r.Use(middleware.Logger, middleware.WithValue("DB", pgdb))

    //routes for our service
    r.Route("/comments", func(r chi.Router) {
        r.Post("/", createComment)
        r.Get("/", getComments)
    })

    //test route to make sure everything works
    r.Get("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("up and running"))
    })

    return r
}

// -- Responses

type CreateCommentRequest struct {
    Comment string `json:"comment"`
    UserID  int64  `json:"user_id"`
}

type CommentResponse struct {
    Success bool            `json:"success"`
    Error   string          `json:"error"`
    Comment *models.Comment `json:"comment"`
}

type CommentsResponse struct {
    Success  bool              `json:"success"`
    Error    string            `json:"error"`
    Comments []*models.Comment `json:"comments"`
}

//-- UTILS --

func handleErr(w http.ResponseWriter, err error) {
    res := &CommentResponse{
        Success: false,
        Error:   err.Error(),
        Comment: nil,
    }
    err = json.NewEncoder(w).Encode(res)
    //if there's an error with encoding handle it
    if err != nil {
        log.Printf("error sending response %v\n", err)
    }
    //return a bad request and exist the function
    w.WriteHeader(http.StatusBadRequest)
}

func handleDBFromContextErr(w http.ResponseWriter) {
    res := &CommentResponse{
        Success: false,
        Error:   "could not get the DB from context",
        Comment: nil,
    }
    err := json.NewEncoder(w).Encode(res)
    //if there's an error with encoding handle it
    if err != nil {
        log.Printf("error sending response %v\n", err)
    }
    //return a bad request and exist the function
    w.WriteHeader(http.StatusBadRequest)
}

// -- handle routes

func createComment(w http.ResponseWriter, r *http.Request) {
    //get the request body and decode it
    req := &CreateCommentRequest{}
    err := json.NewDecoder(r.Body).Decode(req)
    //if there's an error with decoding the information
    //send a response with an error
    if err != nil {
        handleErr(w, err)
        return
    }
    //get the db from context
    pgdb, ok := r.Context().Value("DB").(*pg.DB)
    //if we can't get the db let's handle the error
    //and send an adequate response
    if !ok {
        handleDBFromContextErr(w)
        return
    }
    //if we can get the db then
    comment, err := models.CreateComment(pgdb, &models.Comment{
        Comment: req.Comment,
        UserID:  req.UserID,
    })
    if err != nil {
        handleErr(w, err)
        return
    }
    //everything is good
    //let's return a positive response
    res := &CommentResponse{
        Success: true,
        Error:   "",
        Comment: comment,
    }
    err = json.NewEncoder(w).Encode(res)
    if err != nil {
        log.Printf("error encoding after creating comment %v\n", err)
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    w.WriteHeader(http.StatusOK)
}

func getComments(w http.ResponseWriter, r *http.Request) {
    //get db from ctx
    pgdb, ok := r.Context().Value("DB").(*pg.DB)
    if !ok {
        handleDBFromContextErr(w)
        return
    }
    //call models package to access the database and return the comments
    comments, err := models.GetComments(pgdb)
    if err != nil {
        handleErr(w, err)
        return
    }
    //positive response
    res := &CommentsResponse{
        Success:  true,
        Error:    "",
        Comments: comments,
    }
    //encode the positive response to json and send it back
    err = json.NewEncoder(w).Encode(res)
    if err != nil {
        log.Printf("error encoding comments: %v\n", err)
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    w.WriteHeader(http.StatusOK)
}

Get Comment By Id

We already have the function to get a comment by id in our models package, let's add a route and a function to handle that route in our API package now:

        // here's the route
        r.Get("/{commentID}", getCommentByID)

And here's the function to handle it:

func getCommentByID(w http.ResponseWriter, r *http.Request) {
    //get the id from the URL parameter
    //alternatively you could use a URL query
    commentID := chi.URLParam(r, "commentID")

    //get the db from ctx
    pgdb, ok := r.Context().Value("DB").(*pg.DB)
    if !ok {
        handleDBFromContextErr(w)
        return
    }

    //get the comment from the DB
    comment, err := models.GetComment(pgdb, commentID)
    if err != nil {
        handleErr(w, err)
        return
    }

    //if the retrieval from the db was successful send the data
    res := &CommentResponse{
        Success: true,
        Error:   "",
        Comment: comment,
    }
    //encode the positive response to json and send it back
    err = json.NewEncoder(w).Encode(res)
    if err != nil {
        log.Printf("error encoding comments: %v\n", err)
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    w.WriteHeader(http.StatusOK)
}

PUT

Now that we have READ and POST let's focus on updating our data by handling a PUT request. First let's edit our models package (comment.go) by adding this function:

func UpdateComment(db *pg.DB, req *Comment) (*Comment, error) {
    _, err := db.Model(req).
        WherePK().
        Update()
    if err != nil {
        return nil, err
    }

    comment := &Comment{}

    err = db.Model(comment).
        Relation("User").
        Where("comment.id = ?", req.ID).
        Select()

    return comment, err
}

Then let's add the route inside the API package (api.go):

r.Put("/{commentID}", updateCommentByID)

And finally let's write the function that handles the request, gets it from the user then interacts with the database by updating the data:

func updateCommentByID(w http.ResponseWriter, r *http.Request) {
    //get the data from the request
    req := &CommentRequest{}
    //decode the data
    err := json.NewDecoder(r.Body).Decode(req)
    if err != nil {
        handleErr(w, err)
        return
    }
    pgdb, ok := r.Context().Value("DB").(*pg.DB)
    if !ok {
        handleDBFromContextErr(w)
        return
    }
    //get the commentID to know what comment to modify
    commentID := chi.URLParam(r, "commentID")
    //we get a string but we need to send an int so we convert it
    intCommentID, err := strconv.ParseInt(commentID, 10, 64)
    if err != nil {
        handleErr(w, err)
        return
    }

    //update the comment
    comment, err := models.UpdateComment(pgdb, &models.Comment{
        ID:      intCommentID,
        Comment: req.Comment,
        UserID:  req.UserID,
    })
    if err != nil {
        handleErr(w, err)
    }
    //return successful response
    res := &CommentResponse{
        Success: true,
        Error:   "",
        Comment: comment,
    }
    //send the encoded response to responsewriter
    err = json.NewEncoder(w).Encode(res)
    if err != nil {
        log.Printf("error encoding comments: %v\n", err)
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    //send a 200 response
    w.WriteHeader(http.StatusOK)
}

Note that we have to convert the string commentID into the int ID, there's a difference between receiving data as json and how we handle the data inside structs, that's why inside structs you have things like

type Comment struct {
    ID      int64  `json:"id"`
    Comment string `json:"comment"`
    UserID  int64  `json:"user_id"`
    User    *User  `pg:"rel:has-one" json:"user"`
}

For example the ID is how we name the commentID in our codebase but if we send or receive that ID through JSON then we will just name it "id".

Now modify your request.json (change the comment data) and use this command to send a PUT request:

curl -X PUT localhost:8080/comments/1 -d "@request.json" | jq

If everything went well you should see this in your terminal:

SuccResponse DRY

Since our success response is the same everywhere let's refactor it to follow the DRY principle, we will call this function succ because I'm childish, so refactor the code in our API package:

func succCommentResponse(comment *models.Comment, w http.ResponseWriter) {
    //return successful response
    res := &CommentResponse{
        Success: true,
        Error:   "",
        Comment: comment,
    }
    //send the encoded response to responsewriter
    err := json.NewEncoder(w).Encode(res)
    if err != nil {
        log.Printf("error encoding comment: %v\n", err)
        w.WriteHeader(http.StatusBadRequest)
        return
    }
    //send a 200 response
    w.WriteHeader(http.StatusOK)
}

Delete

Let's finish this, go to your models package and add the Delete function:

func DeleteComment(db *pg.DB, commentID int64) error {
    comment := &Comment{}

    err := db.Model(comment).
        Relation("User").
        Where("comment.id = ?", commentID).
        Select()
    if err != nil {
        return err
    }

    _, err = db.Model(comment).WherePK().Delete()

    return err
}

And inside the API package add the route and the function to handle that route:

r.Delete("/{commentID}", deleteCommentByID)

func deleteCommentByID(w http.ResponseWriter, r *http.Request) {
    //parse in the req body
    req := &CommentRequest{}
    err := json.NewDecoder(r.Body).Decode(req)
    if err != nil {
        handleErr(w, err)
        return
    }

    //get the db from ctx
    pgdb, ok := r.Context().Value("DB").(*pg.DB)
    if !ok {
        handleDBFromContextErr(w)
        return
    }

    //get the commentID
    commentID := chi.URLParam(r, "commentID")
    intCommentID, err := strconv.ParseInt(commentID, 10, 64)
    if err != nil {
        handleErr(w, err)
        return
    }

    //delete comment
    err = models.DeleteComment(pgdb, intCommentID)
    if err != nil {
        handleErr(w, err)
    }

    //send successful response
    succCommentResponse(nil, w)
}

There are huge codebases of RoR from famous companies such as Github, Gitlab, Airbnb, Shopify, Groupon and even Twitch that are still using it. However the ecosystem is definitely in decline and companies are struggling to recruit senior developers.

Overall taking the approach of saying "tech X is dead" has never worked - and be cautious of whoever tells you otherwise. In the web dev world we have been hearing about how PHP has been dead for a while yet there are still millions of jobs and the ecosystem keeps evolving with amazing frameworks like Laravel. There are still jobs for COBOL devs FFS, so no, Ruby on Rails isn't dead but it's harder to get in as a beginner compared to other frameworks and languages although I consider it a great starting full stack framework, let me elaborate.

It's a pain to start (on Windows)

As a developer you should be using Linux, of course, but I'm on my Windows machine right now and it has been an absolute hellish experience, so as DX goes RoR takes a big L from the start if you are a Windows user.

Let me help you a bit with the installation.

How to install Ruby on Rails on Windows

There are guides out there but the best resource to not run into any compatibility or dependency issues is just to follow this 27 minutes video from FreeCodeCamp and you should be good to go. No, it's not a joke.

Alternatively you can use WSL2 with Docker. Good luck!

How to switch between Ruby on Rails versions on Windows

If you happily installed the latest version of ruby and have decided to fork a project that uses an older version, well, it's not compatible so tough luck. Luckily you can use a 12 years old no longer maintained project called PIK to switch Ruby versions. Again, not a joke.

This article has been very helpful and it guides you through the whole process. Overall Onghu's blog seems very helpful if you are a Ruby developer.

Good for beginners

A fairly famous site to get into Web Development called The Odin Project teaches Ruby on Rails to beginners, while it might seem overwhelming at first I believe RoR will teach anyone good foundations to understand how a full stack application works, let's see why.

Funnily enough even The Odin Project doesn't have a guide on how to install RoR on Windows.

MVC

Ruby on Rails allows you to develop a modular monolith following the MVC pattern. It's a heavily opinionated framework that will make you understand MVC and stick to it, you want it or not, enforcing the good practices - something that's excellent for a beginner.

image source

You will understand how everything is interconnected inside your application: anything data related is handled by your Model, the controller is there to apply business logic to the data you extract through the Model and the View is there to display and get the data from the user.

I feel as if I started as a developer with RoR I'd have an easier time understanding how a full stack application works (or software in general).

Double edged sword of abstraction

As someone who just saw a glimpse of RoR, I feel like the framework can abstract too much away or have too many layers if you just want to jump in and start working. This means that you need to read and study a bit before getting your hands dirty in code. For me this is another negative point when it comes to developer experience.

I don't mean that RoR is bad but it is harder to get started than a lot of other frameworks which is something that might discourage developers from learning it, at the end of the day you want to build your project for fun, not read outdated docs and posts just to get started.
Which is another issue: you have a lot of posts and answers on SO that are fairly old and while they might be helpful it's confusing for newer devs as things have changed since then.

I believe it's not easy to start but once you put enough time you will get excellent results as the abstraction and all the built in features allow you to make a CRUD app blazingly fast.
You won't have microservices but instead you will have a solid modular monolith, which I'd say in a lot of cases is far better.

Get weird with Ruby

After using Ruby for a bit the adjective that comes to my mind is fun. Depending who you are or how you work it might be the best or the worst thing ever. Here's a Stackoverflow post asking for a difference between && vs and to illustrate my point.

Image source

Ruby is a language that allows you to write code however you want, there are various ways of doing the same thing which might sound wonderful BUT if you are working in a team your beautiful one-liner or the way you did something might confuse your less experienced co-workers.

Overall I think Ruby is amazing and it seems to allow a lot of freedom but there's a cost to having so many choices: your team needs to follow and enforce a particular code style.
If you want to dig a bit deeper then check this article about closures in Ruby.

RoR gets magically fast

Once you learn a bit of the framework you can make a full CRUD app in a few hours thanks to the generate command which allows you to make all the files you'd need for your CRUD operations, including routes and testing.
Here's an article that dives a bit in scaffolding with the generate command and keep in mind all these files will be created inside an already well structured project so all you have to do is write a bit of logic and you are good to go.

Additionally you have partials and concerns, to sum it up very poorly (the links explain it much better):

• partials: view components that you can move around
• concerns: mixins that allows you dependency injection

If you are unfamiliar with the terms mixin or DI, this article explains concerns a bit more in detail.
I like that the second paragraph is as follows:
"The Rails API documentation doesn't help, either. It starts with the assumption that the reader is already familiar with the problem concerns are trying to solve, and goes on to provide the solution, only adding to the confusion"

It really illustrates the struggles of someone who is new to the ecosystem and one of the biggest negative points about RoR right now.

Pros and Cons

Ruby on Rails is great if

• You are a beginner and want to learn fundamentals of a full stack app
• You want to make a fast CRUD app with a bit of time investment beforehand
• You are a senior software engineer who wants to transition into a new ecosystem

Ruby on Rails is bad if

• You have services where performance is extremely important

Let me quote a post from Gtilab: "while Rails is excellent technology for our purposes, it does have a few drawbacks, one of them being performance. Luckily, only a tiny part of most codebases is actually performance critical. We use our own gitaly daemon written in Go to handle actual git operations, and PostgreSQL for non-repository persistence".

However the biggest con of Ruby on Rails is its lack of support in terms of documentation, up to date tutorials and overall DX.
If the RoR community doesn't put more effort into its ecosystem and training junior developers or anyone coming from other frameworks it will suffer a slow and agonizing death where top companies will fight for a small portion of talent and others will try to rewrite their codebase by any means necessary due to lack of RoR developers.

Fin

If you have enjoyed this article or you want to scream at me because I'm wrong feel free to leave a comment, subscribe for more or hit me up on Twitter @tekbog.

PS: if you want to learn Ruby on Rails and don't know where to start then head to their page and click on Getting Started.

Here is the source code of the project from my Github.

This post is part of the MERN series, if you are lost or don't feel confident I'd recommend you read through the series to get familiar with NodeJS and Express. However you will find additional resources and links throughout the article.

If you want to yell at me you can find me on Twitter @tekbog.

Auth

What some people might call AUTH in reality is authentication and authorization. To sum it up I like to think about it this way:

Authentication: who are you?

Authorization: what can you do?

Authentication is about knowing the identity of the user while authorization is knowing what action the user is allowed to do, I'm paraphrasing from this .NET article.

If you still want to read more or a comparison check out this auth0 article.

Protect the password

In order to make our database secure for our users we will deal with the passwords by hashing and salting them. So bad agents can't access your user's data like pic related.

I'd recommend you read more about these concepts but here's a very quick summary that define them fairly well and you can also watch this video from ByteByteGo.

Hashing: map data of arbitrary size to fixed-size values

Salting: random data that is used as an additional input to a one-way function that hashes data

We are going to put user's data through a hash function with salt to make it secure. In order to do that we will use the bcrypt.js library.

Tokens

Picture source

For us to identify (authenticate) the user every time the user wants to do something (authorize) we will be using JWT (JSON Web Token).

Picture source

We generate a new token for every user that registers and store it in the database, afterwards we verify that the token is correct by comparing it with the same secret string we used to make the token.

So, to simplify:

1. User registers or logs in and gets a token
2. The token is stored in a cookie in our browser
3. Every time the user wants to do something the user sends the token
4. Every time the server gets a request from the user it checks if the token is correct

By using tokens the user doesn't have to log in every time, however the tokens expire after a while so from time to time we need to log in again.

We will see how tokens work and how to use them by storing them in a cookie.

To use JWT with NodeJS we will be using jsonwebtoken and to read (parse) cookies we will be using cookie-parser

Disclaimer

Before we start keep in mind this isn't a final product and it's just a tutorial (not ready for prod), if you are going to be implementing auth in production check passport.js and of course remember Helmet.
Additionally remember to ask your users for a strong password and make sure you don't make users without the right parameters.

The Project

Setup

For testing the REST API I will be using Postman and as the database I'm running Mongodb locally, however you can use Atlas or Docker.
For local variables I will use an .env file through the package dotenv. You can store your secrets there and afterwards use them inside your NodeJS project by typing process.env.YOUR_VAR.

If you are confused by any of this I'd recommend you start with the first entry of this series that you can find here. Or pick one from my MERN series.

Dependencies

Initialize the project with

npm init

Let's install everything we will need:

npm i bcryptjs cookie-parser dotenv express jsonwebtoken mongoose

Add the dev dependency

npm -D nodemon

Finally modify your package.json scripts like this:

  "scripts": {
    "start": "node index.js",
    "dev": "nodemon index.js"
  }

Now you can do npm start or npm run dev, if you are not familiar with nodemon then just know that it reloads the server with every change you make.

Project structure

NB: Before you freak out keep in mind I've separated the files a lot so it would be easier to navigate, there's no unique way that every project has to be structured so just do whatever works for you as long as it makes sense: some people prefer more files and imports/exports while others prefer to keep one file with all the logic inside.

Additionally keep in mind instead of routes you can have a folder with controllers and no services, I think it depends a lot on your background and what you are used to. This structure works for me, however it shouldn't work for you or your organization.

Structure TL;DR

API folder: this is where our API is going to be aka our endpoints, I've made testroute.js so you start by getting familiar with imports/exports and making sure the server runs and you get familiar with express.Router().

Then inside api/session-exporer we will have a router with two routes to explore what tokens and cookies are.

Afterwards there's a middleware folder with middleware to check cookies and verify tokens as well as to check if the logged user is an admin.

The last auth folder is where our REST api is located, we have one router and inside the routes folder you will find the different routes/endpoints. However the logic is stored inside servicess folder which has a subfolder for utils functions.

The last folder I haven't mentioned is models that contains our mongoose schema.

.env

One of the first things we should do is generate the signature key for our server, we can it with this command:

node -e "console.log(require('crypto').randomBytes(256).toString('base64'));"

Then store the result in our .env file (which is included in .gitignore)

PORT=5000
TOKEN_KEY=8bf0698c0908f00edf6b4efeb837cd52baef165b9029c7a06648ef23e0a0fdfda1cdd758e67597480f74424793848596

Database

Note that you might need to modify this, depending where your database is located but overall your db.js file should look something likes this:

const mongoose = require("mongoose");
const localDB = "mongodb://localhost:27017/auth";

const connectDB = () => {
  try {
    mongoose.connect(localDB, {
        useNewUrlParser: true,
        useUnifiedTopology: true,
    });
    console.log("MongoDB Connected...");
  } catch (err) {
    console.error(err.message);
    process.exit(1);
  }
};

module.exports = connectDB;

Entry point: index.js

This is the file Node runs when you start the server

//interact with our .env file
require("dotenv").config();
// --- routers ---
const authRouter = require("./api/auth/router");
const sessionExplorerRouter = require('./api/session-explorer/router');
const { routerTest } = require('./api/testroute');

//get our db connection
const connectDB = require("./db");
//import express package
const express = require("express");
//middleware to read cookies
//http://expressjs.com/en/resources/middleware/cookie-parser.html
const cookieParser = require('cookie-parser');

//get our express app
const app = express();
//get port from our .env file
const PORT = process.env.PORT || 5000;

// -- MIDDLEWARE
app.use(express.json());
app.use(cookieParser());
// -- MIDDLEWARE

// -- ROUTES --

//hello world on: http://localhost:5000/
app.get("/", (req, res) => res.send("Hello World!"));

//make sure everything is set up correctly
//http://localhost:5000/test
app.use('/test', (req, res) => res.send("from test!"));

//router test to make sure you know how to use middleware and imports/exports
//call example: http://localhost:5000/api/cookie
app.use('/api', routerTest);

//session explorer endpoints to understand cookies and jwt
app.use('/api', sessionExplorerRouter);

//get the router from api/auth/router and use the endpoints (routes) from api/auth/routes
//call example: http://localhost:5000/api/auth/login
app.use('/api/auth', authRouter);

// -- ROUTES --

//connect to db
connectDB();

//start listening on the designed port
app.listen(PORT, () => console.log(`Example app listening on port ${PORT}!`));


// The unhandledRejection listener
//The unhandledRejection event is emitted whenever a promise rejection is not handled. 
//NodeJS warns the console about UnhandledPromiseRejectionWarning and immediately terminates the process. 
//The NodeJS process global has an unhandledRejection event.
process.on('unhandledRejection', error => {
    console.error('unhandledRejection', error);
});

User schema

Inside models/user.js you will find the Mongoose schema that we will use to interact with MongoDB. You make a schema for your data, export it and then you can interact with the data in your database through Mongoose predefined methods like User.findById().

The most complicated thing we will be doing is updating the database, if you are unfamiliar with the term atomic check out this article.

const mongoose = require('mongoose')

const UserSchema = new mongoose.Schema({
    username: {
        type: String,
        required: true,
        unique: true
    },
    password: {
        type: String,
        required: true,
        minlength: 6,
    },
    role: {
        type: String,
        default: 'Basic',
        required: true
    },
    token: {
        type: String
    }
})

const User = mongoose.model('User', UserSchema)
module.exports = User;

Services

Before going into our API let's check our logic that lives inside the services folder. Each service gets exported so it's easier to use inside our API folder, however this is on the over-complicated side.
Depending on your project you could just add everything into your routes/controllers or perhaps you'd want to add responses to services as well and break it into more folders or files.
As I said the file structure and overall organization depends on you, this is what I found useful for this project. Later you will see how we just import our authService inside our API folder and use the various functions (if you are familiar with OOP you can consider the auth service as our main object and functions the methods that we call after initializing the object through an import).

authService.js contains the main chunk of logic however we also have a small token.js file inside our utils folder.

authService.js:

const User = require('../models/user')
const bcrypt = require('bcryptjs')
require('dotenv').config()
const utils = require('./utils/token')

exports.getUserByUsername = async (username) => {
    console.log('inside authService')
    return await User.findOne({ username })
}

exports.getUserById = async (id) => {
    return await User.findById(id)
}

exports.getUserByIdAndDelete = async (id) => {
    return await User.findByIdAndDelete(id)
}

exports.registerUser = async (username, password) => {

    //generate salt, it can take a while so we use await
    const salt = await bcrypt.genSalt(10);
    //hash password
    const hashedPassword = await bcrypt.hash(password, salt);

    //2 hours to expire in seconds
    const maxAge = 2 * 60 * 60
    //get token from our utils folder
    const token = utils.getToken(username, maxAge)
    console.log({ token })
    //create new user using mongoose schema from models
    const newUser = await User.create({
        username: username,
        password: hashedPassword,
        token: token
    });

    console.log('making USER after token b4 cookie----')
    console.log({ newUser })
    //return all the elements we are going to use
    return { newUser, token, maxAge }

}

exports.loginUser = async (username, password) => {
    //get the user
    const user = await User.findOne({ username })
    //validate the hashed password we have in our database
    const validPassword = await bcrypt.compare(password, user.password)
    //2 hours to expire in seconds
    const maxAge = 2 * 60 * 60
    //set up the jwt token
    const token = utils.getToken(username, maxAge)

    //add token to the user
    user.token = token
    //remember to save your modification
    //https://masteringjs.io/tutorials/mongoose/update
    await user.save()

    console.log({ token })

    return { user, validPassword, token, maxAge }
}

exports.updateUser = async (id) => {

    //get the user
    const userToUpdate = await User.findById(id)
    //check if the user is already an admin
    if(userToUpdate.role === 'Admin'){
        throw new Error('The user is already an admin')
    }
    userToUpdate.role = "Admin"
    userToUpdate.save()

    return userToUpdate
}

utils/token.js

const jwt = require('jsonwebtoken')

//sign and make a new token for the user
exports.getToken = (username, expirationTime) => {
    //example of seeting up expiration time
    //2 hours to expire in seconds
    //const maxAge = 2 * 60 * 60

    //generate token for the user
    const token = jwt.sign(
        { username },
        process.env.TOKEN_KEY,
        {
            expiresIn: expirationTime //2h in seconds
        }
    )

    return token
}

API

This is the most important part of the project. If you don't know routing, HTTP request and response methods or your way around middleware in Express I'd recommend you check some quick tutorials to understand the concepts before digging in but here's a very small and incomplete summary:

• Routing: by creating different endpoints (or routes? - see link for differences) we allow the user to go/call those specific endpoints with parameters and different HTTP requests to get the data they want.

• HTTP methods are conventions that indicate the desired actions we request and respond with.

• Middleware is the middle layer that goes between requests and/or responses, say you want to modify or add some data to the request of the user before it reaches the desired endpoint? That's what middleware does.
  In our case we verify that the user is logged in and that it's an admin before sending it to our updateuser route.

testroute.js

This is just a test route to make sure everything works in our server:

const express = require("express");
const routerTest = express.Router();

routerTest.route("/test").get((req, res) => res.send("GET endpoint from api/testroute"));

module.exports = {
    routerTest,
}

Inside index.js we import it and then define the route:

//import the route
const { routerTest } = require('./api/testroute');

....

//router test to make sure you know how to use middleware and imports/exports
//call example: http://localhost:5000/api/test
app.use('/api', routerTest);

This is the response we get from calling that route

API/auth

Let's jump into our auth folder and see how it works.
This is router.js:

const express = require("express");
const router = express.Router();
const { login } = require("./routes/login");
const { register } = require("./routes/register");
const { updateUserToAdmin } = require("./routes/update");
const { deleteUser } = require("./routes/delete");

//get middleware
const verifyAccessToken = require("../middleware/token");
const checkAdmin = require("../middleware/checkAdmin");

router.route("/register").post(register);
//we can have the same route with different request types
router.route("/register").get((req, res) => res.send("Hello World!"));

//call example: http://localhost:5000/api/auth/login
router.route("/login").post(login);

router.route("/deleteuser").delete(deleteUser);

//load middleware
//make sure the user is logged
router.use(verifyAccessToken);
//make sure the logged user is an admin
router.use(checkAdmin);
//if the previous middleware doesn't throw an error then we update the user
//call example: http://localhost:5000/api/auth/updateuser/62c8ed790c6812f9ddcc1100
router.route("/updateuser/:id").put(updateUserToAdmin);

module.exports = router;

For now let's just focus on the register route.

API/auth/routes/register

const authService = require("../../../services/authService");

exports.register = async (req, res, next) => {
    console.log("inside register");

    const { username, password } = req.body;
    console.log("req body is ", req.body);

    //check if the username already exists
    const oldUser = await authService.getUserByUsername(username);

    console.log({ oldUser });
    if (oldUser) {
        return res.status(409).send("user already exists");
    }

    //simple rule for the password
    if (password.length < 6) {
        return res.status(400).json({ message: "password less than 6 characters" });
    }

    try {
        //define variables with destructuring
        //https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment
        const { newUser, token, maxAge } = await authService.registerUser(
            username,
            password
        );
        //send our cookie with the token
        res.cookie("jwt", token, {
            httpOnly: true,
            maxAge: maxAge * 1000, //convert 2h to ms; maxAge uses miliseconds
        });

        //response status and display the created user
        return res.status(201).json({
            message: "User created successfully",
            newUser,
        });
    } catch (err) {
        //if there are any errors we will get a message back
        return res.status(401).json({
            message: "User not created",
            error: err.message,
        });
    }
};

Check how we are using our function from the services folder through authService.registerUser().

Let's use Postman with raw data as JSON to make the following call on http://localhost:5000/api/auth/register endpoint.

It should look like this:

And if we check our MongoDB we should see the user we just created here:

Let's make a second user as well.

Notice that once we create a user we get a cookie stored in postman, if you click on "Cookies" button you should see something like:

This is our JSON Web Token (JWT):

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6InRlc3QtdXNlcjEiLCJpYXQiOjE2NTg3NzA2OTIsImV4cCI6MTY1ODc3Nzg5Mn0.dPkQroBqkkdi2g_2MwNCm1N16fcchtMBcbbkRxRDCIc

If we go to jwt.io and paste the token this is what we will see:

Once we paste our signature (TOKEN_KEY from our .env file) you should see how the site changes as well:

Finally I'd like to mention that I left a lot of console.logs around for you to see how everything works and why it's doing what, so feel free to add your own logs or at least check the console when calling different endpoints:

API/auth/routes/delete

We have made our users but let's see a very simple way to delete one of them. Note that I'm not using query parameters (we will do something more complicated with the update endpoint) but instead I'm sending the id through the request body.

The delete route is as follows:

const authService = require('../../../services/authService')

exports.deleteUser = async (req, res, next) => {
    const { id } = req.body;
    //if there's no id then we can't do anything
    if (!id) {
        return res.status(400).json({ message: "missing id" });
    }

    try {
        //this service returns the deleted user if it exists
        const user = await authService.getUserByIdAndDelete(id)
        //if user doesn't exist then we can't do anything
        if (!user) {
            return res.status(404).json({
                message: "Could not delete user",
                error: "User not found",
            });
        }
        //return success if the user is deleted
        return res.status(200).json({
            message: "User deleted successfully",
            user,
        });
    } catch (err) {
        return res.status(409).json({
            message: "User not deleted",
            error: err.message,
        });
    }
}

Let's try it by hitting this endpoint with a delete method:

http://localhost:5000/api/auth/deleteuser

Note that we need to get the ObjectId from our users, which you can find in MongoDB:

62ded50470566254e90bb059

And this is the call we do in postman:

If we hit it again we can see that the error message is handled correctly, same if we type the user id wrong:

API/auth/routes/login

Let's look at our login endpoint, it also gives us a cookie with a newly made token so we stay logged in, this is the login.js file:

const authService = require("../../../services/authService");

exports.login = async (req, res, next) => {
    const { username, password } = req.body;
    console.log("login req body is ", req.body);
    //simple checker for username and password
    if (!username || !password) {
        return res.status(400).json({ message: "username or password is missing" });
    }

    try {
        //get the data from our auth service
        const { user, validPassword, token, maxAge } = await authService.loginUser(username, password)
        //check if the user exists
        if (!user) {
            return res.status(401).json({
                message: "Login not successful",
                error: "User not found",
            });
        }
        //return error if the password is incorrect
        if (!validPassword) {
            return res.status(401).json({
                message: "Login not successful",
                error: "Password is incorrect",
            });
        }
        //send our cookie with the token
        res.cookie("jwt", token, {
            httpOnly: true,
            maxAge: maxAge * 1000, //convert 2h to ms; maxAge uses miliseconds
        });

        //if everything is good return the user
        return res.status(200).json({
            message: "Login successful",
            user,
        });

    } catch (err) {
        res.status(401).json({
            message: "Login not successful",
            error: err.message,
        });
    }
};

This is what we get if we call our endpoint:

http://localhost:5000/api/auth/login

as long as the user exists in the database.

We get our token through our authService and afterwards we send it to the user:

        //send our cookie with the token
        res.cookie("jwt", token, {
            httpOnly: true,
            maxAge: maxAge * 1000, //convert 2h to ms; maxAge uses miliseconds
        });

Inside authService on L52 you can see the following:

    //set up the jwt token
    const token = utils.getToken(username, maxAge)

we generate the token thanks to the function in services/utils/token.js

    //generate token for the user
    const token = jwt.sign(
        { username },
        process.env.TOKEN_KEY,
        {
            expiresIn: expirationTime //2h in seconds
        }
    )

If you check the console of our server (the node app we are running) you can see the generated token, I recommend adding more logs at every bit of code that you are curious about although there are better ways to debug it:

API/Session-explorer

Now, before we go to our last endpoint updateuser let's understand a bit better how tokens and the communication with the server works, as well as middleware.

api/session-explorer/router

const express = require("express");
const router = express.Router();
const verifyTokenMiddleware = require("../middleware/token")

//route to read our cookies after loggin in
router.route("/cookie").get((req, res) => {
    console.log('cookies: ', req.cookies)
    res.status(200).json({
        cookies: req.cookies
    })

});

//read more on middleware here: https://expressjs.com/en/guide/using-middleware.html
//token endpoint to check out how verifyToken middleware works
//if the token isn't correct then our middleware in /middleware/token gives an error responde
router.route("/token").post( verifyTokenMiddleware, (req, res) => {
    // in case you want to read everything from the request header
    // console.log({req})
    res.status(200).send("token correct: access granted!")
})


module.exports = router

After logging in and having our cookie let's get a request from

http://localhost:5000/api/cookie

Which should return our cookie where the JWT token is stored:

Note that we are able to get cookies thanks to the middleware cookie-parser and interact with json thanks to express.json, we declared that middleware in our index.js file:

// -- MIDDLEWARE
app.use(express.json());
app.use(cookieParser());
// -- MIDDLEWARE

So in order to access our cookies we just need to type:

req.cookies

Middleware

middleware/token

Before jumping to our /token endpoint let's understand the middleware we have.

We are using token.js to verify that the token we get is correct:

const jwt = require("jsonwebtoken");
require("dotenv").config();

//this only works if you use cookie-parser
const checkCookie = (req) => {
    console.log('inside checkCookie')
    console.log('all our cookies are: ', req.cookies)
    //we get the jwt cookie we need
    return req.cookies['jwt']
}

//middleware for verifying the JWT
const verifyAccessToken = (req, res, next) => {
    //define token
    let token;

    //authenticate through Bearer token
    if (req.headers.authorization && req.headers.authorization.startsWith("Bearer ")) {
        token = req.headers.authorization.split(' ')[1]
        //logs so you see what's happening
        console.log('auth bearer token')
        console.log({ token })
    }
    else {
        token = req.headers["x-access-token"] || checkCookie(req) || null
        //logs
        console.log('our token is from x-access-token header, a cookie or null')
        console.log({ token })
    }

    //if we can't get our token anywhere then the response is an error
    if (!token) {
        return res.status(403).send("A token is required for authentication");
    }
    try {
        //we use the JWT library to verify the token
        //and we need to know the token_key which we encrypted our information
        const decoded = jwt.verify(token, process.env.TOKEN_KEY);
        //log
        console.log({ decoded })
        //the middleware adds the user information to the request and sends it to the route
        req.user = decoded;
    } catch (err) {
        return res.status(401).send("Invalid Token");
    }
    //if you have doubts check Express middleware docs: http://expressjs.com/en/guide/using-middleware.html
    return next();
};

module.exports = verifyAccessToken;

Note how we can get our token from a cookie or headers: req.headers.authorization and req.headers.authorization.

Now while logged in (having our cookie) let's hit our /token endpoint which is using this middleware (remember to use a POST method):

Inside the token.js file we are using this function:

const checkCookie = (req) => {
    console.log('inside checkCookie')
    console.log('all our cookies are: ', req.cookies)
    //we get the jwt cookie we need
    return req.cookies['jwt']
}

to check if the cookie exists (or get it). Then we store the token in the variable:

token = req.headers["x-access-token"] || checkCookie(req) || null

And afterwards verify the token through:

        //we use the JWT library to verify the token
        //and we need to know the token_key which we encrypted our information
        const decoded = jwt.verify(token, process.env.TOKEN_KEY);
        //log
        console.log({ decoded })

This is what we get on the console:

Reminds you of something? Yeah, that's the decoded payload we got from the JWT site, let's type our token there again to see the exact results:

remember to have your secret stored in an .env file, this is mine:

PORT=5000
TOKEN_KEY=8bf0698c0908f00edf6b4efeb837cd52baef165b9029c7a06648ef23e0a0fdfda1cdd758e67597480f74424793848596

The last line inside the try block is:

        //the middleware adds the user information to the request and sends it to the route
        req.user = decoded;

Although it's not helpful for us now what the middleware is doing is add the decoded data to the request and sends that request to the next middleware (or the endpoint it has to reach).

Now if we delete the cookie from postman:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6InRlc3QtdXNlciIsImlhdCI6MTY1ODc3MzcxNywiZXhwIjoxNjU4NzgwOTE3fQ.-4feclw9asuqMf5MQJRd3D3hleqBbNxlOGkxzG8QSF4

Then go to authorization, select "Bearer Token" and paste our cookie it should work the same:

Check the console:

But this is not all, if we select "No Auth" and then go to Headers and add

x-access-token

as key and our token as value our authorization is going to work as well:

Finally if we get rid of everything and we don't have a token in our cookie or our headers then we won't be able to authenticate:

middleware/checkadmin

The other file inside our middleware folder is checkAdmin.js which is a middleware that checks if the user is an admin: if the user is an admin we go to the next middleware and if it's not we return a 403 status

const User = require('../../models/user')

const checkAdmin = async (req, res, next) => {
    //we get the logged user from our token middleware
    //check token @ L42
    console.log('inside checkadmin middleware')
    console.log('user from req is: ', req.user)
    const loggedUsername = req.user.username
    // console.log({loggedUsername})
    const loggedUser = await User.findOne({ username: loggedUsername })
    // console.log({loggedUser})

    if(loggedUser.role === "Admin"){
        //if the logged user is the admin
        //we go to the next action (middleware)
        //in this case the next action is calling the /updateuser endpoint
        return next()
    }
    //if the user isn't an admin return a 403
    return res.status(403).send("You are not an admin");
}

module.exports = checkAdmin

API/auth/update

Now that we understand how middleware and tokens work let's dig into how one user can update another as long as the logged user is an admin. So a logged in admin will be able to update another user to be an admin as long as we pass the id of the user we want to update.

Let's see the code from update.js

const authService = require("../../../services/authService");

//after loading the middleware we can finally call to update the user
exports.updateUserToAdmin = async (req, res, next) => {
    //get the id from the query params
    const id = req.params.id

    if (!id) {
        return res.status(400).json({ message: "missing id" });
    }

    try {
        console.log('we have passed all the middleware and we are inside updateUsertoAdmin')
        const updatedUser = await authService.updateUser(id)

        return res.status(200).json({
            message: "User updated successfully",
            updatedUser,
        });
    } catch (err) {
        return res.status(400).json({
            message: "User not updated",
            error: err.message,
        });
    }
}

It is fairly simple because all our logic is inside the authService.js:

exports.updateUser = async (id) => {

    //get the user
    const userToUpdate = await User.findById(id)
    //check if the user is already an admin
    if(userToUpdate.role === 'Admin'){
        throw new Error('The user is already an admin')
    }
    userToUpdate.role = "Admin"
    userToUpdate.save()

    return userToUpdate
}

updateUser just updates the new user we want to admin, after getting it from the database through the id.

And this logic is simple because we have abstracted the heavy lifting to our middleware, if you rememeber api/auth/router.js we have the following:

//get middleware
const verifyAccessToken = require("../middleware/token");
const checkAdmin = require("../middleware/checkAdmin");

...

//load middleware
//make sure the user is logged
router.use(verifyAccessToken);
//make sure the logged user is an admin
router.use(checkAdmin);
//if the previous middleware doesn't throw an error then we update the user
//call example: http://localhost:5000/api/auth/updateuser/62c8ed790c6812f9ddcc1100
router.route("/updateuser/:id").put(updateUserToAdmin);

Note that if the middleware fails to see a logged in user (verifyAccessToken) or fails to see that the user is an admin (checkAdmin) we won't be able to reach our /updateuser/:id route and we will get back an error.

Now let's see how everything works, to begin with we need at least two users and one of them needs the role of "Admin". Since we register users with only username and password we will have to make our user an admin manually by editing the database directly (bad stuff!).

Let's create our user:

Now modify the database (note we are only doing this because it's a tutorial):

Log in with our new user who's an admin to get the token in a cookie:

This is our database, get the id of the test-user:

Now let's do a PUT request to our endpoint:

http://localhost:5000/api/auth/updateuser/62ded42c70566254e90bb056

If we refresh the database again we will see that both of our users are admins:

Now if we update both users back to normal and try to update:

We will get an error:

In this case our request reached verifyAdmin middleware and it failed:

You can find the exact line of code for this @ middleware/checkAdmin.js L19:

    //if the user isn't an admin return a 403
    return res.status(403).send("You are not an admin");

C'est fini

You can find the whole project on my Github.

If you want to reach out feel free to leave a comment or hit me up on Twitter as well as follow me there or on Hashnode.

Keep in mind that I haven't used any linterns or specific style guides so you might find some inconsistencies with semicolons here and there. I wanted to make a small project on auth at first but it got out of hand, hopefully this is helpful - most of the articles I saw about NodeJS and auth weren't going into as much detail or were outdated.

Thanks for reading.

This article is for those who want to quickly glance over Golang and build a small project, it serves as an introduction into the language.

After going through the post you will know how to build a simple CRUD app and you will be somehow familiar with Go syntax to start your own project.

If you have feedback you can reach me out on Twitter @tekbog or leave a comment here.

The repo for this project can be found HERE.

If you are unfamiliar with API/backend development I have starting series with both Node and Spring Boot you can check.

Why Golang?

If you want to build loosely coupled micro services that are low on memory and scalable Go is probably one of the best choices, if not the best, considering the fact that Kubernetes, Docker and Terraform.

Additionally many other companies use it such as Uber and of course the tech giant Google is using it heavily internally.

Overall Go excels in the back end with Goroutines and Channels. It offers quick development with a minimalist C-like language.

Go Syntax

You can use this page as the Go syntax cheat sheet or start the Go tour from the official site.

Concurrency

I'm not going to go into concurrency, however here's a good enough tutorial to get you started.
To paraphrase a bit:

Goroutine is a lightweight thread.

Channel allows to send data between Goroutines.

Now keep in mind that while looking for Go docs the last release as of today 1.18 has introduced Generics so some of the old tutorials might be outdated.

Let's get through some basics to get you started.

Imports and exports

In every new file you will need to declare its package right at the start such as: package services

Then in order to use export (or expose) a function you need to start it with a capital letter. In the example above you can see that the function GetFeedTweets starts with a capital letter, this means I'm making it "public".
Afterwards just import the package like: import ("project-folder-name/package") and you will be use the function by invoking the package name followed by the dot and the function like: package.FunctionName.

Note that when working with Structs you have to make the fields start with capital letter if you want to expose them. Otherwise you will end like me:

Golang is case sensitive

REMEMBER THIS OR YOU WILL SUFFER

Pointers

I won't be covering pointers in detail as it's a lengthy topic, however here's an article that should make a good introduction.

The Go tour also offers a quick introduction.

To sum it up (badly, please read up on it):

//declare int variable that equals to 42
var i = 42
//make p an int pointer
var p *int
//assign the i memory address to p
p = &i

//p is just a pointer holding memory
//will print 0xc0000b8000
fmt.Println(p)

//p* points to the memory address it's holding
//will print 42
fmt.Println(*p)

You will see functions getting parameters like func FuncName (p *pointerType) just use p as you normally would.

Variadic functions:

If you come from the JavaScript world you will know how useful this is as it works the same as the spread operator, here's a quick tutorial.

Here's a quick example from StackOverflow:

package main
import "fmt"

func my_func( args ...int) int {
   sum := 0
  //the underscore in golang is called the "blank identifier"
   for _, v := range args {
      sum = sum + v
   }

   return sum;
}

func main() {
    arr := []int{2,4}
    sum := my_func(arr...)
    //prints: Sum is  6
    fmt.Println("Sum is ", sum)
}

However it's easier to understand how it works with strings:

package main

import (
    "fmt"
    "strings"
)

func joinString(element ...string) string {
    return strings.Join(element, ",")
}

func main() {

    //will print "join,this"
    fmt.Println(joinString("join", "this"))
    //will print "a,a,a"
    fmt.Println(joinString("a", "a", "a"))

}

Make

The quickest way to make a slice, which is a dynamic array, just do

a := make([]int, 5)

Here's the page from the Go tour

Range

Golang doesn't have a forEach function however it uses range instead so if you want to loop through all the elements of an array or slice just do:

for index, element := range yourSlice{
        fmt.Println(index)
        fmt.Println(element)
}

You can find more examples in this site with golang code examples.

Maps

Maps, hash tables or dictionaries - in Golang they are used as follows:

    m := map[string]int{
        "first-key": 5,
        "second-key":  10,
    }
    fmt.Println(m) 

    for key, value := range m { // Order not specified
        fmt.Println(key, value)
    }

Here's a quick summary about maps.

Methods:

Golang doesn't have classes however it has methods. It sounds counter intuitive but it works great, if you have worked with OOP before just think of Structs as classes/objects. You define a function for them and then use that function on the Struct with a dot call, here's an example.

import "fmt"

type User struct {
    Id   int
    Name string
}

//method to set user's name
func (u *User) setName(newName string) {
    (*u).Name = newName
}

func main() {

    //initialize a new user
    newUser := User{
        Id:   0,
        Name: "First Name",
    }
    //User's ID:  0
    fmt.Println("User's ID: ", newUser.Id)
    //User's Name:  First Name
    fmt.Println("User's Name: ", newUser.Name)

    // make a pointer to the newUser
    pointerNewUser := &newUser

    // call your method
    pointerNewUser.setName("Name From Pointer")
    //User's Name:  First Name
    fmt.Println("Same User's Id: ", pointerNewUser.Id)
    //User's new name: Name From Pointer
    fmt.Println("User's new name:", pointerNewUser.Name)
}

Print

When you want to print something to the console remember to use the proper verbs:

%v    the value in a default format when printing structs, the plus flag (%+v) adds field names
%#v    a Go-syntax representation of the value
%T    a Go-syntax representation of the type of the value
%%    a literal percent sign; consumes no value

That's from the official documentation.

For example a normal expression would be:

fmt.Printf("checking user: %v\n", user)

TypeOf: Reflection

If you want to examine types during runtime then you need to use the reflect package:

package main

import (
    "fmt"
    "reflect"
)

type user struct {
    Id   int
    Name string
}

type blog struct {
    User_id int
    Data    string
    Date    string
}

//this is a generic function since I'm using the keyword "any"
func checkTypeAndValue(x any) {
    userType := reflect.TypeOf(x)
    userValue := reflect.ValueOf(x)
    fmt.Println("Type ", userType)
    fmt.Println("Value ", userValue)

}
func main() {
    newUser := user{
        Id:   1,
        Name: "myUser",
    }
    checkTypeAndValue(newUser)

    newBlog := blog{
        User_id: 1,
        Data:    "this is my first blog",
        Date:    "01-01-1990",
    }
    checkTypeAndValue(newBlog)

}

Blank identifier

You might find blank identifiers in loops or when you import a package, like:

  //the underscore in golang is called the "blank identifier"
   for _, v := range args {
      sum = sum + v
   }

You can find a good explanation from Effective Go about it. However to put it simple: the compiler complains if you aren't using a variable, so if you need to store something but never use it then use the underscore.

Generics

Finally here! The type of jokes no longer work:

With 1.18 Golang has introduced generics, so be careful when you look at old tutorials.

I'd recommend you to read on Generics if you haven't heard the term before but the gist of it is that you can use one function for many types of data, just like I did in the example of TypeOf: Reflection, there's a generic function there that accepts any type of data:

//this is a generic function since I'm using the keyword "any"
func checkTypeAndValue(x any) {
    userType := reflect.TypeOf(x)
    userValue := reflect.ValueOf(x)
    fmt.Println("Type ", userType)
    fmt.Println("Value ", userValue)

}

This FreeCodeCamp article makes it quick to understand. And you can also refer to the official Golang docs.

The project: mini Twitter backend

Notes before you start

Keep in mind this project was made mainly to show off Golang a bit, you can copy the architecture but the project is missing proper ORM, there's no authentication or authorization in place, I'm completely disregarding middleware and there are no tests.

I will talk about all this issues in its own sections but it's important that you are aware that this is not ready for production.

If I had to start from scratch or remake the project I'd add libraries such as sqlx and Gorm. As well as improve the API and other changes that I go through below.

Additionally I'd like to address the routing library I'm using: Fiber. Note that it's version 2, however a lot of tutorials and blog posts are showing and talking about v1, a bit has changed since then so when looking for additional information make sure the import is github.com/gofiber/fiber/v2.

Also, you should definitely have a configuration file and an .env in your project to hide your data.

There's also a disclaimer in the README:

Disclaimer

This is an introductory project to look at Golang a bit. This project is not production ready and it has bad practices like the way pagination works or how we interact with the database.

Through the codebase you will find different "prints" that are used to debug the project, feel free to play with them. What's missing for production ready?

You should add a proper logger, configuration, middleware, a different way to handle the data, perhaps an ORM, and a better way to handle the pagination, as well as a better API.

Note that you can make yourself vulnerable to SQL ingections if you copy and paste the code. This has been made for learning purposes.

Database

You should probably have Docker running MySQL (or any other SQL database). If not you can always install MySQL on your system and use something like MySQL Workbench to work with it.

Database design

I'm using QDBD for the diagram.

The database itself is fairly simple, you have the users that tweet tweets and a table of followers to keep the data of who follows who. The followers table is mainly to implement the feed/timeline for our users.

Create Database

In the scripts folder you will find the main script to run to start your MySQL database:

use twitterdb;

DROP TABLE IF EXISTS tweets;
DROP TABLE IF EXISTS users;
DROP TABLE IF EXISTS followers;

CREATE TABLE users (
  user_id INT NOT NULL AUTO_INCREMENT,
  user VARCHAR(255) NOT NULL,
  passhash VARCHAR(40) NOT NULL,
  email VARCHAR(255) NOT NULL,
  first_name VARCHAR(255) NOT NULL,
  last_name VARCHAR(255) NOT NULL,
  dob DATE,
  PRIMARY KEY (user_id)
);

CREATE TABLE tweets (
  tweet_id INT NOT NULL AUTO_INCREMENT,
  user_id INT NOT NULL,
  tweet VARCHAR(140) NOT NULL,
  date_tweet DATETIME NOT NULL,
  PRIMARY KEY (tweet_id),
  FOREIGN KEY user_id(user_id) REFERENCES users(user_id) 
  ON UPDATE CASCADE ON DELETE CASCADE
);

CREATE TABLE followers (
  id_user INT NOT NULL REFERENCES users (user_id),
  id_follower INT NOT NULL REFERENCES users (user_id),
  PRIMARY KEY (id_user, id_follower)
);

INSERT INTO users (user, passhash, email, first_name, last_name, dob) VALUES
("foo", "asdsad1", "test@gmail.com", "bob", "bobbinson", "2006-01-01"),
("foo2", "asdsad2", "test2@gmail.com", "bob2", "bobbinson2", "1992-01-01"),
("foo3", "asdsad3", "test3@gmail.com", "bob3", "bobbinson3", "1993-01-01"),
("foo4", "asdsad4", "test4@gmail.com", "bob4", "bobbinson4", "1994-01-01"),
("foo5", "asdsad5", "test5@gmail.com", "bob5", "bobbinson5", "1995-01-01"),
("foo6", "asdsad6", "test6@gmail.com", "bob6", "bobbinson6", "1996-01-01"),
("foo7", "asdsad7", "test7@gmail.com", "bob7", "bobbinson7", "1925-01-01"),
("foo8", "asdsad8", "test8@gmail.com", "bob8", "bobbinson8", "1980-01-01"),
("foo9", "asdsad9", "test9@gmail.com", "bob9", "bobbinson9", "1980-01-01"),
("foo10", "asdsad10", "test10@gmail.com", "bob10", "bobbinson10", "1970-01-01");

INSERT INTO tweets(user_id, tweet, date_tweet) VALUES
(1, "test tweet", "2001-01-01 22:00:00"),
(2, "test tweet2", "2002-01-01 22:00:00"),
(3, "test tweet3", "2003-01-01 22:00:00"),
(4, "test tweet4", "2004-01-01 22:00:00"),
(5, "test tweet5", "2005-01-01 22:00:00");

INSERT INTO followers(id_user, id_follower) VALUES
(5,1),
(4,1),
(3,1),
(2,1),
(6,1),
(2,5),
(4,5);

The other files are examples of queries that we will be using later.

Project Architecture

If you are familiar with building software this should be familiar if not, first let me tell you what's missing and afterwards we will go through every folder.

The project is missing folders like config, middleware, logger and testing. And if you would organize the controller better you'd have a routes folder where you organize the API better.

Quick note on imports

In this project you might get confused by imports such as:

import (
    "goexample/database"
    "goexample/models"
    "goexample/services/utils"
)

The goexample here is the name of the project, I just renamed my repo afterwards so it would make sense, so instead of having "mini-twitter-clone/database" we use "goexample/database" to import the database package.
With any other new projects just use the name of the folder.

API

You store your endpoints/routes in controller.go:

package api

import (
    "goexample/services"

    "github.com/gofiber/fiber/v2"
)

func SetupRoutes(app *fiber.App) {

    api := app.Group("/api")

    //get all unordened users
    api.Get("/users", services.GetUsers)
    //get all users ordered by age ASC
    api.Get("/users/age", services.GetUsersByAgeAsc)

    //get all unordened tweets from db
    api.Get("/tweets", services.GetTweets)

    //http://localhost:3000/api/feed/1
    //get MOST RECENT feed/timeline for the user
    api.Get("/feed/:id", services.GetFeedTweets)
    //pagination
    api.Get("/feed/:id/:limit/:offset", services.GetFeedTweetsPaginated)
    //can try https://github.com/gofiber/fiber/issues/193#issuecomment-591976894
    //a whole presentation on why you shouldn't do what I did:
    //https://use-the-index-luke.com/no-offset

}

I'm using the Fiber library, which is similar to Express, here we have several endpoints that we call with a Get request and once the server gets a request its response is to call the functions that I have exposed from the services package - that's where our business logic is.

NB: pagination isn't properly implemented.

This is just a quick example, this endpoint has security issues just like others but you should understand how the library and Golang works through the example.

Note that in the /feed/:id the id parameter pertains to the user we want to get the feed for.

Start the project

You can start the project by doing go run . inside the folder.
Your terminal should look like this:

Let's visit the different endpoints to see the responses, I'm going to use a normal browse but you should check Postman if you are not familiar with debugging the backend.

Let's run the endpoints from the controller:

http://127.0.0.1:3000/api/users

I left a lot of prints so if you check your terminal with every call you should see stuff like:

Next is a call where we order users by age:

http://127.0.0.1:3000/api/users/age

When we ask for tweets we get all of them:

http://127.0.0.1:3000/api/tweets

And now we get the feed or timeline for the user with the id 1:

http://127.0.0.1:3000/api/feed/1

Remember that pagination isn't production ready but the core concepts are the same:

http://127.0.0.1:3000/api/feed/1/2/1

All the calls work thanks to the business logic we have in the services package and the functions we have exposed.

Services

Inside the services package we have the functions that are called when someone or something hits one of the endpoints. Remember that we expose the functions by using a capital letter in the package.

Let's look at timeline_tweets.go which contains two functions for two different endpoints in the controller.go file:

package services

import (
    "fmt"
    "goexample/database"
    "goexample/models"
    "goexample/services/utils"
    "log"

    "github.com/gofiber/fiber/v2"
)

func GetFeedTweets(c *fiber.Ctx) error {

    //you shouldn't do this by the way, but it's just a demo
    // dbQuery := fmt.Sprintf("SELECT users.user_id, users.user, users.first_name, users.last_name, tweets.tweet, tweets.date_tweet FROM users INNER JOIN tweets ON users.user_id = tweets.user_id INNER JOIN followers ON users.user_id = followers.id_user WHERE followers.id_follower = %s ORDER BY tweets.date_tweet DESC;", c.Params("id"))
    // rows, err := database.DB.Query(dbQuery)

    //avoid the SQL injection by rewriting it like
    dbQuery := "SELECT users.user_id, users.user, users.first_name, users.last_name, tweets.tweet, tweets.date_tweet FROM users INNER JOIN tweets ON users.user_id = tweets.user_id INNER JOIN followers ON users.user_id = followers.id_user WHERE followers.id_follower = ? ORDER BY tweets.date_tweet DESC;"
    rows, err := database.DB.Query(dbQuery, c.Params("id"))

    //check for errors
    if err != nil {
        return utils.DefaultErrorHandler(c, err)
    }
    //close db connection
    defer rows.Close()

    //create a slice of tweets
    var timelineTweets []models.TimelineTweet
    //loop through the result set
    for rows.Next() {
        timelineTweet := models.TimelineTweet{}
        err := rows.Scan(&timelineTweet.User_id, &timelineTweet.User, &timelineTweet.First_name, &timelineTweet.Last_name, &timelineTweet.Tweet, &timelineTweet.Date_tweet)
        if err != nil {
            log.Fatal(err)
        }
        timelineTweets = append(timelineTweets, timelineTweet)
    }
    fmt.Print(timelineTweets)

    utils.ResponseHelperJSON(c, timelineTweets, "timeline", "No timeline found")

    return err
}

func GetFeedTweetsPaginated(c *fiber.Ctx) error {

    // dbQuery := fmt.Sprintf("SELECT users.user_id, users.user, users.first_name, users.last_name, tweets.tweet, tweets.date_tweet FROM users INNER JOIN tweets ON users.user_id = tweets.user_id INNER JOIN followers ON users.user_id = followers.id_user WHERE followers.id_follower = %s ORDER BY tweets.date_tweet DESC LIMIT %s OFFSET %s;", c.Params("id"), c.Params("limit"), c.Params("offset"))
    // avoid a SQL injection by rewriting it like
    dbQuery := "SELECT users.user_id, users.user, users.first_name, users.last_name, tweets.tweet, tweets.date_tweet FROM users INNER JOIN tweets ON users.user_id = tweets.user_id INNER JOIN followers ON users.user_id = followers.id_user WHERE followers.id_follower = ? ORDER BY tweets.date_tweet DESC LIMIT ? OFFSET ?;"

    rows, err := database.DB.Query(dbQuery, c.Params("id"), c.Params("limit"), c.Params("offset"))
    if err != nil {
        return utils.DefaultErrorHandler(c, err)
    }

    defer rows.Close()

    var timelineTweets []models.TimelineTweet
    for rows.Next() {
        timelineTweet := models.TimelineTweet{}
        err := rows.Scan(&timelineTweet.User_id, &timelineTweet.User, &timelineTweet.First_name, &timelineTweet.Last_name, &timelineTweet.Tweet, &timelineTweet.Date_tweet)
        if err != nil {
            log.Fatal(err)
        }
        timelineTweets = append(timelineTweets, timelineTweet)
    }
    //TODO: implement a response with pages and all that pagination jazz
    utils.ResponseHelperJSON(c, timelineTweets, "timeline", "No timeline found")

    return err
}

The first that you can notice is how we pass the context to GetFeedTweets and afterwards we use the variable 'c' to use that context as the Fiber library says.

Afterwards we use the exposed variable DB from the package database to open the DB, read the data and close it afterwards.

In order to store and scan the data properly we use a struct from our models package.

Afterwards you will see several functions from the utils package. Those functions are mainly helpers so you can check out how Golang does loops and certain other things.

The two other files inside our services package are very similar, our work here is get the data from the database, scan it through our array of structs and send it back to the user in JSON format. As well as we run some functions from the utils package.

Utils

This package contains helper functions, perhaps the most interesting being inside user_helper as it has functions that interact with data from slices, however be careful, they are not as well implemented as you might think.

Let's look at the most helpful one, response_helper.go:

package utils

import (
    "github.com/gofiber/fiber/v2"
)

//response JSON for services after you loop and scan
func ResponseHelperJSON(c *fiber.Ctx, data any, dataType string, dataError string) {
    if data != nil {
        c.Status(200).JSON(&fiber.Map{
            "success": true,
            dataType:  data,
        })
    } else {
        c.Status(404).JSON(&fiber.Map{
            "success": false,
            "error":   dataError,
        })
    }
}

Note that this functions contains a generic type of data as I'm using the keyword any and I'm interacting with the context given from Fiber through the variable 'c'.

Models

The models folder contains the different structs we use as entities to interact with our database as shown in the services folder.

Here's an example, note that even that if you want to expose the entire struct everything has to start with capital letter:

package models

type UserWithAge struct{
    Id         int   `json:"id"`
    User       string `json:"user"`
    Passhash   string `json:"passhash"`
    Email      string `json:"email"`
    First_name string `json:"first_name"`
    Last_name  string `json:"last_name"`
    Age        int    `json:"age"`
}

Database

The database package is very straightforward and remember to use a config file and an .env to store your sensitive data.

package database

import (
    "database/sql"
    "fmt"
    "log"
)

var DB *sql.DB

func Connect() error{
    var err error
    //use a config file for this
    DB, err = sql.Open("mysql", "root:password@tcp(127.0.0.1:3306)/twitterdb")

    if err != nil {
        log.Fatal(err)
        return err
    }

    if err = DB.Ping(); err != nil {
        log.Fatal(err)
        return err
    }

    fmt.Println("Connected to database")

    return nil

}

Main.go

Finally, this is where we start our server:

package main

import (
    "github.com/gofiber/fiber/v2"
    "goexample/database"
    "log"

    "goexample/api"

    _ "github.com/go-sql-driver/mysql"
)

func main() {

    if err := database.Connect(); err != nil {
        log.Fatal(err)
    }

    app := fiber.New()
    api.SetupRoutes(app)


    log.Fatal(app.Listen(":3000"))

}

Bonus Ukrainian meme

This post goes through basic concepts that every developer should know or at least be familiar with in some way or the other, it's an entry point to good practices. Most people already do all this without thinking, perhaps the post will just server to illustrate certain actions or thoughts.
However and as we all know there's never a clear answer when it comes to software engineering, it boils down to two magical words: "it depends".

Feel free to correct me or provide feedback, you can find me on twitter @tekbog

OOP: what a beauty

You look outside and it's a beautiful day, you definitely can miss this weather so you decide to stay home and code.

After making a beautiful hello world program you feel good and empowered to try something more advanced, and you wonder, how should you organize or even where to start. So you ask the kind souls at StackOverflow, at which some reply and mention OOP before your thread gets deleted because code monkeys decide to fight over Objected Oriented Programming VS Functional Programming.

So you start looking at OOP and it makes a lot of sense! You have everything organized in objects that call other objects and are related to one another. It seems secure and it can be as complex as you wish, yet simple to understand. What a concept!

BUT YOU ARE WRONG

However with good practices and proper object modeling we can avoid having such a mess.

Concepts

Objects are made through classes, what does this mean? Classes are blueprints for whatever you want to make. You use that blueprint to create (or instantiate) the object that will run in memory. The initialised object will have:

• state (characteristics)
• behaviour (what can it do or what can it be done to it)

NB: in languages like C++ you need to take care of the memory usage, however a lot of languages have garbage collection implemented nowadays so you don't have to worry about when your objects disappear from memory.

So, to make things clearer, you have a design called class then you use that design to create a new object of that class with which you will be able to interact and use. The interactions are defined (just like instructions) inside the class through state and behaviour. The class defines what you can do with the object and how other objects will be able to interact with said object once it's created.

Both the state and behaviour are modified through:

• attributes: parameters that define the state of the object
• methods: functions that define the behaviour of the object

The strengths of OOP are as follow

Encapsulation

You want to have the data/variables and its behaviour closed in one single unit. Inside you define its behaviour through functions (methods) that you can make public or private to keep the object as hidden and pure as possible.

Abstraction

External agents don't need to know what's going on inside a specific object, they just need to know how to use it if needed. With abstraction you hide the internal implementation.
To explain the concept for example take a random controller, you need to know HOW it works but you don't need to know WHY it works.

If this sounds too similar to encapsulation you can think of it as: abstraction is the theory while encapsulation is the implementation.
Additionally you can read this article. The picture is from this repo.

Inheritance

or something a bit more complex

Inheritance provides security and a cleaner -more defined- architecture by basing one object on another, either through inheritance or sub-classes. To use inheritance you can either define an interface and then implement it on a new object or you can use an existing class and extend it to make a sub-class, with the original class called a super-class.
You have interfaces, classes and then there's something in between called abstract classes which are just like ordinary classes but unfinished so you can't instantiate them but you can extend them and finish building them.

Polymorphism

Poly stands for many;
Morph stands for form.
The main concept of polymorphism is the ability to have many forms.
For example an interface can have many implementations. In order to apply polymorphism you just need a core to pull from and morph the original object with many (poly) changes.

Polymorphism vs Inheritance

Things get a bit confusing with polymorphism and inheritance but think about how a class can inherit properties from a superclass while an interface can be implemented in many different ways making itself polymorphic. Henceforth classes work mostly through inheritance while interfaces work through polymorphism and inheritance.

Method Overriding

Also known as Dynamic Polymorphism and Run Time Polymorphism.
It consists of allowing the subclass to change (override) the method (function) of the superclass. If you have a super class with the method (function) that says run and prints 'superclass runs' you can change the same method in the subclass to print 'subclass runs'.

Method Overloading

Also known as Static Polymorphism and Compile Time Polymorphism.
You define two different methods that are called the same but behave differently with different parameters and at compile time the program decides which method to use.

Here's a StackOverflow answer that goes into both method overriding and overloading with a code example.

Duck Typing

The principle of duck typing is related to method overloading and compile-time polymorphism, although mostly known and used in Python.
The saying goes as "if it walks like a duck and quacks like a duck.. it is a duck". We can apply this logic to our objects, we don't care what they are per se but we care what they do and how their functions work. Therefore you can overload a function (through polymorphism) to do whatever you want, so in this scenario if you have a bird object that does everything a duck is capable of then you call that object a duck.

However in languages like Java or C# duck typing doesn't work as we are usually very concerned as to what type of objects we are working with.

Inheritance Relationships

There are two key relationships in OOP:

• IS A

IS-A stands for "the object I'm referring to IS An object of another type I'm extending from" in this case if we have a subclass Dog that extends from the Animal class the relationship would be: Dog class IS-A Animal class.
This example is based on classes but you could also have an interface called Animal and a class called Dog that implements it, also a IS-A relationship.

• HAS A

This relationship is more about usage, it means that an object is using another object without a need to extend or implement it.
For example our Dog object could have a Toy object and interact with it. Here Dog HAS-A relationship with Toy.

SOLID Principles

Those were dark times, the dot-com bubble made developers aggressive and savage, nobody followed any rules and the bugs reigned untouchable as software engineers were fighting amongst themselves, blaming each other and not writing tests. In order to stop this violence and establish the order back in the workplace a prophet called Robert C. Martin came with 5 solid principles to put an end to this madness of developers fighting each other due to unreadable and unmaintainable code.

The SOLID principles are five main ideas to make your code understandable, flexible and maintainable.

Single-responsibility principle

SRP is summed up nicely in: "every class/function/module should have only one responsibility and encapsulate that part". And so microservices were born!
To put an example, if you have a class or a function that has many inputs, conditionals and it's fairly long you probably need to refactor that ASAP.

Open–closed principle

Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification

This means that you should be able to always extend a class with new functionality without having to change how it works. It's a key to maintainable code and it's easily achievable with interfaces, you can keep expanding it without the need to change the original interface.

Liskov substitution principle

LSP is a bit confusing but basically the Liskov principle says that the subclass should be substitutable for the superclass. This will prevent you from making the wrong decisions when it comes to inheritance and polymorphism, it's easy to extend a class but make sure it's the correct order and implementation.

It's much easier to understand through code, check this answer from StackOverflow.

Interface segregation principle

ISP is intended to keep a system decoupled and thus easier to refactor, change, and redeploy. No code should be forced to depend on methods it does not use.

ISP wants you to split the large interfaces and keep it simple, stupid (KISS). The official definition is as follows:

Clients should not be forced to depend upon interfaces that they do not use.

You don't want a bloated interface that has methods for multiple responsibilities. Split it.

Dependency inversion principle

DIP is summed as follows:

• High-level modules should not import anything from low-level modules. Both should depend on abstractions (e.g., interfaces).
• Abstractions should not depend on details. Details (concrete implementations) should depend on abstractions.

It might sound simple but people tend to overlook this principle, however by following the open-closed principle and LSP you will be able to achieve the level of abstraction needed to separate both low and high level components so they never depend on each other.
If your components are designed to be closed for modification but open for extension (open/closed principle) and you can replace with your implementations your interface (LSP) then you are following DIP. Separate the concern for design through layers of abstraction.

FreeCodeCamp has a nice definition while talking about Dependency Injection (which is a way to follow the dependency inversion principle):

According to the principles, a class should concentrate on fulfilling its responsibilities and not on creating objects that it requires to fulfill those responsibilities.

Association: Composition and Aggregation

Both composition and aggregation are part of association, here are some UML diagrams, which is a way of saying two objects are related (by association). However composition and aggregation differ.

Composition is when a sum of parts build the object, so you need each other, following a two-way association. If you get rid of the composition its parts get deleted.
IBM offers a nice explanation:

A composition association relationship specifies that the lifetime of the part classifier is dependent on the lifetime of the whole classifier.

Aggregation is a more lax composition where objects don't depend on each other for the aggregation to exist. You can have two different objects that work together to build an aggregation but if the aggregation disappears the objects remain, as they are fully separate entities.
99guru defines the aggregation relationship really well:

an object of one class can own or access the objects of another class.

People tend to mix a bit the terms, more than often you will hear composition but in reality they are talking about aggregation. So just keep in mind when someone mentions composition they might be talking about a lax composition, meaning aggregation. Sometimes it doesn't matter but it might be relevant if you are designing complex systems.

Composition over Inheritance

If our code base deviates from the SOLID principles (or any other reason) instead of having a tightly coupled structure based on inheritance you might want to use composition instead.
Composition is usually just a way of saying that classes might use other classes to build complex interactions instead of relying as heavily on inheritance, this leads to HAS-A relationships (or PART-OF). The term is loosely used and sometimes people confuse aggregation with composition. When someone is talking about composition over inheritance they can mean composition or aggregation, either way what they want to achieve is:

• Stronger encapsulation: inheritance needs a link with its subclasses which might weaken encapsulation.
• Easier testing: if you have separate objects without inheritance you can access and test them easier as they are fully separate units.
• Better refactoring: swapping things around is easier when you don't have a whole family of objects depending on it. By adding interfaces and making different classes work with each other you add flexibility which is almost always good.

It's especially strong if you bring Dependency Injection into the matter, which is a form of Inversion of Control.

FreeCodeCamp has a nice little tutorial about DI.

Replace Conditional with Polymorphism

Another benefit and strong point for OOP is how you can use Polymorphism to replace Conditionals. For example if you have a class with a lot of if-else or switch statements you can refactor it with classes through polymorphism. Afterwards just use whatever class you need.
Refactor Guru offers a nice example.

Engineering Principles

Once again xkcd points out our reality, however by applying certain principles we can get a bit better at organizing the pattern of lights.

Measure twice and cut once

The point here is to think ahead. This depends entirely on what you are doing, for example if you are planning to make a platform but don't know how the schema database is going to look then use NOSQL over SQL.

Don’t Repeat Yourself (DRY)

DRY is a key to be a good programmer. A good way to know when you are repeating yourself is to adhere to the Rule of Three. If you are using the same code three times you need to refactor!

Keep It Simple Stupid (KISS)

Software engineers have a lot of tools at our disposal which makes us incredible prone to over-engineer everything instead of following KISS. Keep things simple and avoid complexity, just make sure you can extend your code when you need to.

This video about microservices from KRAZAM sums it up or take a look at the one from fireship if you are familiar with web development. Although over-engineering also apply to YAGNI rule.

You Aren’t Gonna Need It (YAGNI)

YAGNI is about focusing on the problem at hand while (just like KISS) minimizing complexity. Focus on keeping things simple yet open to added complexity later on, use the open/closed principle.

Avoid Premature Optimization

premature optimization is the root of all evil

While we need to keep in mind optimization to avoid performance issues most likely you don't have to optimize that function and make it less readable.

Principle Of Least Astonishment

POLA is about being boring. Make sure to not surprise the users or yourself both the code and the UX have to be consistent and predictable.

Law of Demeter

LOD or principle of least knowledge is summed up in 3 points:

• Each unit should have only limited knowledge about other units: only units "closely" related to the current unit.
• Each unit should only talk to its friends; don't talk to strangers.
• Only talk to your immediate friends.

Read more here.

This introduces the concepts of coupling and cohesion.

• Decoupling: try to reduce the number of connections between different classes.
• Cohesion: the associated classes must be in one module/package/directory.

OOP Modeling

You know the good principles, so time to write code! But wait, first you should model your ideas for a simpler implementation. Software engineers use UML to design their systems.
I will have another blog post on the topic, what follows is a very small introduction with resources.

UML basics

The point of a UML diagram is to put together classes, abstract classes and interfaces together (amongst other things) so you can visualize how your software is going to work and start implementing the correct OOP hierarchies.
You can find a small introduction from TutorialsPoint here. And if you want to watch a video FreeCodeCamp is your answer.

One of the most basic things you should be familiar with are the UML arrows:

Design Patterns

Design patterns are solutions to common architecture problems. If you have ever heard about Factory Method or Singleton those are creational design patterns, there are three types: Creational, Structural and Behavioral. Refactoring Guru is one of the best at explaining them.

Additionally I highly recommend this video series from Christopher Okhravi if you are unfamiliar with the matter.

We will be using the code from the 2nd part of this series.

You can find the code in this Github repository, note that you are on the testing branch.

hmu @tekbog if you find any bugs or want to annoy me.

About testing and QA

People still fight over definition and how it should be done, however this article from IBM might give you some insight if you are unfamiliar with software testing.

To largely simplify testing we can define some concepts:

• Unit tests: as the definition implies we will test simple units of code, one function at a time.
• Integration tests or I&T: is when you test several functions or different pieces of code that work together
• End to End testing: as the name implies you fully test your whole application, often as an emulated user
• Test Driven Development: write the test for the function you want, then write the function

A lot of different tools exist for almost any language, in JavaScript the most famous ones are Jest, Mocha and Cypress for E2E. However the overall most famous E2E tool might be Selenium.

Git

I'm not going to go into detail here, if you unfamiliar with Git and want to learn more about check out this tutorial from Atlassian. Start at the beginning if you are confused. A few key points for newbies:

• Git is a version control tool
• Github is the hub that hosts Git projects, it's owned by others and there are other companies such as Bitbucket and Gitlab that do the same.

I'm using my old project so I'm going to make a new branch with

git checkout -b testing main

and stay here as this will be the branch for this article.

Jest (or Mocha)

We will be using Jest since we want to focus on React, however if you are working with Node.js and Mongoose it's discouraged even in the official Mongoose docs so feel free to use an alternative like Mocha. We will use Jest as our application isn't complex enough to run into problems and we will focus on React in the future.

Let's install Jest

npm install --save-dev jest

Afterwards inside package.json modify the test script

"test": "jest --verbose"

You should have this:

Start testing

Now that we have Jest installed let's make some dummy tests to see how it works.

NB: I will be using normal import/export as ES6 module exports might give you some issues.

Let's make a helper file inside utils folder that will contain our function we want to test test_helper.js:

//return the sum of two numbers
const sumFunction = (a,b) => {
    return a+b
}

module.exports = {
    sumFunction
}

Next we will make a folder called tests and inside a file called dummy.test.js where we will test the function above:

const { sumFunction } = require("../utils/test_helper")
//tests take a description and a function
test('test sumFunction, expected to pass', () => {
    const a = 5
    const b = 6
    const result = sumFunction(a, b)
    //through expect we indicate the expected result of the test
    //the .toBe method expects a specific value you pass
    //https://jestjs.io/docs/expect#tobevalue
    expect(result).toBe(11)
})
test('test sumFunction, expected to fail', () => {
    const a = 5
    const b = 6
    const result = sumFunction(a, b)

    expect(result).toBe(12)
})

Note that .test is needed so Jest knows to read your tests from this file. Now let's do

npm run test

This should be the result:

Check out many of the Expect methods Jest has here

Testing a RESTful API

We will be using ES6 modules so we need to make some changes in our package.json as follows:

"test": "node --experimental-vm-modules node_modules/jest/bin/jest.js --verbose --runInBand --forceExit"

Jest gives you more information here.

We will be using Supertest, so let's install it

npm install supertest

Now let's make a file inside tests called api.test.js where we will be importing everything we need to make our tests:

import supertest from 'supertest'
import { app } from '../app.js'
//use the supertest object as our API
const api = supertest(app)

import { Item } from '../models/item'

Then remember to run our server with npm start dev so it restarts every time we make changes and open a new window terminal so we can type our Jest commands.

GET test

Since we are getting our data from a server we will need to use async functions, keep that in mind.

This is a test for our GET call expecting a 200 status:

test('GET call', async () => {
    await api
        .get('/api/items')
        .expect(200)
})

You can run this test through this command

run npm test -- -t "GET call"

That's how you run individual tests.

In order to make the test a bit more complete we will also add a method to expect JSON data:

test('GET call', async () => {
    await api
        .get('/api/items')
        .expect(200)
        .expect('Content-Type', /application\/json/)
})

Your api.test.js file should look like this by now

import supertest from 'supertest'
import { app } from '../app.js'
//use the supertest object as our API
const api = supertest(app)

//run npm test -- -t "GET call"
//test GET or READ call on localhost:3001/api/items endpoint
test('GET call', async () => {
    await api
        .get('/api/items')
        .expect(200)
        .expect('Content-Type', /application\/json/)
})

POST test

In order to interact with the database properly we will use our Mongoose model

import { Item } from '../models/item'

Afterwards we will proceed to make a POST call

test('POST call', async () => {
    //build a new item
    const newItem = {
        description:"sent from Jest!",
        likes: 10
    }
    //we send the item object to the DB through the API
    //we expect a successful result
    await api
        .post('/api/items')
        .send(newItem)
        .expect(201)
    //get all the items in our DB
    const items = await Item.find({})
    //let's check that the last item added was indeed newItem object
    //it should contain the description "sent from Jest!"
    expect(items[items.length-1].description).toBe("sent from Jest!")
})

GET item by id

In this test we are going to check if the GET route by specific id works as expected.

test('GET one', async () => {
    //get all the items
    const items = await Item.find({})
    //get the the first item parsed to JSON
    const firstItem = items[0].toJSON()
    //get the result expecting success and JSON data
    const resItem = await
        api.get(`/api/items/${firstItem.id}`)
        .expect(200)
        .expect('Content-Type', /application\/json/)
    //check if the item has the same id and the route works as expected
    expect(resItem.body.id).toEqual(firstItem.id)
})

Check all items have an id

Let's add one more simple test

    test('check all items have ids', async () => {
        //get all the items
        const items = await api.get('/api/items')
        //check that every item in our DB has the id property
        for(const item of items.body){
            expect(item.id).toBeDefined()
        }
    })

Refactoring GET calls

Let's use describe to put our three GET calls together as follows:

describe('GET calls', () => {
    //run npm test -- -t "GET call"
    //test GET or READ call on localhost:3001/api/items endpoint
    test('GET call', async () => {
        await api
            .get('/api/items')
            .expect(200)
            .expect('Content-Type', /application\/json/)
    })
    //npm test -- -t "GET one"
    //GET item by id
    test('GET one', async () => {
        //get all the items
        const items = await Item.find({})
        //get the the first item parsed to JSON
        const firstItem = items[0].toJSON()
        //get the result expecting success and JSON data
        const resItem = await
            api.get(`/api/items/${firstItem.id}`)
            .expect(200)
            .expect('Content-Type', /application\/json/)
        //check if the item has the same id and the route works as expected
        expect(resItem.body.id).toEqual(firstItem.id)
    })
    test('check all items have ids', async () => {
        //get all the items
        const items = await api.get('/api/items')
        //check that every item in our DB has the id property
        for(const item of items.body){
            expect(item.id).toBeDefined()
        }
    })
})

Afterwards we can use this cmd to call this block of tests

npm test -- -t "GET calls"

If everything is successful we will get this in the console:

DELETE

Let's check that we can delete an item and all its data is gone

test('DELETE item', async () => {
    //get items and parse the one you want to delete to JSON
    const itemsAtStart = await Item.find({})
    const itemToDelete = itemsAtStart[0].toJSON()
    //delete the item by id
    await api
        .delete(`/api/items/${itemToDelete.id}`)
        .expect(204)
    //get all items from the database again
    const itemsNow = await Item.find({})
    //check if the number of current items is one less than before
    expect(itemsNow).toHaveLength(itemsAtStart.length-1)
    //get an array of all the descriptions inside the DB
    //could get any other info like the id
    const itemsDescriptions = itemsNow.map(i => i.toJSON().description)
    //expect the description from the deleted item to not be there
    expect(itemsDescriptions).not.toContain(itemToDelete.description)
})

beforeEach

If you want to run any code before tests start you need to use beforEach. In our case we want our database to reset every time we start testing so let's make some data to insert inside a file called test_helper.js

const initialItems = [
    {
        description: "first item",
        likes: 5
    },
    {
        description: "second item",
        likes: 7
    },
    {
        description: "third item",
        likes: 10
    }
]

export {
    initialItems
}

then import that data and use it to reset our database every time we start testing. We delete everything and then insert our initial data again:

beforeEach(async () => {
    await Item.deleteMany({})
    await Item.insertMany(initialItems)
})

afterAll

If you want to run something after all our tests are done then use afterAll

afterAll(() => mongoose.connection.close())

In our case we just close the connection to the database.

You will learn how to make a full RESTful API that connects with a real database. The endpoints contain all the CRUD operations and you will apply modern techniques like ES6 export modules and async/await functions, as well as the relevant libraries to make building the project a breeze.

Here's the repository for the project and you can find me on Twitter @tekbog.

Additionally you can find the first chapter of the series that connects to an in-memory database here.

Setup

Do

npm init

The install all the dependencies

npm install express nodemon dotenv cors express-async-errors mongoose

Afterwards modify the package.json by adding

"type":"module",

right after

"main": "index.js",

this will allow us to use ES6 exports.

Once that's done we will modify the "scripts" part. You run those scripts by typing

npm run name_of_script

We will add:

    "start": "node index.js",
    "dev": "nodemon index.js",

That way whenever you type

npm run dev

it will be equal to

nodemon index.js

Nodemon is a tool that restarts the server every time a line of code is modified. It's only used for development purposes.

Your package.json should look like this now:

Project Structure

The project structure is going to look as follows:

Controllers folder will contain our endpoints and the way we interact with them.
Models folder will have the model of our database, we will use that model in our controller items.js.
Utils is generally a folder for anything non-specific but you don't want floating around in the project, in our case it will contain a configuration file.
App.js is the server itself where we will be importing from controllers and models.
Index.js is the main point that starts everything, we will import our server/app there.

Then there's package.json that contains all the dependencies and information about the project.

.gitignore && .env

.gitignore is there if you are using Git, make sure to ignore your node_modules folder as it should only be installed locally or on the server. We will also be ignoring our .env file through .gitignore since it will contain information we can't expose.
.env is there so you can use the dotenv library and use all the keys and passwords without exposing anything outside. If you aren't pushing your code anywhere then feel free to ignore process.env variables and write them as consts in your code.

MongoDB Atlas

We will be working with a real database and its free tier. If you are unfamiliar with it then have a quick look at their guide here.
However the TL;DR is: get your mongodb URI and put it into the .env file like this

MONGODB_URI=mongodb+srv://YOUR_USER:YOUR_PASSWORD@cluster0.pekyz.mongodb.net/YOUR_DATABASE?retryWrites=true&w=majority
PORT=3001

Note that you have to use the information of your account but don't confuse your own login with your database account.

Mongoose Schema

Let's start with the models folder. Inside we will put the schema of our data. Object databases don't need schemas however it's always good practice to have one. If you don't want to dig into databases right now just know that here you are defining a model of the data your database will be storing.
The easiest way to interact with MongoDB is the mongoose library.

item.js

//in order to communicate with mongoDB we will use mongoose
//it simplifies the process
import mongoose from "mongoose"
//we define the schema of the database
//description field is a string and likes is a number
const itemSchema = new mongoose.Schema({
    description: String,
    likes: Number
})
//we modify the schema and pass it to JSON
//mongoDB adds two fields we don't necessarily want
//_id is the id the objects get randomly assigned
//we transform _id into a string variable called .id and delete _id
//we also delete __v
//try the data without deleting __v
itemSchema.set('toJSON', {
    transform: (document, returnedObject) => {
        returnedObject.id = returnedObject._id.toString()
        delete returnedObject._id
        delete returnedObject.__v
    }
})
//mongoose needs you to define a 'model' with your schema
//then use that model to operate with different functions
const Item = mongoose.model('Item', itemSchema)

export { Item }

Endpoints with async functions

The controllers folder could also be named API as it serves the routes/endpoints for our server. Once we have the user requesting HTTPS methods on those endpoints we will direct those requests to the database through our Mongoose model as you will see in the code.
It works as follows: the user interacts with the frontend, we receive an event through it that triggers our server's backend through the api we have designed, the api then interacts with the database through the different models. The database sends back the information to the server and the server sends it back to the frontend.

user -> frontend sends an event -> server receives the event through a HTTP method (GET, POST, UPDATE, DELETE) -> server calls the database through the model

Then depending if the interaction with the database is successful we send information back to the user

DB -> server -> frontend -> user

Here we will be using the Router object from Express as it simplifies development. We will then use the mongoose schema to interact with the database. Everything is done with async/await functions instead of promises, if you are unfamiliar with them the main thing is: those functions wait for the data so everything works properly.

NB: You will find the concept Middleware everywhere in the backend, it just means a function that modifies the data object and usually passes it to the next point.

items.js

import { Router } from "express"
import { Item } from "../models/item.js"
//define our router object
const itemsRouter = Router()
//GET or READ endpoint
//it's an async function that waits for the data thanks to await
//we get the data from the database by using our mongoose model
//one of the methods at the moongoose model is .find where you can get whatever you want
//find is empty so we get all the data
//afterwards we return a successful status and the JSON data
itemsRouter.get('/', async (req, res) => {
    const items = await Item.find({})
    res.status(200).json(items).end()
})
//in order to use a paramater we need to put ":id"
//then we can use the value of that "id" through req.params.id
//req stands for request, res stands for response
itemsRouter.get('/:id', async (req, res) => {
    const item = await Item.findById(req.params.id)
    //check if the object exists and deliver the relevant status
    if(item){
        res.status(200).json(item).end()
    } else{
        res.status(404).end()
    }
})
//POST or CREATE ednpoint
//we get the data sent to the server through req.body
itemsRouter.post('/', async (req, res) => {
    const body = req.body
    //if the request is missing the description field deliver a 404
    if(!body.description) return res.status(400).end()
    //make the object we are going to insert into the database
    //we get the content from body
    //the id is self generated so we don't have to make one
    //if the likes doesn't exist we will put 0
    const itemObj = new Item({
        description: body.description,
        likes: body.likes || 0
    })
    //save the new object into our database and return a CREATED status
    const savedItem = await itemObj.save()
    res.status(201).json(savedItem).end()
})
//DELETE endpoint
//we search for the object through the id and delete it from our DB
itemsRouter.delete('/:id', async (req, res) => {
    await Item.findByIdAndRemove(req.params.id)
    res.status(204).end()
})
//PUT or UPDATE endpoint
itemsRouter.put('/:id', async (req, res) => {
    const body = req.body
    //we make a new object just like with POST
    const itemObj = {
        description: body.description,
        likes: body.likes
    }
    //findAndUpdate requires the id and then the object you are inserting
    const updatedItem = await Item.findByIdAndUpdate(req.params.id, itemObj)
    res.status(201).json(updatedItem).end()
})

export { itemsRouter }

Utils

This is just good practice to hide sensitive information in your .env file.

config.js

//we need this library to use our .env file
import 'dotenv/config'
//get the variables from the .env file
const PORT = process.env.PORT
const MONGODB_URI = process.env.MONGODB_URI

export {
    PORT,
    MONGODB_URI
}

Server

Here's is where will unite everything and establish a connection to our database. We use mongoose to connect to MongoDB Atlas through an async function. Note that we are using a express-async-errors library so we never had to catch the errors, however in this particular case you can see how to deal with a try/catch block in case the connection to the database fails.
After the database connection we use middleware to make our server work, first one allows the server to parse everything as JSON and second is our Router object that has all our endpoints. If we were to put this on the internet you'd also need to use CORS.

app.js

//example on how to import the whole object called config
import * as config from './utils/config.js'
//get the express object and assign it to app
import express from "express";
const app = express()
//express router
import { itemsRouter } from './controllers/items.js'
//mongoose is needed to connect to the database
import mongoose from 'mongoose'
//async function so we can connect to the database
//throws an error if the connection fails
const connectToDB = async () => {
    try {
        await mongoose.connect(config.MONGODB_URI)
        console.log(`connected to the database`)
    } catch (e) {
        console.log(`error connecting to the db: ${e}`)
    }
}
//run the function so we are connected to the database
connectToDB()
//------------- middleware -------------//
//allow the server read json data
app.use(express.json())
//define the endpoint: localhost:3001/api/items for the router
app.use('/api/items', itemsRouter)

export { app }

Then we import everything into index.js which is our entry point.

//import the app object
import { app } from './app.js'
//get the port from .env
import { PORT } from './utils/config.js'
//run the server on the app object with the defined port
app.listen(PORT, () => {
    console.log(`server running on port ${PORT}`)
})

MERN stack stands for MongoDB, Express, React and Node. It's widely popular due to the fact that you can use JavaScript on the backend and frontend, or TypeScript. It's also fast as scalable, with millions of developers writing code and contributing to the documentation and the ecosystem.

There are variations where instead of React for the frontend you will find Angular, Vue, or even Svelte nowadays, although this last one is still not as popular. Additionally you also have Next.js, Nuxt.js, Remix.js, etc. You can do everything with JavaScript or TypeScript nowadays.

This is the first article of the MERN stack series that will progress in complexity. At the end of it you will be able to create a full stack web app and deploy it anywhere with Docker. But now we will focus on building the simplest RESTful API with Node and Express.

NB: There's an 'older' stack, called LAMP) that consists of Linux, Apache, MySQL, PHP/Perl/Python, depending on your focus you might want to check it out

You can find the next chapters of this series here: RESTful API with Node.JS, Express and MongoDB.

Concepts and memes

Node.JS is a runtime environment for JavaScript, it basically allows you to ruin everything with JavaScript anywhere.

Express or Express.js is the most famous backend framework for Node.js. It allows you to build web applications and APIs.

Once you have your project you can put Express.js on top of Node.js in a machine and it will act as a server for requests, you can also connect it to a database and it will server you as an intermediary between user events and the information you want to show.

Write code, not docs - our first server

No, you should totally write documentation but let's start this tutorial.

The first thing we need to do is go to our directory and npm init this will start the package.json, you can just press yes to everything. We will have all the dependencies stored in this file. Instead of npm you could also check out another package manager called Yarn.

Once that's done follow with npm install express and make a index.js file, that's where our code is going to be. If you are using git or any sort of version control remember to ignore the node_modules folder. That's where all your dependencies are, you shouldn't be touching that folder at all.

Let's write our first server, put this into index.js:

//we import the express dependency
const express = require('express')
//assign express to a variable so it's easiet to work with
const app = express()
//this is our first route
//we want to get http://localhost:3001/
//the app already knows that we are on localhost
//we just need to indicate that's the entry point with '/'
//req stands for request and res for response
app.get('/', (req, res) => {
    //we reply to the get endpoint by sending a text back
    res.send('hello world!')
})
//define the port where our app is gonna run
const PORT = 3001
//make the server listen to that port
app.listen(PORT, () => {
    //the console.log is for your CLI
    //so you know the server has started
    console.log(`server running on localhost with port ${PORT}`)
})

Once you have it you can run your server through CLI by typing node index.js in your terminal (remember you have to be in the folder where index.js is). We are using our Node runtime to execute the code in our index.js file.

NB: Every time you make a change in your code you need to restart your server, you exit it with CTRL+C from the terminal and every time you want to run it again you need to type 'node index.js'. In order to get around this you could install nodemon

Afterwards in your browser go to http://localhost:3001/ You should see the following:

RESTful API

There are several discrepancies between the theory and the practice of building a RESTful API, as well as different definitions, here we will go with a CRUD example. Which stands for Create, Read, Update and Delete. We will build endpoints for all these operations.

Before starting make sure you have this middleware on top of your file

//middleware so the server can parse json data
app.use(express.json())

and also make a "database" like this:

let infoDB = [
    {
        id: 1,
        info: "example of information",
        date: "2022-04-03T20:00:22.158Z"
    },
    {
        id: 2,
        info: "example of information2, put anything here",
        date: "2022-03-03T20:00:22.158Z"
    }
]

This is how the top of your file should look like:

Additionally I will be using Postman to make calls to my endpoints, it's free and simplifies the process of development, however there are many alternatives.

GET or Read

The code is very straightforward, we make a call to /api/info and get a response with all the data back in JSON format.

//an endpoint to return all our infor back
app.get('/api/info', (req, res) => {
    //we send the information back in JSON format
    //along with a 200 status which means it's successful
    res.status(200).json(infoDB)
})

The call to Postman should return:

We will also add a way to get a specific item from the database through the id but you can check it out later, first get familiar with the rest of the code.

//and endpoint to return an object with a specific id
app.get('/api/info/:id', (req, res ) =>{
    //we parse the request id since it's a string
    const id = Number(req.params.id)
    //find the object the user wants in our database
    const infoObject = infoDB.find(info => info.id === id)
    //return with the needed status and the object if you find it
    infoObject
        ? res.status(200).json(infoObject).end()
        : res.status(404).json({
            error: `an object doesn't exist with the id ${id} in our database`
        }).end()
})

POST or Create

In order to make this endpoint we will need an additional helper function to generate ids for our data:

//a helper function to generate an id
//if you want to make it simpler just return infoDB.length + 1
const generateId = () => {
    const maxId = infoDB.length > 0
        ? Math.max(...infoDB.map(i => i.id))
        : 0
    return maxId + 1
}

If you are unfamiliar with the methods what you do here is first make an array of items with the ids through map then we use the spread operator (the three dots) to spread the array and pass it to Math.max which will return the maximum item.

Afterwards we use a ternary operator to assign either the maximum id or 0.

Here's the Express endpoint for a POST method:

//an endpoint to add a note
app.post('/api/info', (req, res) => {
    //without the middleware express.json() on L6 we wouldn't be able to read this data
    //extract the body from the request
    const body = req.body
    //check if the body doesn't contain any info
    if(!body.info) {
        //return an error status and a json response
        return res.status(400).json({
            error: 'you need to type the information'
        })
    }
    //we make an object from the request to add it to our database
    //in order to generate the id and the date we use helper functions
    //read more about how to use Date: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date
    const infoObject = {
        id: generateId(),
        info: body.info,
        date: new Date()
    }
    //we modify our database by adding our object to it
    //first we copy the data (so we don't modify existing data), add our object and finally store everything back
    //feel free to read about ACID: Atomicity, Consistency, Isolation, and Durability
    infoDB = infoDB.concat(infoObject)
    //we send a response with the stored object
    //and a status of 201 which means created
    res.status(201).json(infoObject)
})

In order to make a call to our post endpoint and send the data it needs we use Postman as follows:

Make sure to select all the parameters in the picture: body -> raw and JSON format. After clicking the SEND button you should get back this:

DELETE

//an endpoint to delete information from our db
//the ":id" at the end means we will be using that as a parameter
//we get the id from params.id in the request
app.delete('/api/info/:id', (req, res) => {
    //parse the data as a number since req.params.id is a string
    const id = Number(req.params.id)
    //filer back all the data that doesn't contain the requested id
    infoDB = infoDB.filter(i => i.id !== id)
    //return a code with no content and close the operation
    res.status(204).end()
})

If we want to delete the information object with the id 3 this is our call on Postman:

PUT or Update

We will be using a helper function to obtain the object we want to modify and its id:

//helper function for the update operation
//we get the id of the object we want to modify
//then return the object to modify and its index in infoDB
const getInfoObjectAndIndex = (id) => {
    //declare our variables
    let infoToModify = {}
    let indexOfInfo = 0
    //find the object through the id and assign our values
    for(let i = 0; i < infoDB.length; i++){
        if(id === infoDB[i].id) {
            infoToModify = infoDB[i]
            indexOfInfo = i
        } 
    }
    //return our values in form of an array
    //so we can use destructuring to assign our variables
    return [infoToModify, indexOfInfo]
}

The actual endpoint call is as follows:

//an endpoint to update info
app.put('/api/info/:id', (req, res) => {
    //get the data from the body
    const body = req.body
    //if we don't get the information for our object throw an error
    if(!body.info) return res.status(400).json({
        error: 'the content is missing'
    })
    //check what you are getting(feel free to add console logs everywhere)
    console.log(body)
    //parse the date into a number
    const id = Number(req.params.id)
    //we use destructuring assignment here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment
    const [infoToModify, indexOfInfo] = getInfoObjectAndIndex(id)
    //a console log to show the object we got
    console.log(infoToModify)
    //we make a new object with the new values
    const modifiedInfo = {
        id: infoToModify.id,
        info: body.info,
        date: infoToModify.date
    }
    //insert (or overwrite) our object into the database
    infoDB[indexOfInfo] = modifiedInfo
    //send a success status
    res.status(200).json(modifiedInfo).end()
})

In order to update our data through Postman the call should be:

(that's the data you get after you press SEND, check through a GET call that you have all your data in the database)

GitHub and disclaimer

You can find the source code on my Github here you have the index.js alone.

Note that this is the first chapter in a series of guides, here I haven't followed the best practices and this guide was solely made for learning purposes. If you are developing a proper product remember to use the best security practices.

To get in direct touch with me you can hit me up on Twitter @tekbog

This guide is on how to build a full RESTful API with Spring Boot so you can make CRUD operations on MySQL database.

You can find the code to the full project on my GitHub.

If you are unfamiliar with Spring Boot I have better introductory articles:

• Make a single GET endpoint
• APIs with Spring Boot

For any questions you can reach me on Twitter @tekbog

Architecture

The project will have two layers:

• Business (where the logic is): Service and Controller
• Data access layer: Repository and Entity

Additionally you would have a "view" layer to display the data properly using a front-end framework or JSP but this tutorial will only cover how to handle the API. If you are unfamiliar with some of these terms search for "MVC" and "3 tier" architectures.

Connect to the database

Before starting to code anything make sure your database is working and it's connected to your project. If everything is running then proceed to modify resources/application.properties file:

#autotable creation
spring.jpa.hibernate.ddl-auto=update
#db connected
spring.datasource.url=jdbc:mysql://localhost:3306/db_restful_example_spring
spring.datasource.username=root
spring.datasource.password=password

Entity and Lombok

The very first thing is to know the data we are going to store in our database, in this case it will be as follows:

• id (Long)
• data (String)
• email (String)
• name (String)

Once that's clear we will proceed to make an Entity Class:

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.NonNull;

import javax.persistence.*;

//we define this is an entity
@Entity
//using lombok
//generates all the boilerplate that is normally associated with simple POJOs (Plain Old Java Objects)
// and beans: getters for all fields, setters for all non-final fields, and appropriate toString, equals and hashCode
//https://projectlombok.org/features/Data
@Data
//generate constructors
// https://projectlombok.org/features/constructor
@NoArgsConstructor
@AllArgsConstructor
public class DataExample {
    //all entity objects need an id
    //in this case every time new data added to the object
    //we get a new id
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @Column(name = "data_id", nullable = false)
    private Long dataId;

    //make sure the email is not null
    //https://projectlombok.org/features/NonNull
    @NonNull
    private String email;
    private String name;
    private String data;

}

We use Lombok to generate all the boilerplate code as explained above.

If you start the app you will generate the following table:

Set up the Repository

The Repository annotation allows us to access and modify the data however we want, and it also gives us a lot ofimplemented methods thanks to JPA. Additionally we can also use keywords to make methods we want without writing JPQL or additional queries.

import com.example.restfultutorialmysql.entity.DataExample;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

import java.util.List;

//this is the implementation of the DAO interface
//data access object which lets you access and modify data
//the parameters are <TypeOfData, TypeOfId>
@Repository
public interface DataExampleRepository extends JpaRepository<DataExample, Long> {

    //this are JPA additional methods
    //follow the docs to know how to create them
    //https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#jpa.query-methods.query-creation
    //uncomment this and try to implement them in the Service yourself

//    //get data by email
//    List<DataExample> findDataExampleByEmail(String email);
//
//    //get data by name
//    List<DataExample> findDataExampleByName(String name);
//
//    //get data by id
//    List<DataExample> findDataExampleByDataId(Long id);
}

Service

Following good practice first I will set up the interface and afterwards implement all the methods.

The interface:

import com.example.restfultutorialmysql.entity.DataExample;

import java.util.List;

public interface DataExampleService {
    //GET
    List<DataExample> getAllData();

    //POST
    DataExample saveData(DataExample dataExample);

    //PUT
    DataExample updateDate(DataExample dataExample);

    //DELETE
    void deleteDataById(Long id);

}

The implementation:

import com.example.restfultutorialmysql.entity.DataExample;
import com.example.restfultutorialmysql.repository.DataExampleRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.List;

//we define this cass as a service
//so we can use it as a bean in our Controller
@Service
public class DataExampleServiceImpl implements DataExampleService{

    //bean injection
    //https://stackoverflow.com/questions/39890849/what-exactly-is-field-injection-and-how-to-avoid-it
    //the link above is to clarify if you get any kind of warning with @Autowired
    //we get the repository that will interact
    //with the database through our entity objects
    @Autowired
    DataExampleRepository dataExampleRepository;

    //implement all the CRUD methods
    //READ
    @Override
    public List<DataExample> getAllData() {
        return dataExampleRepository.findAll();
    }
    //CREATE
    @Override
    public DataExample saveData(DataExample dataExample) {
        return dataExampleRepository.save(dataExample);
    }
    //UPDATE
    @Override
    public DataExample updateDate(DataExample dataExample) {
        return dataExampleRepository.save(dataExample);
    }
    //DELETE
    @Override
    public void deleteDataById(Long id) {
        dataExampleRepository.deleteById(id);
    }
}

Note that we have implemented the CRUD operations: create, read, update and delete.

Controller: setting up our endpoints

Now that we have all good practices in place it's time to make our endpoints.

import com.example.restfultutorialmysql.entity.DataExample;
import com.example.restfultutorialmysql.service.DataExampleServiceImpl;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.*;

import java.util.List;

//@RestController annotation will make this class the controller
//meaning that it's listening to htttp://localhost:8080 calls
//and replying with whatever endpoints you have made
@RestController
//if you want to add the api version, which is good practice
//add this annotation
//localhost:8080/api/v1
//@RequestMapping("api/v1")
public class Controller {

    //we inject the service bean
    //and use it to handle the data
    //after every call to the endpoint we will do dataExampleService.method(methodParam);
    @Autowired
    DataExampleServiceImpl dataExampleService;

    //get all the data from the database using the service
    //once the call is successful respond with an OK status
    @GetMapping("/data")
    public ResponseEntity<List<DataExample>> getData(){
        return new ResponseEntity<List<DataExample>>(dataExampleService.getAllData(), HttpStatus.OK);
    }
    //you can also just get all the data
    //without making a ResponseEntity object
//    @GetMapping("/data")
//    public List<RandomData> getData(){
//        return dataService.getRandomData();
//    }

    //save the data you send and return CREATED
    //valid docs: https://commons.apache.org/proper/commons-lang/javadocs/api-3.1/org/apache/commons/lang3/Validate.html
    //difference with @Validated: https://stackoverflow.com/questions/36173332/difference-between-valid-and-validated-in-spring
    //ReQuestBody means you are expected to introduce data into your request
    @PostMapping("/data")
    public ResponseEntity<DataExample> saveDate(@Validated @RequestBody DataExample dataExample){
        return new ResponseEntity<>(dataExampleService.saveData(dataExample), HttpStatus.CREATED);
    }

    //update the data
    @PutMapping("/data")
    public ResponseEntity<DataExample> updateData(@RequestBody DataExample dataExample){
        return new ResponseEntity<>(dataExampleService.updateDate(dataExample), HttpStatus.OK);
    }

    //delete the data
    //PathVariable means you are expected to pass a parameter/variable
    //{id} translates into the Long id that you interact with afterwards
    @DeleteMapping("/data/{id}")
    public ResponseEntity<HttpStatus> deleteData(@PathVariable Long id){
        dataExampleService.deleteDataById(id);
        return new ResponseEntity<>(HttpStatus.NO_CONTENT);
    }

}

Testing our app

We will be using Postman for this purpose.

POST

First we will test the POST method:

Make sure you send the data as raw->JSON

{
    "email": "email3@email.com",
    "name": "name example3",
    "data": "this is an example of data3"
}

As you can after we send the data we get it back with an assigned ID and a 201 OK status code.

The {{url}} is just a local variable that means:

You can define local variables while working with Postman, in our case is http://localhost:8080 make sure you are using the correct port, if you want to make a call on postman without defining variables just write http://localhost:8080/data

We have inserted data into our database, let's see if it's stored correctly:

GET

And now let's test the GET method:

The returned status is 200, so everything works as expected.

PUT

DELETE

Finally we test the delete method by deleting one of the items in our database:

Check afterwards that it got deleted:

The tech industry is continuously evolving, changing, new tools are made every day and new frameworks are created or mass adopted. The point of this post is to glimpse at what's going on, check the current trends and where the industry is headed.

Languages

If you ever want to know what language is the most popular a good and don't know where to look the best place might be TIOBE:

Python currently being king but closely followed by C and Java. Let me give a quick explanation on why those languages are the most used.

Python

One of the selling points of Python is just how easy the syntax is and how it almost reads as pseudocode.

Python has libraries for all your needs and despite being criticized as slow it's widely used everywhere. Most projects related to data science or machine learning will rely on Python, not to mention it has three of the most popular web frameworks right now: Django, Flask and FastAPI.

It's the perfect language for beginners but at the same time it's widely adopted everywhere, if you don't know where to start learn Python but good looking trying to figure out the best environment:

There's always a relevant xkcd

C

C isn't as trendy or flashy but it's everywhere and does everything. Operating systems, embedded programming, etc. Both Git and Linux are written mostly in C.

If it's good enough for Linus then it's good enough for the rest of us.

Java

Java, owned by Oracle, is enterprise king. Google, Netflix, Amazon, etc use Java in one way or another.

Most Apache projects are also written in Java - if you are curious to why, here's a HN reply.

Learn Java, learn Spring and you will never be out of a job. Check out some of the articles and tutorials on this blog about Java and Spring if you don't know where to start. The first post is: How to Make your first API.

PHP

Before JavaScript there was PHP, after JavaScript there's still PHP.

This language went through a lot of iterations over the years to improve the developer's experience. Nowadays it's mainly used as a backend language on the web. It has two of the biggest frameworks: Laravel and Symfony.

However PHP's main use comes from WordPress. If you haven't heard or used WordPress, you should have a look. We don't know the exact statistics but everyone keeps saying that it powers over 33% of the web. Why? It's fast to set up, design and have an e-commerce site ready or anything you'd want, there are millions of plugins and it's fairly easy to use for no-coders.
The perfect CMS, you can have a site with a blog in less than a day and focus on SEO (marketing loves WordPress for how easy it makes positioning) or whatever the business needs.

If you ever hear that PHP is dead or dying, well, just look at the amount of jobs there are.

JavaScript

Not long ago The State of JS 2021 was released. It doesn't take a very large pool of votes but it's widely known in the dev community and worth looking into. It's probably one of the most important surveys to consider if you have anything to do with JavaScript. Most tools/frameworks work with TypeScript as well, if you were in doubt.

Front-end Frameworks

The big three are there as always there: React, Angular and Vue:

Last year Svelte made a lot of noise and it's slowly being adopted, the docs have improved a lot and after trying it for a bit I consider myself a fan as well for its ease of use.
However it's not the only new kid on the block, I've seen a lot of praise for both SolidJS and AlpineJS.

Back-end Frameworks

The results aren't going to surprise anyone, if you are a backend developer you have to know Express:

Although the amount of new tools we got this past year has been impressive:

From what I've seen I believe Remix is the one receiving the most hype right now but Astro and SvelteKit could be here to stay.

Testing? Do people actually..?

Yes, TDD is a thing.
Jest or Mocha at the top. The usual.

Bloated apps for mobile and desktop

Don't freak out if you open your Discord desktop app and you can inspect its source code as if it was an ordinary website. Everything is JavaScript now, well, TypeScript actually.

Electron and React Native shouldn't surprise anyone, with Cordova and Ionic following close.

It will be fun to see how Tauri can compete or find its market considering Flutter is here.

Build Tools

Nothing out of the ordinary, however I expect Vite to be widely adopted by next year.

Libraries

Here we have Axios, Lodash, Moment, Redux, etc. Nothing out of the ordinary.

Out of the list you should know Redux and Tailwindcss.

Google shenanigans

Google has been hard at work this last couple of years when it comes to their tools, and I'm not talking about Kubernetes here or the awful state of the search engine.

Go

If you are a backend developer you probably have noticed the increased adoption of the Go language. Considering that K8s is written in Go and the amount of companies that have started to use it I'd keep an eye on the language and its ecosystem.

Flutter

Google has decided Facebook (Meta) is too cool with their React Native and who doesn't want to get rid of Electron? Enter Flutter. They might be getting a bit too ambitious here but working with Flutter and Dart is a breeze.

I've directed a multi-platform project this past year using Flutter + Firebase. It's fairly easy to setup and make an MVP, I recommend it highly for a fast paced project where the client doesn't want a pixel-perfect design, perfect for startups.

For anything else? Well, as always, it depends.

What else is out there?

Terraform, serverless architecture, Rust, micro frontends and everything involved in Web 3.0: blockchain, crypto, NFTs, all that jazz.

C'est fini

The cover art is from Joan Cornellà.

If you think I'm wrong we can argue on Twitter: hmu @tekbog.

Disclaimer: I haven't talked about C#, C++, Ruby, Swift, R, etc because they are out of my area of expertise.

TL;DR

1. Make a database
2. Connect database to our project
3. Use GET calls to read from SQL database with Spring Boot
4. Source code on Github
5. Architecture:

For any questions you can ask me on Twitter @ tekbog

Set up the database (and make sure it's running)

In my case I'm gonna use MySQL Workbench to create a database called "mydb" and then add a table called "my_table"

Once that's done run this query to put some data in your database:

USE mydb;

CREATE TABLE my_table 
(
    id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    name VARCHAR(255) NOT NULL,
    email VARCHAR(255) NOT NULL
);

INSERT INTO my_table (name, email) VALUES ("random_name", "random@random.com");
INSERT INTO my_table (name, email) VALUES ("random_name1", "random1@random.com");
INSERT INTO my_table (name, email) VALUES ("random_name2", "random2@random.com");

Starting with Spring Boot

Connect your database to the Spring Boot app by changing

application.properties

The port is default, make sure it's the same. You can check the port number when you start the Spring Boot app.

The username and password pertain to your MySQL database.

spring.datasource.url=jdbc:mysql://localhost:3306/mydb
spring.datasource.username=root
spring.datasource.password=password

And the dependencies are:

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

    </dependencies>

You can use Spring Initializr as well to set up your project, it's highly recommended.

Postman

In order to make calls to the server I'm going to be using Postman Although there are many alternatives.

Architecture and code

The "folders" are called packages, once you create them: inside you can make classes, interfaces, etc.

Entity

UserData class makes it possible to interact with the database

package com.example.sqldbexample.entity;


import lombok.Getter;

import javax.persistence.*;

//lombok getter so we don't have to write getters
@Getter
//The entity interacts with the database
@Entity
@Table(name = "my_table")
public class UserData {
    //an id is always needed to index the data
    //the annotation GeneratedValue allows the data to be generated automatically
    //and we add the name of the columns where the data goes
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @Column(name = "id", nullable = false)
    private Long id;

    @Column(name = "name")
    private String name;

    @Column(name = "email")
    private String email;
}

To begin we import Lombok so we don't have to write the getters for the class. Afterwards we define the class as Entity and map the Java objects to the database we are accessing. The most confused annotation could be GeneratedValue: we always need an id and in this case we made our id to be generated automatically. Spring annotations have the following syntax:

@AnnotationName(parameter="parameter value")

Repository

The Repository interface is a DAO (data access object) bean that simplifies our interaction with SQL thanks to JPA, the interface takes 2 parameters: the type of object and the type of id, after which it will allow you to use its extended methods:

In our case we want to interact with the entity object we have created called UserData and the id is Long type.

Note that Repository annotation also indicates that this is a type of bean we can use anywhere in our project.

package com.example.sqldbexample.repository;

import com.example.sqldbexample.entity.UserData;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

//repository to interact with the entity
//we use the JpaRepository because of all its methods
@Repository
public interface UserDataRepository extends JpaRepository<UserData, Long> {
}

Service

UserDataService interface is gonna allow us to use the repository we have just created to get the data we want

package com.example.sqldbexample.service;

import com.example.sqldbexample.entity.UserData;

import java.util.List;

//interface to handle the entity UserData
public interface UserDataService {
    List<UserData> getUserData();
}

Now the implementation is as follows:

package com.example.sqldbexample.service;

import com.example.sqldbexample.entity.UserData;
import com.example.sqldbexample.repository.UserDataRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.List;

@Service
public class UserDataServiceImpl implements UserDataService{

    //get the repository bean
    @Autowired
    UserDataRepository userDataRepository;

    //we use the repository bean to get access to all the methods
    //like findAll
    @Override
    public List<UserData> getUserData() {
        return userDataRepository.findAll();
    }
}

Thanks to the JPA repository we get access to all its methods without having to implement any of them, it just works with our MySQL database. So in order to get all the users from the table we use .findAll() method.

API

You can read my previous article if you are unfamiliar with the Controller

//@RestController annotation will make this class the controller
//meaning that it's listening to htttp://localhost:8080 calls
//and replying with whatever endpoints you have made
@RestController
//localhost:8080/api/v1
@RequestMapping("api/v1")
public class Controller {

    @Autowired
    private UserDataService userDataService;

    @GetMapping("/getdata")
    public List<UserData> getData(){
        return userDataService.getUserData();
    }
}

The main difference is the fact that we add RequestMapping so everything goes to api/v1 path after localhost:port/. Putting the version of your API is good practice.

To sum it up we use the UserDataService bean (service), the Autowired annotation allows us to get the bean through Inversion of Control (Spring magic) and use that interface's implementation without calling it ourselves.

Test that everything works

Start the server, make sure your database is running and then using Postman call localhost:8080/api/v1/getdata

you should get this back:

Java is often hated for various reasons one of them is being too verbose.

However it's a great language that's continuously evolving and it's an absolute king of enterprise software, for the good or the bad. One of its main strengths is the Spring Framework, and for anyone who wants to make a project quickly Spring Boot is the fastest and easiest way to make your backend with Java. Spring is a tested framework that has evolved through the years, just like Java, it's secure and you can find any bugs you will encounter on StackOverflow with ease.

Under the hood Spring is very complex and let's you trivialize a lot of needed implementations thanks to its inversion of control and dependency injection, you use all the power of the framework through annotations.

Start the project

Start the project with Spring Initializr if your IDE doesn't have it built-in just download it (generate) from the site and include the dependency of Spring Web as follows:

RestController

After you have started your project all you need to do is make a new class anywhere and just add the @RestController this will instantly tell Sping that this class will have RESTful endpoints:

@RestController
public class Controller {
}

RestController annotation will make this class the controller meaning that it's listening to htttp://localhost:8080 calls and replying with whatever endpoints you have made.

For clarification:

• 8080 is the default port but you can change it in application.properties
• all the Spring annotations start with @

GetMapping

This GetMapping annotation makes the htttp://localhost:8080/api endpoint, after the call it will return or do whatever you put in the api() method. The name of the method is irrelevant, and it can also be void.

If you want your endpoint to be something other than /api just write the name you want.

@GetMapping("/api")
public String api(){
    return "call to /api";
}

When you type in the browser the URL you will see the following:

RequestParam

If you want to extract a query parameter in your get method then you need to add the RequestParam as the following:

@GetMapping("/call")
public String call(@RequestParam(value = "val") String val){
    return "the val is "+val;
}

This is an example of a call:

The enpoint htttp://localhost:8080/call will read the parameter you pass in the browser, for example

http://localhost:8080/call?val=myparam

Controller class code & TL;DR

//@RestController annotation will make this class the controller
//meaning that it's listening to htttp://localhost:8080 calls
//and replying with whatever endpoints you have made
@RestController
public class Controller {

    //@GetMapping annotation makes the htttp://localhost:8080/api endpoint
    //it will return or do whatever you put in the api() method
    @GetMapping("/api")
    public String api(){
        return "call to /api";
    }

    //the enpoint is htttp://localhost:8080/call
    //it also reads the parameters you pass in the browser
    //for example http://localhost:8080/call?val=myvalue
    //it can read the parameters because of the @RequestParam annotation
    @GetMapping("/call")
    public String call(@RequestParam(value = "val") String val){
        return "the val is "+val;
    }

}

Official Spring documentation:

• RequestParam
• GetMapping

LeetCode defines itself as

LeetCode is the best platform to help you enhance your skills, expand your knowledge and prepare for technical interviews.

Is it truly the best? That's debatable (I will explain in another article) however it's very useful and it's been a standard for interviews - especially from FAANG companies (or MAANG now) - since the tech boom and the need to filter candidates. I personally like it because I generally enjoy challenging programming problems but most people use the platform just to prepare for interviews.

JavaScript Arrays

LeetCode has something called "Explore Cards" so I decided to dive into Arrays

Next you will find all the problems and its solutions with comments.
The full repository is here. And you can find me on Twitter @tekbog

Max Consecutive Ones

Given a binary array nums, return the maximum number of consecutive 1's in the array.

Solution:

var findMaxConsecutiveOnes = function(nums) {
     //initialize a variable to keep track of the max consecutive ones
    var contmax = 0;
    var cont = 0;
    for(let i = 0; i < nums.length; i++){
        // console.log("i is: "+nums[i]);
        if(nums[i]){
            //simple counter, if there's something in the array then add 1 to the counter
            cont++;
            // console.log("cont is "+cont);
        } else {
            console.log("it's a zero " +i)
            console.log("cont is "+cont)
            //check if the current consecutive ones is greater than the max consecutive ones we have already
            contmax = Math.max(cont, contmax);
            cont = 0;
        }
    }
    // console.log("contmax is "+contmax)
    // this is a ternary operator
    //https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Conditional_Operator
    // check which counter is higher and return it in case the last counter is higher
    return cont > contmax ? cont: contmax;
};

Find Numbers with Even Number of Digits

Given an array nums of integers, return how many of them contain an even number of digits.

Solution:

 var findNumbers = function(nums) {
     //log for debugging
    console.log(...nums)
    //initialize a variable to keep track of the even numbers of digits
    let count = 0;
    //iterate through the array
    for (let num of nums){
        console.log(num.toString().split(''))
        //convert the number to a string to use string methods
        //we can clearly see the number of digits in every number now
        let innerNum = num.toString().split('')
        if (innerNum.length % 2 === 0){
            //if the number of digits is even, add one to the count
            count++
        }
    }
    return count;
};

Squares of a Sorted Array

Given an integer array nums sorted in non-decreasing order, return an array of the squares of each number sorted in non-decreasing order.

Solution:

 var sortedSquares = function(nums) {
    //create a new array to store the squares
    let arr = [];
    //push the squares of the numbers into the array
    for (let i of nums){
        arr.push(i*i)
    }
    console.log(arr)
    //sort the array in non-decreasing order
    arr.sort((a,b) => a-b)
    console.log(arr)
    return arr
};

Duplicate Zeros

Given a fixed-length integer array arr, duplicate each occurrence of zero, shifting the remaining elements to the right.

Note that elements beyond the length of the original array are not written. Do the above modifications to the input array in place and do not return anything.

Solution:

var duplicateZeros = function(arr) {
    for (var i=0; i<arr.length; i++) {
        if (arr[i] === 0) {
            //use the method splice to insert a new zero after the current zero
            arr.splice(i, 0, 0);
            //get rid of the last element since we just inserted a new zero so there's less space in the array
            arr.pop();
            //increment i to skip over the new zero we just inserted
            i+=1
        }
    }
};

Merge Sorted Array

You are given two integer arrays nums1 and nums2, sorted in non-decreasing order, and two integers m and n, representing the number of elements in nums1 and nums2 respectively.

Merge nums1 and nums2 into a single array sorted in non-decreasing order.

The final sorted array should not be returned by the function, but instead be stored inside the array nums1. To accommodate this, nums1 has a length of m + n, where the first m elements denote the elements that should be merged, and the last n elements are set to 0 and should be ignored. nums2 has a length of n.

Solution:

 var merge = function(nums1, m, nums2, n) {
    // console.log(nums1.slice(0,m))
    for (let i = 0; i < n; i++){
        console.log(nums1[m])
        //since we know that from 'm' the array will be free we can start inserting elements there
        nums1[m] = nums2[i]
        console.log(nums1[m])
        //incremenet the index 'm' so we can insert the next element properly
        m++
    }
    //remember to sort the array in a non-decreasing order
    nums1.sort((a,b) => a - b);
    console.log(nums1)
};

Remove Element

Given an integer array nums and an integer val, remove all occurrences of val in nums in-place. The relative order of the elements may be changed.

Since it is impossible to change the length of the array in some languages, you must instead have the result be placed in the first part of the array nums. More formally, if there are k elements after removing the duplicates, then the first k elements of nums should hold the final result. It does not matter what you leave beyond the first k elements.

Return k after placing the final result in the first k slots of nums.

Do not allocate extra space for another array. You must do this by modifying the input array in-place with O(1) extra memory.

Solution:

const removeElement = (nums, val) => {
    for (let i = 0; i < nums.length; i++) {
      if (nums[i] === val) {
        // use the method splice to remove the element that's equal to val
        nums.splice(i, 1);
        // decrement i by 1 to account for the splice
        i--;
      }
    }
    //since we removed elements from the array we just return the length of the array
    return nums.length;
  };

Remove Duplicates from Sorted Array

Given an integer array nums sorted in non-decreasing order, remove the duplicates in-place such that each unique element appears only once. The relative order of the elements should be kept the same.

Since it is impossible to change the length of the array in some languages, you must instead have the result be placed in the first part of the array nums. More formally, if there are k elements after removing the duplicates, then the first k elements of nums should hold the final result. It does not matter what you leave beyond the first k elements.

Return k after placing the final result in the first k slots of nums.

Solution:

var removeDuplicates = function(nums) {
    // nums = [...new Set(nums)]
    // return nums.length
    // doesn't work cuz it has to be in place
    for (let i = 0; i < nums.length; i++) {
        //remove the next element if it's the same as the current element
        if (nums[i] === nums[i + 1]) {
            //use the method splice to remove the element
            nums.splice(i, 1)
            //decrement i to account for the splice
            i--
        }
    }
};

Check If N and Its Double Exist

Given an array arr of integers, check if there exists two integers N and M such that N is the double of M ( i.e. N = 2 * M).

Solution:

 var checkIfExist = function(arr) {
    //make a flag to check if there is a double
    //update it if it exists
    let res = false;
    console.log(`arr is ${arr}`);
    //use forEach to iterate through all the elements in the array
    arr.forEach( (e, i) => {
        //check if the double of e exists in the array with indexOf
        if(arr.indexOf(e*2) !== -1){
            //check we are not iterating over the same element
            if(arr.lastIndexOf(e*2) !== arr.indexOf(arr[i])){
                res = true
            }
            }
        }
    )
    return res
};

Valid Mountain Array

Given an array of integers arr, return true if and only if it is a valid mountain array.

Solution:

 var validMountainArray = function(arr) {
     //get the max of the array to know the "top" of the mountain
    let max = Math.max(...arr);
    //get its index as well
    let maxIndex = arr.indexOf(max);
    //debugging 101
    console.log(`max is ${max}`);
    console.log(`maxIndex is ${maxIndex}`);
    //check for edge cases
    if(maxIndex === 0 || maxIndex === arr.length-1){
        return false;
    }
    //iterate through the first half of the "mountain"
    for(let i = 0; i < maxIndex; i++){
        console.log('first half');
        console.log(`is is ${i} i+1 is ${i+1}`);
        //if the mountain isn't strictly increasing, return false
        if(arr[i] > arr[i+1] || arr[i] === arr[i+1]){
            return false;
        }
    }
    //iterate through the second half of the "mountain"
    for(let i = maxIndex; i < arr.length; i++){
        console.log('second half');
        console.log(`is is ${i} i+1 is ${i+1}`);
        //if the mountain isn't strictly decreasing, return false
        if(arr[i] < arr[i+1] || arr[i] === arr[i+1]){
            return false;
        }
    }
    //return true if the mountain is valid
    return true;
};

Replace Elements with Greatest Element on Right Side

Given an array arr, replace every element in that array with the greatest element among the elements to its right, and replace the last element with -1.

After doing so, return the array.

Solution:

 var replaceElements = function(arr) {
     console.log(arr);
    for(let i = 0; i < arr.length-1; i++){
        //assign max as the next element to check
        let max = arr[i+1];
        //iterate through the right side of the array
        for(let j = i+1; j<arr.length; j++){
            //check if there's an element that's greater than the current max
            if(arr[j] > max){
                max = arr[j];
            }
        }
        //put the max element as you finish the array
        arr[i] = max;
    }
    //put the last element as -1 since the problem asks for it
    arr[arr.length-1] = -1;
    return arr;
};

Remove Duplicates from Sorted Array

Given an integer array nums sorted in non-decreasing order, remove the duplicates in-place such that each unique element appears only once. The relative order of the elements should be kept the same.

Since it is impossible to change the length of the array in some languages, you must instead have the result be placed in the first part of the array nums. More formally, if there are k elements after removing the duplicates, then the first k elements of nums should hold the final result. It does not matter what you leave beyond the first k elements.

Return k after placing the final result in the first k slots of nums.

Do not allocate extra space for another array. You must do this by modifying the input array in-place with O(1) extra memory.

Solution:

 var removeDuplicates = function(nums) {
    // nums = [...new Set(nums)]
    // return nums.length
    // doesn't work cuz it has to be in place
    for (let i = 0; i < nums.length; i++) {
        //remove the next element if it's the same as the current element
        if (nums[i] === nums[i + 1]) {
            //use the method splice to remove the element
            nums.splice(i, 1)
            //decrement i to account for the splice
            i--
        }
    }  
};

###Move Zeroes Given an integer array nums, move all 0's to the end of it while maintaining the relative order of the non-zero elements.

Solution:

 var moveZeroes = function(nums) {
     console.log(nums);
     //edge case in case arr is empty
     if (nums.length === 0) return
     //make two pointers
     let positiveEleTracker = 0, j = 0
     //loop through array finding positive elements and filling the zeroes
     while (j < nums.length) {
         if (nums[j] !== 0) {
             //once we find a positive element swap it and keep count of positive elements
             nums[positiveEleTracker] = nums[j]
             positiveEleTracker++
         }
         //keep iterating through array
         j++
     }
     console.log(nums);
     //use the positiveEleTracker to fill the rest of the array with zeroes
     //fill the rest of the array with zeroes
     while (positiveEleTracker < nums.length) {
         nums[positiveEleTracker] = 0
         positiveEleTracker++
     }
     console.log(nums);
 }

Sort Array By Parity

Given an integer array nums, move all the even integers at the beginning of the array followed by all the odd integers.

Return any array that satisfies this condition.

Solution:

 var sortArrayByParity = function(nums) {
     //edge case if arr length is 0
    if(nums.length === 0) return nums;
    //keep track where the even index is
    let evenIndex = 0;
    for(let i =0; i< nums.length; i++){
        //if even
        if(nums[i] % 2 == 0){
            console.log(nums[i]);
            //saving the val that's gonna get rewriten
            //this is the odd value
            let oddTempVal = nums[evenIndex];
            console.log("temp is "+oddTempVal);
            //if the value is even it should go at the beginning
            //there are no strict rules so any even value will do
            nums[evenIndex] = nums[i];
            //keep track of the even index
            evenIndex++;
            // place the odd value where the even value was
            nums[i] = oddTempVal;
        }
    }
    console.log(nums);
    return nums;
};

Remove Element

Given an integer array nums and an integer val, remove all occurrences of val in nums in-place. The relative order of the elements may be changed.

Since it is impossible to change the length of the array in some languages, you must instead have the result be placed in the first part of the array nums. More formally, if there are k elements after removing the duplicates, then the first k elements of nums should hold the final result. It does not matter what you leave beyond the first k elements.

Return k after placing the final result in the first k slots of nums.

Do not allocate extra space for another array. You must do this by modifying the input array in-place with O(1) extra memory.

Solution:

const removeElement = (nums, val) => {
    for (let i = 0; i < nums.length; i++) {
      if (nums[i] === val) {
        // use the method splice to remove the element that's equal to val
        nums.splice(i, 1);
        // decrement i by 1 to account for the splice
        i--;
      }
    }
    //since we removed elements from the array we just return the length of the array
    return nums.length;
  };

Height Checker

A school is trying to take an annual photo of all the students. The students are asked to stand in a single file line in non-decreasing order by height. Let this ordering be represented by the integer array expected where expected[i] is the expected height of the ith student in line.

You are given an integer array heights representing the current order that the students are standing in. Each heights[i] is the height of the ith student in line (0-indexed).

Return the number of indices where heights[i] != expected[i].

Solution:

 var heightChecker = function(heights) {

    //first copy the array using the spread operator
    //then sort the array to check the differences
    const sorted = [...heights].sort((a,b) => a - b)

    console.log(heights);
    console.log(sorted);
    //counter for the different elements
    let diffCount = 0;
    for(let i = 0; i<heights.length; i++){
        if(heights[i] !== sorted[i]){
            console.log("diff");
            diffCount++;
        }
    }
    console.log(diffCount);
    return diffCount;
};

Third Maximum Number

Given an integer array nums, return the third distinct maximum number in this array. If the third maximum does not exist, return the maximum number.

Solution:

 var thirdMax = function(nums) {
    //make a copy of the array without dupes
    const nonDupArr = [...new Set(nums)];
    //order the array from largest to smallest
    nonDupArr.sort((a,b) => b-a);
    console.log(nonDupArr);
    //get the third max number after sorting
    const thirdNum = nonDupArr[2];
    //if the array is less than 3 return the maximum number
    //if it's greater than 3 return the third maximum
    return nonDupArr.length < 3 ? Math.max(...nonDupArr) : thirdNum;
};

Find All Numbers Disappeared in an Array

Given an array nums of n integers where nums[i] is in the range [1, n], return an array of all the integers in the range [1, n] that do not appear in nums.

Solution:

 var findDisappearedNumbers = function(nums) {
     //get the max range meaning it goes from [1,max]
    const maxRange = nums.length;
    console.log(maxRange);
    //array to store the missing numbers
    let res = [];
    //iterate from 1 to the range we need
    for(let i = 1; i <= maxRange; i++){
        //if the number is not in the array, push it to the array
        let found = false;
        //iterate through the array to see if the number is in the array
        for(let j = 0; j < nums.length; j++){
            if (i === nums[j]){
                //if the number is in the array, set found to true
                found = true;
            }
        }
        //only push the numbers that are not found
        if (!found){
            res.push(i);
        }
    }
    console.log(res);
    return res;
};

Squares of a Sorted Array

Given an integer array nums sorted in non-decreasing order, return an array of the squares of each number sorted in non-decreasing order.

Solution:

 var sortedSquares = function(nums) {
    //create a new array to store the squares
    let arr = [];
    //push the squares of the numbers into the array
    for (let i of nums){
        arr.push(i*i)
    }
    console.log(arr)
    //sort the array in non-decreasing order
    arr.sort((a,b) => a-b)
    console.log(arr)
    return arr
};

What happens when you have an idea or side project and don't know where to start? Easy: Wordpress.

WordPress is quick, cheap and easy to use.

It is SEO friendly and you can build anything you want in a matter of hours. It is also probably the best CMS right now and if you are working with non-tech literate people, they can do anything they want through plugins - although this is a double-edged sword.

Hosting solutions

However, the issue is always how/where to host your project - whether to go with shared hosting, dedicated server or VPS. There is also the cost to consider - after all this is a minor side project. I didn't want to spend any valuable time dealing with servers and just wanted to build something as quick as possible.

If the website would have been for a client, and they wanted Wordpress, I would have chosen Kinsta or Pantheon. I have used both in the past and I highly recommend them. The majority of my time was dealing with Kinsta, and while they both have amazing support (especially Kinsta), they were out of my budget.

List of cheap hosting services for WordPress:

1. AwardsSpace
2. NameCheap - EasyWP
3. x10Premium
4. Ionos (2022 update: STAY AWAY FROM IONOS)
5. Bluehost
6. Dreamhost

Popular hosting services that aren't dirt cheap:

1. WPEngine
2. TFWPH
3. FlyWheel

I ended up going with NameCheap - EasyWP, I buy my domains there and it's been a good experience so far.

2022 edit: I had some issues with them but the service is worth checking out, from what I understand they have improved quite a bit.

So what's the best and cheap hosting for Wordrpess?

After my horrible experience I ended up looking into something a bit more complicated to set up, but still quite simple.

DigitalOcean with Plesk

DigitalOcean with Plesk allows you to get a free Plesk license with 3 email accounts. The installation is really simple, you can follow my guide on how to install WordPress with Plesk on DigitalOcean here, the price is almost unbeatable for the amount of features you get.

This is the end result: BaitGaming, the site still needs optimization everywhere and I need to set up cache. But it's the v1.0.0 and we are up and running in record time, the rest of the team can start building online presence and in the near future I will rework the site. It is most likely going to be Next.JS with Wordpress as headless CMS.

To get in touch with me the quickest way is @tekbog and if you are looking for more WordPress content @natmiletic is a good and reliable source and so is @edanbenatar.

DigitalOcean compared to GCP, AWS, Azure, etc, offers a one click deployment with minimal configuration that allows you to host several WordPress sites (or CMS of your choice) through Plesk. This allows you to have personalized email accounts using your domain without additional cost. It's perfect for small businesses, MVPs or early projects.

To start you need a DigitalOcean account. From the referral link you get 100$ for 60 days.

The guide is as follows:

1. Install Plesk on DigitalOcean
2. Set up Plesk and install your WordPress site
3. Set up SSL for WordPress
4. Set up the webmail on Plesk
5. Set up backups or snapshots for your DigitalOcean droplet

If you get lost and need technical support I can ignore you on Twitter @tekbog

Install Plesk on DigitalOcean

What we are going to do is go to marketplace, search for Plesk and click on Create Plesk Droplet:

After we select our machine (remember to save the password, I personally use KeePassXC ).

Set up Plesk and install WordPress

We go into our droplet and copy the ip, which will lead us to this page:

Don't be too quick!

We will need to make an username and password, I'd recommend using a password manager like KeePassXC.

Plesk ready to use:

Click on add a new domain, if you don't have one just select temporary domain and go to the install WordPress step:

Once you have added your domain you will have to configure your DNS, just follow the link:

In my case I had to follow NameCheap 's instructions but you can pick your registrar.

What we want to do next is go to WordPress and install the site:

It will ask for your admin's username and password:

after you set that up it will install WordPress:

Set up SSL for our WordPress site

The first thing you will notice when you have your site ready is that it lacks a security certificate

Web servers work through HTTP however in order to add security and encryption we add an SSL certificate (the 'S' in HTTPS stands for secure, so when you see https it just means the data is encrypted and secure). Just click on the "no certificate button" and "get SSL/TLS certificate":

You will see this page:

Feel free to buy one if you want, however a Let's Encrypt certificate is more than enough, click on install.

We want to secure everything since we will be using the webmail as well:

In order to validate our domain we will need to add a TXT record in our DNS:

Plesk does this automatically.

While this is going on your server (droplet) might suffer a bit, just be patient:

Once you are done you will see that your site contains https:

Set up the webmail on Plesk

Our encryption is ready for our site, let's set up our email. Digital Ocean's default Plesk license allows 3 emails. Go to Mail and click on Create Email Address:

The next thing is to input the data, however I'd recommend on deactivating "Can be used to log in to Plesk" for security reasons:

In order to use our email we can open it through this button: or just go to: https://webmail.your-domain.com

However due to security policies our outgoing mail is ending in spam, to fix this what we need to do is enable DKIM, go to mail, mail settings and click on your account, you will see the following:

Check the DKIM box, click on apply and then OK:

And in order to get rid of the

You cannot send emails from Plesk because outbound connections on TCP port 25 is blocked. Check the firewall settings or contact your hosting provider. If you are sure that the ports are already open, Plesk can recheck them. Start the recheck

Just click on start Recheck, if you are not sure if it has worked go to your DNS settings and check for a DKIM record. I've had problems with this issue before, there's even a support article from Plesk here however you shouldn't worry about doing anything with the firewall. Another possible issue related to DKIM records can also be found here. The solutions is literally to turn it off and on, like always.

It's very important to double check that our email works properly before launching our site or starting a marketing campaign, make sure it doesn't land in spam!

Set up backups for your DigitalOcean droplet

DO NOT FORGET BACKUPS!

You can also use Plesk for backups but we are going to just snapshot our entire VM.

Go to droplets, click on your droplet and then click on backups:

This costs an additional $1, however you can also just do Snapshots on your own, which is what I do for new projects that haven't exactly started yet:

Once you have created your snapshot all you have to do is restore when things go wrong: