Nowadays, using Docker with Microservices is common practice, as it brings isolation, portability, scalability, and more. It will be a surprising for a backend engineer / full-stack engineer not to familiar with Docker (of course, combined with Kubernetes). In the article, I will introduce the core concept of Docker from the practical perspective, whether you are a college student who are interested in leraning Docker or a junior software developer just beginning your career in the software industry, you should be able to grasp the essentils of Docker that you need in practice in 10 minutes by reading the article.

The workflow with Docker in industrial projects

I found the following chart that perfectly explain the workflow with Docker in practice Docker Tutorial for Beginners [FULL COURSE in 3 Hours].

As shown above, the steps of the workflow are (all Docker related terms will be explained in the next section):

• After you complete the application development and commit the code to Git, Jenkins will be notified via webhooks

• Jenkins will buid the docker images according to the dockerfile provide
• Jenkins will push the built images to the Docker repository, like Docker Hub or AWS ECR (Elastic Container Registry)
• The server will pull the images from the Docker repository, then create and run containers according to the Docker Compose files (*compose.yaml files)

The Core Concepts of Docker

In the above workflow, I mentioned many essetianls in Docker. In this section, I will explain them in details.

Docker Sever (Docker Deamon)

It’s responsible for managing Docker containers, images, networks, and volumes. It is run on the same machine where Docker is installed. The Docker Client will send all requests to Docker Server.

Docker Client (CLI)

The Docker Client is the interface that users interact with when using Docker.

Docker Hub

Docker Hub is a Docker Repository. It is the default public registry for Docker images, created and maintained by Docker, Inc.

Docker Container

An instance of an image is called a container. It is the running image.

Docker Image

Stopped container. Docker image are read-only layers, and the docker container is the writable layer on top of a image.

Docker Compose

Docker Compose is a powerful tool for defining and managing multi-container Docker applicaitons. In practice, you can sepcify your applicaton’s services, networks, and volumes in a docker-compose.yml file, and then use a single command to start
them all.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

version: '3.9'

services:
  mongodb:
    image: mongo:6.0 # Replace with the desired MongoDB version
    container_name: mongodb-container
    restart: always
    ports:
      - '27017:27017' # Expose MongoDB's default port
    environment:
      MONGO_INITDB_ROOT_USERNAME: admin # MongoDB root username
      MONGO_INITDB_ROOT_PASSWORD: password # MongoDB root password
    volumes:
      - mongodb_data:/data/db # Persistent data storage

volumes:
  mongodb_data:
    driver: local

Dockerfile

A Dockerfile is a script that contains a series of instructions to build a Docker image, inccluding the base image,
dependencies, configurations, and commands to run the application.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

# Use the official Node.js LTS image as the base image
FROM node:18-alpine

# Set the working directory inside the container
WORKDIR /app

# Copy package.json and package-lock.json files to the container
COPY package*.json ./

# Install dependencies
RUN npm install --production

# Copy the rest of the application files
COPY . .

# Expose the port the app runs on
EXPOSE 3000

# Define the command to run the application
CMD ["npm", "start"]

About CMD ["npm", "start"]: Specifies the command to start the Node.js application. Make sure your package.json has a start script defined (e.g., “start”: “node index.js”).

Private Docker Registry

Normally, we push our Docker images to pricate docker registry rather than a public one. A commonly used private Docker registry is AWS ERC (Elastic Container Registry).

Docker Volumes

Docker Volumes is used to persist data in Docker and prevent data loss in stateful applications. They allow folders from the host’ physical file system (e.g. /home/mount/data) to be mounted into the virtual file system (e.g. /var/lib/mysql/data) of Docker.

There are three types of Docker volumes:

• Docker-managed volumes: Managed entirely by Docker and stored in its default location.
• Anonymous volumes: Automatically created without a specific name, typically for temporary use.
• Named volumes: Explicitly named and can be referenced by their names. Named volumes are the most common choice in production environments due to their reusability and ease of management.

Summary

The above covers the core concepts of Docker, providing a quick start for beginners. You can explore each part in greater details based on your interests.

Comment and share

Let us start with requirements

Business Requirements

The system needs to accept subcriptions from users with certain period time. After the paid subscription expires, the subscription status in user profile should be downgraded from a subscriped user to a normal user.

Make A Choice among Two Options

MongoDB TTL Index vs Node Schedule

As introduced in one of my previous writing regarding the System Architect , the system uses MongoDB to store user data. To add the function into the system, there were two choices that immediately came to my mind: MongoDB TTL Indexes and Node Schedule

• Node Schedule
  Node schedule runs on the server side, for example, node-schedule. But since the requirement is to change user’s subscription status after certain amount of time (relatively long, like a week, a month etc.), and it will only run once, putting such burden on the server side for such task is not worth it.

• MongoDB TTL Indexes
  Compare to Node Schedule, MongoDB TTL Indexes run on the database side, it will not create extra jobs on the server, it is a better choice for such requirement.

MongoDB TTL Indexes

Directly create MongoDB TTL Indexes

Clearly, in my case, I need to create MongoDB TTL Indexes with give parameters after the schema is generated. And after searching for a while, I realised that in mongoose, I can only create TTL indexes when create the schema, it is not possible to create TTL indexes with give parameters when the app is running (please correct me if I am wrong here). Within mongoose, it only can introduce TTL Index within schema definition, as shown below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import mongoose from 'mongoose';
const { Schema } = mongoose;

const XXXSchema = new Schema(
  {
    username: {
      type: String,
      required: true,
      unique: true,
    },
  },
  // timestamp of the creation data of the user
  { timestamps: true }
);

XXXSchema.index(
  { createdAt: 1 },
  {
    expireAfterSeconds: 30,
    partialFilterExpression: { username: { $eq: 'xxx' } },
  }
);

Indirectly create MongoDB TTL Indexes

Luckily, there is an alternative method to save my life: using expireAt in the schema. Under the table, the expireAt generates TTL indexes in MongoDB. Below are how I did it:

1. introduce expireAt in the schema with expires property

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

import mongoose from 'mongoose'
const { Schema } = mongoose

const XXXSchema = new Schema(
  {
    username: {
      type: String,
      required: true,
      unique: true,
    },
    expireAt: {
      type: Date,
      default: Date.now,
      expires: 0,
    },
  },
  // timestamp of the creation data of the user
  { timestamps: true },
)
...

2. when receiving the parameters, create the data with expireAt is set.

1
2
3
4
5
6
7
8

...
const newXXX = new XXX({
  username: user.username,
  expireAt: Date.now() + duration * 1000,
});

await newXXX.save();
...

3. After the code in step 2 is ran, in the MongoDB, a TTL index should be seem. Let us take MongoDB Atlas as an example:
   

Job done :)

Comment and share