CI/CD 🚀
This articles aims to give you an idea and pointers on how to create a fast CI/CD environment for your enterprise software. Do you even need a CI/CD pipeline? How long should the process take? How do I know it’s fast? How do I keep the cost low? All this and more we will be answering in this article.
Do I need CI/CD ?
The answer is it depends. If you’re working on a small project for yourself than no but some CI can help you reduce errors. If you think the extra work for a CI/CD configuration is not worth it than also no. On the other hand if you’re working on a larger scale project and or you are working on enterprise software and need to ship your application to consumers than yes.
Why
- Small you most likely will only need to adjust one project, build, and deploy it sometimes. In this case doing that manually might be good enough. But we are engineers right? Over engineering is in our blood so my answer is I always do CI/CD because I’m lazy.
- Enterprise without CI/CD it’s the wildwest if you don’t test and check for standards that go into your applications it will get messy.
Requirements for CI/CD ?
CI/CD depends on a OS environment means we need a machine to execute scripts / instruction on. In this case we need a provider which lets us use a VM to lint,format,test,build and ship our application. Some of those providers might be GitHub Actions or CircleCi for instance. Let’s say we opted for one provider what else do we need? As mentioned earlier we to execute scripts and instructions meaning we need to write a configuration for our provider. Most configurations are written in YAML so we probably should get some idea on how the YAML syntax looks like. And for the scripts and Instructions we also need to know some aspects of linux and bash scripting. A lot of CI/CD providers now day’s have a vast amount of script collections maintained by providers for various task from building the app and shipping the app. This reduces your need to write complex setups yourself and can just reuse them. Everything I’ve spoken about so far is actually mostly the CI part of things next is the CD part. So how and where do you wanna deploy your app? On Premis? Google Cloud? Amazons AWS? Vercel? Firebase? Just to name a few if you don’t know them yet. Let’s go with Google Cloud or short GCP. You’ve chosen your cloud provider now you need to ask yourself how you wanna deploy? There are all kinds of ways to get your app up and running. We have Kubernetes to easily scale our applications if needed which you will need to consider if you have a lot of traffic. We have Cloud Run which can boot up a container for us, and lets say we have a web app in which case we want to connect our domain to the container which we can also supported with GCP Cloud Run. Now that we know what we need for CD we need to integrate the communication from the CI to our Cloud Provider so we can get our CD aspects running.
How do I know it is fast?
Generally speaking almost every app should be able to be build and be deployed in < 10min obviously this number varies on size of the project. The app container should be about 50-60MB. Most pipelines take a long time when it comes to the test step running unit tests, and integration tests can take a long time. If you end up in a situation where your test step takes a long time consider splitting up tests and execute them all at the same time on different machines..
Detecting Issues
Detecting issues and figuring out bottlenecks is all about what depends on what and than eliminating synchron tasks and parallelise them. Linting, Formatting, Type check, Build App can all run parallelised. It’s on you if you want to run multiple jobs or run a shell script to offload processes in the shell in one job.
Keeping Cost Low
This depends on your provider there might be cost involved for: each triggered pipeline, execution time of the triggered pipeline, machine/vm/docker specs CPU/MEM. Individual features such as shared workspace for multiple jobs, how many users can access it, how many concurrent jobs, and provider support etc.
Examples
Always make use of the various plugins available to your CI provider such as circleci’s ORBS
CircleCI
Showing you a PR workflow which does linting, format, type check, test this is a node project so in this case we use the node orb which allows us to install any node version we need without any need of do the scripting part ourself.
orbs:
node: circleci/node@5.0.2
version: 2.1
workflows:
version: 2
build-feature:
when:
not:
equal: [master, << pipeline.git.branch >>]
jobs:
- lint:
context: gen-env
- format:
context: gen-env
- type-check:
context: gen-env
- test:
context: gen-env
jobs:
lint:
resource_class: medium
working_directory: ~/project
docker:
- image: 'cimg/base:stable'
environment:
YARN_NPM_AUTH_TOKEN: ${GH_REGISTRY_TOKEN}
steps:
- checkout
- node/install:
node-version: 14.17.5
install-yarn: true
- node/install-packages:
cache-path: ~/project/node_modules
pkg-manager: yarn
- run:
name: Lint
command: "yarn lint"
GitHub Actions
This example shows how to build a docker container and push it with the use of the actions repository from GitHub which as you can see means I don’t have to do any scripting myself.
name: ci
on:
workflow_dispatch:
inputs:
nvim-version:
description: "use neovim (stable) or neovim-git (nightly)"
default: "neovim"
required: true
jobs:
docker:
runs-on: ubuntu-latest
name: docker
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
-
name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push stable
uses: docker/build-push-action@v2
if: ${{ github.event.inputs.nvim-version == 'neovim' }}
with:
context: .
push: true
build-args: |
version=${{ github.event.inputs.nvim-version }}
tags: danielnehrig/nvim:latest
-
name: Build and push nightly
uses: docker/build-push-action@v2
if: ${{ github.event.inputs.nvim-version == 'neovim-git' }}
with:
context: .
push: true
build-args: |
version=${{ github.event.inputs.nvim-version }}
tags: danielnehrig/nvim:nightly
Epilogue
Overall CI/CD helps to reduce workload and keep your codebase clean with automatized tasks. CI/CD usually will cost you money but it’s worth it. Github Actions is available for free for Open Source projects 🎉.
