Posts by tags
  • Popular
  • Kubernetes 72
  • tools 24
  • databases 24
  • migrations 13
  • observability 12
  • A-Z
  • AIOps 1
  • ARM 1
  • AWS 3
  • benchmarking 2
  • best practices 7
  • business 4
  • caching 3
  • Calico 1
  • Cassandra 2
  • Ceph 5
  • cert-manager 1
  • CI/CD 9
  • CLI 4
  • ClickHouse 3
  • CNI 2
  • CockroachDB 1
  • comparison 9
  • databases 24
  • eBPF 2
  • Elasticsearch 5
  • etcd 4
  • failures 11
  • FinOps 1
  • Fluentd 1
  • GitLab 4
  • Helm 5
  • hyperconvergence 1
  • Ingress 3
  • Kafka 2
  • Keycloak 1
  • KeyDB 3
  • Kubernetes 72
  • Kubernetes operators 11
  • Linux 4
  • logging 5
  • Logstash 1
  • market 5
  • memcached 1
  • migrations 13
  • MongoDB 2
  • MySQL 2
  • networking 7
  • nginx 1
  • observability 12
  • Palark 7
  • PHP 1
  • PostgreSQL 10
  • Prometheus 4
  • Python 4
  • RabbitMQ 1
  • Redis 4
  • Rook 3
  • security 7
  • serverless 2
  • software development 2
  • SSL 1
  • storage 10
  • success stories 2
  • Terraform 3
  • tools 24
  • troubleshooting 8
  • Vault 1
  • Vector 2
  • virtualization 1
  • VPN 1
  • werf 3
  • YAML 2
  • ZooKeeper 1

A brief introduction to werf, a CI/CD tool and now CNCF project

Meet werf, an Open Source project that has grown leaps and bounds in part through our continuous effort and engagement. werf’s mission is to make software delivery in Kubernetes more efficient by leveraging existing technologies and marrying them with best practices. Now that it has officially become a CNCF project, we’re pleased to share its story and the gist of what it offers to users.

The weather is cloudy… and exceptionally diverse

CI/CD leveraging containers and Kubernetes is what we all do today or, at least, where we focus our efforts. Tempted by one-button deployments and dynamic environments promptly popping up for each new feature we want to test, we imagine our development process being rendered more comfortable than ever before. At the same time, our business owners are excited by the promise of such great time-to-market acceleration to stay competitive.

However, when you step into these waters, you find yourself in a murk of great cloud-native projects that you will naturally want to examine and even try. You have probably seen this meme that recently circulated:

Well-known meme depicting the boundless landscape of cloud-native projects

By the way, did you know there’s a separate Continuous Delivery Landscape in addition to the most widely known Cloud Native one? The good news is that it’s much smaller! That said, there’s still a considerable amount to choose from:

Continuous Delivery Landscape from the Continuous Delivery Foundation as of February 2023

A huge ecosystem is both good and bad at the same time. The challenging part here is not so much about finding the best implementation of another essential function but more so about gluing different parts into a desired solution. You are literally creating your CI/CD pipeline out of those building blocks.

The software delivery lifecycle involves many steps you must complete for your code to be built, tested, released, and deployed. Thus you’re going to have to choose a tool for each of those steps and somehow glue them together, all of that added on top of your favourite CI system, of course.

This journey will undoubtedly be exciting and insightful! However, based on real-world experience, the likely outcome of that approach is that it will yield a solution that is not easy to maintain and comes with a number of loopholes that can lead to unexpected — i.e. unwanted — results.

Enter werf.

What is werf?

werf is an Open Source CLI tool that aspires to improve our CI/CD experience. This project originated in a professional services company providing DevOps as a Service for its customers. In the process of implementing CI/CD pipelines for various projects, the desire arose to automate and standardise the way it was done, as well as the tools that were used. Likely even more critical, however, was the need to minimise the risks of the abovementioned random consequences of those using this established CI/CD workflow who could — unintentionally ­— ruin everything in production.

That’s why the best practices are such a crucial aspect of the ideas behind werf. What are they all about? Here are the most notable examples of what’s under the hood:

  • Sophisticated caching system that ensures the optimal building time: owes to layer-by-layer caching, incremental rebuilds based on the Git history, and shared caches for layers and images;
  • Content-based tagging: improves this even further by preventing extra builds (and, consequently, redeployments of them to K8s) when they are unnecessary;
  • Kubernetes-native (in-cluster) container image building;
  • Giterminism: a term derived from joining “Git” and “determinism,” meaning the use of Git as a single source of truth — to guarantee builds’ reproducibility;
  • Advanced Kubernetes deployment process: bringing observability to the next level via the K8s resource tracking and fail-fast mechanism;
  • Original container registry cleanup mechanism: saves up space for you by detecting unused container images with regard to your development flow.

Leveraging these features and following the requirements they imply enforces compliance of some kind in your CI/CD: “This is how things are to be done for us so that we can guarantee what we’ll get in production, the way we will rollback in the case any issues arise, how much time & resources will end up being spent, etc.”

werf aims to cover the full software delivery process

What is inside werf?

To achieve this, werf relies heavily on well-known, existing cloud-native projects. Remember the building blocks? It’s mainly Buildah that’s used in building container images along with an extended version of Helm for deploying them. However, looking at the bigger picture, it’s essential to mention that werf also integrates with:

  • Git: using the Git history for rebuilds and relying on Git in Giterminism;
  • various container registries: storing and cleaning up images;
  • Argo CD: implementing the GitOps pattern if you need it;
  • Kubernetes;
  • various CI systems: executing your pipeline.

As for the last item on the list, while GitLab CI/CD and GitHub Actions are supported out of the box, instructions are available for others as well.

It’s also essential to note that werf inherits the configuration files of the tools it uses under the hood, such as Dockerfiles and Helm charts. It significantly improves the learning curve and ensures compatibility. Using three files, we can illustrate a simplistic werf project configuration:

1. werf.yaml which serves as the main configuration file binding the build and deploy instructions together:

configVersion: 1
project: hello
---
image: hello
dockerfile: ./Dockerfile

2. Dockerfile describing how the container image is built:

FROM node
WORKDIR /app
COPY . .
RUN npm ci
CMD ["node", "server.js"]

3. .helm/templates/deployment.yaml telling us how the application will be deployed in Kubernetes:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
spec:
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      labels:
        app: hello
    spec:
      containers:
      - image: {{ .Values.werf.image.hello }}

These listings should be self-explanatory if you have used Docker and Helm before. Followed by the werf converge command, this configuration will result in a container image being built, pushed to the container registry, and deployed to Kubernetes. All the magic described above — the caching system, content-based tagging, etc. — will happen behind the scenes.

If you’re interested in trying werf out in practice, its quickstart is a good starting point. If you think you’re ready to take a deep dive — these guides compiled for different frameworks can be your best friend. They should serve as a perfect complement even for developers with very limited prior Kubernetes experience.

To sum it up, werf preaches an opinionated approach, but it is this approach that assists you in the swim through the spacious waters of cloud-native tools.

Palark and werf

Palark is playing a vital role in the werf Open Source project and actively contributing to both its codebase and future plans.

We have had a positive experience relying on werf for our internal software projects as well as the production-grade CI/CD for our customers’ applications. Moreover, we see the comprehensive experience of leveraging werf as one of our added values. As it was mentioned above, werf is pretty much about accumulating and automating industry best practices. We can clearly see the benefits the tool has to offer its end users: greater efficiency and reliability in building and deploying containerised applications to Kubernetes, better Dev & Ops cooperation, and easier CI/CD maintenance.

Since werf has for years now been used to help address the production needs of various companies, we can be certain it’s sufficiently mature at this point, which cannot be overlooked when you are dealing with bleeding-edge technologies. Its well-thought-out release process with five stability channels varying from Alpha to Rock-solid adds even more points to its credibility.

Afterword

Following its seven years of existence as Open Source, werf became a CNCF Sandbox project. Now it has a vendor-neutral home and is in the good company of many other amazing Open Source gems. werf has already been integrated with some of them, and we can expect more collaboration to happen going forward. With that in mind, werf will continue chasing its dream of CI/CD with Kubernetes becoming ever more integral and efficient.

Read more information about werf on GitHub and the project’s blog. To stay updated about werf-related news, follow werf on Twitter and Telegram.

Comments

Your email address will not be published. Required fields are marked *