Contact us
Posts by tags
  • Popular
  • Kubernetes 74
  • tools 26
  • databases 25
  • migrations 13
  • observability 12
  • A-Z
  • AIOps 1
  • ARM 1
  • AWS 3
  • benchmarking 2
  • best practices 7
  • business 4
  • caching 3
  • Calico 1
  • Cassandra 2
  • Ceph 5
  • cert-manager 1
  • CI/CD 9
  • CLI 4
  • ClickHouse 3
  • CNI 2
  • CockroachDB 1
  • comparison 9
  • databases 25
  • eBPF 2
  • Elasticsearch 5
  • etcd 4
  • failures 11
  • FinOps 1
  • Fluentd 1
  • GitLab 4
  • Helm 5
  • hyperconvergence 1
  • Ingress 3
  • Kafka 2
  • Keycloak 1
  • KeyDB 3
  • Kubernetes 74
  • Kubernetes operators 11
  • Linux 4
  • logging 5
  • Logstash 1
  • market 5
  • memcached 1
  • migrations 13
  • MongoDB 2
  • MySQL 2
  • networking 7
  • nginx 1
  • observability 12
  • Palark 7
  • Patroni 2
  • PHP 1
  • PostgreSQL 11
  • Prometheus 4
  • Python 4
  • RabbitMQ 1
  • Redis 4
  • Rook 3
  • security 7
  • serverless 2
  • software development 2
  • SSL 1
  • storage 10
  • success stories 2
  • Terraform 3
  • tools 26
  • troubleshooting 9
  • Vault 1
  • Vector 2
  • virtualization 1
  • VPN 1
  • werf 3
  • YAML 2
  • ZooKeeper 1

Exploring Cloud Native projects in CNCF Sandbox. Part 3: 14 arrivals of 2024 H1

10 March 2025
By Dmitry Shurupov, co-founder

Last year, we reviewed 25 new CNCF Sandbox projects accepted in 2023. We’re continuing this series with our brief introductions to the projects added to the Sandbox in April, June, and July of 2024. Following our tradition, we’ll list them by their formal categories, starting with those featuring more new projects.

Application Definition & Image Build

1. Radius

This project aims to foster collaboration between developers and operators handling Kubernetes workloads by empowering them with a user-friendly IaC (Infrastructure as Code) solution.

To use it, you’ll need to define so-called Radius resources, where you specify what your applications and their environments are and what they need to be run: specific workloads in the specified Kubernetes cluster, specific interconnections between them, port forwards, etc. Then you can deploy resources via Radius recipes to the targets of two types:

  • Azure by using Bicep to define your resources;
  • AWS, Azure, and Kubernetes by using relevant Terraform providers. (GCP support is mentioned as being in the project’s backlog.)

Importantly, Radius provides a Web UI where you can see your resources, applications, and everything related. (By the way, this dashboard is built on Backstage!)

Radius can even create an application graph to help you better understand how your app’s services and infrastructure are related.

Radius integrates nicely with another CNCF project that originated in Microsoft: Dapr. You can easily use your Dapr building blocks as resources in Radius and enjoy all its benefits. The project’s documentation has tutorials on getting started not only with the existing Dapr microservices application but also with a Helm chart. The docs even cater to those willing to start gradually adding Radius to their app.

Finally, Radius docs’ FAQ boasts a huge section comparing this project with many others (such as Bicep, Terraform, Crossplane, Backstage, KubeVela, etc.), which is a good read to better grasp the role of Radius in the Cloud Native ecosystem.

2. Stacker

Stacker is a tool to build OCI images that comes as one statically built binary and is configured in YAML. Its main benefits are:

  • Security: rootless builds, with no privileges on the host required;
  • Reproducibility: hermetically sealed builds based on LXC containers;
  • Efficiency: incremental builds, i.e. rebuilds are performed only if an input is changed.

You can easily integrate Stacker with GitHub Actions via its ready-to-use build-and-push action. The project’s website also suggests using Stacker with another CNCF Sandbox project that originated in Cisco: the Zot container image registry.

3. Score

Score is not a particular software but a specification. Focused on developers, it allows you to define your workload configuration that would work across various platforms (including Docker Compose and Kubernetes) and environments (local and remote).

With Score specification, you create a YAML file describing which services your app will use and leverage one of the Score implementations to generate ready-to-use manifests. Currently, it features two reference implementations:

  • score-compose for Docker Compose, which generates the docker-compose.yaml files;
  • score-k8s for Kubernetes, which generates manifests.yaml to deploy to your K8s cluster.

As the Score Roadmap document outlines, “To expose its full potential, additional implementations for popular tooling such as Kustomize, Amazon ECS, Google Cloud Run, or Nomad are needed.” However, they have not arrived yet.

What will be inside your specifications? A web server (nginx) with a service port might look like this:

apiVersion: score.dev/v1b1
metadata:
  name: workload-a
containers:
  example:
    image: nginx
    readinessProbe:
      exec:
        command: ["/usr/bin/curl", "-f", "-m", "5", "http://localhost"]
service:
  ports:
    web:
      port: 8080
      targetPort: 80

A MySQL server with its 3308 port published can be described as:

resources:
  db:
    type: mysql
    metadata:
      annotations:
        "compose.score.dev/publish-port": "3308"

Doing so allows you to use the same configuration for apps that would be run via Docker Compose for local development and later deployed to Kubernetes.

Security & Compliance

4. Bank-Vaults

Bank-Vaults provides various tools for secret management based on HashiCorp Vault. They include:

  • bank-vaults — a CLI tool simplifying Vault configuration. It allows you to automatically init and unseal Vault as well as continuously configure it via external YAML/JSON configuration. Unsealing for KMS is supported for AWS (S3-based), Azure Key Vault, Google Cloud (GCS-based), and Alibaba Cloud.
  • Vault Operator — a Kubernetes operator that builds on top of the abovementioned CLI functionality (e.g., automatic and continuous Vault configuration, automatic unsealing). It also features TLS support, capabilities for backup (when used with Velero) and monitoring (with Prometheus), etc.
  • Secrets Webhook — a Kubernetes mutating webhook that directly injects secrets into Pods.
  • Vault SDK — several Go packages for interacting with Vault. They add Kubernetes support to the official Vault client, allow you to generate self-signed TLS certificates, store JWT bearer tokens, and more.
  • Vault Secrets Reloader — a Kubernetes controller to reload workloads on secret changes.

Here’s a general overview of how Bank-Vaults operates and its main components:

Since the OpenBao project emerged recently (after HashiCorp changed the license for its Open Source products, including Vault, in 2023), the community has raised relevant questions regarding OpenBao support in Bank-Vaults. There are some efforts in this direction, yet OpenBao doesn’t seem to be a first-class citizen in this project yet.

5. TrestleGRC

Trestle is part of a bigger OSCAL Compass project, which might be interesting for those implementing compliance and working in DevOps. (OSCAL itself stands for Open Security Controls Assessment Language, which is NIST’s set of formats expressed in XML, JSON, and YAML with their Metaschema tooling.) The OSCAL Compass is made up of several projects that are focused on creating, validating, and governing documentation artifacts for compliance needs. The tools implemented by these projects offer an opinionated approach and leverage OSCAL as a standard data format.

Currently, there are 3 main components in the set:

  1. Trestle — a compliance-as-code platform. It operates as a CI/CD pipeline for compliance artifacts stored in Git. When generated, these artifacts are passed to various tools that render enforcement, measurement, and compliance reporting. Trestle allows editing and manipulation of OSCAL documents while ensuring the schemas are enforced, transforming documents from other formats to OSCAL, and governing author compliance content in markdown and draw.io.
  2. Agile Authoring — a GitOps automated workflow for Trestle using GitHub Actions. It supports the authoring of OSCAL artifacts such as catalogs, profiles, and component-definitions managed in Git. Its features include release management, semantic versioning, and access control.
  3. Compliance to Policy — a bridge for Compliance as Code and Policy as Code. It deploys compliance-as-code defined in OSCAL into policy validation or enforcement engines and then produces OSCAL assessment results. It supports Auditree as a generic compliance framework and Kyverno and OCM (Open Cluster Management) Policy Framework for Kubernetes resources. OPA/Gatekeeper for Kubernetes resources and Ansible are also mentioned in its roadmap.

In essence, Trestle “brings the maintenance of control descriptions into the DevOps domain. The goal is to have changes to the system (for example, updates to configuration management) easily related to the controls impacted, and to enable modification of those controls as required in concert with the system changes.”

The project features a demos repository where you can find demonstrations of how to enable version-controlled editing of System Security Plan responses, convert CIS controls from an Excel spreadsheet, use Trestle SDK, etc.

6. bpfman

As its name suggests, bpfman is an eBPF manager. It simplifies loading/unloading, modifying and monitoring eBPF programs on Linux hosts and Kubernetes clusters.

More specifically, bpfman aims to be a powerful solution for those interested in leveraging eBPF capabilities in Kubernetes, which comes with various challenges. The project has a Kubernetes operator to address these challenges and run eBPF programs with automated lifecycle management. Technically, it deploys to K8s a DaemonSet with bpfman, which is the project’s main system daemon, and bpfman-agent, which ensures the desired state for eBPF programs on a given node. The eBPF programs that you want to run on Kubernetes are represented as a set of CRDs.

For example, deploying the go-xdp-counter sample Go program (described in the XdpProgram CRD) on K8s via bpfman-operator and running it will result in the following workflow, which is greatly explained in the bpfman’s docs:

Following this approach, only the main daemon (bpfman) has the privileges required to load eBPF programs. You can control access to the Kubernetes API via regular RBAC. You can also benefit from having multiple eBPF programs from multiple users and running multiple XDP programs on a single interface (via the libxdp multiprog protocol).

Scheduling & Orchestration

7. Koordinator

Koordinator is a scheduling system created to efficiently run microservices, AI, and big data workloads on Kubernetes. It focuses on latency-sensitive workloads and batch jobs and boasts prominent production users, such as Alibaba and Xiaomi.

Technically, Koordinator consists of two control planes (scheduler and manager) and one DaemonSet (called Koordlet):

To improve resource utilization, Koordinator features a priority and QoS mechanism for co-locating different types of workloads and enhancing their performance. It combines numerous approaches, such as:

  • resource overcommitment for low-priority Pods, which is performed by reclaiming high-priority resources that have been requested and not used;
  • multi-hierarchy elastic resource quota management allowing busy quota groups to temporarily borrow the resources from the idle groups;
  • Pod packing, i.e. scheduling based on the bin-packing algorithm;
  • a QoS manager, which “eliminates the long-tail jitter of latency-sensitive workloads by continuously adjusting the resource isolation parameters of each Pod.”

The project documentation contains detailed examples of leveraging load-aware and fine-grained scheduling, co-location capabilities, and even a best practices section with helpful hints on running specific software (e.g., Apache Spark jobs and Apache Hadoop YARN) with Koordinator.

8. KubeSlice

KubeSlice is a vendor-neutral framework implementing overlay networking across Kubernetes clusters deployed in diverse infrastructures: bare metal, cloud, multi-cloud, hybrid cloud, or edge. In a nutshell, it delivers versatile features — such as application connectivity, network policies, RBAC, DNS entries, automatic service discovery, traffic prioritization, and microservices isolation — across clusters and clouds.

The project’s main components are KubeSlice Controller (the central management unit) and Slice Operator (this Kubernetes operator is installed on each worker cluster). Additional components, such as Slice VPN Gateway, Slice Router, Slice Gateway Edge, KubeSlice DNS, etc., are responsible for rendering networking functions and are all managed by the Operator.

KubeSlice integrates nicely with other CNCF projects:

  • Network Service Mesh (NSM) capabilities are used to set up the data plane and connect application Pods to the overlay network;
  • SPIRE, the reference implementation of the SPIFFE APIs, is used to establish the workload identities of the NSM control plane and data plane;
  • Optionally, you can integrate KubeSlice with Istio by setting up ingress and egress gateways using Istio resources.

You can also integrate KubeSlice with Slack for event monitoring and metric alerting.

Automation & Configuration

9. Atlantis

Atlantis is an application that automates Terraform pull requests. It listens to new events via webhooks, performs specific Terraform commands (plan, import, apply) remotely, and adds their output by commenting on relevant pull requests. Initially created in a company, it was soon moved to an independent repo and maintained by the community for many years.

You can integrate Atlantis with various Git repos: GitHub, GitLab, Gitea (including compatible forks, such as Forgejo), Bitbucket Cloud and Bitbucket Server, and Azure DevOps. As an application, Atlantis will receive webhooks from your Git host and execute Terraform commands locally. To make it possible, you must deploy the app, establish its communication with a Git host, and configure your webhooks. The app itself can be deployed as a simple binary or Docker container, or to Kubernetes. If you switched to OpenTofu from Terraform, there is a blog article covering such an integration.

Being a long-living and massively used project, Atlantis’s issue with the request to join CNCF Sandbox received an impressive 900+ upvotes from the community, which I believe is unprecedented.

10. Kubean

Kubean builds on top of kubespray and aims to streamline and automate your Day 2 operations for Kubernetes clusters running in production. It steps in when you have an existing cluster and is mainly implemented as a Kubernetes operator comprising several controllers that monitor various objects: cluster nodes, manifests, and so on.

Therefore, you define all your settings in CRDs that are processed by the Operator. It applies to:

  • clusters themselves: their nodes, IPs, etc.;
  • operations: deployment, upgrade, etc.;
  • versions of software deployed in your clusters (think of containerd, cni, etcd…);
  • offline software packages (called local artifacts) that you want to keep to be used whenever needed;
  • custom actions to be performed on the clusters (e.g., Ansible playbooks or shell scripts).

Offline packages are an essential feature of Kubean, and its documentation outlines other details on using Kubean to operate Kubernetes clusters in air-gapped environments. For example, this guide shows how to prepare and install a cluster without the Internet by leveraging Kubean’s ClusterOperation task.

Remote Procedure Call

11. Connect

Connect is a cross-language framework for Protobuf RPC, whose motto is “simple, reliable, interoperable.” Created as an answer to “a maximalist design ethos that has led to extraordinary complexity” of today’s gRPC libraries, Connect simplifies building browser and gRPC-compatible HTTP APIs.

To achieve this, the project delivers only the essential features in its libraries built on familiar primitives (e.g., net/http in Go) and ensures they are small and reliable. This idea was embraced by many in the industry, and today’s Connect adopters include big names, such as Dropbox, Bluesky, Chick-fil-A, and others.

The project currently has implementations for Go, TypeScript, Swift, and Kotlin, with Dart being its most recent addition.

Wasm / Edge

12. Kairos

Kairos is an immutable Linux distribution for Kubernetes clusters running on edge. Basically, it allows you to create bootable images for your edge devices with the Linux distribution of your choice and Kubernetes.

For Linux flavours, it offers ready-to-use images with Alpine, Debian, Fedora, openSUSE, Rockylinux, and Ubuntu. It’s easy to customise these images by adding the required packages, choosing specific versions of the Linux kernel, adding custom mounts, and so on. You can also turn any other Linux into an image with Kairos by leveraging the so-called Kairos Factory.

There are numerous ways to install Kairos: via an interactive installer (available as a CLI or Web UI), cloud-init, or even by leveraging Kubernetes CRDs, where you can also define your configurations. In the latter case, a Kubernetes controller will create a Pod to build the requested ISO and make it accessible for downloading.

Kairos supports Secure Boot. Moreover, following the SENA (Secure Edge Native Architecture) requirements, the project implements Trusted Boot, which combines Secure Boot, Measured Boot, and FDE (Full Disk Encryption). In terms of networking, Kairos creates self-coordinated, fully meshed clusters at the edge by combining P2P and VPN. The EdgeVPN project, based on libp2p, is used to achieve that. You can also use Kairos to run confidential workloads (to encrypt all the data, including the in-memory one), yet this feature is still considered experimental.

For Kubernetes, Kairos relies on K3s, a lightweight distribution that is also a CNCF project. The documentation comes with various examples of working with Kairos and K3s. Particularly, it covers creating ISOs for air-gapped installations, configuring high availability for K3s, bootstrapping self-coordinated clusters, and running LocalAI.

API Gateway

13. Kuadrant

Kuadrant provides a set of tools for implementing gateway policies in Kubernetes. It allows operators and developers to “connect, secure, protect and observe service endpoints.”

Kuadrant can work with Envoy Gateway or Istio as your Gateway API provider and can be installed on Kubernetes and OpenShift. The project’s components reside in both the control plane (policy controllers and CRDs for them) and the data plane (policy enforcement configuration and third-party components’ integration).

Several Kubernetes operators are in charge of Kuadrant functionality:

  • Kuadrant Operator that provides the core APIs and manages the lifecycle of all the project’s components;
  • Limitador Operator that enforces rate limits. It’s written in Rust and can be used with Envoy as a gRPC service that implements the Envoy Rate Limit protocol;
  • Authorino Operator that implements a lightweight Envoy external authorization. It supports JWT authentication, mTLS, pattern-matching authz, OPA, Kubernetes ServiceAccount tokens and RBAC, and more;
  • DNS Operator that integrates with cloud DNS providers. It supports AWS Route 53, Google Cloud DNS, and Azure;
  • cert-manager that manages TLS certificates for the components and the gateways, ensuring that traffic is secure.

There is also a kuadrantctl CLI tool that helps you to manage Kuadrant configurations and resources.

Database

14. openGemini

openGemini is a distributed TSDB (time series database). It is focused on delivering high-performance reads and writes, together with data analysis capabilities. You can find openGemini performance comparison with some other databases here.

Other project features include great scalability (thanks to the MPP [Massively Parallel Processing] architecture), data compression (the authors promise that storage would cost only 1/10 of that for NoSQL solutions), and built-in observability data (metrics, logs, and traces).

openGemini’s storage engine ensures high cardinality featuring support for the InfluxDB Line and Apache Arrow Flight protocols. This database is also compatible with Prometheus’s standard interfaces (remote_write/remote_read) to serve as remote storage for Prometheus. You can query data via InfluxQL or PromQL.

The two most common use cases for openGemini are:

  1. Observability. Combining it with other tools, such as the abovementioned Prometheus or OTel Collector, or Telegraf, opens new opportunities for real-time status monitoring, anomaly detection, statistics analysis, and more.
  2. Internet of Things. You can leverage it for energy consumption monitoring and analysis, environmental monitoring, fault predictions, and other similar tasks.

Afterword

Most of these projects were launched 2-3 years prior to joining CNCF, which reassures the trend we’ve seen in the Sandbox during the second half of 2023. However, we still see that software “veterans” are also coming: Stacker, Bank-Vaults, and Atlantis were born in 2017-2018.

This batch stands out, with the biggest number of new entries falling into the Application Definition & Image Build category and no additions to the Observability, which was the most popular before.

Finally, the most obvious and persistent insight is that the Go language and Apache 2.0 license are the default choices for Cloud Native projects. However, you can always find a few exceptions relying on Rust.

P.S. Other articles in this series

  • Part 1: 13 arrivals of 2023 H1: Inspektor Gadget, Headlamp, Kepler, SlimToolkit, SOPS, Clusternet, Eraser, PipeCD, Microcks, kpt, Xline, HwameiStor, and KubeClipper.
  • Part 2: 12 arrivals of 2023 H2: Logging operator, K8sGPT, kcp, KubeStellar, Copa, Kanister, KCL, Easegress, Kuasar, krkn, kube-burner, and Spiderpool.

Comments

Your email address will not be published. Required fields are marked *

Related articles

Contact us