Tech Recipe Book
My Services
  • Book
    • About the author
    • Architect
      • Algorithms
        • DB index algorithms
          • How does database indexing work
        • Neural network optimization
          • Neural Network Optimization
        • Route search
          • Road network in a database to build a route
          • Traveling Salesman Problem (TSP)
      • Architecture Frameworks
        • DODAF
        • TOGAF
        • Enterprise Architecture (EA) Tools Reviews 2023 | Gartner
      • Zero Trust
      • Billing
        • SHM billing system
      • Bots
        • Discord
        • Telegram
          • Chat GPT Telegram bot
          • Получаем статистику Telegram-канала при помощи api и python или свой tgstat с регистрацией и смс
          • Как хостить телеграм-бота (и другие скрипты на Python) на Repl.it бесплатно 24/7
          • Создание Telegram бота на PHP #1: основные понятия для работы с API
          • Создание Telegram бота на PHP #2: создание первого бота для Telegram
          • Создание Telegram бота на PHP #3: примеры отправки сообщений с кнопками в Telegram
          • Создание Telegram бота на PHP #4: отправка файлов и изображений в Telegram
          • Создание Telegram бота на PHP #5: работа с хуками
      • Business intelligence
      • Cloud Storage
        • Ceph
        • Virtual Distributed File System
      • Cryptography
        • Open Source PKI Software
        • OpenPGP
          • Email Encryption
          • Kleopatra
          • Miscellaneous Tools
          • Server side applications
      • Message broker
        • Kafka
          • Kafka UI-tools
          • Kafka streams ksqlDb
        • RabbitMQ
      • DB
        • MySQL
          • Auto sharding
          • MariaDB Zabbix monitoring
          • MySQL and MariaDB replication with Zabbix monitoring
        • Postgres
          • HA PostgreSQL with Patroni, Haproxy, Keepalived
          • Mass parallel requests - Greenplum
          • PostgreSQL cluster for development and testing
        • Vitess - Scalable. Reliable. MySQL-compatible. Cloud-native. Database.
      • Identity and Access Management (IDM)
        • FreeIPA - Identity, Policy, Audit
        • FreeIPA as an Enterprise solution
        • Keycloak
          • Keycloak HA cluster
        • Open Identity Platform
        • SSO
          • Keycloak for Java app
          • OpenAM
          • OpenIG
      • Firewall
        • nftables
      • Infrastructure As a Code
        • Ansible
        • IaC Packer Ansible Teraform
        • Installing Jenkins using terraform in Kubernetes in Yandex Cloud with letsencypt
        • Teraform Crosplan Pulumi
        • Yandex IaC solutions
      • Kubernetes
        • Installation
          • Install Kubernetes cluster
          • Deploying a Kubespray cluster to OpenStack using Terraform
          • Kube deploy in Yandex cloud
        • Frameworks
          • Deckhouse
            • LDAP authentification
            • On premise Install
            • Yandex Cloud Install
          • K3S
          • OpenShift OKD
          • RKE2
          • Rancher
            • Rancher Install
        • Auth
          • Keycloak in k8s
          • LDAP
        • GUI management Lens
        • Monitoring
          • Monitoring with Falco
          • Network monitoring
          • Nginx ingress
          • Prometheus Graphana for sample Nodejs app
          • Rsource monitoring Avito
        • Exposing services
          • Exposing Kubernetes Services
          • Cilium BGP
        • CNCF
        • Helm
          • Repositories
            • Artifact Hub | official
            • Bitnami | vmware
          • Awesome helm charts and resources
          • Essential Services for Modern Organizations
          • Security and Compliance
          • Additional charts
        • Isolation
          • vcluster - Virtual Kubernetes Clusters
          • Kiosk
          • KubeArmor
          • Control Plane Hardening
          • Hierarchical namespaces
        • Security Center
          • Minesweeper
          • NeuVector by SUSE
          • SOAR in Kubernetes
          • Security Сenter for Kubernetes
        • Terraform CI security
          • Terraform plan analysis with Checkov and Bridgecrew
          • Yandex Terraform scan
        • Vulnerability management
          • Aqua
          • Sysdig
          • Kyverno
          • GitLab
          • NeuVector by SUSE
        • Image scanning
          • Snyk
          • Sysdig
          • Harbor
          • Trivy
        • Signature verification
          • Sigstore
        • Control plane security
          • Gatekeeper
            • Applying OPA Gatekeeper
          • Kyverno
            • Policy as a code. Kyverno
        • Runtime Security
          • Osquery
          • Falco
          • ClamAV
        • Network security
          • Cilium
          • Control Plane Hardening (API restriction)
          • Network policy recipes
          • Service mesh
            • Istio HA, LoadBalance, Rate limit
          • mTLS Autocert
        • Honeypot
          • Building honeypot using vcluster and Falco
        • Backup
          • Kasten K10
        • Secrets
          • Vault CSI Driver
      • Load Balance
        • Nginx
        • HAProxy
          • Proxy methods
          • HAProxy for RDP
          • Payment gateway A/B test with HAProxy
          • HAPRoxy for Percona or Galera
      • Monitoring
        • Zabbix
          • Apache Zabbix
          • Disc Quota
          • Nginx Zabbix
          • SSL certificates Zabix
          • Zabbix notifications
        • Nagios
          • Datacenter monitoring
        • Prometheus and Grafana
      • Windows
        • Sysmon enhanced Windows audit
        • Sysmon to Block Unwanted File
      • Linux
        • Rsync
        • Debian based
          • Apt-Cacher NG
          • Unattended Upgrades in Debian / Ubuntu
        • RedHat basede
          • RPM Server
        • Logs analysis
        • Build armhf qemu
      • NGFW
      • CI/CD
        • DevSecOps
          • DAST
            • Burp
              • Dastardly
            • StackHawk
            • ZAP and GitHub Actions
          • SAST
            • Checkmarx
            • OSV by Google
            • Snyk
            • SonarQube
        • GitLab Runner in Yandex Cloud
        • Dynamic Gitlab Runners in Yandex Cloud
        • GitLab runner in Kubernetes with Werf
        • Kubernetes deploy strategies
        • Kubernetes highload deploy. part 1
        • Kubernetes highload deploy. part 2
        • Kubernetes Argo Rollouts
        • Jenkins in Kubernetes
        • Ansible Semaphore
        • Image storage, scaning and signing
        • Install WireGuard with Gitlab and Terraform
        • CI/CD example fror small web app
        • Threat matrix for CI CD Pipeline
      • SIEM / SOC
        • Datadog
        • Splunk
          • Splunk — general description
        • MaxPatrol
          • MaxPatrol 8 and RedCheck Enterprise
        • QRadar IBM
        • Cloud Native Security Platform (CNAPP) - Aqua
        • OSSIM | AT&T
          • AlienVault (OSSIM) install
        • Wazuh
        • EDR
          • Cortex XDR | Palo Alto Networks
          • Cynet
          • FortiEDR | Fortinet
          • Elastic
        • Elastic
          • Install Elasticsearch, Logstash, and Kibana (Elastic Stack) on Ubuntu 22.04
          • Setting Up Elastic 8 with Kibana, Fleet, Endpoint Security, and Windows Log Collection
        • Threat Intelligence
          • MISP
          • msticpy Microsoft
          • X-Force | IBM
          • Elastic
      • VPN
        • Full-Mesh VPN fastd, tinc, VpnCloud
        • Wireguard
          • WireGuard for Internet access
          • WireGuard on MikroTik and Keenetic
          • WireGuard site to site
        • SoftEther VPN Project
        • Cisco AnyConnect client
        • OpenConnect
        • SSTP python server
      • OS hardening
        • CIS Benchmarks
      • Cloud Providers
      • OpenNebula
        • OpenNebula Edge Cloud - Open Source Cloud & Edge Computing
        • Discover OpenNebula – Open Source Cloud & Edge Computing Platform
        • OpenNebula Multi-Cloud
        • Kubernetes on OpenNebula
        • The Open Source Alternative to Nutanix
        • The Simple Alternative to OpenStack
        • OpenNebula Partner Ecosystem
      • OpenStack
        • Install manual
        • Install with DevStack
      • VM
        • Create a VHD file from a Linux disk
        • Backup / Migration
          • Coriolis
          • Proxmox Backup Server
        • oVirt
        • VMware vCenter
        • Proxmox
      • Docker
        • Container optimization
        • Ubuntu RDP container
      • LXC
        • LXD on Ubuntu 18.04
        • Install, Create and Manage LXC in Ubuntu/Debian
    • Big Data
      • OLAP data qubes
      • Storage and autoscale in Lerua
    • Machine Learning
      • Yandex YaLM 100B. GPT model
      • Kaggle Community Datasts Models
      • AI in video production
      • Image search
      • Chat bots
        • You.com
        • Chat GPT
          • Implementing GPT in NumPy
        • Jailbreak Chat
      • Coding plugins CodeWhisperer
    • Malware
      • Isiaon/Pitraix: Modern Cross-Platform Peer-to-Peer Botnet over TOR
      • theZoo A repository of LIVE malwares
    • Pentest
      • Red Team
        • MITRE ATT&CK matrix
        • C2 Frameworks
          • Brute Ratel C4
          • Cobalt Strike
          • Covenant
          • Havoc Framework
          • Merlin
          • Metasploit
          • Sillenttrinity
          • Sliver
        • Manage and report
          • Dradis Framework
          • Hexway
        • Underground
      • Social engineering
        • Social Engineer Toolkit setoolkit
      • OSINT
        • OSINT for comapny
        • Instagram fishing
      • Forensics
        • Forensics tools
      • Pentesting Methodology
      • Web
      • CI/CD Methodology
      • Cloud Methodology
        • Hacking The Cloud
      • Kubernetes Pentesting
      • Android
        • SSL Unpinning for Android applications
      • iOS
        • SSL unpinning iOS and macOS applications
      • HackBar tool
      • CyberChef Tools
      • Python virtualenv
      • IppSec - YouTube
      • Hacktricks.xyz
    • Compliance
      • 152 ФЗ. Personal data
      • PCI DSS and ГОСТ Р 57580.1-2017
      • Cloud compliance
      • ГОСТ Р 57580.1-2017 для Kubernetes
      • Kubernets as DevSecOps and NIST compliance
      • NIST SP 800-61 cyberincidece control
      • CIS Kubernetes Benchmark v1.6 - RKE2 v1.20
      • CIS Kubernetes Benchmark v1.23 - RKE2
      • Requirements for Russian Banks
      • Tools
        • Chef InSpec
        • Elastic SIEM
    • Asset management
      • CMDBuild
    • Project management
    • Incident management SRE
    • Risk management
      • IT risk management
      • BSI-Standard 200-3
    • Web Dev
      • Cookie security
      • OWASP Top 10 2021
      • Docker nginx php mysql
      • Docker tor hiddenservice nginx
      • Docker Compose wp nginx php mariadb
      • Dependency Checking
        • Nexus Analyzer
        • OWASP dependency-check
      • Yii skeeks cms
      • YiiStudio
    • Art
      • GTK Themes
      • Themes for Xfce Desktop
      • XFCE / Xubuntu Windows 95
      • Moscow events
      • Photo goods
      • Russian style gifts
    • Cryptocurrency
      • News
      • Arbitrage
      • Stocks
      • Exchange aggregators
      • Where to use
      • Prepaid cards
        • BitFree
        • Pyypl Your Money at Your Fingertips
    • IT magazines
      • WIKI and Writeups tools
        • BookStack
        • GitBook
        • MkDocs
        • Wiki.js
        • DokuWiki
    • Languages
    • Learning
      • (ISC)2
        • CISSP
      • Offensive Security
        • OSCP
        • OSEP
        • OSED
      • DevSecOps
        • Certified DevSecOps Professional (CDP)
        • Certified DevSecOps Expert (CDE)
      • Web Security Academy: PortSwigger
    • Relocation
      • London experience
      • IT visas in 2022
      • Remote work
      • Running business in UAE
    • Freenet
      • Independent online services: the philosophy of a free Internet
      • Tor Project Anonymity Online
      • I2P Anonymous Network
    • Services
      • SMS Registration
        • Registering ChatGPT in Russia
      • Local and regional eSIMs for travellers - Airalo
      • Digital busines cards
      • No KYC services and exchanges
Powered by GitBook
On this page
  • Kyverno
  • Kubernetes Native Policy Management
  • Kyverno is a policy engine designed for Kubernetes
  • About Kyverno
  • How Kyverno works
  • Quick Start Guides

Was this helpful?

  1. Book
  2. Architect
  3. Kubernetes
  4. Control plane security

Kyverno

Last updated 1 year ago

Was this helpful?

Kyverno

Kubernetes Native Policy Management

Kyverno is a policy engine designed for Kubernetes

Policies are managed as Kubernetes resources and no new language is required to write policies.

This allows using familiar tools such as kubectl, git, and kustomize to manage policies.

Kyverno policies can validate, mutate, generate, and cleanup Kubernetes resources, and verify image signatures and artifacts to help secure the software supply chain.

The Kyverno CLI can be used to test policies and validate resources as part of a CI/CD pipeline.

About Kyverno

Kyverno (Greek for “govern”) is a policy engine designed specifically for Kubernetes. Some of its many features include:

  • policies as Kubernetes resources (no new language to learn!)

  • validate, mutate, generate, or cleanup (remove) any resource

  • verify container images for software supply chain security

  • inspect image metadata

  • match resources using label selectors and wildcards

  • validate and mutate using overlays (like Kustomize!)

  • synchronize configurations across Namespaces

  • block non-conformant resources using admission controls, or report policy violations

  • self-service reports (no proprietary audit log!)

  • self-service policy exceptions

  • test policies and validate resources using the Kyverno CLI, in your CI/CD pipeline, before applying to your cluster

  • manage policies as code using familiar tools like git and kustomize

Kyverno allows cluster administrators to manage environment specific configurations independently of workload configurations and enforce configuration best practices for their clusters. Kyverno can be used to scan existing workloads for best practices, or can be used to enforce best practices by blocking or mutating API requests.

How Kyverno works

Kyverno policies can match resources using the resource kind, name, label selectors, and much more.

Policy enforcement is captured using Kubernetes events. For requests that are either allowed or existed prior to introduction of a Kyverno policy, Kyverno creates Policy Reports in the cluster which contain a running list of resources matched by a policy, their status, and more.

The diagram below shows the high-level logical architecture of Kyverno.

The Webhook is the server which handles incoming AdmissionReview requests from the Kubernetes API server and sends them to the Engine for processing. It is dynamically configured by the Webhook Controller which watches the installed policies and modifies the webhooks to request only the resources matched by those policies. The Cert Renewer is responsible for watching and renewing the certificates, stored as Kubernetes Secrets, needed by the webhook. The Background Controller handles all generate and mutate-existing policies by reconciling UpdateRequests, an intermediary resource. And the Report Controllers handle creation and reconciliation of Policy Reports from their intermediary resources, Admission Reports and Background Scan Reports.

Quick Start Guides

This section is intended to provide you with some quick guides on how to get Kyverno up and running and demonstrate a few of Kyverno’s seminal features. There are quick start guides which focus on validation, mutation, as well as generation allowing you to select the one (or all) which is most relevant to your use case.

First, install Kyverno from the latest release manifest.

kubectl create -f https://github.com/kyverno/kyverno/releases/download/v1.10.0/install.yaml

BASH

Next, select the quick start guide in which you are interested. Alternatively, start at the top and work your way down.

Validation

Add the policy below to your cluster. It contains a single validation rule that requires that all Pods have the team label. Kyverno supports different rule types to validate, mutate, generate, cleanup, and verify image configurations. The field validationFailureAction is set to Enforce to block Pods that are non-compliant. Using the default value Audit will report violations but not block requests.

`kubectl create -f- << EOF apiVersion: kyverno.io/v1 kind: ClusterPolicy metadata: name: require-labels spec: validationFailureAction: Enforce rules:

  • name: check-team match: any:

    • resources: kinds:

      • Pod validate: message: "label 'team' is required" pattern: metadata: labels: team: "?*" EOF`

YAML

Try creating a Deployment without the required label.

kubectl create deployment nginx --image=nginx

BASH

You should see an error.

`error: failed to create deployment: admission webhook "validate.kyverno.svc-fail" denied the request:

resource Deployment/default/nginx was blocked due to the following policies:

require-labels: autogen-check-team: 'validation error: label ''team'' is required. Rule autogen-check-team failed at path /spec/template/metadata/labels/team/'`

BASH

In addition to the error returned, Kyverno also produces an Event in the same Namespace which contains this information.

**NoteKyverno may be configured to exclude system Namespaces like kube-system and kyverno. Make sure you create the Deployment in a user-defined Namespace or the default Namespace (for testing only).

Now, create a Pod with the required label.

kubectl run nginx --image nginx --labels team=backend

BASH

This Pod configuration is compliant with the policy and is allowed.

Now that the Pod exists, wait just a few seconds longer and see what other action Kyverno took. Run the following command to retrieve the Policy Report that Kyverno just created.

kubectl get policyreport

BASH

Notice that there is a single Policy Report with just one result listed under the “PASS” column. This result is due to the Pod we just created having passed the policy.

NAME PASS FAIL WARN ERROR SKIP AGE cpol-require-labels 1 0 0 0 0 2m46s

BASH

If you were to describe the above policy report you would see more information about the policy and resource.

**Results**: **Message**: validation rule 'check-team' passed. **Policy**: require-labels **Resources**: **API Version**: v1 **Kind**: Pod **Name**: nginx **Namespace**: default **UID**: 07d04dc0-fbb4-479a-b049-a3d63342b354 **Result**: pass **Rule**: check-team **Scored**: **trueSource**: kyverno **Timestamp**: **Nanos**: 0 **Seconds**: 1683759146

YAML

Policy reports are helpful in that they are both user- and tool-friendly, based upon an open standard, and separated from the policies which produced them. This separation has the benefit of report access being easy to grant and manage for other users who may not need or have access to Kyverno policies.

Now that you’ve experienced validate policies and seen a bit about policy reports, clean up by deleting the policy you created above.

kubectl delete clusterpolicy require-labels

BASH

Mutation

Mutation is the ability to change or “mutate” a resource in some way prior to it being admitted into the cluster. A mutate rule is similar to a validate rule in that it selects some type of resource (like Pods or ConfigMaps) and defines what the desired state should look like.

Add this Kyverno mutate policy to your cluster. This policy will add the label team to any new Pod and give it the value of bravo but only if a Pod does not already have this label assigned. Kyverno has the ability to perform basic “if-then” logical decisions in a very easy way making policies trivial to write and read. The +(team) notation uses a Kyverno anchor to define the behavior Kyverno should take if the label key is not found.

`kubectl create -f- << EOF apiVersion: kyverno.io/v1 kind: ClusterPolicy metadata: name: add-labels spec: rules:

  • name: add-team match: any:

    • resources: kinds:

      • Pod mutate: patchStrategicMerge: metadata: labels: +(team): bravo EOF`

YAML

Let’s now create a new Pod which does not have the desired label defined.

kubectl run redis --image redis

BASH

**NoteKyverno may be configured to exclude system Namespaces like kube-system and kyverno. Make sure you create the Pod in a user-defined Namespace or the default Namespace (for testing only).

Once the Pod has been created, get the Pod to see if the team label was added.

kubectl get pod redis --show-labels

BASH

You should see that the label team=bravo has been added by Kyverno.

Try one more Pod, this time one which does already define the team label.

kubectl run newredis --image redis -l team=alpha

BASH

Get this Pod back and check once again for labels.

kubectl get pod myredis --show-labels

BASH

This time, you should see Kyverno did not add the team label with the value defined in the policy since one was already found on the Pod.

Now that you’ve experienced mutate policies and seen how logic can be written easily, clean up by deleting the policy you created above.

kubectl delete clusterpolicy add-labels

BASH

Generation

Kyverno has the ability to generate (i.e., create) a new Kubernetes resource based upon a definition stored in a policy. Like both validate and mutate rules, Kyverno generate rules use similar concepts and structures to express policy. The generation ability is both powerful and flexible with one of its most useful aspects being, in addition to the initial generation, it has the ability to continually synchronize the resources it has generated. Generate rules can be a powerful automation tool and can solve many common challenges faced by Kubernetes operators. Let’s look at one such use case in this guide.

We will use a Kyverno generate policy to generate an image pull secret in a new Namespace.

First, create this Kubernetes Secret in your cluster which will simulate a real image pull secret.

kubectl -n default create secret docker-registry regcred \ --docker-server=myinternalreg.corp.com \ --docker-username=john.doe \ --docker-password=Passw0rd123! \ --docker-email=john.doe@corp.com

BASH

Next, create the following Kyverno policy. The sync-secrets policy will match on any newly-created Namespace and will clone the Secret we just created earlier into that new Namespace.

`kubectl create -f- << EOF apiVersion: kyverno.io/v1 kind: ClusterPolicy metadata: name: sync-secrets spec: rules:

  • name: sync-image-pull-secret match: any:

    • resources: kinds:

      • Namespace generate: apiVersion: v1 kind: Secret name: regcred namespace: "{{request.object.metadata.name}}" synchronize: trueclone: namespace: default name: regcred EOF`

YAML

Create a new Namespace to test the policy.

kubectl create ns mytestns

BASH

Get the Secrets in this new Namespace and see if regcred is present.

kubectl -n mytestns get secret

BASH

You should see that Kyverno has generated the regcred Secret using the source Secret from the default Namespace as the template. If you wish, you may also modify the source Secret and watch as Kyverno synchronizes those changes down to wherever it has generated it.

With a basic understanding of generate policies, clean up by deleting the policy you created above.

kubectl delete clusterpolicy sync-secrets

BASH

Kyverno runs as a in a Kubernetes cluster. Kyverno receives validating and mutating admission webhook HTTP callbacks from the Kubernetes API server and applies matching policies to return results that enforce admission policies or reject requests.

Mutating policies can be written as overlays (similar to ) or as a . Validating policies also use an overlay style syntax, with support for pattern matching and conditional (if-then-else) processing.

Kyverno also supports high availability. A highly-available installation of Kyverno is one in which the controllers selected for installation are configured to run with multiple replicas. Depending on the controller, the additional replicas may also serve the purpose of increasing the scalability of Kyverno. See the for more details on the various Kyverno controllers, their components, and how availability is handled in each one.

These guides are intended for proof-of-concept or lab demonstrations only and not recommended as a guide for production. Please see the for more complete information on how to install Kyverno in production.

In the validation guide, you will see how simple an example Kyverno policy can be which ensures a label called team is present on every Pod. Validation is the most common use case for policy and functions as a “yes” or “no” decision making process. Resources which are compliant with the policy are allowed to pass (“yes, this is allowed”) and those which are not compliant may not be allowed to pass (“no, this is not allowed”). An additional effect of these validate policies is to produce Policy Reports. A is a custom Kubernetes resource, produced and managed by Kyverno, which shows the results of policy decisions upon allowed resources in a user-friendly way.

Note that how although the policy matches on Pods, Kyverno blocked the Deployment you just created. This is because Kyverno intelligently applies policies written exclusively for Pods, using its feature, to all standard Kubernetes Pod controllers including the Deployment above.

Congratulations, you’ve just implemented a validation policy in your Kubernetes cluster! For more details on validation policies, see the .

Congratulations, you’ve just implemented a mutation policy in your Kubernetes cluster! For more details on mutate policies, see the .

Congratulations, you’ve just implemented a generation policy in your Kubernetes cluster! For more details on generate policies, see the .

https://kyverno.io/
Learn More
Get Started
Documentation
Sample Policies
dynamic admission controller
Kustomize
RFC 6902 JSON Patch
high availability page
installation page
Policy Report
rule auto-generation
validate section
mutate section
generate section
Policy as a code. Kyverno