Tech Recipe Book
My Services
  • Book
    • About the author
    • Architect
      • Algorithms
        • DB index algorithms
          • How does database indexing work
        • Neural network optimization
          • Neural Network Optimization
        • Route search
          • Road network in a database to build a route
          • Traveling Salesman Problem (TSP)
      • Architecture Frameworks
        • DODAF
        • TOGAF
        • Enterprise Architecture (EA) Tools Reviews 2023 | Gartner
      • Zero Trust
      • Billing
        • SHM billing system
      • Bots
        • Discord
        • Telegram
          • Chat GPT Telegram bot
          • Получаем статистику Telegram-канала при помощи api и python или свой tgstat с регистрацией и смс
          • Как хостить телеграм-бота (и другие скрипты на Python) на Repl.it бесплатно 24/7
          • Создание Telegram бота на PHP #1: основные понятия для работы с API
          • Создание Telegram бота на PHP #2: создание первого бота для Telegram
          • Создание Telegram бота на PHP #3: примеры отправки сообщений с кнопками в Telegram
          • Создание Telegram бота на PHP #4: отправка файлов и изображений в Telegram
          • Создание Telegram бота на PHP #5: работа с хуками
      • Business intelligence
      • Cloud Storage
        • Ceph
        • Virtual Distributed File System
      • Cryptography
        • Open Source PKI Software
        • OpenPGP
          • Email Encryption
          • Kleopatra
          • Miscellaneous Tools
          • Server side applications
      • Message broker
        • Kafka
          • Kafka UI-tools
          • Kafka streams ksqlDb
        • RabbitMQ
      • DB
        • MySQL
          • Auto sharding
          • MariaDB Zabbix monitoring
          • MySQL and MariaDB replication with Zabbix monitoring
        • Postgres
          • HA PostgreSQL with Patroni, Haproxy, Keepalived
          • Mass parallel requests - Greenplum
          • PostgreSQL cluster for development and testing
        • Vitess - Scalable. Reliable. MySQL-compatible. Cloud-native. Database.
      • Identity and Access Management (IDM)
        • FreeIPA - Identity, Policy, Audit
        • FreeIPA as an Enterprise solution
        • Keycloak
          • Keycloak HA cluster
        • Open Identity Platform
        • SSO
          • Keycloak for Java app
          • OpenAM
          • OpenIG
      • Firewall
        • nftables
      • Infrastructure As a Code
        • Ansible
        • IaC Packer Ansible Teraform
        • Installing Jenkins using terraform in Kubernetes in Yandex Cloud with letsencypt
        • Teraform Crosplan Pulumi
        • Yandex IaC solutions
      • Kubernetes
        • Installation
          • Install Kubernetes cluster
          • Deploying a Kubespray cluster to OpenStack using Terraform
          • Kube deploy in Yandex cloud
        • Frameworks
          • Deckhouse
            • LDAP authentification
            • On premise Install
            • Yandex Cloud Install
          • K3S
          • OpenShift OKD
          • RKE2
          • Rancher
            • Rancher Install
        • Auth
          • Keycloak in k8s
          • LDAP
        • GUI management Lens
        • Monitoring
          • Monitoring with Falco
          • Network monitoring
          • Nginx ingress
          • Prometheus Graphana for sample Nodejs app
          • Rsource monitoring Avito
        • Exposing services
          • Exposing Kubernetes Services
          • Cilium BGP
        • CNCF
        • Helm
          • Repositories
            • Artifact Hub | official
            • Bitnami | vmware
          • Awesome helm charts and resources
          • Essential Services for Modern Organizations
          • Security and Compliance
          • Additional charts
        • Isolation
          • vcluster - Virtual Kubernetes Clusters
          • Kiosk
          • KubeArmor
          • Control Plane Hardening
          • Hierarchical namespaces
        • Security Center
          • Minesweeper
          • NeuVector by SUSE
          • SOAR in Kubernetes
          • Security Сenter for Kubernetes
        • Terraform CI security
          • Terraform plan analysis with Checkov and Bridgecrew
          • Yandex Terraform scan
        • Vulnerability management
          • Aqua
          • Sysdig
          • Kyverno
          • GitLab
          • NeuVector by SUSE
        • Image scanning
          • Snyk
          • Sysdig
          • Harbor
          • Trivy
        • Signature verification
          • Sigstore
        • Control plane security
          • Gatekeeper
            • Applying OPA Gatekeeper
          • Kyverno
            • Policy as a code. Kyverno
        • Runtime Security
          • Osquery
          • Falco
          • ClamAV
        • Network security
          • Cilium
          • Control Plane Hardening (API restriction)
          • Network policy recipes
          • Service mesh
            • Istio HA, LoadBalance, Rate limit
          • mTLS Autocert
        • Honeypot
          • Building honeypot using vcluster and Falco
        • Backup
          • Kasten K10
        • Secrets
          • Vault CSI Driver
      • Load Balance
        • Nginx
        • HAProxy
          • Proxy methods
          • HAProxy for RDP
          • Payment gateway A/B test with HAProxy
          • HAPRoxy for Percona or Galera
      • Monitoring
        • Zabbix
          • Apache Zabbix
          • Disc Quota
          • Nginx Zabbix
          • SSL certificates Zabix
          • Zabbix notifications
        • Nagios
          • Datacenter monitoring
        • Prometheus and Grafana
      • Windows
        • Sysmon enhanced Windows audit
        • Sysmon to Block Unwanted File
      • Linux
        • Rsync
        • Debian based
          • Apt-Cacher NG
          • Unattended Upgrades in Debian / Ubuntu
        • RedHat basede
          • RPM Server
        • Logs analysis
        • Build armhf qemu
      • NGFW
      • CI/CD
        • DevSecOps
          • DAST
            • Burp
              • Dastardly
            • StackHawk
            • ZAP and GitHub Actions
          • SAST
            • Checkmarx
            • OSV by Google
            • Snyk
            • SonarQube
        • GitLab Runner in Yandex Cloud
        • Dynamic Gitlab Runners in Yandex Cloud
        • GitLab runner in Kubernetes with Werf
        • Kubernetes deploy strategies
        • Kubernetes highload deploy. part 1
        • Kubernetes highload deploy. part 2
        • Kubernetes Argo Rollouts
        • Jenkins in Kubernetes
        • Ansible Semaphore
        • Image storage, scaning and signing
        • Install WireGuard with Gitlab and Terraform
        • CI/CD example fror small web app
        • Threat matrix for CI CD Pipeline
      • SIEM / SOC
        • Datadog
        • Splunk
          • Splunk — general description
        • MaxPatrol
          • MaxPatrol 8 and RedCheck Enterprise
        • QRadar IBM
        • Cloud Native Security Platform (CNAPP) - Aqua
        • OSSIM | AT&T
          • AlienVault (OSSIM) install
        • Wazuh
        • EDR
          • Cortex XDR | Palo Alto Networks
          • Cynet
          • FortiEDR | Fortinet
          • Elastic
        • Elastic
          • Install Elasticsearch, Logstash, and Kibana (Elastic Stack) on Ubuntu 22.04
          • Setting Up Elastic 8 with Kibana, Fleet, Endpoint Security, and Windows Log Collection
        • Threat Intelligence
          • MISP
          • msticpy Microsoft
          • X-Force | IBM
          • Elastic
      • VPN
        • Full-Mesh VPN fastd, tinc, VpnCloud
        • Wireguard
          • WireGuard for Internet access
          • WireGuard on MikroTik and Keenetic
          • WireGuard site to site
        • SoftEther VPN Project
        • Cisco AnyConnect client
        • OpenConnect
        • SSTP python server
      • OS hardening
        • CIS Benchmarks
      • Cloud Providers
      • OpenNebula
        • OpenNebula Edge Cloud - Open Source Cloud & Edge Computing
        • Discover OpenNebula – Open Source Cloud & Edge Computing Platform
        • OpenNebula Multi-Cloud
        • Kubernetes on OpenNebula
        • The Open Source Alternative to Nutanix
        • The Simple Alternative to OpenStack
        • OpenNebula Partner Ecosystem
      • OpenStack
        • Install manual
        • Install with DevStack
      • VM
        • Create a VHD file from a Linux disk
        • Backup / Migration
          • Coriolis
          • Proxmox Backup Server
        • oVirt
        • VMware vCenter
        • Proxmox
      • Docker
        • Container optimization
        • Ubuntu RDP container
      • LXC
        • LXD on Ubuntu 18.04
        • Install, Create and Manage LXC in Ubuntu/Debian
    • Big Data
      • OLAP data qubes
      • Storage and autoscale in Lerua
    • Machine Learning
      • Yandex YaLM 100B. GPT model
      • Kaggle Community Datasts Models
      • AI in video production
      • Image search
      • Chat bots
        • You.com
        • Chat GPT
          • Implementing GPT in NumPy
        • Jailbreak Chat
      • Coding plugins CodeWhisperer
    • Malware
      • Isiaon/Pitraix: Modern Cross-Platform Peer-to-Peer Botnet over TOR
      • theZoo A repository of LIVE malwares
    • Pentest
      • Red Team
        • MITRE ATT&CK matrix
        • C2 Frameworks
          • Brute Ratel C4
          • Cobalt Strike
          • Covenant
          • Havoc Framework
          • Merlin
          • Metasploit
          • Sillenttrinity
          • Sliver
        • Manage and report
          • Dradis Framework
          • Hexway
        • Underground
      • Social engineering
        • Social Engineer Toolkit setoolkit
      • OSINT
        • OSINT for comapny
        • Instagram fishing
      • Forensics
        • Forensics tools
      • Pentesting Methodology
      • Web
      • CI/CD Methodology
      • Cloud Methodology
        • Hacking The Cloud
      • Kubernetes Pentesting
      • Android
        • SSL Unpinning for Android applications
      • iOS
        • SSL unpinning iOS and macOS applications
      • HackBar tool
      • CyberChef Tools
      • Python virtualenv
      • IppSec - YouTube
      • Hacktricks.xyz
    • Compliance
      • 152 ФЗ. Personal data
      • PCI DSS and ГОСТ Р 57580.1-2017
      • Cloud compliance
      • ГОСТ Р 57580.1-2017 для Kubernetes
      • Kubernets as DevSecOps and NIST compliance
      • NIST SP 800-61 cyberincidece control
      • CIS Kubernetes Benchmark v1.6 - RKE2 v1.20
      • CIS Kubernetes Benchmark v1.23 - RKE2
      • Requirements for Russian Banks
      • Tools
        • Chef InSpec
        • Elastic SIEM
    • Asset management
      • CMDBuild
    • Project management
    • Incident management SRE
    • Risk management
      • IT risk management
      • BSI-Standard 200-3
    • Web Dev
      • Cookie security
      • OWASP Top 10 2021
      • Docker nginx php mysql
      • Docker tor hiddenservice nginx
      • Docker Compose wp nginx php mariadb
      • Dependency Checking
        • Nexus Analyzer
        • OWASP dependency-check
      • Yii skeeks cms
      • YiiStudio
    • Art
      • GTK Themes
      • Themes for Xfce Desktop
      • XFCE / Xubuntu Windows 95
      • Moscow events
      • Photo goods
      • Russian style gifts
    • Cryptocurrency
      • News
      • Arbitrage
      • Stocks
      • Exchange aggregators
      • Where to use
      • Prepaid cards
        • BitFree
        • Pyypl Your Money at Your Fingertips
    • IT magazines
      • WIKI and Writeups tools
        • BookStack
        • GitBook
        • MkDocs
        • Wiki.js
        • DokuWiki
    • Languages
    • Learning
      • (ISC)2
        • CISSP
      • Offensive Security
        • OSCP
        • OSEP
        • OSED
      • DevSecOps
        • Certified DevSecOps Professional (CDP)
        • Certified DevSecOps Expert (CDE)
      • Web Security Academy: PortSwigger
    • Relocation
      • London experience
      • IT visas in 2022
      • Remote work
      • Running business in UAE
    • Freenet
      • Independent online services: the philosophy of a free Internet
      • Tor Project Anonymity Online
      • I2P Anonymous Network
    • Services
      • SMS Registration
        • Registering ChatGPT in Russia
      • Local and regional eSIMs for travellers - Airalo
      • Digital busines cards
      • No KYC services and exchanges
Powered by GitBook
On this page
  • Pentesting CI/CD:
  • Methodology
  • VCS
  • Pipelines
  • VCS Pentesting Methodology
  • Pipelines Pentesting Methodology
  • Exploitation Benefits
  • More relevant info
  • Top 10 CI/CD Security Risk
  • Labs
  • References

Was this helpful?

  1. Book
  2. Pentest

CI/CD Methodology

Last updated 1 year ago

Was this helpful?

Pentesting CI/CD:

Methodology

Dev environments have become a major part of today’s attack surface. And within them, the most lucrative assets are the systems responsible for CI and CD — those that build, test, and deploy code — and typically possess the secrets and access to the most critical assets of the organization. So it’s only natural that attackers are continuously on the lookout for novel ways to gain access to these systems.

VCS

VCS stands for Version Control System, this systems allows developers to manage their source code. The most common one is git and you will usually find companies using it in one of the following platforms:

  • Github

  • Gitlab

  • Bitbucket

  • Gitea

  • Cloud providers (they offer their own VCS platforms)

Pipelines

Pipelines allow developers to automate the execution of code (for building, testing, deploying... purposes) after certain actions occurs: A push, a PR, cron... They are terrible useful to automate all the steps from development to production.

However, these systems need to be executed somewhere and usually with privileged credentials to deploy code.

VCS Pentesting Methodology

Even if some VCS platforms allow to create pipelines for this section we are going to analyze only potential attacks to the control of the source code.

Platforms that contains the source code of your project contains sensitive information and people need to be very careful with the permissions granted inside this platform. These are some common problems across VCS platforms that attacker could abuse:

  • Leaks: If your code contains leaks in the commits and the attacker can access the repo (because it's public or because he has access), he could discover the leaks.

  • Access: If an attacker can access to an account inside the VCS platform he could gain more visibility and permissions.

    • Register: Some platforms will just allow external users to create an account.

    • SSO: Some platforms won't allow users to register, but will allow anyone to access with a valid SSO (so an attacker could use his github account to enter for example).

    • Credentials: Username+Pwd, personal tokens, ssh keys, Oauth tokens, cookies... there are several kind of tokens a user could steal to access in some way a repo.

  • Webhooks: VCS platforms allow to generate webhooks. If they are not protected with non visible secrets an attacker could abuse them.

    • If no secret is in place, the attacker could abuse the webhook of the third party platform

    • If the secret is in the URL, the same happens and the attacker also have the secret

  • Code compromise: If a malicious actor has some kind of write access over the repos, he could try to inject malicious code. In order to be successful he might need to bypass branch protections. These actions can be performed with different goals in mid:

    • Compromise the main branch to compromise production.

    • Compromise the main (or other branches) to compromise developers machines (as they usually execute test, terraform or other things inside the repo in their machines).

    • Compromise the pipeline (check next section)

Pipelines Pentesting Methodology

The most common way to define a pipeline, is by using a CI configuration file hosted in the repository the pipeline builds. This file describes the order of executed jobs, conditions that affect the flow, and build environment settings. These files typically have a consistent name and format, for example — Jenkinsfile (Jenkins), .gitlab-ci.yml (GitLab), .circleci/config.yml (CircleCI), and the GitHub Actions YAML files located under .github/workflows. When triggered, the pipeline job pulls the code from the selected source (e.g. commit / branch), and runs the commands specified in the CI configuration file against that code.

Therefore the ultimate goal of the attacker is to somehow compromise those configuration files or the commands they execute.

The Poisoned Pipeline Execution (PPE) vector abuses permissions against an SCM repository, in a way that causes a CI pipeline to execute malicious commands. Users that have permissions to manipulate the CI configuration files, or other files which the CI pipeline job relies on, can modify them to contain malicious commands, ultimately “poisoning” the CI pipeline executing these commands.

For a malicious actor to be successful performing a PPE attack he needs to be able to:

  • Have write access to the VCS platform, as usually pipelines are triggered when a push or a pull request is performed. (Check the VCS pentesting methodology for a summary of ways to get access).

    • Note that sometimes an external PR count as "write access".

  • Even if he has write permissions, he needs to be sure he can modify the CI config file or other files the config is relying on.

    • For this, he might need to be able to bypass branch protections.

There are 3 PPE flavours:

  • D-PPE: A Direct PPE attack occurs when the actor modifies the CI config file that is going to be executed.

  • I-DDE: An Indirect PPE attack occurs when the actor modifies a file the CI config file that is going to be executed relays on (like a make file or a terraform config).

  • Public PPE or 3PE: In some cases the pipelines can be triggered by users that doesn't have write access in the repo (and that might not even be part of the org) because they can send a PR.

    • 3PE Command Injection: Usually, CI/CD pipelines will set environment variables with information about the PR. If that value can be controlled by an attacker (like the title of the PR) and is used in a dangerous place (like executing sh commands), an attacker might inject commands in there.

Exploitation Benefits

Knowing the 3 flavours to poison a pipeline, lets check what an attacker could obtain after a successful exploitation:

  • Secrets: As it was mentioned previously, pipelines require privileges for their jobs (retrieve the code, build it, deploy it...) and this privileges are usually granted in secrets. These secrets are usually accessible via env variables or files inside the system. Therefore an attacker will always try to exfiltrate as much secrets as possible.

    • Depending on the pipeline platform the attacker might need to specify the secrets in the config. This means that is the attacker cannot modify the CI configuration pipeline (I-PPE for example), he could only exfiltrate the secrets that pipeline has.

  • Computation: The code is executed somewhere, depending on where is executed an attacker might be able to pivot further.

    • On-Premises: If the pipelines are executed on premises, an attacker might end in an internal network with access to more resources.

    • Cloud: The attacker could access other machines in the cloud but also could exfiltrate IAM roles/service accounts tokens from it to obtain further access inside the cloud.

    • Platforms machine: Sometimes the jobs will be execute inside the pipelines platform machines, which usually are inside a cloud with no more access.

    • Select it: Sometimes the pipelines platform will have configured several machines and if you can modify the CI configuration file you can indicate where you want to run the malicious code. In this situation, an attacker will probably run a reverse shell on each possible machine to try to exploit it further.

  • Compromise production: If you ware inside the pipeline and the final version is built and deployed from it, you could compromise the code that is going to end running in production.

More relevant info

Top 10 CI/CD Security Risk

Labs

  • On each platform that you can run locally you will find how to launch it locally so you can configure it as you want to test it

References

*** is an open-source tool for auditing your software supply chain stack for security compliance based on a new . The auditing focuses on the entire SDLC process, where it can reveal risks from code time into deploy time.

Check this interesting article about the top 10 CI/CD risks according to Cider:

Gitea + Jenkins lab:

***: Checkov is a static code analysis tool for infrastructure-as-code.

Github Security
Gitea Security
Concourse Security
CircleCI Security
TravisCI Security
Jenkins Security
Apache Airflow Security
Terraform Security
Atlantis Security
Cloudflare Security
Okta Security
Ansible Tower / AWX / Automation controller Security
Chain-bench
CIS Software Supply Chain benchmark
https://www.cidersecurity.io/top-10-cicd-security-risks/
https://github.com/cider-security-research/cicd-goat
Checkov
https://www.cidersecurity.io/blog/research/ppe-poisoned-pipeline-execution/?utm_source=github&utm_medium=github_page&utm_campaign=ci%2fcd%20goat_060422
https://cloud.hacktricks.xyz/pentesting-ci-cd/pentesting-ci-cd-methodology