Skip to main content

Command Palette

Search for a command to run...

CI/CD Security for Product Security Engineers: A Practical Enterprise Guide

Updated
•29 min read•View as Markdown
S
I like breaking things that are supposed to be secure. When I’m not hunting vulnerabilities, I’m exploring systems, architectures, and the assumptions behind them.

Imagine you are a Product Security Engineer at a large enterprise.

Your company is building a modern SaaS platform. Hundreds of developers are committing code every day. Applications are built as containers, deployed to Kubernetes, and released multiple times a day.

One morning, someone asks you:

"Can you review the security of our CI/CD pipeline?"

You open the pipeline and see:

Developer
   ↓
GitHub
   ↓
GitHub Actions
   ↓
Docker Build
   ↓
SAST + SCA + Secret Scan
   ↓
Container Image
   ↓
JFrog Artifactory
   ↓
Helm
   ↓
Kubernetes
   ↓
Production

Suddenly, you realize something important:

CI/CD is not just a DevOps topic. It is part of the application's security architecture.

A vulnerability in the application is only one possible attack path.

An attacker could instead compromise:

  • the source repository,

  • a CI runner,

  • a build credential,

  • a dependency,

  • a container image,

  • an artifact repository,

  • a Helm chart,

  • a Kubernetes service account,

  • or the deployment pipeline itself.

That is why Product Security Engineers should understand the entire software delivery chain.

This article walks through that chain as it exists in real enterprise environments.


1. The Big Picture: Your Application Has a Delivery Supply Chain

Before looking at individual tools, build one mental model.

A modern enterprise application often travels through this lifecycle:

Code
  ↓
Review
  ↓
Build
  ↓
Test
  ↓
Security Scan
  ↓
Package
  ↓
Sign
  ↓
Store
  ↓
Promote
  ↓
Deploy
  ↓
Run
  ↓
Monitor

Every arrow is a potential security boundary.

For example:

Developer
   ↓
Git Repository
   ↓
CI Pipeline
   ↓
Build Runner
   ↓
Artifact
   ↓
Registry
   ↓
Kubernetes
   ↓
Production

A security engineer should continuously ask:

Who controls this step, what credentials are available here, and what happens if this step is compromised?

That question will take you surprisingly far.


2. CI and CD: Start With the Basics

Let's first understand what CI/CD actually means.

2.1 Continuous Integration — CI

CI automatically builds and validates code whenever developers push changes or create pull requests.

A typical flow:

Developer
   ↓
Git Push / Pull Request
   ↓
Build
   ↓
Unit Tests
   ↓
SAST
   ↓
SCA
   ↓
Secret Scan
   ↓
Artifact

Real-life example:

A developer modifies an authentication service and opens a pull request.

The pipeline automatically:

  1. Builds the application.

  2. Runs unit tests.

  3. Runs SAST.

  4. Checks dependencies.

  5. Searches for leaked secrets.

  6. Produces a build artifact.

The developer doesn't manually execute all of these steps.

Security question

Can a developer bypass the security checks simply by editing the pipeline?

That brings us to an important concept.


3. Pipeline-as-Code: The Pipeline Is Code Too

Modern pipelines are usually stored in Git as configuration files.

Examples:

Jenkinsfile
.github/workflows/build.yml
.gitlab-ci.yml
azure-pipelines.yml

Pipeline-as-Code means the CI/CD workflow itself is version-controlled and executable.

Example:

build:
  script:
    - npm install
    - npm test
    - npm build

At first glance this looks harmless.

But imagine an attacker modifies the pipeline:

build:
  script:
    - cat ~/.aws/credentials

Now the pipeline may become a credential-stealing mechanism.

Real-life security scenario

A developer account is compromised.

The attacker cannot directly access AWS production.

But the developer can modify the CI workflow.

The CI runner has access to cloud credentials.

Therefore:

Compromised Developer Account
          ↓
Modified Pipeline
          ↓
CI Runner
          ↓
Cloud Credentials
          ↓
Cloud Account

This is why:

Your CI/CD configuration is part of your application's attack surface.


4. Git: The First Security Boundary

Before code reaches the pipeline, it lives in a source-code repository.

Common platforms include:

  • GitHub

  • GitLab

  • Bitbucket

  • Azure Repos

Git is where application source code and often CI/CD configuration are maintained.

Important concepts:

Repository

The central location containing application source code, configuration, documentation and pipeline definitions.

Branch

A separate line of development used to isolate changes.

Pull Request / Merge Request

A formal request to review and merge a change into another branch.

Protected Branch

A branch that requires specific controls such as review or status checks before changes can be merged.

CODEOWNERS

A mechanism that requires designated owners to review changes to specific files or directories.

Commit Signing

A mechanism used to provide stronger assurance about who authored a commit.


5. What Should a Product Security Engineer Look For in Git?

Suppose your company's production deployment configuration is stored here:

/deployment/prod/

Ask:

  • Who can modify it?

  • Is production branch protection enabled?

  • Is two-person review required?

  • Can authors approve their own changes?

  • Are CODEOWNERS configured?

  • Are security-sensitive files specially protected?

  • Is MFA enabled?

  • Are old credentials still valid?

  • Are repository actions audited?

A very common mistake is protecting application code while forgetting to protect:

Jenkinsfile
.github/workflows/
Dockerfile
Helm charts
Terraform
Kubernetes manifests

These files can sometimes be more security-sensitive than application code, because they influence how the application is built and deployed.


6. CI Runners: Where the Pipeline Actually Executes

A pipeline needs a machine on which to run.

That machine is typically called a:

  • Runner

  • Agent

  • Build Agent

  • Worker

A CI runner is the execution environment in which build and pipeline commands run.

Architecture:

CI Controller
      ↓
Build Runner
      ↓
Build Commands

Examples include:

  • Jenkins agents

  • GitHub Actions runners

  • GitLab runners


7. Why Build Runners Are Security-Critical

Imagine the runner has:

AWS credentials
Docker access
Git credentials
Registry credentials
Kubernetes credentials

Now imagine a malicious pull request executes:

env

or:

cat ~/.config/...

The attacker may discover credentials available to the build.

This is why enterprise environments increasingly use:

Ephemeral Runners

A fresh build environment is created for a job and destroyed after the job finishes.

Instead of:

One permanent runner
       ↓
Job 1
       ↓
Job 2
       ↓
Job 3

you use:

Fresh Runner → Job → Destroy
Fresh Runner → Job → Destroy
Fresh Runner → Job → Destroy

This reduces cross-job contamination and persistence.


8. Secrets in CI/CD

Almost every enterprise pipeline needs secrets.

For example:

AWS credentials
GitHub tokens
Docker registry credentials
Signing keys
Database credentials
API keys
Kubernetes credentials

The dangerous approach is:

AWS_SECRET_ACCESS_KEY: "my-super-secret-key"

inside source control.

Instead, enterprises typically use dedicated secret managers.

Examples:

  • AWS Secrets Manager

  • HashiCorp Vault

  • Azure Key Vault

  • Google Secret Manager

  • CI/CD secret stores

A secret manager stores sensitive credentials separately from application source code and makes controlled retrieval possible.


9. Secret Injection

The next question is:

"How does the pipeline actually get the secret?"

Typical flow:

Secret Manager
      ↓
CI Pipeline
      ↓
Build Job

A mature design tries to use:

  • Short-lived credentials

  • Least privilege

  • Just-in-time access

  • Secret masking

  • Restricted scope

Example

Instead of giving CI permanent AWS administrator credentials:

CI
 ↓
Long-lived AWS Admin Key

a better architecture may look like:

CI Identity
   ↓
Assume Role
   ↓
Temporary Credentials
   ↓
Required AWS Permissions

That leads to another important cloud-security concept:

Workload identity.


10. Workload Identity

Workload identity allows a workload or pipeline to obtain cloud permissions through an identity rather than embedding long-lived access keys.

For example:

GitHub Actions
      ↓
Federated Identity
      ↓
AWS IAM Role
      ↓
Temporary Credentials

This is much safer than storing:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

for months.

For a Product Security Engineer, understanding CI identity → cloud role → permissions is extremely valuable.


11. Build: Turning Source Code Into Software

Now the pipeline begins building your application.

Source Code
    ↓
Compiler / Build System
    ↓
Binary / Package / Container

Depending on the application, the output could be:

JAR
WAR
ZIP
Python wheel
npm package
Binary
Container image

A build is the process of converting source code and dependencies into a deployable software artifact.

The security question is:

Can we trust the build process itself?

Because if the build environment is compromised, even secure source code can produce a malicious artifact.


12. SAST: Look for Security Bugs in Code

The next step is usually static analysis.

SAST analyzes source code without running the application to identify potential security issues.

Examples:

  • Fortify

  • Checkmarx

  • Semgrep

  • CodeQL

  • SonarQube

Pipeline:

Code
 ↓
SAST
 ↓
Findings
 ↓
Policy
 ↓
Pass / Fail

Example:

String query =
    "SELECT * FROM users WHERE id=" + userInput;

A SAST tool may identify a potential SQL injection issue.


13. SCA: Secure the Dependencies

Your developers didn't write everything.

Applications depend on open-source libraries.

For example:

Application
 ├── Spring
 ├── Jackson
 ├── Log4j
 ├── OpenSSL
 └── Hundreds of transitive dependencies

SCA identifies vulnerabilities and risks in third-party software dependencies.

Examples:

  • Snyk

  • Mend

  • Black Duck

  • Dependabot

  • OWASP Dependency-Check

The famous Log4Shell incident demonstrated why dependency visibility matters.

A vulnerability doesn't need to exist in your own code.

It may exist inside something your application imports.


14. SBOM: Know What Is Inside Your Software

This leads naturally to another concept:

SBOM — Software Bill of Materials

An SBOM is an inventory of the components and dependencies contained in a software product.

Think of it as:

Software = Food
SBOM = Ingredient List

Example:

payment-service
 ├── Spring 6.x
 ├── Jackson 2.x
 ├── Netty
 ├── OpenSSL
 └── PostgreSQL driver

Common formats:

  • SPDX

  • CycloneDX

Why does this matter?

Suppose a new critical vulnerability is announced in a library.

You can ask:

"Which of our products contain this component?"

Without an SBOM, that question can become a painful manual investigation.


15. Secret Scanning

Another common CI security control is secret scanning.

Secret scanning searches source code and commits for credentials such as API keys, tokens and private keys.

Examples:

  • Gitleaks

  • TruffleHog

  • GitHub Secret Scanning

Example:

AKIA....
-----BEGIN PRIVATE KEY-----
ghp_....

A mature pipeline attempts to detect secrets before they reach production or, ideally, before they are committed.

One important lesson:

Removing a secret from the current file does not necessarily remove it from Git history.

The credential may already exist in previous commits.


16. Container Security: Packaging the Application

Modern enterprise applications increasingly run inside containers.

A simplified flow:

Application Code
      ↓
Dockerfile
      ↓
Docker Build
      ↓
Container Image

A container image is a portable package containing the application and its runtime dependencies.

Example:

FROM python:3.12

COPY application.py /app/

CMD ["python", "/app/application.py"]

Now Product Security enters the container world.


17. Container Security Questions

As a security engineer, inspect:

Base Image

Where did the image originate?

python:3.12
ubuntu:24.04
alpine:3.x

Root User

Does the application run as root?

USER root

Running as a non-root user is generally preferable when feasible.

Package Installation

Are unnecessary operating-system packages installed?

Secrets

Are passwords or tokens copied into the image?

Bad:

COPY secrets.env /app/

Privileges

Does the container require unnecessary Linux capabilities?

Image Lifecycle

How often are base images patched?

These are classic container-security review questions.


18. Container Image Scanning

After building the image, scan it.

Docker Build
     ↓
Container Scan
     ↓
CVE Findings
     ↓
Policy

Tools include:

  • Trivy

  • Grype

  • Prisma Cloud

  • Wiz

  • Snyk

  • JFrog Xray

A scanner may report:

openssl
Version: 3.0.x
Severity: Critical

The pipeline can then decide whether deployment should continue.


19. Container Registry

Where does the container image go after it is built?

Usually to a registry.

Examples:

  • Amazon ECR

  • Azure Container Registry

  • Google Artifact Registry

  • JFrog Artifactory

  • Harbor

A container registry stores and distributes container images and related artifacts.

Typical enterprise flow:

CI
 ↓
Build Image
 ↓
Scan Image
 ↓
Sign Image
 ↓
Registry
 ↓
Kubernetes

Security controls include:

  • Authentication

  • RBAC

  • Immutable artifacts

  • Vulnerability scanning

  • Signing

  • Audit logs


20. Image Tags vs Image Digests

You will often see:

myapp:latest
myapp:1.5.2

But container images also have immutable digests:

sha256:abc123....

A tag can be moved to a different image.

A digest identifies the exact image content.

Therefore, from a security and supply-chain perspective:

Tag
   ↓
"Which version?"

Digest
   ↓
"Exactly which content?"

This distinction becomes important when investigating supply-chain attacks.


21. Container Image Signing

Now we reach one of the most important modern software-supply-chain concepts.

Imagine your CI pipeline builds:

payment-service:v12

How does Kubernetes know that this image was actually produced by your trusted build pipeline?

This is where signing comes in.

Image signing provides a cryptographic mechanism for establishing authenticity and integrity of container images.

Typical flow:

Build Image
    ↓
Scan
    ↓
Sign Image
    ↓
Registry
    ↓
Verify Signature
    ↓
Deploy

Common technologies include:

  • Sigstore

  • Cosign

  • Notation

The important security question becomes:

"Can we prevent an untrusted image from being deployed?"


22. Artifact Signing and Provenance

Container images are only one type of artifact.

You may also produce:

JAR
ZIP
Binary
Python package
Helm chart

Artifact signing provides integrity and authenticity for software artifacts.

But signatures alone don't answer:

"Where did this artifact come from?"

That's where provenance comes in.

Build provenance records information about how and where an artifact was produced.

Conceptually:

Production Image
      ↓
Digest
      ↓
Signature
      ↓
Build Attestation
      ↓
CI Pipeline
      ↓
Commit SHA
      ↓
Source Repository

This gives security teams a much stronger chain of trust.


23. SLSA: Understanding Software Supply-Chain Assurance

You don't need to memorize every SLSA specification detail for most interviews.

Understand the idea.

SLSA is a framework for improving the security and provenance of software supply chains.

The basic question is:

Can you establish where software came from, how it was built, and whether the build process can be trusted?

This is becoming increasingly important as organizations defend against software supply-chain attacks.


24. Helm: The Kubernetes Packaging Layer

Now our application is ready to be deployed.

But Kubernetes deployments can involve many YAML files:

Deployment
Service
Ingress
ConfigMap
Secret
ServiceAccount
NetworkPolicy

Managing them separately across:

DEV
QA
STAGING
PROD

can become messy.

Enter Helm.

Helm is a package manager and templating mechanism commonly used to package and deploy Kubernetes applications.

A Helm chart may look like:

my-app/
├── Chart.yaml
├── values.yaml
└── templates/
    ├── deployment.yaml
    ├── service.yaml
    ├── ingress.yaml
    └── configmap.yaml

You might have:

values-dev.yaml
values-qa.yaml
values-prod.yaml

Then deploy:

helm upgrade --install my-app ./chart \
    -f values-prod.yaml

25. Why Helm Matters to Product Security

Helm isn't "just a deployment tool."

A Helm chart can control security-sensitive Kubernetes settings.

For example:

securityContext:
  runAsNonRoot: true

or:

hostNetwork: true

or:

privileged: true

Therefore a malicious or poorly configured chart could introduce serious security weaknesses.

When reviewing a Helm chart, ask:

  • Does the container run as root?

  • Is privileged mode enabled?

  • Are Linux capabilities restricted?

  • Are host paths mounted?

  • Is host networking enabled?

  • Are secrets handled securely?

  • Does the workload have excessive RBAC?

  • Is the image trusted and signed?

  • Is a NetworkPolicy defined?

  • Is the service publicly exposed?

This is where Kubernetes security and CI/CD security meet.


26. Kubernetes: The Deployment Runtime

Helm may generate Kubernetes manifests, but Kubernetes is the platform that actually runs the workload.

Kubernetes orchestrates containers and manages their deployment, networking, scaling and lifecycle.

At minimum, a Product Security Engineer should understand:

Cluster
 ├── Namespace
 ├── Pod
 ├── Deployment
 ├── Service
 ├── Ingress
 ├── ConfigMap
 ├── Secret
 ├── ServiceAccount
 ├── Role
 ├── RoleBinding
 └── NetworkPolicy

You don't need to become a Kubernetes administrator.

But you should understand its security model.


27. Kubernetes RBAC

One of the most important security concepts is Kubernetes authorization.

Typical model:

Workload
   ↓
ServiceAccount
   ↓
Role
   ↓
Permissions

Kubernetes RBAC controls which users and workloads can perform which actions against Kubernetes resources.

Example:

A service only needs:

GET pods

but accidentally receives:

create pods
delete pods
get secrets
create secrets

That is excessive privilege.

A compromised workload could abuse these permissions.


28. Kubernetes Service Accounts

A ServiceAccount provides an identity for workloads running inside Kubernetes.

Think:

Pod
 ↓
ServiceAccount
 ↓
Kubernetes API permissions

This is an important bridge between:

Application Security

and:

Cloud / Infrastructure Security

Because a compromised application may attempt to use the identity assigned to its workload.


29. Kubernetes Secrets

Kubernetes supports:

kind: Secret

But Product Security Engineers should understand that:

A Kubernetes Secret object is not the same thing as a complete enterprise secrets-management strategy.

Security depends on:

  • RBAC

  • Encryption at rest

  • Access controls

  • Secret rotation

  • External secret managers

  • Audit logging

A mature architecture may integrate Kubernetes with systems such as:

AWS Secrets Manager
HashiCorp Vault
Azure Key Vault

30. NetworkPolicy

Kubernetes NetworkPolicy controls which workloads can communicate with which other workloads.

Imagine:

Internet
   ↓
Frontend
   ↓
API
   ↓
Database

A database should not normally accept connections from every pod in the cluster.

A NetworkPolicy can enforce:

Frontend → API
API → Database
Anything else → DENY

This is essentially micro-segmentation for Kubernetes workloads.


31. Infrastructure as Code — IaC

The application isn't the only thing represented as code.

Infrastructure is increasingly code too.

Examples:

  • Terraform

  • CloudFormation

  • Pulumi

  • Bicep

Infrastructure as Code allows infrastructure resources to be defined, reviewed and deployed programmatically.

Example:

Terraform
    ↓
VPC
    ↓
EKS
    ↓
Load Balancer
    ↓
Security Groups
    ↓
IAM

32. Why IaC Matters to Product Security

Imagine a Terraform change opens:

0.0.0.0/0

to port:

5432

on a database security group.

The application code might be perfectly secure.

The infrastructure configuration has still created a serious exposure.

This is why security teams scan IaC before deployment.

Tools include:

  • Checkov

  • KICS

  • Trivy

  • Terrascan

  • tfsec

Typical pipeline:

Terraform Code
      ↓
IaC Scan
      ↓
Policy
      ↓
Pass / Fail

33. Security Gates: When Should CI/CD Stop?

At this point, your pipeline may perform:

SAST
SCA
Secret Scan
IaC Scan
Container Scan

But there is another question:

When should the pipeline block deployment?

That is the purpose of security gates.

Example:

Critical vulnerability
        ↓
   Deployment BLOCKED

But mature enterprises rarely use:

"Any vulnerability = block everything."

Instead they often apply risk-based conditions.

For example:

Critical
+
Reachable
+
Production exposed
+
Fix available
        ↓
BLOCK

Whereas:

Low severity
+
Not exploitable
+
No production exposure
        ↓
Continue + Track

The exact policy varies by organization, but the important concept is:

Security controls need an enforcement policy, not just a scanner.


34. Policy as Code

How do you enforce those rules consistently?

Through Policy as Code.

Policy as Code means security and compliance rules are expressed in machine-readable policies and evaluated automatically.

Examples:

  • Open Policy Agent — OPA

  • Kyverno

  • Conftest

  • HashiCorp Sentinel

Example policy:

Production containers must not run as root.

Or:

Only signed images may be deployed.

Or:

Public databases are prohibited.

The advantage is consistency.

Instead of:

Engineer reviews deployment manually

you have:

Deployment
   ↓
Policy Engine
   ↓
PASS / FAIL

35. Environment Promotion

Enterprise applications usually have multiple environments:

DEV
 ↓
QA
 ↓
STAGING
 ↓
PRODUCTION

A mature software-supply-chain design generally tries to build once and promote the same artifact.

For example:

Build Image
   ↓
Image Digest X
   ↓
DEV
   ↓
QA
   ↓
STAGING
   ↓
PRODUCTION

Rather than:

Build DEV image
Build QA image
Build PROD image

Why?

Because rebuilding can produce different artifacts.

The security objective is:

Test the artifact and then promote that same trusted artifact.


36. Deployment Strategies

Once the artifact is approved, how does it reach users?

There are several common strategies.

Rolling Deployment

Instances are gradually replaced with the new version.

v1 v1 v1 v1
   ↓
v1 v1 v2 v2
   ↓
v2 v2 v2 v2

Blue/Green Deployment

Two environments exist and traffic switches from the old version to the new version.

Blue  → Current
Green → New

Canary Deployment

The new version is exposed to a small percentage of traffic before wider rollout.

95% → v1
 5% → v2

then:

90/10
70/30
50/50
0/100

These are deployment concepts, but Product Security Engineers should understand them because security-sensitive releases may require controlled exposure.


37. GitOps: When Git Becomes the Deployment Source of Truth

Traditional deployment might look like:

CI/CD
  ↓
helm upgrade
  ↓
Kubernetes

A GitOps model is different.

GitOps uses Git as the desired-state source for infrastructure and application deployment.

Typical flow:

Developer
   ↓
Git
   ↓
Helm / Kubernetes Configuration
   ↓
Argo CD
   ↓
Kubernetes

The GitOps controller continuously reconciles the cluster with the desired state.

Popular tools:

  • Argo CD

  • Flux


38. Argo CD

Argo CD is a GitOps continuous-delivery tool commonly used to deploy applications to Kubernetes.

Example:

Git Repository
      ↓
Helm Chart / Kubernetes Manifest
      ↓
Argo CD
      ↓
Kubernetes

From a security perspective, understand:

  • Argo CD RBAC

  • Repository credentials

  • Cluster credentials

  • Application permissions

  • Sync policies

  • Secrets

  • Project isolation

If an attacker compromises Argo CD, the deployment layer itself may be compromised.


39. Admission Control: The Last Gate Before Kubernetes Runs the Workload

This is a more advanced but very valuable concept.

Imagine the CI pipeline missed something.

Can Kubernetes still reject the deployment?

Yes.

Admission control evaluates Kubernetes API requests before objects are persisted or accepted into the cluster.

Conceptually:

Deployment Request
        ↓
Admission Policy
        ↓
 ┌───────────────┐
 │ Is image      │
 │ trusted?      │
 │ Is container  │
 │ privileged?   │
 │ Is policy OK? │
 └───────┬───────┘
         ↓
      Allow / Deny

Tools such as:

  • OPA Gatekeeper

  • Kyverno

can enforce policies at the Kubernetes boundary.

This creates defense in depth:

CI Security Gate
        +
Kubernetes Admission Policy

40. Runtime Security: CI/CD Doesn't End at Deployment

Many organizations make one mistake:

They secure the pipeline but forget what happens after deployment.

Your application is now running:

Container
   ↓
Kubernetes
   ↓
Cloud
   ↓
Production

Runtime security can include:

  • Container runtime monitoring

  • Kubernetes monitoring

  • Cloud threat detection

  • Network detection

  • SIEM

  • EDR

  • CNAPP capabilities

For AWS environments, security engineers should at least understand the role of services such as:

  • CloudTrail

  • GuardDuty

  • Security Hub

  • Inspector

  • AWS Config

The exact architecture varies, but the principle is universal:

Preventive security controls reduce risk; runtime detection helps identify what got through.


41. CI/CD Supply-Chain Attacks

Now let's step back.

Why do we care about all of this?

Because an attacker may decide not to attack your application directly.

Instead, they may attack the software delivery chain.

For example:

Compromised Developer
        ↓
Git Repository
        ↓
Modified Pipeline
        ↓
CI Runner
        ↓
Malicious Artifact
        ↓
Container Registry
        ↓
Kubernetes
        ↓
Production

Or:

Developer
   ↓
Malicious Dependency
   ↓
Build
   ↓
Artifact
   ↓
Production

Or:

Registry
   ↓
Tampered Image
   ↓
Deployment

This is the essence of software supply-chain security.


42. Dependency Confusion

Imagine your company uses:

internal-auth-library

An attacker publishes a public package with the same name.

If your build system resolves the public package instead of the internal one:

CI
 ↓
Package Manager
 ↓
Attacker's Package
 ↓
Build
 ↓
Production

This is called dependency confusion.

Security engineers should understand:

  • Internal package repositories

  • Package scopes/namespaces

  • Dependency pinning

  • Lock files

  • Registry priority

  • Package provenance


43. Typosquatting

Another classic supply-chain technique is typosquatting.

For example:

requests

vs.

requsets

An attacker publishes the fake package.

A developer accidentally installs it.

The result may be:

Developer Error
      ↓
Malicious Package
      ↓
CI
      ↓
Production

The attack is simple.

The impact can be significant.


44. Build Reproducibility

Here is a more advanced concept.

Imagine you build:

commit ABC

today and get:

artifact XYZ

Tomorrow, you rebuild exactly the same commit and get:

artifact DEF

Why did the output change?

This raises concerns around:

  • build dependencies

  • timestamps

  • external package changes

  • compromised build infrastructure

Reproducible builds aim to produce the same artifact from the same source and build inputs.

You don't need to implement reproducible builds yourself in every organization, but understanding the concept is useful for advanced supply-chain discussions.


45. Immutable Infrastructure and Artifacts

A mature enterprise environment tries to reduce the ability to silently modify deployed artifacts.

For example:

Image Digest
+
Signed Artifact
+
Immutable Registry
+
Controlled Promotion

Instead of:

latest
latest
latest

The goal is:

Know exactly what was built, what was tested, what was approved and what was deployed.


46. Security Observability

A secure pipeline must also be observable.

You want records of:

Who changed the code?
Who approved the PR?
Who ran the pipeline?
What artifact was built?
Who pushed the artifact?
Who deployed it?
Which cluster received it?
Which identity performed the action?

Important logs may come from:

Git
CI/CD
Artifact Registry
Kubernetes
Cloud
IAM
Security Tools

This becomes incredibly valuable during incident response.


47. A Real-World Enterprise Example

Let's put the entire story together.

Suppose your organization builds:

Customer Identity Service

The developer commits code:

GitHub
   ↓
Pull Request

Branch protection requires:

Code Review
+
Automated Tests
+
Security Checks

The CI pipeline starts:

GitHub Actions
   ↓
Ephemeral Runner

The pipeline performs:

SAST
SCA
Secret Scan
Unit Tests

Then:

Docker Build
   ↓
Container Scan
   ↓
SBOM
   ↓
Image Sign

The image is pushed to:

JFrog Artifactory

The deployment configuration is defined through:

Helm

The deployment targets:

Kubernetes

Kubernetes enforces:

RBAC
NetworkPolicy
Admission Policy

Argo CD manages deployment:

Git
 ↓
Argo CD
 ↓
Kubernetes

Finally:

Production
 ↓
Runtime Monitoring
 ↓
Cloud Detection
 ↓
SIEM

Now imagine an attacker gets access to a developer account.

The security architecture should ideally create multiple barriers:

Compromised Developer
        ↓
Branch Protection
        ↓
PR Review
        ↓
CI Security Gates
        ↓
Artifact Scanning
        ↓
Image Signing
        ↓
Registry Controls
        ↓
Admission Policy
        ↓
Kubernetes RBAC
        ↓
Runtime Detection

That is defense in depth across the software supply chain.


48. The CI/CD Security Stack You Should Know

As a Product Security Engineer, don't memorize only product names.

Understand the category and its purpose.

Area What it does Examples
Git Stores source and pipeline code GitHub, GitLab
CI/CD Automates build/test/deployment Jenkins, GitHub Actions, GitLab CI
SAST Finds vulnerabilities in source code Fortify, Semgrep, CodeQL
SCA Finds vulnerable dependencies Snyk, Black Duck, Mend
Secret Scanning Finds leaked credentials Gitleaks, TruffleHog
SBOM Lists software components SPDX, CycloneDX
Container Build Packages application as image Docker, Buildah
Container Scan Finds image vulnerabilities Trivy, Grype
Registry Stores artifacts/images JFrog, ECR, Harbor
Signing Establishes artifact/image authenticity Cosign, Notation
Provenance Records how artifacts were built SLSA, attestations
Helm Packages Kubernetes deployments Helm
Kubernetes Runs containerized workloads Kubernetes
RBAC Controls access Kubernetes RBAC
NetworkPolicy Restricts workload communication Kubernetes NetworkPolicy
IaC Defines infrastructure as code Terraform, CloudFormation
IaC Security Detects insecure infrastructure configuration Checkov, KICS
Policy as Code Automates security rules OPA, Kyverno
GitOps Manages deployment from Git Argo CD, Flux
Admission Control Blocks unsafe Kubernetes objects Kyverno, Gatekeeper
Runtime Security Detects runtime threats GuardDuty, CNAPP platforms
SIEM Centralizes and analyzes security logs Splunk, Microsoft Sentinel

49. What You Should Know for Product Security Interviews

You don't need to become a Jenkins administrator.

You need to be able to explain the security architecture.

For example, an interviewer may ask:

"How would you secure a CI/CD pipeline?"

A strong answer should naturally cover:

Source Control
      ↓
Branch Protection
      ↓
PR Review
      ↓
Pipeline Security
      ↓
Ephemeral Runners
      ↓
Short-Lived Credentials
      ↓
SAST / SCA / Secret Scan
      ↓
SBOM
      ↓
Container Scanning
      ↓
Artifact Signing
      ↓
Artifact Registry
      ↓
Security Gates
      ↓
Helm / Kubernetes
      ↓
RBAC + NetworkPolicy
      ↓
Admission Policies
      ↓
Runtime Detection

That's much stronger than simply saying:

"We run SAST and DAST in Jenkins."


50. Your Product Security Mental Model

When reviewing an enterprise CI/CD architecture, think in these five layers.

Layer 1 — Source

Ask:

Can unauthorized or malicious code enter the system?

Think:

Git
PR
Branch Protection
CODEOWNERS
MFA
Commit Signing

Layer 2 — Build

Ask:

Can the build process be compromised?

Think:

CI
Runners
Secrets
Workload Identity
Build Isolation
Dependencies

Layer 3 — Artifact

Ask:

Can we trust what was produced?

Think:

SBOM
Scanning
Signing
Provenance
Registry
Immutable Artifacts

Layer 4 — Deployment

Ask:

Can an attacker deploy something they shouldn't?

Think:

Helm
Kubernetes
Argo CD
RBAC
Admission Control
Policy as Code

Layer 5 — Runtime

Ask:

What happens if something still gets through?

Think:

Runtime Security
Logging
Monitoring
Detection
Incident Response

51. The One Diagram You Should Remember

If you remember only one diagram from this entire article, remember this:

                         SOFTWARE SUPPLY CHAIN
                                  │
                                  ▼
                              SOURCE
                                  │
                         Git / Pull Request
                                  │
                                  ▼
                                BUILD
                                  │
                     CI / Runner / Credentials
                                  │
                                  ▼
                              SECURITY
                                  │
                SAST / SCA / Secrets / IaC / SBOM
                                  │
                                  ▼
                              PACKAGE
                                  │
                       Container / Artifact
                                  │
                                  ▼
                           VERIFY & SIGN
                                  │
                    Signature / Provenance
                                  │
                                  ▼
                              STORE
                                  │
                     JFrog / ECR / Registry
                                  │
                                  ▼
                             DEPLOY
                                  │
                     Helm / Argo CD / K8s
                                  │
                                  ▼
                              ENFORCE
                                  │
                 RBAC / NetworkPolicy / OPA
                                  │
                                  ▼
                              RUNTIME
                                  │
                  Monitoring / Detection / SIEM

52. Final Takeaway

A Product Security Engineer does not need to know every DevOps tool.

You need to understand how software moves from a developer's laptop to a production workload — and where trust can be broken along the way.

The most useful progression to remember is:

CODE
 ↓
REVIEW
 ↓
BUILD
 ↓
TEST
 ↓
SCAN
 ↓
SBOM
 ↓
PACKAGE
 ↓
SIGN
 ↓
STORE
 ↓
PROMOTE
 ↓
HELM
 ↓
KUBERNETES
 ↓
ENFORCE
 ↓
PRODUCTION
 ↓
MONITOR

And at every stage, ask four questions:

Who controls this?

What credentials are available?

Can this step be bypassed or tampered with?

How do we prove that the final production artifact is the one we intended to deploy?

That is the mindset that turns CI/CD knowledge into Product Security expertise.


Product Security Engineer's CI/CD Learning Path

If you're building this knowledge specifically for a Product Security role, a practical order is:

1. Git + Pull Requests
        ↓
2. CI/CD Fundamentals
        ↓
3. Pipeline-as-Code
        ↓
4. CI/CD Secrets + Identity
        ↓
5. SAST + SCA + Secret Scanning
        ↓
6. Docker + Container Security
        ↓
7. SBOM
        ↓
8. Container Registry
        ↓
9. Image Signing + Provenance
        ↓
10. Helm
        ↓
11. Kubernetes Security
        ↓
12. IaC Security
        ↓
13. Policy as Code
        ↓
14. GitOps / Argo CD
        ↓
15. Supply-Chain Security
        ↓
16. Runtime Security

Once you understand this journey, terms such as Helm, Argo CD, SBOM, Cosign, SLSA, OPA, Kubernetes RBAC, IaC, runners and provenance stop looking like disconnected DevOps buzzwords.

They become pieces of one larger system:

The secure software delivery supply chain.

9 views

Cloud & Kubernetes Security

Part 1 of 1

Cloud & Kubernetes Security