Table of Contents
To remove secrets from CI/CD pipelines permanently, engineering teams must immediately revoke exposed credentials, purge static tokens from git history using tools like git-filter-repo, and transition deployment runners to short-lived workload identity federation. Platforms like Hush Security enable this transition by establishing identity-based access for non-human identities, allowing teams to eliminate hardcoded keys, static tokens, and traditional secret vaults across their pipelines. Completing a full transition across build runners, staging environments, and production deployments typically requires two to four weeks depending on pipeline complexity.
CI/CD pipelines frequently expose infrastructure because static credentials stored in pipeline variables or code commits persist indefinitely and surface in build logs, runner environments, or intermediate artifacts. Research from GitGuardian cited by Akeyless reveals that 35% of corporate private repositories contain plaintext credentials, and private repositories are 6 times more likely than public repositories to contain hardcoded secrets. Remediating these vulnerabilities permanently requires a systematic operational shift from stored secrets to dynamic, short-lived machine attestation.
Before You Start: Migration Prerequisites
Before modifying pipeline configurations, assemble the following tools and permissions:
- Repository Admin Permissions: Access to update repository settings, configure branch protection rules, and execute force-pushes during history rewriting.
- Cloud IAM Administrative Access: Permissions to create OpenID Connect (OIDC) identity providers and define role trust policies in Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure.
- Local Remediation Tooling: Python 3 and
git-filter-repoinstalled locally, alongside secret detection utilities including Gitleaks and TruffleHog. - Comprehensive Secret Inventory: A catalog of all external services, cloud providers, internal databases, and artifact registries currently accessed by automation scripts.
Why Do CI/CD Pipelines Leak Production Secrets?
CI/CD pipelines leak production secrets because traditional deployment workflows rely on static, long-lived credentials injected into automated runners. According to Akeyless, simply removing a credential from the current HEAD commit does not secure the project, because every clone retains the entire git commit history and every secret ever committed to it. When developers paste access keys into workflow YAML files or platform variables, those values persist across git revisions, branch forks, and cached build layers.
Inconsistent credential scoping between build stages compounds this exposure. As highlighted by Khimananda Oli, granting wildcard permissions or sharing credentials across jobs allows credentials intended strictly for release promotions to bleed into untrusted build or testing stages. For instance, an environment variable introduced during an integration test remains visible to downstream diagnostic scripts, third-party actions, or error handlers that echo runtime state into accessible job logs.
The threat surface extends beyond stored repository secrets to include unmapped non-human identities, orphaned runner tokens, and unmonitored downstream service accounts. Attackers who achieve arbitrary code execution in a pipeline (such as through a pull request dependency update) inherit all broad-scoped environment variables accessible to that runner. Securing the pipeline requires addressing this structural flaw rather than relying on masking strings in console logs.
| Pipeline Exposure Vector | Root Cause | Primary Failure Mode |
|---|---|---|
| Git History Retention | Committing keys to repository tracking | Secrets remain readable across clones even after HEAD deletion |
| Broad Stage Scoping | Shared environment variables across all jobs | Test scripts inherit production deployment privileges |
| Unscoped Cloud IAM | Wildcard permission grants on pipeline roles | Breached runners allow lateral cloud account compromise |
| Console Logging | Verbose build script output | Plaintext credentials echo into pipeline run records |
How Do You Audit and Discover Non-Human Identities in CI/CD Workflows?
Auditing non-human identities in CI/CD workflows requires scanning both code history and active pipeline execution paths to identify every automation credential in use. As outlined in DevOps incident response guidelines documented by beefed.ai, the initial phase requires running automated scans across main and active feature branches using open-source scanners like Gitleaks and TruffleHog. Running these scanners identifies existing keys, database strings, and API tokens embedded across historical commits.
To audit code history for leaked credentials, execute a baseline Gitleaks detection scan across the entire repository:
gitleaks detect --source . --report-path gitleaks-report.json
This command evaluates git commits against signature patterns, outputting identified tokens, file locations, and commit hashes into a structured JSON report. However, static repository scanning only inspects static strings; it fails to track non-human identities that are generated dynamically or assigned broad privileges inside cloud platforms.
To achieve complete visibility, organizations use Hush Security to discover, attribute, govern, delegate, and audit non-human identities. Operating across pipeline workloads and agent interactions, Hush Security provides runtime discovery of non-human identities, ensuring every machine actor is identified, attributed to a specific owner, and monitored across every downstream call. Platform teams must evaluate their non-human identity inventory against four criteria:
- Credential Lifespan: Identify whether the credential is static (such as a long-lived cloud access key) or dynamic (such as an ephemeral STS token).
- Execution Scope: Determine whether the identity is scoped strictly to a single repository branch or shared across multiple development teams.
- Privilege Granularity: Verify if the identity uses least-privilege role definitions or wide wildcard administrator policies.
- Auditability: Confirm that every pipeline authentication event produces an immutable audit record in central security logs.
How Does Workload Identity Federation Replace Static Pipeline Secrets?
Workload identity federation replaces static pipeline secrets by allowing the CI/CD platform to mint short-lived OpenID Connect (OIDC) identity tokens that external clouds exchange for temporary access credentials. As explained by beefed.ai in their DevOps credential architecture guide, platforms like GitHub Actions and GitLab CI generate a cryptographically signed JSON Web Token (JWT) at job execution. Cloud providers like AWS, GCP, and Azure validate this token against their native IAM identity providers without requiring pre-shared static keys.
Federation eliminates the traditional “secret zero” bootstrap problem, in which an automation system requires a persistent secret simply to authenticate against a secret vault. Because the cloud provider verifies the authenticity of the token using the CI/CD platform’s public signing keys, no secret is ever stored within the pipeline configuration.
+--------------------------------------------------------------------------------+
| WORKLOAD IDENTITY FEDERATION FLOW |
| |
| +-------------------+ 1. Mint OIDC JWT +----------------------+ |
| | CI/CD Runner | ---------------------------> | GitHub / GitLab IDP | |
| | (Job Environment) | <--------------------------- | (Signs Token) | |
| +-------------------+ 2. Signed Token +----------------------+ |
| | |
| | 3. AssumeRoleWithWebIdentity(Token) |
| v |
| +-------------------+ 4. Validate Claims +----------------------+ |
| | Cloud Provider | <--------------------------- | Cloud IAM Trust | |
| | (AWS / GCP / Az) | ---------------------------> | Policy | |
| +-------------------+ 5. Ephemeral STS +----------------------+ |
| | Credentials |
| v |
| +-------------------+ |
| | Target Resources | (Job runs with temporary, scoped access) |
| +-------------------+ |
+--------------------------------------------------------------------------------+
While workload federation removes static credentials, implementing raw OIDC across dozens of microservices can lead to fragmented trust policies and policy sprawl. Each repository requires distinct IAM role trust boundaries, claims mapping, and environment restrictions. Without centralized governance, platform engineering teams struggle to track which repositories maintain trust relationships with production cloud accounts, creating an unmonitored management layer.
What Is the Difference Between Stored Secrets, Vaults, and Identity-Based Access?
Organizations handling CI/CD automation typically manage machine credentials through one of three architectural patterns: static stored secrets, centralized secrets vaults, or identity-based access platforms. Each model involves clear tradeoffs regarding operational complexity, rotation burden, and exposure risk.
+-----------------------+-----------------------+-----------------------------+
| Stored Pipeline Keys | Secrets Vaults | Identity-Based Access |
+-----------------------+-----------------------+-----------------------------+
| Static long-lived | Central storage; | No static secrets or keys; |
| secrets in repo vars; | short-lived leases; | short-lived, just-in-time |
| manual rotation; high | requires auth tokens; | runtime authorization; full |
| exfiltration blast | complex operations | non-human identity audit |
| radius. | and maintenance. | visibility. |
+-----------------------+-----------------------+-----------------------------+
Stored Pipeline Variables
Storing static tokens in repository settings or CI tool variables is the legacy approach. While simple to configure, static tokens create significant operational overhead and security debt. These credentials rarely rotate automatically, are frequently echoed into build outputs, and offer no granular scoping. A compromise of the CI variable store exposes long-lived production access.
Centralized Secrets Vaults
Centralized vaults like HashiCorp Vault address static storage by offering dynamic secrets engines that generate temporary database users or cloud credentials with explicit time-to-live leases, as described by beefed.ai. GitLab CI supports direct integration via secrets:vault stanzas combined with id_tokens to fetch credentials dynamically at job start. While vaults isolate secrets from source code, they introduce significant operational burdens: platform teams must deploy, unseal, monitor, and scale highly available vault clusters, while maintaining pipeline integration agents and managing bootstrap authentication tokens.
Identity-Based Access for Non-Human Identities
Identity-based access platforms eliminate stored secrets, static keys, and secret vaults entirely by authenticating workloads based on verified runtime context. Through Hush Security, organizations adopt identity-based access for non-human identities, enabling just-in-time, scoped access with full auditability. Shifting to identity-based access eliminates hardcoded, static, and long-lived access across the technology stack, significantly reducing the operational maintenance overhead typically borne by DevOps and SecOps teams.
Same styling, adjusted for four columns (first column a bit narrower, the other three equal):
“`html
| Evaluation Metric | Stored Pipeline Secrets | Centralized Secrets Vault | Identity-Based Access (Hush) |
|---|---|---|---|
| Credential Format | Static, long-lived API keys | Dynamic, leased secrets | Ephemeral, just-in-time authorization |
| Storage Overhead | Repository variable stores | Dedicated vault infrastructure | Secretless architecture (no vaults) |
| Rotation Burden | Manual or scheduled script | Automated lease expiration | Fully automatic per-action evaluation |
| Bootstrap Secret Required | Yes (stored in pipeline) | Yes (token or app role ID) | No (cryptographic identity token) |
| Non-Human Identity Audit | Fragmented CI logs | Vault access audit logs | Comprehensive end-to-end outcome audit |
How Do You Configure Ephemeral, Least-Privilege Access for Cloud Deployments?
Configuring ephemeral, least-privilege access requires establishing cryptographic trust between your CI/CD platform and cloud IAM, restricting permissions to distinct deployment environments, and purging exposed legacy credentials.
Step 1: Revoke and Purge Compromised Repository Credentials
When removing credentials from a pipeline, assume any token previously committed or stored in pipeline variables has been compromised. Immediate revocation at the issuing provider is mandatory. After revoking the active token, purge the string from your repository’s git history using git-filter-repo, as documented by beefed.ai:
echo 'AKIAIOSFODNN7EXAMPLE' > secrets-to-remove.txt
git filter-repo --replace-text secrets-to-remove.txt
git push --force --mirror origin
Check before moving on: Run a fresh gitleaks detect --source . scan to verify that zero instances of the compromised string remain in branch histories or tag references.
Step 2: Establish OIDC Trust in Cloud IAM
Configure an OpenID Connect identity provider within your cloud account pointing to your CI provider (such as GitHub Actions at https://token.actions.githubusercontent.com). Create an IAM role with a trust policy that restricts assumption to your exact organization, repository, and branch:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
},
"StringLike": {
"token.actions.githubusercontent.com:sub": "repo:example-org/production-service:ref:refs/heads/main"
}
}
}
]
}
Check before moving on: Verify that the condition blocks assumption attempts from feature branches or outside repositories.
Step 3: Implement Least-Privilege Environment Scoping
Adhere to the least-privilege principles detailed by Khimananda Oli. Scope deployment roles strictly to individual environments such as staging or production, and grant read-only or targeted write permissions rather than wildcard policies. Furthermore, do not pass secrets directly between pipeline jobs. Jobs should fetch their own scoped credentials independently, or encrypt intermediate artifacts with ephemeral keys if data transfer between stages cannot be avoided.
In your pipeline definition (such as a GitHub Actions workflow), request OIDC token minting permissions and assume the role dynamically at runtime:
name: Deploy Production Workload
on:
push:
branches:
- main
permissions:
id-token: write
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
environment: production
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Configure AWS Credentials via OIDC
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/cicd-production-deployer
aws-region: us-east-1
- name: Deploy Infrastructure
run: |
aws s3 sync ./build s3://production-workload-assets-bucket/
Check before moving on: Execute the pipeline and confirm in cloud audit logs that authentication succeeded using temporary session credentials without referencing long-lived access keys.
Step 4: Audit IAM Policies and Environment Protections
As advised by Khimananda Oli, configure environment protections requiring manual approvals or passing status checks before release workflows can access deployment roles. Additionally, schedule quarterly IAM policy reviews to eliminate stale role bindings, inspect trust policies for over-permissive wildcards, and continuously reduce the attack surface across your automation identities.
What Are the Phased Milestones to Achieve Secretless CI/CD?
Eliminating secrets across an enterprise delivery pipeline cannot happen in an unmanaged, single-day rollout. Attempting an immediate cutover across all repositories disrupts deployment operations. Instead, engineering teams should follow a structured, phased implementation roadmap based on identity maturity.
+--------------------------------------------------------------------------------+
| SECRETLESS CI/CD IMPLEMENTATION ROADMAP |
| |
| Phase 1: Discovery & Triage (Days 1-5) |
| + Run history scans with Gitleaks & TruffleHog |
| + Map all non-human identities and cloud permissions |
| + Identify hardcoded keys and plaintext environment variables |
| |
| Phase 2: Remediation & Git Scrubbing (Days 6-12) |
| + Revoke all detected static keys at the issuing service |
| + Rewrite git history with git-filter-repo and force-push |
| + Add pre-commit hooks and PR scanner blocking gates |
| |
| Phase 3: Workload Identity Federation (Days 13-20) |
| + Configure OIDC trust providers in cloud IAM |
| + Migrate workflow definitions to ephemeral role assumption |
| + Enforce environment-level protection rules and approval gates |
| |
| Phase 4: Identity-Based Governance (Days 21+) |
| + Decommission static vault credentials and access keys |
| + Implement centralized runtime NHI governance and outcome auditing |
| + Conduct quarterly least-privilege IAM policy reviews |
+--------------------------------------------------------------------------------+
Common Pitfalls to Avoid at Each Step
When transitioning pipelines to secretless authentication, watch for these frequent implementation failures:
- Deleting from HEAD Without History Rewriting: Removing an access key in a new commit leaves the credential exposed across prior git commits. Always purge history using
git-filter-repoand immediately revoke the token. - Permissive OIDC Trust Policy Conditions: Setting the OIDC
subclaim condition to a wildcard (repo:example-org/*:*) allows any repository or fork in the organization to assume production deployment roles. Always lock the subject claim to specific branches and environments. - Reusing Credentials Across Stages: Sharing an authentication token between testing and deployment stages exposes high-privilege credentials to untrusted third-party code. Always re-fetch credentials independently at each job stage.
- Neglecting Non-Human Identity Governance: Implementing OIDC across hundreds of workflows without runtime discovery creates unmonitored identity sprawl. Centralize identity-based governance across your non-human workforce to ensure continuous visibility and auditability.
Resolving Your Next Operational Step
Platform and security engineering teams aiming to eliminate static secrets should begin by running automated baseline discovery scans across their core repositories. After cataloging existing non-human identities and revoking exposed credentials, transition deployment runners to workload identity federation and adopt identity-based access across all automated pipelines.