Developer Pushed AWS Credentials to Public GitHub — Incident Response
A developer accidentally committed live AWS access keys to a public GitHub repository. Walk through the immediate containment, investigation, and prevention strategy.
It's 10:30 AM. Your security monitoring fires: 'GitHub Secret Scanning detected AWS credentials in a public repository.' The keys belong to a CI user with broad S3 and EC2 read/write access. The commit was pushed 47 minutes ago. You don't know if the keys have already been used by someone else.
The Problem
Leaked AWS credentials are one of the most dangerous security incidents. Attackers have automated scrapers that monitor GitHub in real time — your keys can be picked up within minutes of a push. Every second you don’t act is a second an attacker could be spinning up crypto miners, exfiltrating data, or creating persistence backdoors.
Step 1: Revoke the Keys Immediately (First 2 Minutes)
Do not investigate first. Revoke first.
# Deactivate the key (reversible — do this first)
aws iam update-access-key \
--access-key-id AKIAIOSFODNN7EXAMPLE \
--status Inactive \
--user-name ci-deploy-user
# Then delete it permanently
aws iam delete-access-key \
--access-key-id AKIAIOSFODNN7EXAMPLE \
--user-name ci-deploy-user
If the compromised credential was an IAM role (not a user), issue a policy that denies everything immediately:
aws iam put-role-policy \
--role-name compromised-ci-role \
--policy-name EmergencyDenyAll \
--policy-document '{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny",
"Action": "*",
"Resource": "*"
}]
}'
Step 2: Check CloudTrail for Unauthorized Usage
Now investigate what happened during the 47-minute window:
# Look up all API calls made with this access key
aws cloudtrail lookup-events \
--lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIAIOSFODNN7EXAMPLE \
--start-time "2024-01-15T09:43:00Z" \
--end-time "2024-01-15T10:30:00Z" \
--query 'Events[*].{Time:EventTime,Event:EventName,IP:CloudTrailEvent}' \
--output table
Look for these red flags in the output:
| Suspicious Action | What It Means |
|---|---|
RunInstances | Attacker spun up EC2 (likely crypto miner) |
CreateUser / CreateAccessKey | Attacker created persistence |
GetObject on sensitive buckets | Data exfiltration |
AssumeRole | Lateral movement to other roles |
| Requests from unusual IPs/regions | Clear attacker activity |
# Pull the raw CloudTrail event to see the source IP
aws cloudtrail lookup-events \
--lookup-attributes AttributeKey=AccessKeyId,AttributeValue=AKIAIOSFODNN7EXAMPLE \
--query 'Events[*].CloudTrailEvent' | python3 -c "
import json, sys
for line in json.load(sys.stdin):
event = json.loads(line)
print(event.get('sourceIPAddress'), event.get('eventName'), event.get('awsRegion'))
"
Step 3: Assess Blast Radius
Based on what CloudTrail shows, determine the scope:
# If RunInstances was called — find and terminate attacker's instances
aws ec2 describe-instances \
--filters "Name=tag:CreatedBy,Values=AKIAIOSFODNN7EXAMPLE" \
--query 'Reservations[*].Instances[*].InstanceId'
# Check all regions — attackers always use unexpected regions
for region in $(aws ec2 describe-regions --query 'Regions[*].RegionName' --output text); do
echo "Checking $region..."
aws ec2 describe-instances --region $region \
--filters "Name=instance-state-name,Values=running" \
--query 'Reservations[*].Instances[*].{ID:InstanceId,Region:Placement.AvailabilityZone,Launch:LaunchTime}' \
--output table
done
# Check for new IAM users or keys created during the window
aws iam list-users --query 'Users[?CreateDate>=`2024-01-15T09:43:00Z`]'
aws iam list-access-keys --query 'AccessKeyMetadata[?CreateDate>=`2024-01-15T09:43:00Z`]'
Step 4: Check GuardDuty Findings
If GuardDuty is enabled, it will have already flagged anomalous activity:
aws guardduty list-findings \
--detector-id $(aws guardduty list-detectors --query 'DetectorIds[0]' --output text) \
--finding-criteria '{
"Criterion": {
"updatedAt": {
"GreaterThanOrEqual": 1705312980000
}
}
}'
GuardDuty finding types to look for:
UnauthorizedAccess:IAMUser/MaliciousIPCallerUnauthorizedAccess:IAMUser/TorIPCallerRecon:IAMUser/MaliciousIPCallerCryptoCurrency:EC2/BitcoinTool.B
Step 5: Rotate Everything That Was Accessible
Even if you find no evidence of attacker access, rotate defensively:
# Rotate the CI user's credentials (generate new key for pipeline)
aws iam create-access-key --user-name ci-deploy-user
# If CI user had access to RDS — rotate the DB password
aws secretsmanager rotate-secret \
--secret-id prod/rds/ci-user-password \
--force-delete-without-recovery
Update your CI/CD pipeline (GitHub Actions secrets, etc.) with the new credentials — and this time, use OIDC federation instead of long-lived keys:
# GitHub Actions — OIDC (no stored credentials at all)
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::123456789012:role/github-ci-role
aws-region: us-east-1
Step 6: Prevention Strategy
| Layer | Tool | What It Does |
|---|---|---|
| Developer Machine | detect-secrets pre-commit hook | Blocks the commit before it happens |
| CI Pipeline | truffleHog / GitGuardian | Scans every PR for secrets |
| Repository | GitHub Secret Scanning | Auto-revokes detected AWS keys |
| AWS Native | IAM Access Analyzer | Flags overly permissive policies |
| Monitoring | GuardDuty | Detects anomalous API activity in real time |
| Architecture | OIDC instead of keys | Eliminates stored credentials entirely |
Set up the pre-commit hook:
pip install detect-secrets
detect-secrets scan > .secrets.baseline
# .pre-commit-config.yaml
repos:
- repo: https://github.com/Yelp/detect-secrets
rev: v1.4.0
hooks:
- id: detect-secrets
args: ['--baseline', '.secrets.baseline']
Post-Incident Report Template
Incident: Leaked AWS Credentials
Date: [date]
Duration: 47 minutes (push to revocation)
Severity: High
Timeline:
09:43 - Keys pushed to public repo
10:30 - GitHub Secret Scanning alert triggered
10:32 - Keys revoked
10:45 - CloudTrail investigation complete (no attacker usage found)
Root Cause: Developer hardcoded credentials in .env file committed to repo
Corrective Actions:
1. All CI pipelines migrated to OIDC (due: this sprint)
2. detect-secrets pre-commit hook added to all repos (due: this week)
3. GuardDuty real-time alert to PagerDuty configured (complete)
4. IAM Access Analyzer enabled on all accounts (complete)
- The exact sequence of actions to take in the first 15 minutes
- How to use CloudTrail to determine if keys were used by an attacker
- How to assess blast radius from unauthorized API calls
- A layered prevention strategy using pre-commit hooks, CI scanning, and native AWS tools
Have a similar scenario to share?
Production incidents are the best teachers. Submit your real-world scenario and help others learn.
Open Google FormRelated Scenarios
EC2 Instance Communicating with Malicious IP — Incident Response
The Problem A running EC2 instance may be compromised — malware installed, data being exfiltrated, or the instance being used as a pivot to …
AWS Cloud Engineer Learning Path
Who Is This Path For? This path is designed for complete beginners who want to break into cloud computing as an AWS engineer. If you know …
AWS Cloud Foundations — Fresher Learning Path
How to Use This Path Each section below shows an AWS architecture diagram. Click any coloured block to see: