The Situation
FinOps automation — pull Cost Explorer data and recommendations programmatically to build a monthly savings report instead of logging into the console.
Problem Statement
Your AWS bill jumped 40% last month. Engineering lead asks: “What are we spending on EC2, and what can we do about it?” Rather than clicking through the Cost Explorer console, this script pulls the data and recommendations programmatically — ready to be emailed, Slacked, or fed into a dashboard.
Cost Explorer Key Concepts
| Term | Meaning |
|---|
| On-Demand | Pay-as-you-go — highest rate, no commitment |
| Savings Plan (Compute) | 1 or 3-year hourly spend commitment — up to 66% savings, flexible across instance types |
| Reserved Instance (RI) | 1 or 3-year instance commitment — up to 72% savings, specific instance type |
| Blended Cost | Average cost across on-demand and committed spend |
Complete Script
import boto3
from datetime import datetime, timedelta
import json
def analyze_and_recommend_savings(lookback_days: int = 30) -> dict:
"""
Analyses EC2 on-demand costs and surfaces AWS-generated
Savings Plan and Reserved Instance recommendations.
Cost Explorer is a GLOBAL service — the client always uses
region "us-east-1" regardless of where your resources are.
TimePeriod dates must be ISO 8601 format: "YYYY-MM-DD".
End date is exclusive (same as Python range() semantics).
"""
# Cost Explorer is only available in us-east-1
ce = boto3.client("ce", region_name="us-east-1")
end = datetime.utcnow().strftime("%Y-%m-%d")
start = (datetime.utcnow() - timedelta(days=lookback_days)).strftime("%Y-%m-%d")
print(f"Analysing EC2 costs from {start} to {end}...\n")
# ── Step 1: Current on-demand EC2 spend by instance type ──────
# get_cost_and_usage() is the primary Cost Explorer API.
#
# Filter: We want only EC2 compute on-demand charges.
# The "And" operator requires ALL nested conditions to match.
# SERVICE dimension filters by the AWS service name.
# PURCHASE_TYPE dimension filters by pricing model.
#
# Metrics: "BlendedCost" is the dollar amount.
# "UsageQuantity" is hours of usage.
#
# GroupBy: Breaks the cost by INSTANCE_TYPE dimension so we can
# see which instance types are driving the spend.
cost_response = ce.get_cost_and_usage(
TimePeriod={"Start": start, "End": end},
Granularity="MONTHLY",
Filter={
"And": [
{
"Dimensions": {
"Key": "SERVICE",
"Values": ["Amazon Elastic Compute Cloud - Compute"],
}
},
{
"Dimensions": {
"Key": "PURCHASE_TYPE",
"Values": ["On-Demand"],
}
},
]
},
Metrics=["BlendedCost", "UsageQuantity"],
GroupBy=[{"Type": "DIMENSION", "Key": "INSTANCE_TYPE"}],
)
# Aggregate costs across months (in case lookback spans multiple months)
instance_costs: dict[str, float] = {}
for result in cost_response["ResultsByTime"]:
for group in result["Groups"]:
instance_type = group["Keys"][0]
cost = float(group["Metrics"]["BlendedCost"]["Amount"])
instance_costs[instance_type] = instance_costs.get(instance_type, 0) + cost
# ── Step 2: Savings Plans recommendation ──────────────────────
# get_savings_plans_purchase_recommendation() calls AWS's recommendation
# engine which analyses your historical usage and computes the optimal
# hourly commitment to maximise savings.
#
# SavingsPlansType options:
# "COMPUTE_SP" — most flexible, applies to any EC2, Lambda, Fargate
# "EC2_INSTANCE_SP" — higher savings, locked to a region/instance family
#
# TermInYears: "ONE_YEAR" or "THREE_YEARS"
# PaymentOption: "NO_UPFRONT", "PARTIAL_UPFRONT", "ALL_UPFRONT"
sp_response = ce.get_savings_plans_purchase_recommendation(
SavingsPlansType="COMPUTE_SP",
TermInYears="ONE_YEAR",
PaymentOption="NO_UPFRONT",
LookbackPeriodInDays="THIRTY_DAYS",
)
sp_summary = sp_response.get(
"SavingsPlansPurchaseRecommendation", {}
).get("SavingsPlansPurchaseRecommendationSummary", {})
# ── Step 3: Reserved Instance recommendation ──────────────────
# get_reservation_purchase_recommendation() gives per-instance-type
# RI recommendations — more specific than Savings Plans but higher savings.
ri_response = ce.get_reservation_purchase_recommendation(
Service="Amazon EC2",
TermInYears="ONE_YEAR",
PaymentOption="NO_UPFRONT",
LookbackPeriodInDays="THIRTY_DAYS",
)
ri_recommendations = ri_response.get("Recommendations", [])
# ── Print report ──────────────────────────────────────────────
total_cost = sum(instance_costs.values())
print("=" * 60)
print("EC2 ON-DEMAND SPEND BY INSTANCE TYPE (Last 30 Days)")
print("=" * 60)
for itype, cost in sorted(instance_costs.items(), key=lambda x: -x[1]):
bar = "█" * min(int(cost / total_cost * 30), 30)
print(f" {itype:<20} ${cost:>10.2f} {bar}")
print(f"\n {'TOTAL ON-DEMAND':<20} ${total_cost:>10.2f}")
print("\n" + "=" * 60)
print("SAVINGS PLAN RECOMMENDATION (Compute SP, 1-Year, No Upfront)")
print("=" * 60)
if sp_summary:
monthly_savings = float(sp_summary.get("EstimatedMonthlySavingsAmount", 0))
savings_pct = sp_summary.get("EstimatedSavingsPercentage", "N/A")
hourly_commit = float(sp_summary.get("HourlyCommitmentToPurchase", 0))
current_spend = float(sp_summary.get("CurrentOnDemandSpend", 0))
print(f" Current On-Demand Spend: ${current_spend:>10,.2f}")
print(f" Recommended Hourly Commit: ${hourly_commit:>10.4f}/hr")
print(f" Estimated Monthly Savings: ${monthly_savings:>10,.2f}")
print(f" Savings Percentage: {savings_pct}%")
print(f" Annual Savings Estimate: ${monthly_savings * 12:>10,.2f}")
else:
print(" No Savings Plan recommendation available (need 30+ days of data)")
print("\n" + "=" * 60)
print("RESERVED INSTANCE RECOMMENDATIONS (Top 5, 1-Year, No Upfront)")
print("=" * 60)
for i, rec in enumerate(ri_recommendations[:5], 1):
details = rec.get("RecommendationDetails", [{}])[0]
spec = details.get("InstanceDetails", {}).get("EC2InstanceDetails", {})
print(f"\n [{i}] {spec.get('InstanceType', 'N/A')} in {spec.get('Region', 'N/A')}")
print(f" Platform: {spec.get('Platform', 'Linux')}")
print(f" Recommended: {details.get('RecommendedNumberOfInstancesToPurchase', 'N/A')} instances")
print(f" Monthly Save: ${float(details.get('EstimatedMonthlySavings', 0)):,.2f}")
print(f" Upfront Cost: ${float(details.get('UpfrontCost', 0)):,.2f}")
result = {
"period": {"start": start, "end": end},
"total_on_demand_usd": round(total_cost, 2),
"instance_breakdown": {k: round(v, 2) for k, v in instance_costs.items()},
"savings_plan": {
"monthly_savings_usd": round(float(sp_summary.get("EstimatedMonthlySavingsAmount", 0)), 2),
"savings_pct": sp_summary.get("EstimatedSavingsPercentage", "N/A"),
"hourly_commitment": round(float(sp_summary.get("HourlyCommitmentToPurchase", 0)), 4),
} if sp_summary else {},
}
# Save JSON report
with open("cost_report.json", "w") as f:
json.dump(result, f, indent=2)
print("\nFull report saved to cost_report.json")
return result
if __name__ == "__main__":
analyze_and_recommend_savings(lookback_days=30)
Key Commands Explained
| Command | What it does |
|---|
ce.get_cost_and_usage(TimePeriod, Granularity, Filter, Metrics, GroupBy) | Returns cost data broken down by any dimension |
Granularity="MONTHLY" | Aggregates data per calendar month |
Filter["And"] | Both conditions must match (service = EC2 AND purchase type = on-demand) |
GroupBy=[{"Type": "DIMENSION", "Key": "INSTANCE_TYPE"}] | Breaks results by instance type |
group["Metrics"]["BlendedCost"]["Amount"] | Cost as a string — cast to float() |
get_savings_plans_purchase_recommendation(SavingsPlansType, TermInYears, PaymentOption) | AWS-generated SP recommendation |
get_reservation_purchase_recommendation(Service, TermInYears, PaymentOption) | AWS-generated RI recommendation |
LookbackPeriodInDays="THIRTY_DAYS" | How much history the recommendation engine uses |
Common Issues
DataUnavailableException — Cost Explorer needs at least 14 days of usage data before it can generate recommendations. Enable Cost Explorer in the billing console and wait.
Empty recommendations — On-demand spend must be significant enough for AWS to recommend commitments. Very small accounts may not get RI recommendations.
Region must be us-east-1 — Cost Explorer API is only available in us-east-1 regardless of where your resources are deployed.
🔍 Line-by-Line Code Walkthrough
Imports
| Line | Why It’s Used |
|---|
import boto3 | AWS SDK — needed for the Cost Explorer client |
from datetime import datetime, timedelta | datetime.utcnow() for the current date, timedelta(days=30) to compute the lookback start date |
import json | Saves the final report as a JSON file |
Cost Explorer Client
ce = boto3.client("ce", region_name="us-east-1")
| Line | Explanation |
|---|
boto3.client("ce", ...) | "ce" is the service name for AWS Cost Explorer |
region_name="us-east-1" | Cost Explorer is a global service but the API endpoint only exists in us-east-1. You MUST hardcode this, regardless of which region your resources are in |
Date Setup
end = datetime.utcnow().strftime("%Y-%m-%d")
start = (datetime.utcnow() - timedelta(days=lookback_days)).strftime("%Y-%m-%d")
| Line | Explanation |
|---|
datetime.utcnow() | Current UTC date as a naive datetime |
.strftime("%Y-%m-%d") | Formats as "2025-01-20" — the ISO 8601 format required by Cost Explorer. The API rejects other formats |
timedelta(days=lookback_days) | Subtracting 30 days from today gives the start of the analysis window |
| End date is exclusive | Like Python’s range(), the end date is NOT included. To see data through Jan 20, set end to Jan 21 |
get_cost_and_usage(...)
cost_response = ce.get_cost_and_usage(
TimePeriod={"Start": start, "End": end},
Granularity="MONTHLY",
Filter={
"And": [
{"Dimensions": {"Key": "SERVICE", "Values": ["Amazon Elastic Compute Cloud - Compute"]}},
{"Dimensions": {"Key": "PURCHASE_TYPE", "Values": ["On-Demand"]}},
]
},
Metrics=["BlendedCost", "UsageQuantity"],
GroupBy=[{"Type": "DIMENSION", "Key": "INSTANCE_TYPE"}],
)
| Parameter | Explanation |
|---|
TimePeriod | Date range. Both Start and End are in "YYYY-MM-DD" format |
Granularity="MONTHLY" | Aggregates data by calendar month. Other options: "DAILY" or "HOURLY" (hourly only available for last 14 days) |
Filter["And"] | Logical AND — all nested conditions must match. Here: service must be EC2 AND purchase type must be On-Demand |
"Key": "SERVICE" | Filter by AWS service name. "Amazon Elastic Compute Cloud - Compute" is the exact service name for EC2 instances |
"Key": "PURCHASE_TYPE" | Filter to "On-Demand" only — excludes Savings Plan and Reserved Instance charges |
Metrics=["BlendedCost", "UsageQuantity"] | Which numbers to return. BlendedCost = dollar amount. UsageQuantity = hours of usage. You can request multiple metrics |
GroupBy=[{"Type": "DIMENSION", "Key": "INSTANCE_TYPE"}] | Splits the result by instance type (e.g., t3.medium, c5.xlarge). Without this, you’d get one total number |
for result in cost_response["ResultsByTime"]:
for group in result["Groups"]:
instance_type = group["Keys"][0]
cost = float(group["Metrics"]["BlendedCost"]["Amount"])
instance_costs[instance_type] = instance_costs.get(instance_type, 0) + cost
| Line | Explanation |
|---|
cost_response["ResultsByTime"] | List of one dict per time period (one per month when Granularity=MONTHLY) |
result["Groups"] | List of groups — one per instance type for this month |
group["Keys"][0] | The first key in the GroupBy key list. Since we group by INSTANCE_TYPE, Keys[0] is the instance type string (e.g., "t3.medium") |
group["Metrics"]["BlendedCost"]["Amount"] | The cost as a string (e.g., "123.456789"). Must be cast to float() for arithmetic |
instance_costs.get(instance_type, 0) + cost | Accumulates costs across multiple months. .get(key, 0) returns 0 if the key doesn’t exist yet |
Savings Plan Recommendation
sp_response = ce.get_savings_plans_purchase_recommendation(
SavingsPlansType="COMPUTE_SP",
TermInYears="ONE_YEAR",
PaymentOption="NO_UPFRONT",
LookbackPeriodInDays="THIRTY_DAYS",
)
| Parameter | Explanation |
|---|
SavingsPlansType="COMPUTE_SP" | Compute Savings Plan — the most flexible type. Applies to EC2 (any instance type, any region), Lambda, and Fargate. Alternative: "EC2_INSTANCE_SP" (higher savings but less flexible) |
TermInYears="ONE_YEAR" | 1-year commitment. "THREE_YEARS" offers higher discounts but more risk |
PaymentOption="NO_UPFRONT" | Pay monthly with no upfront cost. "ALL_UPFRONT" saves more but requires capital. "PARTIAL_UPFRONT" is the middle ground |
LookbackPeriodInDays="THIRTY_DAYS" | How much historical usage data the recommendation engine analyses. "SEVEN_DAYS" and "SIXTY_DAYS" are also valid |
sp_summary = sp_response.get(
"SavingsPlansPurchaseRecommendation", {}
).get("SavingsPlansPurchaseRecommendationSummary", {})
| Line | Explanation |
|---|
.get("SavingsPlansPurchaseRecommendation", {}) | Returns an empty dict if no recommendation exists (insufficient data). Using .get() with a default prevents KeyError |
.get("SavingsPlansPurchaseRecommendationSummary", {}) | The nested summary dict containing EstimatedMonthlySavingsAmount, HourlyCommitmentToPurchase, etc. |
Report Printing
for itype, cost in sorted(instance_costs.items(), key=lambda x: -x[1]):
bar = "█" * min(int(cost / total_cost * 30), 30)
print(f" {itype:<20} ${cost:>10.2f} {bar}")
| Line | Explanation |
|---|
sorted(..., key=lambda x: -x[1]) | Sorts by cost descending (highest spend first). x[1] is the cost value; negating it reverses the sort order |
cost / total_cost * 30 | Computes the proportional bar length: 30 characters = 100% of total cost |
min(..., 30) | Caps the bar at 30 characters to prevent line overflow |
int(...) | "█" * 12.7 is invalid — we need an integer |
{itype:<20} | Left-aligned string padded to 20 characters. Makes columns line up |
{cost:>10.2f} | Right-aligned float with 2 decimal places, padded to 10 chars |
Reserved Instance Recommendation
ri_response = ce.get_reservation_purchase_recommendation(
Service="Amazon EC2",
TermInYears="ONE_YEAR",
PaymentOption="NO_UPFRONT",
LookbackPeriodInDays="THIRTY_DAYS",
)
| Parameter | Explanation |
|---|
Service="Amazon EC2" | Which service’s RIs to recommend. Other values: "Amazon RDS", "Amazon ElastiCache" |
| Note the difference | RI recommendations are per specific instance type in a specific region. SP recommendations are for a flexible hourly spend commitment |
ri_response["Recommendations"] | List of RI purchase recommendations, each with details about instance type, count, and savings |