Scenario Advanced Python Python AWS Scripting

EC2 Cost Analysis & Savings Plan Recommendations with Cost Explorer

Python script to analyse EC2 on-demand spend via AWS Cost Explorer and surface Savings Plan and Reserved Instance purchase recommendations with estimated savings.

January 20, 2025 8 min read ~20 min to complete DB
The Situation

FinOps automation — pull Cost Explorer data and recommendations programmatically to build a monthly savings report instead of logging into the console.

5 Steps
3 Services Used
~20 min Duration
Advanced Difficulty

Problem Statement

Your AWS bill jumped 40% last month. Engineering lead asks: “What are we spending on EC2, and what can we do about it?” Rather than clicking through the Cost Explorer console, this script pulls the data and recommendations programmatically — ready to be emailed, Slacked, or fed into a dashboard.


Cost Explorer Key Concepts

TermMeaning
On-DemandPay-as-you-go — highest rate, no commitment
Savings Plan (Compute)1 or 3-year hourly spend commitment — up to 66% savings, flexible across instance types
Reserved Instance (RI)1 or 3-year instance commitment — up to 72% savings, specific instance type
Blended CostAverage cost across on-demand and committed spend

Complete Script

import boto3
from datetime import datetime, timedelta
import json


def analyze_and_recommend_savings(lookback_days: int = 30) -> dict:
    """
    Analyses EC2 on-demand costs and surfaces AWS-generated
    Savings Plan and Reserved Instance recommendations.

    Cost Explorer is a GLOBAL service — the client always uses
    region "us-east-1" regardless of where your resources are.

    TimePeriod dates must be ISO 8601 format: "YYYY-MM-DD".
    End date is exclusive (same as Python range() semantics).
    """
    # Cost Explorer is only available in us-east-1
    ce = boto3.client("ce", region_name="us-east-1")

    end   = datetime.utcnow().strftime("%Y-%m-%d")
    start = (datetime.utcnow() - timedelta(days=lookback_days)).strftime("%Y-%m-%d")

    print(f"Analysing EC2 costs from {start} to {end}...\n")

    # ── Step 1: Current on-demand EC2 spend by instance type ──────
    # get_cost_and_usage() is the primary Cost Explorer API.
    #
    # Filter: We want only EC2 compute on-demand charges.
    # The "And" operator requires ALL nested conditions to match.
    # SERVICE dimension filters by the AWS service name.
    # PURCHASE_TYPE dimension filters by pricing model.
    #
    # Metrics: "BlendedCost" is the dollar amount.
    #          "UsageQuantity" is hours of usage.
    #
    # GroupBy: Breaks the cost by INSTANCE_TYPE dimension so we can
    # see which instance types are driving the spend.
    cost_response = ce.get_cost_and_usage(
        TimePeriod={"Start": start, "End": end},
        Granularity="MONTHLY",
        Filter={
            "And": [
                {
                    "Dimensions": {
                        "Key":    "SERVICE",
                        "Values": ["Amazon Elastic Compute Cloud - Compute"],
                    }
                },
                {
                    "Dimensions": {
                        "Key":    "PURCHASE_TYPE",
                        "Values": ["On-Demand"],
                    }
                },
            ]
        },
        Metrics=["BlendedCost", "UsageQuantity"],
        GroupBy=[{"Type": "DIMENSION", "Key": "INSTANCE_TYPE"}],
    )

    # Aggregate costs across months (in case lookback spans multiple months)
    instance_costs: dict[str, float] = {}
    for result in cost_response["ResultsByTime"]:
        for group in result["Groups"]:
            instance_type = group["Keys"][0]
            cost = float(group["Metrics"]["BlendedCost"]["Amount"])
            instance_costs[instance_type] = instance_costs.get(instance_type, 0) + cost

    # ── Step 2: Savings Plans recommendation ──────────────────────
    # get_savings_plans_purchase_recommendation() calls AWS's recommendation
    # engine which analyses your historical usage and computes the optimal
    # hourly commitment to maximise savings.
    #
    # SavingsPlansType options:
    #   "COMPUTE_SP" — most flexible, applies to any EC2, Lambda, Fargate
    #   "EC2_INSTANCE_SP" — higher savings, locked to a region/instance family
    #
    # TermInYears: "ONE_YEAR" or "THREE_YEARS"
    # PaymentOption: "NO_UPFRONT", "PARTIAL_UPFRONT", "ALL_UPFRONT"
    sp_response = ce.get_savings_plans_purchase_recommendation(
        SavingsPlansType="COMPUTE_SP",
        TermInYears="ONE_YEAR",
        PaymentOption="NO_UPFRONT",
        LookbackPeriodInDays="THIRTY_DAYS",
    )
    sp_summary = sp_response.get(
        "SavingsPlansPurchaseRecommendation", {}
    ).get("SavingsPlansPurchaseRecommendationSummary", {})

    # ── Step 3: Reserved Instance recommendation ──────────────────
    # get_reservation_purchase_recommendation() gives per-instance-type
    # RI recommendations — more specific than Savings Plans but higher savings.
    ri_response = ce.get_reservation_purchase_recommendation(
        Service="Amazon EC2",
        TermInYears="ONE_YEAR",
        PaymentOption="NO_UPFRONT",
        LookbackPeriodInDays="THIRTY_DAYS",
    )
    ri_recommendations = ri_response.get("Recommendations", [])

    # ── Print report ──────────────────────────────────────────────
    total_cost = sum(instance_costs.values())

    print("=" * 60)
    print("EC2 ON-DEMAND SPEND BY INSTANCE TYPE (Last 30 Days)")
    print("=" * 60)
    for itype, cost in sorted(instance_costs.items(), key=lambda x: -x[1]):
        bar = "█" * min(int(cost / total_cost * 30), 30)
        print(f"  {itype:<20} ${cost:>10.2f}  {bar}")
    print(f"\n  {'TOTAL ON-DEMAND':<20} ${total_cost:>10.2f}")

    print("\n" + "=" * 60)
    print("SAVINGS PLAN RECOMMENDATION (Compute SP, 1-Year, No Upfront)")
    print("=" * 60)
    if sp_summary:
        monthly_savings  = float(sp_summary.get("EstimatedMonthlySavingsAmount", 0))
        savings_pct      = sp_summary.get("EstimatedSavingsPercentage", "N/A")
        hourly_commit    = float(sp_summary.get("HourlyCommitmentToPurchase", 0))
        current_spend    = float(sp_summary.get("CurrentOnDemandSpend", 0))

        print(f"  Current On-Demand Spend:   ${current_spend:>10,.2f}")
        print(f"  Recommended Hourly Commit: ${hourly_commit:>10.4f}/hr")
        print(f"  Estimated Monthly Savings: ${monthly_savings:>10,.2f}")
        print(f"  Savings Percentage:        {savings_pct}%")
        print(f"  Annual Savings Estimate:   ${monthly_savings * 12:>10,.2f}")
    else:
        print("  No Savings Plan recommendation available (need 30+ days of data)")

    print("\n" + "=" * 60)
    print("RESERVED INSTANCE RECOMMENDATIONS (Top 5, 1-Year, No Upfront)")
    print("=" * 60)
    for i, rec in enumerate(ri_recommendations[:5], 1):
        details = rec.get("RecommendationDetails", [{}])[0]
        spec    = details.get("InstanceDetails", {}).get("EC2InstanceDetails", {})

        print(f"\n  [{i}] {spec.get('InstanceType', 'N/A')} in {spec.get('Region', 'N/A')}")
        print(f"      Platform:     {spec.get('Platform', 'Linux')}")
        print(f"      Recommended:  {details.get('RecommendedNumberOfInstancesToPurchase', 'N/A')} instances")
        print(f"      Monthly Save: ${float(details.get('EstimatedMonthlySavings', 0)):,.2f}")
        print(f"      Upfront Cost: ${float(details.get('UpfrontCost', 0)):,.2f}")

    result = {
        "period":              {"start": start, "end": end},
        "total_on_demand_usd": round(total_cost, 2),
        "instance_breakdown":  {k: round(v, 2) for k, v in instance_costs.items()},
        "savings_plan": {
            "monthly_savings_usd": round(float(sp_summary.get("EstimatedMonthlySavingsAmount", 0)), 2),
            "savings_pct":         sp_summary.get("EstimatedSavingsPercentage", "N/A"),
            "hourly_commitment":   round(float(sp_summary.get("HourlyCommitmentToPurchase", 0)), 4),
        } if sp_summary else {},
    }

    # Save JSON report
    with open("cost_report.json", "w") as f:
        json.dump(result, f, indent=2)
    print("\nFull report saved to cost_report.json")

    return result


if __name__ == "__main__":
    analyze_and_recommend_savings(lookback_days=30)

Key Commands Explained

CommandWhat it does
ce.get_cost_and_usage(TimePeriod, Granularity, Filter, Metrics, GroupBy)Returns cost data broken down by any dimension
Granularity="MONTHLY"Aggregates data per calendar month
Filter["And"]Both conditions must match (service = EC2 AND purchase type = on-demand)
GroupBy=[{"Type": "DIMENSION", "Key": "INSTANCE_TYPE"}]Breaks results by instance type
group["Metrics"]["BlendedCost"]["Amount"]Cost as a string — cast to float()
get_savings_plans_purchase_recommendation(SavingsPlansType, TermInYears, PaymentOption)AWS-generated SP recommendation
get_reservation_purchase_recommendation(Service, TermInYears, PaymentOption)AWS-generated RI recommendation
LookbackPeriodInDays="THIRTY_DAYS"How much history the recommendation engine uses

Common Issues

DataUnavailableException — Cost Explorer needs at least 14 days of usage data before it can generate recommendations. Enable Cost Explorer in the billing console and wait.

Empty recommendations — On-demand spend must be significant enough for AWS to recommend commitments. Very small accounts may not get RI recommendations.

Region must be us-east-1 — Cost Explorer API is only available in us-east-1 regardless of where your resources are deployed.


🔍 Line-by-Line Code Walkthrough

Imports

LineWhy It’s Used
import boto3AWS SDK — needed for the Cost Explorer client
from datetime import datetime, timedeltadatetime.utcnow() for the current date, timedelta(days=30) to compute the lookback start date
import jsonSaves the final report as a JSON file

Cost Explorer Client

ce = boto3.client("ce", region_name="us-east-1")
LineExplanation
boto3.client("ce", ...)"ce" is the service name for AWS Cost Explorer
region_name="us-east-1"Cost Explorer is a global service but the API endpoint only exists in us-east-1. You MUST hardcode this, regardless of which region your resources are in

Date Setup

end   = datetime.utcnow().strftime("%Y-%m-%d")
start = (datetime.utcnow() - timedelta(days=lookback_days)).strftime("%Y-%m-%d")
LineExplanation
datetime.utcnow()Current UTC date as a naive datetime
.strftime("%Y-%m-%d")Formats as "2025-01-20" — the ISO 8601 format required by Cost Explorer. The API rejects other formats
timedelta(days=lookback_days)Subtracting 30 days from today gives the start of the analysis window
End date is exclusiveLike Python’s range(), the end date is NOT included. To see data through Jan 20, set end to Jan 21

get_cost_and_usage(...)

cost_response = ce.get_cost_and_usage(
    TimePeriod={"Start": start, "End": end},
    Granularity="MONTHLY",
    Filter={
        "And": [
            {"Dimensions": {"Key": "SERVICE",       "Values": ["Amazon Elastic Compute Cloud - Compute"]}},
            {"Dimensions": {"Key": "PURCHASE_TYPE",  "Values": ["On-Demand"]}},
        ]
    },
    Metrics=["BlendedCost", "UsageQuantity"],
    GroupBy=[{"Type": "DIMENSION", "Key": "INSTANCE_TYPE"}],
)
ParameterExplanation
TimePeriodDate range. Both Start and End are in "YYYY-MM-DD" format
Granularity="MONTHLY"Aggregates data by calendar month. Other options: "DAILY" or "HOURLY" (hourly only available for last 14 days)
Filter["And"]Logical AND — all nested conditions must match. Here: service must be EC2 AND purchase type must be On-Demand
"Key": "SERVICE"Filter by AWS service name. "Amazon Elastic Compute Cloud - Compute" is the exact service name for EC2 instances
"Key": "PURCHASE_TYPE"Filter to "On-Demand" only — excludes Savings Plan and Reserved Instance charges
Metrics=["BlendedCost", "UsageQuantity"]Which numbers to return. BlendedCost = dollar amount. UsageQuantity = hours of usage. You can request multiple metrics
GroupBy=[{"Type": "DIMENSION", "Key": "INSTANCE_TYPE"}]Splits the result by instance type (e.g., t3.medium, c5.xlarge). Without this, you’d get one total number
for result in cost_response["ResultsByTime"]:
    for group in result["Groups"]:
        instance_type = group["Keys"][0]
        cost = float(group["Metrics"]["BlendedCost"]["Amount"])
        instance_costs[instance_type] = instance_costs.get(instance_type, 0) + cost
LineExplanation
cost_response["ResultsByTime"]List of one dict per time period (one per month when Granularity=MONTHLY)
result["Groups"]List of groups — one per instance type for this month
group["Keys"][0]The first key in the GroupBy key list. Since we group by INSTANCE_TYPE, Keys[0] is the instance type string (e.g., "t3.medium")
group["Metrics"]["BlendedCost"]["Amount"]The cost as a string (e.g., "123.456789"). Must be cast to float() for arithmetic
instance_costs.get(instance_type, 0) + costAccumulates costs across multiple months. .get(key, 0) returns 0 if the key doesn’t exist yet

Savings Plan Recommendation

sp_response = ce.get_savings_plans_purchase_recommendation(
    SavingsPlansType="COMPUTE_SP",
    TermInYears="ONE_YEAR",
    PaymentOption="NO_UPFRONT",
    LookbackPeriodInDays="THIRTY_DAYS",
)
ParameterExplanation
SavingsPlansType="COMPUTE_SP"Compute Savings Plan — the most flexible type. Applies to EC2 (any instance type, any region), Lambda, and Fargate. Alternative: "EC2_INSTANCE_SP" (higher savings but less flexible)
TermInYears="ONE_YEAR"1-year commitment. "THREE_YEARS" offers higher discounts but more risk
PaymentOption="NO_UPFRONT"Pay monthly with no upfront cost. "ALL_UPFRONT" saves more but requires capital. "PARTIAL_UPFRONT" is the middle ground
LookbackPeriodInDays="THIRTY_DAYS"How much historical usage data the recommendation engine analyses. "SEVEN_DAYS" and "SIXTY_DAYS" are also valid
sp_summary = sp_response.get(
    "SavingsPlansPurchaseRecommendation", {}
).get("SavingsPlansPurchaseRecommendationSummary", {})
LineExplanation
.get("SavingsPlansPurchaseRecommendation", {})Returns an empty dict if no recommendation exists (insufficient data). Using .get() with a default prevents KeyError
.get("SavingsPlansPurchaseRecommendationSummary", {})The nested summary dict containing EstimatedMonthlySavingsAmount, HourlyCommitmentToPurchase, etc.

Report Printing

for itype, cost in sorted(instance_costs.items(), key=lambda x: -x[1]):
    bar = "█" * min(int(cost / total_cost * 30), 30)
    print(f"  {itype:<20} ${cost:>10.2f}  {bar}")
LineExplanation
sorted(..., key=lambda x: -x[1])Sorts by cost descending (highest spend first). x[1] is the cost value; negating it reverses the sort order
cost / total_cost * 30Computes the proportional bar length: 30 characters = 100% of total cost
min(..., 30)Caps the bar at 30 characters to prevent line overflow
int(...)"█" * 12.7 is invalid — we need an integer
{itype:<20}Left-aligned string padded to 20 characters. Makes columns line up
{cost:>10.2f}Right-aligned float with 2 decimal places, padded to 10 chars

Reserved Instance Recommendation

ri_response = ce.get_reservation_purchase_recommendation(
    Service="Amazon EC2",
    TermInYears="ONE_YEAR",
    PaymentOption="NO_UPFRONT",
    LookbackPeriodInDays="THIRTY_DAYS",
)
ParameterExplanation
Service="Amazon EC2"Which service’s RIs to recommend. Other values: "Amazon RDS", "Amazon ElastiCache"
Note the differenceRI recommendations are per specific instance type in a specific region. SP recommendations are for a flexible hourly spend commitment
ri_response["Recommendations"]List of RI purchase recommendations, each with details about instance type, count, and savings
Services Used
Cost Explorerboto3IAM
Prerequisites
  • Python 3.8+
  • boto3
  • Cost Explorer enabled
  • IAM: ce:GetCostAndUsage, ce:GetSavingsPlansPurchaseRecommendation, ce:GetReservationPurchaseRecommendation
What You Learned
  • Cost Explorer API structure
  • Granularity and GroupBy parameters
  • Savings Plans vs Reserved Instances
  • TimePeriod formatting requirements

Have a similar scenario to share?

Production incidents are the best teachers. Submit your real-world scenario and help others learn.

Open Google Form

Related Scenarios