Datadog Cost Control With 45% Lower Spend and Zero Lost Alerts Using Terraform
We cut a fintech's monthly Datadog spend by 45% across 400+ Kubernetes nodes, with every critical alert on payment services kept intact.
lower monthly Datadog spend
critical alerts lost on payment services
of logs still archived and recoverable
A Datadog invoice that tripled in 18 months, and a team afraid to cut it
Our client is a high-growth fintech running customer-facing financial services on Kubernetes on AWS, across more than 400 nodes. In 18 months, its Datadog invoice tripled as the platform grew. Finance called the spend unsustainable, while engineering warned that cutting telemetry blindly could leave on-call teams without the signals that protect payment services.
As a regulated business, the client depends on reliable monitoring, alerting, and audit-ready logs. It partnered with Ksolves to shrink the bill and keep every critical alert, so observability costs could scale with the business instead of ahead of it.
C H A L L E N G E
01
Unmanaged Custom Metrics Teams had shipped custom metrics freely, and many were no longer queried by any dashboard or monitor yet were billed every month.02
High-Cardinality Tags Per-pod, per-request, and per-user tags multiplied unique timeseries across 400+ nodes, pushing metric volume up with every deployment.03
Every Log Line Indexed All logs, including debug output and health-check noise, were indexed at full price, even though only a fraction were ever searched.04
Oversized APM Ingestion Trace sampling had never been tuned, so high-volume, low-value traces were ingested at the same rate as critical transaction paths.05
Fear of Losing Visibility Engineering would not accept blind spots on payment-critical services, which had stalled earlier cost-cutting attempts.06
No Ongoing Cost Accountability Nobody could see which teams or services drove usage, so even a successful clean-up would have drifted back within months.S O L U T I O N
Cut what is unused, protect what keeps production safe
Ksolves, an AI-first DevOps services company, ran a structured Datadog cost optimization built on one rule: cut what is unused or redundant, never what protects production. The team audited usage across metrics, logs, and APM, mapped every billable signal to the dashboards and monitors that used it, and agreed a protected list of critical alerts with the client's SRE team before changing anything. Each fix below answers the challenge with the same number.
- 01 Usage audit and custom metric clean-up Every custom metric was checked against dashboard and monitor usage, and those no longer used by any view or alert were removed.
- 02 Tag cardinality controls Tag policies on the Datadog Agent drop or aggregate high-cardinality tags at source across the Kubernetes estate, cutting unique time series without losing service-level visibility.
- 03 Log exclusion filters with archive-then-rehydrate Datadog Log Pipelines index only operationally relevant logs, while every log line is archived to Amazon S3 and can be rehydrated on demand for audits or incident reviews.
- 04 Right-sized APM sampling Tuned sampling rules keep full fidelity on critical transaction paths and reduce ingestion of high-volume, low-value traces.
- 05 Protected critical alerting Every change was validated against the agreed list of critical monitors before and after rollout, so no payment-critical alert lost its underlying signal.
- 06 Cost dashboard and config as code A governance dashboard shows usage trends by product, and Terraform manages the filters, sampling rules, and monitors so savings cannot silently regress.
Architecture
Technology Stack
| Category | Technology |
|---|---|
| Platform | Datadog (Metrics, Logs, APM) |
| Log Processing | Datadog Log Pipelines |
| Orchestration | Kubernetes on AWS (400+ nodes) |
| Archive Storage | Amazon S3 |
| Infrastructure as Code | Terraform |
D E L I V E R Y
How a change reaches production without touching a critical alert
Ksolves rolled changes out in stages, each checked against the protected alert list agreed with the client’s SRE team. The filters, sampling rules, and monitors are codified in Terraform, and a shared cost dashboard shows usage by product. The client team can now see what drives spend, adjust rules through code, and recover any archived log from Amazon S3. As the Kubernetes footprint and customer base grow, observability cost no longer has to grow with them.
From a tripling invoice to a governed spend line, with every critical alert intact
Ksolves replaced an unmanaged, fast-growing Datadog bill with a lean, governed setup that ingests and indexes what matters and archives the rest cheaply in Amazon S3. Cost controls live in Terraform and on a shared dashboard, so observability spend is now an accountable budget line. The client can grow without observability costs scaling out of control.
- 45% Lower Monthly Datadog Spend The invoice had tripled in 18 months and was flagged as unsustainable. Monthly spend is now 45% lower.
- 0 Critical Alerts Lost Cost cuts had stalled for fear of losing visibility. Every monitor on the agreed critical list kept its full signal.
- 100% of Logs Still Retained Every log line used to be indexed at full cost. Only relevant logs are indexed now, and 100% remain archived in Amazon S3 and can be rehydrated on demand.
- 400+ Nodes, 1 Cost View No one could see which services or teams drove usage. One dashboard now tracks usage by product, giving finance and engineering a shared view of spend.
T E A M
Meet the Engineers on Your Project
Every engagement is staffed by senior engineers from our DevOps team, who will design, build, and support your platform.
READY TO BUILD YOURS?
Cut Your Datadog Bill Without Cutting Your Visibility
Tell us what your Datadog usage looks like today. The DevOps team who built this will walk you through what a usage audit and a protected-alert list could look like for your metrics, logs, and APM.