Project Name
Starburst Cut Loan Decisioning to 6 Seconds and Killed 47 ETL Pipelines for an Indian Fintech
A leading Indian fintech processing 2.8 million loan applications monthly had grown its data infrastructure into three disconnected layers: Oracle for customer profiles and KYC, Cassandra capturing 400M+ daily fraud events, and ClickHouse aggregating risk metrics across 50M+ customers. A team of 12 data engineers maintained 47 ETL pipelines just to produce overnight risk reports. Loan decisions were made on data 12 to 24 hours stale. Applying its AI-First approach, Ksolves deployed Starburst Enterprise to unify all three systems under a single federated SQL layer, cutting loan decisioning latency from 14 minutes to 6 seconds and enforcing PII governance without modifying any source system.
- Siloed Risk Data Across Three Systems: Oracle held credit profiles, Cassandra captured fraud events, and ClickHouse stored risk metrics with no unified query layer. A single Customer 360 view required coordinating three separate systems.
- ETL Sprawl Creating 12 to 24 Hour Data Lag: 47 Spark and Airflow pipelines on staggered 4-hour, 8-hour, and nightly cycles. Loan risk decisions routinely made on stale data.
- Regulatory Reporting Bottleneck: RBI NPA classification and credit exposure reports required 3-day preparation cycles with 8 analysts, creating compliance risk during high-volume periods.
- Distributed Transaction Consistency: Oracle ACID, Cassandra eventual consistency, and ClickHouse columnar snapshots operated on different freshness guarantees, requiring careful federated query design.
- Oracle Legacy Schema Complexity: 14 years of schema evolution produced 340+ tables with no consistent customer key and 3 deprecated identifier schemes, requiring custom connector configuration.
- Query Governor Limits at Scale: Federation queries on 50M customers exceeded coordinator memory thresholds, requiring resource groups, pagination, and materialised view pre-computation.
Ksolves deployed Starburst Enterprise on a 3-coordinator, 12-worker cluster in AWS Mumbai co-located with all three source systems. The governing principle was federation without migration: every source system retained its existing architecture while all cross-system query, governance, and decisioning was delivered through the Starburst layer.
- Starburst Enterprise Federation Layer: Three catalogs (oracle_crm, cassandra_fraud, clickhouse_risk) expose all source data through a single Trino-compatible SQL interface with no source system modifications.
- Customer 360 Unified View: Six materialised views pre-join common cross-source patterns. The customer_360_live view delivers credit score, fraud flag, transaction health, and KYC status in a single SELECT, averaging 3.8 seconds on 50M customers.
- Real-Time Loan Decision API: Starburst JDBC integrated into the loan origination system, replacing a 7-step data assembly pipeline. Decisioning latency dropped from 14 minutes to under 6 seconds.
- Centralised PII Governance: 12 role-specific RBAC policies with column-level masking for PAN, Aadhaar, phone, and account numbers enforced at the Starburst layer. Zero source system modifications.
- Cassandra Pushdown Optimisation: Cross-partition queries designed using Starburst pushdown capabilities to avoid full-table scans on the 400M+ daily fraud events dataset.
Technology Stack
| Category | Technology |
|---|---|
| Federation | Starburst Enterprise |
| Source Systems | Oracle RDS + Apache Cassandra + ClickHouse |
| Analytics | Customer 360 Materialized Views |
| Integration | Starburst JDBC into Loan Origination System |
| Governance | Starburst RBAC and Column-Level Masking |
- 47 ETL Pipelines Decommissioned: All 47 Spark and Airflow pipelines replaced by Starburst federation. 12 data engineers freed from ETL maintenance.
- Loan Decisions Cut From 14 Minutes to 6 Seconds: JDBC integration replaced a 7-step assembly pipeline. Cross-system risk scores delivered at the point of application without data assembly delays.
- Customer 360 Across 50M Customers in 3.8 Seconds: credit score, fraud flag, transaction health, and KYC status in a single SELECT. Complete risk profile with no ETL lag.
- 400M+ Daily Fraud Events in Real Time: Cassandra fraud events queryable directly in federated SQL. Behavioural signals included in loan risk scoring at decisioning time, not 12 to 24 hours later.
- Regulatory Reporting 3x Faster: RBI NPA and credit exposure reports produced directly from the federated layer. 3-day analyst preparation cycles eliminated. 100% SLA adherence maintained.
- PII Governance Across 12 Consumer Roles: Column-level masking enforced uniformly at the Starburst layer. Zero source system modifications required.
“For the first time, our risk team can run a single SQL query spanning our entire data estate and get a live answer in seconds. What used to take overnight and a team of engineers now takes one query and under 10 seconds.”
– Chief Data Officer / Head of Risk Technology.
A leading Indian fintech trapped by 47 ETL pipelines, 12- to 24-hour data lag, and 14-minute loan decisioning was transformed through Ksolves’ big data services. Starburst Enterprise federates Oracle, Cassandra, and ClickHouse under one SQL interface. 47 pipelines decommissioned. Loan decisions in 6 seconds. Customer 360 in 3.8 seconds. 400M+ fraud events queryable in real time. Regulatory reporting 3x faster. PII governance enforced without touching a single source system.
Is Your Risk Team Still Waiting on Overnight ETL to Make Decisions on Yesterday’s Data?
