Project Name
How Ksolves Built a Stateful Geo-Fencing Engine to Eliminate Location-Spoofing Fraud
A leading Telecommunications operator in the UAE approached Ksolves with a critical fraud challenge in their Real-Time Marketing unit. Location-based offers were being exploited by bad actors who spoofed geolocation signals to fraudulently trigger promotions intended exclusively for subscribers physically present in specific cities, districts, or venues.
The most visible attack pattern was Impossible Travel: a subscriber’s mobile number would appear in Riyadh to claim a location-triggered offer, while their last verified network activity had been registered in Dubai only minutes earlier, a physical journey that no mode of transport could complete. With 150 to 250 million geolocation events processed per day and traffic surging beyond 25,000 events per second during major UAE commercial events, the detection system had to operate continuously, tracking every active subscriber’s last known position and flagging implausible location transitions before any fraudulent offer could be served.
Ksolves engineered a stateful geo-fencing engine built on Apache Spark Structured Streaming and Apache Kafka, combining real-time subscriber state management with inline Haversine geospatial computation to deliver sub-second fraud flagging at enterprise scale.
- Stateful Tracking at Scale: Maintaining last-known latitude, longitude, and timestamp for tens of millions of active subscribers required a distributed state store that remained consistent and low-latency across all Spark executors simultaneously.
- High-Variance Throughput: Average throughput of 2,000 to 5,000 EPS was manageable in steady state, but spikes exceeding 25,000 EPS during Eid, Dubai Shopping Festival, and New Year's Eve represented a 5x surge that state operations could not be permitted to bottleneck.
- Geospatial Computation in a Streaming Context: Calculating Haversine distances and deriving implied travel speeds had to happen inline for every incoming event, within the latency budget of a micro-batch cycle, without offloading to external systems.
- State Expiry and Memory Management: Subscriber state had to be retained long enough for meaningful fraud detection while expiring appropriately to prevent unbounded memory growth across long-running streaming jobs.
- False Positive Management: Legitimate fast-location transitions such as passengers in commercial aircraft or GPS noise events had to be accurately distinguished from genuine fraud signals without over-suppressing real marketing engagement.
Ksolves designed and implemented a real-time stateful geo-fencing engine that combines Kafka event streaming with Spark Structured Streaming's stateful processing capabilities. Every active subscriber is tracked as a continuously updated geospatial profile, enabling instant comparison of any new location event against the subscriber's last verified position.
-
Stateful Subscriber Profiles via MapGroupsWithState
- Per-Subscriber State Objects: Spark's MapGroupsWithState operator maintains a keyed state object per subscriber MSISDN, storing the most recent latitude, longitude, and timestamp observed for that subscriber.
- Atomic State Updates: On each new marketing trigger event, the system retrieves the current state for the relevant MSISDN, computes the geospatial delta, and updates the state with the latest values in a single atomic operation.
- Intelligent State Expiry: State timeout policies are configured to expire inactive subscriber records, preventing memory leaks in long-running streaming jobs while retaining state for all commercially active subscribers.
-
Impossible Travel Detection Engine
- Haversine Distance Calculation: For each incoming event, the system calculates the Haversine distance in kilometres between the event's reported coordinates and the last known coordinates stored in state, accounting for Earth's curvature at any coordinate pair.
- Speed-Based Fraud Classification: The system then computes the implied travel speed by dividing the distance by the elapsed time in hours. If the implied speed exceeds 1,000 km/h, a threshold calibrated beyond commercial aviation speeds, the event is immediately classified as high-risk.
- Operationally Calibrated Threshold: The 1,000 km/h threshold was operationally validated to eliminate false positives from legitimate air travel while reliably catching location spoofing and session hijacking at the MSISDN level.
-
Event Routing and Compliance Logging
- Kafka-Driven Offer Suppression: High-risk events are written to a dedicated fraud-flagging Kafka topic, consumed in real time by the marketing execution layer to suppress pending offer delivery before it reaches the subscriber.
- Full Audit Trail: All flagged events, including MSISDN, coordinates, timestamps, and computed travel speed, are logged to a structured audit store for compliance review, downstream investigation, and fraud pattern analysis.
-
Future Enhancement with Apache Flink
The current Spark-based architecture delivers near-real-time detection within micro-batch cycles. A planned enhancement to Apache Flink will eliminate this micro-batch latency entirely. Flink's native keyed state management and event-time processing would enable state updates and Impossible Travel calculations to occur as each individual event arrives, achieving genuine sub-second fraud flagging even at 25,000+ EPS peak conditions.
Technology Stack
| Layer | Technology / Tool | Purpose in the Solution |
|---|---|---|
| Stream Processing | Apache Spark Structured Streaming | Stateful micro-batch event processing with MapGroupsWithState for per-MSISDN location tracking |
| Message Broker | Apache Kafka | High-throughput event ingestion of geo-location triggers and dedicated fraud-flagging topic for offer suppression |
| State Management | Spark MapGroupsWithState | Keyed state store maintaining last known latitude, longitude, and timestamp per subscriber MSISDN |
| Geospatial Algorithm | Haversine Formula | Inline great-circle distance calculation between GPS coordinate pairs accounting for Earth's curvature |
| Fraud Detection Logic | Speed-Threshold Engine (1,000 km/h) | Implied travel speed computation (Distance / Time Elapsed) with calibrated threshold beyond commercial aviation speeds |
| Compliance & Audit | Structured Event Log Store | Full audit trail of flagged events including MSISDN, coordinates, timestamps, and computed travel speed |
| Future Enhancement | Apache Flink | Native keyed state management and event-time processing for genuine sub-second fraud flagging at 25,000+ EPS |
- Near-Real-Time Fraud Prevention at 150–250M Events/Day: Impossible Travel fraud is now detected in near real-time across 150 to 250 million daily geo-location triggers, preventing location-spoofed offers from reaching fraudulent sessions before they can be actioned.
- 5x Spike Resilience During Peak Commercial Events: The engine sustains accurate stateful detection through event spikes exceeding 25,000 EPS during Eid, Dubai Shopping Festival, and New Year's Eve — precisely the high-value commercial windows when fraud attempts are most concentrated and the cost of a missed detection is highest.
- Zero False Positives for Legitimate Air Travellers: The 1,000 km/h travel speed threshold has been operationally validated to capture genuine spoofing and session hijacking while generating zero false positives for subscribers legitimately travelling by commercial aircraft.
- Geo-Verified Offer Delivery Across All Campaigns: Location-triggered marketing campaigns are now executed with full confidence that every geo-targeted offer is served exclusively to subscribers physically present in the intended geography, protecting both campaign ROI and brand credibility.
- Secondary Analytics Asset- Real-Time Subscriber Location Intelligence: The per-MSISDN state log created as a by-product of fraud detection has become a high-value secondary asset, enabling network analytics, real-time personalisation, and audience segmentation use cases beyond fraud prevention.
- Future-Ready: Sub-Second Detection with Apache Flink: With a clear migration path to Apache Flink already scoped, the architecture is engineered to move from near-real-time to genuine sub-second fraud flagging, future-proofing the platform against evolving fraud techniques and growing event volumes.
This engagement demonstrates what it means to build intelligent infrastructure at the intersection of streaming data engineering and real-time fraud prevention. Ksolves did not retrofit a batch fraud detection system for streaming use, instead, the team designed a purpose-built stateful engine where geospatial awareness, subscriber identity continuity, and event-time reasoning operate as first-class capabilities within the streaming architecture itself.
The result is a platform that protects the commercial integrity of location-triggered marketing at a scale and speed that batch or conventional streaming architectures cannot match. With a clear migration path to Apache Flink for genuine sub-second detection, the architecture is built to grow alongside the operator’s ambitions.
Prevent fraud in real time, before it impacts revenue and trust.
