Develop a data processing engine
with multi-step processing
Objective
To develop a domain-specific data processing engine with multi-step processing & minimal coding requirements.
What’s the need?
The data processing should remain at the scale of Spark infrastructure without writing the code in Spark.
Solution
An intelligent Spark program that delivers data Without Coding
Discuss Your Problem With Our Experts!
Connect With ExpertsImplementation Approach
01
An execution engine
processes data in
multiple stages
processes data in
multiple stages
02
A program which reads/writes
data from multiple data
points/targets
data from multiple data
points/targets
03
Completely configurable
engine with JSON DSL
engine with JSON DSL
04
The end user computation logic
is in the form of SQL for each
data processing stage.
The engine executes &
processes data without
coding in Spark/Scala
is in the form of SQL for each
data processing stage.
The engine executes &
processes data without
coding in Spark/Scala
05
The Fault-tolerant engine
recovers from its execution
if the job fails due to system failure
recovers from its execution
if the job fails due to system failure
Discuss Your Idea!
Request a call back