Real-time data stream processing
I. Elements of Case Study
What is Case Study: investigating trends and specific situations in scientific disciplines
-
The Problem
i. Identify the problem
ii. Explain why the problem is important
iii.How was the problem identified?
iv. Was the process for identifying the problem effective? -
Steps taken to address the problem
-
Results
-
Challenges and how they were met
-
Beyond Results
-
Lessons Learned
II. Background information
-
What is stream processing?
a method of continuous computation that happens as data is flowing through the system. -
What is real-time system?
has tight deadlines
low-latency -
Sourses of data using stream processing
Sensors
Web feeds
Social networking -
Use cases of stream processing
business intelligence
real time data analysis
trends detection
smart order routing
log processing
real time stock quotes
fraud detection
Route Planning -
Real-time stream processing tools
storm: a distributed computation framework for event stream processing, doing for realtime processing what Hadoop did for batch processing.
spark(streaming’s batch processing): can be seen as a potential replacement for the MapReduce functions of Hadoop
III. Reference
Real Time Data Streaming Case Study Example
Fraud detection
Realtime Event Processing in Hadoop
distributed real time
Strom and spark
Case Study Research Design