realtime

Real-time data stream processing:Case Study

Posted by gycg on September 25, 2016

Real-time data stream processing

I. Elements of Case Study
What is Case Study: investigating trends and specific situations in scientific disciplines

  1. The Problem
    i. Identify the problem
    ii. Explain why the problem is important
    iii.How was the problem identified?
    iv. Was the process for identifying the problem effective?

  2. Steps taken to address the problem

  3. Results

  4. Challenges and how they were met

  5. Beyond Results

  6. Lessons Learned

II. Background information

  1. What is stream processing?
    a method of continuous computation that happens as data is flowing through the system.

  2. What is real-time system?
    has tight deadlines
    low-latency

  3. Sourses of data using stream processing
    Sensors
    Web feeds
    Social networking

  4. Use cases of stream processing
    business intelligence
    real time data analysis
    trends detection
    smart order routing
    log processing
    real time stock quotes
    fraud detection
    Route Planning

  5. Real-time stream processing tools
    storm: a distributed computation framework for event stream processing, doing for realtime processing what Hadoop did for batch processing.
    spark(streaming’s batch processing): can be seen as a potential replacement for the MapReduce functions of Hadoop

III. Reference
Real Time Data Streaming Case Study Example
Fraud detection
Realtime Event Processing in Hadoop
distributed real time
Strom and spark
Case Study Research Design