Stream processing vs batch processing
Examples
Stream processing: Log files, real time sensor readings, gamer actions e.g. moba, fps. Batch processing: collating incoming requests for customers, handling the batch of requests at the end of the day.
Differences
| Batch | Stream | |
|---|---|---|
| Freq | infrequent, produce results when job finished | jobs continuously run and produce results |
| perf | high latency (minutes to hours) | low latency (seconds to milliseconds) |
| data src | db, apis, static files | msg queues, event streams, transactoins |
| analysis | complex analysis | generally simpler, rising wave is complex |
| processing | process after store | process and then store |
