Stream processing vs batch processing


Stream processing: Log files, real time sensor readings, gamer actions e.g. moba, fps. Batch processing: collating incoming requests for customers, handling the batch of requests at the end of the day.


Freqinfrequent, produce results when job finishedjobs continuously run and produce results
perfhigh latency (minutes to hours)low latency (seconds to milliseconds)
data srcdb, apis, static filesmsg queues, event streams, transactoins
analysiscomplex analysisgenerally simpler, rising wave is complex
processingprocess after storeprocess and then store