Map of Streaming Data Systems

Streaming data systems are built to handle a constant and endless flow of data (sometimes described as events.)

This is fundamentally different from the way most software works, where systems are expected to handle a bounded set of actions initiated by a user or a scheduler.

It’s useful to break into two areas where streaming data is used to make learning more manageable:

• Distributed Applications (Microservices)
• Data Engineering

Streaming System: Actions taken by users on an e-commerce site (Login, Update Name, Add to Cart, Update Cart, Checkout) are appended to an event stream. Separate services read from the stream to create audit logs and trigger suspicious behavior alerts for security, assign promotions and push notifications to eligible users.

Traditional System: Actions taken by users on an e-commerce site trigger transactions to a database, an application reads from the database periodically to scan for suspicious behavior, promotions and push notifications are created in the same application code that is updating the database.

Systems in Depth#

Go deeper into Systems via the following articles.

External Resources#

Below is a curated list of high-quality external resources on the topic of Systems.