About
Producer / Consumer is concurrency model (ie two threads/process communication) where:
The data send and received is called a message.
Articles Related
Pipeline
When the consumer becomes also a producer for another consumer, they form a chain called a pipeline.
See also Event - Provider
Upstream faster than downstream
What solutions do we have when there is a fast producer and a slow consumer (ie if the upstream is faster, than the downstream)
- Backpressure - the slower consumer request a given amount of data from the faster producer
- Buffer the messages in a queue
- One consumer: as buffers have limited capacity, the application will run out of memory sooner or later.
- Start another consumers in parallel that will consume the same buffer. see Parallel Programming - Producer Consumer problem (Bounded-buffer problem)
- Drop some message - networking hardware does that
- Block the producer until the consumer catch up. It slows down the entire pipeline.