System design to induce delay in the Kafka Producer using Redis expired notifications (pub/sub)

1. Client
POST API : /delay-message
External APIs
API producing
events
ingest message in mongo
Layer 1 : Delay API Microservice
(Exposed to client and other external APIs)
ingest the messageID in
redis with a specific TTL (delayTime)
Triggers event to produce expired message in Kafka topic
Layer 2 : Redis Subscriber
(Internal API microservice to subscribe to redis expired
events and trigger Kafka producer to push expired
message in Kafka Topic)
Subscribes to Redis to get
the expired message ID
Fetches the expired message from persistent storage from messageID expired in redis
Message :
{"payload":OBJECT,
"delayTime": INTEGER}
SYSTEM DESIGN
Push message in Kafka Topic
Layer 3 : KAFKA PRODUCER
(Internal API microservice to produce messages in Kafka Topic)
Client
External APIs
API consuming messages
KAFKA CONSUMERS
KAFKA DELAY
PRODUCER
SIDDHARTH RAJA
SWE @ARISTA NETWORKS