Dec 28, 2021

[system design] Scalable and Reliable Logging at Pinterest (DataEngConf SF16)

Reference:
https://youtu.be/DphnpWVYeG8?t=2250


Although this talk was back in 2016, there are several points to take away:

1. the way project 'merced' operates reminds me of paper 'Sparrow'

https://cs.stanford.edu/~matei/papers/2013/sosp_sparrow.pdf

And their use of zk as message delivery is fun.

2. back then Kafka doesn't support exactly once delivery and transaction commit;

thus they've spent quite of efforts for that.

3. back then K8S an't popular/feature rich enough which addresses HA issues~

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.