Yelp enhanced data streaming architecture using Apache Beam and Flink to unify pipelines, reducing discrepancies and maintenance challenges in offline data stores.
Previously, separate pipelines caused weak encapsulation, data discrepancies, and schema synchronization issues between online and offline databases at Yelp.
By abstracting internal system details, Yelp aimed to provide consistent data streams for business properties, simplifying analytical processes and maintenance.
Legacy data pipelines using MySQL Replication Handler and Cassandra Source Connector were replaced by unified Apache data streaming projects at Yelp.
[
Collection
]
[
|
...
]