GCP for Apache Kafka Users: Stream Ingestion and Processing (Cloud Next '19)

Опубликовано 12 апреля 2019, 22:48

In private and public clouds, stream analytics commonly means stateless processing systems organized around Apache Kafka or a similar distributed log service. GCP took a somewhat different tack, with Cloud Pub/Sub, Dataflow, and BigQuery, distributing the responsibility for processing among ingestion, processing and database technologies.
We compare the two approaches to data integration and show how Dataflow allows you to join and transform and deliver data streams among on-prem and cloud Kafka clusters, Cloud Pub/Sub topics and a variety of databases. The session will have a mix of architectural discussions and practical code reviews of Dataflow-based pipelines.

Trusted Cloud Access Through Chrome Browser → bit.ly/2K9lfQr
Get Chrome Browser for Enterprise → bit.ly/2TWkABa

Watch more:
Next '19 Data Analytics Sessions here → bit.ly/Next19DataAnalytics
Next ‘19 All Sessions playlist → bit.ly/Next19AllSessions

Subscribe to the G Suite Channel → bit.ly/G-Suite1


Speaker(s): Ricardo Ferreira, Karthi Thyagarajan


Session ID: DA305


event: Google Cloud Next 2019; re_ty: Publish; product: Cloud - Data Analytics - BigQuery; fullname: Ricardo Ferreira, Karthi Thyagarajan;