Autoscaling Streaming Applications in Cloud Dataflow with Scotiabank (Cloud Next '18)

Опубликовано 26 июля 2018, 16:58

This talk provides and overview of how Cloud Dataflow autoscales streaming applications on GCP. Often the production streaming pipelines are over provisioned in order to handle peak loads. Autoscaling improves resource utilization by scaling up or down based on the load. Better understanding of various factors that influence its decisions in a pipeline is helpful in development and management of production pipelines. We also talk about how common streaming sources like PubSub and Kafka interact with autoscaling.

DA222

Event schedule → g.co/next18

Watch more Data Analytics sessions here → bit.ly/2KXMtcJ
Next ‘18 All Sessions playlist → bit.ly/Allsessions

Subscribe to the Google Cloud channel! → bit.ly/NextSub


event: Google Cloud Next 2018; re_ty: Publish; product: Cloud - Data Analytics - Dataflow; fullname: Reuven Lax;