How Twitter Migrated its On-Prem Analytics to Google Cloud (Cloud Next '18)

Опубликовано 9 августа 2018, 18:09

Twitter is migrating its existing on-premises revenue analytics infrastructure into Google Cloud. Using technologies such as Bigtable, BigQuery, and Dataflow, we have been able to provide a more robust analytic solution for significantly less cost.

This talk will focus on our work building an ETL pipeline to migrate an existing Lambda architecture into Google Cloud and building a query service to manage analytics queries to BigTable.

We will cover:
- Quickly and efficiently exporting data from our on-prem data stores into BigQuery
- Replacing analytic MapReduce jobs with BigQuery
- Loading results from BigQuery into Bigtable for high throughput online queries
- Querying data from Bigtable

Event schedule → g.co/next18

Watch more Data Analytics sessions here → bit.ly/2KXMtcJ
Next ‘18 All Sessions playlist → bit.ly/Allsessions

Subscribe to the Google Cloud channel! → bit.ly/NextSub

event: Google Cloud Next 2018; re_ty: Unlisted; product: Cloud - Migration - Application migration, Cloud - Databases - Database Migration Service;