Community Tools for Analysis of Earth Science Data in the Cloud

Опубликовано 21 июня 2019, 17:17

The Pangeo project is building a flexible, open framework for scientific analysis and visualization based on tools widely used in the Python ecosystem. We will demonstrate the flexibility of the system, showing how the user needs only their browser to access the framework via JupyterHub. Data is stored in cloud-friendly formats on Amazon S3, while Xarray provides tools for efficient data manipulation. Dask schedules fine-grained parallelization on a Kubernetes cluster, and PyViz provides interactive visualization. We will also discuss performance, security, and transferability across public cloud platforms, costs to operate, and approaches to encourage a cultural shift in scientific computation.