Data Lakes

December 17th 2021, 14:00 - 17:00

In recent years, classic HPC users have seen an ever-increasing interest in the public cloud that is used as part of traditional HPC workflows. There are many reasons for this, e.g. special hardware components such as TPUs or special GPUs are available in the cloud earlier than in a local data center. In addition, there is a need for users to store any data for analysis using AI methods in different data silos and to be able to access them flexibly from HPC and cloud systems. A central role for data analytics workflows is the flexible data migration and provision in the data lake. For this purpose, highly-scalable object storage has long been established in the cloud area, which is mostly used via an S3 interface. Another advantage from the user's point of view for a consistent data management strategy as offered by a data lake is the uniform and consistent view that it allows for the individual data silos.

Data centers can share their plans and services regarding high-performance data analytics (HPDA) and big data analytics (BDA) and scientific data management. This workshop aims to foster interaction between data centers, administrators, and, most importantly, researchers.

Our motto is: Let's build a bridge to the data lake together

The videos of the event are now online on YouTube.

You can register for this workshop here. If you would like to give a talk, please contact Julian Kunkel.

This workshop is funded by the GWDG and supported by the NHR.