Interaction between HPC, AI, and Research Data

Acronym: HAI

Coordination: Dr.  Matthias Lieber, NHR@TUD

Other researchers involved: Prof. C.B. (TUDa), Dr. Charlotte Debus (NHR@KIT), Dr. Siavash Ghiasvand (TUD), Prof. Harald Koestler NHR@FAU,  Jaison Lewis (NHR@Göttingen), Prof. Sarah Neuwirth (NHR@SW), Lincoln Sherpa (NHR@TUD), Dr. Christian Terboven (NHR4CES@RWTH).

Centers involved: NHR4CES@RWTH, NHR4CES@TUDa, NHR@FAU,  NHR@Göttingen, NHR@KIT, NHR@SW, NHR@TUD

Motivation:  Data analytics and AI pipelines place high demands on software development and computational resources

Goals and methods: This project aims to contribute strategies and tools for efficient and collaborative development as well as high computational efficiency with three focus points: (1) data processing pipelines with LLM-automated data engineering for rapid development; (2) efficient use of HPC resources for scalable model training; and (3) collaborative code development and execution, paired with FAIR data management practices.

Innovations und perspektives: HAI will align to use cases of NHR users and increase their productivity and competences. It will contribute strategies for increased efficiency and usability of HPC resources by providing tools, tutorials, and documentation. Software will be provided open source and can also be used by other computing centers.

Projectduration: 24 months, start: Q4-2024