Polyglot Workshop – Developing LLMs for Low-Resource Languages

9 Mar 2026, 13:00 → 13 Mar 2026, 17:00 Europe/Berlin

2. OG (IMPULSE-Haus)

Aniket Sen, Nicholas Kluge, Shiza Fatimah, Sophia Falk

“Polyglot Workshop – Developing LLMs for Low-Resource Languages” is a four-day, hands-on program focused on sharing expertise in building large language models, with a special emphasis on underrepresented languages. Our goal is to empower a new generation of practitioners to advance LLM development beyond high-resource settings. The Workshop covers all key stages of an LLM training pipeline, making this knowledge accessible to a broad and diverse audience.

 

In-person registration is closed, but online attendance over zoom is possible. Please note that, as this is a hands-on workshop, several sessions involve practical exercises on the University of Bonn's HPC cluster Marvin. Due to technical and security constraints, remote access to the cluster for these exercises cannot be provided to the online participants.

 

To register for online participation, please send an email to polyglot@uni-bonn.de. Zoom details will be sent to registered online participants before the workshop. If you have any other questions, don't hesitate to reach out at the same address.

 

Monday 9 March

13:00 → 16:00 Setting up your Workstation (optional!)

Introduction to basic concepts that are prerequisites for attendees to take full advantage of the workshop (e.g., SSH basics, Introduction to Linux, etc.).

Conveners: Alexander Ermakov, Aniket Sen, Nicholas Kluge, Shiza Fatimah

Tuesday 10 March

08:00 → 12:00 Introduction to LLMs

Participants are introduced to the fundamentals of LLMs. What are they? How do they work? What is cool about them? What will you be learning during the next days?

Conveners: Aniket Sen, Nicholas Kluge

12:00 → 13:00 Lunch Break

13:00 → 16:00 Success Stories

Case studies from teams and researchers that have developed LLMs or NLP tools for low-resource languages, highlighting both obstacles and breakthroughs.
Speaker 1: Marcellus Amadeus
Speaker 2: TBC.

Convener: Marcellus Amadeus

17:00 → 18:00 Marvin Tour

A guided tour of the Marvin Cluster. How does it operate? How was it built? Who are the brave souls that maintain it?

Convener: Jan Steiner

Wednesday 11 March

08:00 → 12:00 Introduction to HPC

Participants are introduced to the fundamentals of High-Performance Computing (HPC), including how HPC infrastructures can support LLM training, with hands-on demos.

Convener: Jan Steiner

12:00 → 13:00 Lunch Break

13:00 → 17:00 Working with Datasets

Participants are introduced to the fundamentals of how to work with text datasets (e.g., downloading, documenting, deduplication, filtering, and synthetic creation). Extra: participants will also receive an introduction to the basics of tokenization/text encoding.

Conveners: Nicholas Kluge, Shiza Fatimah

Thursday 12 March

08:00 → 12:00 LLM Pretraining 101

Participants are introduced to the fundamentals of pretraining LLMs (e.g., model architecture, parallel training, optimization, monitoring, evaluation).

Conveners: Aniket Sen, Nicholas Kluge

12:00 → 13:00 Lunch Break

13:00 → 17:00 Post-Training and Alignment

Participants are introduced to the fundamentals of fine-tuning and aligning LLMs (e.g., SFT, DPO, Constitutional AI).

Conveners: Florian Mai, Nicholas Kluge

Friday 13 March

08:00 → 12:00 Environmental Practices to LLM Development

An exposé on the ecological impact of training LLMs, with emphasis on sustainable practices.

Conveners: Nicholas Kluge, Sophia Falk

12:00 → 13:00 Lunch Break

13:00 → 17:00 LLMs for Humanities

A lecture on the possibilities of application for LLMs in Humanities research.

Conveners: Alexander Ermakov, Nicholas Kluge