After ChatGPT: where do we stand with language models?

Large language models (such as BERT, GPT3, ChatGPT... and soon GPT4) have deeply changed the landscape of research in natural language processing. These models have permitted previously unseen results on many tasks and in many languages. At the same time, their internal mechanisms remain rather opaque, and their full potential is still largely unknown.

What? half-day workshop
Where? Ecole normale supérieure, Salle des résistants, RDC, 45 rue d’Ulm 75005 Paris
When? Wednesday 11 January, from 2pm to 5:30pm
Registration: free entry, but please indicate your name here

(remote attending should be possible; the link will be sent to participants who have indicated their email address in the file above)

Argument

Language models (such as BERT, GPT3, ChatGPT… and soon GPT4) have deeply changed the landscape of research in natural language processing in recent years. These models have permitted previously unseen results on many tasks and in many languages. At the same time, their internal mechanisms remain rather opaque, and is the subject of intense research (Bertology). This situation raises many questions.

      • Can we say that these models ‘understand’ language? And if so, in what way? To what extent?
      • What is their interest and their benefits for research outside NLP? For creative work?
      • On a practical side, how can we deal with them and/or integrate them into our research, given the computing power required to train them? Have we become dependent on the major (often private) players in the field?
      • What are the limits of these models and their potential dangers?

We will probably not have all the answers to these questions on January 11, but this workshop will at least be an opportunity to think about these models, with various actors in the field, both private and public.

Programme

(presentations will be in English)

      • 2.00-2.45pm – Thea Sommerschield (U. Ca Foscari, Venice):  »Fair AI for ancient languages: a proposal for dos and don’ts », presentation following her experience in the framework of the Ithaca project (Thea Sommerschield will also give a presentation the day before, more directly dedicated to the Ithaca project, in the framework of the DHAI seminar, https://dhai-seminar.github.io/
      • 2:45-3.30pm – Tm Van De Cruys (KU. Leuven, Belgium): Using language models for poetry generation, language models and creation
      • 3:30-4:15pm Laurent Daudet and Olga Lopusanschi (LightOn, Paris) « I need my own ! Developing private Large Language Models« , on the development of language models within a start-up like LightOn.
      • 4:30 – 5:30pm : Round table on the current situation, research and future of these models. Discussion with the speakers and with Anne Bouverot (Abeona Foundation)
      • 5:30pm : Thierry Poibeau : Wrap-up

Sponsors

Organized with the support of Lattice (https://www.lattice.cnrs.fr/) and of Prairie (https://prairie-institute.fr/).

 

Slides

      • Unfortunately, Thea Sommerschield‘s slides are not available. You can however find the paper on Ithaca here
      • Tim Van de Cruys presentation is attached here
      • Laurent Daudet and Olga Lopusanschi’s (LightOn) presentation is attached here (general presentation, by Laurent) and here (the art of the prompt, by Olga)

 

 

 

 

A lire aussi