Bell Labs Internship on Increasing the Reliability of LLM Applications with Knowledge Estimation (PhD)

Job Description:

Large Language Models (LLMs) present an interesting opportunity for Nokia to increase its efficiency and improve the usability of our products (by automating routine tasks, by augmenting human experts in specialized areas, etc.). However, state-of-the-art LLMs remain unreliable: slightly different prompts can lead to very different responses, answers are not guaranteed to be factual, etc.
Knowledge estimation [1, 2], is an emerging topic that is concerned with estimating the extent to which LLMs have knowledge about a given subject. In this PhD internship, you will investigate how knowledge estimation can help increase the trustworthiness of LLMs for industry applications.
[1] https://arxiv.org/pdf/2404.12957
[2] https://arxiv.org/abs/2406.12673

Duration: flexible, to be agreed (typically 3-4 months), starting time is flexible
Location: Antwerp (Belgium)

* You will review scientific literature on knowledge estimation
* You will design techniques that leverage knowledge estimation to make LLMs more reliable (e.g., to improve a RAG pipeline, enable more directed fine-tuning, etc.)
* Evaluate the effectiveness of the proposed techniques on representative problems.
* Consolidate your research in a scientific publication

Candidate Requirements:

* Student enrolled in Ph.D. Computer Science/Engineering in Machine Learning
* Strong programming skills in Python
* Language skills: English
* Experience in AI with any of the following is a big plus: knowledge estimation, uncertainty estimation, robustness, interpretability.
* A strong publication record is a big plus