About me

I am a second year PhD student and researcher in the Multilinguality and Language Technology lab at the German Research Center for Artificial Intelligence, working under the supervision of Josef van Genabith. My research is in the field of explainable artificial intelligence (XAI), where I work on interpreting the inner workings of large-scale pre-trained language models, as well as understanding their limitations. My background is in computational linguistics, computer science and cognitive science.

Thesis topics

last updated: October 2024

We are always looking for talented master students of Language Science and Technology or Computer Science, who want to work on a thesis in the area of interpretable, robust, or trustworty LLMs. Feel free to approach me with your own research questions or let me know if you are interested in any of the topics in the list below and want to talk about details.

Note: Due to the nature of the topics I am interested in, you need to have some experience with LLMs and at least a somewhat decent theoretical understanding of neural networks/deep learning.

Specific ideas:

• Detection of indirect prompt injections through XAI methods in RAG models
• How robust are LLMs to editing?
• Encoding of (different kinds of) concepts in LLM activation spaces
• Evaluation of activation spaces in response to different fine-tuning techniques (full, adapters, soft prompts)
• Comparison of parameter efficiency of fine-tuning techniques (full, adapters, soft prompts)

Topics that I am very interested in, but where we would need to find a research question together:

• Any topic on (arithmetic) reasoning skills of LLMs
• Any topic on investigating the flow of information within LLMs (Mechanistic Interpretability)
• Any topic on task arithmetics
• Any topic in the scope of Sparse Autoencoders