Research Questions
- How can persona usage in LLMs be systematically classified?
- Is a persona assigned to the LLM (Role-Playing), or is it applied to the user (Personalization)?
- How can personality assessment be conducted for LLMs?
Results
- The fundamental distinction in persona research is between Role-Playing (the LLM adopting a persona) and Personalization (the system adapting to the user’s persona).
- Role-Playing is used to improve task performance in domains such as software development, gaming, and medicine.
- Personalization focuses on meeting user needs in areas like search, recommendations, education, and healthcare.
- Evaluation typically relies on psychometric tests such as the Big Five Inventory (BFI) or the MBTI, though LLMs exhibit limited personality consistency.
Findings
- Role-Playing:
- Applied in single-agent scenarios (e.g., Voyager in gaming) or multi-agent scenarios (e.g., software development, medical workflows).
- In multi-agent settings, dynamics such as voluntary cooperation, coordination, and even destructive behaviours (toxicity, jailbreaking) can emerge.
- Personalization:
- Generates tailored recommendations and experiences by incorporating user history and contextual information.
- Application Overlap:
- In some cases, Role-Playing and Personalization can be used together, though their objectives differ.
- Risks:
- Persona assignment can introduce bias and create safety vulnerabilities.
- Assessment:
- Although LLMs often mimic intended persona traits, the generalisability of human psychometric tests to LLMs remains questionable.
Scores
- LLM Models: 5
- Synthetic Data: 3
- Method: 4
- Speed: 1
- Ethics: 4
- Accuracy: 2
- Demographics: 2
If you would like to explore this research in more detail, click here to read the full paper.