Research Questions
- Can LLMs be used as Subpopulation Representative Models (SRMs)?
- Which techniques can steer LLM behaviour, and which SRM applications already exist?
- How should the SRM lifecycle (design, development, and operation) be structured?
- What are the benefits and risks associated with this approach?
Results
- LLMs can serve as powerful tools for capturing public opinion and representing subpopulations.
- SRMs provide opportunities to overcome data scarcity when survey response rates are low.
- SRM applications are being explored in politics, sociology, and commercial domains.
- However, significant risks exist, including misinformation, bias, privacy violations, and potential misuse.
- Tasks range widely, from low-complexity classification to multi-turn, open-ended dialogue.
Findings
- Advantages:
- LLMs enable low-cost, open-ended analysis and can generate human-like representatives of subpopulations.
- Historical Parallel:
- SRMs echo early efforts such as the 1960s “People Machine”, signalling renewed interest in public opinion modelling.
- Application Areas:
- SRM use is growing in tasks such as forecasting elections, collecting consumer opinions, and tracking brand perception.
- Risks:
- Key concerns include misinformation generation, poor performance for marginalised groups, and the potential for social manipulation.
- Evaluation Framework:
- A five-criteria framework is proposed: fidelity, necessity, robustness, sensitivity, and fairness.
Scores
- LLM Models: 5
- Synthetic Data: 4
- Method: 5
- Speed: 3
- Ethics: 5
- Accuracy: 4
- Demographics: 5
If you would like to explore this research in more detail, click here to read the full paper.