IS 617 Large Language Models for the Economic and Social Sciences - HWS25 (University of Mannheim)

A course developed and taught by Indira Sen, Abigail Hayes, and Georg Ahnert

This course aims to equip students with the theoretical foundations and practical skills necessary to leverage Large Language Models (LLMs) in computational social science research. Students will explore how LLMs can be used for analyzing social and economic data, modeling human behavior, and generating insights from large-scale data sources. They will also learn about the challenges of using LLMs for social research and how social science principles can help audit and evaluate LLMs.

This is a project-based course. Be creative in thinking of a project, but here are some ideas to get you started. Please check the grading rubric to see what criteria will be used to score your projects as well as the course policy on the use of AI tools in conducting project work.

Schedule and Materials

Week	Lecture	Readings	Tutorial
1	Course Introduction & Demystifying LLMs 1: Tokens, Text Representation and Classification	1. How to read a paper 2. Can Generative AI improve social science?	Setup
2	Demystifying LLMs 2: Word Embeddings and Transformers	1. Word Embeddings 2. Introduction to Transformers	NLP Basics
3	Demystifying LLMs 3: Generative LLMs	1. Language Modeling 2. Embers of autoregression show how large language models are shaped by the problem they are trained to solve.	HuggingFace
4	Interacting with and Steering LLMs	1. The prompt report: a systematic survey of prompt engineering techniques. 2. Fine-tuning	LLM Inference
5	Infrastructure powering LLMs	1. Illustrating Reinforcement Learning from Human Feedback (RLHF) 2.Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research.	Prompting
6	Content Analysis and Project Pitches	1. ChatGPT outperforms crowd workers for text-annotation tasks Are Chatbots Reliable Text Annotators? Sometimes	Content Analysis
7	AI-augmented Surveys	1. Out of one, many: Using language models to simulate human samples 2. Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies	Survey Simulations
8	Social Media Simulations	1. Social Simulacra: Creating Populated Prototypes for Social Computing Systems 2. Simulating social media using large language models to evaluate alternative news feed algorithms	LLM Agents
9	Project Discussion		Midway Project Presentations
10	Machine Behavior + Guest Lecture by Jana Jung	1. Machine behaviour 2. AI Psychometrics: Assessing the Psychological Profiles of Large Language Models Through Psychometric Inventories	AI Psychometrics
11	Ethical Impacts	1. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?	Ethical Impacts
12	AI Safety and Alignment	1. What is ‘AI psychosis’ and how can ChatGPT affect your mental health? 2. The PRISM alignment project	AI Safety
13	Auditing LLMs	1. Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. 2. Auditing large language models: a three-layered approach.	LLM Auditing
14	Summary and Project Discussions		Final Presentation

Credits

This course is based on a course Indira co-taught and co-created with David Garcia at the University of Konstanz. We're also grateful to the following related courses and resources for informing the materials in this one:

• Daniel Jurafsky and James H. Martin. 2025. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models, 3rd edition. Online manuscript released August 24, 2025. https://web.stanford.edu/~jurafsky/slp3.
• Elena Voita, 2020. NLP Course for you. https://lena-voita.github.io/nlp_course.html