This repository contains material for the advanced cognitive modeling course (Aarhus University). All Monday will be allocated to lectures, the practice and applications will be on Fridays. We will use Python, PyMC3 for Bayesian modelling, Tensorflow and OpenGym for deep/reinforcement learning).
Prerequisites: This course will be run using Python. Being familiar with variables, lists, dicts, the numpy and scipy libraries as well as plotting in matplotlib is required. If you have never programmed in Python, or if you have limited experience, you might consider preparing with the following tutorials:
The portfolio will consist of 3 Jupyter notebook exercises (Bayesian modeling on weeks 11, Reinforcement learning on week 16 and Deep reinforcement learning on week 19).
| Notebook | Colab | nbViewer |
|---|---|---|
| Coin-flipping problem - Bayes' rule |  |
 |
| Thinking probabilistically | |
|
| Introduction to PyMC3 | |
|
| Normal distribution- Linear Regression | |
|
| 7 scientists problem - Measurement of IQ | |
|
| Psychophysics | |
|
| Exam scores | |
|
| Memory retention | |
|
| Model Comparison | |
|
| Comparing gaussian means | |
|
| GLM | |
|
| Mixture models | |
|
| Notebook | Colab | nbViewer |
|---|---|---|
| OpenAI Gym | |
|
| Q-learning | |
|
| Deep Q-learning | |
|
| Notebook | Colab | nbViewer |
|---|---|---|
| Portfolio 1 - deadline: 03.22.2021 | |
|
| Portfolio 2 - deadline: 04.30.2021 | |
|
| Portfolio 3 - | |
|
| Friday | 12:00 – 14:00 | Presentation – Introduction to cognitive and computational modelling |
|---|
-
Lei Zhang - BayesCog Summer 2020 Lecture 09 - Intro to cognitive modeling & Rescorla-Wagner model
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016). Computational psychiatry as a bridge from neuroscience to clinical applications. Nature Neuroscience, 19(3), 404–413. https://doi.org/10.1038/nn.4238
Kriegeskorte, N., & Douglas, P. K. (2018). Cognitive computational neuroscience. Nature Neuroscience, 21(9), 1148–1160. https://doi.org/10.1038/s41593-018-0210-5
Lewandowsky, S. & Farrell, S. (2011). Computational modeling in cognition : principles and practice. Thousand Oaks: Sage Publications. Chapter 1.
Forstmann, B. & Wagenmakers. (2015). An introduction to model-based cognitive neuroscience. New York, NY: Springer. Chapter 1: An Introduction to Cognitive Modeling.
| Monday | 14:00 – 16:00 | The process of modeling |
|---|---|---|
| Friday | 12:00 – 14:00 | Programming Probabilistically – Bayes' rule, distributions |
Bodner, K., Brimacombe, C., Chenery, E. S., Greiner, A., McLeod, A. M., Penk, S. R., & Vargas Soto, J. S. (2021). Ten simple rules for tackling your first mathematical models: A guide for graduate students by graduate students. PLOS Computational Biology, 17(1), e1008539. https://doi.org/10.1371/journal.pcbi.1008539
Wilson, R. C., & Collins, A. G. (2019). Ten simple rules for the computational modeling of behavioral data. ELife, 8. https://doi.org/10.7554/elife.49547
Blohm, G., Kording, K. P., & Schrater, P. R. (2020). A How-to-Model Guide for Neuroscience. Eneuro, 7(1), ENEURO.0352-19.2019. https://doi.org/10.1523/eneuro.0352-19.2019
- Neuromatch Academy (W1D2 - Intro) - Video - Slides
- Neuromatch Academy (W1D2 - Outro) - Video - Slides
- Neuromatch Academy (W1D2 - Tutorials) - Videos - Slides
| Monday | 14:00 – 16:00 | Introduction to PyMC3 – MCMC – Parameter estimation |
|---|---|---|
| Friday | 12:00 – 14:00 | Normal distributions - Linear regression |
This week we will focus on Chapters 1 and 2 from the book (Bayesian analysis with Python).
Martin, O. (2018). Bayesian analysis with Python : introduction to statistical modeling and probabilistic programming using PyMC3 and ArviZ. Birmingham, UK: Packt Publishing. Chapter 1 and 2.
Kruschke, J. (2015). Doing Bayesian data analysis : a tutorial with R, JAGS, and Stan. Boston: Academic Press. Chapter 6 and 7.
Pilon, C. Bayesian methods for hackers : probabilistic programming and Bayesian inference. New York: Addison-Wesley. Chapter 1 and 2.
- An introduction to Markov Chain Monte Carlo using PyMC3 by Chris Fonnesbeck [first half of the conference] - Video - Code
| Monday | 14:00 – 16:00 | – 7 scientists - Measurement of IQ |
|---|---|---|
| Friday | 12:00 – 14:00 | Psychophysics |
Martin, O. (2018). Bayesian analysis with Python : introduction to statistical modeling and probabilistic programming using PyMC3 and ArviZ. Birmingham, UK: Packt Publishing. Chapter 3.
Kruschke, J. (2015). Doing Bayesian data analysis : a tutorial with R, JAGS, and Stan. Boston: Academic Press. Chapter 9.
- BayesCog Summer 2020 Lecture 11 - Hierarchical Bayesian modeling - Video
| Monday | 14:00 – 16:00 | Hierarchical Bayesian modelling |
|---|---|---|
| Friday | 12:00 – 14:00 | Memory retention |
Kruschke, J. (2015). Doing Bayesian data analysis : a tutorial with R, JAGS, and Stan. Boston: Academic Press. Chapter 10.
- Bayesian modeling without the math: An introduction to PyMC3- Video
| Course content | ||
|---|---|---|
| Monday | 14:00 – 16:00 | Models comparison Ch.5 (part I) - Comparing linear models |
| Friday | 12:00 – 14:00 | Models comparison Ch.5 (part II) - Bayes factors, exercises with group difference, one sample t test, repeated measures |
Martin, O. (2018). Bayesian analysis with Python : introduction to statistical modeling and probabilistic programming using PyMC3 and ArviZ. Birmingham, UK: Packt Publishing. Chapter 5.
Lee, M. & Wagenmakers. (2013). Bayesian cognitive modeling : a practical course. Cambridge New York: Cambridge University Press. Chapter 8.
van de Schoot, R., Depaoli, S., King, R., Kramer, B., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J., & Yau, C. (2021). Bayesian statistics and modelling. Nature Reviews Methods Primers, 1(1). https://doi.org/10.1038/s43586-020-00001-2
- The Bayesian Workflow: Building a COVID-19 Model by Thomas Wiecki [Part 1] - Video
- BayesCog Summer 2020 Lecture 12 - Model comparison - Video
- Intro to Bayesian Model Evaluation, Visualization, & Comparison Using ArviZ | SciPy 2019 Tutorial - Video
| Course content | ||
|---|---|---|
| Monday | 14:00 – 16:00 | Generalizing linear models - Course and live coding covering Ch.4 and exercises on using GLM during the second hour |
| Friday | 12:00 – 14:00 | Mixture models - Course and live coding covering Ch.6 and exercises with mixture models during the second hour |
Martin, O. (2018). Bayesian analysis with Python : introduction to statistical modeling and probabilistic programming using PyMC3 and ArviZ. Birmingham, UK: Packt Publishing. Chapter 4 & 6.
Kruschke, J. (2015). Doing Bayesian data analysis : a tutorial with R, JAGS, and Stan. Boston: Academic Press. Chapter 15.
GLM - Linear regression (PyMC3 documentation) GLM - Robust linear regression (PyMC3 documentation) GLM - Hierarchical linear regression (PyMC3 documentation)
| Course content | ||
|---|---|---|
| Monday | 14:00 – 16:00 | Time series - Introduction to Markov processes and hidden markov models for sequential data analysis in base Python |
| Friday | 12:00 – 14:00 | Time series - Hidden Markov models using PyMC3 |
- Markov Models From The Bottom Up, with Python, Eric Ma - link
-
Hierarchical Time Series With Prophet and PyMC3 by Matthijs Brouns - Video
-
Scott Linderman, Machine Learning Methods for Neural Data Analysis, 2021 Stanford University - Github repos
-
Neuromatch, W2D3 Decision Making Intro - Video
-
Ankan, A. & Panda, A. (2018). Hands-On Markov Models with Python. Birmingham: Packt Publishing.
For the (deep) reinforcement learning part of the course, we will be using Reinforcement leasrning: An introduction (Sutton & Barto, 2018). You can download the book here. The core concepts of reinforcement learning are nicely introduced in Chris Willcoks' Reinforcement learning course (see the Videos and Slides. The Neuromatch academy session focused on reinforcement learning is also highly recommended (see course material here). You can find Colab notebooks accompagning all these courses online.
| Course content | ||
|---|---|---|
| Friday | 12:00 – 14:00 | Introduction to reinforcement learning - Key concepts of reiinforcement learning and deep reinforcement learning. Introduction to the OpenAI Gym environment. Going through the first part of the chapter during the second hour. |
- Neuromatch Reinforcement Learning Intro - Doina Precup Video
- Neuromatch Reinforcement Learning Outro - Tim Behrens Video
- Reinforcement Learning: Machine Learning Meets Control Theory - Video
- Deep Reinforcement Learning: Neural Networks for Learning Control Laws - Video
- Reinforcement Learning 1: Foundations - Chris Willcocks - Video - Slides
| Course content | ||
|---|---|---|
| Monday | 14:00 – 16:00 | OpeAI Gym |
| Friday | 12:00 – 14:00 | Markov Decision Process. Bellman Equations. |
Géron, A. (2019). Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow : concepts, tools, and techniques to build intelligent systems. Sebastopol, CA: O'Reilly Media, Inc. Ch.18. Reinforcement learning.
Sutton, R. & Barto, A. (2018). Reinforcement learning : an introduction. Cambridge, Massachusetts London, England: The MIT Press. Ch.3. Finite Markov Decision Processes
- Reinforcement Learning 2: Markov Decision Processes - Chris Willcocks - Video
- Reinforcement Learning 3: OpenAI Gym - Adam Leach - Video
Pulcu, E., & Browning, M. (2019). The Misestimation of Uncertainty in Affective Disorders. Trends in Cognitive Sciences, 23(10), 865–875. https://doi.org/10.1016/j.tics.2019.07.007
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019). Reinforcement Learning, Fast and Slow. Trends in Cognitive Sciences, 23(5), 408–422. https://doi.org/10.1016/j.tics.2019.02.006
| Course content | ||
|---|---|---|
| Monday | 14:00 – 16:00 | Markov Decision Process Dynamic programming. Value and Policy iteration algorithms. |
| Friday | 12:00 – 14:00 | Markov Decision Process Frozen lake environment. Applications to computational psychiatry Zorowitz et al. (2020) |
Sutton, R. & Barto, A. (2018). Reinforcement learning : an introduction. Cambridge, Massachusetts London, England: The MIT Press. Ch.4. Dynamic Programming
- Reinforcement Learning 4: Dynamic Programming - Chris Willcocks - Video -
Juechems, K., & Summerfield, C. (2019). Where Does Value Come From? Trends in Cognitive Sciences, 23(10), 836–850. https://doi.org/10.1016/j.tics.2019.07.012
Zorowitz, S., Momennejad, I., & Daw, N. D. (2020). Anxiety, Avoidance, and Sequential Evaluation. Computational Psychiatry, 4(0), 1. https://doi.org/10.1162/cpsy_a_00026
| Course content | ||
|---|---|---|
| Monday | 14:00 – 16:00 | Q-learning |
| Course content | ||
|---|---|---|
| Monday | 14:00 – 16:00 | Deep Q-learning - CartPole example |
| Friday | 12:00 – 14:00 | Deep Q-learning - Portfolio 2 (correction) |
Géron, A. (2019). Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow : concepts, tools, and techniques to build intelligent systems. Sebastopol, CA: O'Reilly Media, Inc. Ch.18. Reinforcement learning.
MIT 6.S091: Introduction to Deep Reinforcement Learning (Deep RL) - Video
| Course content | ||
|---|---|---|
| Monday | 14:00 – 16:00 | Deep Q-learning |
| Wednesday | 12:00 – 14:00 | Invited lecture: Joshua Skeve - National Inequality and Individual Readiness to Cooperate |
Skewes, J. (2020, October 29). National Inequality and Individual Readiness to Cooperate. https://doi.org/10.31234/osf.io/f79rw Fischbacher, Urs, and Simon Gächter. 2010. "Social Preferences, Beliefs, and the Dynamics of Free Riding in Public Goods Experiments." American Economic Review, 100 (1): 541-56.