Draft chapter outline #1
Conversation
| 1. Chapter summary (Learning objectives + their solutions in a sentence) | ||
| - E.g. instead of stating a LO like "Apply faceting operations to charts", | ||
| we would say "Apply the `facet` method to group data into multiple subset | ||
| based on the unique values of a variables. This is helpful when we want to | ||
| compare subsets of data in a more explicit manner than by e.g. | ||
| coloring the points according to their value/group." |
There was a problem hiding this comment.
I like the idea of having learning outcomes that are not just floating statements but grounded in why it matters. In this example it sounds less like a solution and more like the underlying practical purpose of the learning outcome.
We could separate it into the WHAT and the WHY to make each LO more digestible. You could use a table for this.
There was a problem hiding this comment.
OBJECTIVE and RATIONALE could work too
There was a problem hiding this comment.
Maybe LOs are split into two different types? "Apply" where the solution is the code and "Understand" (or higher blooms) and the solution there would be the same as the "rationale/why".
| - I wonder if it would be convenient if these were somehow linked to the sections in the chapter | ||
| that describe each LO in more detail. | ||
| It seems like it could easily become redundant with the ToC, | ||
| but a reminder to students when they read each section | ||
| that they have just "completed" a LO would maybe be helpful | ||
| both to feel that they are making progress | ||
| and to remind them about what was important in the section they just read? | ||
| - Maybe a note or even a progress indicator in the margin? | ||
| "Learning objective N completed" / "Chapter 20% complete" / "You just learned that..." | ||
| There could also be exercises after each LO, more on that later. |
There was a problem hiding this comment.
I really like the idea of being able to click on the learning outcome and go to the correct section. Is there a possibility of making the progress indicator dynamic? Are we using Quarto Books? I guess I could add this in using Javascript / CSS? That way you could essentially have a progress bar at the bottom with the learning outcome.
There was a problem hiding this comment.
This is more of a later stage addition. We can initially explore if there is something in Quarto natively. If not, make an issue and revisit if we want to make a custom solution.
The main idea here is to structure the content so that it follows the LOs closely. We want to make LOs as modular as possible with the constraint that the formatting of the book should not reduce the quality of the content/LOs
| 2. Required readings? It's somewhat odd to have this in a textbook | ||
| the same way I now have it in the lecture notes. | ||
| But I would like to somehow | ||
| guide the learner toward additional resources | ||
| that the can use to complement what we are teaching, | ||
| e.g. a more in depth chapter/section in "Fundamentals of data visualization". |
There was a problem hiding this comment.
Absolutely, we could even use callout boxes at exactly the points in the text where they can explore deeper with another book / resource. Like a recurring type of callout called Dive Deeper or something. It could be a motif so the reader gets used to it.
There was a problem hiding this comment.
Maybe this can be combined with the challenging section? Is a "deep dive" necessarily challenging? If so, they could all link to the end of the chapter where the challenging section is. When it is more of a reference, it is not challenging... To start, let's try out separate Deep Dive callouts, not related to challenging section.
| - I think this is ideally something like a problematic chart or statement | ||
| illustrating how what students have learned up until this point | ||
| is not sufficient to solve the problem we are demonstrating, | ||
| so that they appreciate the need and importance to learn more. | ||
| - Maybe this could be the students studying a viz | ||
| to define the problem with applying only what we have learned so far | ||
| to solve the scenario we set up | ||
| (ideally a scenario with direct practical implications). | ||
| In general, I believe that learning to think about about what the problem is | ||
| given a certain (business) question is beneficial | ||
| instead of that we spoon-feed them the problem statement | ||
| and let them only think about the solution. | ||
| I like the idea of making them come up with as much as possible themselves, | ||
| then they are motivating why the chapter is important on their own | ||
| and there is less "convincing" needed. | ||
| - E.g. | ||
| > In the previous chapter we learned about | ||
| visualizing each row in a dataframe using a point mark | ||
| and best practices when mapping data to graphical objects/elements. | ||
| In the following chart we have followed these best practices | ||
| when plotting ...{insert some scenario}, | ||
| but it is not easy to extract the main takehome message | ||
| (or any info for that matter). | ||
| How can this be given that we followed everything we know so far? | ||
| Study the chart and think for yourself what has gone awry here | ||
| and write down a suggestion for what you would do to fix it. | ||
| {maybe provide a space in the book where they can write, | ||
| it's ok that it is not saved, | ||
| it just for them to formally put their thoughts into words | ||
| which makes it easier to remember}. | ||
| {show chart} |
There was a problem hiding this comment.
It makes a lot of sense to show the necessity for the next concept by demonstrating how the previous methods fall short. And I think showing a flawed chart piques their curiosity to find out how to improve it. I wonder if we can provide a bit of structure to the space where they write their ideas for instance a table with "Flaws", "Consequences" and "Suggested Solutions". Or "identify at least three issues with this chart...". I think the idea of consequences is useful here as it gets the reader thinking not just about how the chart is flawed but why those flaws matter.
We could use a kind of mirroring where the exercise at the beginning of the chapter is revisited at the end to show resolution or improvement. And you could even see the consequences changing. "With your revised chart the board was able to..." You basically apply all of the concepts you learned during the chapter to the initial problem.
There was a problem hiding this comment.
but why those flaws matter.
YES. 100% agree that thinking about what the downstream consequences of the chart issues are. WHY is it a problem?
I also like the improvement of the chart at the end and it is explained what actions .
The introductory problem should be limited to understand the limitations, and then the chapter introduces the solutions gradually. But also avoid disconnection between problem and solution. One mention of the problem can be more superficial (maybe in the beginning?), and then there is a deeper dive together with the solution. E.g. for overplotting, just say that we can't read the chart but not introduce terminology like "overplotting"/"saturated". The terminology comesi n with the solution.
| - Ideally, there would be some continuity between chapters in this narrative, | ||
| but I'm hesitant to make that a requirement at this point, | ||
| as I also want us to be able to use data from different domains. | ||
| The only way I see us connecting such widely different data | ||
| is through a narrative like that in labs 1-3, | ||
| where we go a bit quirky and very "out there", | ||
| which could be quite appreciated, | ||
| but I would also be OK with having a less humorous narrative | ||
| and the connection is only how the visualization principles tie into each other | ||
| but the data is more independent for each chapter. |
There was a problem hiding this comment.
Yes it is difficult to use a variety of datasets and form a coherent interwoven narrative. If we do go down the narrative route I have some ideas. One is to aim for an overall scenario but the reader takes on different roles within that scenario (maybe in one chapter they are a public health expert vs business analyst in another).
In terms of an overarching scenario it would have to be broad enough to encompass different topics, something like a smart city project could work, where have to consider all aspects of society, business and human wellbeing. I think grounding it in a location could help.
We could aim for an overarching narrative that stays somewhat 'real' feeling, although I'm sure it would be quite challenging to fit in some datasets without introducing a bit of absurdity.
There was a problem hiding this comment.
Problem: Society is broken in some way, or less strong: there is some dissatifaction in this fictional country/city and different people have different ideas of how to solve it. Could be another planet studying our world, but doesn't have to be. Can also just be a fictional world.
The students will embody different roles in society. Each role will care about and prioritize different aspects, e.g. the public health officer will focus on the inequalities in the world and argue that solving those will bring .... , someone else could argue that a more bottom up approach is required where individuals first fix themselves which fixes society (meditation, psychology, personal development data), business people will take a free market approach (some financial data), environmentlist focus on climate data.
| - One thing I wonder is if there is value in that the chapter design evolves over time | ||
| akin to how levels in video games don't follow quite the same pattern | ||
| even if there are some similar elements like boss fights etc. | ||
| Is there value in introducing this novelty to make students stay engaged? | ||
| I think learning apps like Duolingo also have slightly different level design | ||
| although not as extensive as in a good video game. | ||
| - I'm curious about this, | ||
| but it is an "out there" idea I would say | ||
| and we would be on uncharted waters | ||
| if we start to evolve the book structure | ||
| as the story progresses. | ||
| I think it sounds fun, | ||
| and someone must have written something about this idea in the past | ||
| so maybe we can find a few academic articles to learn more | ||
| if it is considered an effective learning strategy. | ||
| - Maybe we can spend a little time looking at the idea, | ||
| but I don't want this to be a time sink up front | ||
| (although the more I think about it the more it enthralls me =p). |
There was a problem hiding this comment.
Obviously I really like this idea. I wonder if there is a danger of making things addictive or engaging without actually enhancing the learning. Maybe if the gamification is intrinsically tied to gaining a deeper understanding.
Having said that, here are some gamification ideas:
- Use escalating narrative throughout the book. The stakes get higher. Maybe the student is aware that they will have to do something very difficult at the end.
- Skill trees: different chapters / sections could fill in a skill tree that expands as the student progresses through the book. Similar to Duolingo skill tree. It could fill in as they complete exercises.
- We could frame exercises as Challenges or something more game like.
- Perhaps more challenging exercises start to appear later in the book and you get more points for them
- If you do have a server setup you can consider info like x % of students completed this question or tell students what percentile they are in in terms of the number of points they've gained from completing challenges.
There was a problem hiding this comment.
I wonder if there is a danger of making things addictive or engaging without actually enhancing the learning. Maybe if the gamification is intrinsically tied to gaining a deeper understanding.
Good point that we should have in the back of our heads when designing.
Maybe some novel elements OK later as students get familiar with the book, but not complete chapter revamp since that would be confusing.
Maybe present LOs like a skill tree? Then students would be able to overview what is in the course and how they make progress as they complete exercises.
Some indication of which exercises are more challenging (points? label?)
Server would be really nice from tracking perspective to be able to demonstrate the impact on the book and how students use it. Plausible.io/gogole analytics is another option here but they are not as granular
| - Should we enable comments on the entire book? There are technologies such as `hypothes.is` | ||
| that are easily enabled in quarto | ||
| and allows student to highlight sections in the book and take notes. | ||
| They can even opt to make their notes public to other students | ||
| and have discussions in the comments | ||
| (maybe preferential and more direct vs having discussion on slack?), | ||
| but there would be no notifications and we are splitting discussion into two avenues. | ||
| Maybe more a space to exchange reflections on the content? |
There was a problem hiding this comment.
I like the idea of students being able to share reflections. hypothes could be good if it's unobtrusive. probably only very keen students would be interested in this.
I was also wondering if students can sort of annotate the charts in the book when identifying problems etc. I don't know how this would work technologically.
There was a problem hiding this comment.
Needs critical mass to be interesting. Unlikely is not mandatory? Useful outside MDS.
Unsure how it works when the content changes...
| - Should there be a dedicated section for common pitfalls? | ||
| This seems hard to execute and I would prefer to bring up pitfalls | ||
| as we go through each section in the chapter | ||
| rather than collect them all in a single section. | ||
| - An alternative to simply bringing it up in the text would be to | ||
| have callout boxes so it is easy to identify the pitfalls for each chapters, | ||
| but they are also occurring right after the section they are related to. | ||
| - Maybe this organization into different callout boxes (or similar semantic unit) | ||
| would allow us to have alternative ways of viewing the content. | ||
| E.g., maybe there could be a way to collate all callout boxes into a single page | ||
| (and like-wise all learning objectives, all excericese, etc separate pages) | ||
| either "on demand" or during the build process of the book. |
There was a problem hiding this comment.
I think this callout box approach is really promising. Dive Deeper, Pitfalls, even coding tips.
| - Should we store student answers to exercises in a cookie | ||
| so that they are saved when students navigate away from the page? | ||
| If we offered an account and login, | ||
| then they could really track their progress in a more organized way, | ||
| but I'm not sure there is any effective way of doing that | ||
| and now we would have to store it on a server instead. |
There was a problem hiding this comment.
A server could be useful if we want any collaborative / competitive / peer review elements. However if we just want to store the student's progress we could just use browser memory.
| - Should we have a "how to use this book" chapter? | ||
| Or even lay out the philosophy behind some of our choices? | ||
| Maybe at least a short version in some type of foreword, | ||
| but I don't want to students to have to read a manual-like compilation |
There was a problem hiding this comment.
Makes sense if we are transcending a typical textbook. Given that most of the elements we have discussed are visual most of this how to use could visual too like an annotated summary of the motifs we use in the book.
There was a problem hiding this comment.
The suggestion here is to create a diagram/visual guide of the motifs/callout boxes in the book. An "IKEA" manual rather than a text heavy one.
Ideally, the elements of the chapters would be self-explanatory and we don't need this section.
| 1. Chapter summary (Learning objectives + their solutions in a sentence) | ||
| - E.g. instead of stating a LO like "Apply faceting operations to charts", | ||
| we would say "Apply the `facet` method to group data into multiple subset | ||
| based on the unique values of a variables. This is helpful when we want to | ||
| compare subsets of data in a more explicit manner than by e.g. | ||
| coloring the points according to their value/group." |
There was a problem hiding this comment.
Maybe LOs are split into two different types? "Apply" where the solution is the code and "Understand" (or higher blooms) and the solution there would be the same as the "rationale/why".
| - I wonder if it would be convenient if these were somehow linked to the sections in the chapter | ||
| that describe each LO in more detail. | ||
| It seems like it could easily become redundant with the ToC, | ||
| but a reminder to students when they read each section | ||
| that they have just "completed" a LO would maybe be helpful | ||
| both to feel that they are making progress | ||
| and to remind them about what was important in the section they just read? | ||
| - Maybe a note or even a progress indicator in the margin? | ||
| "Learning objective N completed" / "Chapter 20% complete" / "You just learned that..." | ||
| There could also be exercises after each LO, more on that later. |
There was a problem hiding this comment.
This is more of a later stage addition. We can initially explore if there is something in Quarto natively. If not, make an issue and revisit if we want to make a custom solution.
The main idea here is to structure the content so that it follows the LOs closely. We want to make LOs as modular as possible with the constraint that the formatting of the book should not reduce the quality of the content/LOs
| 2. Required readings? It's somewhat odd to have this in a textbook | ||
| the same way I now have it in the lecture notes. | ||
| But I would like to somehow | ||
| guide the learner toward additional resources | ||
| that the can use to complement what we are teaching, | ||
| e.g. a more in depth chapter/section in "Fundamentals of data visualization". |
There was a problem hiding this comment.
Maybe this can be combined with the challenging section? Is a "deep dive" necessarily challenging? If so, they could all link to the end of the chapter where the challenging section is. When it is more of a reference, it is not challenging... To start, let's try out separate Deep Dive callouts, not related to challenging section.
| - I think this is ideally something like a problematic chart or statement | ||
| illustrating how what students have learned up until this point | ||
| is not sufficient to solve the problem we are demonstrating, | ||
| so that they appreciate the need and importance to learn more. | ||
| - Maybe this could be the students studying a viz | ||
| to define the problem with applying only what we have learned so far | ||
| to solve the scenario we set up | ||
| (ideally a scenario with direct practical implications). | ||
| In general, I believe that learning to think about about what the problem is | ||
| given a certain (business) question is beneficial | ||
| instead of that we spoon-feed them the problem statement | ||
| and let them only think about the solution. | ||
| I like the idea of making them come up with as much as possible themselves, | ||
| then they are motivating why the chapter is important on their own | ||
| and there is less "convincing" needed. | ||
| - E.g. | ||
| > In the previous chapter we learned about | ||
| visualizing each row in a dataframe using a point mark | ||
| and best practices when mapping data to graphical objects/elements. | ||
| In the following chart we have followed these best practices | ||
| when plotting ...{insert some scenario}, | ||
| but it is not easy to extract the main takehome message | ||
| (or any info for that matter). | ||
| How can this be given that we followed everything we know so far? | ||
| Study the chart and think for yourself what has gone awry here | ||
| and write down a suggestion for what you would do to fix it. | ||
| {maybe provide a space in the book where they can write, | ||
| it's ok that it is not saved, | ||
| it just for them to formally put their thoughts into words | ||
| which makes it easier to remember}. | ||
| {show chart} |
There was a problem hiding this comment.
but why those flaws matter.
YES. 100% agree that thinking about what the downstream consequences of the chart issues are. WHY is it a problem?
I also like the improvement of the chart at the end and it is explained what actions .
The introductory problem should be limited to understand the limitations, and then the chapter introduces the solutions gradually. But also avoid disconnection between problem and solution. One mention of the problem can be more superficial (maybe in the beginning?), and then there is a deeper dive together with the solution. E.g. for overplotting, just say that we can't read the chart but not introduce terminology like "overplotting"/"saturated". The terminology comesi n with the solution.
| - Ideally, there would be some continuity between chapters in this narrative, | ||
| but I'm hesitant to make that a requirement at this point, | ||
| as I also want us to be able to use data from different domains. | ||
| The only way I see us connecting such widely different data | ||
| is through a narrative like that in labs 1-3, | ||
| where we go a bit quirky and very "out there", | ||
| which could be quite appreciated, | ||
| but I would also be OK with having a less humorous narrative | ||
| and the connection is only how the visualization principles tie into each other | ||
| but the data is more independent for each chapter. |
There was a problem hiding this comment.
Problem: Society is broken in some way, or less strong: there is some dissatifaction in this fictional country/city and different people have different ideas of how to solve it. Could be another planet studying our world, but doesn't have to be. Can also just be a fictional world.
The students will embody different roles in society. Each role will care about and prioritize different aspects, e.g. the public health officer will focus on the inequalities in the world and argue that solving those will bring .... , someone else could argue that a more bottom up approach is required where individuals first fix themselves which fixes society (meditation, psychology, personal development data), business people will take a free market approach (some financial data), environmentlist focus on climate data.
| - AI feedback where we provide an LLM | ||
| with both the student solution and the correct solution | ||
| and asks it to compare the two | ||
| and then help guide the student on the right path | ||
| similar to how an educator would do it | ||
| rather than giving the answer directly. | ||
| The LLM could ingest the content of the book, | ||
| with a higher fine-tuning weight on the current chapter/section maybe | ||
| (if a general one is not good enough)? | ||
| If this works well, | ||
| it would be fantastic | ||
| and students would essentially have access to a tutor right there in the textbook. | ||
| Both special made (Chat)GPTs and Khanmigo (an education bot from Khanacedmy) | ||
| could be promising candidates. | ||
| - I'm quite interested in this direction as it is a lower stakes environment | ||
| than labs/quizzes where students receive marks | ||
| and we could get useful feedback from students on how well it works. |
There was a problem hiding this comment.
Maybe not yet possible when the student creates a chart (too low performance), but possible for text questions where we can feed it a textual solution? This could work well if we give a bad example of the chart and the student write down the text solution so that the LLM can use the solution to simulate e.g. a board outcome.
This is a bit experimental, but let's try to make prototype.
Maybe could use the VL json schema from Altair and have the model critique that instead of the visual itself? Would have to test performance to see where it is better.
| - One thing I wonder is if there is value in that the chapter design evolves over time | ||
| akin to how levels in video games don't follow quite the same pattern | ||
| even if there are some similar elements like boss fights etc. | ||
| Is there value in introducing this novelty to make students stay engaged? | ||
| I think learning apps like Duolingo also have slightly different level design | ||
| although not as extensive as in a good video game. | ||
| - I'm curious about this, | ||
| but it is an "out there" idea I would say | ||
| and we would be on uncharted waters | ||
| if we start to evolve the book structure | ||
| as the story progresses. | ||
| I think it sounds fun, | ||
| and someone must have written something about this idea in the past | ||
| so maybe we can find a few academic articles to learn more | ||
| if it is considered an effective learning strategy. | ||
| - Maybe we can spend a little time looking at the idea, | ||
| but I don't want this to be a time sink up front | ||
| (although the more I think about it the more it enthralls me =p). |
There was a problem hiding this comment.
I wonder if there is a danger of making things addictive or engaging without actually enhancing the learning. Maybe if the gamification is intrinsically tied to gaining a deeper understanding.
Good point that we should have in the back of our heads when designing.
Maybe some novel elements OK later as students get familiar with the book, but not complete chapter revamp since that would be confusing.
Maybe present LOs like a skill tree? Then students would be able to overview what is in the course and how they make progress as they complete exercises.
Some indication of which exercises are more challenging (points? label?)
Server would be really nice from tracking perspective to be able to demonstrate the impact on the book and how students use it. Plausible.io/gogole analytics is another option here but they are not as granular
| - Should we enable comments on the entire book? There are technologies such as `hypothes.is` | ||
| that are easily enabled in quarto | ||
| and allows student to highlight sections in the book and take notes. | ||
| They can even opt to make their notes public to other students | ||
| and have discussions in the comments | ||
| (maybe preferential and more direct vs having discussion on slack?), | ||
| but there would be no notifications and we are splitting discussion into two avenues. | ||
| Maybe more a space to exchange reflections on the content? |
There was a problem hiding this comment.
Needs critical mass to be interesting. Unlikely is not mandatory? Useful outside MDS.
Unsure how it works when the content changes...
| - Should we have a "how to use this book" chapter? | ||
| Or even lay out the philosophy behind some of our choices? | ||
| Maybe at least a short version in some type of foreword, | ||
| but I don't want to students to have to read a manual-like compilation |
There was a problem hiding this comment.
The suggestion here is to create a diagram/visual guide of the motifs/callout boxes in the book. An "IKEA" manual rather than a text heavy one.
Ideally, the elements of the chapters would be self-explanatory and we don't need this section.
| I want to be able to tell students that | ||
| if they go through all the exercises in the book (and the labs), | ||
| then they are well-prepared for the quiz | ||
| (without me having to give out any worksheets | ||
| and maybe even without having to give out a practice quiz, | ||
| but that's less important). |
There was a problem hiding this comment.
This will also serve as additional motivation for students doing the exercises and for them not peaking at the solution right away. So we have this working in our favor in addition to making engaging exercises. I'm imaging that student do the exercises once when they go through the chapter as part of the lecture, and that they can then also practice them repeated times for the quiz.
The majority of the quiz will be based on the exercises in the book, but there might be a few questions that are not (ideally there would be zero and the harder questions are more about linking and applying several concepts from the exercises to answer a practical scenario (rather than some obscure detail that's in the text but not in an exercise)).
joelostblom
force-pushed
the
main
branch
4 times, most recently
from
2b68f7b to
6b00da4
Compare
close #4