- Spring 2020, Wednesdays 1:10-2:25PM (Barnard-only), Fridays 2-5PM
- Locations:
- The Media Commons at 370 Jay Street, Brooklyn NY
- The Movement Lab at Barnard College, Manhattan, NY
- Detailed schedule for Times and Locations
- Instructors: Mimi Yin, [email protected], NiNi Dongnier
- Office Hours Signup
Course Structure | See introductory description below.
Over the course of the semester, we will cover 3 topic areas that correspond to 3 parameters of choreography: Pathways, The Body and Space. Topics will be introduced through movement-based improvisation exercises. Computational strategies will be examined through code examples. For each topic, students will create small movement studies with the aim of fully exploring how technology "intervenes" and changes the choreographic process.
The class will culminate in a showing of student work. The final project will take the form of a live performance.
This class runs in conjunction with a course in the Barnard/Columbia Dance Department and will comprise of students from both NYU and Barnard/Columbia.
Because course topics are organized around movement concepts, technical topics will be introduced and built upon week to week.
- What do cameras see? What don't they see?
- Skeletons, Depth Maps and Contours
- Position versus speed of movement
- Scaling and Mapping values
- 2D Position tracking
- 2D Projection mapping
- Computational approaches to choreographing pathways of motion
- Computational approaches to dividing and defining space
- Modes of interaction
"Soft Skills" we will practice:
- Sketching (as in drawing on paper) choreographic ideas.
- Designing and defining rule sets for movement.
- Designing and defining rules of interaction.
- Deconstructing choreography into parameters for code.
- Extracting choreographic ideas from code.
Each topic will be introduced through paper-based drawing exercises and code examples. Google Drive Link
- Select chapters from The Viewpoints Book | PDF | NYU Library
- Select chapters from Dynamic Alignment Through Imagery | Google Books | NYU Library
- Forsythe Technique Videos on Youtube
- Hello!
- What is interaction?
- Introduction to Computation and Interaction through Movement
- Cartesian Plane, Linear Motion, Drawing with the Body with the Kinect
- Resources
- Examples
- p5.js Code! Tutorials
- Introduction
- Drawing
- Animation (Video 2.6 is optional)
- Total Time: ~2.5 hours
- Continue working on your opening choreography in code by animating movement. Make something at least 30 seconds long. Get as far as you can get. Post a link to your sketch here.
- Why are we here?
- What is computational thinking? What is choreographic thinking?
- What are different modes of interaction?
- What are the parameters of choreography?
- How is code an expressive medium?
- Walking improv.
- 1-on-1 walking duet.
- Can a pathway be expressive?
- How do you choreograph pathways in drawings?
- Espaece
- GLOW - Chunky Move, 2006 -
- Drawing Expressive Lines
- Drawing a Trail
- In the Kinect Azure folder of this repository:
- Skeleton_Switch_Joints
- Skeleton_Draw_Lines
- Skeleton_Draw Trails
- Linear Pathway
- Linear Pathway - Deconstructed
- Read and complete Chapters 0 and Chapter 1.0-1.2 of the Workbook
- Print it out and complete the exercises by hand. Staple it. Put your name on it. Submit it next Friday.
- For background on approaches to composition and movement improvisation:
- What is composition? Read chapter 2 of Viewpoints.
- Watch Forsythe Technique Videos: They range from 10s to 1 minute. Very short!
- p5.js Code! Tutorials
- Interaction (~1.5 hours)
- Duplicate your sketch and keep working on your 30 second opening choreography in code. Try finding a way to add in a loop. Get as far as you can get. Give it a real title! Post a link here.
- What can a line express?
- Interacting with a static line
- Interacting with a moving line
- Linear Pathways
- Create a duet or trio that involves at least 1 moving line() that will be projected on the floor. Design interactions with the line(s) with drawings. Specify facing, direction of movement and speed. Specify the parameters of movement you are working with to develop your choreography.
- Timespan: 30-90 seconds.
- Your sketch should be 1920x1080.
- Consider these parameters.
- Example of Line class to make lots of lines easily
- Post your drawings and a link to your sketch here: Assignment Page
- Examples from class:
- p5.js Code! Tutorials
- Modularity and Reusability (~30 minutes)
- Object-oriented Programming (~35 minutes)
- Workshop Study No. 1: Linear Pathways.
- When does random() feel most random?
- Random facings.
- Random facings through sound.
- Trios: Activating random spots.
- Interacting with random spots on the floor.
- Complete Chapter 1.5 of the Workbook
- REVIEW Functions and Returns and see if you can figure out how to simplify the update() function so that it's only updating a single pair of x,y coordinates for both lines. Post your (attempt at a) solution here..
- NEW p5.js Code! Tutorials
- Repetition and Duplications (7.1-7.3, 16.4) (~45 minutes)
- What does a circle represent to you?
- Random improv.
- Circular floor work.
- Coding Tutorials
- Video
- Circular Pathway Examples
- Choreograph a 30 - 90 second generative circular pathway. Create a floor choreography that interacts with that circular pathway.
- Take into consideration the dynamics and timing inherent to circular motion.
- Make use of the whole body and it's ability to move circularly and linearly.
- Discuss and diagram choreographic possibilities for how to interact with your pathway.
- Post a link to your drawings and code you will demo with here: Assignment Page
- Add to your Line() class so that you can assign start and end framecounts to control when the line moves and when it doesn't. Post a link here.
- Workshop Study No. 2: Circular Pathways.
- What makes a body recognizably human?
- What do computers see when they see a human body?
- How would you describe the human body to an alien?
- The body as a constellation.
- Hello Kinect + Kinectron
- Hello Kinect Skeleton
- Resources
- Read Chapter 7 of Dynamic Alignment: Google Books | PDF
- Watch Forsythe Technique Videos: They range from 10s to 1 minute. Very short!
- Complete sections 2.0-2.2 of Chapter 2 of the Workbook
- After you've done the reading and workbook, make a crazy skeleton. Start with the Skeleton_Bones example. Put it up in the p5 editor and point the IP address at 'localhost' and post a link to it here.
- Bring a piece of clothing that alters (confines, expands, distorts) your body to class next week. Look here for inspiration:
- Danse Des Masques
- Martha Graham: Lamentations and Satyric Song
- Issey Miyake and Irving Penn
- David Byrne: Stop Making Sense
- Rei Kawakubo: Shifting your center of gravity. Scenario by Merce Cunningham
- Example from class: Array of Line() objects
- Replace lines 48-53 of the example above so that you can update and display individual lines by name. What else do you need to add to make that work?
- Post a link to your sketch here
lines[GEORGE].update();
lines[GEORGE].display();- How can we re-architect the body?
- How can a body choreograph interaction? How can interaction choreograph a body?
- Clothing improv.
- Statue improv: Duets and Trios.
- Driving interaction through speed and distance.
- Examples
- averageYs
- Get all the code at once in the Kinect_Azure folder of this code repository:
- Create a joint dance for up to 3 dancers. See guidelines. Post write-up and code here.
- Workshop The Joint Dance.
- Introduce webcam based skeleton tracking with ml5.js and Posenet
- Introduce phone sensors
- What are we doing with ourselves for the next 6 weeks?
- Building a Choreography Sketchbook
- Webcam Based Skeleton Tracking
- Phone Sensors
- Recording of Class
- ml5.js PoseNet
- ml5.js PoseNet Video Tutorial | Code
- Debugging iOS
- Debugging Android
- How to screen record: Mac | iOS | Windows | Android
- Create a joint dance for up to 3 dancers. See guidelines.
- Record the dance.
- Record the screen.
- Record the sound.
- Throw it all into a single video.
- Post the final video to your sketchbook here. )
- Hello Projection Mapping: Video Tutorials | The Set-up Tutorial | The Code Tutorial | p5 Tutorial
- Processing Base Examples
- How does the webcam define space?
- How does the laptop screen define space?
- What does it mean to interact with these virtual spaces through movement?
- How do these spaces re-define the relationship between performer and audience?
- Notes
- Recording of Last Hour of Class
- Critique The Joint Dance
- Chair improv
- Preview new technologies
- Streaming Skeleton data between remote performers and audience members
- Face-Tracking
- Hand-Tracking
- WebRTC for building your own live streaming interface
- Embedding Youtube LiveStream in your sketch | How to use Youtube LiveStream.
- Play with the examples above.
- Please post both your written response and chair dance study by 10AM Friday so that we have enough time on Friday to review and respond to the material before class. Before class, I will ask you review everyone's examples of online performance and chair studies as well as leave written responses on at least 3 other people's work.
- Write a short response to the following examples of online performance. (~3-5 sentences about each example.) What's the angle? Would it be possible in a live theater set-up? Would it be the same performance? What would be lost? Add a 4th of your own to analyze. It doesn't need to be a dance performance. It doesn't even need to be good! Just thought provoking about the medium. Post a link to your writing response here.
- Chair Dance Study: Create 3 movements where each of the 6 parameters we identified in class is specified. Write-up your parameters in a document and link to it on the assignments page. Practice each movement until it feels like a movement. Connect them together in a pattern. Video yourself repeating the pattern 5 times. Please post your video and parameters here.
- Think about who you would like to work with on the final project. (I will send out link to spreadsheet.)
- Discuss online performances
- Critique chair dances
- Introduce new technologies
- Streaming Skeleton data between remote performers and audience members
- Face-Tracking
- Hand-Tracking
- WebRTC for building your own live streaming interface
- Embedding Youtube LiveStream in your sketch | How to use Youtube LiveStream.
- Resources:
- Notes from "What is Online Performance" discussion.
- Recordings of intro to tech and movement research exercises: Shared Screen + Speaker | Gallery-only View
- Watch video tutorials
- If you're feeling good about the tutorials, try to combine Streaming Skeleton with either Face-Tracking or Hand-Tracking. Post a link to your Glitch project.
- Generate 60 minutes of movement research. See details here. Post your video here along with a description of your 4th technique by next Thursday evening. Please comment on all of the videos before class on Friday.
- Start to collect source material this week. Divide and conquer but cover all of these areas: Ideas for physical set-up. Explorations of camera angles. Sound, visuals, video, text. Drawings of choreographic ideas and videos of movement research. Stick them all in your folder on Google Drive. (I will send out an invite to a shared folder.)
Over the next 3 weeks, in groups of 2-4, create an online dance performance that includes at least 1 performer and 1 audience member. You may make use of Zoom. There are no limitations on duration.
- Recording of Class
- TensorFlow Face and Hand tracking
- Examples: Face | Hands
- Written Tutorial
- Tutorials on drawing in 3D with WebGL: Written | Video: Just watching 18.1 is enough.
- Sending Still Frames
- Flesh out your Brainstorm Google Slides for review by Monday evening. Make sure your slides are in this folder with your names as the title of the document.
- Move your project forward for next week. Ideas on how to do that. Post your materials for review by Thursday evening.
- Please post your material for class on Friday here. If you will be presenting pre-rendered video, please post by Thursday evening so we have time to watch it before class on Friday.
- What are the parameters of space?
- What are all the ways to define a space?
- What are all the ways to interact with a space?
- What are all the ways to divide a space?
- What are all the ways to define a boundary?
- What's the difference between a division and a boundary?
- Interacting with Divisions / Boundaries / Areas
- Complete Chapter 3.0-3.3 of the Workbook
- Watch: Jiri Kylian | NDT : Sweet Dreams
- What are all the ways in which Kylian is working with space? How are the spaces defined?
- Why angular spaces? Why sharp edges? How would it be different if he used more traditional spotlights?
- Are defining and interacting with space the same thing?
- Does one preclude the other?
- Interactive Divisions, Boundaries and Areas
- Ways to calculate divisions / boundaries / areas | Tutorial
- Watch Videos 1-3: Transformations | Video 9.22: Drawing Custom Shapes
- 3 Ways To Divide
- 3 Boundaries
- Semi-exclusionary Square
- Hello Projection Mapping: Video Tutorials | The Set-up Tutorial | The Code Tutorial | p5 Tutorial
- Design an interactive space as a storyboard and in code. | See sample.
- Fill out this survey about who you want to work with by Monday Mar 30: ITP | Barnard/Columbia
- Brainstorm what you'd like to explore in your final project. Please see guidelines and example.
- Collect source material this week: Sound, visuals, video, choreography, text, diagrams, drawings, storyboards and stick them in your folder.
- Prepare material to show in class: 3 choreographic ideas. 1 sketch. (You can make more than that.)
Over the next 4 weeks, in groups of 3-4, create a movement-based interactive work that explores one or more modes of interaction and one or more of parameters of choreography from this course. The work can take the form of an installation or a performance. Installations can be time-based or persistent with no definitive start or end time. The only requirement is that the choreography must be interactive, meaning the performers are making choreographic decisions in real-time. This doesn’t mean there is no element of pre-set choreography, however there must be room in the choreographic structure for individual decision-making. You will have a maximum of 10 minutes to show your work. Each group will consist of 1-2 students from ITP and 1-2 students from Barnard.
This course re-conceives interactive media as a form of choreographic intervention. Instead of asking how moving bodies can control media, we will ask how rules of interaction can structure the composition of a dance.
To accomplish this, the class facilitates a semester-long collaboration between ITP students and dancers from the Barnard/Columbia Dance Department. Choreographers will learn to apply computational thinking to choreography and creative coders will learn to apply choreographic thinking to computation. To whatever extent possible, we will attempt to embody code.
Using computer vision and visual media, we will look at directing both how people move (quality of movement) as well as where they move (pathways and spatial relationships).
We will evaluate the strengths and weaknesses of the various sensing technologies available to us today. How wide is the gulf between what we can see and feel (strength, hardness, contortion) and what a computer can see and interpret (locations, contours, velocity)? Class time will be split between movement exercises, playing with examples and deconstructing code.
The class will culminate in a final showing of student work.
All classes will take place at NYU with a separate weekly technical lab for ITP students.
The course is intended both for anyone looking to deepen their practice in working with movement-based interaction regardless of previous experience with movement technique or programming.
As a result, there is no pre-requisite for dance and no pre-requisite for code.
- 40% for showing up (on time!) and participating with curiosity and enthusiasm.
- 15% for each topic study. (Pathways and Body)
- 30% for the final project.
- More than 2 unexcused absences qualifies you for a failure.
- 2 lateness of 15 minutes or more qualifies as 1 unexcused absence.
Please see ITP's statement on Pass/Fail which states that a "Pass" is equivalent to an "A" or a "B" while anything less would be considered a "Fail".
We will have weekly assignments that are relevant to material from the previous class. These assignments are required and you should be prepared to show/talk about them in class. It is expected that everyone in the class will create and maintain a blog for their assignments.
Attendance is mandatory. Please inform your teacher via email if you are going to miss a class. Two unexcused absences is cause for failing the class. (An unexcused lateness of 10 minutes or more is equivalent to 1/2 an absence.)
This class will be participatory, you are expected to participate in discussions and give feedback to other students both in class and participate with their projects. This (along with attendance) is 40% of your grade.
Class will culminate with final projects. You are expected to push your abilities to produce something that utilizes what you have learned in the class that is useful in some manner to yourself or the world. This will comprise 20% of your grade.
Plagiarism is presenting someone else’s work as though it were your own. More specifically, plagiarism is to present as your own: A sequence of words quoted without quotation marks from another writer or a paraphrased passage from another writer’s work or facts, ideas or images composed by someone else.
The core of the educational experience at the Tisch School of the Arts is the creation of original academic and artistic work by students for the critical review of faculty members. It is therefore of the utmost importance that students at all times provide their instructors with an accurate sense of their current abilities and knowledge in order to receive appropriate constructive criticism and advice. Any attempt to evade that essential, transparent transaction between instructor and student through plagiarism or cheating is educationally self-defeating and a grave violation of Tisch School of the Arts community standards. For all the details on plagiarism, please refer to page 10 of the Tisch School of the Arts, Policies and Procedures Handbook, which can be found online at: http://students.tisch.nyu.edu/page/home.html
Please feel free to make suggestions to your instructor about ways in which this class could become more accessible to you. Academic accommodations are available for students with documented disabilities. Please contact the Moses Center for Students with Disabilities at 212 998-4980 for further information.
Your health and safety are a priority at NYU. If you experience any health or mental health issues during this course, we encourage you to utilize the support services of the 24/7 NYU Wellness Exchange 212-443-9999. Also, all students who may require an academic accommodation due to a qualified disability, physical or mental, please register with the Moses Center 212-998-4980. Please let your instructor know if you need help connecting to these resources.
Laptops will be an essential part of the course and may be used in class during workshops and for taking notes in lecture. Laptops must be closed during class discussions and student presentations. Phone use in class is strictly prohibited unless directly related to a presentation of your own work or if you are asked to do so as part of the curriculum.