Skip to content

interaction-lab/MoveToCode

Repository files navigation

MoveToCode: An Augmeted Reality Visual Programming Language

MoveToCode is an open source augmeted reality visual programming language (VPL) created by the Interaction Lab led by Maja J Matric. The VPL is used to study student kinesthetic curiostiy. The VPL can be thought of similar to Scratch in 3D.

MoveToCode is built for the Microsoft Hololens 2, as well as ARCore supported mobile devices, using the Mixed Reality Toolkit. Currently it is set up to work with the Mayfield Kuri robot but can be modified for any ROS supported robot via ROS# and rosbridge. A virtual, high fidelity model, Kuri robot is also currently availble with limited capabilities.

Doxygen documentation can be found here: https://interaction-lab.github.io/MoveToCode/html/index.html

Videos/Gifs

  • MoveToCode + a robot tutor using the Hololens 2 - this is when students coded text based exercises. We have since moved to phone based AR coding a robot (video coming soon).
  • MoveToCode mobile captured from the iPad's point of view. In pairs, students first build mazes out of physical pieces of paper. Once the mazes match, students program the smaller robot to complete these mazes. Students must move around to complete the above tasks presenting programming in a physically situated environment.

Papers and Study Commits

M2C Mobile Elementary Pilot next day outreach changes: 88c7fcbeee11265823d3d979b8108003c1b9a505


M2C Mobile Elementary Pilot: b3780ab4b00eabf99acdcc934e9d9d88c6489ddb


Augmented Reality Appendages for Robots: Design Considerations and Recommendations for Maximizing Social and Functional Perception (Commit:d459dcc9a1fe267fa805064aabaf4a51934ca3f1 )

@inproceedings{goktan2022augmented,
  title={Augmented Reality Appendages for Robots: Design Considerations and Recommendations for Maximizing Social and Functional Perception},
  author={Goktan, Ipek and Ly, Karen and Groechel, Thomas Roy and Mataric, Maja},
  booktitle={5th International Workshop on Virtual, Augmented, and Mixed Reality for HRI},
  year={2022}
}

Commit for all papers below this: 0765f1d1ab46373f1bc1be90d9d5b07fafc2a533

Kinesthetic Curiosity: Towards Personalized Embodied Learning with a Robot Tutor Teaching Programming in Mixed Reality

@InProceedings{10.1007/978-3-030-71151-1_22,
  author="Groechel, Thomas and Pakkar, Roxanna and Dasgupta, Roddur and Kuo, Chloe and Lee, Haemin and Cordero, Julia and Mahajan, Kartik and Matari{\'{c}}, Maja J.",
  editor="Siciliano, Bruno
  and Laschi, Cecilia
  and Khatib, Oussama",
  title="Kinesthetic Curiosity: Towards Personalized Embodied Learning with a Robot Tutor Teaching Programming in Mixed Reality",
  booktitle="Experimental Robotics",
  year="2021",
  publisher="Springer International Publishing",
  address="Cham",
  pages="245--252",
  isbn="978-3-030-71151-1"
}

Adapting Usability Metrics for a Socially Assistive, Kinesthetic, Mixed Reality Robot Tutoring Environment (pdf)

@inproceedings{mahajan2020adapting,
  title={Adapting Usability Metrics for a Socially Assistive, Kinesthetic, Mixed Reality Robot Tutoring Environment},
  author={Mahajan, Kartik and Groechel, Thomas and Pakkar, Roxanna and Cordero, Julia and Lee, Haemin and Matari{\'c}, Maja J},
  booktitle={International Conference on Social Robotics},
  year={2020}
}

External Dependencies

Versions of packages are pushed directly to the repository to avoid versioning issues, licensing permitting.

Tested Platform Support

The following are currently tested versions of required software. Higher versions of each software may work but have not been tested.

Licensing

MoveToCode is open source under the MIT License

Credits

Questions or Contributions

For any questions, you can contact Thomas Groechel at [email protected]. Feel free to fork for your own projects and/or submit pull requests.