Machine learning, body tracking and creativity

The Google Accessibility Blog has a collection of fascinating articles including one by Claire Kearney-Volpe, a Designer and researcher who made Creatability. This is a set of creative tools that can be used with any input device and have encouraged creation by a group of disabled users. The YouTube video about Creatability has captions and an expanded audio-described version of this video lives at:

The various experiments use different input methods from head tracking to switch access. The site suggests that you explore ways to ” make music by moving your facedraw using sight or sound, and experience music visually. “

The system uses Posenet  combined with Tensorflow.js allowing a user to move in front of a webcam and create fun things within a browser – no downloading of programs or storage of data on other people’s servers. The code is open source and can be found on the Google Creativity Lab Github account.

There are lots of experiments that have been shared on the Creatability website with support and further resources.