Google - Creatability Website Design + Development

Creatability is a set of experiments made in collaboration with creators and allies in the accessibility community to formulate a more democratic online experience. There are over 56 million Americans living with a disability, and many of them don’t use digital tools because the tools available don’t cater to them.

We partnered with Google Creative Lab to explore just how far creativity can go with web and AI technology, be it through music, drawing, and more. We’ve created the Component Library, an open-source code with tutorials for makers to start their own projects using four unique features: Keyboard, Body Synth, Clarion Lite, and Sampler.

Each experiment we created utilizes an AI tool called Posenet that tracks head and body movement. We then connected that movement to various instruments in Tone.js. With this combination, anyone with limited mobility can think with their minds and create with their bodies.

We respect your privacy. That’s why none of our features record images and nothing is sent to Google servers.

2x Webby Winner - 2019: Features & Design, Best Use of Machine Learning

We’ve created a music keyboard that you don’t just have to play with your fingers. Use your body and face as your musical canvas. Just turn on your webcam and track a point on your body to start playing. If you want to stick to the traditional keyboard, plug in your MIDI and start customizing your instruments and scales.

Body Synth gives you a hand free music-making experience. This new feature transforms movement into sound, with each part of your body playing a different note or sound. Your movements can be small or big, and you can change the chord and instrument through voice recognition. Just let the AI know what’s on your mind and the technology will follow suit.

We created Clarion Lite in collaboration with Open Up Music to enable music-making through simple shapes on your screen. Our new feature offers many different patterns for you to choose from. Once you’ve selected your pattern, either use your mouse and keyboard or turn on your webcam to use your body as an instrument.

Sampler gives you a series of sounds to choose from to create music using your body, face, mouse, or keys. If what’s available isn’t what you’re looking for, you can import your own samples and control other instruments using your MIDI controller. Sampler was made by Use All Five, Kyle Phillips, Claire Kearney-Volpe, Luisa Pereira, Yotam Mann, and Google Creative Lab. Special thanks to friends at Henry Viscardi School at The Viscardi Center, Tech Kids Unlimited, and ADAPT Community Network.