Music Composition and Production Using Machine Learning
Everyone is aware of the practical applications of machine learning: predicting stock prices, classifying fraudulent activity, but can machine learning be used as a tool in the creative process? Googles new open source research project, Magenta, is proving that you don’t need to be human to be creative. Created by the Google Brain team, Magenta is a python and JavaScript library that allows you to generate art and music utilizing machine learning models. It leverages TensorFlow, an open-source software library that focuses on training and inference of deep neural networks.
Often times musicians are faced with creative writing blocks and don’t know where to even start when creating a new song from scratch. Struggles with melody writing and artists finding their own unique sound are far too prevalent in the music industry. Googles Magenta Studio, built off of the Magenta package, can help combat some of these challenges in music composition and production.
Magenta Studio
Magenta studio is a collection of music plugins for music generation available both as a standalone application and as a plug in for Ableton Live, a music sequencer and digital audio workstation software. The five plugins are: Continue, Generate, Interpolate, Groove, and Drummify. Each plugin relates to a different machine learning model.
Each has a dashboard, standalone dashboards shown below, that allows you to alter existing files, merge existing files, or generate completely new ones. You just choose your output or input location, the number of variations you want generated, and the the temperature, or randomness of the created files (a higher temperature creates a higher variance, meaning that the files that are created will vary more widely).
Generate comes in handy for artists that are experiencing a creative block and are in need of some inspiration. This plugin generates a 4 bar drum beat or melody with no input necessary. It uses a Variational Autoencoder (VAE ) that has been trained on millions of melodies and rhythms. Using latent sampling, Generate learns a summarized representation of musical qualities and chooses a random combination of the summarized qualities to produce a music file.
Continue uses predictive modeling to extend music files by up to 32 measures, creating an entire melody or creating variations on themes. It utilizes a recurrent neural network (RNN) model. RNN models contain loops, allowing for information, or in this case, notes, to be passed from one step in the network to the next. A special kind of RNN is used for this plugin, called Long Short-Term Memory (LSTM) that allows for regulation of the notes that are passed.
Interpolate merges two music files or can be used to create a smooth morphing between them. It uses a VAE in which similar musical patterns are clustered together. Each file is mapped out and interpolate draws a line between these positions in order to return a new, merged music file.
Groove takes in a drum pattern and outputs a music file representing the “feel” of a drummers performance by adjusting the timing and velocity. This model was created using neural networks that were trained on over 15 hours of real drummer performances.
Drumify, our fifth plugin, creates a groove, or drum rhythm, based on input. Input can be either a drum pattern or a melody and the model is trained on the same drum dataset as in Groove. A rhythm is extracted from the input and the model turns it into a groove.
Conclusion
The Magenta Studio is just one of the cool creative tools built on the Magenta library. Other projects include: the NSynth Sound Maker for combining instruments to create new sounds, the Latent Loop and Beat Blender for sketching out melodies or drum beats, and my personal favorite, the Lo-Fi Player for building your own custom lo-fi channel.
