Sound-to-notation app makes noise in the music industry

It is nonetheless a crucial process: many musicians would agree that notation makes for better music. For these musicians, something as revolutionarily easy as a Google Translate for music notation would be a dream come true, as Prof. Sven Ahlbäck of Swedish company DoReMir Music Research AB underlines.

‘Musical ideas, songs or tunes, can suddenly pop up in your mind, and when it happens it is important to catch the idea quickly before it vanishes. To notate music by hand or on computers has always been a slow process that takes time to master and requires you to analyse the syntax of the music, to recognise time signature, tempo, key, etc. Being able to go straight from musical idea to music notation would mean important time-savings for many musicians.’

They need to look no further: ScoreCloud, a virtual notation assistant available as a PC/Mac software and an iOS app, is now available for download under both free and monthly paid plans, thanks to support under the SME Instrument.

You can think of it as a virtual assistant like Amazon’s Alexa or Apple’s Siri, spiced up with an instant-translation feature: sing or play an instrument, and ScoreCloud will notate the music for you, from which point you can develop your ideas into arrangements and composition within the app. Thanks to the my.scorecloud web service, you can even share your notations with the world.

‘For many years, I have been researching how people perceive musical structure and found that this research could be used for building a computer software that makes music notation automatically,’ Prof. Ahlbäck explains.

‘This is how ScoreCloud was born, so as to help all musicians – from beginners and music amateurs to professionals – who need to have their music played by someone else or want to develop it by means of music notation.’

Music complexity isn’t a barrier either. As Prof. Ahlbäck points out, ScoreCloud’s cognitive modelling mimics how we perceive music and tries to come up with a music notation that is not only accurate, but actually readable. It will differentiate between several voices and simplify the musical score, just like a human musician who notates music would.

Making ScoreCloud able to cope with this complexity is actually the very reason that pushed DoReMir to seek EU funding. ‘A major issue for automatic music notation has been to analyse polyphonic musical sound, in which more than one note sound simultaneously.

Just a few years ago the level of performance of polyphonic pitch recognition was far from sufficient to produce musical scores from common polyphonic instruments such as piano or guitar.

We wanted to see if, by using the most modern technology in combination with cognitive modelling, we could break this barrier to make automatic music transcription from polyphonic sound a reality. The EU funding gave us this possibility,’ says Prof. Ahlbäck.

Looking back, the polyphonic pitch transcription system developed under the DOREMIR project even exceeded initial expectations. Prof. Ahlbäck speaks of a performance setting a world record.

‘Beta tests resulted in people describing ScoreCloud as the ‘missing link’ between sounding music and music notation. We have also received interest from other software companies that want to make use of our technology,’ he enthuses.

Although the project was completed in September 2017, Prof. Ahlbäck and his team have “a long line of products” in the pipeline, including cross-platform apps targeting singers, songwriters or children who want to learn about music.

‘This technology makes it possible to analyse the content of music on a level never seen before, which in turn opens the door to applications in the likes of musical content search, music recommendation systems, etc. We are also currently releasing a musical message app, which musifies messaging. The possibilities are endless,’ Prof. Ahllbäck concludes.