development of a multipurpose, collaborative PDF reader for real-time sharing of navigation and annotation on sheet music
Networked Music Performance (NMP) and overall Music making in cooperative environments is spreading quickly.
During the last few years we have seen many institutions adopting hardware and software technologies enabling musicians to play music in an almost seamless experience in terms of audio and video.
A dimension that is usually missing from this environment is the complementary dimension of interaction on the score, which is in fact, usually physically shared in a traditional learning context, as for example during a piano lesson, studying side by side with a teacher.
A possible solution to the issue is an application able to display music scores and provide interaction tools, shared between the participants, acting as if the musical document was a single physical entity. In terms of immersion and accessibility, this might prove more effective than expected in improving the feeling of presence between the distance learners.
Features range from brushes to synchronized page turning, layers for orchestras to stack scores in a single document and allow a director to have a single view in which annotations can be set on different groups or instruments.
Fig. 1 Concept example of interaction between two different app users connected at realtime
Having an interface that emulates the score’s presence provides a key physical feature during the preparatory networked music interaction: pointing on the score, feature that a traditional camera-monitor conference setup cannot provide most of the times.
IMPLEMENTATION
A key role in the implementation of this application comes from the Conservatorio di Musica “Giuseppe Verdi” Milano (ConsMi) and the Intermusic project, acting as hosts in this research and providing the needed material and musicians to test and evaluate the development and design of the app’s first version.
Intermusic
In particular Intermusic is the place in which the idea for this project was born. Named after the concept of ”Interactive Environment for Music Learning and Practicing”(Erasmus+ funded 2017 – 2020), aims to create an online shared Platform for that branch of distance learning dedicated to music teaching and practice that will enable modelling and sharing the best training practices for musicians as well as joint courses and online projects.
The two main objectives of Intermusic are:
- support online synchronous singing, instrumental solo performance and chamber groups classes where teachers and students interact in real time in an audio-video communication environment of the highest quality
- use and integrate methodologies and technologies of Blended Learning and Virtual Learning Environments in music higher education3.
Therefore, the nature of this project is strongly related with Intermusic and its intellectual outputs.
Guidelines
Development planning followed the guidelines below:
- Open Source: further development after Version 1 will be carried and branched freely by anyone who wants to contribute to the project through a public repository system
- Extendable: being open source will allow developers to create their own versions and most importantly, portings to other platforms. In fact, the first version targets the Android platform, and a Windows version is being planned, but the open source code will show anyone interested in porting the application how it is structured inside and they will only have to translate it to the new platform’s code.
- Multipurpose and modular: the features in the application follow a modular design that enables different uses of the software depending on the circumstances.
Users will be able to set up the modalities of interaction, for example:
-
- Performers will be able to open the same score in order to interact on it while rehearsing together.
- Directors will be able to open multiple scores at once and create annotations on single parts or groups of both locally and remotely connected performers.
- Teachers will be able to interact with students while providing feedback with the help of add-ons like tap tempo metronomes and solfege helper tools.
- Lightweight: the application features a very lightweight PDF rendering library, available for many platforms (Windows, MacOS, Android), which is free for open source projects and performs outstandingly well even on low end devices. It was in fact clear since the planning phase that one of the key features in this context would have been performance: avoiding fluidity issues on navigation and overall interaction on the document is vital in order to preserve the feeling of presence.
STAGE OF DEVELOPMENT
The first version of the application is in a pre-alpha stage at the time of writing this article and undergoing further development and design choices. The first version to be published will be on the Android platform since Android tablets are the most diffused and low-price interfaces that will surely enable the widest range of users.
A design and development document will be released with the alpha version and its open source code, its objective is to help creating a community of users and developers that will contribute to the growth of the application and its tools.
Before the code and the documents are published, internal tests with the Intermusic team will document interaction and feedback on the application in order to provide use cases and overall aspects of how the application is perceived.
FUTURE POSSIBILITIES
This first stage of development is crucial to fix guidelines and concepts for further development.
While the main Android branch is expanded with new features and optimizations, it is important to involve more users to use it and join its development, so that the following versions could be shaped after the needs of the community. For this, a forum and a website will be set up all together with all the documentation and code for the beta distribution.
Overall, the objective for the future is to support different platforms with cross-platform connectivity and modularity in order to keep the software as open to possibilities as attainable.