The Possibilities of Emerging Digital Music Formats: Part 1 — An Introduction
Throughout music history, each new format for producing, performing, capturing, sharing, or consuming music has expanded the horizon of musical possibilities. But within the past 100 years, perhaps no other development has been as influential as the development of recording technology. As musicologist Mark Katz explains in his book “Capturing Sound,” in which he traces the history of recording technology and how it has changed the way people both make and listen to music, recordings do more than just capture reality — they also change it. He refers to this change as the “phonograph effect.”
Simply put, a phonograph effect is any change in musical behavior — whether listening, performing or composing — that has arisen in response to sound-recording technology. (Katz, 2010)
Revised in 2010, Katz’s history ends with the effects that web-based file sharing and the MP3 have had on music consumption, a topic more deeply explored several years later by Jonathan Sterne in his book “MP3: The Meaning of a Format.” Neither of these books, however, was published recently enough to include the range of new digital formats that have become available in the last decade through major technological advancements backed by both serious financial resources and rising consumer interest.
August 1, 2012, marked the launch of a Kickstarter campaign (Oculus, 2015) that ultimately raised more than $2 million in pledges to support the development of a new virtual reality (VR) headset by then-teenager Palmer Luckey (Kickstarter, 2016). Two years later, shortly after Luckey delivered the first VR headsets, Facebook acquired his company, Oculus, for $3 billion (Constine, 2014). On October 13, 2016, Sony Interactive Entertainment (SIE) released a head-mounted display called “PlayStation VR,” which a little over a year later had already sold more than 2 million units (Joyce, 2017). By that same time, the Oculus Rift and another VR headset, the HTC Vive, had both sold around 200,000 units each (Matney, 2017). All three of these VR headsets targeted the dedicated video gamer market. Recognizing this, Google’s Cardboard (2014), Samsung’s Gear VR (2015), and more recently Oculus’ Go (2018) offered cheaper, less functional VR systems that sought to entice more mainstream consumers to begin exploring what VR had to offer. According to an eMarketer study, there were at least 9.6 million active users of VR headsets in 2017 (Roettgers, 2017).
While virtual reality is undoubtedly the poster child for new interactive and immersive environments that are finding footholds in global entertainment markets, VR is not alone. On March 13, 2015, Google’s video-sharing service YouTube began supporting 360-degree videos (Hollister, YouTube’s Ready To Blow Your Mind With 360-Degree Videos, 2015). Facebook, not wanting to be left behind, updated its platform to support 360-degree videos several months later (Maher, 2015). Looking to influence the future of these technologies, Apple announced at its 2018 Worldwide Developers Conference (WWDC) the release of its AR Kit 2, which aims to make integrating augmented reality (AR) features into apps easier for developers (Apple, 2018). And although the technology is still very much in its infancy, AR already has acquired a substantial user base: the same eMarketer study indicated there were 40 million active users of AR-enabled apps in 2017.
All of these indicators point to a rapid pace of advancements in entertainment technologies, many of which consumers are adopting faster than creators can even provide content for. But even though these new environments open vast worlds of creative possibilities, revealing new and uncharted territories of artistic expression, most of the content that has already been created only mimics older practices.
Take Netflix’s VR App, which places the experiencer on a couch in a virtual living room with a large, flat-screen television on the wall accompanied by a virtual remote to navigate and watch Netflix’s content just as if they were sitting in their actual living room (Oculus, 2018). Contrast that with Dispatch (Oculus, 2017), a mini-series created by Ed Robles and starring Martin Starr. Each episode of Dispatch places the viewer inside the consciousness of police dispatcher Ted as he receives a stream of gut-wrenching emergency phone calls. The drama is conveyed almost entirely via the sounds that Ted hears through his phone, but as each scenario develops, ghost-like impressions representing Ted’s imagination of the scene begin to fill in around the viewer. Dispatch is an excellent example of how the art of storytelling can be enhanced and expanded through these new digital environments.
Yet while there is a growing body of documentation for these new tools that support the creation of new digital spaces generally, musicians looking to explore the new interactive and immersive digital formats have few examples from the 20th century to draw inspiration from.
There were — and undoubtedly more will be uncovered — isolated endeavors such as Morton Heilig’s 1962 Sensorama (Turi, 2014) (see Figure 1) and Ivan Sutherland’s 1968 Sword of Damocles (Virtual Reality Society, 2015) (see Figure 2) constitute some of the earliest experiments with mixed and virtual realities. Within the last ten years (2008–2018), however, numerous composers have released works that push beyond the familiar formats of musical experiences into uncharted digital territories. Yet studying and creating in these new formats can be challenging for two reasons: the works are often obscure (and thus difficult to find) and experiencing them requires equipment that may not be readily available.
These issues share the same root cause — that is, the ever-increasing rapidity of technological development. Analyzing the number of patents issued by the US Patent Office provides one illustration of how quickly new technologies are emerging (Atlantic Re:Think, 2015). In its first 121 years of operation, the patent office issued one million patents. The next million patents came in only 25 years, and that trend has only continued to accelerate. As of 2018, the office had issued 10 million patents (Patel, 2018). While this indicates a thriving environment for innovation, the haste of technological development has also created an environment where new inventions quickly become incompatible, obsolete, and forgotten. This applies not only to hardware but also, and often more paralyzing, the software running on that hardware. As a result, some of the experiences included in this survey are no longer accessible in their original formats and can only be studied through surviving documentation. Thankfully, manufacturers like Oculus, a prominent manufacturer of VR headsets, have recognized this issue and are working to ensure future hardware and software updates will continue to support the content made today (Constine, Oculus VR Content Will Be Forward-Compatible with ITs Future Headsets, 2018).
The overarching question that series of articles will explore is, how are these formats different? Understanding the musical possibilities made available through new digital formats requires examining the works collected here for how they align with or break from historical music formats? What qualities do they have in common with previously available musical experiences? And perhaps most important, do they offer anything new?
Each of these new formats claims to be different by positioning itself either partially or entirely in a new reality. In researching and experiencing these new realities, however, it became clear that both the works themselves and the developers of the software and hardware that enable these new realities seem to be investing most of their efforts in recreating our current realities. This raises several questions. Of the works included here, what aspects of our reality do they successfully copy? What elements do they leave behind or fail to adapt faithfully? Finally, do they push beyond the limits of our reality in any way? This survey will not concern itself with assigning value to these differences, only with cataloging them. Portions of the final article will reflect on these types of inquiries.
The final and most relevant element this survey seeks to uncover is how each of these formats changes the way music can be engaged. Ultimately, this series aims to uncover what musicians can do within modern musical formats that have never been possible before.
Atlantic Re:Think. (2015, September 9). How Fast Is Technology Accelerating? (T. Atlantic, Producer) Retrieved from www.theatlantic.com/sponsored/prudential-great-expectations/how-fast-is-technology-accelerating/360/
Constine, J. (2018, October 27). Oculus VR Content Will Be Forward-Compatible with ITs Future Headsets. (T. Crunch, Producer) Retrieved from https://www.techcrunch.com/2018/09/26/oculus-compatability/
Patel, N. (2018, July 19). The US Patent Office has issued 10 million patents. Retrieved from The Verge: https://www.theverge.com/2018/6/19/17478898/uspto-utility-patents-10-million-issued
Turi, J. (2014, February 16). The Sights and Scents of the Sensorama Simulator. (Engaget, Producer) Retrieved from www.engadget.com/2014/02/16/morton-heiligs-sensorama-simulator/
Virtual Reality Society. (2015, October 16). History of Virtual Reality. Retrieved from Virtual Reality Society: www.vrs.org.uk/virtual-reality/history.html
