Concepts and Technologies Fueling the Growth of Comhear s Advanced Audio Products

Transcription

1 White Paper Concepts and Technologies Fueling the Growth of Comhear s Advanced Audio Products 2016, 2017, Comhear, Inc. 12/8/16, rev. 07/17/17 Concepts and Technologies Fueling the Growth of

2 Comhear s Advanced Audio Products Contents Executive Summary 3 Introduction / Problem Statement 5 Market Conditions & Opportunities 6 Components and Technology Advantages 6 Patent brief and high level applications 7 Theory: Beamforming, Crosstalk, HRTF 8 3D Audio Product Justification 11 Primary components and form factors of Comhear speaker arrays 11 Parameter control software 13 Summary 14 Citations and Footnotes 14 Plots and Technical Materials: Comhear Beam-forming plots 15 Executive summary 2

3 Compared to other technological fields where rapid advancement is fueled by competition and consequently richly rewarded, the field of consumer audio has progressed in mostly incremental steps. Though advances are continually made in labs and experimental environments around the world, breakthrough consumer audio technology products are relatively rare. Some noteworthy exceptions in the last decade include noise cancelling headphones, headphones as fashion accessories, the sound bar genre, portable bluetooth speakers, semantic interfaces such as Amazon Alexa, and Dolby Atmos / DTS MDA scalable object-based audio methods. But by and large, and even in the case of some of the above examples, audio technologies are, more often than not, viewed by consumers as accessories of incidental value rather than necessities of central importance and high value. Will a marginalized role for audio be the rule for years to come? Will the field continue to be characterized by incremental change? Will audio follow the model of providing visual, music and communications systems with a limited range of accessories? Or are there real opportunities for more significant breakthrough products in the consumer audio eco-system? If so, what technical and market factors will synergize to produce a breakthrough technology? We believe there are opportunities for disruptive products in the consumer audio eco-system, and that market conditions will justify new approaches and products to address old and new categories of audio experience. Alignment of desirable user experiences with new enabling technologies at accessible prices will eclipse good enough and drive innovation in consumer audio. We believe that 3D-audio is the area most ready for and receptive to disruptive technologies. We believe Comhear in partnership with Zylux have the expertise, IP, technology, resources and focus to capture emerging consumer demand for next-generation audio products. What changing market conditions make this possible? Consumers are now exposed to a new generation of immersive media experience, most significantly, VR/AR technologies. These media all benefit from and often require advanced audio systems to provide compelling experience. For now, many of these new experiences are delivered via headphones, but as new media forms mature, we expect that there will be a large demand for shared immersive experience, eclipsing the use of headphones and driving a market for our immersive speaker technologies. The result will allow AR/VR and immersive entertainment to enter the mainstream of shared, social experience. The same enabling technologies of shared AR/VR experience can also be employed to the benefit of virtual conferencing and distance education. These trends will motivate consumers and prosumers to seek a better user experience in the problematic and solution-resistant area of audio quality and intelligibility for conferencing/collaborative communications. The versatility of our platform in fact offers another possibility for disruption. Experience has shown that we deliver an improved user experience for audiophile-quality music reproduction, in addition to visual entertainment (games and cinematics), as well as for communications. The ability for a single compact platform to deliver excellence in all these modalities can represent a shift away from dedicated audio platforms, to a single convergence platform that equals or outperforms dedicated last-generation technologies in every modality. What are the enabling technological advancements Comhear brings to the table? Our primary technology leverages a combination of several powerful approaches, working together to deliver compelling UX in compact and economical packages. These advancements, explained in greater detail herein are: 3

4 Active/Adaptive DSP-driven compact speaker and microphone arrays Sound Field Modeling for trans-aural, steered mic arrays and directed audio Patented beam-forming optimization methods Small form factor speakers and mics configured for unusually large soundstages Employment of high efficiency, low power multichannel class D amplifiers driving small and cost-effective full-range speaker arrays Applied perception science for vivid 3D auditory imaging and speech intelligibility Perfect phase alignment for extremely high quality voice and music reproduction. A few advanced products in professional and installation applications utilize isolated aspects of the advances listed here, though none combine this impressive list of innovations or have access to our patented beam forming technology to achieve the vivid 3D and directed audio imaging of Comhear products. This white paper provides added depth and details of our technology. * * * * * * * * * Our industry partner has now manufactured a ProAV product built around our core technologies in a manner that is designed for easy scalability in a wide variety of applications and form factors. Two consumer products are currently in final stages of manufacturing for introduction in January 2018; 2018 will be a year of accelerated growth and rapidly expanding revenue. We encourage exploration of our technology and consideration for licensing and partnerships. Through partnering we anticipate a strong future for our products and the growth of our companies by providing consumers and business users with breakthrough products that provide deeply satisfying media experiences for consumers and business users alike. For more information, please contact Comhear at ( ) or 4

5 Introduction / Problem Statement Consumer media markets have seen unprecedented qualitative improvements in visualization technology since transitioning from Standard Definition to High Definition. 4K quickly followed HD and immersive visualization is rapidly gaining popularity; 8K is not far behind. Parallel improvements in audio technology have not been adopted at the same rapid pace. Consumers have by and large chosen to prioritize portability, streaming, storage, and ease of accessibility over audio quality. Good Enough often wins out over high quality for a number of complex reasons, and consumers tend to prioritize visual quality over audio quality. Even though Hi-Res Audio and HD Audio equipment content are available (usually at premium prices) the high quality consumer audio market is small in comparison to mainstream audio markets. Compared to other technological fields where rapid advancements are fueled by competition and richly rewarded, the field of consumer audio has adapted to market conditions by advancing in mostly incremental steps. Audio technologies are viewed, more often than not, by consumers as accessories rather than necessities of central importance and high value. Though advances are continually made in labs and experimental environments around the globe, breakthrough consumer audio technology products are relatively rare. Some noteworthy innovations in the last decade and their functional roles in the consumer eco-system include: Bose Noise-Cancelling Headphones comfort, audio quality and fatigue mitigation Beats Headphones fashion accessory, reasonable audio quality Soundbar genre convenient upgrade to mitigate poor audio quality on thin displays; alternative to inconvenient surround sound speakers and equipment Portable speaker genre incremental audio quality improvements; portable, battery operated, multi-room wireless/blutooth, voice commands and semantic interfaces such as Amazon Alexa Dolby Atmos / DTS MDA object based audio for scalable, immersive cinema experience [see footnotes]. Yet despite these areas of progress, a number of under-performing audio technologies are persistently employed, despite problematic results. Some are of particular interest to us: o Poor intelligibility, balance and adaptation in conferencing technologies on all scales: personal, small group, large groups o Lack of comfort and poor situational awareness where immersive media audio is rendered on headphones o The promise of compelling alternatives in home theatre sound has fallen far short of its potential. The vast majority of users cannot afford - or do not desire - permanently installed multichannel surround systems, while content channel count continues to increase. Soundbars purport to provide faux surround sound but in almost every case fall short. And home theatre sound is often not integrated with home audio systems. In many homes TV sound is separate from music sound systems. o AR/VR and next generation cinematic/4dx visual technologies promise a paradigm shift in immersive visual experience. Apart from many competing and evolutionary solutions in headphone sound, no comparable paradigm shift in speaker technology 5

6 (offering more vivid audition than traditional multichannel surround sound) is in the consumer eco-system today. Will a marginalized role for audio be the rule for years to come? Will the field continue to be characterized by incremental change? Will audio follow the model of providing visual, music and communications systems with limited accessories, or are there opportunities for more significant breakthrough products in the consumer audio eco-system? If so, what technical and market factors will synergize to produce a breakthrough technology? Market Conditions & Opportunities We believe there are opportunities for disruptive products in the consumer audio eco-system, and that market conditions will justify new approaches and products to address old and new categories of audio experience. Alignment of desirable user experiences with new enabling technologies at accessible prices can eclipse good enough. Advances in overall personal and home technologies compel parallel responses in the auditory domain and will drive consumers to reassess the value and potential for a much higher quality audio experience. We believe that 3D-audio is the area most ready for and receptive to disruptive technologies. Visual immersive experience will create consumer awareness of 3D audio s full potential, and improve market conditions for a disruptive technology. We believe Comhear in partnership with Zylux have the expertise, IP, technology, resources and focus to capture emerging consumer demand for next-generation audio products. We believe consumers are beginning to widely understand the desirability and benefits of exceptional, immersive auditory experience as they become accustomed to Ultra HD (4K) video with its improved dimensionality and immersive quality, and as thinner TV form factors eliminate even the possibility that TV sound can be good enough. Consumers are now routinely exposed to a new generation of games, location based entertainments, immersive theme park venues, 4DX cinema, and most significantly, VR/AR technologies. These all benefit from and often require advanced audio systems to provide compelling experience. Much attention is focused on delivering these new experiences via headphones, but as these new media forms mature, we believe there will be a large market for shared immersive experience, paralleling the use of headphones and driving a market for our immersive speaker technologies. The result will allow AR/VR and immersive entertainment to enter the mainstream of shared, social experience. On a more practical level, the benefits of virtual conferencing and distance education motivates consumers and business users to seek better user experience in the problematic and solutionresistant area of audio quality and intelligibility in communications technology. Similarly, we have received unexpected attention from the high-end audiophile community, impressed with the ability of our platform to render normal stereo with great clarity, definition and dimensionality. Further exploration reveals an ability to reveal subtlety and remarkable musicality from HiRes Audio and HD music content, which are growing music consumer categories embraced by companies such as Tidal and Sony. The versatility of our platform in fact offers another possibility for disruption. Experience has shown that we deliver improved user experience for audiophile-quality music reproduction, in addition to visual entertainment (games and cinematics), as well as for communications. The ability for a single compact platform to deliver excellence in all these modalities can represent a 6

7 shift away from dedicated audio platforms, to a single convergence platform that equals or outperforms dedicated last-generation technologies in every modality. Technology Backgrounder Components and Technology Advantages If we are correct that market conditions provide opportunities in unique new audio market categories, what are the enabling technological advancements our companies bring to the table? Our primary technology leverages a combination of several powerful approaches, working together to deliver compelling UX in compact and economical packages. These advancements, explained in greater detail herein are: Active/Adaptive DSP-driven compact speaker and microphone arrays Sound Field Modeling for Trans-aural, Mic Array and Directed Audio Patented Beam Forming Optimization Methods Small form factor speakers and mics configured for unusually large soundstages Employment of high efficiency, low power multichannel class D amplifiers driving small and cost-effective full range speaker arrays Applied perception science for vivid 3D Auditory Imaging and Speech Intelligibility Perfect phase alignment for extremely high quality voice and music reproduction A few advanced products in professional and installation media technologies utilize some of the advances listed here, though none combine this impressive list of innovations or our patented beam forming technology to achieve the vivid 3D and directed audio imaging of Comhear products. Patent Brief and High Level Applications Comhear products are based around a core of new, compact, patented DSP and sound field control technology developed at UCSD and by researchers there and from the Univ. of Southampton (Otto, Fazi, Kamdar, Yamada) 1. In brief, this patent, awarded in October 2016, describes an innovative method for the optimization of beamforming, controlling a small speaker array to produce spatialized, localized, and binaural (or transaural) virtual surround or 3D sound. The signal processing method allows a small speaker array to deliver sound in various ways using highly optimized inverse filters, delivering narrow beams of sound to the listener while producing negligible artifacts. Unlike earlier compact beamforming audio technologies, our method does not rely on ultra-sonic or high power amplification. In fact, the technology is extremely low power, producing 98dBA at one meter, while utilizing around 20 watts of peak power. Comhear has further developed the technology for a number of applications and markets, and has exclusively licensed the technology for commercialization. The applications described in the patent are broad, and may be used for both microphone and speaker arrays. In the case of speaker applications, internally referred to as our MY BEAM product, the basic use-case allows sound from a small (10 20 ) linear array of speakers to focus sound in narrow beams to: - Direct sound in a highly intelligible manner where it is needed and effective 7

8 - Limit sound where it is not wanted or where it may be disruptive - Provide non-headphone based, high definition, steerable audio imaging in which a stereo or binaural signal is directed to the ears of the listener to produce vivid virtual and 3D auditory images. In the case of microphonic applications, the basic use-case allows sound from an array of microphones (ranging from a few small capsules to dozens in 1-, 2- or 3-dimensional arrangements) to capture sound in narrow beams. These beams may be dynamically steered and may cover many talkers and sound sources within its coverage pattern, amplifying desirable sources and providing for cancellation or suppression of unwanted sources. Theory: Beamforming, Crosstalk and HRTF MyBeam speaker uses Digital Signal Processing (DSP) on identical, individually powered and perfectly phase-aligned speaker elements in a linear array to produce constructive and destructive interference. Figure 1 shows an acoustical sound field model in which two speakers pass identical sine tones, resulting in a predictable pattern of acoustical reinforcement (constructive) resultant nodes and (destructive) resultant nodes. Figure 2 model shows a more complex interaction in which two signals are applied to 10 elements of a linear array. Using delay and sum beamforming coefficients applied to each element, along with artifact cancellation techniques, we are able to achieve a relatively high level of focus at predicted nodal positions of 0 degrees and + or - 55 degrees. Fig. 1 Fig. 2 Figure 3 shows diffusion of a normal consumer audio stereo speaker. The pink area indicates the predicted balanced listening area, while the purple areas indicate that listeners will perceive sound coming from the nearest source due to the proximity effect or law of the first wave front. 8

9 Fig. 3 For traditional music listening environments, it would seem that larger listening areas should provide the widest range of acceptable listening positions. This is true, but at a cost: a wide stereo listening area will almost always have diminished spatial imaging, due to cross-talk. Figure 4 illustrates stereo crosstalk. Direct sound from speakers reach the ears as you would expect: left speaker to left ear, right speaker to right ear. But due to cross-talk, some sound from the left is heard by the right ear & vice versa. This creates blur in the spatial image, and the dimensionality of a stereo or binaural source is compromised. In traditional stereo music this was an acceptable compromise; in 3D audio for immersive cinema, AR/VR and mission-critical communications, this compromise is not acceptable due to its significant negative impacts on reproducibility of multidimensional auditory imaging, which also degrades intelligibility and tonal clarity. Fig. 4 Figure 5 illustrates how stereo crosstalk is controlled in MyBeam, using beamforming to control directivity of the left and right channels, greatly diminishing crosstalk and allowing the brain of the listener to perceive a stereo image as highly detailed and with exceptional spatial dimensionality. The effect is quite similar in concept to polarized 3D glasses, where the observer s eyes process two slightly differing images to produce stereo visual perception. Headphones can reproduce 3D auditory images as well, but there are perceptual and practical drawbacks that make headphones undesirable or unusable in some contexts 9

10 Fig. 5 Figure 6 illustrates direct sound arrival in a cross-talk cancelled, trans-aural acoustic wave generated by a beam-forming system, showing strong sound pressure in the right ear and a significant acoustic shadow in the left ear. This model vividly illustrates Interaural Intensity Difference, or IID (see below). Fig. 6 Figure 7 depicts binaurally encoded beaming of alerts, alarms, and voice communications. Vivid positional rendering is employed for enhanced situational awareness and intelligibility. Position and loudness of each source may be automated, or interactively positioned by the user. Fig. 7 10

11 Studies in psychoacoustics tell us that well-recorded stereo signals and binaural recordings contain cues that help create robust, detailed 3D auditory images. By focusing left and righ channel signals on the appropriate ear, MYBEAM maintains key psychoacoustic cues and by avoiding crosstalk. Together, these cues are known as Head Related Transfer Functions (HRTF). Briefly stated, HRTF component cues are interaural time difference (ITD, the difference in arrival time of a sound between two locations), the interaural intensity difference (IID, the difference in intensity of a sound between two locations, sometimes called ILD), and interaural phase difference (IPD, the difference in the phase of a wave that reaches each ear, dependent on the frequency of the sound wave and the ITD). Once the brain has analyzed IPD, ITD, and ILD, the location of the sound source can be determined with relative accuracy. [2] 3D Audio Product Justification Why do we care about accurately rendering 3D auditory imaging? Because navigation and situational awareness in virtual environments is key to establishing and maintaining a virtual cognitive model, and humans use both visual and auditory information to determine their position in space and the relative position of objects around them. In fact, when visual and auditory information are not well-correlated in virtual environments, motion sickness will sometimes occur. When applied to speech, 3D auditory imaging can provide a 65% improvement in intelligibility [3][4]. When applied to music and video games/virtual environments, 3D auditory image can provide a vast improvement in detail and image [5]. The average human has the remarkable ability to locate a sound source with better than 5 accuracy in both azimuth and elevation, in challenging environments [2]. Yet due to cost, computing power, materials constraints, and a number of contributing factors, 3D auditory imaging on speakers has largely remain unaddressed and undeveloped in the marketplace. Comhear and Zylux together now manufacture a mature solution that yields excellent 3D auditory imaging in an affordable, robust package. Licensing is available in the following areas of use: Consumer video games: virtual surround / 3D-sound on a small speaker bar Professional installed AV systems for digital signage/directed audio in public venues: arenas, transportation hubs, retail, services POS, etc. 3D trans-aural audio for 4D cinema, Location Based Entertainment, and other advanced entertainment venues Personal desktop audio for music, video and communications Home theatre and high-end audio systems for home entertainment Primary components and form factors of Comhear speaker arrays The core technology reference design of our speaker products consists of a portable code base that can be implemented on custom ASICs, FPGA or DSP architectures, or in more advanced products, a SOC. Our modular I/O architecture may be adapted to various configurations, from communications and conferencing to automotive and consumer audio products. For use with movies, multichannel music and games, Dolby and other codecs can be supported, along with our KAP audio enhancement software and our licensed binaural encoder running on custom or commonly available architectures (Fig. 8). 11

12 Dolby & Standard Codecs MyBeam 12 element speaker array with 2 element mic array based on Comhear Reference Design 12 x Low Power Class D amplifiers DSP Host Architecture - Comhear Proprietary Licensed IP Patented Beam Forming 3D audio object renderer Binaural Encoder, KAP Optical digital input Stereo Line In USB Audio in + USB Parameter Control Mic In Subwoofer out GUI + remote control Fig. 8 A Mybeam speaker is active it contains its own amplifiers and IO and it is highly adaptable. It can be configured to include ambience monitoring for automatic level adjustment, and can adapt its beam forming focus to the distance of the listener. and operate in several distinct modalities, including binaural (transaural), single beam-forming optimized for speech and privacy, near field coverage, far field coverage, multiple listeners, etc. In binaural mode, operating in either near or far field coverage, Mybeam renders a normal PCM stereo music or video signal (compressed or uncompressed sources) with exceptional clarity, a very wide and detailed sound stage, excellent dynamic range, and communicates a strong sense of envelopment (the image musicality of the speaker is in part a result of sample-accurate phase alignment of the speaker array). Running at up to 96K sample rate, and 24-bit precision, the speakers reproduce Hi Res and HD audio with exceptional fidelity. When reproducing a PCM stereo signal of binaurally processed content, highly resolved 3D audio imaging is easily perceived. Height information as well as frontal 180 degree images are well-rendered and rear imaging is achieved for some sources. Reference form factors include 12 speaker, 10 speaker and 8 speaker versions, in widths of ca. 8 to 22 inches. Partners may wish to design other form factors using our scalable algorithms. Systems can be used with or without subwoofer (subwoofer recommended for music and cinematic use). Electronics and our reference speaker cassette structure may be separated for various installation requirements and custom industrial design. 20 components are shown in reference design (Fig. 9). 11 enclosure reference design shown in (Fig. 10). 12

13 Fig. 9 Fig. 10 Parameter control software Comhear has created a MyBeam graphical user interface (Fig. 11). This interface is enabled by connecting the MyBeam speaker array to a Windows-based PC through the USB port. It allows access to all user-configurable parameters on the array. An OSX version is currently in development. 13

14 Fig. 11 Fig. 12 Summary We are proud to report that we are now partnering in the manufacture of a ProAV product that uses our core technologies in a manner that is designed for easy scalability in a wide variety of applications and form factors. As a startup with 2.5 year of growth and refinement, 2017 will be a year of accelerated growth and rapidly expanding revenue through licensing and support of a hero consumer product with prestigious industry partners. 14

15 * * * * * * * * * Citations [1] Method for controlling a speaker array to provide spatialized, localized, and binaural virtual surround sound US Patent Application A1 Inventor Peter Otto Suketu Kamdar Yamada Toshiro Filippo M. Fazi Original Assignee The Regents Of The University Of California The University Of Southampton [update with grant #] [2] [3] Augmented Intelligibility in Simultaneous Multi-talker Environments. Duraiswami, Grant, Mesgarani, Shamma 2003, Proceedings of the International Conference on Auditory Display (ICAD 03). [4] Directional communication using spatial sound in human-telepresence. Shohei Nagai, Shunichi Kasahara, Jun Rekimot, Proceedings of the 6th Augmented Human International Conference, Singapore 2015, ACM New York, NY, USA, ISBN: [5] The Psychology of Music in Multimedia Siu-Lan Tan, Annabel J. Cohen, Scott D. Lipscomb, Roger A. Kendall. Footnotes Spatial separation turned out to be the most effective single cue for improving the intelligibility of multiple, competing speakers. [3] Improving sound quality (Dolby 5.1 Surround versus Dolby stereo) positively affected the players sense of presence and enjoyment of the game. Specifically, participants playing the game with surround sound reported feeling greater engagement, experienced a greater sense of being present in the action of the game world, described the experience as more socially intimate and more perceptually real, and rate the game as more enjoyable than those playing with stereo sound. [4] 15

16 Plots and Technical Materials: Comhear Beam-forming 16