"Expansion of spatial experience with 'sound' created by the corona disaster"

3 months since the emergency declaration was lifted. It seems that it will still take some time for the convergence, but economic and cultural activities will resume, and gradually we will be able to experience real spaces again, such as viewing tourist attractions and museums, and experiencing new things at events and exhibitions. It has become At the same time, the new normal, such as non-contact and avoidance of the three Cs, will not restrict conventional experiences, but will create opportunities to create new spatial experiences, coupled with the rapid increase in digital literacy over the past few months. I don't think so.
This time," From the standpoint of a NOMLAB manager, I would like to introduce the prototype for expanding the spatial experience using "sound."

Extending the exhibition experience by placing sounds in a real space

Mr. Shunichi Takeshita, representative of GATARI, gave us such expectations, and the discussion started. GATARI and NOMLAB focused on expanding the exhibition experience with "sound" as an experience that can be done while maintaining contactless, even with the recent development and spread of audio devices and with COVID-19.
In addition to the proposal from Hitomi Nakamura, NOMLAB's content director, "Isn't it possible to create an experience where the sound is controlled according to the position of the user, and the songs are smoothly connected when walking in the space, making it feel like listening to a single song?" , exhibition experience planning, sound installation, sound engineering, etc. Among them, the mixed reality audio AR platform "Auris" developed and provided by GATARI was used for exhibition production, aiming for a tour-type sound experience service specializing in production, "oto rea ( Otria)".

Features of “oto rea”

With the newly developed oto rea, we aimed to make the best use of the characteristics of both companies to create a presentation that enhances the added value of experiences with stories and realistic spatial experiences.

■ You can create an experience with a story

It is a mechanism that recognizes the exact position of the user and distributes the sound only to the user through headphones. Sounds can be set according to the user's actions such as "move," "stand," "sit," "look," and "approach." You can create an immersive story experience that makes you feel like you've entered a game world that combines real space and sound enhancement. Also, since there is no need to worry about sound mixing, it is possible to create a highly private experience.

■ Production that enhances the added value of a realistic spatial experience

It is possible to hear sounds from above, below, front, back, left and right, express the distance between sounds, and place multiple sounds in a space. . Reverberations and muffled sounds can also be reproduced, so you can produce effects that maximize the unique characteristics of the real space.

Demonstration at NOMURA Co., Ltd. "RESET SPACE"

Opened in 2018 with the concept of "a space where new creativity is born from communication with friends" because it shows that various scenes can be produced, and at the same time, it is possible to manage closeness and corona countermeasures within a visible range. A demonstration was held at NOMURA Co., Ltd. "RESET SPACE". Although it is an experience that can expand the concept of RESET SPACE, we have created three prototypes (1) Chair exhibition (2) Doodle exhibition (3) Living office exhibition, assuming introduction in various facilities. About 120 people from inside and outside the company participated in the experience.

RESET SPACE is a fusion of ingenuity to activate communication

Visitors put on sunglasses-type audio devices, hold smartphones connected via Bluetooth, and start walking around each exhibition.

(1) Chair Exhibition “Feel the background and story of the exhibits”

The chair exhibition uses four unique chairs in the company. As you approach each chair, you will begin to hear sounds, and as soon as you sit down, you will be able to hear the sounds that symbolize the world view of the chair's maker and the era.
Through sound, you can experience the places and times in which the exhibits originally existed, which cannot be understood just by placing them in the exhibition room, and make you think about the background and story of the exhibits. I performed with such a theme.
For example, in the case of a chair made by a Finnish artist, which is said to have been made to give a sense of the warmth of wood in a sophisticated modernist style, when you approach the chair, you can hear the chirping of birds, and the moment you sit down, you can feel the Scandinavian atmosphere. The sound flows as if the air and light of the forest are blowing in. Through this exhibition, visitors were able to experience the technology that can link sound with movement and location information in an easy-to-understand manner.

(2) “Experience the experience of being guided by a special voice” graffiti exhibition

The graffiti exhibition is an exhibition performance where you can hear the sounds of the illustrations drawn on the wall of REST SPACE, and you can hear them talking.
When I'm walking around the exhibition room and I feel some kind of faint sound, such as a cry, a noise, or a human voice... that kind of idea gives me a hint. For example, in the case of a cat illustration on the wall, you can hear the meowing of a cat from somewhere, and rely on that sound when you approach the illustration of a cat and say, "What are you looking at? The cat starts talking in the form of "nya" (the voices of the demonstration were performed by volunteers from GATARI and NOMLAB).
We are creating an experience that gives you a strange feeling that only you are guided by a special voice.

(3) Living Office Exhibition “Experience how sound changes seamlessly depending on your position and movement”

The living office exhibition is an exhibition where you can experience how various sounds change seamlessly according to the various actions of the participants as they walk around the entire RESET SPACE.
In the open space, where uneven and unique chairs and uniquely finished tables with flooring tops are arranged, BGM that reminds us of future-oriented conversations and conversations that pursue space unfolds. In the closed space, which is used when you want to concentrate, you can hear the cry of an owl drawn large as if watching over the employees, and the sound of dishes and water being used from the coffee service counter that reminds you of the time of the event. is flowing ... .... I think that the sound produced the concept of REST SPACE, a space where new creativity is born from communication with friends.

Possibilities of exhibitions possible with "oto rea"

Mao Horii, the planner who worked on the planning of this exhibition, talks about the possibilities of exhibitions using such an excursion-type audio experience service.

It can be attractive as a scene of a story with a sense of presence

“What is common in conventional exhibitions is the experience of ``accumulating information'' from individual exhibition media. , you combine information in your head and build a story, but it is often difficult to get there.By adding the experience of "sound" that seamlessly connects sound and narration, the story becomes If you can feel the atmosphere and the scene in real space, I think it will be possible to make the exhibition attractive in one shot as ``a scene of a story with a sense of presence.''

Able to actively work on the feelings of the participants

“Existing exhibition spaces often remain silent, silent, or have overall background music, and the sound environment does not have much positive influence on the emotions of visitors. It's up to the individual, but by carefully designing the sound environment, it is possible to raise the mood of the visitors in the direction intended by the exhibitor."

"oto rea", which further enhances the added value of the spatial experience by sensing the invisible expanse of space through sound, can be found in historical buildings such as museums, castles, etc., as well as in this project. I think that it is possible to create new experiences in workplaces and commercial spaces.

Future Improvements and Awareness of the Experience of “Listening”

Through the demonstration, I found points for improvement and hints for continuing research. Nakamura says, ``While triggers for actions in real space are not yet widespread, in the future, a world will come where everyday actions such as staring at or looking into things will become a matter of course. I would like to continue my research on how to handle and direct sound." Yoshimoto Masuda, who was in charge of sound engineering and sound creation at GATARI, said, "It was an experience event in a well-maintained environment, such as attending the demonstration, designing the experience route and preparing the equipment, but it was not ideal. There are still issues to be solved in order to realize the experience.Also, I felt that a good experience that makes you want to experience it again and again differs from person to person.We carefully considered the cause and countermeasures, and worked out the development and expression. I wish I could do it, ”he commented for the future.

Personally, I was interested in a book that the general manager of a certain science museum introduced to me because it might give me a hint when thinking about this prototype. “How Do Blind People See the World?” (written by Asa Ito) analyzes the perception of space, how the senses and bodies are used, and the communication methods of visually impaired people. I am questioning that. At the same time as I learned that the act of "seeing" does not equate to "visual information," I also realized how people without visual impairment usually use functions other than vision. This realization made me think about what the experience of "listening" is, and I felt that this approach to "sound" could lead to further expansion of the spatial experience. How do hearing-impaired people hear the sounds of the world? It seems that there are countless perspectives to consider if you delve into the thought of the experience of "listening."

This time, I introduced it from my standpoint as a manager, but discussions are repeated from various perspectives such as service design, space production, audio devices, sound installation, acoustic technology and psychoacoustics. As a laboratory that works to expand the experience of space, we will continue to expand the experience of "sound" while creating chemical reactions between various professionals.

“oto rea”

GATARI
Project Management Shunichi Takeshita
Application Development Takumi Ito Hirotoshi Yoshitaka
Acoustic Engineering Come Sato Yoshimoto Masuda
Sound Creation Come Sato Yoshimoto Masuda

NOMLAB
Project Management Yuichi Tsumoto
Planning Mao Horii
Content Direction Hitomi Nakamura Minori Hayashi
Sound Creation Masahiko Ohi

​Click here for press release

As museums, art galleries, entertainment facilities, and new forms of regional tourism.
Please contact us if you are interested.