What is the evolution of spatial experience through sound led by oto rea? (Part 2)

Jointly developed by GATARI Co., Ltd., an AR/MR startup that aims to create future infrastructure for the future city "super city", and NOMLAB, a laboratory of NOMURA Co., Ltd. with the theme of "digital innovation x place creation". , oto rea, a prototype for augmenting spatial experience through sound. In this article, which is the second part of the dialogue, we will expand on the possibilities of oto rea and the future prospects for post-coronavirus. (Click here for the first part)

3. Market area and potential of oto rea

Okamura
I think that the voices of those who have experienced the game will be the seeds that will lead us in the future.

Hayashi
According to the questionnaire results of the experience session (the question was set as "What kind of market do you think oto rea can be used in?"), adding entertainment elements to facilities that convey historical value such as art museums and museums Isn't it interesting to go? There were many voices like this. For example, it is possible to add information not only to objects, but also to empty spaces, so it is possible to provide experiences such as "BGM changes as you walk down the aisle." Whether it's a museum or an amusement park, if you just move around, you'll get bored and tired, but the experience of finding enjoyment in the movement itself, even if it's in the middle of nature or in a warehouse. I think it can also be applied. I think there is a great possibility that you can create an interesting experience even if you don't have a specific object to look at.

Okamura
It is possible to give value to things that were not conscious of value until now.

Kuroki
Mechanism to rediscover the existence of value It can also be said that

Nakamura
I heard about a facility that offers backyard tours that are not open to the public. By doing so, you may be able to feel more strongly the important functions of the backyard and the breathing of the people working there. In addition, I think that it may be possible to give a more interesting and strong impression by conveying the background of the exhibits in the exhibition facility with a three-dimensional sound. For example, in an exhibition room with a diorama of dinosaurs, if you can hear the sounds of the times when dinosaurs lived, the roar of a volcano, and the voices of other dinosaurs fleeing from the surroundings, you will feel like you have transcended time and space. can provide an experience I think.

Kuroki
What we call "experience of lines, not points", is not just that there is sound in the space, but when you enter and exit a specific space, you can feel the connections and differences between each space. You can express yourself through sound.

Nakamura
At NOMURA Co., Ltd. 's in-house oto rea experience event, we changed the BGM depending on the area, but many people were surprised during the experience.

Okamura
It's important to create a story of your experience.

Nakamura
With oto rea, you can hear the sound three-dimensionally from the front, back, right and left. It's not just three-dimensional, but the sound changes according to your movements, so by linking it to the actual space, you can say that it's a game-like space where you can feel more realistic.​

Takeshita
​ ​Speaking of the added value of the experience, when using audio for commentary and production in conventional facilities, even if you install a single speaker, the sound is narrowed down so as not to interfere with other corners, or it is installed separately. So it was difficult to create an ideal sound environment. Furthermore, when outdoors, there are many cases where sound cannot be produced in the first place due to regulations. oto rea is also a mechanism that allows you to personalize the voice without those restrictions.

Hayashi
With oto rea, you can experience being surrounded by 10 speakers like surround sound.

Takeshita
​ ​It can also be personalized according to each user's hearing ability, such as adjusting the volume to suit each individual, even for elderly people who are hard of hearing. By personalizing not only the volume but also the contents of the content, I feel the possibility of further development.

Okamura
The idea of personalization has great potential not only in facilities that attract customers but also in everyday life.

Takeshita
Currently, oto rea is experienced by hanging a smartphone attached to a strap from the neck, but we call it the "third eye". It is possible to create an unprecedented experience that connects the digital world and the real world with a smartphone camera, so to speak, eyes. In other words, it's a setting where you can see the digital world when your third eye is open. I think that if you wear AR glasses, you can make that third eye a daily reality. If the eyes that can see the digital world become commonplace and become infrastructure, it will not only expand the possibilities of advertising, but it will also be possible to use it for navigation for the visually impaired. . We believe that we can provide an experience that makes foreign tourists who speak different languages feel "accepted in the space". I would like to aim for an infrastructure for community development that accepts a diverse range of people.

Okamura
Will "a variety of people be able to participate" become a requirement for various spaces, including events, in the future?

Nakamura
I feel that there is a growing demand for how to deal with various people and how to respond to the feelings of wanting to experience and share together, even when planning spaces.​

4. The new normal of people, society and digital technology

Okamura
The word “new normal” has been coined due to the corona crisis, and I think that one direction of the new normal is the progress in the fusion of the real and the digital. For example, in a society where AR glasses have become a part of everyday life, will people lose their sense of the existence of technology?

Takeshita
I think it is important to add value by connecting the digital space and the real space in various ways, such as acquiring digital twin information and connecting it with the real space. It's important to be contextually connected, so even if an unfamiliar (digital) character suddenly appeared here (at GATARI's studio, where we're talking now), it would be meaningless to the person who saw it. , If it's a GATARI character, it makes sense because the relationship between the location and the character becomes the context. In front of the vending machine, in front of the elevator, "meaningful / not meaningful" will change. In the case of an office, you need to be able to unlock the door by standing in front of it. In the case of an entertainment space, when you sit in a chair, the ceiling lights will turn off and you will be in the spotlight. I think it is important to create a digital space that is connected to the real space.

Nakamura
In the future, physical switches will gradually disappear, and I believe that a future will come when devices can be controlled by actions (instead of switches). I think that a more comfortable life will evolve, such as turning on the light just by waving your hand. I think that the idea of controlling space through the natural movements of people will continue to spread in the future.

Hayashi
Even now, you can turn it off by telling your smart speaker to turn off the light.

Takeshita
The interface of mobile devices will be operated by touching the screen with a smartphone instead of the pop of a mobile phone, and it will be possible to directly touch the information on the back of the screen by AR. I feel that the real space and the digital space will become equivalent.

Kuroki
By specifically visualizing digital things, it may be possible to develop attachment I personally think. Only you can see it, and by recognizing it, you create a special relationship with yourself (visualized digital things), and there is a kind of attachment that is born there I don't think so.

Hayashi
Maybe it's just me, but when you digitize a city, it looks cuter than the real thing. The "sense of distance" and "synchronization with your own movements" that you feel when you look at a city will make it possible to create a fusion of experiences unique to digital and spatial experiences.

Takeshita
For example, if a grandma and her grandchild are watching a baseball game, the grandchild can watch the game in real time at the baseball stadium, and the grandma can watch the game virtually, but it is possible with current technology. You can have a conversation during the game and see where your opponent is looking at the baseball field.

Okamura
Even if the new corona converges, there will still be a certain number of people who will continue to participate remotely, so it is said that the overall increase will be more than before the corona. When the boundary between real and virtual disappears, the boundary between real participants and remote participants will disappear.

Nakamura
In terms of the fusion of real and virtual, An increasing number of companies want to express digital twins in actual space planning work. There is a growing need for proposals for real experiences connected to digital I feel.

5. Future prospects of oto rea

Okamura
Will otorea evolve into an experience that uses the five senses other than hearing?

Hayashi
In addition to sound, lighting production and vibration production can also be considered. We are also thinking about combining various senses with vibration devices and scent devices.

Takeshita
Since oto rea is made with the assumption that AR glasses will be used as a device, it will eventually be possible to add visual information to hearing. Since it is an effort with future expandability, it is a system that can be kept on the cutting edge according to the needs of society and the spread of devices.

Hayashi
You can increase the richness of the space more and more.

Nakamura
For example, in a facility like a museum, visitors may not always walk along the line of flow envisioned by the person who planned and designed it, but we believe that oto rea can guide them. I feel the possibility of providing a new experience by linking multiple senses, such as guiding people by projecting sound, lighting, images, and creating wind.

Okamura
From what I've heard so far, I feel that oto rea is not only a device that can be used in the entertainment field, but also a device that can solve social issues. Lastly, could each of you talk about your future prospects and expectations for oto rea and joint projects?

Nakamura
The government has advocated society 5.0 and is working on measures to connect the physical space and the virtual space. We also want to boost oto rea in line with the government's efforts, If you are interested in reading this article, I would like you to experience oto rea and participate in new initiatives together. I think.

Takeshita
In response to Mr. Nakamura's talk about society 5.0, one of GATARI's themes, "not creating people left behind," will become important in realizing a smart city. In addition to the development of cutting-edge technology, I would like to proceed with inclusive initiatives as a set.

Hayashi
One of the things that left an impression on me was that someone who experienced oto rea said, "It's nice to be able to experience it while talking with everyone." I think it's the difference between an AR experience where you cover your ears and an AR experience where you don't.

Kuroki
I think the good thing about oto rea is that you can create a wide range of content. Even in the entertainment space, you can create both game-oriented content and attraction-oriented content. For example, attractions are fun to ride, like ride-type attractions, and games have multiple endings, and I think the appeal is that you can change the outcome by choosing yourself. Whether it's an ending that only you can reach (like a ride) or an ending that everyone can enjoy (like a game), otorea is scalable to both, so I'd like to create more and more attractive content.​

Takeshita
It would be great if you could create an experience that makes you smarter while having fun. I was a child who was extraordinarily knowledgeable about the kanji for fish (everyone laughs), but when I played with my friends to see which one I could memorize, I became more familiar with them before I knew it. I would be very happy if I could create an experience that stimulated my curiosity and made me learn while having fun. oto rea is a system that can be used in any space, but the more valuable the space is, the more the value is multiplied. In a place of high cultural value, I feel that it functions as a “multiplication device” that fully conveys the value of the space. With smartphones and AR glasses, we tend to focus on the screen and lose sight of the real world, but I think oto rea is a system that "values (shows) reality."​

Okamura
oto rea is an initiative that has the potential to amplify the diverse values of various spaces for a wide range of visitors. The corona crisis is still going on, but I hope that you will continue to work on it and continue to create new spatial value. Thank you for today.

all together
thank you very much!

About the oto rea experience
You can experience it at NOMURA Co., Ltd. in Odaiba or GATARI Akihabara Studio. For facility operators who are interested in this article, here Please contact Sorry to trouble you, but please specify "I want to experience oto rea" at the beginning of the "Inquiry content" input field. We may not be able to meet your expectations depending on the guidance of the government and the Tokyo Metropolitan Government regarding measures against the new coronavirus infection. Please note.