“We could very quickly find ourselves in a world where reality is controlled by tech corporations”
“We could very quickly find ourselves in a world where reality is controlled by tech corporations”
Augmented reality is like magic that offers endless options. But for Apple, which recently launched the Vision Pro, this is just another avenue to extract more and more information from us. And if we are not careful, explains AR expert Shay Segal, "we could lose touch with reality"
To plant a virtual flower next to a bench on the sidewalk, and return to it a week later to see how it grows. Feel your body as if it were an octopus, an insect or a lion. Observe the "breathing" of a tree in the yard. Dive into a shared psychedelic trip with your partner without using psychedelic substances. The strange magic offered by augmented reality (AR) is endless. But you won't find any hint of any of this in Apple's marketing videos for the new and highly hyped Vision Pro headset. These are marketed as the next iPhone — a productivity-oriented device, with a multitude of applications open at the same time.
"Apple's move to launch the Vision Pro is a sign that the era of 'weirdos' is over," says Shay Segal, game developer, lecturer in design and development of augmented reality at Shenkar and head of XR (Extended Reality) at the startup Resight. "Virtual reality was born in the 1980s from the fevered minds of the weirdest people in Silicon Valley, like Jaron Lanier. Whereas Apple's entry into the field is a stamp of consensus. They are moving away from the rich imagination of the eccentrics to start talking to my mother, my father, and my uncles."
With a price tag of $3,500, there's no way our uncles will adopt this anytime soon.
"This is only a first, experimental version, the purpose of which, among other things, is to test the reaction of the public and, of course, to gather information. When Apple CEO Tim Cook was asked about the high price about two weeks ago, he replied that the reason for this is that the Vision Pro is 'tomorrow's technology - today'".
And what will happen tomorrow, when the technologies become common property? In an article for "The New Yorker'' from the beginning of the month, Lanier, who is considered the father of virtual reality, wrote: "Apple is marketing the Vision Pro as a device you might wear for everyday purposes—to write e-mails or code, to make video calls, to watch football games. But I’ve always thought that V.R. sessions make the most sense either when they accomplish something specific and practical that doesn’t take very long, or when they are as weird as possible.”
According to Lanier, the important question is what will happen when humans take off their headsets: "And so another urgent question is whether people can enjoy the storied reality of finitude after coming down from the high of fake infinity. Can being merely human suffice? Can the everyday miracle of the real world be appreciated enough? Or will the future of culture only be viral? Will all markets become Ponzi-like fantasies? Will people reject physics forever, the moment we have technology that’s good enough to allow us to pretend it’s gone?”
Indeed, the unique characteristics of the technology require a different discourse. "With any technology, as we add more and more mediation to the world instead of experiencing it directly, we pay less and less attention to the world,” says Segal, who works for a startup developing a technological infrastructure to map the world in 3D, enabling the addition of a virtual layer to every location.
"Phones, for example, make us notice less and less of the things we look at, even through the camera lens. But the vision of augmented reality is that we will be able to look at the world through these glasses - and that is already an unusual and completely different thing. On top of that, we will be able to communicate with animals, with plants, inanimate objects and with other people. Maybe we can even hallucinate with other people."
It sounds like a psychedelic experience.
"That's right, both psychedelic and therapeutic experiences, and more layers that we have in our relationship with the world. In ancient societies, magic was an integral part of everyday life, and in my eyes the most significant value of augmented reality is that we can bring back things that we have lost - to share dreams, fantasies, and visions with other people. To talk with creatures and entities we invented. These are creatures that will be part of the world, that will develop anatomically."
Give me an example.
"Consider my ability with one command to say 'and let there be light' in a dark room, and there will be light. It provides strength. And there are even more far-reaching options. We can, for example, transform ourselves into different animals, feel what it's like to be an octopus. If you have a headset and I have a headset, we can connect to one body, and distort the way we perceive time. One of the interesting things is to somewhat challenge the laws of physics or break them.
"On the more romantic side of technology, I imagine myself going somewhere and planting a flower next to a street bench, then coming back a few days later and see the flower growing and curling around that bench. These are small things that can be really exciting.
"And of course there are things that connect me to sustainability. I can learn about things that are usually transparent to us, for example walking on a main street in Tel Aviv and getting current information about everything that exists there. About the amazing trees that we don't look at - what species are they? When were they planted? How do they breathe? Maybe I can make an animation of their breathing and observe it. I can get information about their history, for example what they looked like 50 years ago."
"On the more romantic side of technology, I imagine myself going somewhere and planting a flower next to a street bench, then coming back a few days later and the flower growing and curling around that bench. These are small things that can be really exciting.
"And of course there are things that connect me to sustainability. I can learn about things that are usually transparent to us, for example walking on a main street in Tel Aviv and getting current information about everything that exists there. About the amazing trees that we don't look at - what species are they? When were they planted? How do they breathe? Maybe I can make an animation of their breathing and observe it. I can get information about their history, for example what they looked like 50 years ago."
The disturbing conclusion from Pokemon Go
The first model of virtual reality was presented as early as 1838 by the British scientist and telegraph inventor Charles Wheatstone. The stereoscope he invented simulated the natural way in which the two eyes look at an object from two different angles, in order to produce three-dimensional vision from their combination. The field became commercial in the mid-eighties with the establishment of the start-up VPL Research by Lanier and his partner Thomas Zimmerman. The two, who at the time recruited employees from Apple after the invention of the Macintosh, bet that the technology giant would enter the market in 2010. Although they sold equipment to large customers such as NASA, they knew that end-user adaptation would take much more time. In practice, it took another 14 more years.
And yet even today, says Segal, there is not enough thinking about the field: "It is found in the depths of academic research and not on the front as in the field of artificial intelligence." There is also no regulatory body that will limit the steps of malicious actors - which will surely come, faster than we can imagine. "There is a paradox here because it is a magical world that has infinite freedom, but in the end it will probably be controlled by individual players who may use this magic for bad purposes. The technology companies want the headsets to replace the phone, that they will be the new way we experience the Internet, play games and create content. They are in a race among themselves to map the world, without us knowing about it and before there was a public discourse about it. And this is very dangerous."
What is dangerous about that?
"For years we have been moving towards the concentration of power in the hands of individual rulers, companies and countries, and we have less control over our lives than we think. We are constantly under the illusion of control, and the Internet plays a large part in this. The first years of the Internet were very romantic, and today everything is just a command away. Anywhere, anytime, I'm under the illusion that I'm in control, that I can do anything, I'm connected, I know everything, I can set up a conversation with you and research everything about you on Google, and you can check everything about me.
"But this comes at a price - our presence on the web in general and on social media in particular is surveillance software. The technology giants follow us at any given moment and know what we do, and what we think about. So now, with AR glasses, they will mine information from 3D mapping of the world, which will add more dimensions and give them more power. Take Pokemon Go for example."
What does Pokemon Go have to do with it?
"Pokemon Go (a game in which players try to locate Pokemon in augmented reality using a mobile app) is the only app that was successful in augmented reality. The game was also successful because it rode on beloved and popular content, and there was also a challenge— catching Pokemon out in the world is fun. But all in all, behind the scenes of this thing, the only reason the Pokemon Go app exists is for Google to have a 3D map of the world. Because people running with their phones open are people scanning the world all the time. That information flows to Google, because Pokemon Go was built by Niantic, which is a subsidiary company of Google, so in the end it's always about data.
"If we take it one step further, then AR glasses that are easy and convenient to use are actually a very powerful computer that sits on our face, and knows everything about us at any given moment, from the very fact that it monitors our vision and scans the field of vision and collects all the information we gather this way. And it's not like a phone that's in your pocket at least 20% of the time. It's constantly on your face. We could very quickly find ourselves in a world where reality and our sensory experience are controlled by corporations - and that's not science fiction."
Give me a nightmare scenario.
"It starts with digital distraction. Already today there is personalization of the content we receive online, which is based on the personal information collected from us, so imagine spatial personalization - a certain type of information that will be adjusted for me and another type of information that will be adjusted for you, and someone will constantly shape this world."
What does that mean in general?
"For example, I will see objects in the world that you will not see. And this creates gaps in the experience of reality between me and you, which can turn into real social gaps. People from different classes will experience the same reality differently. It will create a rift between people, when the rift we already know today is only the tip of the iceberg of what could happen in the future."
"Being on the side of humanity"
"Above all, the digital deception bothers me the most," says Segal. "That is, the blurring between the real and the not real - who knows what elements will control the stratified spaces, what manipulations they will want to work on us, what they will plant there, how realistic it will be. Will we lose touch with reality, will we have a way to know that what we see a meter away or 500 meters from us is real or not real?
"One of the problems already today is that people are very captivated when they see realistic images, it creates a kind of 'wow effect', people want to see such images - and the connection of this with the need of malicious entities to play with our minds is problematic. A realistic object that is in space and is controlled by a corporation or a state that wants to do bad things — that's a problem."
Give me an example.
"If I want, for example, to eliminate the presence of all the women in a space, I can do it. A little artificial intelligence, a little augmented reality, a little recognition of people in the space, and boom — I did it. This creates a situation where we are constantly required to question what we see and experience, and live with a feeling of insecurity. Not to mention that there will not be a uniform reality that people will experience. And if you also connect political interests to control stratified spaces - then the social division that we know today from the Internet will be nothing compared to what may still happen."
In what way?
"Today we can still take solace in the fact that we are able to disconnect from the phone and the computer. You can decide to leave Twitter, because there is a reality beyond the phone screen that is full of bots, fake news and manipulations. But in the future reality may be controlled by corporations and political parties. Imagine a country that has a minority of people it doesn't like, a persecuted minority. It can promise some corporation that it will give them a tax break in exchange for designing a reality that suits it."
You are part of the AR industry. Do all the threats you describe affect your actions?
"Yes. I constantly struggle with this conflict myself and try to teach my students - think about what you are building. Media researcher Prof. Douglas Rushkoff talks about being on the side of humanity ('Team Human'), meaning constantly checking that we are building things in a way that really serves us and not because we indulge in some financial or technological spin.
"So this is something I constantly think about: why do I build what I build realistically? Is it just because I have a technical attraction to it? Because it's not good, and we have to overcome it. I also think that artists, designers, technical people, should strive not to imitate reality in a distorted way, but to do fantastic things. This field was created so that we can see things that are a little beyond.
"Already today, people intuitively understand these problems, even people who do not understand anything in the field, both because of the aesthetics of augmented reality and because of the way it is placed in front of our faces. And this aesthetic is symbolic, it signals to us that we are losing a little grip on reality, because it is being hidden from us, because we are being locked in a bubble."
