Why 3D Makes Your Brain Cry

Far more eloquent and well-learned people than me have weighed in on the 3D fad in movies and video games. I'm going to talk about how your eyes and brain actually process 3D, and why current 3D is just not good enough to replicate the effect with complete fidelity.

The first myth about 3D vision that needs to die a horrible, flaming, burning death is the misconception that closing one eye will destroy 3D vision. Human beings perceive depth in many ways, most of which are monocular--they require only one eye. Closer things are more detailed than faraway things. Parallax, the same visual illusion that makes railroad tracks converge in the distance, is another way to tell depth. Relative size of objects, objects overlapping, perspective, and relative apparent movement--these are all monocular cues of depth perception, and they work just fine with one eye shut.

However, there is a benefit to binocular vision; this kind of seeing in 3D is called stereoscopic vision. When the eyes focus on a particular object--let's say, a tree--the tree is in the center of the retina of both eyes. But because there are two eyes that occupy different horizontal positions in the head, slightly different images of the scene surrounding the tree will be projected on the retinas of both eyes. Some neurons of the striate cortex in the brain, part of the brain's visual processing system, respond and become extremely active when a visual stimulus produces retinal disparity--images on different parts of the retina of each eye. It's not that each eye sees a two dimensional scene, then combines them for a three dimensional scene. It's that each eye sees a slightly DIFFERENT three dimensional scene, and the difference between what the eyes "see" is what produces difference in depth perception.

When watching a 2D movie, each eye sees only monocular cues. There are monocular cues for depth within the flat plane of the screen, but no stereoscopic vision. What almost all 3D technology works on is fooling your eyes into producing retinal disparity. 3D movies project two images onto the screen, and place filters over the images to polarize the light differently. Viewers wear 3D glasses that have polarized lenses to match the polarities of the two different images of light.

For example, let's say that there's two images of a tree on the screen. The left image may be polarized to setting X, and the right image may be polarized to setting Y. The left lens of the glasses is also polarized X, so only the left image goes to the left eye, and the right lens is polarized Y so only the right image goes to the right eye. This is, of course, quite a simplification, but that's the essential idea. Bada bing--you have two different images going to your retinas, and hey presto, those little neurons in the striate cortex are fooled into responding to retinal disparity.

Except not really. First of all, the polarization only applies to light that's being projected onto the screen. Looking at other objects doesn't make them appear "more 3D than before," and there isn't a screen yet--unless you're sitting right under an IMAX screen--that eats up your entire visual field. You'll still be able to see the theatre, the walls, your fellow moviegoers, and the visual cues, monocular and binocular alike, from these more mundane things won't mesh with the stereoscopic cues you're receiving from the screen. There's no way that giant tree can be that detailed, and also occupying the same space as the guy sleeping through the movie in front of you. Linear stereoscopic glasses even require you to not move your head due to the way they are polarized--turning your head will wreck the illusion.

In addition, stereoscopic glasses do nothing for improving monocular cues of depth. Things like shadows, light, and edges. Ever notice how 3D in movies resembles a pop-up book more than it does real life? The edges are too sharp and clean, and the transitions between objects and backgrounds are too abrupt. This is especially noticeable in live-action movies that have then been adapted for 3D. Our brains in real life combine monocular and binocular cues for the illusion of depth, but that synergy simply doesn't work on the silver screen. Take something like detail. Most movies are shot or animated in very crisp detail, and while the image may "pop" due to stereoscopic glasses, the monocular cue of less detail in the background may not be present. This leads to a "fake" looking 3D experience in which monocular and binocular cues don't match up.

I'm not saying there's anything WRONG with 3D. It's an interesting technology, and it displays some of what we understand about visual processing between the eyes and brain. But in no way does it capture and readily combine all the cues we use in the real world for 3D.

The Neuroscience of Christopher Nolan's Inception: Part 2

As a movie, I very much enjoyed Inception, and I'm currently prodding my family to see it again with me. But I'm not addressing the movie's merits as a movie.

Now that I've seen the thing and no longer have to rely on conjecture, I have a few more things to say on elements of the movie, mostly based off of areas that I misinterpreted given the limited information before the movie's release. Obviously, spoilers follow.

1) The difficulty of inception: I've already covered why planting an idea in someone's head really isn't all that hard. Interestingly enough, the movie's version of inception is more complex. It involves planting an idea and making sure that person believes the idea is his/her own. Cobb's argument for the difficult of inception is that people can always trace the source of their ideas if they were from another person. Which is, unfortunately, patently untrue. Tracing the source of an idea involves remembering where it came from. Neuroscience tells us that our memory is reconstructive. We don't remember events or facts in perfect detail--instead, the brain reconstructs events in a fashion that makes sense to it. This is why eyewitness reports, despite being prized by juries, are factually less reliable than other evidence. The ability of any individual to trace the exact genesis of an idea is pretty wishy-washy.

2) Knowing what you don't know you know: The concept of subconsciously knowing things that you're consciously unaware of is older than Freud, and actually has some truth to it. Certain kinds of learning do not require conscious knowledge of learning. Muscle memory is an excellent example of this, illustrated by the famous patient H.M., who had a disastrous surgery for epilepsy that left him unable to form new memories, and presumably an inability to learn new things. However, H.M. consistently improved at motor tasks like tennis, even though every time the researchers handed him a tennis racket, he claimed he had no idea how to play. Another good example of being consciously unaware of knowledge comes from split-brain patients, who have had the connection between the right and left hemispheres of the brain severed. The two halves of the brain then act almost independently, without sharing information with one another. From split-brain studies, it appears that for an individual to be consciously aware of a stimulus, the stimulus has to be processed somewhere down the line by the left side of the brain--the side responsible for language. It might be a stretch, but it seems like conscious knowledge of something depends on our ability to put it into words.

3) The nature of reality: The ending didn't really come as a surprise to me. It would be strange if a Christopher Nolan mind-screw of a movie gave us an unambiguous ending. However, the questions it raises are more complicated than the simplistic "are we brains in a vat" version. As I said in an earlier post, the difference between REM brain activity and awake brain activity comes down to the involvement of the prefrontal cortex, and the availability of sensory input. The way the dream machine worked in the movie seemed to preserve prefrontal cortex functioning for everyone in the dream: Cobb, his team, and Fisher. Otherwise, there is no way they would be able to plan, reason, and keep track of the elaborate heist.

Now, part of the reason why we can distinguish dreams from reality is their sheer weirdness. Without the prefrontal cortex to make sense of the activity of the sensory association areas of the brain, dreams flat out don't make much sense when we wake up. However, the dreams of Inception are fundamentally different from "normal" dreams. First of all, they are constructed by not the individual's sensory association cortices, but by an architect who pulls everyone into a shared dream. Secondly, the prefrontal cortex is active during Inception's dreaming. At this point, I'm venturing into the realm of conjecture, but it seems reasonable that a dream in which the prefrontal cortex is active and making sense of things would indeed be very difficult to distinguish from reality.

Overall, though, I did really like the film. And while a lot of Inception's claims either contradict or muddle what science has discovered about sleep and dreams, I have to give the movie props for constructing its own internal logic about dreams, and sticking to it. Hopefully, I'm off to see it again :)

The Neuroscience of Christopher Nolan's Inception

A Google search for "dream interpretation" brings up over eleven million results. Pop psychology certainly loves this bit of Sigmund Freud's psychoanalytic theory. For Freud and his intellectual successors, dreams were a psychological gold mine. They offered a unique glimpse into the unconscious mind while the conscious mind slept. And Freud was hardly alone in his fascination with our nightly visions: almost every human culture has ascribed some kind of importance to dreams. Joan of Arc supposedly predicted her own death in dreams. The "inexplicable cultural phenomenon" (MST3K guys' words, not mine) that is Twilight occurred to author Stephanie Meyer in a dream. And now director Christopher Nolan takes us into a world of dreams with drugs and a strange machine that sends Leonardo DiCaprio into an individual's dreaming mind.

His purpose? To steal ideas. Inception is based around the premise that individuals can share dream states, and that bizarre machine we've seen in snatches of trailers seems to be the key. So how realistic is all this?

First, a quick crash course on the neuroscience of sleep and dreams. Sleep, typically considered a passive state where nothing's happening, is actually a time of great changes in brain activity. When we sleep, we go through cycles of approximately ninety minutes a piece, with multiple stages of sleep per cycle (conveniently numbered in chronological order). Stage 1 and 2 sleep are relatively shallow--in fact, if participants in sleep experiments are awakened during this period, they might not even remember being asleep. Stages 3 and 4 are what scientists call "slow-wave sleep," after the large slow delta waves that show up on EEGs measuring brain activity. If awakened from this very deep sleep, people are typically confused and groggy.

Yet these four stages have nothing to do with dreaming. In fact, it's only after going through stages 1-4 that we reach the dreaming state of sleep--REM sleep. REM, which stands for rapid eye movement, refers to the state of sleep in which the brain is insanely active. Brain activity of a person in REM sleep bears a lot of resemblance to that of a person who's awake. If startled from REM, people will be alert, responsive, and almost always report that they've been dreaming. However, REM itself doesn't account for much of the time we're asleep. With about four to five sleep cycles per night and about a half hour of REM sleep per cycle, we only enter REM for about two to two and a half hours per night.

That's two hours, give or take a half hour, that we're dreaming and conscious. That's right--conscious. Strike one against Inception, which assumes, as common sense would have it, that dreamers are unconscious. Startlingly, it turns out that we're actually not down and out for the count while we're dreaming, and our brains are aware of what's going on. For example, say you're having a dream about running along the beach. The parts of your brain responsible for motor control become active, as if you were actually running. The visual association cortex of your brain--the part responsible for making sense of information from your eyes--is extremely active, literally "seeing" the beach. Even your eye motions are not random--they resemble the eye movements of a conscious person scanning a scene for information. Your heart rate and blood flow to the brain increases, as does your brain's oxygen consumption.

So why, if you're actually conscious, do dreams make so little sense when you wake? How do you get the crazy dream-scapes seen in the trailers of Inception? One possible explanation is the lack of activity in certain parts of the brain during REM. The prefrontal cortex, located behind your forehead and eyeballs, shows low activity during REM. The prefrontal cortex's job is to make sense of things, plan, execute higher order functions like logic and reasoning, among other functions. The lack of activity in this part of the brain while dreaming may explain why dreams are very vivid, but don't make a whole lot of sense upon waking, as Leo informs Ellen Page in the movie trailer.

Here's where the premise of Inception begins to get a bit implausible. There are indeed common patterns among mental states. Generally speaking, individuals show consistent patterns of brain activation depending on what their mental state is. Someone looking at faces has brain activation in the fusiform face area, on the underside of the brain. Someone listening to music shows activation in the temporal lobe, responsible for hearing and sound. In theory, it would be plausible for two people's brains to share the exact same pattern of brain activation at a given moment in time. We obviously have no technology nowadays that would achieve such a feat, whatever movies like Inception or Avatar might imply, but it is theoretically possible.

However, Leo would be in trouble if he had the exact same brain activation pattern as the dreaming person. Remember that prefrontal cortex? The bit of your brain that makes logical sense of things? His prefrontal cortex would have to remain active in order for him to pull off a heist inside someone else's mind. Otherwise, he'd just be sucked into the dream with no particularly coherent sense of time or planning, just like the rest of us when we dream. So whatever technology he's using would have to replicate the mental patterns of the dreaming person in the sensory areas of his brain, but leave Leo's own prefrontal cortex unaltered.

But let's assume for the moment that Leo can share a dream state with another person, and still keep his own sense of planning, logic, and reason active. What will he find in another person's dreams? Deep dark corporate secrets? Not likely. The premise of Inception is that it's easier to steal secrets from someone's dreaming mind, when the mind is supposedly more vulnerable. However, there's no evidence that REM has anything to do with unconscious desires or deep dark secrets. There is evidence that in animals, REM sleep helps with learning new information, though the evidence is somewhat thinner with humans. Learning is correlated with the amount of REM sleep in humans, but there's no telling if one causes the other. Bottom line is this--Leo might have spent all that time and effort getting into someone's dreaming mind to find nothing useful there.

Even if there is something useful there, there is still the fundamental problem that dreamers are conscious. In neuroscience terms, consciousness roughly refers to awareness of thoughts, perceptions, emotions, and memories. When experimenters wake sleeping participants from REM sleep, they are alert and perceptive. When asked to describe their dreams, they show signs of consciousness, giving details of how their dreams looked, what they felt, what they did within the dream, etc. Even if their prefrontal cortices were inactive and their dreams are strange and disjointed, participants were still conscious in their dreams, even if it was different from wakeful consciousness.

Bottom line is this--Leo would most likely go through all that trouble with futuristic technology to enter someone else's dream state, only to find a dream that has none of the information he wants, and a very much conscious dreamer capable of reacting to events in the dream, including his intrusion. But that's just half of the equation. What about inception, his ultimate challenge of planting an idea in a dreamer's mind rather than taking it?

The film synopsis and marketing so far have made a huge deal of the difficulty of inception, but neuroscience and psychology tell us that planting an idea inside someone's head without them noticing is actually surprisingly easy, even when that person is awake. Hysteria about subliminal messages and brainwashing aside, it is possible to "prime" people subliminally in visual experiments. Priming involves rapidly flashing a word, too quickly for it to be consciously processed. If primed with one word, participants will tend to pick related words, like picking the word bedroom over the word elephant if primed with the word "house." This is just one example of the subtle but real ways in which ideas can be influenced without the person's conscious awareness.

In conclusion, Leo would really have an easier time of getting what he wanted if he stuck to good, old-fashioned conversation with a person while conscious. The premise of Inception is not entirely unrealistic, but it certainly is extremely impractical, even if the strange dream-sharing technology in the film did exist.