Consciousness (8): From perception to interpretation

Number eight in my series of posts on consciousness. All the posts are indexed here.
----------------------------

The previous two posts were a tour of ambiguous visual stimuli. Let's use these data to generate ideas about consciousness. Ideally, these ideas will lead to prediction-generating hypotheses about consciousness and clarify the explanatory target for neuroscience.

Interpreting Necker
Let's jump-start our thinking with a familiar example: your experience of the Necker Cube (shown on the right). This time, try looking at it with one eye. You should still experience perceptual bistability.

People seem to naturally gravitate toward describing bistable perception as an alternation between two different interpretations of a stimulus. The psychologists that study bistable perception do the same. For instance, Suzuki and Peterson (2000) say:

Bistable displays are displays that afford at least two potential interpretations even though the physical displays remain unchanged. [...] At any given moment, only one interpretation of a bistable display is seen; over time, the two perceptual interpretations spontaneously and stochastically alternate.

How are we to interpret the view that the brain interprets a stimulus? Is it a metaphor? If so, is it useful? Let's start by considering the nature of interpretation more generally, independently of the issue of conscious perception.

What is interpretation?
In general, to interpret something is to determine what it means. We are probably most aware of the need for an interpretation when we encounter difficult bits of writing. What fan of JRR Tolkien hasn't struggled to interpret Bilbo's pronouncement to his fellow Hobbits, "I like less than half of you half as well as you deserve"? What the heck does that mean?

Interpreting a complicated text can be a painstaking process that often requires a good deal of specialized knowledge. People build careers on their ability to interpret confusing legalese, complex poems, or arcane works of philosophy. Some philosophers are infamous for the patience and charity required to construct an intelligible interpretation of their work. For example, the oft-revered philosopher Ludwig Wittgenstein (1922) wrote, 'The thing is independent, in so far as it can occur in all possible circumstances, but this form of independence is a form of connexion with the atomic fact, a form of dependence.' Most readers will probably agree that it is hard to interpret Wittgenstein's sentence, that the meaning is not transparent.

While our need to interpret text is most obvious when we encounter tortured prose, technically speaking we interpret even the clearest expressions. The meaning of the sentence, 'George Washington was the first President of the United States,' is fairly transparent to most Americans. That is, interpreting the sentence is effortless, given our background knowledge. For someone just learning English, or someone with no knowledge of the United States, the sentence's meaning will not be so clear. For some philosophers, the meaning of the above quote from Wittgenstein might seem transparent. Transparency of meaning is not an intrinsic feature of a chunk of text, but depends on the background knowledge we bring to the text.

We have to be careful, as some texts might not mean anything, or if they do it might not be worth the effort to decipher them. Chomsky (1957) produced the famous sentence, 'Colorless green ideas sleep furiously' as an example of grammatically well-formed nonsense. Of course, we could generate grammatically ill-formed nonsense too: 'Gorp dilettante achieve on.' Whether such strings are literally meaningless is an interesting philosophical question that we won't explore. I include this discussion partly to highlight that I have been throwing around the term 'meaning' without defining it, a point we will revisit in the next post.

So far I've focused on interpreting expressions in natural language. However, people also interpret paintings, dreams, medical test results, pretty much anything. Psychologists used to be quite fond of asking people to interpret random smears of ink on sheets of paper (the Rorschach test). While these cases are interesting, to keep the discussion more manageable, in the next post I'll focus on the analogies between perception and interpretation of expressions in natural language.

With this rudimentary understanding of interpretation in hand, in the next post we will consider ways in which perception and interpretation are similar. While I will ultimately eschew thinking of perception as literally identical to interpretation, it is an analogy worth mining for ideas about conscious visual perception.

References
Chomsky (1957) Syntactic Structures Mouton, The Hague/Paris.

Suzuki and Peterson (2000) Multiplicative effects of intention on the perception of bistable apparent motion, Psychological Science 11: 202–209.

Wittgenstein, L (1922) Tractatus Logico-Philosophicus (Ogden translation) Cosimo Classics.

------------------
Table of Contents of posts on consciousness.

Consciousness (7): More Ambiguous Figures

The seventh in my series of posts on consciousness. All the posts are indexed here.
--------------------------------

In this post we'll finish the tour, started in the previous post, of ambiguous figures.

Motion
Some of the most compelling illusions include things that move. Indeed, every time we watch a movie we succumb to the illusion of apparent motion. As we saw in the previous post, a rotating Necker Cube evokes vivid bistability. The present group of ambiguous figures include moving parts that are essential for the illusion.

Ambiguous structure from motion
The following video looks like a cylinder rotating either clockwise or counterclockwise (its direction is bistable). There is no cylinder drawn in the video, just a bunch of randomly placed spots. The spots' motion is set to match the velocity they would have if painted on the surface of a cylinder, and this motion signal alone is enough to give the impression of a particular shape.



It sometimes takes more than 30 seconds for the percept to switch, so you might need to watch the movie more than once. I am able to make it reverse faster by rotating my finger around the bottom of the imaginary cylinder as if I were pushing it in a new direction.

Bistable See-Saw
In the following animation, the barbell-shaped object should look like it is flipping back and forth like a see-saw. Sometimes the see-saw crosses through the horizontal axis, and other times the vertical axis. You might even see the barbell rotating 'round and 'round in a circle, though in my experience this is rare.



I lock in fairly strongly to the horizontal see-saw, but when I cover up the bottom half of the image for a few seconds, this brings out the other percept.


One Plaid or Two Gratings?
Stare at the black dot in the following animation. While initially you probably see a plaid pattern moving upward, you will eventually see two translucent sinusoidal patterns (often described as 'gratings') sliding past one another. It took me almost 30 seconds the first time before the percept switched, so stick with it.





The Spinning Girl
One of my favorite illusions. This beautiful ballerina was created by Nobuyuki Kayahara. In which direction is the ballerina doing her pirouette? Most people see her rotating clockwise initially, but the stimulus is actually ambiguous, so you can also see her rotating counterclockwise.




If you have trouble getting her to switch directions, cover her body and look only at the shadows at the bottom of the image. With the number of cues reduced, you should be able to see the shadow change direction. Once that happens, slowly lift your hand while maintaining the new direction of rotation to reveal the new pirouette direction.

This illusion has been misinterpreted as a test of handedness, or a test of whether you are right-brained or left-brained. There is no evidence for these claims, and I'm not sure where the rumors originated.


Binocular rivalry
Binocular rivalry has been a workhorse for the study of consciousness. This is partly because, in addition to the extensive psychological studies of binocular rivalry, neuroscientists have locked onto rivalry as a model for the study of the neural basis of consciousness. While we'll look more deeply at rivalry in future posts, for now we'll treat it as just another cool bistable percept.

To experience rivalry in the following image, put a piece of paper perpendicular to the screen between the two images, so your left eye sees the face and your right eye sees the house (your face should be about six inches from the screen). Be sure to fuse the checkered circles in the center of each figure. Once you obtain fusion, hold it for a while and you will experience rivalry.


Most people do not see a simple fusion of the house and face, but rather the patterns alternate. For instance, you might see the house for a few seconds, and then the face will dominate for a while, and so on. That is binocular rivalry. During transitions, the new percept will spread across the old in a kind of traveling wave, in which case you might see a dynamic quilt-like pattern.


Ambiguous forms
In this class of ambiguous figures, perception alternates between often drastically different types of objects (e.g., face and vase). These illusions are probably better known than all of the others. They are used in advertisements and art, and there are so many on the internet that I can only show a tiny sample. I won't say much about them, as the titles suggest what the two objects are supposed to be, and most of them aren't very difficult to see.

Vase versus Face
The old standby in every introductory psychology textbook.



Duck versus Rabbit
Another classic. I like the following version (from Torrey (1970)) because the two interpretations seem equally likely.



Wife/Mother-in-law and Husband/Father-in-law
On the top is a beautiful young socialite and a nasty witch-like banshee. Below is a handsome gadabout and a wretchedly distasteful lecher.



Chef versus Dog
Tilt your head to the left to see the dog, and to the right to see the goofy French chef.



Nude woman versus Reagan face



Kissing Jesters
It alternates between a single jester facing you, and two jesters facing each other, their lips lightly touching.

Gypsy versus Narcissist
The top of the image shows the ambiguous version, while the bottom shows disambiguated versions (gypsy on the left and narcissistic woman looking into the mirror on the right).

Man's face or Woman Reading?
Our last figure. I would be remiss, in a tour of ambiguous forms, if I didn't pay homage to the great surrealist Salvador Dali. His paintings are filled with beautiful and sometimes hauntingly plastic forms. The following painting, 'The Image Disappears,' was painted by Dali in 1938.


It seems somehow appropriate to let Salvador Dali be the last stop in our tour of ambiguous figures. If you have any favorites that I haven't included, please let me know in the comments or via email.

Where we are headed
While ambiguous images are intrinsically cool, they also provide a window into the nature of visual consciousness. Based on these illusions, in the next post I'll make some general hypotheses about the nature of (visual) perception. These hypotheses will give us a target for the neuronal data, to which we will then turn.


Sources of Illusions
The structure-from-motion demo is supplementary material in Krug et al. (2008). The Bistable See-Saw is adapted from the ambiguous quartet illusion, which was described by Ramachandran and Antsis (1985) (a tactile version is described in Carter et al. (2008)). The plaid/grating illusion is from Stoner et al (1990). The Spinning Girl was created by Nobuyuki Kayahara, who works in digital design. The house-face image used for binocular rivalry is from Tong et al. (1998). Vase/face goes back to Rubin (1915), but the one here is from Fischer (1967). The duck-rabbit was published originally by Jastrow (1899), but the one here is from Torrey (1970). The mother-in-law/wife image was originally published by Hill (1915), and the husband/father-in-law was originally published in Botwinick (1961). Kissing Jesters is from Fisher (1967). Chef/Dog is from Wallach and Austin (1954). Nude/Reagan is from Fisher (1968), a paper that shows 30 ambiguous forms from the history of psychology. Gypsy/Narcissist is from Fisher (1967).


References
Botwinick (1961) Husband and father-in-law: A reversible figure. American Journal of Psychology, 74: 312-313.

Carter, O, Konkle, T, Wang, Q, Hayward, V, and Moore C (2008) Tactile Rivalry Demonstrated with an Ambiguous Apparent-Motion Quartet. Current Biology 18: 1050-1054.

Fisher, G (1967), Measuring Ambiguity, American Journal of Psychology 80: 541-557.

Fischer, G (1968) Ambiguity of form: Old and new. Perception and Psychophysics 4: 189-192.

Hill, We (1915) My wife and my mother-in-law. Puck November 6.

Jastrow, J. (1899) The Mind's Eye. Popular Sci. Monthly, 54: 299-312.

Kristine Krug, Emma Brunskill, Antonina Scarna, Guy M Goodwin, Andrew J Parker (2008) Perceptual switch rates with ambiguous structure-from-motion figures in bipolar disorder. Proc. R. Soc. B, 275: 1839-1848.

Ramachandran, V.S., and Anstis, S.M. (1985). Perceptual organization in multistable apparent motion. Perception 14, 135-143.

Rubin, EJ (1915) Synsopleved Figurer: Studier i psykologisk Analyse. [If anyone has the full reference please let me know]

Stoner, GR, Albright TD, and Ramachandran VS (1990) Transparency and coherence in human motion perception. Nature 344: 153-5.

Tong, Nakayama, Vaughan, and Kanwisher (1998) Binocular rivalry and visual awareness in human extrastriate cortex, Neuron 21: 753–759

Torrey, CC (1970) Trace Localization and the Recognition of Visual Form. The American Journal of Psychology, 83: 591-600.

Wallach, H, and Austin, P (1954) Recognition and the localization of visual traces. Am J Psychol, 67:338-40.

-----------------------------------
Table of Contents of posts on consciousness.

Consciousness (6): Reversible Figures

The sixth in my series of posts on consciousness. All the posts are indexed here.

Background: Why do we need psychology?
Instead of just diving into the neural data, let's take some time to examine our target, consciousness. Clarifying the features of conscious awareness will provide a more precise target for our neuronal theories.

The ultimate goal is to develop an understanding of consciousness at both the neuronal and psychological levels. The two approaches should coevolve until they fit together as nicely as our ideas about trait inheritance and DNA, as well as our ideas about action potential generation and single channel biophysics.

To flesh out the analogy with inheritance/DNA, let's consider the work of Gregor Mendel. By controlling the reproduction of different strains of pea plants, Mendel was able to measure many features of inheritance well before anybody had heard of DNA. His work provided an explanatory target for molecular biologists who unraveled the mechanisms only much later. Similarly, psychologists have gained quite a bit of knowledge of consciousness by simply studying consciousness, knowledge gained without focusing much at all on the specific neuronal mechanisms involved.

There are literally thousands of psychological experiments and clinical studies that reveal interesting features of consciousness. Obviously, we'll only be able to look at a tiny subset of these data. This means I'll need to curate the data with some caution: I must be wary of cherry picking, or focusing only on data that lets me push some pet theory. The right approach toward any pet hypothesis is to try to kill it with data. We should actively seek out falsifying evidence, not data that confirms what we already believe.

We have to start somewhere in this galaxy of data, so let's begin with a set of illusions that has entertained and puzzled psychologists, and the general public, for nearly 200 years: ambiguous stimuli. I start with them partly for their cocktail party value, but also because they are an excellent gateway into the psychology of conscious perception.


Ambiguous Visual Stimuli and Bistable Perception
A stimulus is ambiguous when it can evoke different percepts. That is, even though the stimulus is unchanging, our experience of the stimulus oscillates between two "interpretations." The alternating perceptual experiences are known as bistable percepts. (If this is confusing, hold on: many examples are coming up).

In this post I'll focus on cases in which the percepts switch back and forth between two identical objects (e.g., two cubes) that are seen from two different perspectives. They are often called 'reversible figures.'


Necker cube
The Necker Cube is probably the most famous reversible figure. It was first discussed in print in 1832 by a Professor of Minerology, LA Necker (Necker, 1832). The Necker Cube is at the top of the following figure. Looking at the line drawing tends to evoke alternating experiences of two cubes. These cubes are shown, in an unambiguous form, below the Necker Cube.


Perceptually, one of the cubes appears to come out of the page pointing down toward the left (the pink cube on the left), while the other cube appears to come out of the page pointing up to the right (the pink cube on the right).

While most people's visual system automatically generates a percept of one of the cubes (an amazing fact in itself), some people tend to stay locked into one interpretation. That is, they don't spontaneously experience bistability. If this is you, just keep staring and your percept will eventually switch. The longer you look at a reversible figure, the more frequently the perceptual alternation will occur.


Reversible Steeple
It is relatively easy to generate ambiguous drawings similar to the Necker Cube: make a line drawing of an arbitrary 3-D polygon and it is likely to generate bistable percepts. For instance, here is a five-sided solid, the Reversible Steeple:

One of the steeples points toward you (with the rectangular base further away), while the other points away (its base will be closer to you).


Schröder's Staircase
The following figure should appear as a set of steps with either the blue or the pink "wall" closer to you.


When the red wall is closest, it seems you are looking at a staircase from above (the more standard perspective such as when you are approaching a set of stairs to climb). When the blue wall appears closest, it will seem as if you are looking up at a staircase from underneath, or an upside-down staircase, or an overhanging unfinished brick wall (the latter two descriptions are from Wallin's book).


Plush Chair
You can imagine the following is one of those plush velvet chairs with brass buttons on the front and back.



One percept is of a chair facing you: you see the chair from above with the backrest facing you and the seat of the chair is coming out toward you. The other percept is of a chair facing away from you: you see the chair from below, with the back of the backrest facing you and the seat is going away from you.


Inverting Hairbrush
It appears to be a hair brush. It can appear either with the bristles facing you, or the bristles facing away.


Scripture's Blocks
This is one of my favorite reversible figure in this post, one of the more vivid cases of bistability. The image should appear as a set of long rectangular blocks stacked upon each another.


In one percept, each block is oriented down to the left, capped on the bottom by a white face. The hatched shading is the top surface of each block. In the other percept, each block is oriented up to the right with its white face at the top. In this case, the hatched shading coats the front surface of each block.


The scope of perceptual reorganization
I'll finish by illustrating the deep and sometimes startling nature of the perceptual reorganization during alternation. We'll look at two modifications of the Necker Cube.


Arrowhead Cube
I've placed two arrows on the "surface" of the Necker Cube below. Consider two questions. Are the arrows on the inside or outside surface of the cube? In what direction are the arrows pointing? As you probably guess, the answer depends on which cube you see!



When you see the down-left cube, then the arrows appear on the outside of the cube, and seem to point toward you. However, when you see the top-right cube, they appear to be painted on the inside surface of the cube, and to point backwards away from you.

Somehow, when the brain alternates between cubes, it takes note of additional features of the cube and integrates them into the percept in an appropriate way. It does this without you having to think about it, without you consciously knowing how you do it.


Rotating Necker Cube
The final bistable percept is my favorite of the bunch, the Rotating Necker Cube. It is a picture of a cube that is rotated by the same amount (in the same direction) with each time step. You should see a rotating cube. Does the cube still show bistability even when rotating?



Not only does the Rotating Necker Cube still alternate, but when it alternates it reverses its apparent direction of rotation! Once the percept switches, our visual system interprets the exact same movement as rotation in the opposite direction.

I will be devoting a future post to the Necker cube, as it is such a rich source of ideas and data.


Where we are headed
In the next post (maybe even two) we'll continue looking at ambiguous stimuli. This post has been a quick list of reversible figures, without much theory or discussion of consciousness. We will ultimately use these illusions to brainstorm about the nature of visual consciousness. Then we'll have something more precise that we can target from a neuronal perspective.


Philosophical dessert
Just as Mendel's laws were consistent with many possible molecular mechanisms, these visual illusions are consistent with any number of lower-level neuronal explanations. For that matter, the illusions considered in isolation are consistent with dualism (roughly speaking, dualists believe that the mind is not part of nature, that it is a different kind of thing altogether such as a soul). Illusions provide useful data that all people (not just neurophiles like myself) interested in consciousness should struggle to explain. Dualists of the world, get off of your armchairs!

Original Sources of Illusions
The Reversible Steeple is adapted from John Wallin's wonderful monograph Optical Illusions of Reversible Perspective published in 1905 (it is available free at Google Books). Schröder's staircase was first published in Schröder (1858). I got the idea for coloring the two walls of the staircase from planetpurplex.com, a site full of optical illusions. The Inverting Hairbrush and Plush Chair are both adapted from Wallin (1905). Scripture's blocks were introduced by Scripture (1897), though the figure used above is taken from Wallin (1905). The Arrowhead Cube is adapted from Mason et al. (1973).

I am not sure who first noticed bistability in the Rotating Necker Cube. If anyone knows the background, please let me know. Wallin (pages 46-47) says Wheatstone looked at moving Necker Cubes, but it seems Wheatstone just held wire cubes in his hand and contemplated them while he moved them about (see Wheatstone (1838)). Neither Wheatstone nor Wallin remarked on the apparent reversal of rotation, so the first observation was likely after the publication of Wallin's monograph in 1905.

References
Mason, J, Kaszor, P, and Bourassa, C.M. (1973) Perceptual structure of the Necker cube. Nature 244: 54-56.

Necker, LA (1832) Observations on some remarkable Optical Phænomena seen in Switzerland; and on an Optical Phænomenon which occurs on viewing a Figure of a Crystal or geometric Solid. The London and Edinburgh Philosophical Magazine and Journal of Science (3^rd Series) 1, No 5, 329-337.

Schröder, H (1858) Über eine optische Inversion bei Betrachtung verkehrter, durch optische Vorrichtung entworfener physischer Bilder. Annalen der Physik und Chemie 181: 298-311. [Note last name sometimes spelled 'Schroeder' or 'Schroder']

Scripture, E.W. (1897) The New Psychology. Walter Scott Ltd, London.

Wallin, J.E.W. (1905) Optical Illusions of Reversible Perspective: A volume of historical and experimental researches. Stanton Call Press, Stanton IA.

Wheatstone, C (1838) Contributions to the Physiology of Vision.—Part the First. On some remarkable, and hitherto unobserved, Phenomena of Binocular Vision. Philosophical Transactions of the Royal Society of London, 128: 371 - 394.

---------------------------------------
Table of Contents of posts on consciousness.