Friday, May 28, 2010

Consciousness (14): Interpretation Mechanics

Number fourteen in my series of posts on consciousness. Table of Contents is here.

The perception-as-interpretation view, summarized in the previous post, is useful as an informal ordinary-language hypothesis about consciousness. However, it is obviously a long shot from a final literal scientific theory. While it seems to be a useful way of speaking, I think we should look at it as an interesting suggestion, or perhaps even an inspiration that will lead us toward a more specific and fleshed-out theory.

There is a diverse range of theories that turn out to be special cases of the perception-as-interpretation hypothesis. These theories describe the interpretation-building mechanism alternatively as neuronal ‘model building’, ‘emulation,’ ‘virtual reality construction,’ ‘simulation of a world’, or ‘unconscious inference.’ Each specific theory carries slightly different assumptions about how the brain constructs experience, but most of them share two or more of the general features of the interpretation view delineated in previous posts.

Among psychologists, the most influential of these views is that the brain uses an unconscious inference procedure to construct a hypothesis about the source of a retinal projection. Because this theory of perception is so interesting, influential, and useful, I’ll describe it in a little bit of detail before stepping back to speak more generally about all of these theories.

Marty the Brain Scientist
To help us understand this theory, let’s imagine a tiny scientist, Marty, who lives and works in your brain (Figure 1). His sole occupation, every moment, is to monitor the movies playing on your retinae, and to build a hypothesis about their source in the world. Our conscious experience is identical to the specific hypotheses that Marty generates. For instance, if Marty’s best hypothesis about the source of the stimuli is that there is a red ball three feet to your left, that is precisely what you will see. Marty is fairly motivated to generate hypotheses accurately and quickly: after all, if you die, he dies. The more accurate his hypotheses, the better you will be able to interact with the world.

Figure 1: Marty the brain scientist.

Retinal movies are Marty’s primary source of evidence. He uses such evidence, along with various assumptions and background knowledge about how the world works, to generate hypotheses about the source of the observed projections (that is, he makes an inference about the source of the stimuli). We are only conscious of the outputs of Marty’s vocation, not any details of his inference-generating procedures. Hence the hypothesis that perception is unconscious inference.

Hypothesis formation is a special case of inference. It is a type of inference that doesn't enjoy the level of certainty granted to deductive inferences (as you’d find in mathematical proofs). Rather, when we form a hypothesis we are often throwing out our best hunch, an educated guess based on limited evidence and previous assumptions about the way the world works. Philosophers sometimes call this type of inference an ‘inference to the best explanation’ or ‘abductive inference.’

For instance, say the evidence we wish to explain includes late-night scratching sounds in the cupboard and small fecal nuggets deposited in the pantry. We could use such evidence, and our general understanding of how the world works (mice are nocturnal, etc), to construct a hypothesis that would best explain the evidence. In this case, we would likely hypothesize that there are mice living in our kitchen. Perhaps Marty settles on his hypotheses about visual stimuli using a similar process of abductive reasoning.

Obviously, Marty is merely a useful fiction. Nobody thinks there is literally a little man in your head viewing your retinal movies. Advocates of this theory believe that we will ultimately be able to give a more literal story that describes how brains construct hypotheses based on information coming in from the retinae. In the meantime, I should spell out why the unconscious inference theory appeals to psychologists.

The appeal of the theory
There are three main reasons for the theory’s appeal (aside from its impressive intellectual pedigree since Helmholtz (1866)). For one, the theory would explain how certain illusions are generated. For instance, recall Shepard’s Monsters (Figure 2) from post eleven. Shepard explains the illusion as follows:
[T]he linear perspective of the subterranean tunnel (along with other depth cues, such as the relative heights of the projections of the two monsters on our retinas) supports the automatic perceptual inference that one of the two monsters is farther back in depth. The two monsters, nevertheless being exactly the same size in the drawing, subtend the same visual angle at the eye [i.e., their projections occupy the same surface area on the retinae]. The visual system therefore makes the additional inference that in order to subtend the same visual angle, the monster that is farther back in depth must also be larger.
Notice how the idea of inference-making is built into multiple layers of Shepard’s explanation of the illusion. The brain makes inferences about which monster is further away, and then uses this information to make further inferences about which monster is larger, which explains why one monster looks bigger than the other. Note the claim isn’t that the brain only uses inferences in cases of illusions (how would the brain know if it were seeing an illusion or not?), but that illusions help reveal the underlying inferential machinery of normal perception.

Figure 2: Terra Subterranea, or Shepard’s Monsters

A second appeal is that the theory finds a mathematical home in probability theory and statistics. The brain lives in an uncertain world, and even the brain’s own responses to identical stimuli are not the same every single time (that is, the brain itself is a “noisy” processor). In mathematics, the principles of sound inference in such uncertain contexts are provided by statistics. Couching theories of brain function in the language of probability and statistics allows psychologists to state their theories with more rigor than can be done in ordinary language. Perhaps most importantly, such theories allow them to generate quantitative predictions that can be tested against the data.

Figure 3: The eye lives between a noisy brain and an uncertain world.

The third appeal applies to the ‘unconscious’ side of the ‘unconscious inference’ thesis. That is, it seems pretty clear that the processes which generate our perceptual experiences are not consciously accessible to us (as discussed in post ten and post thirteen).

Hopefully this gloss on the unconscious inference theory of perception was half as fair as it was brief. At this point I don’t want to push too hard against it (for instance, you would be right to ask what it means for the brain to perform an inference). Rather, my goal was to showcase the most prominent species of the perception-as-interpretation thesis. More than one-hundred years after Helmholtz initially suggested the hypothesis that perception is unconscious inference, Fodor and Pylyshyn were able to describe the theory, without much overstatement, as the ‘Establishment theory' of perception.

Representations within interpretations
Enough with unconscious inferences: what about all the other theories I mentioned above, such as the view that the brain builds a ‘simulation’ of the world? I am going to avoid jumping down the historical rabbit hole of comparing/contrasting the often subtle differences in this panoply of psychological-level theories of perception. Rather, it will be more productive to extract a common denominator shared by all of these theories, something all of the advocates would agree upon. If such a common factor turns out to be useful and correct, then great. If not, then we will have eliminated an entire class of models of perception with one parsimonious swing of the blade. This seems much easier than starting by contrasting every such theory pair in detail.

The one theoretical commitment shared by all these theories of perception is that the brain constructs representations of the world, and the contents of such neuronal representations are the contents of experience. Our first priority will be to analyze this idea of neuronal ‘representation’: what the heck does it mean, and how far can it take us in our quest to understand visual experiences?

While I won’t analyze the notion yet, the notion of a ‘representation’ should be intuitively familiar to most of us. Three squiggly lines on a map represent water. A photograph of someone represents the person. I’ve already sneaked in the claim that the brain constructs a ‘portrait’ of the world: a portrait of something is one type of representation. The claim we will evaluate is that one component of the brain’s interpretation of a stimulus is an internal representation of the world constructed partly based on that stimulus.

Before heading into brains, however, in the next post I will finish this chapter by posting a broad range of quotations from the literature on the topics we have explored in the last eight posts. This will help us to see how these ideas of interpretation, simulation, representation, etc are used in practice.

Fodor, JA, and Pylyshyn ZW (1981) How direct is visual perception? Cognition 9: 139-196.

Helmholtz, H. von 1866 Concerning the perceptions in general. In Treatise on physiological optics, vol. III, 3rd edn (translated by J. P. C. Southall 1925 Opt. Soc. Am. Section 26, reprinted New York: Dover, 1962).

Shepard, RN (1991) Mind Sights, W.H.Freeman & Co Ltd.

Table of Contents of my posts on consciousness.

Friday, May 07, 2010

How to run R code in Matlab

R (site here) is a great open-source environment for statistical analysis. But I'm a Matlab user. Luckily, it is pretty easy to run R code from Matlab. Since I just set it up in my Matlab environment, I thought I'd write out the recipe I followed. I have only done the following in Windows XP, and I used Matlab version 7.8. I think it will only work in Windows. It assumes you already have R and Matlab properly installed on your computer.

Of course, this doesn't mean I don't have to learn how to use R, it just means I get to do it all in Matlab (and note for fellow Matlab users, there is a great cheat sheet that shows how to translate between the two).

1. Install the R package rscproxy.
In R, enter:
to install the package.

2. Install the R(D)Com server.
Download it here. The server allows Matlab to talk with R. I installed it using the default settings without checking or unchecking any boxes. Note this server is built for Scilab, which is an open source version of Matlab, but it seems to work for Matlab too.

3. Download the Matlab R-Link toolbox
Get here, unzip the contents, and paste MATLAB_RLINK in Matlab's toolbox folder (or whatever folder you want). Be sure to add MATLAB_RLINK to your Matlab path.

4. Restart your computer.

5. Is it working?
To see if the toolbox is working, start Matlab and enter 'Rdemo' at the command prompt. This should evoke:
b =
1 4 9 16 25 36 49 64 81 100

c =
2 5 10 17 26 37 50 65 82 101

6. Have fun!
If Rdemo worked, you are ready to go!

For instance, enter the following in Matlab:
openR; %Open connection to R server
x=[1:50]; %create x values in Matlab
putRdata('x',x); %put data into R workspace
evalR('y<-sqrt(x)'); %evaluate in R
evalR('plot(x,y)') %plot in R
To close the connection to R, and the graphs opened from R, enter:

7. Problems?
If the above doesn't work, go to C:\Program Files\R, open the (D)COM Server folder, go to 'bin', copy 'sciproxy.dll', and paste it in C:\Program Files\MATLAB\R2009a\bin (obviously you may have a different path to Matlab's binary folder). Close Matlab, and restart your computer.

If that doesn't help, I probably won't be able to help, but go ahead and ask as someone might know. The site where you downloaded R-Matlab has some useful Q&A so you might inquire there.

8. Acknowledgments
This is basically an updated version of Kevin Murphy's site. Please let me know if anything here becomes obsolete.

9. Caveat (added 6/18/12)

From the comments section:
After using R(D)COM and Matlab R-link for a while, I do not recommend it. The COM interface has trouble parsing many commands and it is difficult to debug the code. I recommend using a system command from Matlab as described in the R Wiki. This also avoids having to install all of the RAndFriends programs. 

Thursday, May 06, 2010

Consciousness (13): The Interpreter versus the Scribe

Number thirteen in my series of posts on consciousness. Table of Contents is here.

While elaborating on the parallels between perception and language interpretation, we have unpacked many features of the nature of visual perception that should hold up even if we end up finding the view of perception-as-interpretation wanting. In this post I’ll briefly integrate the data and theory from the past seven posts into a more tidy and (hopefully) coherent story.

As we discussed in some detail in post ten, the contents of experience have properties that are, on the surface, quite different from the properties of the underlying neural machinery doing the experiencing. I can see an iridescent jewel two feet in front of me (that’s the content), but the vehicle doing the experiencing is neither iridescent nor two feet in front of me.

We can be intimately familiar with the contents of our experience while remaining in complete ignorance of facts about nervous systems. I hope I don’t offend my fellow neuroscientists when I claim that our species’ great artists, playwrights, musicians, and novelists have revealed more about the contents of experience than any neuroscientist. Yet most of these artists worked without knowing the most basic facts of neuroscience. The vehicles of experience are effectively invisible to us, while the contents of experience are as familiar as breathing. Anyone that has savored an authentic lobster roll from a rundown shack on the coast of Maine knows what it is like to revel in the contents of experience (and those who have not have yet to fully live).

In sum, the contents of our experience seem to be a neurally-constructed portrait of what is happening beyond the brain. The brain faces some rather severe obstacles if its goal is to make this portrait accurate. For one, a great deal of information is lost in the projection from the scene to the retina (a projection we discussed in some detail in post nine).

Consider the case in which a projection onto the retina is square-shaped. What can we say about the object that generated that projection? Assuming there are no distance cues present, the same square shape on the retina could be produced by a tiny square that is extremely close to the eye, a medium-sized square a moderate distance away, or a colossal square that is extremely far away. It could even be generated by non-square shapes transmitted through a distorting funhouse-type medium.

Purves and Lotto state the point nicely:
[T]he retinal output in response to a given stimulus can signify any of an infinite combination of illuminants, reflectances, transmittances, sizes, distances, and orientations in the real world. It is thus impossible to derive by a process of logic the combination of these factors that actually generated the stimulus[.]
In other words, given only retinal movies as data, the brain cannot determine with perfect accuracy the scene in the world that generated said movies. Given the often striking ambiguity of the source of a retinal projection, it is remarkable that our visual system usually locks in on a single perceptual response to a given stimulus. Even during bistable perception we typically experience one object at a time, not a superposition of two objects.

How does the brain settle on a unique percept when provided with an inherently ambiguous retinal projection? It seems the brain uses context (post eleven) as well as background assumptions and knowledge (post twelve) to help narrow down the range of reasonable interpretations. Bistable percepts merely serve to highlight those rare instances when these contributions from the brain are not sufficient to settle on a single interpretation for an extended period of time.

In general, while we know that the retinal movies strongly influence the brain’s construction of experience, our experience is obviously not a mere report or transcription of what is happening in the retinae. If it were, ambiguous stimuli wouldn’t spontaneously reorganize in such drastic ways such as we observe in the Spinning Girl and Rotating Necker Cube (post seven), the angles in Purves’ Plumbing would look the same, the tabletop dimensions in Turning the Tables would look identical (post twelve), the yellow and blue squares in Purves’ Cubes would look grey, Shepard's subterranean monsters would look identical in size (post eleven), etc..

Hopefully the previous seven posts have made it clear why psychologists often say that the brain constructs interpretations of stimuli in a context-sensitive way, based on background knowledge and assumptions, in the light of sometimes intense ambiguity of the actual source of the stimulus. If we were forced to choose between the false dichotomy of saying that experience is an interpretation of what is happening in the retinae versus a transcription of what is happening on the retinae, I think the choice is clear.

Richard Gregory (1966) summed up the view quite well when he said that “Perception is not determined simply by the stimulus patterns; rather it is a dynamic searching for the best interpretation of the available data.” Our visual experience is clearly the result of neuronal events downstream from the stimulus, a construction of an experience whose contents mostly include worldly events beyond the eyes. It is such worldly events that we must engage with, after all, and such engagement with the world determines whether we eat, reproduce, flee, or die.

Next up
In the next couple of posts we’ll persue the idea that perception is interpretation down more specific paths, looking at a prominent view that the mechanism of interpretation is a kind of unconscious inference, and finally we’ll end up heading into the brain, looking at the neuronal basis of these internal “portraits” of the world.

Gregory, RL (1966). Eve and Brain. London: Weidenfeld and Nicolson.

Purves, DP, and Lotto, RB (2003) Why we see what we do: An empirical theory of vision Sinauer Associates.

Table of Contents of posts on consciousness.