Peer commentary on Barsalou’s target article "Perceptual Symbol Systems" to appear
in Behavioral and Brain Sciences.

Murat Aydede

The University of Chicago
Department of Philosophy
1010 East 59^th Street
Chicago, IL 60637, U.S.A.

EMAIL: m-aydede@uchicago.edu

URL: http://humanities.uchicago.edu/faculty/aydede/

Abstract: It is argued that three major attempts by Barsalou to specify what makes a perceptual symbol perceptual fail. It is suggested that one way to give such an account is to employ the symbols’ causal/nomic relation to what they represent, rather roughly in the manner in which contemporary informational psychosemanticists develop their theories. But this fails to draw the distinction Barsalou seems to have in mind.

Barsalou seems to give the following answer(s). Perceptual symbols (PSs) are (1) modal, (2) analogical, i.e. in many cases they resemble their semantic value (e.g. referent), and/or (3) reside (are implemented) mainly in the sensory-motor areas of the brain.

The first answer isn’t helpful since the modal/amodal dichotomy is defined in terms of the perceptual/non-perceptual dichotomy. The most one can get from the text is that "[PSs] are represented in the same systems as the perceptual states that produced them" (§ 1.1). But this doesn’t help much either, because all we are told is that a PS is any symbol in whose causal production a perceptual state essentially figures, and which resides in the same system as the symbols underlying the perceptual state. But for this to work, we must have a workable notion of perceptual/non-perceptual systems. And I am not sure we do independently of a corresponding notion of perceptual/non-perceptual symbols. Hence it looks like defining modal/amodal symbols in terms of the systems in which they reside won’t break the circularity. Furthermore, if causal production by a perceptual state were sufficient to make a symbol perceptual, almost all symbols posited by the recent tradition Barsalou opposes would be perceptual: in a sense all symbols could be causally produced by perceptual states — if ‘causal production’ is understood as ‘involvement in causal etiology.’

The second answer is not really elaborated in the text but is confined to some short remarks, such as that the "structure of a perceptual symbol corresponds, at least, somewhat, to the perceptual state that produced it" (§ 1.1). Later in the article, Barsalou tells us a bit more but not much; not enough, anyway, to calm the obvious worries such terminology provokes. He says, for instance, that perceptual symbols resemble their referents (§ 3.2.8), and that certain kinds of changes in the form of the symbols systematically result in changes in their semantic value, i.e. in what they represent (§ 3.3). He illustrates this sense in his Figure 6. But he never tells us how to understand such language. This is worrisome since such discourse borders on the absurd when taken literally. Let me elaborate.

Mental representations (symbols) live dual lives. They are realized by physical properties in the brain. As such, they are physical particulars, i.e. they can in principle be individuated in terms of brute-physical properties. Indeed, Barsalou repeatedly reminds us that the pictures in his figures are nothing but a convenient notation to refer to certain activation patterns of neurons in the sensory-motor areas of the brain. These are the perceptual symbols. We can, then, talk about symbols as physical entities/events/processes realized in the brain. As such they have a form. But symbols also represent: they have a semantic content. Their content or semantic value is what they represent. The duality in question, then, is one of form/content, or as it is sometimes called, syntax/semantic. PSs must live similar lives. Barsalou’s second answer to my initial question seems to be an attempt to mark the modal/amodal distinction in terms of the form of the symbols, or the syntactic vehicles of semantic content.

Well, then, how is the attempt supposed to go? How are we to translate his language of "resemblance" such that his answer becomes plausible? What is it about the formal properties of vehicles (remind you, brute-physical properties of neuronal activation patterns) such that they are not semantically "arbitrary"? We are not given any clue in the text. But let me suggest an answer on behalf of Barsalou

Perhaps the formal property F of a certain type of symbol S realized in my brain is semantically non-arbitrary in that its tokening is under the causal/nomic control of instantiations of a certain property of a type of physical object that S refers to. But now, notice, if the answer to our question is to be elaborated along anything like this line, we have abandoned the project of specifying modal/amodal distinction in purely formal terms, and instead causally reached out to the world, to the represented. It is not accidental that such a "reaching out" has been at the core of recent philosophical literature on how to naturalize psychosemantics, of attempts to give a naturalistic account of how certain brain states or neuronal events in the brain can represent things/events in the world, can have semantic content. This is a matter of the relational/semantic lives of symbols, not their intrinsic/formal lives, from which I started on behalf of Barsalou.

For Dretske (1981) and Fodor (1987, 1990), two leading informational psychosemanticists, a concept, understood as a symbolic representation realized in the brain, is (partly) the concept it is because it stands in a certain systematic nomological/causal relation to its semantic value. If such relations were crucial to make symbols perceptual, then on this account, many informational theorists would automatically turn out to be defenders of PSS — not a desirable consequence. (Come to think of it, Fodor as a PSS theorist!)

I don’t think that any general attempt to characterize what makes a mental symbol perceptual/modal (as opposed to non-perceptual/amodal) can be given successfully in terms of the formal properties of vehicles. This is plausible if we firmly keep in mind what we are talking about when we talk about mental symbols: activation patterns of neurons in sensory-motor areas of the brain with a lot of physiological properties.

This leaves us with Barsalou’s third answer to my eponymous question. Perhaps this is indeed his main claim: (Almost) all human cognition can be and is done by the kinds of symbols (whatever their forms are) residing in the sensory-motor areas of the brain. But if this is Barsalou’s answer, it doesn’t establish any distinction. For one thing, it’s open to the defender of the necessity of amodal symbols in cognition to claim that amodal symbols, for all we know, may reside in such areas. For another, if this is all that the perceptual/non-perceptual distinction comes to, the target article loses much of its theoretical interest and bite.

Barsalou’s project is to establish a very strong and controversial claim, namely, that all concepts can be exhaustively implemented by mental symbols which are in some important sense exclusively perceptual. Without an independent and non-circular account of what this sense is, of what it is that makes symbols exclusively perceptual, I am not sure how to understand this claim, let alone evaluate its truth. But I claim this rather in the spirit of putting a challenge rather than an insurmountable difficulty, because I know that Barsalou has a lot to say on this and other issues I would have touched on had I had more space. Barsalou’s paper contains extremely rich and important material not all of which are clearly formulated and well defended. I very much hope that the target article would (at least) create the opportunity to remedy this weakness in the future altogether.

Acknowledgement: Many thanks to Jesse Prinz and Philip Robbins for their helpful comments.

References: