The predictive mind

Lynne Malcolm: I'm looking out at a beautiful seascape, the sky is a dazzling light blue, the aquamarine ocean is flicked with bright white waves and glistening yellow reflections of the sun. And nearby there are bottlebrush trees, covered in rich red flowers. If you were standing next to me, can I be sure that you are experiencing the same thing as me?

Bruce Goldstein: The answer is no. What you're referring to is the idea of an experience being private. Let's say two people are looking at some scene or something and they are describing it to each other and it seems like they are experiencing the same thing. Well, it turns out that we know that they perhaps are experiencing maybe about the same thing but we have no way of knowing exactly what they are experiencing. There is a term that philosophers use called qualia. Qualia is the essence of an experience, so if you are having a red qualia, then that is the essence of what redness is to you.

Let me give you an example to illustrate this idea of the private mind. Let's say that there's two people standing on the edge of the Grand Canyon. They are looking out and they are marvelling at how wonderful it looks and they are pointing out things to each other, and it seems like they are getting about the same experience. All of a sudden something called the experience fairy comes along (just bear with me on this) and taps them on the shoulder and all of a sudden their experiences switch. So now you're having the experience of your friend, and your friend is having your experience, and amazingly enough they are not exactly the same. The reds look a little redder perhaps than they did when you were looking at it, the far end of the canyon looks a little further away. There are subtle differences. That's what we mean by qualia. Each person's qualia is their own private experience.

Lynne Malcolm: Hi, it's All in the Mind on RN, I'm Lynne Malcolm. In this episode, a broad ranging exploration of the mind-brain connection and how it creates our conscious and unconscious experience.

With me is Bruce Goldstein, Emeritus Professor in Psychology at the University of Pittsburgh. His recently released the book is called The Mind: Consciousness, Prediction and the Brain.

Bruce Goldstein: The mind and the brain are intimately connected because the brain creates the mind. However, there's a big difference between them. It has often been said that the brain is something that's material. You can take a picture of the brain, it can be photographed, but you can't take a picture of the mind. The mind is invisible. And so that has led to a lot of arguments. There are some people who actually say that they are two totally different things, but most respectable neuropsychologists say that the mind depends on the brain, and most research is trying to make links between the mind and the brain.

Lynne Malcolm: And we'll hear later from Bruce Goldstein that he sees the brain as a predictive machine which has a predictive power that's key to our survival.

But first, let's go back for a moment to the idea we started on; how does our mind and brain create our unique experience of the world around us? Bruce Goldstein told me about an experiment that tried to demonstrate this. It involved studying the brain activity of a boy in a coma named Sam.

Bruce Goldstein: Yes, Sam is one of these people, there are many people who are in comas staring out into space, they are kept alive by life-support, and it doesn't seem as if there is anyone in there. So in this experiment, Sam was shown a film which was excerpts from an Alfred Hitchcock movie in which this little boy gets a hold of a real gun and goes around pointing it at people and threatening to shoot them and there actually are bullets in the gun. Well, there are places in this film where there is very, very high tension because of the drama that's going on. And then there's other places where there is not that much tension.

They put Sam in a scanner, in a brain scanner, and measured his brain activity as he was watching the film, and what happened was that during the places where there was high tension, his brain activity would go wild, and during the other times it wouldn't. And then they did the same thing for people who were not in a coma and they had the same pattern. So Sam was reacting in the same way as people who weren't in a coma. So there must be someone in there, there's just no way to know. This is a really good example of the idea of a deprived mind. You would think Sam would be totally not there, but he actually was, although we have no way of knowing exactly what he was experiencing.

Lynne Malcolm: So how does the nervous system, or our physiology, if you like, create experience?

Bruce Goldstein: That's the $10,000 question. I can tell you that we don't exactly know, but let me try to give you an idea of how people are thinking about that. One way to think about it is the idea of looking at what kind of signals are going on in the nervous system that are causing our experience. And when you do that, what you're looking at is nerve impulses in neurons. And when you look at the neurons you see that what nerve impulses are are sodium and potassium ions moving across membranes. So what you're talking about is a chemical process.

So the basically the answer to the question is that there is a chemical process that somehow causes us to have experience, and then the question is; how in the world would ions moving across the membrane result in experience? As it turns out no one can answer this question, but for good reason this has been called the hard problem of consciousness, and the reason it's called a hard problem is because no one can figure out how to solve it. And in fact people aren't even trying to solve it because it's so hard. They are looking at the relation between the brain and experience in a different way. What people do instead of researching the hard problem, they research what is sort of tongue-in-cheek called the easy problem, and the reason they call it the easy problem is because it's easier than the hard problem but it's still pretty hard.

Lynne Malcolm: And what is that 'easy problem'? Researchers are studying the neural networks of the brain to understand the physiology of experience.

Bruce Goldstein likens this to the way traffic moves through a city.

Bruce Goldstein: You have the highway system and at various times of day the highway might be, for example, lots of traffic going into town in the morning, and then later on traffic going out of town or traffic going to a sporting event. So the activity in the road system is constantly changing. The same thing happens in the brain but we are talking about nerve networks rather than a road system, and depending on how you're thinking, depending on your state of consciousness, whether you're sleepy or not, it changes the activity in this brain network.

Lynne Malcolm: We've known for many years that different areas of the brain are specialised for different functions, but it's emerging in research that this idea, referred to as 'localisation of function', is over-simplified.

Bruce Goldstein: The idea that a particular function was represented in a particular area of the brain. So one of the examples is faces. When people look at faces there's a certain area called the fusiform face area, and that has a lot of activity, and that is true. So there's nothing wrong with that. The problem is that when you actually look at the bigger picture, there's activity in a wide scattered area of the brain, what we call networks.

So the fusiform face area is connected to many other areas of the brain. When you look at a face, many different things happen. You recognise the face, you read an emotion into the face. You might have other kinds of reactions to the face. These all activate different areas of the brain and parts of the network, so we're getting back to this idea of a network with a traffic pattern in it, and depending on how you are reacting, let's say you look at a face and the face is an angry face and you get afraid, all of a sudden the activity in the network is going to start going over towards the fear area of the brain, and so on.

So basically the present-day thinking about understanding the brain and experience is that we are not talking about single areas, we are talking about networks, and they often extend over huge areas of the brain, from the front to the back, from side to side, when you see these pictures of networks.

Also another way to look at it is let's say you're trying to solve a problem, you have certain networks that are involved in thinking, but as you are solving the problem, the activity goes to different parts of the brain as you think of different aspects of the problem. So again, it's changing across time.

Lynne Malcolm: And Bruce Goldstein explains that most of what we experience happens outside of our conscious awareness.

Bruce Goldstein: We're aware of our experience, but we are not very aware of what's going on behind the scenes in the brain that causes that experience. The classic example of that is seeing. You just open your eyes and you see. It becomes hard if, let's say, it's a foggy day, there's not that much light, but most of the time you just open your eyes and you see. What's so complicated about that? Well, it turns out it's so complicated that people have been working on it for over 100 years and they still haven't figured it out completely. There's a lot of processes that go on in the eye, in the brain, that are happening in millisecond timescales that go on without our awareness and yet then all of a sudden we see.

One of my favourite vision examples is what you do when you look out at a scene and you are moving your eyes from one place to another. Typically when you look at a scene, you're moving your eyes two or three times a second from one place to another just so you can see different parts of the scene. If you were to peer into the person's eye who is doing that, you would see that there are little images on the back of the eye, and every time they move their eyes, the image smears across the retina on the back of the eye, which makes sense because the eye is moving. But we don't see the smear. Why don't we see the smear? Because there's a signal being set down to the brain that is telling the perception area; watch out, the eye is about to move, don't be fooled, just keep the scene steady. This is an example of behind-the-scenes activity that is keeping the scene steady, we don't see it smear every time we move our eyes.

Lynne Malcolm: You're with All in the Mind on RN. I'm Lynne Malcom, and I'm with Bruce Goldstein, Emeritus Professor of Psychology at the University of Pittsburgh. In his recent book The Mind: Consciousness, Prediction, and the Brain he highlights our amazing capacity to make predictions about the future. Behind the scenes, our brain is often making predictions in the areas of vision, language, music, memory and social interaction. And this predictive power is essential to our survival.

Bruce Goldstein: It's one of the hottest topics in psychology today. It really started 20 or 30 years ago, and people came up with the idea called prediction error, and what that means is that we are constantly making predictions about things as we are going through our lives.

Let me give you an example. Let's say you're just taking a walk in the woods and you've done this many times before and you sort of know what the sounds are like in the woods and so on. All of a sudden you hear this strange sound coming from the right and it's almost like there's something running through the woods, and you stop, you look, and you find out that perhaps it's an animal coming towards you or something like that.

Okay, so what happened there is you're going along, your predictions are fine as you're walking along, and then all of a sudden something happens that is an error, it violates what your prediction is, and you have to take it into account. We are doing this all the time. Any time we have something that happens that is not what we expect, then you generate a prediction error, and then you have to take action to correct it and to make the prediction error go away. It's a very adaptive way of behaving.

So the idea is that this prediction error actually comes from the brain, it's from what's going on in the brain, we have expectations and we are always testing what's going on against our expectations. There are many mathematical models that try to explain how it works. No one really knows how it works.

Lynne Malcolm: All right, so let's just go through some of the ways in which the brain is a prediction machine, as you say. An example is when we read, we're making a lot of predictions when we read.

Bruce Goldstein: Well, every time you're reading anything, let's say you're reading 'John put on his bathing suit and decided to…' well, you know that it's going to have something to do with jumping into the water or going to the beach or whatever. When you're having a conversation with someone, you very often can predict what the next thing is they're going to say. And if they say something surprising all of a sudden, well, that's a prediction error and you have to do take it into account.

You know, it has been said that people who have lived together for long periods of time, they almost know what someone is going to say before they say it. So there's a lot of predictability in language, both in conversation and in written language.

Lynne Malcolm: And so when we read, is there something physiological happening that contributes to this prediction around what's going to come next, say in a sentence?

Bruce Goldstein: Yes, there is. The brain is always thinking ahead and making predictions, and I can give you an example of a physiological experiment. They measured a response called the event related potential. For example, they would tell someone, here is a sentence; 'There was a nice breeze, so the girl went outside to fly a…' Okay, so what do you think is going to be after 'a'? Well, probably 'kite'. That would be the predictable response.

When you have a predictable response, the event related potential stays pretty low energy, there's not a lot of activity, there's not a big response. But if the sentence went like this; 'There was a nice breeze, so the girl went outside to fly an…' All of a sudden what would 'an' be? I guess it could be an aeroplane or something, but that's very unexpected. And so all of a sudden the event related potential perks up and you get a big response.

So the message here is (and this has been shown in many different experiments) that when there is a prediction error, or if you want to use another term, just if something unexpected happens, the brain perks up and there is a large response. It's like a warning response almost.

Lynne Malcolm: Just as it does with language, the brain's predictive mechanism also influences the way we hear and experience music.

Bruce Goldstein: In music we're not talking about meaning like we do in language, but we are talking about notes, about what notes are going to come next. Let me try doing a great moments in music here, and I'm going to sing you a song. [Sings notes of 'Twinkle Twinkle Little Star'] There was little bit of a pause there and that was a little bit disturbing probably because you wanted to hear that last note. Hearing that last note is what's called return to the tonic. It's actually the same note as the first note, and there's sort of a rule that people tend to want a melody to return to the tonic. That's prediction, they are predicting it's going to happen. It doesn't happen all the time but it happens very, very often in music. That is an example of prediction happening in music that has been shown through many experiments. If you are measuring brain activity and the person is expecting the melody to go back to the tonic and it goes somewhere else, then you get this big burst of activity in the brain, just like I explained for the language.

Lynne Malcolm: And also repetition in music creates a prediction too, doesn't it. You get repetition in even the vocals in music.

Bruce Goldstein: Right, a lot of music, just about all there is is repetition, over and over again. But if you look at classical music, there's often repeating themes over and over, the themes come back even from one movement to the next. And people are expecting that to happen. Of course the classic example of repetition in music is Beethoven's Fifth Symphony. And then if you keep on listening to that movement, I don't know if anyone has ever counted how many times that is repeated but it's many, many times, and you expect that to happen and it's just something that it really does in many respects add to the music because you are predicting and it makes you feel good to be able to predict what's going to happen next.

Lynne Malcolm: So can we predict, for example, from our memory of the past to what might happen in the future?

Bruce Goldstein: That's a good question. What seems to be the consensus now is that what memory is for is actually predicting the future, and it makes sense, when you think about it. Memory is telling us how the world operates. You know, when I did this, a good result happened, when I did this, a not so good result happened. And so when you have that information, that helps us behave correctly, behave in an efficient way in the future.

One of the experiments that makes a link between memory and predicting the future is an experiment where they measured brain activity in a brain scanner of a person remembering a past event, and you got activity in the frontal lobe and some activity further back. And then they asked the person, okay, why don't you think about a similar event that might be happening, say, a week or two in the future. And when people did that, they got activity in exactly the same areas. A little bit different, but mostly the same areas. So the activity of thinking about something from the past and thinking about what might be happening in the future turns out to be very similar.

And, interestingly enough, there are some people who have brain damage who cannot remember what happened from the past, what we call episodic memory, memory for life events from the past. And when they are asked to imagine something that might happen in the future, they can't do it. Let's say imagine going to a party next week, they have a very difficult time doing it and they can't remember a party that they went to a week before. So there is a very close link in the brain between the memory from the past and what we are going to predict is going to happen in the future.

Lynne Malcolm: And how does our capacity to predict influence and impact the way we interact with other people, our social interaction?

Bruce Goldstein: Well, social interaction, that is perhaps the most difficult kind of prediction because you're dealing with other people. There is a term called theory of mind, and what that means is let's say you are interacting with another person, and let's say you don't know them that well but you know them a little and you're having this interaction with them, and what you're probably doing is you're coming up with some kind of idea about what they're going to do next or what do they mean by saying that, you're trying to understand what's going on with them. And when you're trying to do that, you are creating a theory of mind about the person. You're trying to create a theory of what's going on in that person's mind. And we do this constantly, all the time as we are interacting with people. We may not even realise we're doing it. And so we are constantly making predictions. And since people are so interesting and different and unpredictable, sometimes what we predict does not happen. And then of course we can get into trouble in social relationships.

Lynne Malcolm: Young children, especially before the age of about four years old, do not have the same theory of mind as most adults have. It develops through childhood. A famous experiment which demonstrates this theory of mind is called the Sally Anne Task.

Bruce Goldstein: What you have, you have an adult who has two dolls and they are telling a story to a child. Let's say it's a child who is three years old. So the story goes something like this; here's Sally, Sally has a box. And here's Anne, Anne has a basket. Okay, Sally has a doll in her box, and Sally decides to leave the room. So Sally goes out of the room to play. Now that Sally is out of the room, Anne takes the doll from the box and puts it into the basket. But Sally isn't there, so she doesn't see her do this.

Okay, then Sally comes back into the room, and where do you think Sally is going to look for the doll? So if the child is a young child, three years old or less, they will most of the time say, 'Sally is going to look in the basket.' Why will Sally look in the basket? Well, that's because the doll is in the basket. And sometimes there is some questioning, and it almost seems like the child sort of realises that Sally wasn't in the room so didn't see it, but it doesn't register. So they are not taking what's going on in Sally's mind into account, that she couldn't know that the doll was moved to the basket, that she wasn't there.

Now, if you do the same thing on a four- or five-year-old child, this is really interesting, I have seen videos at this, the adult asks the five-year-old, 'Okay, when Sally comes back into the room, where is she going to look for the doll?' And they look at the experimenter like they're crazy, like, 'What do you mean where's she going to look? She's going to look in the box. That's where it was when she left.' So they know, they have a theory of Sally's mind that the three-year-old did not have.

Lynne Malcolm: As adults we also make predictions as we interact socially and this is known as mentalising.

Bruce Goldstein: Let's say you're near a hill and you see a rock rolling down the hill. Well, you don't usually say 'the rock wants to get to the bottom of the hill'. That would be an act of mentalising but we don't usually give mental motivations and so on to rocks. But if you saw a person running down the hill, you might make something up, you might say, 'That person is running down the hill because they want to catch a bus,' because you know there's a bus stop at the bottom of the hill.

And then let's say that a bus turns the corner and the person starts running faster, then you say, 'Oh yes, I'm pretty sure the person is running down the hill in order to catch the bus.' So that's an example of making up a hypothesis about what the person is doing, what's going on in their mind, why they want to go down the hill, based on evidence that you see in the environment.

And we are doing this all the time. Any time someone is doing something, we are often making up some reason why they did it. If you're good at it, and people with good social skills are better at mentalising than people with poor social skills because they can tell why people are doing what they are going to do. And of course this brings up the issue of autism because people who are autistic have a very poor mentalising ability. They are not very good at trying to figure out what someone is doing and why they are doing it.

Lynne Malcolm: What can this ability to predict teach us generally about how the mind works?

Bruce Goldstein: What it teaches us is that the mind is adaptive. The mind works in order so that we can survive, and the way we can survive is to be able to predict what's going to happen in the future, and if something happens that is not what we would predict, that we can take steps to deal with it. It's as simple as that really.

Now, it's very complicated when you start getting into the physiology and the psychology of it, but when you come right down to it, that's what it's all about, it's about doing what the organism needs to do to survive. What this means is we are not just talking about people here, animals also do predictions and have prediction errors. Most lifeforms that have survival value have some kind of prediction mechanism.

Lynne Malcolm: What are some of the newest breakthroughs in mind and brain research which will make a significant difference to our understanding of the mind and consciousness in the years to come?

Bruce Goldstein: Well, there's a lot of different things that are going on. We've been talking about networks and this is one of the big areas. It's like trying to figure out how these networks work, how does traffic pattern change in these networks for different situations. About two weeks ago they had a conference that is usually in Tucson Arizona, which is where I happen to be right now, it's called The Science of Consciousness. And interestingly enough there were a number of sessions in that conference on psychedelics. So this is a big new area of research. You know, it used to be back in the '60s, of course people who will remember that, they will remember Timothy Leary and people who were taking psychedelic drugs to expand their consciousness. Well, now what's happening is that people are taking psychedelic drugs in laboratories to expand their consciousness, but people are put in brain scanners and they are measuring their network activity and they are trying to figure out how changes in consciousness cause changes in activity in the brain. This is pretty exciting stuff. So that's one of the areas of research that is trying to figure out what's going on in the brain in different types of consciousness.

Lynne Malcolm: But still, it's the so-called hard problem which remains unsolved.

Bruce Goldstein: Remember we called it the hard problem of consciousness, and the reason we called it the hard problem is because the hard problem is how does the mind actually cause consciousness to happen? And even if we are looking at the easy problem, which is what is it about different areas of the brain, how the activity occurs in different areas and how traffic patterns change for different states of mind and different tasks and different experiences, these things are still a huge mystery. A tremendous amount of progress has been made but I think we still have a long way to go before we really understand this.

Lynne Malcolm: Bruce Goldstein, Emeritus Professor in Psychology at the University of Pittsburgh, and author of The Mind: Consciousness, Prediction and the Brain.

Thanks to producer Jane Lee and sound engineer today Marty Peralta. I'm Lynne Malcolm, bye till next time.