This is the interview with Matthew Fisher from episode 063 of the You Are Not So Smart Podcast.
Download – iTunes – Stitcher – RSS – Soundcloud
In this episode we explore what happens when a human mind becomes aware that it can instantly, on-command, at any time, search for an answer to any question, and then, most of time, find it.
David: You mentioned early on in the paper that human beings use a variety of cognitive tools to make our lives easier. I think that’s sort of a framing that maybe some people will go, “What are you talking about?” Or, what is some examples of those cognitive tools?
Matthew: Yeah so we actually start – and maybe it’s easier to start with physical tools. And we can think how they extend our physical abilities. So in the paper we talk about – for example – a baseball glove. So you can think of a baseball glove as just an enhanced hand. It’s specifically crafted for a particular goal. And just makes it easier to catch a ball. And it’s perfectly crafted to fit the human biology, and we can take advantage of it and become better at a task because of it. And so – similarly, there are cognitive tools. So these have been around for a millennia. So you can think of writing as a cognitive tool. So without it, if I want to remember something – I need to store that internally. I need to take whatever fact I’m trying to hang onto. And actually spend the mental energy to hang on to that piece of information. But with a piece of paper, I can outsource that task. Instead of me holding onto that memory, I can have the piece of paper do it. So if I jot down your birthday instead of me having to hold onto that from day to day – I can now just always go back and rely on that piece of paper. So without – broad definition of what a cognitive tool is, we can see that they’re all over our environment and that a lot of – what culture is, is sort of building these up and making us more and more efficient at the things we’re trying to accomplish.
David: Yeah and it’s kind of, it’s really neat because your research taps into something that thinkers have been thinking about for a long time. I know that even the Greek philosophers, some of them were like, “We shouldn’t have scrolls, because scrolls externalize our knowledge and that makes us dumb.” And calculators. Even when I was a kid, I remember old people being like, “Yeah, back in my day we didn’t have calculators.” Like, “Well, that sucked for you, ’cause these are very useful.” And that’s sort of what your research is definitely tapping into, something – when you think about these external tools– I mean, I know that there’s a great book by Daniel Dennett called, “Intuition Pumps.” Where he talks about…you were saying language – he was saying a metaphor is a cognitive tool and there’s some like bigger ones in philosophy like thought experiments, the Chinese Room, stuff like that. One thing I think that is a – is a – extremely useful cognitive tool that we often– That we use, every single person uses this and doesn’t realize that it’s a thing. And it’s sort of an extension of like what photo albums are. Anything like that where you store information that queues up your memories is interdependent memory system you write about. But other human beings are also part of our interdependent memory systems that we use every day. One of those things is a transactive memory system. What is that?
Matthew: Yeah so, along the same line as a piece of paper. We’re outsourcing knowledge to a piece of paper. We can create sort of intricate systems of memory with other people. So the term that’s been used to describe these sorts of systems is transactive memory. So one really easy to understand example would be an old couple – and old married couple trying to recount a story from around the time they first met. So you can imagine that on their own, if you were to interrogate them 1 on 1 – they may each have a pretty difficult time fleshing out the details of the story, and coming up with the complete narrative. But you see this interesting thing where if you ask them together, “Hey, what happened on that day back then?” What ends up happening is that one person can remember 1 part, and it sort of cues the memory of another – and of the other. And they end up cueing each other back and forth. And this dynamic process emerges where the story that they’re able to recall is much richer than the sum of the individual parts. And so in these sorts of memory systems, it’s important to know – not only what you have stored internally, but you need to know where to go to find out the information you need. So you can think of this at a broader like societal scale. And what some people have called the division of cognitive labor. So the idea is that each of us are responsible for a particular area of knowledge. And then we are sort of interconnected in this web of ideas to help each other accomplish our goals. So you see this in science a lot. If I’m in a biology lab, I need to be trusting the work of the chemist and the statisticians and the physicist. And kind of together, even though maybe I personally can’t understand all the knowledge in each of those domains – I know who knows what, and I can rely on them. And so, these 2 ideas – transactive memory and the division of cognitive labor – they show how dependent we are on other people to fill in the gaps in our own knowledge. And sort of help us work around areas where we might not know how something works.
David: So cool, ’cause a lot of the best scientists who really put out a lot of papers, and put a lot of material – they often find a buddy in the world, in their silo. And they bounce ideas back and forth with each other. And you find they have– Either they write books together, or they write papers together. And I always think of them as being like old couples. Like Kahneman and Tversky and stuff like that. They are– I just interviewed Lee Ross. And Ross and Nisbet have – did brilliant things together. And I think that – I can only imagine the weird big cloud of knowledge they share, and they don’t– They’re not exactly sure who knows what, but they know – they’re aware of that cloud of info. And it’s also a crazy thing that you – when you think of a scientist, you think of a person who knows everything. Like Doc– Like Marty McFly and Doc Brown. You’re like, you think of Doc Brown. Like what was he a scientist of? Like he just is a scientist of some kind. But when you really get to know scientists, you realize that they’re usually extremely specialized in one thing – and then know almost nothing about anything else.
Matthew: Yeah but – yeah – I think our main point is that even though they may not know the intricacies of those other areas of knowledge, they often are well connected with people who do. And that’s what – that is what matters, yeah.
David: Yeah, it’s always so great to ask – person in one silo about another silo. And they’ll say, “Well I don’t know that, but here’s 16 people you can contact.” And they’re just so used to that being part of how they interact with the world. That they push that on to you, always think that’s so neat. And so talking about this – this – so this transactive memory system. One of the most innermost immersing features of it is that though you have a feeling that you know stuff, and you share a bank of knowledge with either one person like in a marriage – or with lots of people, like in a scientific discipline. There’s a sense, there’s an intuition that you know it, but you don’t – you don’t always know exactly how much of what you think you know is in your head, or is going to require pinging other people inside that system. And it seems to be the leap you’re making into – the technological version of this. Am I on the right path?
Matthew: Yeah, that’s right. So those are sort of the 2 main ideas we are interested in. So first is that the internet can serve as a transactive memory partner. So just in the way – as I was describing, how we rely on others to fill in information. We can do that with the internet – it’s just a much greater extent. So in a certain way it’s, the internet can serve as the ideal memory partner. So it’s fast, it’s accurate for the most part – and it’s always available.
David: It’s always awake, always ready. Always willing, always willing to answer your questions in your pocket. It’s always there, yeah.
Matthew: Exactly, and it doesn’t have to– And you have very little responsibility to it. So it’s not asking you back for questions, it’s just this one way interaction where we can sort of depend on it’s knowledge to fill in our own. And so the second idea we wanted to test was – to what degree does the boundary between the externally stored knowledge and our own internal knowledge get blurred. So we started to think that – we’ve become so reliant on these other forms, or these other– This outsourced knowledge. That perhaps we’re underestimating just how reliant we are. And we may be assuming that the knowledge we have access to is actually our own, and it’ll lead us to think we know more than we actually do. And so that’s what we – we wanted to test in our paper.
David: Okay so let’s get this all on the table. We – this internet is sort of as you write – an expert in all domains, and– But it goes further than a meat based person knowledge source would go. ‘Cause it’s our old school brains really readily accepts the internet into a transactive memory relationship. But we sort of lose track of how much we rely on it. Just like we would with a person. But it’s just – it’s such a magnified version of that. That our – our miscalibration is also magnified in a way. Is that right?
Matthew: Yeah, yeah well so the question of – to what degree we lose track of how much we know and how much the other people know in transactive memory systems – actually hasn’t been tested. So that’s something we’re interested in doing. We’ve been looking at this effect in the internet and asking this new question. To what degree do people conflate their own knowledge with other knowledge? And we’ve been looking at [?] effect with the internet. And it very well could be the case that that applies to other forms of transactive memory, like the social networks. But it’s still an open question. I suspect that it might not apply to the– You may not see as strong an effect. when you look at those systems. But it’s a question that deserves more research.
David: Well that’s awesome, and I’m glad that there’s a – that you’re headed there. And also you’ve accurately estimated your knowledge on the subject. So yeah, but we – anyone who’s listened to this show knows that we already think of ourselves as above average in most regards. We already overestimate our abilities and our competence and our knowledge in general. And so it makes sense that we would over estimate what we know when it comes to a transactive memory relationship. How much we know, versus how much is stored in another location. And people in researched seemed – you hypothesized early on that people really underestimate how much they rely on the internet. Because when you successfully look something up, it feels like in some way you kind of mastered that bit of knowledge. And you actually write – quote – ’cause I love this quote. “You erroneously include knowledge stored outside of your own head as your own.” So the – so what’s important here, and this was a very important distinction for me, because I think that a lot of headlines about your research don’t seem to catch this. Which is that it’s – it’s a misattribution. Not – it’s not– I think the knee jerk reaction is that, “Oh so the internet makes us stupid. Okay, I thought so.” But it’s more of – it’s more about a misattribution and being overconfident, and having a sort of false self-image, right?
Matthew: Yeah, it’s a misattribution of where the knowledge is. So after you’ve been plugged into this sort of omniscient source, then that effect sort of bleeds into your assessment of yourself. So you end up feeling smarter after having accessed the repository of information on the internet.
David: So there’s like a sum total of knowledge, and you just – you think your contribution to it is greater than it really is.
Matthew: Yeah that’s it, that’s a good way to put it.
David: So and before we get into the actual research, there’s a question I want to get out of the way. Which is – so do all of us somewhat savvy internet users and cyber people – do we think we’re smarter than we really are on a level above what the typical person believes?
Matthew: That’s a good question. So you’re right that these sorts of biases just purveyed reasoning more generally. That we just tend to think we’re better than we actually are. So in our experiments, you can sort of assume that that baseline, that sort of high baseline of self-knowledge, of self-assessed knowledge is already built in. And so what we’re finding is that even above and beyond those effects that would already be there – that are sort of the normal processes of the human mind. That the internet can boost those, and make those more extreme. So, I will work through some of the details of the experiment. But we’re – we have controlled conditions showing the people were isolating the effect specifically of the internet. So above and beyond those other biases in human reasoning. And it seems like the internet is exaggerating them to some extent.
David: Oh boy, see now – there’s a headline right there. There’s a big one like, your biases are super charged, or your biases are on steroids. However you want to paint it, I can see the inflammatory headline. Okay, so let’s go through the– How did you study this? How did you research this?
Matthew: Yeah so, what we did is – we ran a series of experiments. So 9 in total. And we used a – I’ll describe the basic paradigm that we used throughout this series of studies. So what we did is we had participants, and we tested them online. And we had them randomly assigned to the internet or the no email internet condition. So participants would be asked a series of questions. Sort of common knowledge questions that were moderately difficult. Things that maybe you have an intuitive answer – but when pressed, may have difficulty coming up with a complete explanation. So we would ask participants things like, “Why are there phases of the moon? How is glass made? Why are there dimples on a golf ball?” And participants in the internet condition were asked to go find the answer to that question online, and then report back to us the URL that they found most helpful. And then – in one version, participants in the no internet condition were told the actual answers to the question. Just directly in the experiment. So what we did is we had – we took the information from the URL that participants found most helpful. And we just provided it directly to participants in the internet condition. So what’s neat about this set up is that participants in both conditions are seeing the exact same information. It’s just in one case, participants are using internet to go look it up. And in the other case, participants are just being provided with it directly.
David: So what kind of – what kind of questions and information are we talking about?
Matthew: Yeah so, things like – the sort of questions that have an intuitive appeal, but you can’t completely answer. So, “How is glass made? How – why are there phases to the moon?” Things like that.
Matthew: And so this was just the induction phase of the experiment. So this is just getting participants to either be used to finding things online or not. And so, after the induction phase, we then had all participants complete the self-assessment phase. So this was when participants told us – this was our way of getting them to tell us how much they think they know. And what we did here, is we asked them a series of questions totally unrelated to the questions that they had learned about previously. So we asked participants to rate how well they could answer questions about things like health or history or pop culture. Things that were just very different from the things they had looked up before. And what we ended up finding is that after having searched the internet, participants rated their ability to answer these unrelated questions higher than the participants who had been just given the information directly, and not used the internet to look it up.
David: This is wild stuff, man. That is so crazy. ‘Cause some of the questions like you have in your paper, like, “Why are there more Atlantic hurricanes in August instead of September – August and September?” And the idea that I would think I could answer that question after searching the internet a little bit on something completely unrelated is really – is really bonkers concept.
Matthew: Yeah so, the way we phrase those questions – so we would – we would give them a sort of sample of questions from a particular domain. So one of the areas that we asked them to assess was questions about the weather. So we would say – like the hurricane question, we would then ask them, “How do tornadoes form? Why are cloudy nights warmer?” And then the – but we actually had them assessed. Was – how well could you answer questions similar to these? So we sort of give them an example of the sorts of questions within that domain. And then have them assess sort of more generally – how well do you think you can do at this sort of task? And so – but we were trying to tap into kind of a more general notion of knowledge, rather than maybe some specific, particular question. And so, what we ended up finding – like I said – is that people sort of generally think they’re able to answer these questions better after having searched the internet.
David: So you this is a very cool finding, and there’s all sorts of things – like we just talked about forever. How – all these implications. But you – what I like about this study is that you go ahead and replicate a number of times, trying to make sure you have all your t’s crossed, and your i’s dotted.
Matthew: That’s right.
David: So what are some ways that you dug deeper and sort of tried to suss out exactly what you were seeing here?
Matthew: Yeah so, there are a number of potential worries you may have with that experimental set up. So things like participants spending more time in the internet condition, and maybe just thinking about these questions for longer – makes you feel smarter. So we ran an experiment, controlling for the amount of time participants took on the task in each condition. We also – we’re worried about the exact question we were asking at the end. So we were, we were telling participants – we’re asking them, “How well could you answer questions similar to these?” And you might be concerned that participants in the internet condition just interpret that question very differently than participants in the no internet condition. So if I’ve just told you to look up a bunch of answers online to these questions I’ve given you – and then I give you some new questions – you may well assume that when I’m asking you to assess your knowledge, what I’m really asking is, “Hey how well could you answer these questions, given that you could look up the answers?” So we ran a couple of experiments to rule out that explanation for our results. So in one study, what we did is – instead of having participants just answer that question, we gave them a little cover story. So we said, “Participants–” We said, “People who explain questions better have more activity in their brain.” And so this was just kind of made neuroscience for the purpose of this experiment. And then we had them – the slider scale, that either went from a barely lit up brain, to a brain with – lit up all over. And we had them assessed using that scale, what their brain would like when they gave an explanation. And the idea there was – we want to really zero in on participants assessing knowledge in their own head, not including things that they have access to. And this was just one way for us to get at that question. We also took a more straightforward approach in a follow up experiment. Where we just included in the instructions how well could you answer questions similar to these, without using any outside sources. And what we found is that – with the new phrasing, or using the brain slider dependent measure – you still get the effect. So effect replicates, and participants in the internet condition give themselves higher ratings.
David: I love that they’re like, “Yeah, that really active brain – that’s me.”
Matthew: That’s it. Yeah, and one other thing we’re interested in is trying to get out the why question. So – what is it about the internet and the process of search that is leading participants to give these sorts of ratings? And so what we did at the end of the paper is tried to uncover the mechanism kind of leading to these results.
David: Yeah. This is the part that I was like, “Oh that makes so much sense.” Of course, in hindsight of course. But yeah, go ahead. You started to really dive in on – maybe it’s the search itself. It’s the actual – the way that we get into the internet. The actual verb of searching the internet. Maybe that’s it. So how did you explore that?
Matthew: Yeah so, we did this a couple of different ways. So, one way we did is – we sent participants to a URL that was a filtered Google search. So maybe you’ve used these settings on Google, maybe if you’re looking for like a recent article. You can filter by things posted within the last week or the last month. And sometimes that’s really helpful. But for the sorts of questions we were asking participants, that was actually really unhelpful. So we would – when participants in the internet condition were looking up questions like, “Why are there phases of the moon?” The results they would see when they searched for it would only be things that had been posted in the last week. And we made sure that the posts that showed up on the Google results were not very helpful to answering that question. And so participants would go through the same task I described earlier. But now participants in the internet condition were getting results that really weren’t that helpful to answering the question. And what we found is that – even with these irrelevant results, participants still rated themselves as knowing more in these other domains during the self-assessment phase of the experiment.
David: Okay, okay, okay – hold on. So the – in one instance they got bad info when they searched. In another instance they got irrelevant or no info when they searched. And still, they still thought that their knowledge, they rated their knowledge as higher – even though they weren’t getting the results out of the internet that would increase their knowledge. So what’s going on here?
Matthew: Exactly. Yeah so, what we think is going on is that – there’s something about being embedded in the search environment that really matters. So just the very act of typing in your Google search query to this engine that you understand can basically access the world’s knowledge. So it leads you to assume that you are contributing to that more than you actually are. That you are less reliant on that source than you actually are. And so, independent of what you’re actually learning from the internet – just the ability to search it seems to matter in terms of inflating our assessments of what we think we know.
David: God, that’s so weird. And the – so what is the– The first thing that pops in my mind, and I’m sure that you guys thought this too. Is – how’s this different from say 1972, some bookworm who loves to go to the library and is always looking up stuff – or maybe a person who works at the library and is always searching for people – patrons who come in and need information. Is there – is this absolutely unique to the internet? Or is it – what’s going on there?
Matthew: Yeah, it’s a great question. And I’m aware of some work at other labs – are starting to explore this question. And what seems to be going on, is that the internet is unique. And the way I see it is that, the internet has a few features that those sources of information just don’t have. It’s – these were – what I mentioned from the onset. And that’s that it’s fast, it’s reliable, and it’s always available. And so, I think to the degree to which other sources of information exhibit those properties, you may see a similar effect. So you can imagine a librarian maybe from an era past that was looking through the card catalog very efficiently, and could access all this information super quickly. Maybe she would exhibit the effect as well. But the ability to just have all of this knowledge in our pockets, on our smart phones at all times – seems like a fundamental shift in just the availability and the degree to which we can rely on this information. So one way to think about it is that the – our effect is basically showing showing that people confuse their knowledge in their head with knowledge stored externally. And so, to the extent that you can make that external knowledge feel more like your own. Then it’s – you’re more likely to see this effect. So while an encyclopedia in the basement is accessible, it doesn’t feel like it’s accessible in the same way that Wikipedia is for example.
David: Right. And even if there is a similar thing – to a much lesser degree with the encyclopedia in the basement or the– Even if there is a similar thing going on with say a person who is constantly searching like a librarian of old times – having it in your pocket, and then you’re sitting around at the party. And somebody says, “Let’s get down to brass tacks.” And then somebody says, “Hey what does that even mean?” And you go clickity clack, it means this. Like it turns us all into the person with the infinite Rolodex that – the infinite– Whatever effect might have occurred within a librarian, is now definitely occurring in everyone and into a big – to a magnified degree. So there’s – that is – to me that’s just a super astonishing thought. And the fact that– Do you think that this is – so I guess this leads to the big question, which is – is this good, bad, neither – just different? I mean, what are the implications for–? I mean is – for a– For we human beings, now with this new ability and this new set of behaviors, and this new illusion of knowledge. What are the implications?
Matthew: Yeah so, I think one interpretation that I very much disagree with is that – hey this proves the internet is evil, let’s all run for the hills. And I don’t think that at all. So first of all, thee internet’s just obviously a wonderful resource and has revolutionized all sorts of domains of human endeavours, and has helped us out in countless ways. And is continuing to do so. And so the way I’ve been thinking about these results is – more as a trade off of the strategy we’re using to store and access information. So it’s more like an unintended consequence that may be important to be aware of. But not any reason to raise the alarm bells or start destroying any internet servers or anything like that.
David: It reminds me of – my dad used to do – he was an electrical contractor, and I used to do – work on construction jobs with him. And there was – everyone always was very aware in that world that when you start to project into the future how long a project will take, you always grossly underestimate how much time it’s going to take to do something. And so they actually will – knowing that’s true, they would put into their quotes and their blueprints and their plans – an extra amount of money that a project would need. An extra amount of time it would need – just to make up for the fact that they knew that human beings were bad at that. And they just intuitively understood it. And I think that’s really cool. They just sort of banked on the fact that they were going to miss– They were going to underestimate. And in the paper you sort of sum up talking about one thing that might happen here. Is people might unwittingly exaggerate how much work they can do from one situation to the next – intellectually. Or maybe how much – how much work it will take, or how much research it will take to come to a certain level. To be competent in something, or a project or research paper or a book or whatever. And it really reminds me of the other thing. And it seems like – since this is just – since we just got this, I mean relatively speaking we aren’t really yet doing that thing that contractors do. Where they factor in the extra time, because they know that they’re underestimating. And it seems to me that maybe there’s a space here, and your work is going to help. At a certain point, we will acknowledge that we’re overestimating our knowledge. And we will maybe factor that into things. Am I sort of in the same – in the right world here?
Matthew: Yeah so, one way to think about it is that – if I think I know how to answer these questions about the weather, but I actually don’t – for much of our everyday life, that may not matter. Because of how pervasive the internet has become. So let’s say I go around thinking I know how tornadoes form, when I actually don’t. And at some point, I actually need to produce that information for some purpose. Then if I realize, “Hey, what was that answer again?” Maybe I don’t actually know it. I can just pull it up in seconds, and I know the answer. So I think for much of the work that we would need to do, this may not be a serious problem. Just because the internet is available to us, and we can rely on it. But I think it does become problematic in some instances. So I think there are domains of thought or domains of knowledge. Where it actually is good to know how much do I, myself know about this topic independent of my reliance on other sources of information?
David: Right. Like if you’re the President or you’re in a situation where your decisions really impact a lot of people. It would be good to know exactly how much you don’t know.
Matthew: Yeah. That is true. And I think – although even in really important situations, if you think about – this is less important than a President. But you can think about a surgeon performing an important surgery. It’s often in those cases that we – we – in a sense, guarantee that the resources that that person needs are around him and accessible. So he has other doctors that know certain things. He has an iPad there that can look up what to do in certain situations. And so when the stakes are raised, I think we’re aware of that. And so we put in the environment, the tools that people need to succeed. But independent of those sorts of situations – one area where I have been thinking this could be a problem – is in the political domain, for example. So if I’m going to vote on a particular issue, or for a particular candidate – it seems like it’s important in that situation for me to know how much– How much do I understand about the issues at hand, independent of my – of the internet or other sources? So I think – this work is suggesting – that in a situation like that, people may often feel like they’re experts in a particular topic. When in fact they just can access information that helps them think about it in a deeper way.
David: That’s really interesting, that’s really interesting. I’ve been spending a lot of time with this new book, talking to political scientists and interacting with potential voters on a bunch of issues. And they – one thing that keeps coming up, especially among the political scientists. Is trying to really hammer home that most people – even when they believe very strongly about a topic, or they are– They have a strong emotional response to a certain issue. They don’t usually have a lot of knowledge to back up that feeling. And it’s something that political scientists are always talking about that. A big part of encouraging people to vote, or changing attitudes or anything along those lines – is just letting– Just giving people a chance to realize they don’t actually know what they’re talking about.
Matthew: Yeah, I think that’s true. And I think the internet is only going to make that harder. Because even though they may not know how to articulate their position – but when forced to, they certainly will be able to Google it and find someone who can. So to fill in their knowledge.
David: So that is – I was going– This is going to be my last– This was going to be my last question. It still will be. But you segued perfectly into it for me. Which is – I can’t – I could not stop– Everywhere throughout your paper, I was thinking – especially when– Especially the part where you talk about how people see the internet as sort of a– The most ultimate sage. This – the internet is our digital Gandalf. So where does confirmation bias fit into all this? Because when you have search, you can confirm everything. So how does that fit into what you’re researching?
Matthew: Yeah. That’s a great question. And I think you’re right, that– Confirmation bias is one of the fundamental features of human reasoning. And the internet gives us access to materials that can support all sorts of points of view. And so, I think the ease with which we’re able to find things that confirm what we believe – is just rising exponentially. And so, I think that is an important area to be thinking about. And I think it would be a great direction for future research.
David: That’s a good answer, there you go, there you go. I don’t know, but we will look it up. We’ll figure it out. I love this, this is such cool work. And I really wanted to have you on the show because I – I love reading about this kind of stuff, but I kept seeing a lot of– To me, it seemed like there was a lot of weird interpretations. And I really enjoyed being able to go through it with you. I know people are going to want to learn more about what you’re up to, and they’re going to maybe want to try to follow you. How can they do that?
Matthew: Yeah, so I have a website through Yale. You can just Google it, that’s kind of the easiest way to find it. Matthew Fisher at Yale University. And I have several papers related to similar topics posted up there. And you can email me at email@example.com, with any questions or ideas. I love hearing feedback about the work that I’m doing.
David: Trying to make us less dumb, yes.
Matthew: That’s right.
David: Thank you so much. You’re such a great guest, and you so eloquently spoke about all these topics. Thank you so much for being on the show.
Matthew: Yeah, it was wonderful. Thanks David.
You must be logged in to post a comment.