I like this post a lot! It covers very different stuff than I would have if I were writing it.
Well, since I'm commenting, I want to expand on some of the things you said at the end with some tips of my own. :P
1. Don't just go drawing conclusions about what your interlocutor believes by taking the "logical" consequences of what they've told you! Your idea of the logical consequences probably in fact depends on hidden assumptions you're making -- and they don't agree with those. This is the infamous "A/B problem". I mean, OK -- it's important to see what the conclusions are and explore whether they agree with them, but like. Don't go attributing your own conclusions to your interlocutor, right? Don't put words in people's mouths. Ask them about it, don't assert that they believe it. (I guess that's a more general principle.)
2. Speaking of which -- yeah, people who disagree with you *actually disagree* with you, meaning they disagree with you on a lot of background assumptions; they don't secretly agree with you but for one small thing. (The "believing atheists are Satanists" problem.) But people aren't good at making such background assumptions explicit, so they won't helpfullly *state* all their disagreements with you; you'll have to tease it out. (Why don't they state them? For the same reason you didn't think to do so, they didn't realize it was a controversial point that someone might disagree with!) This often involves noticing when someone might be using a concept that compresses multiple things together, and taking the time to peel apart the conflation or equivocation.
3. Remember to keep in mind, what disagreements are important? It's usually not productive to argue over something that you don't agree with but which doesn't affect the actual point at hand.
4. Don't accidentally accuse people of bad faith! I see a lot of people on the internet throwing around the word "disingenuous" a lot. I'm not sure they all mean ill by this, but, like... if you call someone "disingenuous", you are accusing them of arguing in bad faith, and they may react appropriately by, y'know, not wanting to argue with you anymore. Seriously, go check a dictionary -- maybe that's not what you thought the word meant, but that is the dictionary definition and it is how a lot of people will reasonably take it. You should likely scrub this word from your productive vocabulary. I see a bunch of people who seem otherwise polite turn things sour by calling someone "disingenuous", I'm not kidding. If there is some other useful concept you were using it to try to express, find a different way to express it.
...really I've said more about all of this in this series of LW posts:
It seems fair to believe people when they say what beliefs they hold, but I’d suggest you can get into trouble when you assume that people hold beliefs that are the logical consequences of beliefs that they hold, and even more trouble when you assume that they hold beliefs that you think are the logical consequences of beliefs that they hold. Many people hold conflicting beliefs, and many, many people hold beliefs with conflicting implications. I hold a bunch of conflicting beliefs myself and I am a reasonably smart guy who likes to think and talk about abstract issues for fun.
I would phrase it more in terms of, what you think are the logical consequences, aren't what they think are the logical consequences, because you're making different background assumptions than they are.
I once read someone on the internet who claimed that the reason they ate chicken (in spite of all the reasons to be vegetarian) was because they hate chickens.
Regarding steelmaning I think the central problem with the dynamic you have identified is that people get confused about whether any actual person holds the view that they are articulating. However, the whole point of the exercise is to come up with the most convincing version of the argument and convincingness of an argument lies in the eye of the beholder. Indeed, the whole reason why we don’t just call it strongmaning is because you’re not really concerned with whether any actual person holds that view. It’s more about finding out what is true because the fact nobody has made an argument or believes it sincerely is no reason to pretend that it might not be valid, especially if it would be convincing to you.
Basically, almost every source I have encountered that defends the practice of steelmaning argue that it’s because you are concerned with something is actually true or false. Not finding out whether a particular person is thinking correctly or not. So of course you come up with those arguments, you would find most convincing for each relevant position.
Indeed, I’m not really sure what strongest possible argument can possibly mean if it doesn’t mean most convincing to me. There isn’t some objective sense of strength of an argument, and even if there was, it doesn’t particularly seem relevant to me if it is not convincingness to me. If there is some amazing argument out there, which is objectively very strong, but I think it’s ridiculous pointing out that some other argument I find more persuasive is objectively weaker seems like a total waste of time.
Sure an actual communist doesn’t believe a lot of stuff I do, but no actual communist is going to actually persuade me of anything if their argument is predicted on precisely those ideas where we disagree so it would be a waste of time to focus on those arguments that I am certain to not find persuasive. Like the chance that an entirely new argument that looks a little communist, which I just invented will change. My mind seems honestly greater than the chance that an actual communist will succeed because there are so many things that that communist believes which I think is obvious nonsense, they have to persuade me on hundreds of things before they could actually convince me of their view.
So I agree that you should not go around, assuming that most of your opponents actually believe what you consider the strongest arguments for their position, but that doesn’t mean it isn’t a valuable exercise to come up with those arguments. You should just make sure you don’t confuse those arguments, you have come up with with the arguments that those people themselves would consider the strongest or even believe in. People very different from you or who have very different factual or moral opinions will just find different arguments to be the most convincing and indeed will consider lots of arguments. You consider very strong to be ridiculous because they consider lots of the relevant premises to be obvious nonsense. This doesn’t contradict what you said about humans sharing the same moral instincts and stuff. It’s just that the process of reflection and peoples very different experiences can lead to very different opinions, even if it’s actually surprising how much those people have in common or how easily you could imagine them coming up with very different factual and moral opinions with surprisingly Small changes in their past experiences.
To be clear, I agree with basically the rest of your article. This is the only point where I had a disagreement and I actually think this was a completely amazing post so this is very much a small disagreement that just got focused on because it was the only point of potential disagreement.
I think it's a good idea to consider counterarguments for claims you believe, but at some point this process stops being related to other people's claims at all. If your argument in favor of Communism isn't believed by any actual Communists, why claim it involves Communism at all? Why not say "this is my argument for vastly expanding employee stock ownership" or whatever concrete claim you're arguing for? It seems like at some point the people you disagree with serve the same role as a writing prompt blog.
I like making up arguments for claims I don't believe but I don't feel myself bound to make arguments for claims anyone else believes either. I came up with a great argument the other day that spending time with people who are in love with you is bad for your epistemic rationality because they believe false things about your positive qualities.
Okay, after reading this response, I don’t actually think we disagree. My own idea is that it’s often worthwhile to put in the effort to come up with the best possible arguments for positions where you think they are worth investigating and the fact that that other people believe something is evidence that some position is worth investigating, but that’s obviously just one form of evidence and indeed, there are lots of positions that I have considered worth thinking about where I’m pretty sure almost nobody actually sincerely believes in the position.
Do we clear I do think it’s a danger sign. If literally no one has ever believed your position, but in practice that’s pretty uncommon and in any case, it’s definitely not proof not to mention that for sufficiently unusual opinions, you might simply not have come across people arguing for the position, even though there are such people in existence and so the fact you are not heard of anybody arguing for it is weaker evidence and you might think. Of course, it is, in fact that other peoples thinking can provide useful material for your own arguments. My ability to steal man arguments for why we should have no government is substantially greater than my ability to steal man arguments for why children should have the right to vote in part because I have read a lot more people arguing for the Former position. So even though my arguments for the position are not the same as the arguments at any actual believer would give, I have much more intellectual ammunition to start with. Still as a factual matter, I think it’s obvious that if you are genuinely trying to find the truth about something and your views are sufficiently unusual, you should expect to in fact, end up using other peoples views as basically writing prompts some of the time or even come up with arguments for positions that nobody you have heard of has ever defended even when you don’t believe those positions because having unusual opinions means the arguments that might convince you will often rely on premises that other people reject.
So yes, I agree that any person who does a lot of steelmaning should often end up completely disconnected from the positions of other people to the point where it’s not useful to actually focus too much on any resemblance to their views. Of course, sometimes you will end up with a position that is very similar to other peoples position, even though your underlying arguments are completely different, but I don’t think you actually reject that possibility.
I dunno, my own experience suggests that most fourteen year olds are probably about as competent and free to choose who to vote for as most eighty year olds... 🤷♂️
That’s kind of my point as a matter of simple probability. I certainly find this position a lot more likely to be correct when compare to the idea that there should be no government, but I can provide much less argumentation to back it up because I don’t have quite as much intellectual ammunition on account of loads of people not having provided arguments for it in the past. Even when your arguments are novel, the amount of intellectual output on the subject that you have previously consumed will affect how much argumentation you are able to come up with, and this is a prime example, because if anything I think most people can agree that teenagers being able to vote is obviously less crazy, then wanting to abolish the government. Even when your own arguments are pretty different, other people having held the position in the past, makes it easier to argue for that position or clarify you thinking as some of the low hanging fruit has already been picked for you so you aren’t starting from ground zero.
>People’s beliefs often come from their moral intuitions. You almost certainly have the same basic moral instincts as any other human, although you might reject some of them upon reflection.
This is a great post and super helpful! I may share it with my students sometime if you are OK with that? It covers a lot of ground I aspire to convey in fostering self-reflection around beliefs, engaging with others, and classroom discussions.
I also really like that you point out that we all have some ideas/beliefs that are so important to us it limits engaging with some topics that are so outside that realm, usually relating to morality. Spot on.
Some people do in fact just joke about hating men, though. And at least in my corner of Europe, there's a type of guy who's fairly common: "I hate immigrants, they're ruining our country, and I vote as such. Except my neighbor Muhammed, he's so kind. My barber Abdul is also very cool. I love Ali, he's really keeping our local corner store alive. No I cannot explain this to you".
Educated people’s belief networks are broad and deep. As long as you have some control over what you spend your time arguing about (“a media rarely lies”), or are willing to imply things that you can’t defend without actually saying them (“motte and bailey”), it’s not that hard to signal anything you want without saying anything you don’t actually believe.
I like this post a lot! It covers very different stuff than I would have if I were writing it.
Well, since I'm commenting, I want to expand on some of the things you said at the end with some tips of my own. :P
1. Don't just go drawing conclusions about what your interlocutor believes by taking the "logical" consequences of what they've told you! Your idea of the logical consequences probably in fact depends on hidden assumptions you're making -- and they don't agree with those. This is the infamous "A/B problem". I mean, OK -- it's important to see what the conclusions are and explore whether they agree with them, but like. Don't go attributing your own conclusions to your interlocutor, right? Don't put words in people's mouths. Ask them about it, don't assert that they believe it. (I guess that's a more general principle.)
2. Speaking of which -- yeah, people who disagree with you *actually disagree* with you, meaning they disagree with you on a lot of background assumptions; they don't secretly agree with you but for one small thing. (The "believing atheists are Satanists" problem.) But people aren't good at making such background assumptions explicit, so they won't helpfullly *state* all their disagreements with you; you'll have to tease it out. (Why don't they state them? For the same reason you didn't think to do so, they didn't realize it was a controversial point that someone might disagree with!) This often involves noticing when someone might be using a concept that compresses multiple things together, and taking the time to peel apart the conflation or equivocation.
3. Remember to keep in mind, what disagreements are important? It's usually not productive to argue over something that you don't agree with but which doesn't affect the actual point at hand.
4. Don't accidentally accuse people of bad faith! I see a lot of people on the internet throwing around the word "disingenuous" a lot. I'm not sure they all mean ill by this, but, like... if you call someone "disingenuous", you are accusing them of arguing in bad faith, and they may react appropriately by, y'know, not wanting to argue with you anymore. Seriously, go check a dictionary -- maybe that's not what you thought the word meant, but that is the dictionary definition and it is how a lot of people will reasonably take it. You should likely scrub this word from your productive vocabulary. I see a bunch of people who seem otherwise polite turn things sour by calling someone "disingenuous", I'm not kidding. If there is some other useful concept you were using it to try to express, find a different way to express it.
...really I've said more about all of this in this series of LW posts:
https://www.lesswrong.com/posts/GrZJCseCviYpXCT5r/doing-principle-of-charity-better
https://www.lesswrong.com/posts/pvsdduXMfo3AeepFF/x-as-phenomenon-vs-as-policy-goodhart-and-the-ab-problem
https://www.lesswrong.com/posts/LC7jaqukdEu5zTAmg/more-on-policy-arguments-and-the-ab-problem
...but this is what I wanted to add quickly. :P
It seems fair to believe people when they say what beliefs they hold, but I’d suggest you can get into trouble when you assume that people hold beliefs that are the logical consequences of beliefs that they hold, and even more trouble when you assume that they hold beliefs that you think are the logical consequences of beliefs that they hold. Many people hold conflicting beliefs, and many, many people hold beliefs with conflicting implications. I hold a bunch of conflicting beliefs myself and I am a reasonably smart guy who likes to think and talk about abstract issues for fun.
Yes, this is the famous A/B problem (as I discussed here: https://www.lesswrong.com/posts/pvsdduXMfo3AeepFF/x-as-phenomenon-vs-as-policy-goodhart-and-the-ab-problem) (does anyone know who came up with that name? I still haven't found the origin)
I would phrase it more in terms of, what you think are the logical consequences, aren't what they think are the logical consequences, because you're making different background assumptions than they are.
I once read someone on the internet who claimed that the reason they ate chicken (in spite of all the reasons to be vegetarian) was because they hate chickens.
My friend is a vegetarian in a rural area where this is relatively uncommon. His quip is always that he's a vegetarian because he hates vegetables.
> The first step to charitable disagreement is humility. (Humility is also the third, fourth, sixth, and eighth steps.)
Many of the rules of humility club are do not brag about humility club.
The first rule of tautology club is the first rule of tautology club.
The first rule of procrastination club will be decided when I get around to it.
Regarding steelmaning I think the central problem with the dynamic you have identified is that people get confused about whether any actual person holds the view that they are articulating. However, the whole point of the exercise is to come up with the most convincing version of the argument and convincingness of an argument lies in the eye of the beholder. Indeed, the whole reason why we don’t just call it strongmaning is because you’re not really concerned with whether any actual person holds that view. It’s more about finding out what is true because the fact nobody has made an argument or believes it sincerely is no reason to pretend that it might not be valid, especially if it would be convincing to you.
Basically, almost every source I have encountered that defends the practice of steelmaning argue that it’s because you are concerned with something is actually true or false. Not finding out whether a particular person is thinking correctly or not. So of course you come up with those arguments, you would find most convincing for each relevant position.
Indeed, I’m not really sure what strongest possible argument can possibly mean if it doesn’t mean most convincing to me. There isn’t some objective sense of strength of an argument, and even if there was, it doesn’t particularly seem relevant to me if it is not convincingness to me. If there is some amazing argument out there, which is objectively very strong, but I think it’s ridiculous pointing out that some other argument I find more persuasive is objectively weaker seems like a total waste of time.
Sure an actual communist doesn’t believe a lot of stuff I do, but no actual communist is going to actually persuade me of anything if their argument is predicted on precisely those ideas where we disagree so it would be a waste of time to focus on those arguments that I am certain to not find persuasive. Like the chance that an entirely new argument that looks a little communist, which I just invented will change. My mind seems honestly greater than the chance that an actual communist will succeed because there are so many things that that communist believes which I think is obvious nonsense, they have to persuade me on hundreds of things before they could actually convince me of their view.
So I agree that you should not go around, assuming that most of your opponents actually believe what you consider the strongest arguments for their position, but that doesn’t mean it isn’t a valuable exercise to come up with those arguments. You should just make sure you don’t confuse those arguments, you have come up with with the arguments that those people themselves would consider the strongest or even believe in. People very different from you or who have very different factual or moral opinions will just find different arguments to be the most convincing and indeed will consider lots of arguments. You consider very strong to be ridiculous because they consider lots of the relevant premises to be obvious nonsense. This doesn’t contradict what you said about humans sharing the same moral instincts and stuff. It’s just that the process of reflection and peoples very different experiences can lead to very different opinions, even if it’s actually surprising how much those people have in common or how easily you could imagine them coming up with very different factual and moral opinions with surprisingly Small changes in their past experiences.
To be clear, I agree with basically the rest of your article. This is the only point where I had a disagreement and I actually think this was a completely amazing post so this is very much a small disagreement that just got focused on because it was the only point of potential disagreement.
I think it's a good idea to consider counterarguments for claims you believe, but at some point this process stops being related to other people's claims at all. If your argument in favor of Communism isn't believed by any actual Communists, why claim it involves Communism at all? Why not say "this is my argument for vastly expanding employee stock ownership" or whatever concrete claim you're arguing for? It seems like at some point the people you disagree with serve the same role as a writing prompt blog.
I like making up arguments for claims I don't believe but I don't feel myself bound to make arguments for claims anyone else believes either. I came up with a great argument the other day that spending time with people who are in love with you is bad for your epistemic rationality because they believe false things about your positive qualities.
Okay, after reading this response, I don’t actually think we disagree. My own idea is that it’s often worthwhile to put in the effort to come up with the best possible arguments for positions where you think they are worth investigating and the fact that that other people believe something is evidence that some position is worth investigating, but that’s obviously just one form of evidence and indeed, there are lots of positions that I have considered worth thinking about where I’m pretty sure almost nobody actually sincerely believes in the position.
Do we clear I do think it’s a danger sign. If literally no one has ever believed your position, but in practice that’s pretty uncommon and in any case, it’s definitely not proof not to mention that for sufficiently unusual opinions, you might simply not have come across people arguing for the position, even though there are such people in existence and so the fact you are not heard of anybody arguing for it is weaker evidence and you might think. Of course, it is, in fact that other peoples thinking can provide useful material for your own arguments. My ability to steal man arguments for why we should have no government is substantially greater than my ability to steal man arguments for why children should have the right to vote in part because I have read a lot more people arguing for the Former position. So even though my arguments for the position are not the same as the arguments at any actual believer would give, I have much more intellectual ammunition to start with. Still as a factual matter, I think it’s obvious that if you are genuinely trying to find the truth about something and your views are sufficiently unusual, you should expect to in fact, end up using other peoples views as basically writing prompts some of the time or even come up with arguments for positions that nobody you have heard of has ever defended even when you don’t believe those positions because having unusual opinions means the arguments that might convince you will often rely on premises that other people reject.
So yes, I agree that any person who does a lot of steelmaning should often end up completely disconnected from the positions of other people to the point where it’s not useful to actually focus too much on any resemblance to their views. Of course, sometimes you will end up with a position that is very similar to other peoples position, even though your underlying arguments are completely different, but I don’t think you actually reject that possibility.
I dunno, my own experience suggests that most fourteen year olds are probably about as competent and free to choose who to vote for as most eighty year olds... 🤷♂️
That’s kind of my point as a matter of simple probability. I certainly find this position a lot more likely to be correct when compare to the idea that there should be no government, but I can provide much less argumentation to back it up because I don’t have quite as much intellectual ammunition on account of loads of people not having provided arguments for it in the past. Even when your arguments are novel, the amount of intellectual output on the subject that you have previously consumed will affect how much argumentation you are able to come up with, and this is a prime example, because if anything I think most people can agree that teenagers being able to vote is obviously less crazy, then wanting to abolish the government. Even when your own arguments are pretty different, other people having held the position in the past, makes it easier to argue for that position or clarify you thinking as some of the low hanging fruit has already been picked for you so you aren’t starting from ground zero.
>People’s beliefs often come from their moral intuitions. You almost certainly have the same basic moral instincts as any other human, although you might reject some of them upon reflection.
Counterpoint: https://slatestarcodex.com/2014/03/17/what-universal-human-experiences-are-you-missing-without-realizing-it/
This is a great post and super helpful! I may share it with my students sometime if you are OK with that? It covers a lot of ground I aspire to convey in fostering self-reflection around beliefs, engaging with others, and classroom discussions.
I also really like that you point out that we all have some ideas/beliefs that are so important to us it limits engaging with some topics that are so outside that realm, usually relating to morality. Spot on.
Some people do in fact just joke about hating men, though. And at least in my corner of Europe, there's a type of guy who's fairly common: "I hate immigrants, they're ruining our country, and I vote as such. Except my neighbor Muhammed, he's so kind. My barber Abdul is also very cool. I love Ali, he's really keeping our local corner store alive. No I cannot explain this to you".
> you should basically take people at their word when they say they believe stuff
Unless they're signalling vice. Or, potentially, virtue, for that matter.
Educated people’s belief networks are broad and deep. As long as you have some control over what you spend your time arguing about (“a media rarely lies”), or are willing to imply things that you can’t defend without actually saying them (“motte and bailey”), it’s not that hard to signal anything you want without saying anything you don’t actually believe.