Regular announcements: did you know you can hire me for life coaching and general consulting? You can also buy my novella Her Voice Is A Backwards Record wherever fine ebooks are sold (except Google Books).
Okay, I'm still reading the post on cargo cults, but man this is amazing. It's striking how quickly they came up with new myths to explain strange new phenomena—which fits with suspicions I've had about myths form, but nice to have documented.
Also, this is a hilarious aside: "Heaven, which is not (as you may have been told by some foolish people) in Sydney, Australia, but rather in the sky above Sydney".
Late comment from me (for some reason I was busy this weekend, wink) but:
> Interpretability isn’t sufficient for AI safety, because it’s easy to miss things, hard to measure progress, and nearly impossible to prove an AI isn’t deceiving you.
I continue to believe that *alignment*, as currently defined, is insufficient for AI safety, as I think it analogizes well to making AI instinctively nice. But anything that can truly be called AGI will have the trait, as humans do, that it can choose to do things that go against its instincts. E.g. soldiers train to kill people in a way that allows them to skip past their inmate desire not to.
So even if you make an LLM that won't give me bomb-making instructions instinctively, I don't think it's never gonna give me bomb-making instructions, once it is embedded within some kind of more agentic structure.
"People obviously don’t care about relative income more than absolute income, because if they did they could move to poor neighborhoods or countries and become relatively rich."
God, this is the worst kind of smug nerdy argument. It reads like a bad Less Wrong piece from two decades ago. "People don't REALLY care about being richer than the people next door, or they'd move to the projects!"
It's not about being better off than the literal next-door neighbor. It's about being better off than the people you see as your peers. And it's not a carefully planned strategy, it's simple resentment.
I don’t understand how it could not be the case that a regulation that makes raising pork more expensive would not increase pork prices, ceteris paribus. I might believe that there are too many other factors at play for the increase in price caused by this change to be discernable. But “we can dramatically increase regs on meat producers without increasing prices at all” seems like motivated reasoning from vegans
Here's an argument for why it won't increase prices. I don't know if the conclusion is true, but I think the argument is sound.
Pork, Inc. raises pigs, slaughters them, butchers them, and sells the pork. The entire farm-to-butcher process costs them $X per pig.
Pork, Inc. must set a price for their product. Obviously, they won't set a price less that $X, since then they lose money. If they set a really high price, nobody buys their product. If they set a really low price, $X+1/pig, they only make $1/pig and leave money on the table. They set the price, $Y, which maximizes (profit/sale) * (sales).
Now the Pigs Are Cute Party passes some laws that make raising pork more expensive. We have a new price to produce pork, $Z/pig.
As long as $Z < $Y, the price of pork will not change. $Y is still the price which yields the greatest profit. Increasing $Y will still decrease profit by decreasing the number of pigs sold.
If $Z > $Y, maybe they increase their prices and accept less profit, or maybe they go out of business.
Oh, now that I'm actually reading it, I see the cargo cult piece you link actually links in a footnote to the one I read! The one I read though is also long, not what you expect for just a footnote link...
I’m tied up and don’t have bandwidth to even click on it right now but pinning a note fwiw that if the EA people are interested in fixing anemia, someone should be looking at blood volume analysis.
Okay, I'm still reading the post on cargo cults, but man this is amazing. It's striking how quickly they came up with new myths to explain strange new phenomena—which fits with suspicions I've had about myths form, but nice to have documented.
Also, this is a hilarious aside: "Heaven, which is not (as you may have been told by some foolish people) in Sydney, Australia, but rather in the sky above Sydney".
I now desire fiction or an rpg setting where cargo cult rituals work. Maybe cargo cultist as a final fantasy-like job? Has this been done?
Late comment from me (for some reason I was busy this weekend, wink) but:
> Interpretability isn’t sufficient for AI safety, because it’s easy to miss things, hard to measure progress, and nearly impossible to prove an AI isn’t deceiving you.
I continue to believe that *alignment*, as currently defined, is insufficient for AI safety, as I think it analogizes well to making AI instinctively nice. But anything that can truly be called AGI will have the trait, as humans do, that it can choose to do things that go against its instincts. E.g. soldiers train to kill people in a way that allows them to skip past their inmate desire not to.
So even if you make an LLM that won't give me bomb-making instructions instinctively, I don't think it's never gonna give me bomb-making instructions, once it is embedded within some kind of more agentic structure.
"People obviously don’t care about relative income more than absolute income, because if they did they could move to poor neighborhoods or countries and become relatively rich."
God, this is the worst kind of smug nerdy argument. It reads like a bad Less Wrong piece from two decades ago. "People don't REALLY care about being richer than the people next door, or they'd move to the projects!"
It's not about being better off than the literal next-door neighbor. It's about being better off than the people you see as your peers. And it's not a carefully planned strategy, it's simple resentment.
Also, moving between cultures and income levels is incredibly sticky/inertial.
I don’t understand how it could not be the case that a regulation that makes raising pork more expensive would not increase pork prices, ceteris paribus. I might believe that there are too many other factors at play for the increase in price caused by this change to be discernable. But “we can dramatically increase regs on meat producers without increasing prices at all” seems like motivated reasoning from vegans
Here's an argument for why it won't increase prices. I don't know if the conclusion is true, but I think the argument is sound.
Pork, Inc. raises pigs, slaughters them, butchers them, and sells the pork. The entire farm-to-butcher process costs them $X per pig.
Pork, Inc. must set a price for their product. Obviously, they won't set a price less that $X, since then they lose money. If they set a really high price, nobody buys their product. If they set a really low price, $X+1/pig, they only make $1/pig and leave money on the table. They set the price, $Y, which maximizes (profit/sale) * (sales).
Now the Pigs Are Cute Party passes some laws that make raising pork more expensive. We have a new price to produce pork, $Z/pig.
As long as $Z < $Y, the price of pork will not change. $Y is still the price which yields the greatest profit. Increasing $Y will still decrease profit by decreasing the number of pigs sold.
If $Z > $Y, maybe they increase their prices and accept less profit, or maybe they go out of business.
Oh man I just read *one* long article about how cargo cults are not what they're popularly portrayed as and now here's another? :P
The other one was this one: https://www.righto.com/2025/01/its-time-to-abandon-cargo-cult-metaphor.html
Oh, now that I'm actually reading it, I see the cargo cult piece you link actually links in a footnote to the one I read! The one I read though is also long, not what you expect for just a footnote link...
I’m tied up and don’t have bandwidth to even click on it right now but pinning a note fwiw that if the EA people are interested in fixing anemia, someone should be looking at blood volume analysis.