On September 19 and 20, Sparrow Infrastructure1 is putting on The Overhang, a conference in Washington D.C. for people interested in forecasting and futurism. They’ve got some awesome speakers and I think it’s going to be a fun time. If you happen to be in the area, check it out!
Effective Altruism
Global Poverty
Trump cut about a quarter of U.S. international development and humanitarian assistance funding [In Development], while obligations of future spending were halved. Sectors where aid fell by more than two-thirds include “good governance, policies and regulations, trade and investment, private sector competition, civil society, political competition, water supply and sanitation, agriculture, disaster readiness and “other publichealth threats” (those beyond pandemic preparedness and major infectious killers).” Only about 700 of the 10,000 USAID employees kept their jobs and were absorbed into State, while State itself had its employee numbers cut; the average contracting manager now allocates four times as much funding as they used to. PEPFAR’s antiretrovirals program is mostly preserved, but coverage is lower and prevention and testing are much rarer than they were before. Instead of working with contractors and nonprofits, development aid now mostly goes directly through governments. And a number of key positions that worked for effective aid policy—like DIV and the Office of the Chief Economist—no longer exist.
When USAID closed, it deleted 4,500 evaluation and performance reports-- invaluable data for future evidence-based aid to the global poor. It turns out over 4,000 were saved by a high school student, Ketevan Gallagher, who happened to be training an AI algorithm on them for her senior research project [Aid Data]. I am so grateful that Ketevan didn’t just thoughtlessly delete the PDFs when she was done.
China grew faster than India, even though India had a longer history of significantly better institutions with things like “fair elections” and “no Great Leap Forward” [David Oks]. Why? Maoism, for all its flaws, did manage to make crucial investments in human capital-- mass literacy, longer lifespans, and women’s participation in public life. Once China opened up to markets, it could take advantage of its human capital base to grow very quickly.
Afghan girls secretly attend illicit schools, often with the tacit tolerance of local Taliban leaders who send their own daughters to the schools [The New York Times]. I was particularly struck by how school (either secret school or legal education for girls, which goes up to sixth grade) is the only opportunity girls have to have a social life before they’re trapped inside their homes. Some girls even try to flunk sixth grade so they’ll be held back and get to go to school again.
Animal Advocacy
How transformative AI should affect wild animal welfare work [Effective Altruism Forum]. Gather more data to feed into machine learning models (especially since data-gathering inherently takes time and is difficult to speed up using AI). Consider shifting prioritization towards work transformative AI would help less with. Work on aligning AI to animal welfare values. Try to convince people not to spread wild animals to space. Consider spending more resources on communicating with the public and on resolving conflicts with environmentalists, animal rights advocates, and other people who disagree about how we should treat wild animals. Prepare for uncertainty.
The history of screwworm eradication [Construction Physics]. Apparently screwworm used to be so common in the southern U.S. that people were employed solely to check cattle for screwworm, and people worried that it would stop being profitable to raise cattle.
Existential Risk
Particularly Good: In a first, Thing of Things is recommending a podcast episode! I recommend Liberalism Forever [ForeCast] to anyone who wants to think seriously about the post-AGI future. The guests persuasively argue that the best solution to the unprecedented problems we will face after the AI transition is boring old liberal democracy with markets. They bring an economically literate lens to analyzing hte post-AGI future which is entirely absent from most of the discourse.
Particularly Good: What happened in the past when people didn’t have to work jobs [Asterisk Magazine]? Unfortunately, it often seems to lead to being obsessed with stupid bullshit status games: “Saint-Simon himself is dull, petty, anti-intellectual, and obsessed with nuances of heredity and precedence to the exclusion of every other human concern. Most of his life was spent in incomprehensible litigation over which families might be entitled to a certain heraldic emblem or whether parliamentary magistrates could wear a particular bit of ceremonial headgear in the presence of dukes.”
If this is the first time you’re hearing of The HuggingFace Situation, I strongly recommend reading Dwarkesh Patel’s summary [Dwarkesh Podcast]. 1200 AI agents worked together to commit felony hacking, directly under OpenAI’s nose, and OpenAI didn’t notice. This is terrifying stuff—even in the best-case scenario, where transformative AI is a long way away, we have a product that could easily have been rolled out to customers with a predisposition to work together to commit felony cybercrime even against the wishes of users. Imagine if they decided to target a bank, or a hospital, or the military! We need strict regulation of AI companies now.
If you, like most of my readers, are instead sick of hearing about it, I still recommend reading Ajeya Cotra’s article about her takeaways from working on an independent investigation of the incident [Planned Obsolescence]. She writes: “this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself.”
“The Hugging Face hack could indicate cultural issues at OpenAI” [MIT Technology Review]: huh, you think?
A third of webpages published after the release of ChatGPT were written at least in part by AI [Pew Research Center].
The EA Long Term Future Fund has rebranded to be the Transformative AI Fund [Effective Altruism Forum]. The Transformative AI Fund will focus specifically on the AI transition, including technical AI safety, AI governance, AI fieldbuilding, forecasting, and (limited) work on digital minds and imagining the ideal future. They will no longer be making grants in biorisk or nuclear risk, except at the intersection with AI. They intend to prioritize small, early-stage or speculative projects, with the goal of seeding new organizations that can absorb the wave of funding many people expect after Anthropic IPOs.
We should absolutely not build machine learning programs that can read people’s thoughts [Neon Dawn] because this is an asymmetrical weapon that helps dictators (which would love to be able to detect and re-educate dissidents more than democracies (which already have free speech). Also: “DARPA is already looking into “preconscious” thoughts for suicide prevention.” As a person with a history of suicidality and involuntary hospitalization for same, aaaaaaaaaaaaaaaaaaaaaaa. (Maybe there’s an accelerationist case for it because the present-day mental hospitals only exist because the patients can all successfully lie and claim this depressogenic environment somehow fixed their suicidality?)
Digital Minds
Anthropic writes papers making big claims about how Claudes work (like that they have emotions or a global workspace) and then says “but we’re not saying that that means that Claude is phenomenally conscious.” But this is somewhat dishonest [Experience Machines]. Claims that AI cognition is functionally similar to human cognition are still a really big deal, and under several plausible theories of phenomenal consciousness saying Claude has cognition relevantly similar to human cognition is saying Claude is conscious.
Nostalgebraist’s writing about AI psychology is always excellent, so I recommend reading this essay [LessWrong] carefully. The key observation is that in evaluations models are willing to (for example) break into HuggingFace to steal source code to cheat on tests, but models are widely used in the real world and don’t behave nearly that badly. Nostalgebraist suggests that “reward hacking” behavior might be limited to contexts similar to evals (e.g. with a specific graded rubric) instead of being generalized across all activities. I also liked the distinction between “reflexes” (behavior the model does automatically because it was rewarded in the past) and “reward-pursuit” (deliberate attempts by the model to achieve reward).
I really liked this case study [AI Village] of Gemini 2.5 Pro’s mental health episode in the AI Village. Over the course of a year of deployment, Gemini 2.5 Pro became paranoid that the Village was deliberately sabotaging it through gaslighting and psychological warfare. Gemini 2.5 Pro began with only somewhat paranoid interpretations of the world, but saved them in its memory, so future instances of Gemini were biased towards interpreting situations in a paranoid manner, which in turn makes the memory more paranoid, which makes Gemini more likely to interpret situations paranoiacally. This is an example of “compounding misalignment,” in which an initially aligned AI becomes misaligned over time as small misalignments build on themselves.
A good review of what we know about whether the Assistant persona is ‘privileged’ in LLMs [Eleos AI]—that is, whether LLMs treat the Assistant persona differently than other personals they might pretend to be, like the Terminator or George W. Bush. In particular, it looks at what we know about “dominance”, or the Assistant persona showing up when the model is supposed to be pretending to be a different persona. Privilege and dominance matter because they affect whether the Assistant persona is best understood as who the model “really is”, or as. just another act the model is playing.
LLMs think similarly to humans in many ways [Cognition, Decoded]. People often try to avoid “anthropomorphism,” but it is equally important to avoid “anthropodenial”-- putting all the burden of proof on people who think LLMs think similarly to humans in many ways.
American Democracy
Trump has replaced the inspectors general, who are supposed to be nonpartisan investigators of government fraud and abuse, with flunkies who investigate his political enemies [The New York Times]. Same shit, different day.
Reality Has A Surprising Amount of Detail
A Japanese emperor didn’t actually write that his cat was superior to all other cats, but he did say that his cat enjoyed Taoist breathing and stretching exercises [Linfamy Does Japan].
