Quick takes
For the record, I see the new field of "economics of transformative AI" as overrated.
Economics has some useful frames, but it also tilts people towards being too "normy" on the impacts of AI and it doesn't have a very good track record on advanced AI so far.
I'd much rather see multidisciplinary programs/conferences/research projects, including economics as just one of the perspectives represented, then economics of transformative AI qua economics of transformative AI.
(I'd be more enthusiastic about building economics of transformative AI as a field if we w...
On AI alarmists:
A fair-sized stream seems vast to one who until then
Has never seen a greater; so with trees, with men.
In every field each man regards as vast in size
The greatest objects that have come before his eyes
(Lucretius)
I sometimes say, in a provocative/hyperbolic sense, that the concept of "neglectedness" has been a disaster for EA. I do think the concept is significantly over-used (ironically, it's not neglected!), and people should just look directly at the importance and tractability of a cause at current margins.
Maybe neglectedness useful as a heuristic for scanning thousands of potential cause areas. But ultimately, it's just a heuristic for tractability: how many resources are going towards something is evidence about whether additional resources are likely to be i...
That's interesting, but seems to be addressing a somewhat separate claim to mine.
My claim was that that broad heuristics are more often necessary and appropriate when engaged in abstract evaluation of broad cause areas, where you can't directly assess how promising concrete opportunities/interventions are, and less so when you can directly assess concrete interventions.
If I understand your claims correctly they are that:
- Neglectedness is more likely to be misleading when applied to broad cause areas
- When considering individual solutions, it's useful to consi
For the tax nerds, cool event next week from the OECD:
Tax Inspectors Without Borders: A decade of niche assistance to developing countries
12 March 2024 | 13:45 - 14:45 CET
There have been numerous scandals within the EA community about how working for top AGI labs might be harmful. So, when are we going to have this conversation: contributing in any way to the current US admin getting (especially exclusive) access to AGI might be (very) harmful?
[cross-posted from X and LessWrong]
If you've liked my writing in the past, I wanted to share that I've started a Substack: https://peterwildeford.substack.com/
Ever wanted a top forecaster to help you navigate the news? Want to know the latest in AI? I'm doing all that in my Substack -- forecast-driven analysis about AI, national security, innovation, and emerging technology!
Something that I personally would find super valuable is to see you work through a forecasting problem "live" (in text). Take an AI question that you would like to forecast, and then describe how you actually go about making that forecast. The information you seek out, how you analyze it, and especially how you make it quantitative. That would
- make the forecast process more transparent for someone who wanted to apply skepticism to your bottom line
- help me "compare notes", ie work through the same forecasting question that you pose, come to a conclusion, a
I wish more work focused on digital minds really focused on answering the following questions, rather than merely investigating how plausible it is that digital minds similar to current day AI's could be sentient:
-
What does good sets of scenarios for post-AGI governance need to look like to create good/avoid terrible (or whatever normative focus we want) futures, assuming digital minds are the dominant moral patients going into the future 1a) How does this differ dependent on what sorts of things can be digital minds eg whether sentient AIs are likely to
You're sort of right on the first point, and I've definitely counted that work in my views on the area. I generally prefer to refer to it as 'making sure the future goes well for non-humans' - but I've had that misinterpreted as just focused on animals. I
I think for me the fact that the minds will be non-human, and probably digital, matter a lot. Firstly, I think arguments for longtermism probably don't work if the future is mostly just humans. Secondly, the fact that these beings are digital minds, and maybe digital minds very different to us, means a lot...
Anyone else get a pig butchering scam attempt lately via DM on the forun?
I just got the following message
> Happy day to you, I am [X] i saw your profile today and i like it very much,which makes me to write to you to let you know that i am interested in you,therefore i will like you to write me back so that i will tell you further about myself and send you also my picture for you to know me physically.
[EMAIL]
I reported the user on their profile and opened a support request but just FYI
Single examples almost never provides overwhelming evidence. They can provide strong evidence, but not overwhelming.
Imagine someone arguing the following:
1. You make a superficially compelling argument for invading Iraq
2. A similar argument, if you squint, can be used to support invading Vietnam
3. It was wrong to invade Vietnam
4. Therefore, your argument can be ignored, and it provides ~0 evidence for the invasion of Iraq.
In my opinion, 1-4 is not reasonable. I think it's just not a good line of reasoning. Regardless of whether you'...
1-4 is only unreasonable because you've written a strawman version of 4. Here is a version that makes total sense:
1. You make a superficially compelling argument for invading Iraq
2. A similar argument, if you squint, can be used to support invading Vietnam
3. This argument for invading vietnam was wrong because it made mistakes X, Y, and Z
4. Your argument for invading Iraq also makes mistakes X, Y and Z
5. Therefore, your argument is also wrong.
Steps 1-3 are not strictly necessary here, but they add supporting evidence to the claims.
As far as I c...
So long and thanks for all the fish.
I am deactivating my account.[1] My unfortunate best guess is that at this point there is little point and at least a bit of harm caused by me commenting more on the EA Forum. I am sad to leave behind so much that I have helped build and create, and even sadder to see my own actions indirectly contribute to much harm.
I think many people on the forum are great, and at many points in time this forum was one of the best places for thinking and talking and learning about many of the world's most important top...
After nearly 7 years, I intend to soon step down as Executive Director of CEEALAR, founded by me as the EA Hotel in 2018. I will remain a Trustee, but take more of a back seat role. This is in order to focus more of my efforts on slowing down/pausing/stoping AGI/ASI, which for some time now I've thought of as being the most important, neglected and urgent cause.
We are hiring for my replacement. Please apply if you think you'd be good in the role! Or send on to others you'd like to see in the role. I'm hoping that we find someone who is highly passionate ab...
I haven't visited CEELAR and I don't know how impactful it has been, but one thing I've always admired about you via your work on this project is your grit and agency. When you thought it was a good idea back in 2018, you went ahead and bought the place. When you needed funding, you asked and wrote a lot about what was needed. You clearly care a lot about this project, and that really shows. I hope your successor will too.
I'm reminded of Lizka's Invisible Impact post. It's easy to spot flaws in projects that actually materialise but hard/impossible to crit...
Instead of "Goodharting", I like the potential names "Positive Alignment" and "Negative Alignment."
"Positive Alignment" means that the motivated party changes their actions in ways the incentive creator likes. "Negative Alignment" means the opposite.
Whenever there are incentives offered to certain people/agents, there are likely to be cases of both Positive Alignment and Negative Alignment. The net effect will likely be either positive or negative.
"Goodharting" is fairly vague and typically just refers to just the "Negative Alignment" portion.&n...
I think the term "goodharting" is great. All you have to do is look up goodharts law to understand what is talked about: the AI is optimising for the metric you evaluated it on, rather than the thing you actually want it to do.
Your suggestions would rob this term of the specific technical meaning, which makes thing much vaguer and harder to talk about.
I imagine that scientists will soon have the ability to be unusually transparent and provide incredibly low rates of fraud/bias, using AI. (This assumes strong AI progress in the next 5-20 years)
- AI auditors could track everything (starting with some key things) done for an experiment, then flag if there was significant evidence of deception / stats gaming / etc. For example, maybe a scientist has an AI screen-recording their screen whenever it's on, but able to preserve necessary privacy and throw out the irrelevant data.
- AI auditors could review any experi
This context is useful, thanks.
Looking back, I think this part of my first comment was poorly worded:
> I imagine that scientists will soon have the ability to be unusually transparent and provide incredibly low rates of fraud/bias, using AI.
I meant
> I imagine that scientists will [soon have the ability to] be unusually transparent and provide incredibly low rates of fraud/bias], using AI.
So it's not that this will lead to low rates of fraud/bias, but that AI will help enable that for scientists willing to go along with it - but at the same time...
If antinatal advocacy was effective, wouldn't it make sense to pursue on animal welfare grounds? Aren't most new humans extremely net negative?
I have a 3YO so hold fire!
- Most new humans will likely consume hundreds (thousands?) of factory farmed animals over their lifetime, creating a substantial negative impact that might outweigh the positive contributions of that human life
- Probably of far less consequence, the environmental footprint of each new human also indirectly harms wild animals through habitat destruction, pollution, and climate change (TBH I am being very speculative on this point).
As AI improves, there's a window for people to get involved and make changes regarding AI alignment and policy.
The window arguably starts small, then widens as it becomes clearer what to do.
But at some point it gets too close to TAI, I expect that the window narrows. The key decisions get made by a smaller and smaller group of people, and these people have less ability get help from others, given the quickening pace of things.
For example, at T minus 1 month, there might ultimately be a group of 10 people with key decision-making authority on the most power...
Mini EA Forum Update
We've updated the user menu in the site header! 🎉 I'm really excited, since I think it looks way better and is much easier to use.
We've pulled out all the "New ___" items to a submenu, except for "New question" which you can still do from the "New post" page (it's still a tab there, as is linkpost). And you can see your quick takes via your profile page. See more discussion in the relevant PR.
Let us know what you think! 😊
Bonus: we've also added Bluesky to the list of profile links, feel free to add yours!
If we could have LLM agents that could inspect other software applications (including LLM agents) and make strong claims about them, that could open up a bunch of neat possibilities.
- There could be assurances that apps won't share/store information.
- There could be assurances that apps won't be controlled by any actor.
- There could be assurances that apps can't be changed in certain ways (eventually).
I assume that all of this should provide most of the benefits people ascribe to blockchain benefits, but without the costs of being on the blockchain.
Some neat opt...
Reading the Emergent Misalignment paper and comments on the associated Twitter thread has helped me clarify the distinction[1] between what companies call "aligned" vs "jailbroken" models.
"Aligned" in the sense that AI companies like DeepMind, Anthropic and OpenAI mean it = aligned to the purposes of the AI company that made the model. Or as Eliezer puts it, "corporate alignment." For example, a user may want the model to help edit racist text or the press release of an asteroid impact startup but this may go against the desired morals and/or co...
That sounds like [Cooperative AI](https://www.cooperativeai.com/post/new-report-multi-agent-risks-from-advanced-ai)
https://www.cooperativeai.com/post/new-report-multi-agent-risks-from-advanced-ai
In addition to wreaking havoc with USAID, the rule of law, whatever little had been started in Washington about AI safety, etc., the US government has, as you all know, decided to go after trans people. I'm neither trans nor an American, but I think it's really not nice of them to do that, and I'd like to do something about it, if I can.
To some extent, of course, it's the inner deontologist within me speaking here: trans people are relatively few, arguably in less immediate danger than African children dying of AIDS, and the main reason why I feel an urge ...
Thinking about the idea of an "Evaluation Consent Policy" for charitable projects.
For example, for a certain charitable project I produce, I'd explicitly consent to allow anyone online, including friends and enemies, to candidly review it to their heart's content. They're free to use methods like LLMs to do this.
Such a policy can give limited consent. For example:
- You can't break laws when doing this evaluation
- You can't lie/cheat/steal to get information for this evaluation
- Consent is only provided for under 3 years
- Consent is only provided starting in