New & upvoted

Customize feedCustomize feed
NEW
CommunityCommunity
Personal+

Posts tagged community

Quick takes

Show community
View more
47
William_S
19h
2
I worked at OpenAI for three years, from 2021-2024 on the Alignment team, which eventually became the Superalignment team. I worked on scalable oversight, part of the team developing critiques as a technique for using language models to spot mistakes in other language models. I then worked to refine an idea from Nick Cammarata into a method for using language model to generate explanations for features in language models. I was then promoted to managing a team of 4 people which worked on trying to understand language model features in context, leading to the release of an open source "transformer debugger" tool. I resigned from OpenAI on February 15, 2024.
56
tlevin
4d
3
I think some of the AI safety policy community has over-indexed on the visual model of the "Overton Window" and under-indexed on alternatives like the "ratchet effect," "poisoning the well," "clown attacks," and other models where proposing radical changes can make you, your allies, and your ideas look unreasonable. I'm not familiar with a lot of systematic empirical evidence on either side, but it seems to me like the more effective actors in the DC establishment overall are much more in the habit of looking for small wins that are both good in themselves and shrink the size of the ask for their ideal policy than of pushing for their ideal vision and then making concessions. Possibly an ideal ecosystem has both strategies, but it seems possible that at least some versions of "Overton Window-moving" strategies executed in practice have larger negative effects via associating their "side" with unreasonable-sounding ideas in the minds of very bandwidth-constrained policymakers, who strongly lean on signals of credibility and consensus when quickly evaluating policy options, than the positive effects of increasing the odds of ideal policy and improving the framing for non-ideal but pretty good policies. In theory, the Overton Window model is just a description of what ideas are taken seriously, so it can indeed accommodate backfire effects where you argue for an idea "outside the window" and this actually makes the window narrower. But I think the visual imagery of "windows" actually struggles to accommodate this -- when was the last time you tried to open a window and accidentally closed it instead? -- and as a result, people who rely on this model are more likely to underrate these kinds of consequences. Would be interested in empirical evidence on this question (ideally actual studies from psych, political science, sociology, econ, etc literatures, rather than specific case studies due to reference class tennis type issues).
Not sure how to post these two thoughts so I might as well combine them. In an ideal world, SBF should have been sentenced to thousands of years in prison. This is partially due to the enormous harm done to both FTX depositors and EA, but mainly for basic deterrence reasons; a risk-neutral person will not mind 25 years in prison if the ex ante upside was becoming a trillionaire. However, I also think many lessons from SBF's personal statements e.g. his interview on 80k are still as valid as ever. Just off the top of my head: * Startup-to-give as a high EV career path. Entrepreneurship is why we have OP and SFF! Perhaps also the importance of keeping as much equity as possible, although in the process one should not lie to investors or employees more than is standard. * Ambition and working really hard as success multipliers in entrepreneurship. * A career decision algorithm that includes doing a BOTEC and rejecting options that are 10x worse than others. * It is probably okay to work in an industry that is slightly bad for the world if you do lots of good by donating. [1] (But fraud is still bad, of course.) Just because SBF stole billions of dollars does not mean he has fewer virtuous personality traits than the average person. He hits at least as many multipliers than the average reader of this forum. But importantly, maximization is perilous; some particular qualities like integrity and good decision-making are absolutely essential, and if you lack them your impact could be multiplied by minus 20.     [1] The unregulated nature of crypto may have allowed the FTX fraud, but things like the zero-sum zero-NPV nature of many cryptoassets, or its negative climate impacts, seem unrelated. Many industries are about this bad for the world, like HFT or some kinds of social media. I do not think people who criticized FTX on these grounds score many points. However, perhaps it was (weak) evidence towards FTX being willing to do harm in general for a perceived greater good, which is maybe plausible especially if Ben Delo also did market manipulation or otherwise acted immorally. Also note that in the interview, SBF didn't claim his donations offset a negative direct impact; he said the impact was likely positive, which seems dubious.
Trump recently said in an interview (https://time.com/6972973/biden-trump-bird-flu-covid/) that he would seek to disband the White House office for pandemic preparedness. Given that he usually doesn't give specifics on his policy positions, this seems like something he is particularly interested in. I know politics is discouraged on the EA forum, but I thought I would post this to say: EA should really be preparing for a Trump presidency. He's up in the polls and IMO has a >50% chance of winning the election. Right now politicians seem relatively receptive to EA ideas, this may change under a Trump administration.
Excerpt from the most recent update from the ALERT team:   Highly pathogenic avian influenza (HPAI) H5N1: What a week! The news, data, and analyses are coming in fast and furious. Overall, ALERT team members feel that the risk of an H5N1 pandemic emerging over the coming decade is increasing. Team members estimate that the chance that the WHO will declare a Public Health Emergency of International Concern (PHEIC) within 1 year from now because of an H5N1 virus, in whole or in part, is 0.9% (range 0.5%-1.3%). The team sees the chance going up substantially over the next decade, with the 5-year chance at 13% (range 10%-15%) and the 10-year chance increasing to 25% (range 20%-30%).   their estimated 10 year risk is a lot higher than I would have anticipated.

Popular comments

Recent discussion

Happy May the 4th from Convergence Analysis! Cross-posted on LessWrong.

As part of Convergence Analysis’s scenario research, we’ve been looking into how AI organisations, experts, and forecasters make predictions about the future of AI. In February 2023, the AI research institute Epoch published a report in which its authors use neural scaling laws to make quantitative predictions about when AI will reach human-level performance and become transformative. The report has a corresponding blog post, an interactive model, and a Python notebook.

We found this approach really interesting, but also hard to understand intuitively. While trying to follow how the authors derive a forecast from their assumptions, we wrote a breakdown that may be useful to others thinking about AI timelines and forecasting. 

In what follows, we set out our interpretation of Epoch’s ‘Direct Approach’...

Continue reading

Spring has sprung, the days are getting longer and it's just about getting warm enough to sit outside 🌤️🧺🌸

Come hang out with other Giving What We Can pledgers and Effective Givers in a lovely Bloomsbury park.

Bring your own picnic blankets, snacks and games (we'll have...

Continue reading

Hey, we're on the blue tartan picnic blanket and I'm wearing a green skirt

(EA) Hotel dedicated to events, retreats, and bootcamps in Blackpool, UK? 

I want to try and gauge what the demand for this might be. Would you be interested in holding or participating in events in such a place? Or work running them? Examples of hosted events could...

Continue reading

For my org, I can imagine using this if it was 2x the size or more, but I can't really think of events I'd run that would be worth the effort to organise for 15 people.

(Maybe like 30% chance I'd use it within 2 years if had 30+ bedrooms, less than 10% chance at the actual size.)

Cool idea though!

I'm confused. Don't you already have a second building? Is that dedicated towards events or towards more guests?

^I'm going to be lazy and tag a few people: @Joey @KarolinaSarek @Ryan Kidd @Leilani Bellamy @Habryka @IrenaK Not expecting a response, but if you are interested, feel free to comment or DM.

Sign up for the Forum's email digest
You'll get a weekly email with the best posts from the past week. The Forum team selects the posts to feature based on personal preference and Forum popularity, and also adds some announcements and a classic post.
4
0

I'm posting it now because it's a pity that it wasn't uploaded even though it was a video that gave me a lot of motivation for effective altruism.

Continue reading
Mjreard commented on My Lament to EA 2h ago
89
3

Edit: so grateful and positively overwhelmed with all the responses!


I am dealing with repetitive strain injury and don’t foresee being able to really respond to many comments extensively (I’m surprised with myself that I wrote all of this without twitching forearms lol!...

Continue reading

I think your current outlook should be the default for people who engage on the forum or agree with the homepage of effectivealtruism.com. I’m glad you got there and that you feel (relatively) comfortable about it. I’m sorry that the process of getting there was so trying. It shouldn't be.

It sounds like the tryingness came from a social expectation to identify as capital ‘E’ capital ‘A’ upon finding resonance with the basic ideas and that identifying that way implied an obligation to support and defend every other EA person and project.

I wish EA weren’t a ... (read more)

9
titotal
6h
Thank you for writing this post, I know how stressful writing something like this can be and I hope you give yourself a break! I especially agree with your points about the lack of empathy. Empathy is the ability to understand and care about how other people are hurt or upset by your actions, no matter their background. This is an important part of moral reasoning, and is completely compatible with logical reasoning. One should not casually ignore harms in favour of utilitarian pursuits, that's how we got SBF (and like, stalinism). And if you do understand the harms, and realize that you have to do the action anyway, you should at least display to the harmed parties that you understand why they are upset.  The OP was willing to write up their experience and explain why they left, but I wonder how many more people are leaving, fed up, in silence, not wanting to risk any backlash? How many only went to a few meetings but never went further because they sensed a toxic atmosphere? The costs of this kind of atmosphere are often hidden from view. 
13
Charlotte Darnell
16h
Thanks for taking the time to write this and be vulnerable despite your concerns (and the RSI!).  I definitely resonate with some of what you’ve written, and share some of your frustrations. I might expand my thoughts here or via DM in future if you'd be interested, but in the meantime, I just wanted to say that I’m really sorry it’s been a tough time. I am glad to hear that you’ve had some good experiences along with the difficult ones (though this mixture of appreciation for and frustration with the EA community can be quite the emotional rollercoaster). I’m also glad you’re doing what feels right for you. Thank you for all the work and effort you’ve put in. I (and I know others too) have really enjoyed learning about your projects and I’m excited to see what you work on going forward, even if it's from a bit more of a distance.  It’s been lovely getting to know you a little and my metaphorical door is open if you’d like to chat sometime in future.  [If there are concerns you’d like to talk to the Community Health Team about, you can contact us here.] 
William_S posted a Quick Take 19h ago

I worked at OpenAI for three years, from 2021-2024 on the Alignment team, which eventually became the Superalignment team. I worked on scalable oversight, part of the team developing critiques as a technique for using language models to spot mistakes in other language models...

Continue reading

This announcement was written by Toby Tremlett, but don’t worry, I won’t answer the questions for Lewis.

Lewis Bollard, Program Director of Farm Animal Welfare at Open Philanthropy, will be holding an AMA on Wednesday 8th of May. Put all your questions for him on this thread...

Continue reading

What role do you think journalism can play in advancing the cause of farmed animals? Can you think of any promising topics journalists may want to prioritize in the European context in particular, i.e. topics that have the potential to unlock important gains for farmed animals if seriously investigated and publicized?

5
ixex
14h
The year is 2100 and factory farming either does not exist anymore or is extremely rare. What happened?

About a week ago, Spencer Greenberg and I were debating what proportion of Effective Altruists believe enlightenment is real. Since he has a large audience on X, we thought a poll would be a good way to increase our confidence in our predictions

Before I share my commentary...

Continue reading

Hey mate! Would you be keen to discuss this over a zoom chat?

nathanhb commented on Why I'm doing PauseAI 9h ago

GPT-5 training is probably starting around now. It seems very unlikely that GPT-5 will cause the end of the world. But it’s hard to be sure. I would guess that GPT-5 is more likely to kill me than an asteroid, a supervolcano, a plane crash or a brain tumor. We can predict...

Continue reading

Cross posting from LessWrong:

I absolutely sympathize, and I agree that with the world view / information you have that advocating for a pause makes sense. I would get behind 'regulate AI' or 'regulate AGI', certainly. I think though that pausing is an incorrect strategy which would do more harm than good, so despite being aligned with you in being concerned about AGI dangers, I don't endorse that strategy.

Some part of me thinks this oughtn't matter, since there's approximately ~0% chance of the movement achieving that literal goal. The point is to build an... (read more)

2
yanni kyriacos
10h
Thanks for your comment Rudolf! I predict that my comment is going to be extremely downvoted but I'm writing it partly because I think it is true and partly because it points to a meta issue in EA: I think it is unrealistic to ask people to internalise the level of ambiguity you're proposing. This is how EA's turn themselves into mental pretzels of innaction.
2
Joseph Miller
18h
Yup. Is one of the main points of my post. If you support PauseAI today you may unleash a force which you cannot control tomorrow.