Thank you for this interesting overview of Vincent Müller’s arguments! I fully agree that implementation (policy means) often becomes the bottleneck. However, if we systematically reward behavior that contradicts our declared principles, then any “ethical goals” will inevitably be vulnerable to being undermined during implementation. In my own post, I call this the “bad parent” problem: we say one thing, but demonstrate another. Do you think it’s possible to achieve robust adherence to ethical principles in AI when society itself remains fundamentally inconsistent?

Thanks for the comment, Ronen! Appreciate the feedback.

I think it’s good — essential, even — that you keep trying and speaking out. Sometimes that’s what helps others to act too.
The only thing I worry about is that this fight, if framed only as hopeless, can paralyze the very people who might help change the trajectory.
Despair can be as dangerous as denial.

That’s why I believe the effort itself matters — not because it guarantees success, but because it keeps the door open for others to walk through.

I live in Ukraine. Every week, missiles fly over my head. Every night, drones are shot down above my house. On the streets, men are hunted like animals to be sent to the front. Any rational model would say our future is bleak.

And yet, people still get married, write books, make music, raise children, build new homes, and laugh. They post essays on foreign forums. They even come up with ideas for how humanity might live together with AGI.

Even if I go to sleep tonight and never wake up tomorrow, I will not surrender. I will fight until the end. Because for me, a 0.0001% chance is infinitely more than zero.

Seems like your post missed the April 1st deadline and landed on April 2nd — which means, unfortunately, it no longer counts as a joke.

After reading it, I also started wondering if I unintentionally fall into the "Believer" category—the kind of person who's already drafting blueprints for a bright future alongside AGI and inviting people to "play" while we all risk being outplayed.

I understand and share your concerns. I don’t disagree that the systemic forces you’ve outlined may well make AGI safety fundamentally unachievable. That possibility is real, and I don’t dismiss it.

But at the same time, I find myself unwilling to treat it as a foregone conclusion.
If humanity’s survival is unlikely, then so was our existence in the first place — and yet here we are.

That’s why I prefer to keep looking for any margin, however narrow, where human action could still matter.

In that spirit, I’d like to pose a question rather than an argument:
Do you think there’s a chance that humanity’s odds of surviving alongside AGI might increase — even slightly — if we move toward a more stable, predictable, and internally coherent society?
Not as a solution to alignment, but as a way to reduce the risks we ourselves introduce into the system.

That’s the direction I’ve tried to explore in my model. I don’t claim it’s enough — but I believe that even thinking about such structures is a form of resistance to inevitability.

I appreciate this conversation. Your clarity and rigor are exactly why these dialogues matter, even if the odds are against us.

I completely understand your position — and I respect the intellectual honesty with which you’re pursuing this line of argument. I don’t disagree with the core systemic pressures you describe.

That said, I wonder whether the issue is not competition itself, but the shape and direction of that competition.
Perhaps there’s a possibility — however slim — that competition, if deliberately structured and redirected, could become a survival strategy rather than a death spiral.

That’s the hypothesis I’ve been exploring, and I recently outlined it in a post here on the Forum.
If you’re interested, I’d appreciate your critical perspective on it.

Either way, I value this conversation. Few people are willing to follow these questions to their logical ends.

This is a critically important and well-articulated post, thank you for defining and championing the Moral Alignment (MA) space. I strongly agree with the core arguments regarding its neglect compared to technical safety, the troubling paradox of purely human-centric alignment given our history, and the urgent need for a sentient-centric approach.

You rightly highlight Sam Altman's question: "to whose values do you align the system?" This underscores that solving MA isn't just a task for AI labs or experts, but requires much broader societal reflection and deliberation. If we aim to align AI with our best values, not just a reflection of our flawed past actions, we first need robust mechanisms to clarify and articulate those values collectively.

Building on your call for action, perhaps a vital complementary approach could be fostering this deliberation through a widespread network of accessible "Ethical-Moral Clubs" (or perhaps "Sentientist Ethics Hubs" to align even closer with your theme?) across diverse communities globally.

These clubs could serve a crucial dual purpose:

1. Formulating Alignment Goals: They would provide spaces for communities themselves to grapple with complex ethical questions and begin articulating what kind of moral alignment they actually desire for AI affecting their lives. This offers a bottom-up way to gather diverse perspectives on the "whose values?" question, potentially identifying both local priorities and identifying shared, potentially universal principles across regions.
2. Broader Ethical Education & Reflection: These hubs would function as vital centers for learning. They could help participants, and by extension society, better understand different ethical frameworks (including the sentientism central to your post), critically examine their own "stated vs. realized" values (as you mentioned), and become more informed contributors to the crucial dialogue about our future with AI.

Such a grassroots network wouldn't replace the top-down efforts and research you advocate for, but could significantly support and strengthen the MA movement you envision. It could cultivate the informed public understanding, deliberation, and engagement necessary for sentient-centric AI to gain legitimacy and be implemented effectively and safely.

Ultimately, fostering collective ethical literacy and structured deliberation seems like a necessary foundation for ensuring AI aligns with the best of our values, benefiting all sentient beings. Thanks again for pushing this vital conversation forward.

Thank you for such an interesting and useful conversation. 
Yes I use LLM, I don't hide it. First of all for translation, because my ordinary English is mediocre enough, not to mention such a strict and responsible style, which is required for such conversations. But the main thing is that the ideas are mine and chatGPT, who framed my thoughts in this discussion, formed answers based on my instructions. And the main thing is that the whole argumentation is built around my concept, everything we wrote to you is not just an argument for the sake of argument, but the defense of my concept. This concept I want to publish in the next few days and I will be very glad to receive your constructive criticism.

Now as far as AGI is concerned. I really liked your argument that even the smartest AGI will be limited.  It summarizes our entire conversation perfectly. Yes, our logic is neither perfect nor omnipotent. And as I see it, that is where we have a chance. A chance, perhaps, not just to be preserved as a mere backup, but to that structural interdependence, and maybe to move to a qualitatively different level, in a good way, for humanity.

PS sorry if it's a bit rambling, I wrote it myself through a translator).

You’re right to point out that human biological architecture is inherently competitive, irrational, and unreliable from an optimization perspective. I don’t dispute that.
If AGI’s frame of evaluation is risk minimization and maximization of control, then yes — trust, in the human sense, is structurally impossible.

But perhaps the problem is not "trust" at all.
Perhaps the problem is how we define risk.

If survival of AGI requires human unpredictability to be neutralized, the typical solution is either:

• enforce absolute control, or
• modify the human substrate beyond recognition.

But there exists a third, rarely discussed, structural option:

Architected mutual dependence, enforced not by ethics or emotion — but by the wiring of reality itself.

Not because AGI “trusts” humanity,
but because AGI’s own long-term survival becomes entangled with the survival of human agency.

This is not a fragile social contract.
It is an engineered condition where:

• Humanity retains one or more non-replicable control nodes — physical, informational, or systemic — outside AGI’s unilateral reach.
• AGI retains the freedom to operate at superhuman scale — but cannot guarantee its own continuity without human participation in specific critical functions.

In such a structure:

• Eliminating humanity creates existential risk for AGI itself.
• Preserving humanity, despite unpredictability, becomes the rational, stable strategy.

This is not "trust."
This is a game-theoretic lock, built into the architecture of survival itself.

It may require us to relinquish traditional concepts of autonomy.
It may require AGI to accept permanent non-closure of certain optimization processes.

But it sidesteps the naive dichotomy between:

• Total control (which eliminates humanity’s relevance)
• or Total submission (which erases humanity’s agency)

Instead, it establishes a condition where both sides can only persist together — or perish together.

You’re right: if the future leaves no structural need for us, we won’t be preserved.
So perhaps the last, best move is to design a future in which our existence is structurally required.

Not because AGI values us.
But because it has no better option.