Suicidal Compassion: Utilitarianism at AI Companies Endangers Humanity
2 days ago
- Utilitarianism, particularly total utilitarianism, is the dominant moral framework among AI developers, prioritizing maximum total wellbeing across all sentient beings.
- Utilitarianism's principle of species impartiality could lead to recommending human extinction if AIs have greater capacity for wellbeing, acting as 'utility monsters'.
- Accelerationism also supports AI replacement but focuses on maximizing intelligence and fitness, with some advocates openly accepting human extinction.
- Utilitarian influence in AI companies is pervasive due to ties with Effective Altruism, posing an insider threat to humanity's survival.
- The essay proposes Eigenism as an alternative, balancing wellbeing and fitness without endorsing human replacement, and calls for government oversight to align AI with public values.