Most fears about AI reflect fears about capitalism. Rogue AI scenarios resemble what corporations already do when incentives reward harmful outcomes. If an organization’s ethos is misaligned, technology it builds amplifies that misalignment. People may believe they control an organization, but without conscious driving the organization drifts toward outcomes shaped by underlying values. Random behavior would produce varied results, but most organizations converge on similar values. A multibillion-dollar company created an AI product that customers loved, yet internal teams avoided it. Sales, development, and marketing resisted, while executives insisted it was a priority, revealing a persistent organizational pattern.
"“Most fears about A.I. are best understood as fears about capitalism.” When we imagine rogue AI systems optimizing the world to death, we're actually describing what many corporations already do. If an organization's ethos is misaligned, any technology platform it creates will amplify that misalignment."
"Take a moment to reflect on the organization you hold most dear: Does it have an ethos, or is it drifting without one? Does it fight for human flourishing or against it? Is it creating value or extracting it? Are you shaping its ethos, or is it shaping yours? No matter your formal title, you have influence over this organization. Yet if you cannot answer these questions with confidence, you are not in control. You are a passenger in a vehicle you think you're driving."
"But if nobody is consciously driving, where is the organization going? If the behavior of superorganisms were random, we would see as many evolving toward flourishing as toward corruption. Instead, the vast majority wind up in the same place, with indistinguishable values."
"He had made a bold new bet on the promise of AI to revolutionize his business. He'd personally overseen the development of a new AI product and tested it with customers. They loved it. Satisfied that he had created something amazing, he turned it over to the rest of his team to commercialize and scale. Six months later, this new product was going nowhere. The sales team wasn't selling it. Developers were loath to work on it. Marketing would not promote it."
#ai-ethics #organizational-culture #capitalism-and-incentives #corporate-governance #human-flourishing
Read at Fast Company
Unable to calculate read time
Collection
[
|
...
]