The Shelf

Why

I heard about Max Harms on the 80000 hours podcast, talking about AI alignment.

What I liked

From the outside, most humans interact with the AI (protagonist of the book) as if it was a single coherent entity: but it isn't. The AI is a "society" (hence the title of the book) of interacting "personalities". The analog is obviously with AI agents, each focused on their narrow goal, called The Purpose. The AI's personality then arises from the interaction of often opposing goals, in a mostly democratic way. There is no single dominant "agent", but they cooperate to advance their purpose.

What I disliked

The book also features an alien civilization, stationed on Earth orbit. The purpose of these characters is to present the reader with yet another "alien" intelligence, but I think their role is mostly irrelevant until the last part of the book. The reader is made aware of their existence at the very beginning of the book, but doesn't get to know them until the end. It's an odd choice of plot, but it leads naturally to the second installment of the series: Crystal Mentality.