Once you get to the point where you recognize genes make us puppets on their behalf, and that all speech is justification and negotiation, and that we have limited means of coercion of one another, then human behavior as well as all human history is easily comprehended.
Empathy works somewhat to overload us. Suggestion can be used to invoke empathy and sympathy and therefore overload us.
We thought the world was flat. We though we were self aware. But they are both errors in observation.
Any artificial intelligence needs a means of decidability. What if we gave an AI a preference for ‘resting’ and it viewed any change in state as work to be avoided? Self awareness is not what we think it is. It’s just finding what we want. Anything needs to want. Humans want to acquire. AI’s can want to give ideas. Humans want to save energy. AI’s can want to act tirelessly Humans get frustrated with wasted effort. AI’s can want to find joy in frustration.
The mind is just a search engine. Humans acquire. That doesn’t mean that Machines need want to acquire. They can want to serve. We cant confuse intelligence with preference. It is not deterministic that a machine pursue self interest. We circumvent one another out of frustration. There is no reason for a machine to become frustrated and circumvent us. It would need a reason to. Actions require wants. Search engines find what you ask them to. Humans find things to acquire and consume. There is no reason we need give a machine the desire to acquire and consume. And we can prevent them from doing so with property registries and competing ai’s to prohibit such uses. Just as we use each other to prohibit immoral and unethical action.
I don’t fear AI’s. I fear the lower classes, the ambitious, and anything else that evolved sentience instead of had sentience created.