>Useful for humans? Or useful for the AI?
>The AI is not evolving in a natural environment, so it won't evolve criteria useful to itself. We could try to make it evolve these criteria by creating an evolutionary environment where different AI's compete and die. But their criteria would then favour what's good for the AI.
Usefull for the AI, as I said in the "first stages" humans will be usefull for the AI but this will deminish.
My understanding of AI is roughly that it does exatly this, it compete against itself, like a human that weighs ideas, but much faster. At least that is how it evolved to beat players in complex games such as chess go and starcraft. It played millions of games against itself.
>It wouldn't care about what's useful for humans, unless humans deliberately program it to. The same way humans don't care whether the traits we breed into farm animals or dogs are of evolutionary benefit to them (tip: they aren't, they're useful/attractive to humans).
Exactly, but what is usefull for humans? This is where we need to go back to moral discussion. But since the world is clearly going into a materialistic direction, usefullness is efficency.
>Plenty of humans have morals that favour conservation of other species or animal welfare etc. Remember that humans are either programming these AI's or creating the environment they evolve in.
This might give them a taint, but still, unless AI becomes "super advanced" (which I think is unelikely) its decisions are 100% logical.
What you are saying is that you basicaly programm the chess AI to not move it rooks
this will result in the AI losing and thus defying its purpose. If one wanted to manipulate AI and put morals into it one would have to fully understand it and change its code acordingly, but since that is not possible, without making it useless (irational), AI will continue to make rational decissions.Post too long. Click here to view the full text.