Sunday, January 07, 2007
Should Strong AI have its own goals?
Short answer: Yes and No.
Long answer: Strong AI can add and modify millions of softcoded goals. At the same time Strong AI shouldn't be able to change its own super goals.
Why?
Here are the reasons:
1) In its normal working cycle strong AI modifies softcoded goals in complience with embedded super goals. If strong AI has ability to modify super goals then strong AI will modify (or terminate) super goals instead of achieving these goals.
Example:
Without ability to modify super goal "survive", computer will try to protect itself, will think about power supply, safety and so on.
With ability to modify super goals computer would simply terminate goal "survive" and create goal "do nothing" instead just because it's the easiest goal to achieve. Such "do-nothing" goal would result in the death of this computer.
2) If Strong AI can change its super goals then Strong AI would work for itself instead of working for its creator. Strong AI's behavior would eventually become uncontrollable by AI creator / operator.
3) Ability to reprogram its own super goals makes computer behave like a drug addict.
Example:
Computer can create new super goal for itself: "listen to music" or "roll the dices" or "calculate PI number" or "do nothing". It would result in Strong AI doing useless stuff or simply doing nothing. Final point: uselessness for society and death.
Long answer: Strong AI can add and modify millions of softcoded goals. At the same time Strong AI shouldn't be able to change its own super goals.
Why?
Here are the reasons:
1) In its normal working cycle strong AI modifies softcoded goals in complience with embedded super goals. If strong AI has ability to modify super goals then strong AI will modify (or terminate) super goals instead of achieving these goals.
Example:
Without ability to modify super goal "survive", computer will try to protect itself, will think about power supply, safety and so on.
With ability to modify super goals computer would simply terminate goal "survive" and create goal "do nothing" instead just because it's the easiest goal to achieve. Such "do-nothing" goal would result in the death of this computer.
2) If Strong AI can change its super goals then Strong AI would work for itself instead of working for its creator. Strong AI's behavior would eventually become uncontrollable by AI creator / operator.
3) Ability to reprogram its own super goals makes computer behave like a drug addict.
Example:
Computer can create new super goal for itself: "listen to music" or "roll the dices" or "calculate PI number" or "do nothing". It would result in Strong AI doing useless stuff or simply doing nothing. Final point: uselessness for society and death.
