Friday, June 09, 2006
Motivational system
1) I agree that direct reward has to be in-built
(into brain / AI system).
2) I don't see why direct reward cannot be used for rewarding mental
achievements. I think that this "direct rewarding mechanism" is
preprogrammed in genes and cannot be used directly by mind.
This mechanism probably can be cheated to the certain extend by the
mind. For example mind can claim that there is mental achievement when
actually there is none.
That possibility of cheating with rewards is definitely a problem.
I think this problem is solved (in human brain) by using only small
dozes of "mental rewards".
For example, you can get small positive mental rewards by cheating your
mind to like finding solutions to "1+1=2" problem.
However, if you do it too often you'll eventually get hungry and would
get huge negative reward. This negative reward would not just stop you
doing "1+1=2" operation over and over, it would also re-setup your
judgement mechanism, so you will not consider "1+1=2" problem as an
achievement anymore.
Also, we all familiar with what "boring" is.
When you solve a problem once - it's boring to solve it again.
I guess that that is another genetically programmed mechanism with
prevents cheating with mental rewards.
3) Indirect rewarding mechanisms definitely work too, but they are not
sufficient for bootstrapping strong-AI capable system.
Consider a baby. She doesn't know why it's good to play (alone or with
others). Indirect reward from "childhood playing" will come years later
from professional success.
Baby cannot understand human language yet, so she cannot envision this
success.
AI system would face the same problem.
My conclusion: indirect reward mechanisms (as you described them) would not be
able to bootstrap strong-AI capable system.
Back to real baby: typically nobody explains to baby that it's good to play.
But somehow babies/children like to play.
My conclusion: there are direct reward mechanisms in humans even for
things which are not directly beneficial to the system (like mental
achievements, speech, physical activity).
(from AGI email list).
(into brain / AI system).
2) I don't see why direct reward cannot be used for rewarding mental
achievements. I think that this "direct rewarding mechanism" is
preprogrammed in genes and cannot be used directly by mind.
This mechanism probably can be cheated to the certain extend by the
mind. For example mind can claim that there is mental achievement when
actually there is none.
That possibility of cheating with rewards is definitely a problem.
I think this problem is solved (in human brain) by using only small
dozes of "mental rewards".
For example, you can get small positive mental rewards by cheating your
mind to like finding solutions to "1+1=2" problem.
However, if you do it too often you'll eventually get hungry and would
get huge negative reward. This negative reward would not just stop you
doing "1+1=2" operation over and over, it would also re-setup your
judgement mechanism, so you will not consider "1+1=2" problem as an
achievement anymore.
Also, we all familiar with what "boring" is.
When you solve a problem once - it's boring to solve it again.
I guess that that is another genetically programmed mechanism with
prevents cheating with mental rewards.
3) Indirect rewarding mechanisms definitely work too, but they are not
sufficient for bootstrapping strong-AI capable system.
Consider a baby. She doesn't know why it's good to play (alone or with
others). Indirect reward from "childhood playing" will come years later
from professional success.
Baby cannot understand human language yet, so she cannot envision this
success.
AI system would face the same problem.
My conclusion: indirect reward mechanisms (as you described them) would not be
able to bootstrap strong-AI capable system.
Back to real baby: typically nobody explains to baby that it's good to play.
But somehow babies/children like to play.
My conclusion: there are direct reward mechanisms in humans even for
things which are not directly beneficial to the system (like mental
achievements, speech, physical activity).
(from AGI email list).
Comments:
<< Home
Strong AI only needs one initial reward. It does, however, need the ability to create rewards by association.
Your real baby (and I'm new here, so forgive me, and I probably won't be back) starts with one reward: hunger. Mother satisfies that need. But mother only satisfies that need when she's happy (or something). So making mother happy becomes a reward. Mother's happy when baby does something new. So THAT becomes a reward.
Does that make sense? AI is a hobby of mine and I've been thinking and analyzing human behavior for much of my life.
r dot adam dot mitchell at gmail dot com
Feel free to email me.
Post a Comment
Your real baby (and I'm new here, so forgive me, and I probably won't be back) starts with one reward: hunger. Mother satisfies that need. But mother only satisfies that need when she's happy (or something). So making mother happy becomes a reward. Mother's happy when baby does something new. So THAT becomes a reward.
Does that make sense? AI is a hobby of mine and I've been thinking and analyzing human behavior for much of my life.
r dot adam dot mitchell at gmail dot com
Feel free to email me.
# posted by 
: 7:04 PM
: 7:04 PM << Home
