On Monday, 8 July 2013 at 21:46:24 UTC, Walter Bright wrote:
>
> I'm sure you're self-aware, as I'm sure Siri and Watson are not.

But there is no way for you to prove to me that you are self-aware. It could be that you are simply programmed to appear to be self-aware; think of an infinite loop containing a massive switch statement, where each case represents a different situation in life and the function to execute in each case represents what you did in that situation in life. As long as we can't test whether and entity is self-aware or not, for our purposes, it kind of doesn't matter whether it is or not.

If we ever are able to define what consciousness is (and I'm quite sure we will), I suspect it's going to be some kind of a continuous feedback loop from sensory data to brain, and from brain to muscles and through them back to sensory data again. Consciousness would be kind of your ability to predict what kind of sensory data would be likely to be produced if you sent a certain set of signals to your muscles.

I like this guy's take on consciousness:
http://www.youtube.com/watch?v=3jBUtKYRxnA

On Tuesday, 9 July 2013 at 06:07:12 UTC, Tommi wrote:
> Consciousness would be kind of your ability to predict what kind of sensory data would be likely to be produced if you sent a certain set of signals to your muscles.

...and the better you are at predicting those very-near-future sensory signals, the more you feel that you're conscious.

On Monday, 8 July 2013 at 18:37:30 UTC, Walter Bright wrote:
> On 7/8/2013 6:31 AM, Dicebot wrote:
>> Well, second one is not really a scientific problem, it is a philosophical one.
>> Self-awareness is a very vague term with a lot of space for personal
>> interpretation. I don't even think it is worth speaking about.
>
> If you consider that our brains evolved, and self-awareness was a result of evolution, then self-awareness presumably offers some sort of survival benefit.
>

Not necessarily. If the change is neutral, it can still develop in some species. Arguably, as our brain consume 20% of our energy, this is highly likely that it has benefit, so you still have a point.

> Following that line of reasoning, self-awareness becomes a real phenomenon with a scientific basis.

How is it defined in science ? The concept seems hard to define in proper ways to me.

On Monday, 8 July 2013 at 21:46:24 UTC, Walter Bright wrote:
> Just because we have difficulty defining something is not a reason to dismiss it as irrelevant or non-existent.
>
> I'm sure you're self-aware, as I'm sure Siri and Watson are not.
>

It is proven that at least 70% of what we perceive as being our decisions are in fact backward rationalization. I don't think the idea is that absurd that it may be 100%

On Monday, 8 July 2013 at 21:46:24 UTC, Walter Bright wrote:
> Just because we have difficulty defining something is not a reason to dismiss it as irrelevant or non-existent.

Sure, but there is an important difference between "dismissing" and "dismissing as a relevant scientific term to discuss". Speaking about possible self-awareness of computers is perfectly fine for a forum discussion but not acceptable for a scientific one. One needs a common well-defined terms to make progress.

> I'm sure you're self-aware, as I'm sure Siri and Watson are not.

I'll take it as a compliment :) But that is exactly what I am talking about - question if you consider someone self-aware is extremely interesting from the psychological point of view (probably even social psychology). For AI research important question is what properties do self-aware being has.

Those are related but different.

In a former case exact meaning of self-awareness is not important as you primarily study a person who makes a statement, not statement itself. In other words, it is not important what one means by "self-aware" but what thinking processes result in such tag.

The latter relies on research done in previous step to define properties of "self-aware" state that target AI needs to meet to be recognized as such by a wide variety of people. And, of course, as this relies on a common consensus, such concept is naturally very volatile. That is the main idea behind Turing test as far as I understand it.

> ... nor does it mean that personhood is not a very useful and meaningful construct.

Even worse, now you use "personhood" as a replacement for self-awareness! :) It is a very dangerous mistake to use common words when speaking about consciousness and thinking - too much self-reflection involved.

On Tuesday, 9 July 2013 at 10:38:11 UTC, Dicebot wrote:
>> ... nor does it mean that personhood is not a very useful and meaningful construct.
>
> Even worse, now you use "personhood" as a replacement for self-awareness! :) It is a very dangerous mistake to use common words when speaking about consciousness and thinking - too much self-reflection involved.

You're looking at it in the wrong context. Walter was talking about personhood as an analogy, not at all conflating it with self-awareness.

I agree 100% about the language point. By and large our languages (and language abilities) have evolved to identify and communicate day-to-day opportunities and risks.
They are very specialised DSLs running on very specialised hardware, not well suited to performing complex runtime introspection or large-scale formal logic :p

Interestingly, it doesn't take a huge change in design to unleash some very different abilities, e.g. autistic savants.