"Only of a living human being and what resembles (behaves like) a living human being can one say: it has sensations; it sees; is blind; hears; is deaf; is conscious or unconscious." (Wittgenstein)


(1) Preliminary deliberations - Some thoughts about concepts

Human concepts often depend on each other, such as "movement", "space" and "time", each of which cannot be contemplated or used without invoking the other two. They stand in mutual support of each other, which speaks in favour of a holistic view of our conceptual world(s).

On the other hand it is undeniable that we learn to use language in a step by step fashion. Children start with attenuated variants of concepts, for example those of arithmetic [1], or of mental and cognitive vocabulary such as "thought", "memory" or "perception". We judge their competence according to certain criteria (e.g. arithmetic tests and exams) and at some point call them fully competent users of language [2]. So here we have an argument in favour of a gradual view on the meaning and use of words.

These two views are not in opposition of each other, even though they may seem to be. Wittgenstein gave the image of a roman arc bridge. Only the final capstone renders the bridge complete and able to sustain - it enables the other stones to support each other in bearing the weight. However, we speak of a "bridge" in all stages of its construction, even when it is still unusable in its final purpose (which in this analogy corresponds to being "fully competent"). This does not mean that it is not possible to meaningfully (usefully) speak of a "bridge" before its completion, as much with regard to its final purpose as to other aspects such as architectural or aesthetic details.

Just like "bridge", the concept of "thought" can be used in preliminary stages of linguistic competence. If necessary, the not yet fully competent speaker will be corrected (see [2]).

In summary, we note that in the process of learning to cope with complexes of concepts there occurs a gradual incline of linguistic competence with regard to every single concept, however, the dependencies between different concepts are retained throughout. If a speaker wants to increase her competence, she will have to increase her skills all over the field of the complex. It is not wrong to say that her competence increases gradually with regard to every single concept of the complex, but it is an error to conclude that her competence can be improved "pointwise": Her skill in the use of one particular concept cannot be furthered in isolation. As Donald Davidson has noted, someone who has mastered only one concept has not mastered any concept at all.


(2) AI - Plucking the mental from its human context

Focussing on any one particular concept, it is easy to succumb to a notion of independence among the linguistic competencies of a speaker. This can lead to the erroneous conclusion that a machine (in particular a computer) may be able to think, or be in a preliminary state of thought or mental abilities, only because it is able to achieve certain performances which we would ascribe to mental abilities in a person. However, the concept of human thought cannot be isolated from its human context so easily: The ascription of thought is bound up with other concepts of human abilities, expressions and behaviour. As long as the machine cannot emulate these as well, it will lack the necessary criteria for an ascription of mental capabilities and thus the complexity to have sufficient intersection with the human world: Its global repertoire of abilities tied to what we call the mental realm in humans will be too small, despite individual performances equal to or perhaps even surpassing those of humans. A "thinking machine" like that cannot be understood (and treated) by us in the same way we understand (and treat) a thinking person. We remain alienated from the machine in important ways. It is not performances alone, with regard to what we call a mental achievement, which lead to the ascription of "thought" [3].

Once we realize that human thought cannot be isolated from the human world in which it is situated, we can note that this is a big problem for supporters of artificial intelligence: The isolation process is precisely the promise of AI. The "artificial" in "AI" describes just the idea that intelligence can be thought of independently from its material or worldly context. Why? Because a system which mimics all features of human intelligence (including phenomena and performances such as facial expressions) can no longer be called artificial. A favoured argument among AI supporters reads "if the simulation is good enough, it becomes the real thing". Which is true, but comes at the cost of no longer being able to call it a simulation. Then the controversy about "men being nothing more than machines" vanishes together with the simulation ascription. In the case of a complete simulation, the reduction of man to machine threatened by AI disappears. In this case, the original task of AI -- the attempt to build a machine with human abilities -- would have to make way for a new program, the construction of an android, an artificial human being. But this contradicts "strong" AI's original purpose, which regards isolated intelligent performances.

We finally note that in the case of artificial intelligence no "fully competent state" can be named, to which a machine could or should aspire. (X) This differentiates AI from human intelligence, the limits of the latter we are well acquainted with due to our abilities and experiences. We understand our mental abilities in the light of certain "fully developed" competencies. This acquaintance consists in more than experiences of performance, since we ascribe intelligence to ourselves and others even in certain cases of failure ("actually he knows how to do it, he is just having a bad day"). But because of (X), the alleged mental abilities of machines cannot be measured against well known paradigmatic application cases (like an arithmetic exam; see also [2]). The fact that mental concepts are learned gradually is thus not sufficient to claim that there are no categorial differences between the performances of humans and machines. The ascription of mental abilities, intelligence, etc. is not justifiable by a graduation argument alone.


---


[1] Even in arithmetic one does not come to know of the use of "1", "2", "+", etc. independently. This demonstrates the non-empirical character of arithmetic. If we knew what "1", "2" and "+" are "each for themselves", one could discover that 1+1=2. It would then also be conceivable that 1+1=2 may be false, just like there might be no oxygen in earth's atmosphere.

[2] This happens via paradigmatic application cases which are seen as rules for the use of the concept which they characterise, and not by directly or indirectly acquired definitions. (This difference will later become important when we consider machine intelligence.)

[3] Note that "inner states" play no role with regard to the question of thought in humans or machines. They never can, since there are no criteria according to which one could compare the "insides" of humans and machines (or humans and humans).

Until its clear whether we are a simulation or not, this is an futile attempt at anything.

If we (universe) are a simulation, how then dichotomize between the simulating system (universe) and the in-system simulation (AI)? Or between the simulated human and the human simulated AI? At what level of which system the "I" is created?

There seems to be some confusion here about what a simulation is and what the implications of such an assumption are. If our entire world is a simulation, it is a simulation only to those who simulate it, that is, to those who have us in their fish tank. The argument I gave above, which is purely conceptual and does not rely in any way on material particularities (such as inside of who else's system we might exist), nevertheless applies to those inside the fish tank. On a different level of meaning it may or may not apply to those outside the tank, but that is not even a concern here, since the question was about us and our machines, not about fictional entities which may or may not exist.

So how do we dichotomize? For the case of ourselves and the AIs we have created, the article above shows how: By examining the concept of intelligence as we apply it to humans and comparing it to the abilities of the machines we have built. The result is negative: There is no sufficient match between the world of humans and the world of machines to warrant the ascription of human intelligence to both.

With regard to any hypothetical simulators of our world, it is questionable whether these outsiders can be said to simulate us in the same way that we use the term "simulate", as long as the relationship between our world and theirs has not been made explicit. It is unclear whether that relationship could possibly be the same as the relationship between humans and our AIs. It is therefore not a very fruitful question until you can come up with some factual support for such a scenario.

As for the question of where "the I" is created, this is another conceptual error. There is no such thing as "the I". In fact, the phrase "the I" is ungrammatical and therefore excluded from language. The use of "I" expresses my linguistic ability to refer to myself, it is not a description of a mysterious entity called "the self" (as opposed to "you" and "he/she/it" which denote proper entities). As Peter Hacker has remarked, the term "I" is a limiting case of a pronoun, similar to a point being the limiting case of a circle (with radius zero).

Otherwise one may discuss philosophy and zombies in the chinese room as much as one wants. It might be fun, but not much more relevant than trying to establish humans outside of animals, and possible create an illusion of humans as godlike creatures above everything else.

Unfortunately I have no idea what you are trying to say here.

Just a small immediate clarification. "I" refers to the "I" in "AI" or the "I" in the human version.

So expand it about to the following: At what level of which system the "Intelligence" is created?

The answer is largely the same: Intelligence is not created, it is ascribed. It is a term we make use of, not a thing to be discovered "out there". (It has that in common with numbers, see footnote [1] in the original article.) Your reification of intelligence gets you off of the wrong foot altogether and sets you spinning in all kinds of directions in a confused way.

The simulation hypothesis is well known in different variants. And there are at least one (or a few) experiments described to make sure if we are living in a simulation or not. So its not clear at the moment whether we "live" a simulation or not...

Again, this is beside the point I make in the article. The question of whether we should call machines that are able to achieve certain performances "intelligent" is independent of any simulation hypotheses.

The last parts (about simulation) are available around in different scientifi writings if needed, and of course also in popularized versions. Look for Nick Bostrom for latest variants. The theories are way back though and also in physics etc.

I was concerned with the philosophy of intelligence, not with science fiction scenarios. 99.9% of all sci-fi writing makes the same or similar erroneous assumptions as you do.

Modern physics has confirmed the non-determinacy of our world, and, perhaps even more important, the notion that reality itself is observer-dependent (quantum physics). This is a major obstacle for all speculation about our world being a computer simulation, since all computer simulations are both, deterministic and non-observer dependent. Additionally, simulations that go beyond computer programs (or procedures which are equivalent to them) are (at least currently) beyond our grasp. The imagination of sci-fi writers may be great but so is their confusion about the basic nature (concept) of intelligence. The actual results of modern physics usually do not play out in their favour. It's called fiction for a reason.

Being concerned with philosophy of intelligence, and not catch the ref to pilosophical zombies is somewhat weird. That is actually what you are trying to discuss. I believe. Correct me if Im wrong.

It's not what I discuss in the article. The question of zombies is part of the question of "inner states" and the philosophy of experience. It can certainly be connected to questions of intelligence but it doesn't have to be. I chose not to in the article, it is long enough already. Machine intelligence can be discussed separately: See footnote [3] of the article. (Actually, read the article, from your responses I gather you haven't.)

And it is absolutely necessary to make sure to have a clear understanding of whats real before trying to dichotomize between real (human) and unreal (AI). But Ill leave it at that, and let some of the greater physicists including some Nobel laureates battle you on the fiction part.

No real/unreal distiction is necessary to discuss machine intelligence. The term "real" doesn't appear in the article except in connection with simulations. It is not central to the argument. The question was: Can machines think? Or less controversially put: Should we ascribe "intelligence", "thought" and/or "mental abilities" to machines which deliver performances which we would call mental achievements if delivered by humans? There are no "real" or "unreal" thoughts. The question is whether it makes sense to speak of the same concept as far as human abilities and machine abilities are concerned.

Building on what I wrote one could conclude that there are no philosophical zombies (for the conceptual reasons I mention), but this is not the core of my argument, merely a possible corollary, which would need several extra lines of argumentation to make it solid.

To hide behind "greater physicists including some Nobel laureates" without even mentioning a single name or actual idea is of course the worst possible way to lead an argument. Speaking of arguments, you haven't brought forth a single one yet, only a lot of diffuse handwaving to ideas of others you couldn't even be bothered to explain. What should your readers think of that?

I thought we should concentrate on what you actually wanted to discuss, and not have a feast of herrings. Thats why I didnt bother to give you any refs. It wasnt up for discussion. And you had it already finshed off as fiction so whats the need for discussing it?

Now we're in trolling territory. I have addressed all of your concerns as best I could. You chose to come back with an accusation of "a feast of herrings".

"Refs" won't do, only actual arguments will. As in proper and elaborate responses to what others write. Nothing you have written so far comes even close to that. I've explained what the article was about 3 times by now. Put up or shut up, as they say.

No accusations. I was trying to avoid it, and actually stay on your subject. "a feast of herring" was as much pointed at me as anyone else, if anyone else...

Understood.

So called many-world theories (which in fact claim that wave packet collapse is an illusion)haven't yet been debunked right now so we are not actually sure that the world is undeterministic.

If we believe in Church-Turing thesis then any device created by man can do so few things that I wouldn't call that "thinking". The possibility that something which doesn't even live, can think is counter intuitive imho.