During a recent discussion with Tristan, the subject of the Turing Test arose. For those who are unfamiliar, the test is intended as a way to determine if a machine has intelligence. You set it up so that it can converse with a human being – for instance, through a text-based instant message type conversation – and if the person thinks they are talking with another human, it can be taken as evidence that the machine is intelligent.
Setting aside the question of how good an intelligence test this really is (a computer could pretty easily trawl a database of human conversations to produce convincing conversation), it seems like there is another sort of test that would be demonstrative of a different kind of intelligence. Namely, it would be when a machine or a computer program first becomes aware of itself as being a machine or computer program.
It is possible that no machine made by humans will ever develop that level of self awareness. Perhaps it is impossible to replicate whatever trick our brains use to turn flesh into consciousness. If it did happen, however, it seems like it could help to illuminate what self-understanding means, and what sort of mechanisms it requires.