Can Machines Think? — Part II

French AI researcher Yann LeCun once said: “Whether we’ll able to use new methods to create human-level intelligence, there’s probably another 50 mountains to climb, including ones we can’t even see yet. We’ve only climbed the first mountain.”[1]

AI today is an engineering discipline based on incomplete fragmentary science. True AI will be possible after a much deeper understanding of the brain. One need not reverse-engineer the human brain, but a true AI machine will have to emulate certain aspects of the brain. The exploration of the human brain may be compared with the exploration of a new continent.

In 1492, Italian merchant sailor Christopher Columbus led a fleet of three Spanish ships to sail across the Atlantic, using the Chinese compass. Their aim was to establish a new trade route to India. Columbus and his sailors never made it to India. They landed on the Bahamas, and then Cuba. Columbus returned to Spain to inform the Spanish Queen of his discovery and the prospects for futures riches. He made three more trips to Caribbean Islands, reaching the mainland of the New World only on his last voyage. Columbus believed that he had reached Asia.

It took another Italian explorer, A. Vespucci, to realize Columbus had reached a new continent. The printing press was introduced to Europe by German inventor Johannes Gutenberg in 1440 CE. In 1507, German cartographer Martin Waldseemüller published the first map of the New World, based on Vespucci’s journals. He designated the new continent with Amerigo Vespucci’s name. In the following decades the name Amerigo took hold as America. In late 16th century, the Americas were mostly uncharted territory for European explorers. The explorations continued well into the 19th century through many cycles of great expectations, followed by setbacks.

After a number of major breakthroughs, AI scientists probably know as much about the brain today as the first explorers knew about the Americas back in the early 16th century. The road to human-level Artificial Intelligence will be a long journey of great advances, setbacks, unintended negative consequences, accidental discoveries, mathematical theories leading to technological breakthroughs, and great opportunities for doing away with repetitious labor across a vast number of domains.

AI has evolved as an amalgam of layers from different fields of knowledge, different levels of abstraction of various aspects of the brain. There have been mainly two general approaches in Artificial Intelligence.

I- Symbolic AI theories see the mind mainly as an inference or symbolic AI machine. Following the paradigm of classical programming, symbolic AI machines implemented rules of logical reasoning to process a certain knowledge of the world. In rule-based machine translation one coded all of the grammatical rules of a given language and the entire body of vocabulary in that language, cross correlating that with those of another language. For instance, to translate a sentence from English to German, the user fed the given sentence to the machine. The machine parsed through all the relevant grammar rules and vocabulary and compiled a corresponding sentence in German.

Symbolic AI made important breakthroughs. The expert systems that were first introduced in 1965 and fell out of favor in late 1980’s paved the way for modern business workflow engines. Modern computer chess game was also made possible through advances in symbolic AI. But symbolic AI machines were hard to program, unable to learn much on their own. Pure symbolic AI turned out to be inefficient for translation and many natural language processing (NLP) tasks. Every human language comes with grammatical rules and exceptions due to that language’s history of evolution. And at times the meaning of a word depends on the context: the word apple can refer to the fruit, to a computer company, to the city of New York (Big Apple), or the proverb ‘apples and oranges’ which if rendered literally in another language, may not convey the same meaning.

II- Connectionism (aka Machine Learning) attempts to explain mental phenomena through a concept of neural networks. Inspired by inter-connections of neurons in the brain, here the focus is on enabling machines to learn from the ground up, by re-purposing massive amounts of data. This approach is known as deep learning. In 1965, Soviet mathematician Alexey Ivakhnenko published the first general working learning algorithms for deep neural networks.[2] In 1971, another paper described an eight-layer-neural network.[3]

From a mathematical point of view a neural network is a well-structured set of thousands of matrix multiplications carried out in rapid succession, in a hierarchy of layered filters. Each consecutive layer of the network looks for a pattern in the preceding layer — and it is more important for these processes to be fast than exact. In Machine Learning, the user provides the input data and the corresponding output, and the neural network will form its own statistical rules to be applied to new batches of input data. Once a Machine Learning system has been deployed, it needs to be trained rather than programmed. Neural networks can learn from large datasets, by identifying patterns.

In medical applications, an AI machine need not be coded with a priori knowledge of what a tumor looks like. After processing a massive amount of photos tagged by humans, showing variations of what a given tumor looks like, the machine can recognize new instances of that tumor. The more training data you have, the better the results.

In spite of advances in 1980’s, AI was for a time mostly limited to the realm of research. Artificial Neural Networks (ANNs) finally overcame many of the problems faced by symbolic AI. So the connectionist approach gained support in the 1990s. Nuance Verifier (2001), one of the first ANN products, was based on research done at Stanford Research Institute (SRI).[4] It was used to authenticate a speaker’s voice in telephony-based speech recognition.

After a period of relative disillusion, the ship of AI gained new momentum due to availability of massive datasets and advances in graphics processing units (GPU’s) that made parallel processing more powerful, faster, and cheaper. Starting in 2012, the works of Jürgen Schmidhuber, Yann LeCun, Yoshua Bengio, Geoff Hinton, Andrew Ng, and others finally gave results. Papers by Ciresan et al. and Krizhevsky et al. showed how convolutional neural networks can improve vision benchmark records over other machine learning methods.[5]

Since 2015, the pace of innovation has accelerated. Deep neural networks have enabled breakthroughs in a large number of domains, including autonomous driving, image recognition (e.g. for medical diagnosis, business document processing), and machine translation.

[1] https://www.theverge.com/2019/3/27/18280665/ai-godfathers-turing-award-2018-yoshua-bengio-geoffrey-hinton-yann-lecun

[2] https://www.studmed.ru/ivahnenko-ag-lapa-vg-kiberneticheskie-predskazyvayuschie-ustroystva_302f07ded3f.html

[3] http://people.idsia.ch/~juergen/firstdeeplearner.html accessed on April 11, 2020

[4] https://www.microsoft.com/en-us/research/wp-content/uploads/2000/01/1-s2.0-S0167639399000771-main.pdf

[5] https://arxiv.org/abs/1202.2745 and https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf