March 2020 | Przegląd Zbożowo-Młynarski (Grain and Milling Review)
An introduction to the algorithm of convolutional neural networks
In 1805 Joseph Jacquard developed a method of embroidering patterns by means of punched cards. By means of software the machine was taught how it was to guide the threads so that the programmed pattern would come into being. At the same time the pianola came into being – an instrument resembling a piano, which plays musical pieces on the basis of perforated sheets of paper.
The next significant step of humanity on the road to artificial intelligence was the development of methods of optimisation and mathematical prediction. These methods spread during the Second World War, minimising losses and optimising the logistical processes of the Allied forces.
Every time we were dealing with a certain device which received a defined pattern or program and acted according to it. The machines worked on the principle of reaction to stimuli which were foreseen by the creators of those machines. Man created machines. The machines acted as their creators had designed them to.
W-MOSZCZYNSKI pzm 3-20This seemingly unshakeable participation of man in creation began to wobble with the appearance of infinite quantities of data and the necessity of processing them quickly. So much data appeared that the designers could not keep up with designing systems which would be able to process it in a predictable way. In the new situation the role of creative man became ineffective, because it turned out that machines can program themselves much more efficiently.
Thus the concept of artificial intelligence appeared. The average human being is thousands of times more intelligent than the most intelligent machine. A machine, through its simple analytical techniques repeated thousands of times, is able to search and predict far more accurately than the most intelligent human being. So new machines appeared which themselves learn new data, without human help.
Mathematical algorithms which learn by themselves have been known to humanity for a very long time. The first such algorithms existed as early as the 19th century. I have in mind the first linear regression equations. A machine’s artificial intelligence is something else. It is the ability to draw conclusions on the basis of completely new data. It is the creation of new patterns of behaviour from the combination of several other patterns. Every human being has the ability to cope in a completely new situation by making use of previous experience. Every intelligent organism, in the course of learning, makes use of elements of earlier experience. This ability is now being taken over by machines before our eyes.
The essence of evolution is change. In the past cars looked different, and still earlier there were no cars. People travelled by railway. Television is beginning to give way to the internet, and traditional spreadsheets are giving way to integrated BI-type systems. What is disturbing, however, is that this time it is we, people, who are being sidelined. Machines, despite their still meagre analytical abilities, are beginning to surpass us in intelligence.
Of course it is not a matter of Skynet appearing at some moment with its humanoid robots and beginning the extermination of humanity. It is a matter of people gradually becoming less efficient and more troublesome than machines. We already went through this lesson at the beginning of the 19th century, when modern steam machines were displacing individual producers. Then everything ended well, because as humanity we were the owners of the processes. Today processes are entrusted to machines which design the next machines. We as people increasingly often feel like tourists who do not understand what is happening in the factory they are visiting.
In February 1996 the IBM supercomputer named Deep Blue won at chess against the champion Garry Kasparov. The computer won because it probably knew all the combinations of moves and the course of all the chess games of the last century. The computer was programmed by people. So people won, even though it was the computer that was the victor. The computer in this case was not intelligent; it had everything given to it and behaved as its creators had foreseen.
In October 2015 the computer program AlphaGo, belonging to Google’s company DeepMind, defeated the professional European champion Fan Hui at go. In March 2016, in a match with one of the best professional players, Lee Sedol, the AlphaGo program won 4:1. Go is a game of which it is said that it has more combinations of moves than there are atoms in the observable universe. In this game there is no possibility of programming a ready-made algorithm which would win against a human being. So it is not enough to enter into the computer’s memory the history of all go games. This time the machine itself had to cope, improvising.
After defeating humanity at go, the DeepMind experts created a further neural network algorithm named AlphaGo Zero, which itself learned to play go by conducting millions of games against itself. Each time the algorithm improved something in its play. After not quite a year the new algorithm defeated the previous winner, the AlphaGo program. What is interesting is that a further variety of this algorithm, named Alpha Zero, learned to play three games at once: chess, go and Japanese chess. This program defeated the AlphaGo Zero algorithm at go.
It was noticed then that, similarly to a human being, when a neural network learns one game it then learns the next game much faster and is better at it. Similar abilities were noticed in the learning of foreign languages by neural networks. With every new language the network learned faster.
None of the researchers and futurologists supposed that artificial intelligence would overtake man so quickly.
Machine Learning and Deep Learning
There are several families of artificial intelligence models counted in the Machine Learning group. The first were regression models based on linear regression. Then models based on decision trees appeared, among which the most popular is the Random Forest family of models. Models from the Support Vector Machine family, based on the principle of dividing the population, are also quite commonly used. All these models, however, have a certain common limitation. They do not learn in a recurrent way, that is, they are unable to learn by processing their own calculations millions of times.
The systems which are able to do this are artificial neural networks. It is they that won at go. It is neural networks that find images in Google search engines and recognise voices on Facebook.
It should also be said that recently a new family of machine learning models has come into being, in which elements of the techniques applied by neural networks are used (CatBoost, LightGBM, XGBoost). Neural networks are models counted in the area of Deep Learning.
The history of artificial neural networks
The first neural network without feedback was built in 1943 by Warren McCulloch and Walter Pitts. The modern recurrent artificial neural network was invented in 1982 by the American physicist John Hopfield.
For decades artificial neural networks were treated in the categories of a scientific experiment, a mathematical curiosity without any specific application.
With the dynamic development of the internet, the world needed more intelligent ways of processing information without human supervision. The biggest players on the internet market, Google and Facebook, developed their own artificial intelligence systems. In November 2015 Google released the first library of artificial neural networks, named TensorFlow. Not long afterwards, in September 2017, Facebook, in cooperation with Microsoft, released the neural network library PyTorch.
Artificial neural networks came into being as an imitation of biological nervous systems. The brains of living creatures consist of millions of neurons arranged in a network of connections. Information from the external world flows into the neurons in the form of electrical charges. The neurons process this information, passing it on further to successive neurons. Creatures receive feedback about the effects of their actions and in this way learn how to act. Patterns of behaviour are passed on into the genetic code, as a result of which animals undergo evolutionary development.
Artificial neural networks work in a similar way. The individual neurons receive signals in the form of digital information, amplify them or weaken them and pass them on further to successive neurons. At the end the network receives feedback about the accuracy of its classifications, corrects its analytical processes and begins the analysis from the beginning.
In the next article I will explain step by step how a neural network works. Then we will carry out an exercise consisting in building a neural network whose task will be to predict events on the basis of input information.
Wojciech Moszczyński
Wojciech Moszczyński — graduate of the Department of Econometrics and Statistics of Nicolaus Copernicus University in Toruń; specialist in econometrics, finance, data science, and management accounting. He specializes in the optimization of production and logistics processes. He conducts research in the area of the development and application of artificial intelligence. For years he has been engaged in the popularization of machine learning and data science in business environments.
Dodaj komentarz