February 2022 | Przegląd Zbożowo-Młynarski (Grain and Milling Review)
DATA SCIENCE IN THE MILLING INDUSTRY
Data science is a field of study whose goal is to obtain as many benefits as possible from data. Models are able to predict a drought and a market collapse, they are able to analyse the structure of sales and find effective logistical channels. By means of modern techniques, threats can be identified and frauds unmasked. It is worth asking, however, what has really changed with the appearance of data science? Mathematical models for forecasting and optimisation existed, after all, before. There was, however, less data before, and it was harder to obtain. Computers were slower and harder to operate. The undoubted development of computer science caused a growth in the quantity of data. Cheap computers and new software contributed to the popularisation of modelling and data processing. Despite everything, what we observe at present already existed before, only on a smaller scale and in a more primitive form. It is as if we were saying that 50 years ago there were fewer cars and fewer roads, and now that has changed, we have new cars, a lot of roads and safety standards. However, we are far from flying our own aerial vehicles.
If, however, we stopped at this comparison, we could say in the context of data science that once there were few cars, and today, instead of driving cars, we all fly in vehicles piloted by autonomous computer pilots.
So progress has indeed appeared in data science, not only in the area of quantity and prevalence — that is, simple evolutionary modernisation. As humanity we have stood at the threshold of a very steep staircase leading upward. We do not know where it will take us; soon nothing will be as it was before.
W-MOSZCZYNSKI pzm 2-22Man versus machine
The first model to win a chess game against the world champion Garry Kasparov was the algorithm installed in the IBM supercomputer Deep Blue. This happened on 10 February 1996. This game became known as the computer’s victory over man, even though Kasparov won three subsequent games and drew two, ultimately beating Deep Blue by a score of 4:2.¹ The number of possible chess combinations is called the Shannon number and amounts to 10¹²⁰. Claude Shannon derived this number to prove that a computer is not able to analyse every possible move, because calculating one move variant per microsecond „would take it until the end of the universe to finish the calculations”.² In building the distributed Deep Blue system, the method of event probability described in Markov chains was applied. On the basis of the current situation, the computer eliminated all impossible arrangements and drew historical chess arrangements from the past as concepts, which it then chose. In a chess game, the first move (the so-called opening) is limited to a dozen or so possibilities, the next move gives further limited possibilities. The model was therefore based on decision trees based on historical games. Thanks to this, a significant limitation of variants was gained, which is why the system played chess relatively fluently.
A game which until then had seemed impossible to program was the Chinese game of Go. In the case of Go it is the opposite of chess — the first move has several hundred possibilities (19 × 19 = 361). During the game the number of possibilities falls until the moment of the first capture — then it rises again. It is therefore to some degree impossible to make use of earlier experience, and the number of move variants grows exponentially.
There is a well-known maxim: „If someone thinks that chess is the king of games, then Go is its emperor.”³ Until recently it was thought impossible to create a Go algorithm which would be able to defeat grandmasters of this game.
In 2014 Mark Zuckerberg, the creator of Facebook, mentioned that he had hired a special team of scientists engaged in training AI to play Go. Google’s management sent similar signals. Rivalry began. Both companies applied an innovative method of machine learning called „reinforcement learning”. Deep neural networks began playing Go against each other for months. Every game, lasting a few minutes, slowly improved the machines’ abilities.
Google invited the three-time European Champion (2013, 2014, 2015) Fan Hui, who has been perfecting his Go since the age of 12, to play against the AlphaGo system. In October 2015 he lost to AlphaGo by a score of 0:5. Fan Hui was hugely impressed. He stated: „a very strong and stable program. If nobody had told me AlphaGo was a computer, I would think it was a slightly strange, but very strong player.”⁴
Models and their simplicity
It is hard to call a mathematical model artificial intelligence. The simplest model is the moving weighted average. The result of such a model is some generalisation, an average value for a certain period. Using such a model we can find out what the probable value (average level of prices, precipitation, births) will be at a defined point in the future. Basing ourselves on the probability distributions of time series, we can predict various phenomena, their scale and trend.
With time the first regression models appeared, which were a matrix expansion of simple models based on averages. Then models based on the phenomenon of entropy appeared, having the ability to detect and make use of the phenomenon of ordering chaos in a closed system.
The real breakthrough was the invention of neural networks. These are models imitating the thought processes taking place in the brains of living creatures. There was a very long stagnation in the development of these models. After the invention of the single- and double-layer network, researchers were not able to take the next step. The first neural networks worked like a filter in a vacuum cleaner. Information flew through their neurons, distorted by weights. By increasing and decreasing the importance of particular pieces of information, neural networks delivered proper answers, correct results. However, they did not differ much from the simplest linear regression models. Just as air blown into a church organ turned into a beautiful sound. We must not, however, know that a church organ is a very complicated and sophisticated instrument. The mechanism of springs, pipes and cogwheels worked as it had before, but now all of this took on a digital form.
The phenomenon of recursion
Someone came up with the idea of applying feedback to the neural network, called recursion (from Latin recurrere — to run back). Information blown into the organ, instead of turning into sound on the other side of the pipe, began to circulate in a finite loop. In the model something appeared that began to change its character from the inside. It was as if organ makers had a mechanical device which changed the construction of the pipes from the inside, so that after longer learning they would produce the sound expected by the composer. I know it sounds absurd, and that was only the beginning of the „signs”.
So how does a recurrent neural network work? It consists of many layers of neurons, among which the inner layers have a built-in system of self-correcting errors. Information is processed repeatedly until satisfactory results are obtained at the output. This phenomenon resembles learning, but it is something more. It is a process which we know from evolution. It is a kind of inter-generational learning. Successive generations learn on the basis of the knowledge and experience of their predecessors.
Convolutional neural network
It would seem that something which can learn from its own mistakes infinitely would reach towering cognitive abilities. Such abilities were indeed achieved and, curiously, turned out to be the greatest curse of recurrent neural networks. It turned out, namely, that these models, learning in finite loops from training data, reached an almost unimaginable level of accuracy in imitating processes. Unfortunately, when the models were given new, previously unknown data, they became helpless, unable to cope at all with interpreting it.
It turned out that training a recurrent network without proper configuration leads to a lack of generalisation of the phenomenon. Instead of looking for patterns, interdependencies and connections, the model learned a large part of the data by heart. This part of the data was noise — that is, random data unconnected with the process.
The phenomenon described here is called overfitting and constitutes the fundamental problem in working with advanced deep learning and machine learning models. Over time researchers learned to cope with this phenomenon; unfortunately building complex neural networks is not an easy skill. It is a very laborious process which is hard to automate. It requires high qualifications and experience. Business cannot wait for rare experts; modelling is needed urgently. Innovative research in the area of deep learning consumes a lot of time and resources.
Ensemble Learning
The idea of recurrent neural networks initiated a new way of thinking about the application of models from the area of machine learning. This area is somewhat simpler and more practical than deep learning. Until now, most models worked on the principle of an algorithm which was launched and, in a one-off manner, tried to learn, find dependencies, understand and finally create an equation describing the process being studied.
An important step was the idea of ensemble learning. It was decided to apply very many identical models, which would learn the same data simultaneously, in order finally to gather, summarise and obtain a common average result. According to the basic principle of ensemble learning, one model, even the best one, will always be worse than a crowd of weak models. For the group result of several hundred identical models to be good, the models must differ from one another. Since identical models cannot differ from one another, different sets of randomly selected data were applied for these models. This method is called bagging. Explanatory variables also began to be selected randomly (feature randomness). In this way, within a single model called random forest, a population of an enormous number of decision trees arose, differing from one another in the information they possessed. Random forest models gained enormous popularity, often surpassing the forecasting abilities of other models.
Boosting Machine Learning — models with reinforcement
Humanity has achieved its knowledge and technological advancement thanks to generations of researchers and scientists. Thanks to the ability to write, successive generations of scientists did not have to invent everything anew. Researchers could make use of knowledge, draw conclusions from the successes and failures suffered by earlier generations.
The ensemble learning method applied in the random forest model cited earlier more closely resembles an electoral process. A great number of citizens similar to one another, each of whom has their own randomly collected information, casts a vote, and as a result of the joint voting a result arises. The solution which has gained a greater number of votes wins.⁵
Have electoral processes contributed to scientific and technical development? As I mentioned, neural networks were invented, which, in an infinite process of internal learning, reached very high levels of accuracy in interpreting processes. Someone decided to apply this same mechanism to the decision tree model. Instead of grouping a great number of decision trees in a single row and requiring a simultaneous answer from them, a single decision tree was created, which, after carrying out the learning process, assessed its own learning, and then created the next tree, which learned from the mistakes of its predecessor. The achievements of successive trees are analysed by a loss function — that is, practically the same mechanism as that used by the convolutional neural networks described earlier. A model arose which considerably surpassed previous solutions, contributing to a further great step in the development of artificial intelligence.
Does artificial intelligence already exist?
At present, artificial intelligence is developing precisely in the direction of convolutional and recurrent neural networks and in the area of models based on reinforcement learning. This is an intuitive solution based on the evolutionary, inter-generational character of scientific progress. What will happen next? Undoubtedly these methods are only a milestone on the road to the creation of autonomous, intelligent artificial beings. There are still many currently known but unused directions of development. These include, among others, methods combining operational programming and reinforcement learning within modular neural networks. Quantum computer technology is also being developed, on the basis of which quantum models with extraordinary cognitive abilities may arise.
If a genuine, replicable artificial intelligence comes into being, we will not wait decades for breakthrough inventions, but days or even hours. It is thought that the appearance of artificial superintelligence will lead to a technological development which cannot even be imagined now.
We are at present witnesses to a process in which competing super-algorithms play endless games with one another, learn to identify images and behaviours, and find improbably unique phenomena. The cognitive abilities of the human brain have remained far behind specialised analytical systems. These systems are too clumsy to be able to change their specialisations into universal skills. In other words, they are far from becoming as versatile as humans, or more broadly, animals. It is probably only a matter of time; their development continues.
Wojciech Moszczyński
1. Wikipedia, the Free Encyclopedia, https://pl.wikipedia.org/wiki/Deep_Blue
2. Szachy (home.agh.edu.pl) https://home.agh.edu.pl/~zobmat/2019/2_nagroda_2/math.html
3. „Another bastion where man reigned has fallen — a computer has won at go”; Hubert Taler, 29 January 2016; https://spidersweb.pl/2016/01/komputer-wygral-w-go.html
4. „A computer defeated the Go master (and this is not the end)”, Wojciech Kulik, www.benchmark.pl
5. It should be pointed out that random forest models are not intended solely for classification tasks, but also perform excellently in the regression process.
Wojciech Moszczyński — graduate of the Department of Econometrics and Statistics of Nicolaus Copernicus University in Toruń; specialist in econometrics, finance, data science, and management accounting. He specializes in the optimization of production and logistics processes. He conducts research in the area of the development and application of artificial intelligence. For years he has been engaged in the popularization of machine learning and data science in business environments.
Dodaj komentarz