Mathematical Models in Data Science
Creating optimization and implementation of artificial intelligence solutions is primarily based on creating mathematical models.
Mathematical models are the simplification of reality based on historical data. These data intersect each other, are interrelated, interrelated.
Models use these dependencies, find the most common, recurring phenomena and create some kind of reality mockups,
the type of feedback of the probability-based behaviour.
An entrepreneur who plans to implement artificial intelligence solutions does not need to know about building mathematical models.
It is worth knowing, however, what models are, what their most important characteristics are, and above all, what conditions should exist for such models to be built.
W-MOSZCZYNSKI-2023-8-30Data needed to build mathematical models
Each company r has databases, which may include sales registers, stock records, customer registers or information related to the flow of raw materials.
These are internal bases. External databases, such as historical weather data, are also used to create models.
information on macroeconomic moods or historical developments of customer activity on the market.
We distinguish five basic types of databases that exist in almost every company that are used to build mathematical models. These are:
The five types are: CRM databases, register databases, stationary databases, financial databases, transaction (receipt) databases and process databases.
The CRM database is a record of cooperation with customers. It is a historical record of deadlines sent to offers, meetings, negotiation notes related to specific customers.
Databases are stationary. They contain permanent information about specific objects such as cubature, address or distance.
These databases may also contain fixed information about customers such as age, sex, business address or education.
They may also include address of retail stores, production capacity of certain machines or the marginal capacity of certain installations.
Financial databases are data collections containing information on revenue, costs and financial results obtained at specific cost generating sites (MPK).
These are the bases of companies that use management accounting solutions.
The transaction bases (also known as receipt bases) are, for example, sales registers at fiscal cash registers in retail stores, shipping registers and goods admissions
in warehouses. Such databases have a time series format, because their main element is the time of transaction.
They are excellent for setting trends, searching for relationships and links with external factors.
Process databases have a similar format of time series to transaction databases. These are time-based registers, so they are time series.
They shall reflect the process recorded according to specific physical or chemical phenomena occurring in production or transport installations.
Each complex enterprise has at least one of the five databases listed here.
As mentioned above, the mathematical madele is a simplification of reality. The first step in creating a model is to determine its purpose.
The most common models are created to predict the future, less often used to interpret current phenomena.
Suppose we're talking about predicting what the price of a tonne of flour will be in the market in two months.
You should collect information about the price level in the following weeks over the last three or four years.
that the price of flour was changing in repeated cycles, there were some trends, in some months the price of flour was lower,
In other countries, it increased rapidly by creating a periodic maxim. We can observe that for example, always the highest price of a particular raw material occurred in March and April.
This is the first observation, a simple generalization of reality that can be used businessly. We made it without the help of a model, just looking at the chart.
We know that the price of flour was influenced not only by the month, but also by the weather, currency prices, the overall economy, consumption levels and local import volumes of cereals.
Many phenomena had a more or less subtle effect on raw material prices, some phenomena had an impact on prices with a long delay.
Events such as export volumes, fertiliser costs and fuel prices are closely correlated to form complex combinations of dependency.
These links are expressed by the probability account, relationships are most often shifted over time, so they cannot be detected without statistical methods.
It is difficult for an average person without the help of data processing technology to link all the facts. To interpret complex data is a mathematical model.
Building a mathematical model
indicate which data, known as independent variables, will be of the character of the descriptive series and which data will be the resulting variables.
If we build a model predicting the price of flour in the future, the resulting variable will be the historical price of flour per month.
The aggregated data in the form of time series containing descriptive variables shall be compiled in the algorithm of the mathematical model with the resulting variables.
The model finds its own dependency and draws concrete conclusions.
As a result, on the basis of historical variables describing, there is a simplification of the reality resulting in a forecast of flour prices in the future.
Types of mathematical models
The basic breakdown of mathematical models used in data science techniques are classification models and regression models.
The classification models give the result in a so-called discrete format. It is an indication whether it will rain (the result marked as: 1) or not (the result marked as: 0).
For example, the classification model may predict which type of customers will react negatively to changes in sales.
Thus, the model can predict, for example, that the most spontaneous reactions from customers will occur in the 30-45-year-old group (the result marked as group: 0).
At the same time, the model will indicate that the most resilient group of customers will be a group of customers aged 18-25 (the result marked as group: 1).
The completely different nature of the information provided are regression models. They indicate exactly what the projected size will be.
So the model can say that in 10 days the price of the raw material may cost 175 PLN per tonne or that the temperature tomorrow at 12:00 will be 14.5 degrees Celsius.
How do models predict the future?
As I mentioned, models are based on generalizations created on the basis of historical data observations.
These data have certain sizes, their compilation creates some generalization leading to the result.
How can a model create a forecast of the future if we do not know what the level of variables describing in the future will be?
There are two solutions that can work simultaneously. The first way is to use trends.
The description variables may show some historical trends, so that they can interpolate their size in the future.
Based on historical behavior and trends, interpolating the size of the describing variables is possible.
If, therefore, the price of fuel, which is the reference variable, was highest in four years always in May and this price was, for example, 22% of the average price
in a year, it is with a relatively high probability that we can determine what the price of raw material will be in May this year.
But as we can easily see then we cannot rely solely on interpolation but we must run one of the simple models of ARMA or ARIMA mobile autoaggression.
It's a pretty embarrassing way, which is why you can use another method.
The question was how to use a model that already contains variables describing when we do not know the level of these variables describing.
The proven method is the use of ready-made forecasts for describing variables that are created by other specialized models.
Therefore, if we have information on the expected prices of raw materials, the level of exports,
We can build a mechanism for the price of poppy in the next two weeks based on these forecasts.
Learning with supervision and modeling
When describing the most important data science modelling techniques, we need to mention what supervised learning is.
It's a simple method, which is that we usually use 70% of the data we have to train the model.
The remaining 30% of the data are used to test the quality of the forecasts created.
In the case of our model forecasting the price of flour, suppose we have 36 months of historical data. We use 30 months to train the model.
The other six are used to test the model.
The testing of forecasts consists in checking how precisely the model predicts prices after substitution of the variables describing from the said six months.
The model creates forecasts, and we know what the prices have been in these last six months we can compare if the forecasts differed from the actual prices.
This is what we call model validation.
Thanks to the validation of the model, we can find out whether the models provide for the correct way, but whether further work is needed to improve its effectiveness,
Conclusion
Modeling is the most important technique of creating solutions from the area of artificial intelligence. Models are possible primarily based on historical data.
Mathematical models are a simplification of reality, it is important that this simplification does not affect somehow the forecasts or interpretations of phenomena.
who has commissioned the development of mathematical models must be aware that the final accuracy of the models' forecasts is mainly influenced by data quality, reliability and completeness.
Therefore, the quality of data collection and storage is important for the future of the company.
Wojciech Moszczyński — graduate of the Department of Econometrics and Statistics of Nicolaus Copernicus University in Toruń; specialist in econometrics, finance, data science, and management accounting. He specializes in the optimization of production and logistics processes. He conducts research in the area of the development and application of artificial intelligence. For years he has been engaged in the popularization of machine learning and data science in business environments.
Dodaj komentarz