Artykuł A Mini Recommendation System for a Customer Service Centre pochodzi z serwisu THE DATA SCIENCE LIBRARY.

Ta treść jest chroniona hasłem. Aby ją wyświetlić, wpisz poniżej swoje hasło:

Hasło:

Artykuł Zabezpieczone: szkolenie Maksa pochodzi z serwisu THE DATA SCIENCE LIBRARY.

Artykuł Practical notes on applying OCEAN/HEXACO psychological profiling in recommender systems for selling luxury and investment goods (Part 2) pochodzi z serwisu THE DATA SCIENCE LIBRARY.

Artykuł Strategy as the Choice of a Specific Direction of Development DEA-CCR pochodzi z serwisu THE DATA SCIENCE LIBRARY.

Artykuł Factorization Machines – how a baker can build an effective recommendation system based on client log data pochodzi z serwisu THE DATA SCIENCE LIBRARY.

Low Digital Integration

In most cases our businesses are well saturated with digital systems. Phones contain call histories, integrated accounting systems communicate with warehouses. Those in turn communicate with logistics systems that issue or receive goods. Generally speaking, there is some integration of systems, but it is neither coherent nor uniform. For example, our mobile phone is not integrated with the ERP system, though it could be. Cars do not have onboard computers integrated with the main computer responsible for the enterprise’s logistics, though such correspondence between systems could exist. Grain-milling devices or loading scales are usually independent of warehouse systems. To connect all this we need a uniform nervous system—i.e., a network of digital connections enveloping the entire organization. And that will not exist for some time yet, because it would require close cooperation among various vendors, manufacturers of machines and measuring devices, and software—according to some agreed standard. And there is no sign of that for now. I mean a standard like what we have at Google, where a mobile phone, a computer, and a TV can operate under one identity and all of these devices can save history, data, and all kinds of information to a shared database in Google’s cloud. That solution is, let’s call it, a civil solution. It is hard, however, to find similar solutions in the professional world. The solutions supplied for warehouses, transport, machines, and systems are not integrated with one another—and won’t be for a long time.

W-Moszczynski (4)

Internet Access

Systems will be able to be integrated freely with each other when all these devices have internet access. Why is that so important? Because internet APIs make it possible to integrate individual elements of a system with clouds. Clouds are not only for data storage and simple calculations. Practically all analytical tools and methods can be found in the clouds. Their use will soon be necessary so that our business does not lose the competitive race.

In the grain-milling industry, a very important element is price speculation at the point of purchase and sale of grain and flour. In other words, analysis of the commodities exchange, macroeconomic assessments, agricultural market analysis, historical weather forecasts, and unforeseen weather phenomena must be carefully analyzed. Creating more advanced forecasting models requires hiring a very well-educated data analyst who should additionally be a practitioner. Such an expert is quite costly and certainly will not be physically available nearby, so they will work remotely. Since we have no way to supervise them and we are unable to evaluate their work substantively, it would be advisable to cooperate with a company specializing in forecasts—and here the difficult process begins again: finding such a company and establishing cooperation with it.

In the end, we would have to pay more than for a single data analyst, and we still would not be able to control their work.

The Cloud

The cloud differs from a data analyst in that it is predictable and it’s rather hard to expect that it will cheat us on working hours or go on sick leave. The cloud also has a kind of professional experience, because its systems are based on large language models like ChatGPT. In other words, cooperation with the cloud reduces most of the problems associated with cooperation with a human. First: using the cloud costs a fraction of an analyst’s price. Second: if we start treating the cloud like a very advanced calculator, we ourselves can become experts in forecasts and building forecasting models. In the cloud we can optimize warehouse size, analyze the movement of the flour-truck fleet, or check the efficiency of human resource utilization in production; we can compare everything, draw conclusions, and make decisions based on well-processed data.

Artificial Intelligence Will Destroy the Human Race, for Sure!

There has long been a discussion about what digitization actually is. What is this famous artificial intelligence that is supposed to enter our lives and turn us into not-very-bright people for whom machines do the thinking? Is there anything to fear? Usually after such an introduction, some laconic deliberations begin—more or less successful predictions of the future, scaremongering or raising hopes. Whenever something new and groundbreaking appears, there is no shortage of panic-mongers and critics. From today’s perspective, people’s statements from 100 years ago seem laughable.

So as not to speak without examples, I will cite a few instances of how people reacted to the emergence of the railways.

“Traveling at such a speed is impossible and will certainly cause illness.” — Dr. Dionysius Lardner, professor at the University of London, 1830. Back then, “high speed” was considered 30–40 km/h.
“The railway will be the end of civilization. It will destroy family, religion, and morals.” — The Quarterly Review (a conservative British periodical), ca. 1825.
“Traveling at such a speed can cause madness. People are not made for such a pace.” — an anonymous physician quoted in British newspapers in the 1830s. Many doctors warned that the rapid movement of the landscape could lead to disorientation or even insanity. They spoke of dizzying speeds on the order of 40 km/h.
“When people travel at 15 miles per hour, the blood will boil in their veins and they will die.” — Thomas Tredgold, engineer and early railway author, 1825.
“Horses will be useless. Many professions will collapse. This is a threat to the social order.” — a statement by a representative of the London cab-drivers’ guild, 1830s–1840s.

In other words, the fear—then as now—of job losses.

We have plenty of similar statements regarding the introduction of airplanes, electricity, and telephones. How could telephones destroy the human race?

“Telephone conversations will be demoralizing—women will speak with men they do not know!” — letter to the editor of the New York Times, 1880.
“This device disturbs the peace of the home and destroys family bonds.” — a pastor from Massachusetts, 1881.
“The telephone is too complicated to be useful to ordinary people.” — Western Union internal memo, 1876.
“Who would want to talk to someone they cannot see?” — comment attributed to a London aristocrat, late 19th century.

And electricity? How frightening it was!

“If we allow the widespread use of electricity, people will die by the hundreds. Alternating current is murder.” — ca. 1889.
“Electric lamps will blind people and ruin their eyesight. Gas light is gentler and safer.” — an anonymous physician quoted in The Times, London, ca. 1882.

Let’s imagine that we are a person who has never seen a railway. We have read various Cassandra-like statements spread by experts. How can we be convinced there’s nothing to fear? A newspaper will not handle it, in which other, better experts will praise this invention. You simply have to take a ride on that “terrible” railway. I am sure that everyone who rode the railway racing at the dizzying speed of 40 km/h understood how nice it is to get somewhere far without effort and at breathtaking speed.

Now let us take a ride on the “railway of our time.” In the end, we will understand what our weakness is and why this truth can set us free.

A Grain Purchase Price Forecast Made by Artificial Intelligence

Today everyone knows artificial intelligence that you can chat with—ask who invented the flush toilet or have it write an article for us or draw a picture. It’s like an expanded Internet, only you can do everything faster and better. Along the way, professions such as graphic designer, editor, or language teacher disappear from the market. However, ChatGPT in this version is no great breakthrough.

Real business needs very concrete things done massively at large scale. For example: analysis of surveillance video, processing gigantic amounts of data and creating forecasting systems that predict events, autonomous vehicle control, or organizing huge numbers of scanned documents. So let’s do one such example—not to learn how to do it, but to “take a ride on the railway.”

To build (in 15 minutes) a grain purchase price forecasting model, you need to click around a bit in the cloud.

It Was Not Always This Easy

In the 1990s, spreadsheets appeared with an option to create trends. Back then, building a forecast took a moment; unfortunately, they were unreliable and weak. In the second decade of the 21st century, the possibility of programming in Python and R appeared. Libraries emerged that enabled the creation of forecasting models. They were simply ready-made instructions to execute. You didn’t have to write the entire model algorithm from scratch.

In practice, when someone wanted a forecasting model for their sizeable company (e.g., a grain elevator) placed in the cloud, they hired a team of experts—data analysts (a “project team”)—who would spend months creating such a model. This is still how most such undertakings are carried out. I think companies implementing these projects deliberately do it the old way, because it’s profitable. Sometimes it’s not worth enlightening the client.

Welcome Aboard the Railway

Below are a few slides showing how this process looks in practice. I included the slides not so that everyone learns how to do it, but to make it clear that building a forecasting model is a matter of a dozen or so clicks in the cloud. For the experiment I chose GCP (Google Cloud Platform), one of the most popular on the market. Using this cloud is free; anyone can log in. After exceeding certain limits, the cloud starts charging fees—very low ones anyway.

Step 1. Log in to the cloud, create a project. In the slide this is the project “Wojtek1.” Choose the “BigQuery” tab.

Step 2. Upload data. The cloud needs them to create a model. Pasting the data is another few clicks.

Our data table contains 48 rows and looks like this:

We see a column with years and months; we have three grain elevator locations: Miłkowo, Połoć, and Stare Wolice. We added columns with temperature and precipitation levels. We have to paste this table into the cloud.

Step 3. Prepare the query.
This step may seem difficult if someone cannot write SQL. Currently the cloud helps us with artificial intelligence. I will not describe how to talk to it and ask for help. Artificial intelligence writes the query for us.

We press Enter—and we have a model.

We get a beautiful, hopeless model that is useless.

The Role of the Human in the Process

With this example, I wanted to show that using artificial intelligence relieves us of tedious work while simultaneously forcing us to make an intellectual effort. And that is the new quality. We no longer have to program sluggishly, create queries, search libraries, validate, and generally toil. New times require us to think, and we cannot excuse ourselves by saying we are not mathematicians or professional data analysts. Now everyone will have to think!

So what model did we get?

R-squared (R²) = 0.0979 — a very low value → it means the model explains only ~9.8

So we ask artificial intelligence what we can improve:

1. Add important features (feature engineering):
   • e.g., baseline_monthly_trend, _in_harvest_peak, rainy_days
   • encode base as a categorical feature (one-hot encoding)

2. Use a more advanced model (e.g., tree-based):
   • BigQuery ML is simple linear regression;
   • In Vertex AI you can use XGBoost/AutoML Tables, which handle non-linearities better.

3. More data:
   • one year is too little—we need at least 5–10 years of history;
   • data on inventories, acreage, previous yields, etc.

4. Check data quality:
   • whether there are random errors;
   • whether variable scales are not dominating the model.

What have we learned so far?

First, the machine did all the work for us—and the result turned out to be hopeless. If we had done it “by hand,” it would have been just as bad, only it would have taken longer. The machine told us what to change.

What conclusions follow?

First: anyone can make (click together) a model.
Second: the machine will not clean up our company for us.
Third: from this example, we learned what is important and what is not in the model—that is, we learned something. This last conclusion is quite significant, because as passive observers of the classic path of model creation, lacking experience, we truly would not have learned anything.

What does the machine advise? “Point 1: Add important features (feature engineering).” The idea is that we, experienced millers or brokers on the purchase price market, must devise the features that we should place in our database. This could be the inflation rate or fuel prices, or a consumer optimism index. We can introduce any variable. The variables we introduced in the form of precipitation and temperature turned out to be irrelevant.

I would rather ignore point two proposed by the machine. It doesn’t matter which model you have—if the error is that large in scale, the data are to blame, not the model.

In points three and four, the machine tells us directly that we have a mess in the data. The data may be untrue, falsified, or simply entered incorrectly.

What, specifically, might be messy?

For example, transaction entry dates may be delayed; there may appear the phenomenon typical for high warehouse turnover—“debt shifting.” This consists in the warehouse manipulating volume over time or between warehouses, e.g., to avoid paying VAT. There are many other, non-tax factors that will render our data useless. In our example, the database contained a column with average temperature and average precipitation. These data were recorded at the moment of grain intake. Temperature and precipitation primarily concern the harvest of the grain, not the moment when the purchase took place. This is a typical error made by data analysts. The phenomenon is called shift—i.e., a displacement.

A Forceful Conclusion

Did the arrival of the car make people worse and weaker, did they lose the ability to walk? No—people became more technologically advanced. They also acquired many new, astonishing capabilities in carrying heavy things and moving through space. A typical car user, wanting to take advantage of these new superpowers, had to learn to drive a car, had to understand what the oil level is, what coolant is, or how to change a tire. If the car had not appeared, people would still be walking—significantly more slowly.

Today something has appeared again—some astonishing technology. And again we must take a step toward our development. The lack of IT education or the lack of programming skills will not excuse our lack of thinking this time. We must prepare the data; we must understand the business; we must simply start thinking. The example I cited today, relatively simple and intuitive, showed how the machine exposes our weaknesses, our laziness, and our lack of understanding of the business. The data we provided and which seemed decent turned out to be useless.

Artykuł Creating a Grain Purchase Price Forecasting Model, or How the Machine Showed Us Who We Really Are pochodzi z serwisu THE DATA SCIENCE LIBRARY.

One Promising Method: Isolation k-means

One promising method is Isolation k-means — an approach that combines the advantages of classical k-means clustering with the isolation mechanism characteristic of anomaly detection techniques such as Isolation Forest.

Isolation k-means is a hybrid data analysis method based on two fundamental assumptions.
First, it uses the k-means algorithm to group data into clusters — identifying natural structures and recurring patterns within a dataset.
Second, it analyzes the distance between data points and the centers (centroids) of their assigned clusters — assuming that data points lying far from a cluster center are potential anomalies.

Moreover, the algorithm can apply additional isolation criteria to separate points that are not only distant but also rarely co-occur with other observations.

This method becomes particularly valuable in the food industry, where detecting irregularities in real time can prevent serious problems such as product contamination, improper temperature conditions, or incorrect ingredient proportions. Thanks to its flexibility and ability to operate on unlabeled data, Isolation k-means can be effectively used to monitor production parameters, identify deviations from standard process profiles, and support decision-making by operators and quality engineers.

The following sections describe how the Isolation k-means method can be adapted to time-series data in food production and what benefits its application may bring to real-world quality control systems.

Unsupervised Learning

In data science and machine learning, we distinguish between supervised and unsupervised learning.
In supervised learning, the machine is guided by a human, whereas in unsupervised learning it must learn everything on its own.

Unsupervised learning creates the ability to work with unlabeled data. This means that the algorithm does not need previously prepared labels — information indicating which data are correct (normal) and which are faulty (anomalies).

For example, in classical supervised learning, the data must be labeled beforehand: a temperature of 60 °C is proper, while exceeding 95 °C is too high.

In practice — especially in food production — we often do not have such critical thresholds. It is difficult to pre-define which cases are anomalies, particularly when the production line frequently changes its assortment, leading to very short production batches.

That is precisely why algorithms operating on unlabeled data, that is, according to the principle of unsupervised learning, are flexible and well suited to this type of production environment. These algorithms learn patterns directly from the data, independently identifying what is “typical” and what deviates significantly from it.

W-MOSZCZYNSKI ps 5-25

The k-means Clustering Method

k-means clustering is one of the most popular exploratory data analysis methods in machine learning.
It is the most frequently used clustering technique — grouping objects based on features — for example, in customer behavior analysis, image segmentation, genetic data exploration, or anomaly detection.

It is a myth that clustering can only be done in two-dimensional (2D) or three-dimensional (3D) space. In practice, data often have a dozen, dozens, or even hundreds of features (e.g., sensor data from a production line), and k-means still works — just without visualization, since analysis takes place in multidimensional space.

Why mention this?
Because research is often conducted for a client who usually needs to see the division — which leads to generating cluster plots. When the studied phenomenon has, for example, 14 features, techniques such as PCA (Principal Component Analysis) or nonlinear methods like t-SNE (t-distributed Stochastic Neighbor Embedding) are used to reduce those 14 features to 2–3 key ones.

In production practice, using k-means clustering might look as follows:
We have a sample of meat with slightly lower temperature and 23 other measured features. These features place it in Cluster A. Another sample, with different values of those 24 features, falls into Cluster B. We also have Clusters C, D, and E.

In practical terms, this clustering does not necessarily mean anything — it simply reflects differences between products on the production line. Products can be assigned to clusters due to minimal variations in feature magnitudes. The process may be conducted for purely technical reasons.

The k-means method belongs to the group of unsupervised learning algorithms, meaning it does not require pre-labeled data. The algorithm independently finds structures (clusters) within the dataset.

The main goal of k-means is to divide a dataset into k non-overlapping groups (clusters) so that points within the same cluster are as similar as possible to each other and as different as possible from points in other clusters (e.g., groups of meat samples).

The quality of clustering is measured using the Sum of Squared Errors (SSE), also known as Within-Cluster Sum of Squares (WCSS) — the sum of squared distances between each point and its cluster centroid. The smaller the SSE, the tighter the clusters — meaning the points are closer to their centers.

In practice, this means that meat samples from one production batch can be significantly distinguished from each other. This can be useful for analyzing events related to, for example, bacterial contamination or harmful impurities in products.

The Isolation k-means Anomaly Detection Method

Isolation k-means combines two approaches:

• clustering (k-means) for identifying typical data patterns, and

• isolation of anomalies similar to methods such as Isolation Forest, focusing on identifying points that significantly deviate from “normal” groups (clusters).

Initially, data are grouped using the classical k-means algorithm. Centroids — cluster centers representing the most frequent patterns — are created.

For each data point, the distance from its assigned centroid is calculated. The greater the distance, the less the point fits its cluster.

For example, a quality-monitoring system on a packaging line may detect five different clusters of sausages. It’s still the same product (one SKU), but the system has found subgroups. Isolation k-means evaluates how isolated each point is from others. Points very distant from any subgroup are treated as potential anomalies.

Although this is an unsupervised method, it is possible to adjust anomaly detection sensitivity, for instance by setting a 95th percentile distance threshold, above which points are considered anomalies, or by using a normalized distance scale such as Z-score.

How to Prepare Data for Isolation k-means

Because the method is based on k-means, data must be properly prepared — just as in classical clustering.

Feature standardization (scaling): all features should have similar scales (e.g., mean = 0, standard deviation = 1). Otherwise, features with larger numerical values will dominate the distance metric. Alternatively, normalization can be used to transform feature values into a defined range (e.g., −1 to 1).

Mathematical models usually interpret numerically large values as more significant than smaller ones. For instance, in the case of meat, temperature of 4 °C would be considered less important than humidity of 62

Feature selection: choose variables that meaningfully represent process “normality.”
In food production, these may include temperature, production time, concentration levels, pressure, and humidity.
In industrial environments, many parameters (e.g., motor power or water usage) may have no relation to product quality and can unnecessarily clutter analysis.

Use of historical data:
The dataset should contain as much data as possible so that k-means can learn typical clusters.
In streaming (real-time) measurements, sample windows should include at least 100 products.

Food-production sensor data usually take the form of time series. They can be transformed into feature vectors using rolling windows, moving averages, standard deviations, etc.

Unlike classical k-means clustering, which assigns every point (product) to some cluster, Isolation k-means allows that certain points may not belong to any cluster — they are then interpreted as anomalies.

Practical Example: Isolation k-means in a Dairy

We have a dairy production line manufacturing yogurt. The process is strictly controlled and includes several key technological parameters that must remain within defined ranges to ensure safety and final product quality.

We measure six key features:

1. Milk pasteurization temperature (°C)

2. Fermentation time (minutes)

3. Fat content (

4. Protein content (

5. Final product pH

6. Number of live bacterial cultures (log CFU/ml)

Data from sensors and analyzers are collected in real time in transactional form. Each batch of 40 yogurts has a unique production number.
Each batch can thus be represented as a point in a six-dimensional feature space.
We do not analyze each feature separately, but treat them as one coherent observation — a complete process profile for that batch.

The k-means algorithm groups many earlier correct batches into clusters representing typical parameter configurations. For each cluster, a centroid is created.

When a new batch is produced, its profile (six features) is compared to the centroids.
If it lies significantly far from the nearest cluster (e.g., very short fermentation time combined with high pH and low bacterial culture count), it is treated as a potential anomaly.

In streaming mode, data are analyzed continuously.
Thanks to the ability to quickly compute distances to centroids, Isolation k-means can immediately identify batches with unusual profiles — before the product reaches packaging or distribution.
When combined with an alert system, operators can react instantly — stopping the line or conducting additional laboratory tests.

Advantages of the Isolation k-means Method

• Considers the full feature profile, not just individual limits.
  Many issues arise from combinations of parameters that individually fall within limits but together form an unnatural case.

• No need for predefined thresholds or quality standards, which is especially useful for short production series. Production anomalies are rare and difficult to define in advance.

• Simplicity and speed.
  Isolation k-means is relatively simple and efficient, allowing real-time use in quality monitoring systems.

• Robustness to scale differences.
  After appropriate feature standardization or normalization, the algorithm operates stably even with varying measurement units.

Summary

This article demonstrated how the Isolation k-means method can be successfully applied to anomaly detection in food production processes.
Combining classical clustering with isolation analysis allows for a more effective quality assessment approach than traditional threshold-based methods.

By analyzing complete process feature profiles rather than individual values, the method can identify subtle yet potentially critical deviations.
Its flexibility, lack of dependence on predefined limits, and real-time applicability make it particularly attractive for production environments where reliability and rapid response are crucial.

In the era of digital transformation in the food industry, methods like Isolation k-means may become the foundation of intelligent quality control and predictive maintenance systems — supporting consumer safety and process efficiency.

Artykuł Application of the Isolation k-means Method in Quality Control of Food Production pochodzi z serwisu THE DATA SCIENCE LIBRARY.

Cyberattack? What does that even mean? It doesn’t concern me.
I’m a calm pastry chef in a small town where everyone knows me. Cyberattacks are about banks, advertising agencies, or corporations in big cities. Surely no one would attack an ordinary craftsman who honestly works for his community.

Digital Aquarium

We are now deeply immersed in the digital world.
We have bank accounts, social media accounts, YouTube channels, accounting systems, and websites where we sell our products.
An integrated accounting system controls sales systems in our stores, opens and settles cash registers, and configures production systems that carry out daily production plans.
Recipe and warehouse systems are also integrated with oven control systems.

Everything is connected within a defined structure that can be paralyzed by taking over access codes.
Most people try to work efficiently by saving time and simplifying processes.
We usually have one password for everything — sometimes we modify it by changing one or two letters.
All passwords are stored in Google’s password manager.

Our professional life and the safety of our money depend on the systems we have on our smartphones — because that’s the most intuitive and simple approach.
Bank codes, system codes, BLIK codes, and access to our online store — everything in Google Password Manager.

But what if someone took control of everything? Changed all the codes?
We assume nobody will — so we don’t worry.
The situation is like leaving bicycles unlocked in front of a shop.
When a country is safe, and stealing bikes is unprofitable, nothing happens.
We leave our cars unlocked in front of our houses, expensive garden tools on open properties.

But maybe it’s worth taking an interest in cybersecurity?
The situation can change drastically if we fall victim to a hacker who, for fun or profit, paralyzes our lives.

W-MOSZCZYNSKI ppic 6-25

Why Would Someone Hack Me for Fun?

It’s not exactly about fun.
Currently, there’s a significant downturn in the IT services market.
Many highly qualified professionals in information technology have lost their jobs due to the implementation of artificial intelligence — which turned out to be much more efficient at coding than humans.

At the same time, because of the war in Ukraine and the growing importance of artificial intelligence, there is now huge demand for cybersecurity specialists.
The problem is that you can’t learn cybersecurity at university.
There, you only acquire basic knowledge.

The best cybersecurity experts come from hacker circles — they usually call themselves ethical hackers.
They claim to have hacking experience but say they’ve never done anything bad (of course, we believe that).
To become a hacker, you must start hacking — and to gain practice, you must act.
You can’t learn how to break into systems in artificial, virtual environments.
It’s best to hack into existing secured systems.

A Hooded Figure in a Dark Room with a Monitor?

Let’s imagine this situation.
I go to a café.
I sit down with my phone, maybe I want to check my account balance or send an email to a supplier.
I connect to Wi-Fi called “Café_Free” — sounds fine, right?
Who would think that behind it hides someone who just connected to my life?

This type of attack is called “man-in-the-middle” — the hacker sits in the middle of the communication, between your phone and the internet.

Someone creates a fake Wi-Fi network — identical to the one you usually see at cafés, airports, hotels, or on a Pendolino train.
They give it a familiar name, like Starbucks_WiFi or LOT_Free_WiFi — and you’re trapped.

The hacker has just grabbed your bank password, access to your online store, or even your browsing history and secret confectionery recipes.

The hacker, like a spider, sets up a fake network and waits for someone naive enough to connect.
You don’t have to click any suspicious link or download any strange file — just connect.
From that moment, all your data goes through their computer before reaching the real internet.

It’s like writing a letter to your bank — but first handing it to a stranger who reads it, copies it, and only then mails it.

Some people say: “But I’m just an ordinary person — I have nothing interesting.”
And I say: for a hacker, everything is interesting — even access to your email inbox.
They can use it for further attacks.
Your contact list can easily be sold to an advertising agency.

And since, as I mentioned, we usually have one password for everything — it becomes a problem.
Such an attack can happen anywhere — someone can create a fake network called free_Warsaw_WiFi, and people will immediately connect.

And to make it even more cinematic — some wireless keyboards and mice, the “modern” ones, can be taken over via Bluetooth.
A hacker with the right device can control your computer while sitting in a car outside your office.
You wouldn’t even know who that person is.
They can type your password, send emails, or simply log you out.

Sometimes it’s worth going back to something old and reliable — a wired keyboard and mouse.

The Extra Technician

If we have a company that employs more people than can fit on a city bus, we usually deal with a multitude of processes, services, and suppliers.
Many different specialists come through our business — plumbers, electricians, camera technicians, or accounting system operators.
We can’t monitor them all.

Now imagine this scene:
A technician comes in to “check the camera signal.”
He unplugs a cable and plugs something back in.
It looks like a regular Ethernet adapter — an inconspicuous little box.
We have plenty of such unknown devices in our companies.

But inside that little box sits a computer.
Once you plug it in, it automatically connects to the hacker’s server.
From that moment, they have access to your world.

Would you even notice?
Nobody checks what’s behind the router or under the desk.
Everything works, the internet is up, the store terminal functions — why bother?

The Hacker’s Credit Card Trick

Now one more quick trick — the credit card.
The chip, once a symbol of innovation and convenience, has become a weak point for cyberattacks.
Someone can walk right past you with a small reader in their pocket — no contact required.
You walk through a shopping mall and… boom — your card number is read.
And you have no idea.

Fortunately, there’s still the CVV number (Card Verification Value).
The card number alone won’t help the hacker unless they also gain access to your phone.
In general, attacks on bank accounts require higher expertise and are better protected by law enforcement.
That’s why hackers often target something easier — your home server.

A Beginner Hacker’s Exercise: How to Break Into a Home Server

Picture someone sitting in a car outside your house.
That’s how a hacker attack on your home Wi-Fi network might look.

Remember the café example? The hacker who accessed a phone through free Wi-Fi can now see the password to your home server through Google Password Manager.

Most people use default IP addresses like 192.168.0.1 or 192.168.1.1 — and never change them because “it works.”
The router login panel is standard, accessible through a browser.

But — not only you know that address.
A hacker who once connected to your Wi-Fi, or that technician who had temporary access, can scan your entire local network in seconds.
They’ll see your home server at 192.168.0.105, your router at 192.168.0.1, and your laptop at 192.168.0.103.

If the hacker doesn’t already have your password, they try default ones — using a list of the 150 most common passwords worldwide.
Their software checks all of them in less than a minute.
Many people log into their server panels with passwords like:
admin/admin, admin/1234, user/password, qwerty, or MyCompany2023.
Because who would hack a pastry chef’s home server?

Once the hacker gets in, they use a password dictionary — an automatic program that tries every common combination.
A minute or two, and they’re inside.
They can see your saved passwords (stored in a .txt file on your desktop), install malware, or open a backdoor for permanent access.

They can redirect your web traffic so that everyone visiting your online store or blog receives fake files, ads, or even malware — and you won’t even notice.
Everything looks normal.

All this happens simply because your server had a default address and default password.
Because you “didn’t bother to change it.”

We’re pastry chefs — honest people.
We work with flour, not with computers.

The Cherry on the Cake

The modern car key — a wireless module that opens your vehicle.
It transmits a coded signal — and the hacker can intercept it, record it, and wait.
Then, at night, they approach your car and click — it’s open.
No noise. No broken windows.

It sounds like science fiction, but it’s not.
It’s not the future — it’s happening now.
And it can happen to you.
All because someone left a strange little box in the cabinet next to your modem.

Why Updates Matter

We can now decide — either we fall into permanent cyber-paranoia, or we classify the author of this article among the “cyber-freaks,” people who can turn even the brightest day into a dark night.

The choice about your mental comfort is yours — but remember one thing: keep everything updated.

That’s the simplest solution.
Update everything — routers, phones, computers, refrigerators, robot vacuums, even your toaster if it has Wi-Fi.
Each update doesn’t just add icons — it fixes vulnerabilities that someone could exploit.

Most hacking attacks are done remotely and rely on system vulnerabilities.
These are gradually patched by software providers.
If you don’t update, you’re leaving the door open with a note saying “Welcome, hackers.”

Also, change your passwords, memorize them, or write them on paper — not in .txt files.
And don’t let Google Password Manager remember them for you.

Because the truth is:
Today’s hacker doesn’t have to be a genius.
All it takes is you forgetting to click “Install update,” connecting to free airport Wi-Fi, or leaving your router’s default password unchanged.

And suddenly your digital infrastructure becomes a testing ground for some beginner hacker learning the trade.

Artykuł A Cyberattack on My Pastry Shop? That’s Absurd pochodzi z serwisu THE DATA SCIENCE LIBRARY.

Isolation Forest is a machine learning model that can be used to detect anomalies on a food production line, for example, a meat packaging line. Such an installation is equipped with a number of sensors that continuously send information. Receiving this data on a personal computer is relatively simple. The data stream can be directed straight from the sensors to an analytical cloud, such as Azure or AWS, where a ready-made model can be launched. This model will automatically send alerts related to anomalies detected in the production process.

How the Isolation Forest Model Works

This solution is easy, simple (for an average data analyst), and often free.
Let’s assume we have two measurement sensors on a meat packaging line:

• Weight sensors, which check whether each piece has the correct weight,

• Temperature sensors, which measure the temperature of the meat, indicating its freshness.

Thanks to these sensors, for each product a set of data is collected — weight and temperature.

The Isolation Forest model learns from many products that have passed through the line. In this way, it learns what a “typical” product looks like. Suddenly, an unusual product may appear — for example, one that is too heavy or too warm, which may indicate that the meat is starting to spoil. Such a product will then be recognized by the Isolation Forest model as an anomaly.

Why does this happen? Because a product that differs from the rest (has a different weight or temperature) will be quickly isolated by the model’s decision trees. The model needs only a few splits to notice that this product “does not fit” the standard products normally appearing on the line.

W-MOSZCZYNSKI ps 4-25 w2

How Isolation Forest Detects Anomalies

The Isolation Forest model is used to detect anomalies in a set of elements — it identifies something that doesn’t fit the rest. The model builds trees randomly; the selection of features is random (e.g., customer gender, race, age, income). Thus, the structure of trees is not predetermined.

From a database of clients, a tree may first randomly divide them by gender (women and men), then by place of residence, age, etc. If something entirely unusual suddenly appears (a horse, a cat), such a data point will be very quickly separated — after one or two splits — forming a single, short branch of the tree. This short branch indicates that the point is an anomaly.

The key lies precisely in the speed of isolating anomalous points (in the above example — animals from humans). Identifying anomalies is based on identifying branches that are very short. The fewer splits required to separate a point from others, the more likely it is an anomaly.

Each tree is created by randomly dividing data into increasingly smaller groups. Normal observations are similar to one another, so separating them requires many splits — many branches are created. Anomalies, on the other hand, are isolated after only a few splits. The shorter and more singular the branch, the greater the anomaly.

The deciding factor for whether an observation is anomalous is therefore the number of branches (splits) needed to isolate it from the rest of the data.

Isolation Forest is particularly well-suited for analyzing streaming data for several reasons:

• It is very fast, building simple trees randomly and not requiring complex computations.
  Thanks to this, it processes data in real time, without delays.

• It does not require storing full historical data or creating complex patterns.

• It is ideal for rapid detection of individual unusual points as they appear.

• It can be easily updated — being an unsupervised model, it flexibly adapts to new data, which is essential for changing data streams.

• It requires little memory, as trees do not rely on stored historical data — they only retain the historical structure of the trees.

In summary, Isolation Forest is fast, computationally lightweight, and well-suited for detecting anomalies on the fly during real-time data processing. In its basic version, the model analyzes static data (datasets previously collected), but it can also be adapted to handle dynamic (streaming) data.

Supervised and Unsupervised Learning

In data science, analyses are divided into static and dynamic ones.
A static analysis is based on a fixed dataset containing historical data. The model learns from this data, and once trained, it can be applied to dynamic data. This approach is called supervised learning, because the model learns using defined explanatory variables that affect the outcome.

Supervised learning resembles teaching a child to recognize animals by showing pictures and saying, “This is a cat, and this is a dog.” Here, the model knows exactly what result it should achieve for given examples because it has a clearly defined output variable — what we are trying to predict. The model thus learns how features (e.g., size, fur length, sound) affect the result (cat or dog).

Unsupervised learning works differently. It is like showing a child many pictures of animals without saying their names. The child must independently find similarities or differences and create groups or categories. In this approach, there is no output variable (no “cat,” “lion,” or “horse” label). The model tries to understand the data’s structure, divide it into logical groups, or detect differences between them. When something does not fit any group, it is considered an anomaly.

In summary, the key difference is that in supervised learning, the model receives clear guidance on the expected result, whereas in unsupervised learning, the model must find patterns and structure by itself, because no correct answers are predefined.

Static vs. Dynamic (Streaming) Analysis

In static (batch) analysis, we work with a complete, closed dataset (e.g., a year’s worth of customer transactions). Decision trees are built once for this dataset.

In dynamic (streaming) analysis, data points arrive in real time — there is no fixed, large dataset, only continuously incoming data, such as online store transactions or sensor data from production systems.

Isolation Forest needs historical data to build its initial forest of random decision trees. Then, as new data arrives, it checks these points against the existing trees, determining whether they are anomalies. Thus, the model does not need to store historical data — it only keeps the tree structure, which summarizes the characteristics of previously analyzed data.

How Isolation Forest Works for Streaming Processes

For streaming data, Isolation Forest is used in the form of the Incremental Isolation Forest.
Based on the initial data, the first tree is built. As new data arrives in the streaming process, the model continuously updates the trees.

In this way, it constantly “learns” new patterns and reacts quickly to changes.

Returning to our earlier example — if the bank customer dataset (where a horse appeared as an anomaly) is later expanded to include many cats, the model will treat them as a group and begin to identify subcategories such as fur color, gender, and age. It dynamically creates new trees.

Thus, in practice, for dynamic data, the Isolation Forest model can continuously adapt to the data stream.

Summary

To summarize, Isolation Forest models are an effective tool for automatic anomaly detection because they can simultaneously analyze hundreds of different variables and quickly identify deviations from the norm.

The use of such models is not limited to production lines — they can also be applied to:

• analyzing products stored in warehouses,

• raw materials received at entry scales in processing plants,

• monitoring auxiliary processes such as workforce utilization or production support systems.

Importantly, once implemented in cloud infrastructure, Isolation Forest models operate fully automatically and practically maintenance-free.
This means that, once deployed, they can function indefinitely, with minimal susceptibility to failure or technical errors — further increasing the reliability of the entire quality control and production monitoring process.

The practical implementation of this solution involves connecting the data stream from the production line directly to the analytical cloud via an API interface.
The streaming data are then automatically processed by an analytical module based on the Incremental Isolation Forest model, which continuously classifies products as “normal” or “anomalous.”

The results are sent as time series, containing a clear classification for each analyzed product. These alerts can be delivered directly to mobile devices — for example, the production manager’s or quality inspection staff’s smartphones — allowing immediate response to potential quality or technological problems, significantly improving the efficiency of the entire quality control process.

Artykuł Application of the Incremental Isolation Forest Model for Improving the Quality of Food Production pochodzi z serwisu THE DATA SCIENCE LIBRARY.

A hacker attack is a deliberate action aimed at paralyzing a network server or, in its active form, taking control over it. In the past, this was done by introducing a virus; today, antivirus systems are so advanced that viruses have become a secondary threat.

A network server can be compared to the human body. Human skin has an extremely high ability to block all pathogens, germs, and chemical substances. It is like a wall through which nothing can pass. Infections occur only when the skin is damaged, has deep cracks, or is penetrated by a sting or another type of attack that destroys its structure. The human body has two main gateways through which germs, viruses, and pathogens enter: the alimentary canal and the respiratory tract. As we know, these two gates cannot be closed, because that would deprive us of life.

An identical mechanism exists in servers; these devices have entry gates through which vast amounts of content flow, often containing harmful software. Closing the inflow of information to the server would mean it could not be used — it would cease to perform its function.

Therefore, both servers and living organisms cannot completely shut themselves off from the external environment. They must build systems within themselves that can identify and eliminate threats. This function must be performed in a streaming system, that is, in real time. Threats must be eliminated immediately — here and now.

W-Moszczynski (5)

Operation of Machine Learning Models

In general, data science models can be divided into supervised, unsupervised, and autoregressive models.

Supervised Models

The learning process in supervised learning involves randomly dividing a dataset into training (used to train the model) and test (used to check its efficiency and accuracy) sets, usually in an 80/20 proportion. The trained model’s task is to detect information that is anomalous, that is, deviating from the standard behavior of data on the server. It takes the form of an algorithm or mathematical equation. It is usually small in size, though depending on its technology, it may absorb significant server resources.

The problem of consuming large analytical resources is best illustrated by the example of military drones. At present, drones are mostly operated by humans. However, Machine Learning models could easily identify and eliminate enemy armored vehicles. The problem is that drones themselves have no computers onboard. Computers like Raspberry Pi can be installed in them, but they are relatively expensive, and their capabilities are insufficient to smoothly handle such artificial intelligence models.

The moment when widely available microcomputers become both cheap and powerful will be the moment when we witness the mythical war of robots against humans. To put it less dramatically — we will observe completely autonomous robots killing people. For now, this phenomenon is rare on the battlefield for technical reasons. As we can see, Asimov’s Laws of Robotics (created by Isaac Asimov, American science-fiction writer and science popularizer) can be thrown into the trash — before robots even appeared for good, these laws lost all significance.

Thus, supervised learning models work effectively but do not learn during operation. In other words, if server administrators do not ensure that the model is systematically retrained, hackers — or some form of artificial intelligence — will find new forms of cyberattack that the model will not recognize, because it did not have the chance to learn them.

Unsupervised Models

Unsupervised learning models for anomaly detection learn by identifying normal patterns in data and marking unusual observations as anomalies. These models learn without supervision, meaning they have no information about which data points are anomalies and which are normal — this is called learning from unlabeled data.

Such models mainly analyze the structure of the data and create a pattern of normality. New data entering the server are evaluated by comparison, identifying differences relative to the learned pattern.

Typical unsupervised algorithms for anomaly detection include:

• Clustering methods,

• Statistical methods (e.g., PCA, KDE) identifying outliers in a distribution,

• Autoencoders (based on neural network technology),

• Entropy-based models, e.g., Isolation Forest, which quickly isolates anomalies.

The Role of Logs

The human body communicates with the brain using protocols generated by the nervous system. If an internal organ is attacked by pathogens, we feel pain in a specific area. Pain is a protocol sent directly to the brain, whose purpose is to inform the decision-making center about a problem.

At the same time, there are procedures independent of our will. If dust enters the respiratory tract, we start sneezing and coughing.

Server logs work in a similar way, except they are generated and sent continuously — whether there is a problem or not. Logs are continuous information about server activity, sent to an analytical center.

At the current stage of development, this analytical center — the “brain” — is still a human. However, the evolution of AI systems introduces an intermediate problem identification layer between logs and humans, so that the human is not flooded with endless streams of information but instead receives synthetic summaries about the system’s state.

The key is to make the system intelligent enough to identify and immediately eliminate threats. This is exactly what artificial intelligence models are designed for.

Thus, the log register on a server is one of the key data sources for analyzing cyber threats.

We can distinguish five main types of logs:

1. Access logs,

2. Error logs,

3. Authentication and authorization logs,

4. Firewall logs,

5. Application logs.

When identifying anomalies that often turn out to be malicious software or a veiled form of cyberattack, application logs are the most useful, as they analyze user behavior on the server.

Application Logs

Application logs play a key role in both recommendation systems and cyberattack defense systems because they provide valuable information about user behavior.

They show:

• clicks on products or content,

• time spent on a product page or article,

• scrolling behavior,

• cart actions (add/remove product, purchase),

• ratings and reviews.

Hacker attacks are usually carried out by bots programmed to imitate human behavior. They can scroll pages, move to subpages, and perform actions similar to human users. However, bots are created according to specific templates and replicated, so they behave identically. Their uniformity makes them detectable and removable.

Moreover, bots do not respond to graphic or contextual cues as humans do. Of course, advanced AI can detect nuances in images, but bots are simple — they must be. If each bot contained a neural network for image recognition, the hacker’s computer controlling thousands of such bots would be heavily overloaded and inefficient. This would make mass attacks impossible — which, as we know, are one of the most common forms of cyberattacks (DDoS attacks).

DDoS-Type Cyberattacks

Bots are primarily designed to attack the application layer (Layer 7). The attack consists in running the most resource-intensive queries or functions of a website.

All DDoS (Distributed Denial of Service) attacks aim to paralyze the server by maximally consuming its resources. Other forms include volume-based attacks, where massive amounts of “empty” queries are sent to the server, and protocol attacks (e.g., SYN Flood), involving interrupted or incomplete external login procedures.

Intrusive Attacks

DDoS attacks aim to overload a server artificially, but logs can also be used to fight intrusive attacks such as SQL Injection, XSS, or RCE. These are far more dangerous because they are active and aim for specific goals — such as data theft or taking control of the server.

• SQL Injection involves injecting malicious code into database queries, allowing the attacker to read, modify, or delete data, and sometimes gain full control over the database.

• Cross-Site Scripting (XSS) injects malicious JavaScript code into an application, which can lead to session hijacking, fake content display, or account takeover.

• Remote Code Execution (RCE) allows arbitrary code to be executed on the server, potentially taking control of the system. Variants include LFI (Local File Inclusion) and RFI (Remote File Inclusion).

How Machine Learning Uses Application Logs to Detect Attacks

The best method of defense against cyberattacks is detecting anomalies using unsupervised Machine Learning algorithms. Here are the key models:

Isolation Forest

Like the well-known Random Forest, it builds many decision trees (e.g., 100–200). Each tree divides a random subset of data to check how quickly a given point can be “isolated.” If a point is isolated very quickly, it is an anomaly.

Each data point (e.g., username, IP, server query) passes through multiple trees. For each, the model computes the average number of splits required for isolation. Normal points need many splits; anomalies require few.

Autoencoders

An unsupervised model that learns during operation.
It works in two steps:

1. Encoding – compressing large datasets (e.g., user logs) into a smaller representation.

2. Decoding – reconstructing the original data from the compressed form.

The model learns to reconstruct “normal” transactions. If a suspicious transaction appears (e.g., strange activity, too fast or too repetitive), the reconstruction error grows, and the model flags it as anomalous.

This can be illustrated simply: we expect an elderly woman on a tram to behave calmly. Suddenly, a person dressed as a dragon enters, shaking uncontrollably — unpredictable. Our brain cannot anticipate what happens next. This is a stochastic or anomalous behavior. Autoencoders are based on recurrent neural networks (RNNs) capable of such contextual learning.

LOF (Local Outlier Factor)

LOF compares user activity with that of neighbors, measuring local density. If a point is in a dense cluster, it is typical; if it lies alone, it’s suspicious. The algorithm is similar to statistical parametric tests based on normal distribution density.

Why Use Autoregressive Models for Anomaly Detection?

Besides supervised and unsupervised models, autoregressive models are very important in anomaly detection because they can capture what others cannot — temporal changes.

Isolation Forest, LOF, and Autoencoders detect anomalies in static data, but they do not analyze time-based dynamics. Autoregressive models detect anomalies in trends and sequences.

Models such as ARIMA (Autoregressive Integrated Moving Average) and ARMA (Autoregressive Moving Average) belong to statistical time-series forecasting. They model dependencies between observations over time.

Autoregressive models detect anomalies in temporal dynamics — for example, they can see that someone performs the same action every hour or that there are strange pauses in server activity.

If a server usually handles 500 requests per minute and the autoregressive model predicts 480–520, but suddenly it receives 700, the model marks this as an anomaly. Such models are ideal for detecting DDoS attacks, unusual logins, and sudden changes in user behavior.

Summary

Network security is a key condition for the development of the digital economy. Over millions of years of evolution, living organisms have developed systems that eliminate invisible biological threats such as microbes, viruses, and bacteria. The same is now happening in digital evolution.

Today, cybersecurity no longer relies solely on firewalls or simple associative rules. Increasingly sophisticated forms of cyberattacks require equally sophisticated and effective countermeasures.

Artykuł Review of the Most Important Machine Learning Models for Detecting and Eliminating Hacker Attacks pochodzi z serwisu THE DATA SCIENCE LIBRARY.