September 2022 | Przegląd Piekarski i Cukierniczy (Baking and Confectionery Review)
Kaplan–Meier survival analysis, the Cox proportional hazard model
One of the most important problems in management is maintaining the stability of processes. The destabilising factor is the turnover of the fixed elements of the model. These are steady key customers, machines which theoretically should not break down, key employees and key raw-material suppliers.
The moment one of the elements mentioned drops out, withdraws or leaves, the whole functioning process of the company undergoes greater or lesser destabilisation. The role of the management is to predict various scenarios of events and, to the extent that information allows, to counteract them.
Survival analysis
Quoting Wikipedia:¹ „Survival analysis tries to answer certain questions, e.g. what is the proportion of a population which will survive past a certain time? Of those who survive, at what rate will they die or fail? Can multiple causes of death or failure be taken into account? How do particular circumstances or characteristics increase or decrease the probability of survival?”
The birthplace of survival analysis was a hospital, in which it was analysed how long a patient would survive after hearing a diagnosis. Contrary to appearances, the answer to this question was not simple. Some patients died, some discontinued treatment and disappeared from the observed group, others died but from other causes, some patients recovered. The study lasted several years; some had taken part in it from the beginning, others began treatment a few days before the end of the research project. It was therefore necessary to create a special tool which would be able to extract statistically valuable conclusions from the research.
An analysis of position turnover at the Jupiter.pl confectionery company
W-MOSZCZYNSKI ppic 9-22Low unemployment and competition between industries influence a high level of employee turnover. Employees who are attractive on the labour market — that is, those with higher education and sought-after specialisations, such as IT or mechanical engineering — are particularly exposed.
Employee turnover is a very burdensome phenomenon for the confectionery industry. The time needed to introduce a new employee to the intricate technological processes is long. During this time they are not able to work effectively. That is why it is important for them to stay in the company as long as possible after training. In extreme cases, the departure of a key employee may cause a slowdown or even an interruption of the production process. On the other hand, employing a larger number of workers, so as to create sufficiently large human reserves, is connected with an increase in labour costs, which consequently does not allow high salaries to be maintained for key employees.
The management of the Jupiter.pl factories commissioned research in order to better understand the problem and thereby counteract the phenomenon of employee turnover.
An analysis of the phenomenon of employee departures from the Jupiter.pl factories
The number of employees who have worked and are working at the Jupiter.pl factories in Poland amounts to 2121 people. This total includes people who were employed in manual and office posts, in production as well as in logistics departments. Importantly, among the 2121 people there are 616 people who left the company.
Initially, attempts were made to obtain information about the reasons for leaving by conducting survey research. This did not bring the expected results, because many people had left before this idea was born. Some departing employees did not want to fill in surveys; others entered evidently untrue reasons for leaving. It was therefore decided to make use of survival analysis.
Preparing the data
In order to use the test, the data has to be organised to contain categorical values. From the employee database, the following categorical variables were extracted:
Level_of_education:
primary (1277 people),
vocational school (335 people),
secondary school (214 people),
post-secondary school (152 people),
higher education (143 people).
Employee_programme:
Yes — employee belongs to the employee programme (1351 people),
No — employee does not belong to the employee programme (770 people).
Family_members: this variable says how large the family of the employee covered by the study is. Four categories were distinguished:
single person (1%),
wife, no children (4%),
wife, one child (11%),
wife, two or more children (84%).
Which_job: employees were divided into those for whom this was their:
first job (41%),
second job (31%), and
employees for whom the job at Jupiter.pl was a subsequent job (28%).
Professional_qualifications:
employee does not have professional qualifications (0),
employee has professional qualifications (1).
Job_position: these are the 14 various departments of the company in which the employees covered by the study were employed.
Additionally, a continuous variable was introduced: km_to_home, giving how many kilometres separate the workplace and the employee’s place of residence.
Finally the variable employee_left was obtained, where: 1 — means that the employee left (616 people), and 0 — means that the employee is still working (1505 people).
The Kaplan–Meier test
Thanks to the use of the Kaplan–Meier test, within the framework of Survival Test technology, we can check what the influence of a chosen feature is on the level of probability of employee departures. To calculate the test we will use the lifelines library in the Python programming environment.
In order to calculate the dependencies, one first has to choose a specific variable which theoretically influences the level of employee departures. The management of the Jupiter.pl factories first of all wanted to find out what influence possessing professional qualifications has on the level of departures.
from lifelines import KaplanMeierFitter
kmf_NO = KaplanMeierFitter() # no professional qualifications
kmf_YES = KaplanMeierFitter() # has professional qualifications
NO_PQ = SKO.query(„Professional_qualifications == 0”)
YES_PQ = SKO.query(„Professional_qualifications == 1”)
A Kaplan–Meier query is created.
kmf_NO.fit(
durations=NO_PQ[„work_days”],
event_observed=NO_PQ[„Employee_left”],
label=”Employee has no professional qualifications”,
)
kmf_YES.fit(
durations=YES_PQ[„work_days”],
event_observed=YES_PQ[„Employee_left”],
label=”Employee has professional qualifications”,
)
Chart 1 presents the curve of probability that an employee will not leave. On the X axis are the employees’ work days, counted from the moment of employment to the end of the period covered by the analysis. The probability of remaining in the company after 400 days of work for employees possessing professional qualifications amounts to about 85%, while for employees without professional qualifications it is as much as 98%.
By professional qualifications we mean any licences for repairing machines and installations, as well as a heavy-goods-vehicle driving licence, licences for operating cranes and overhead cranes, and running advanced production processes. Possessing special professional qualifications significantly reduces the survival level of employees at the Jupiter.pl plants.
As can be seen in Chart 2, the level of education significantly influences the length of employment at the Jupiter.pl confectionery factories. The probability that after a year an employee with higher education will still be working for the company is barely 40%. In the case of people with secondary–post-secondary education, the probability that an employee will still be working for the company a year after being employed amounts to about 70%.
In Chart 3 it is clearly visible that the size of an employee’s family is of fundamental importance for their decision to stay in the company. A single person is extremely unpredictable, as evidenced by the wide band around the blue probability line. As can be seen, employees with two or more children maintain high employment stability.
The Cox proportional hazard model
Let us move on, then, to the next tool from the area of survival analysis. The Cox model shows what the statistical influence of many factors is on the phenomenon of survival. Again we will use the Python programming environment.
In order to find out more, we should generate a statistical table.
cph = CoxPHFitter()
cph.fit(SKO, duration_col=”work_days”, event_col=”Employee_left”)
cph = cph.print_summary()
Statistical Table 1. Statistical parameters of employee survival in the context of the chosen variables
| coef | exp(coef) | se(coef) | coef lower 95% | coef upper 95% | exp(coef) lower 95% | exp(coef) upper 95% | cmp to | z | p | -log2(p) | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ID_Pracownika | -0.00 | 1.00 | 0.00 | -0.00 | 0.00 | 1.00 | 1.00 | 0.00 | -0.05 | 0.96 | 0.06 |
| km_do_domu | -0.00 | 1.00 | 0.00 | -0.01 | 0.01 | 0.99 | 1.01 | 0.00 | -0.26 | 0.79 | 0.34 |
| Poziom_wykształcenia | 1.58 | 4.87 | 0.05 | 1.48 | 1.69 | 4.38 | 5.42 | 0.00 | 29.11 | <0.005 | 616.42 |
| Program_pracowniczy | -0.44 | 0.65 | 0.10 | -0.64 | -0.24 | 0.53 | 0.79 | 0.00 | -4.34 | <0.005 | 16.10 |
| Członków_rodziny | -0.81 | 0.45 | 0.07 | -0.95 | -0.67 | 0.39 | 0.51 | 0.00 | -11.12 | <0.005 | 92.95 |
| Które_miejsce_pracy | -0.33 | 0.72 | 0.06 | -0.44 | -0.21 | 0.64 | 0.81 | 0.00 | -5.39 | <0.005 | 23.74 |
| Uprawnienia_zawodowe | 0.03 | 1.04 | 0.09 | -0.15 | 0.22 | 0.86 | 1.24 | 0.00 | 0.37 | 0.71 | 0.49 |
| Stanowisko_pracy | 0.01 | 1.01 | 0.01 | -0.01 | 0.02 | 0.99 | 1.02 | 0.00 | 0.66 | 0.51 | 0.97 |
The exp(coef) column tells us about the significance of the individual variables. The value 1 means that the given variable has no statistical influence on the level of employee departures. If the coef column contains a negative value, this means that the chosen variable has a positive influence on survival. So „Employee_programme” has a positive influence on employees’ decision to stay in the company. The exp(coef) value means that people who belong to the programme leave the company 35% less often. Employees possessing special professional qualifications, on the other hand, leave 4% more often than people who do not possess such qualifications.
Summary
The method presented here has an application in many aspects of management. Thanks to it we can find out and analyse what factors influence the departure or retention of customers, what causes breakdowns in the production process, and what one has to eat to protect oneself from serious illness. Thanks to the use of the versatile programming tools of the Python environment, carrying out the analysis is relatively easy.
As can be seen in Chart 4, the number of family members, participation in the employee programme and length of service have a clear influence on employees staying in the company. The higher the length of service and the number of family members, the greater the employee’s chances of staying in the company. On the other side one can see the variables which negatively influence the employee’s chances of staying in the company. These are: „Level of education” and „Professional qualifications”.
Wojciech Moszczyński
1. https://en.wikipedia.org/wiki/Survival_analysis
Wojciech Moszczyński — graduate of the Department of Econometrics and Statistics of Nicolaus Copernicus University in Toruń; specialist in econometrics, finance, data science, and management accounting. He specializes in the optimization of production and logistics processes. He conducts research in the area of the development and application of artificial intelligence. For years he has been engaged in the popularization of machine learning and data science in business environments.
Dodaj komentarz