Survival analysis is a branch of statistics for analyzing the expected duration of time until one event occurs, such as death in biological organisms and failure in mechanical systems. This topic is called reliability theory, reliability analysis or reliability engineering in engineering, duration analysis or duration modelling in economics, and event history analysis in sociology. Survival analysis attempts to answer certain questions, such as what is the proportion of a population which will survive past a certain time? Of those that survive, at what rate will they die or fail? Can multiple causes of death or failure be taken into account? How do particular circumstances or characteristics increase or decrease the probability of survival?
To answer such questions, it is necessary to define "lifetime". In the case of biological survival, death is unambiguous, but for mechanical reliability, failure may not be well-defined, for there may well be mechanical systems in which failure is partial, a matter of degree, or not otherwise localized in time. Even in biological problems, some events (for example, heart attack or other organ failure) may have the same ambiguity. The theory outlined below assumes well-defined events at specific times; other cases may be better treated by models which explicitly account for ambiguous events.
More generally, survival analysis involves the modelling of time to event data; in this context, death or failure is considered an "event" in the survival analysis literature – traditionally only a single event occurs for each subject, after which the organism or mechanism is dead or broken. Recurring event or repeated event models relax that assumption. The study of recurring events is relevant in systems reliability, and in many areas of social sciences and medical research.
Introduction to survival analysis
Survival analysis is used in several ways:
To describe the survival times of members of a group Life tables Kaplan–Meier curves Survival function Hazard function To compare the survival times of two or more groups Log-rank test To describe the effect of categorical or quantitative variables on survival Cox proportional hazards regression Parametric survival models Survival trees Survival random forests
Definitions of common terms in survival analysis
The following terms are commonly used in survival analyses:
Event: Death, disease occurrence, disease recurrence, recovery, or other experience of interest Time: The time from the beginning of an observation period (such as surgery or beginning treatment) to (i) an event, or (ii) end of the study, or (iii) loss of contact or withdrawal from the study. Censoring / Censored observation: Censoring occurs when we have some information about individual survival time, but we do not know the survival time exactly. The subject is censored in the sense that nothing is observed or known about that subject after the time of censoring. A censored subject may or may not have an event after the end of observation time. Survival function S(t): The probability that a subject survives longer than time t.
Example: Acute myelogenous leukemia survival data
This example uses the Acute Myelogenous Leukemia survival data set "aml" from the "survival" package in R. The data set is from Miller (1997) and the question is whether the standard course of chemotherapy should be extended ('maintained') for additional cycles.
The aml data set sorted by survival time is shown in the box. +Aml data set sorted by survival timeobservationtime (weeks)statusx1251Nonmaintained1351Nonmaintained1481Nonmaintained1581Nonmaintained191Maintained16121Nonmaintained2131Maintained3130Maintained17160Nonmaintained4181Maintained5231Maintained18231Nonmaintained19271Nonmaintained6280Maintained20301Nonmaintained7311Maintained21331Nonmaintained8341Maintained22431Nonmaintained9450Maintained23451Nonmaintained10481Maintained111610Maintained
Time is indicated by the variable "time", which is the survival or censoring time Event (recurrence of aml cancer) is indicated by the variable "status". 0= no event (censored), 1= event (recurrence) Treatment group: the variable "x" indicates if maintenance chemotherapy was given
The last observation (11), at 161 weeks, is censored. Censoring indicates that the patient did not have an event (no recurrence of aml cancer). Another subject, observation 3, was censored at 13 weeks (indicated by status=0). This subject was in the study for only 13 weeks, and the aml cancer did not recur during those 13 weeks. It is possible that this patient was enrolled near the end of the study, so that they could be observed for only 13 weeks. It is also possible that the patient was enrolled early in the study, but was lost to follow up or withdrew from the study. The table shows that other subjects were censored at 16, 28, and 45 weeks (observations 17, 6, and9 with status=0). The remaining subjects all experienced events (recurrence of aml cancer) while in the study. The question of interest is whether recurrence occurs later in maintained patients than in non-maintained patients.
Kaplan–Meier plot for the aml data
The survival function S(t), is the probability that a subject survives longer than time t. S(t) is theoretically a smooth curve, but it is usually estimated using the Kaplan–Meier (KM) curve. The graph shows the KM plot for the aml data and can be interpreted as follows:
The x axis is time, from zero (when observation began) to the last observed time point. The y axis is the proportion of subjects surviving. At time zero, 100% of the subjects are alive without an event. The solid line (similar to a staircase) shows the progression of event occurrences. A vertical drop indicates an event. In the aml table shown above, two subjects had events at five weeks, two had events at eight weeks, one had an event at nine weeks, and so on. These events at five weeks, eight weeks and so on are indicated by the vertical drops in the KM plot at those time points. At the far right end of the KM plot there is a tick mark at 161 weeks. The vertical tick mark indicates that a patient was censored at this time. In the aml data table five subjects were censored, at 13, 16, 28, 45 and 161 weeks. There are five tick marks in the KM plot, corresponding to these censored observations.
Life table for the aml data
A life table summarizes survival data in terms of the number of events and the proportion surviving at each event time point. The life table for the aml data, created using the Rsoftware, is shown.
+ Life Table for the aml Data time n.risk n.event survival std.err lower 95% CI upper 95% CI 5 23 2 0.913 0.0588 0.8049 1 8 21 2 0.8261 0.079 0.6848 0.996 9 19 1 0.7826 0.086 0.631 0.971 12 18 1 0.7391 0.0916 0.5798 0.942 13 17 1 0.6957 0.0959 0.5309 0.912 18 14 1 0.646 0.1011 0.4753 0.878 23 13 2 0.5466 0.1073 0.3721 0.803 27 11 1 0.4969 0.1084 0.324 0.762 30 9 1 0.4417 0.1095 0.2717 0.718 31 8 1 0.3865 0.1089 0.2225 0.671 33 7 1 0.3313 0.1064 0.1765 0.622 34 6 1 0.2761 0.102 0.1338 0.569 43 5 1 0.2208 0.0954 0.0947 0.515 45 4 1 0.1656 0.086 0.0598 0.458 48 2 1 0.0828 0.0727 0.0148 0.462
The life table summarizes the events and the proportion surviving at each event time point. The columns in the life table have the following interpretation:
time gives the time points at which events occur. n.risk is the number of subjects at risk immediately before the time point, t. Being "at risk" means that the subject has not had an event before time t, and is not censored before or at time t. n.event is the number of subjects who have events at time t. survival is the proportion surviving, as determined using the Kaplan–Meier product-limit estimate. std.err is the standard error of the estimated survival. The standard error of the Kaplan–Meier product-limit estimate it is calculated using Greenwood's formula, and depends on the number at risk (n.risk in the table), the number of deaths (n.event in the table) and the proportion surviving (survival in the table). lower 95% CI and upper 95% CI are the lower and upper 95% confidence bounds for the proportion surviving.
Log-rank test: Testing for differences in survival in the aml data
The log-rank test compares the survival times of two or more groups. This example uses a log-rank test for a difference in survival in the maintained versus non-maintained treatment groups in the aml data. The graph shows KM plots for the aml data broken out by treatment group, which is indicated by the variable "x" in the data.
The null hypothesis for a log-rank test is that the groups have the same survival. The expected number of subjects surviving at each time point in each is adjusted for the number of subjects at risk in the groups at each event time. The log-rank test determines if the observed number of events in each group is significantly different from the expected number. The formal test is based on a chi-squared statistic. When the log-rank statistic is large, it is evidence for a difference in the survival times between the groups. The log-rank statistic approximately has a Chi-squared distribution with one degree of freedom, and the p-value is calculated using the Chi-squared test.
For the example data, the log-rank test for difference in survival gives a p-value of p=0.0653, indicating that the treatment groups do not differ significantly in survival, assuming an alpha level of 0.05. The sample size of 23 subjects is modest, so there is little power to detect differences between the treatment groups. The chi-squared test is based on asymptotic approximation, so the p-value should be regarded with caution for small sample sizes.
Cox proportional hazards (PH) regression analysis
Kaplan–Meier curves and log-rank tests are most useful when the predictor variable is categorical (e.g., drug vs. placebo), or takes a small number of values (e.g., drug doses 0, 20, 50, and 100 mg/day) that can be treated as categorical. The log-rank test and KM curves don't work easily with quantitative predictors such as gene expression, white blood count, or age. For quantitative predictor variables, an alternative method is Cox proportional hazards regression analysis. Cox PH models work also with categorical predictor variables, which are encoded as {0,1} indicator or dummy variables. The log-rank test is a special case of a Cox PH analysis, and can be performed using Cox PH software.
Example: Cox proportional hazards regression analysis for melanoma
This example uses the melanoma data set from Dalgaard Chapter 14.
Data are in the R package ISwR. The Cox proportional hazards regression usingR gives the results shown in the box.
The Cox regression results are interpreted as follows.
Sex is encoded as a numeric vector (1: female, 2: male). The Rsummary for the Cox model gives the hazard ratio (HR) for the second group relative to the first group, that is, male versus female. coef = 0.662 is the estimated logarithm of the hazard ratio for males versus females. exp(coef) = 1.94 = exp(0.662) - The log of the hazard ratio (coef= 0.662) is transformed to the hazard ratio using exp(coef). The summary for the Cox model gives the hazard ratio for the second group relative to the first group, that is, male versus female. The estimated hazard ratio of 1.94 indicates that males have higher risk of death (lower survival rates) than females, in these data. se(coef) = 0.265 is the standard error of the log hazard ratio. z = 2.5 = coef/se(coef) = 0.662/0.265. Dividing the coef by its standard error gives the z score. p=0.013. The p-value corresponding to z=2.5 for sex is p=0.013, indicating that there is a significant difference in survival as a function of sex.
The summary output also gives upper and lower 95% confidence intervals for the hazard ratio: lower 95% bound = 1.15; upper 95% bound = 3.26.
Finally, the output gives p-values for three alternative tests for overall significance of the model: