Within statistics, the Kolmogorov–Zurbenko (KZ) filter was first proposed by A. N. Kolmogorov and formally defined by Zurbenko.I. Zurbenko. The Spectral Analysis of Time Series. North-Holland Series in Statistics and Probability, 1986. It is a series of iterations of a moving average filter of length m, where m is a positive, odd integer. The KZ filter belongs to the class of low-pass filters. The KZ filter has two parameters, the length m of the moving average window and the number of iterations k of the moving average itself. It also can be considered as a special window function designed to eliminate spectral leakage.
Background A. N. Kolmogorov had the original idea for the KZ filter during a study of turbulence in the Pacific Ocean. Kolmogorov had just received the International Balzan Prize for his law of 5/3 in the energy spectra of turbulence. Surprisingly the 5/3 law was not obeyed in the Pacific Ocean, causing great concern. Standard fast Fourier transform (FFT) was completely fooled by the noisy and non-stationary ocean environment. KZ filtration resolved the problem and enabled proof of Kolmogorov's law in that domain. Filter construction relied on the main concepts of the continuous Fourier transform and their discrete analogues. The algorithm of the KZ filter came from the definition of higher-order derivatives for discrete functions as higher-order differences. Believing that infinite smoothness in the Gaussian window was a beautiful but unrealistic approximation of a truly discrete world, Kolmogorov chose a finitely differentiable tapering window with finite support, and created this mathematical construction for the discrete case.
The KZ filter is robust and nearly optimal. Because its operation is a simple Moving Average (MA), the KZ filter performs well in a missing data environment, especially in multidimensional time series where missing data problem arises from spatial sparseness. Another nice feature of the KZ filter is that the two parameters have clear interpretation, enabling easy adoption by specialists in different areas. A few software packages for time series, longitudinal and spatial data have been developed in the popular statistical software R, which facilitates the use of the KZ filter and its extensions in different areas. The I. Zurbenko Postdoctoral position at UC Berkeley with Jerzy Neyman and Elizabeth Scott provided many applications, supported by contacts with Murray Rosenblatt, Robert Shumway, Harald Cramér, David Brillinger, Herbert Robbins, Wilfrid Dixon, Emanuel Parzen.
Definition KZ Filter
Let be a real-valued time series, the KZ filter with parameters and is defined as
where coefficients
are given by the polynomial coefficients obtained from equation
From another point of view, the KZ filter with parameters and can be defined as time iterations of a moving average (MA) filter of points. It can be obtained through iterations.
The first iteration applies an MA filter over the process ,
The second iteration applies the MA operation to the result of the first iteration,
Generally the kth iteration is an application of the MA filter to the (k − 1)th iteration. The iteration process is computationally cheap due to relying on simple MA operations.
Properties The impulse response function of the product of filters is the convolution of impulse responses. The coefficients of the KZ filter , can be interpreted as a distribution obtained by the convolution of k uniform discrete distributions on the interval where m is an odd integer. Therefore, the coefficient forms a tapering window which has finite support . The KZ filter has main weight concentrated on a length of with weights vanishing to zero outside. The impulse response function of the KZ filter has continuous derivatives and is asymptotically Gaussian distributed. Zero derivatives at the edges for the impulse response function make from it a sharply declining function, what is resolving in high frequency resolution. The energy transfer function of the KZ filter is
It is a low-pass filter with a cut-off frequency of
Compared to a MA filter, the KZ filter has much better performance in terms of attenuating the frequency components above the cutoff frequency. The KZ filter is essentially a repetitive MA filter. It is easy to compute and allows for a straightforward way to deal with missing data. The main piece of this procedure is a simple average of available information within the interval of m points disregarding the missing observations within the interval. The same idea can be easily extended to spatial data analysis. It has been shown that missing values have very little effect on the transfer function of the KZ filter.
Arbitrary k will provide k power of this transfer function and will reduce side lobe value to . It will be a perfect low pass filter. For practical purposes a choice of k within a range 3 to 5 is usually sufficient, when regular MA (k = 1) is providing strong spectral leakage of about 5%.
Optimality The KZ filter is robust and nearly optimal. Because its operation is a simple moving average, the KZ filter performs well in a missing data environment, especially in multidimensional time and space where missing data can cause problems arising from spatial sparseness. Another nice feature of the KZ filter is that the two parameters each have clear interpretations so that it can be easily adopted by specialists in different areas. Software implementations for time series, longitudinal and spatial data have been developed in the popular statistical package R, which facilitate the use of the KZ filter and its extensions in different areas.
KZ filter can be used to smooth the periodogram. For a class of stochastic processes, Zurbenko considered the worst-case scenario where the only information available about a process is its spectral density and smoothness quantified by Hölder condition. He derived the optimal bandwidth of the spectral window, which is dependent upon the underlying smoothness of the spectral density. Zurbenko compared the performance of Kolmogorov–Zurbenko (KZ) window to the other typically used spectral windows including Bartlett window, Parzen window, Tukey–Hamming window and uniform window and showed that the result from KZ window is closest to optimum.
Developed as an abstract discrete construction, KZ filtration is robust and statistically nearly optimal. At the same time, because of its natural form, it has computational advantages, permitting analysis of space/time problems with data that has much as 90% of observations missing, and which represent a messy combination of several different physical phenomena. Clear answers can often be found for "unsolvable" problems. Unlike some mathematical developments, KZ is adaptable by specialists in different areas because it has a clear physical interpretation behind it.
Extensions
Extensions of KZ filter include KZ adaptive (KZA) filter, spatial KZ filter and KZ Fourier transform (KZFT). Yang and Zurbenko provided a detailed review of KZ filter and its extensions. R packages are also available to implement KZ filtrationB.Close, I.Zurbenko, Kolmogorov–Zurbenko adaptive algorithm, Proceedings JSM, 2011
KZFT
KZFT filter is design for a reconstruction of periodic signals or seasonality covered by heavy noise. Seasonality is one of the key forms of nonstationarity that is often seen in time series. It is usually defined as the periodic components within the time series. Spectral analysis is a powerful tool to analyze time series with seasonality. If a process is stationary, its spectrum is a continuous form as well. It can be treated parametrically for simplicity of prediction. If a spectrum contains lines, it indicates that the process is not stationary and contains periodicities. In this situation, parametric fitting generally results in seasonal residuals with reduced energies. This is due to the season to season variations. To avoid this problem, nonparametric approaches including band pass filters are recommended. Kolmogorov–Zurbenko Fourier Transform (KZFT) is one of such filters. The purpose of many applications is to reconstruct high resolution wavelet from the noisy environment. It was proven that KZFT provides the best possible resolution in spectral domain. It permits the separation of two signals on the edge of a theoretically smallest distance, or reconstruct periodic signals covered by heavy noise and irregularly observed in time. Because of this, KZFT provides a unique opportunity for various applications. A computer algorithm to implement the KZFT has been provided in the R software. The KZFT is essentially a band pass filter that belongs to the category of short-time Fourier transform (STFT) with a unique time window.
KZFT readily uncovers small deviations from a constant spectral density of white noise resulting from computer random numbers generator. Such computer random number generations become predictable in the long run. Kolmogorov complexity provides the opportunity to generate unpredictable sequences of random numbers.
Formally, we have a process ,, the KZFT filter with parameters m and k, computed at frequency ν0, produces an output process, which is defined as following:
where is defined as: , ,..., and the polynomial coefficients is given by . Apparently filter is equivalent to the application of filter to the process . Similarly, the KZFT filter can be obtained through iterations in the same way as KZ filter.
The average of the square of KZFT in time over S periods of will provide an estimate of the square amplitude of the wave at frequency ν0 or KZ periodogram (KZP) based on 2Sρ0 observations around moment t:
Transfer function of KZFT is provided in Figure 2 has a very sharp frequency resolution with bandwidth limited by . For a complex-valued process , the outcome is unchanged. For a real-valued process, it distributes energy evenly over the real and complex domains. In other words, reconstructs a cosine or sine wave at the same frequency ν0. It follows that correctly reconstructs the amplitude and phase of an unknown wave with frequency ν0. Figure below is providing power transfer function of KZFT filtration. It clearly display that it perfectly captured frequency of interest ν0 = 0.4 and provide practically no spectral leakage from a side lobes which control by parameter k of filtration. For practical purposes choice of k within range 3–5 is usually sufficient, when regular FFT (k = 1) is providing strong leakage of about 5%.
Example: Simulated signal normal random noise N(0,16) was used to test the KZFT algorithm's ability to accurately determine spectra of datasets with missing values. For practical considerations, the percentage of missing values was used at p=70% to determine if the spectrum could continue to capture the dominant frequencies. Using a wider window length of m=600 and k=3 iterations, adaptively smoothed KZP algorithm was used to determine the spectrum for the simulated longitudinal dataset. It is apparent in Figure 3 that the dominant frequencies of 0.08 and 0.10 cycles per unit time are identifiable as the signal's inherent frequencies.