In computer science, radix sort is a non-comparative sorting algorithm. It avoids comparison by creating and distributing elements into buckets according to their radix. For elements with more than one significant digit, this bucketing process is repeated for each digit, while preserving the ordering of the prior step, until all digits have been considered. For this reason, radix sort has also been called bucket sort and digital sort.
Radix sort can be applied to data that can be sorted lexicographically, be they integers, words, punch cards, playing cards, or the mail.
History
Radix sort dates back as far as 1887 to the work of Herman Hollerith on tabulating machines. and Radix sorting algorithms came into common use as a way to sort punched cards as early as 1923.Donald Knuth. The Art of Computer Programming, Volume 3: Sorting and Searching, Third Edition. Addison-Wesley, 1997. . Section 5.2.5: Sorting by Distribution, pp. 168–179.
The first memory-efficient computer algorithm for this sorting method was developed in 1954 at MIT by Harold H. Seward. Computerized radix sorts had previously been dismissed as impractical because of the perceived need for variable allocation of buckets of unknown size. Seward's innovation was to use a linear scan to determine the required bucket sizes and offsets beforehand, allowing for a single static allocation of auxiliary memory. The linear scan is closely related to Seward's other algorithm — counting sort.
In the modern era, radix sorts are most commonly applied to collections of binary strings and integers. It has been shown in some benchmarks to be faster than other more general-purpose sorting algorithms, sometimes 50% to three times faster.
Digit order
Radix sorts can be implemented to start at either the most significant digit (MSD) or least significant digit (LSD). For example, with 1234, one could start with 1 (MSD) or 4 (LSD).
LSD radix sorts typically use the following sorting order: short keys come before longer keys, and then keys of the same length are sorted lexicographically. This coincides with the normal order of integer representations, like the sequence [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]. LSD sorts are generally stable sorts.
MSD radix sorts are most suitable for sorting strings or fixed-length integer representations. A sequence like [b, c, e, d, f, g, ba] would be sorted as [b, ba, c, d, e, f, g]. If lexicographic ordering is used to sort variable-length integers in base 10, then numbers from 1 to 10 would be output as [1, 10, 2, 3, 4, 5, 6, 7, 8, 9], as if the shorter keys were left-justified and padded on the right with blank characters to make the shorter keys as long as the longest key. MSD sorts are not necessarily stable.
Other than the traversal order, MSD and LSD sorts differ in their handling of variable length input. LSD sorts can group by length, radix sort each group, then concatenate the groups in size order. MSD sorts must effectively 'extend' all shorter keys to the size of the largest key and sort them accordingly, which can be more complicated than the grouping required by LSD.
However, MSD sorts are more amenable to subdivision and recursion. Each bucket created by an MSD step can itself be radix sorted using the next most significant digit, without reference to any other buckets created in the previous step. Once the last digit is reached, concatenating the buckets is all that is required to complete the sort.
Examples
Least significant digit Input list: [170, 45, 75, 90, 2, 802, 2, 66]
Starting from the rightmost (last) digit, sort the numbers based on that digit:
[{170, 90}, {2, 802, 2}, {45, 75}, {66}]
Sorting by the next left digit:
[{02, 802, 02}, {45}, {66}, {170, 75}, {90}]
Notice that an implicit digit 0 is prepended for the two 2s so that 802 maintains its position between them.
And finally by the leftmost digit:
[{002, 002, 045, 066, 075, 090}, {170}, {802}]
Notice that a 0 is prepended to all of the 1- or 2-digit numbers.
Each step requires just a single pass over the data, since each item can be placed in its bucket without comparison with any other element.
Some radix sort implementations allocate space for buckets by first counting the number of keys that belong in each bucket before moving keys into those buckets. The number of times that each digit occurs is stored in an array.
Although it is always possible to pre-determine the bucket boundaries using counts, some implementations opt to use dynamic memory allocation instead.
Most significant digit, forward recursive
Input list, fixed width numeric strings with leading zeros: [170, 045, 075, 025, 002, 024, 802, 066]
First digit, with brackets indicating buckets: [{045, 075, 025, 002, 024, 066}, {170}, {802}]
Notice that 170 and 802 are already complete because they are all that remain in their buckets, so no further recursion is needed
Next digit: [{ {002}, {025, 024}, {045}, {066}, {075} }, 170, 802]
Final digit: [ 002, { {024}, {025} }, 045, 066, 075 , 170, 802]
All that remains is concatenation: [002, 024, 025, 045, 066, 075, 170, 802]
Complexity and performance Radix sort operates in time, where is the number of keys, and is the key length in digits (e.g. 8-bit digits). LSD variants can achieve a lower bound for of 'average key length' when splitting variable length keys into groups as discussed above.
LSD Radix sort can run in linear time , where is the number of digits in each array element with each digit in the range of 0 to , such as 0 to 255 for an 8-bit digit. This holds when is constant, i.e. and is upper bounded by , i.e. . No matter how large the value of , it makes passes through the data.T. H. Cormen, C. E. Leiserson, R. L. Rivest, C. Stein, "Introduction to Algorithms", fourth edition, 2022, p. 213-214R. Sedgewick, K. Wayne, "Algorithms", fourth edition, 2011, p. 708-709 MSD Radix Sort uses just one pass in the best case, and in the worst case, it performs like LSD Radix sort.R. Sedgewick, K. Wayne, "Algorithms", fourth edition, 2011, p. 717
Optimized radix sorts can be very fast when working in a domain that suits them. They are constrained to lexicographic data, but for many practical applications this is not a limitation. Large key sizes can hinder LSD implementations when the induced number of passes becomes the bottleneck.
Performance of LSD radix sort correlates with the number of passes over the array, which is . Two passes over the array are performed per digit: one pass for counting, followed by the permutation pass. The number of passes can be reduced to by performing counting for all the digits of the key in a single pass. This is possible because the counts are always over the entire array, and permuting elements of the array does not change the resulting counts.
Specialized variants
In-place MSD radix sort implementations Binary MSD radix sort, also called binary quicksort, can be implemented in-place by splitting the input array into two bins - the 0s bin and the 1s bin. The 0s bin is grown from the beginning of the array, whereas the 1s bin is grown from the end of the array. The 0s bin boundary is placed before the first array element. The 1s bin boundary is placed after the last array element. The most significant bit of the first array element is examined. If this bit is a 1, then the first element is swapped with the element in front of the 1s bin boundary (the last element of the array), and the 1s bin is grown by one element by decrementing the 1s boundary array index. If this bit is a 0, then the first element remains at its current location, and the 0s bin is grown by one element. The next array element examined is the one in front of the 0s bin boundary (i.e. the first element that is not in the 0s bin or the 1s bin). This process continues until the 0s bin and the 1s bin reach each other. The 0s bin and the 1s bin are then sorted recursively based on the next bit of each array element. Recursive processing continues until the least significant bit has been used for sorting.R. Sedgewick, "Algorithms in C++", third edition, 1998, p. 424-427 Handling signed two's complement integers requires treating the most significant bit with the opposite sense, followed by unsigned treatment of the rest of the bits.
In-place MSD binary-radix sort can be extended to larger radix and retain in-place capability. Counting sort is used to determine the size of each bin and their starting index. Swapping is used to place the current element into its bin, followed by expanding the bin boundary. As the array elements are scanned the bins are skipped over and only elements between bins are processed, until the entire array has been processed and all elements end up in their respective bins. The number of bins is the same as the radix used - e.g. 16 bins for 16-radix. Each pass is based on a single digit (e.g. 4-bits per digit in the case of 16-radix), starting from the most significant digit. Each bin is then processed recursively using the next digit, until all digits have been used for sorting.
Neither in-place binary-radix sort nor n-bit-radix sort, discussed in paragraphs above, are stable algorithms.
Stable MSD radix sort implementations MSD radix sort can be implemented as a stable algorithm, but requires the use of a memory buffer of the same size as the input array. This extra memory allows the input buffer to be scanned from the first array element to last, and move the array elements to the destination bins in the same order. Thus, equal elements will be placed in the memory buffer in the same order they were in the input array. The MSD-based algorithm uses the extra memory buffer as the output on the first level of recursion, but swaps the input and output on the next level of recursion, to avoid the overhead of copying the output result back to the input buffer. Each of the bins are recursively processed, as is done for the in-place MSD radix sort. After the sort by the last digit has been completed, the output buffer is checked to see if it is the original input array, and if it's not, then a single copy is performed. If the digit size is chosen such that the key size divided by the digit size is an even number, the copy at the end is avoided.
Hybrid approaches Radix sort, such as the two-pass method where counting sort is used during the first pass of each level of recursion, has a large constant overhead. Thus, when the bins get small, other sorting algorithms should be used, such as insertion sort. A good implementation of insertion sort is fast for small arrays, stable, in-place, and can significantly speed up radix sort.
Application to parallel computing This recursive sorting algorithm has particular application to parallel computing, as each of the bins can be sorted independently. In this case, each bin is passed to the next available processor. A single processor would be used at the start (the most significant digit). By the second or third digit, all available processors would likely be engaged. Ideally, as each subdivision is fully sorted, fewer and fewer processors would be utilized. In the worst case, all of the keys will be identical or nearly identical to each other, with the result that there will be little to no advantage to using parallel computing to sort the keys.