A Lower Bound for Comparison Sort Pedro Ribeiro DCC/FCUP 2014/2015 Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 1 / 9
On this lecture Upper and lower bound problems Notion of comparison-based sort An Ω(n log n) worst-case bound for a deterministic comparison 2 player game view of algorithm analysis Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 2 / 9
Upper Bound Problem One typical question when designing algorithms is: given some problem X, can we construct an algorithm that runs in time O(f (n)) on inputs of size n? This can be seen as an upper bound problem, and our goal is to make g(n) as low as possible. Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 3 / 9
Lower Bound Problem In this lecture, the question is different: given some problem X, what is g(n) such that any algorithm must take time Ω(g(n)) on inputs of size n? This can be seen as an lower bound problem, and our goal is to make g(n) as high as possible. Lower bound help understand the intrinsic difficulty of the problem and how close we are to the best possible solution. Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 4 / 9
Lower Bound Problem For today we will consider the class of comparison-based sorting algorithms. These are sorting algorithms that operate only by comparing elements and moving them around based on the result of these comparisons Comparison-Based Sorting Algorithm A comparison-based sorting algorithm takes as input an array [a 1, a 2,..., a n ] with n items, and can only gain information about the items by comparing pairs of them. Each comparison ( is a i < a j? ) returns YES or NO. In the end, the algorithm must output a permutation of the input in which all items are in sorted order. Ex: QuickSort, MergeSort, HeapSort, InsertionSort, SelectionSort or BubbleSort are all comparison-based sorting algorithms. Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 5 / 9
Lower Bound for the Deterministic Case Theorem Any deterministic comparison-based sorting algorithm must perform Ω(n log n) comparisons to sort n elements in the worst case. We will now prove this theorem, by showing that given any deterministic comparison-based sorting algorithm A, for all n 2 there exists an input I of size n such that A makes at least log 2 (n!) = Ω(n log n) comparisons to sort I. Note that we need to prove for any possible deterministic algorithm, and not for a specific choice of pivot on QuickSort, or for a specific merge operation on MergeSort. Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 6 / 9
20 questions games Your goal is to determine the correct ordering of n elements (that you do not know in advance) Imagine you can ask me questions about the result of comparing a pair of elements I m answering this questions with the goal of delaying as much as possible your final answer How many questions do you need to ask? Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 7 / 9
Proving the Ω(n log n) bound There are n! permutations that could be the output of the algorithm For each of these permutations, there is an input for which that permutation is the only correct answer Let S be set of permutations that are consistent with the questions already made Initially, S = n! What happens when you start asking questions? Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 8 / 9
Proving the Ω(n log n) bound A comparison will answer YES or NO This answer will split the permutations still left in S in two groups Now suppose the adversary answering the question will always choose the answer leading to the largest group This means at each time we can only cut S by a factor of 2 By design, the algorithm can only stop when S = 1, so that it knows what permutation to output This means the algorithm must make log 2 (n!) questions! (i.e., comparisons) log 2 (n!) = log 2 (n)+log 2 (n 1)+log 2 (n 2)+...+log 2 (2) = Ω(n log n) (we could use Stirling s Approximation to prove this last step) Pedro Ribeiro (DCC/FCUP) A Lower Bound for Comparison Sort 2014/2015 9 / 9