Insertion Sort

Insertion sort is a simple $$O(n^2)$$ sorting algorithm that uses $$O(1)$$ memory. It essentially works by "inserting" every element into the list one at a time.

Insertion sort is adaptive, meaning it takes less time on almost-sorted arrays than random ones. It is also stable, which means it preserves the relative order of equal items.

Algorithm
Initially, the sort only considers the first element. In this case, the sublist is already sorted by definition.

Then, the sort will start inserting elements into the already sorted sublist. To do this, it compares the element with the previous element in the sublist, starting at the element in the sublist closest to the current one, until it finds an element in the sublist that is not greater than the current one. Then, it simply inserts the element into the sublist so that it goes just before the one we found.

Laconic
Insertion sort moves an item backwards until it meets with a lesser item, then repeats the process with the next one until finished.

Example
Consider the array:

Start at the second element (3): [7|3 6 5 1 4 2]

Insert it into its correct position: [3 7|6 5 1 4 2]

Insert the third element (6) into its correct position: [3 6 7|5 1 4 2]

Insert 5 into its correct position: [3 5 6 7|1 4 2]

Insert 1 into its correct position: [1 3 5 6 7|4 2]

Insert 4 into its correct position: [1 3 4 5 6 7|2]

Insert 2 into its correct position: [1 2 3 4 5 6 7]

Sorted!

Pseudocode
The process that the sort uses to insert the elements is writes-optimal and is often misimplemented, resulting in a totally new algorithm (see optimized gnome sort). It saves the element to be inserted into an auxiliary variable, writes every element before it up to the position one element up, and finally writes the element to be inserted in its correct spot. This takes exactly $$n+1$$ writes, where $$n$$ is the amount of spaces back the element needs to be pushed.

procedure insertion_sort(array arr) do # Arrays are 0-indexed, so this starts at the second element for index = 1 to length(arr)-1 do elem element = arr[index] int i = index-1 while i >= 0 and arr[i] > elem do arr[i+1] = arr[i] i -= 1 end while arr[i+1] = element end for end

Gnome sort
It is similar to insertion sort, but it does not use nested loops. It works by comparing the element to the next one, initially at the starting point. When the comparison is in order, move to the next element by one step and compare again, otherwise, swap and move to the previous element. Do this until the end of the list is reached.

procedure gnome_sort(array arr) do index = 0 while index + 1 < length(arr) do if arr[i] > arr[i+1] do swap arr[i] and arr[i+1] index -= 1 if index < 0 do index = 0 end if else index += 1 end if end for end

Tiny Gnome sort
The sorting algorithm above can be made shorter. Whenever a swap occurs, it goes back to the beginning of the list instead, to compare-and-swap again, moving rightward. It does this until the following array is sorted. This algorithm runs in $$O(n^3)$$ average and worst case.

Optimized gnome sort
Optimized gnome sort is a variant of insertion sort that is arguably simpler but is less writes effective. The only difference between regular insertion sort and optimized gnome sort is the insertion process. Rather than doing what is described above, it simply swaps the element to be inserted with the previous one until it is in its place. This results in the same permutation of the array but takes more data moves ($$3n$$ instead of $$n+1$$).

Binary insertion sort
Binary insertion sort is a variant of insertion sort that usually takes less comparisons than standard insertion sort. Instead of using a linear search, it uses a binary search to find where the element must be inserted. This results in a search over $$n$$ elements taking $$\log_2 n$$ comparisons, which makes it better on most arrays but a little worse on almost-sorted arrays.