Problem Statement

Sort an array of elements using the Shell sort algorithm, a diminishing increment sort. The Shell sort   (also known as Shellsort or Shell's method)   is named after its inventor, Donald Shell, who published the algorithm in 1959. Shell sort is a sequence of interleaved insertion sorts based on an increment sequence. The increment size is reduced after each pass until the increment size is 1. With an increment size of 1, the sort is a basic insertion sort, but by this time the data is guaranteed to be almost sorted, which is insertion sort's "best case". Any sequence will sort the data as long as it ends in 1, but some work better than others. Empirical studies have shown a geometric increment sequence with a ratio of about 2.2 work well in practice. [1] Other good sequences are found at the On-Line Encyclopedia of Integer Sequences.

Let's start with the solution:

shellsort := proc(arr)
	local n, gap, i, val, j;
	n := numelems(arr):
	gap := trunc(n/2):
	while (gap > 0) do #notice by 1 error
		for i from gap to n by 1 do
			val := arr[i];
			j := i;
			while (j > gap and arr[j-gap] > val) do
				arr[j] := arr[j-gap];
				j -= gap;
			end do;
			arr[j] := val;
		end do;
		gap := trunc(gap/2);
	end do;
end proc;
arr := Array([17,3,72,0,36,2,3,8,40,0]);
shellsort(arr);
arr;