In 1992, it was demonstrated that the Hamiltonian cycle problem (HCP) may be embedded in a Markov decision process (MDP). This breakthrough enabled a number of new theoretical and algorithmic developments for HCP. In particular, optimisation models which are equivalent to HCP were constructed. However, the development of numerical procedures based on these models has lagged the rapid development of new theory. In this monograph, we progress a number of new algorithmic approaches that take advantage of the MDP perspective. The work presented within is separated into three primary chapters, each describing a different algorithmic approach to solving HCP.