Comets have not only blazed across the celestial vault throughout human history, they have embellished the night sky since the Earth itself formed some 4.5 billion years ago. Comets were among the first-born solid bodies in the solar system, and their frozen nuclei tell of the primordial chemistry and chaos that ultimately resulted in the formation of the planets, the evolution of life and us. For all this, however, comets have long been celestial oddities: they baffled our distant ancestors, and human society continues to marvel and speculate wildly at their appearance even to the present day. Cutting against the perceived constancy of the stars, comets seemingly present themselves at random times and they are often interpreted as harbingers of terrestrial change - both good and ill. How then are we to tame the comet: where do they form, how do they move, and can their appearances be predicted? Such questions have preoccupied astronomers for centuries but comets have only gradually allowed the secrets of their wayward flight to be revealed. In this book I explore the historical struggle to understand not only the place of comets within a societal context, but also the scientific quest to make their paths amenable to mathematical certitude. The latter narrative is the more technical in content, but it took tactile form with the invention, in 1732, of the cometarium, a mechanical device to demonstrate the first two of Kepler's planetary laws. And, this development was later paralleled, in the mid to late 19th century, by the development of various mechanical devices (analog computers) to help obtain solutions to Kepler's Problem - the problem which asks, exactly where in its orbit is a given comet at some specific set time. The telling of the wayward comets story covers the past two millennia of human history, and it takes us from the phenomenological musings of Aristotle, through the exactitude of Newton's gravitational theory and calculus, to the truly incredible study of comet 67P/Churyumov-Gerasimenko, by the European Space Agency's Rosetta spacecraft, in the modern era.