Hilbert space theory is an invaluable mathematical tool in numerous signal processing and systems theory applications. Hilbert spaces satisfying certain additional properties are known as Reproducing Kernel Hilbert Spaces (RKHSs). This primer gives a gentle and novel introduction to RKHS theory. It also presents several classical applications. It concludes by focusing on recent developments in the machine learning literature concerning embeddings of random variables. Parenthetical remarks are used to provide greater technical detail, which some readers may welcome, but they may be ignored without compromising the cohesion of the primer. Proofs are there for those wishing to gain experience at working with RKHSs; simple proofs are preferred to short, clever, but otherwise uninformative proofs. Italicised comments appearing in proofs provide intuition or orientation or both. A Primer on Reproducing Kernel Hilbert Spaces empowers readers to recognize when and how RKHS theory can profit them in their own work.