This book is aimed at providing a coherent, essentially self-contained, rigorous and comprehensive abstract theory of Feynman's operational calculus for noncommuting operators. Although it is inspired by Feynman's original heuristic suggestions and time-ordering rules in his seminal 1951 paper An operator calculus having applications in quantum electrodynamics, as will be made abundantly clear in the introduction (Chapter 1) and elsewhere in the text, the theory developed in this book also goes well beyond them in a number of directions which were not anticipated in Feynman's work. Hence, the second part of the main title of this book.The basic properties of the operational calculus are developed and certain algebraic and analytic properties of the operational calculus are explored. Also, the operational calculus will be seen to possess some pleasant stability properties. Furthermore, an evolution equation and a generalized integral equation obeyed by the operational calculus are discussed and connections with certain analytic Feynman integrals are noted. This volume is essentially self-contained and we only assume that the reader has a reasonable, graduate level, background in analysis, measure theory and functional analysis or operator theory. Much of the necessary remaining background is supplied in the text itself.