The paper obtains two principal results. First, using a new definition of higher-order (>2) matrix derivatives, the paper derives a recursion for computing any Gaussian multivariate moment. Second, the paper uses this result in a perturbation method to derive equations for computing the 4th-order Taylor- series approximation of the objective function of the linear-quadratic exponential Gaussian (LQEG) optimal control problem. Previously, Karp (1985) formulated the 4th multivariate Gaussian moment in terms of MacRae's definition of a matrix derivative. His approach extends with difficulty to any higher (>4) multivariate Gaussian moment. The present recursion straightforwardly computes any multivariate Gaussian moment. Karp used his formulation of the Gaussian 4th moment to compute a 2nd-order approximation of the finite-horizon LQEG objective function. Using the simpler formulation, the present paper applies the perturbation method to derive equations for computing a 4th-order approximation of the infinite-horizon LQEG objective function. By illustrating a convenient definition of matrix derivatives in the numerical solution of the LQEG problem with the perturbation method, the paper contributes to the computational economist's toolbox for solving stochastic nonlinear dynamic optimization problems.