This book presents how a two-class discrimination model with or without prior knowledge can be extended to the case of multi-categorical discrimination with or without prior knowledge. The prior knowledge of interest is in the form of multiple polyhedral sets belonging to one or more categories, classes, or labels, and it is introduced as additional constraints into a classification model formulation. The solution of the knowledge- based support vector machine (KBSVM) model for two- class discrimination is characterized by a linear programming (LP) problem, and this is due to the specific norm (L1 or L8) that is used to compute the distance between the two classes. The proposed solutions to a classification problem is expressed as a single unconstrained optimization problem with (or without) prior knowledge via a regularized least squares cost function in order to obtain a linear system of equations in input space and/or dual space induced by a kernel function that can be solved using matrix methods or iterative methods.