Visual recognition remains an extremely challenging problem in computer vision. Most previous approaches have been evaluated on small datasets. However, ImageNet dataset with millions images for thousands classes poses more challenges for the next generation of vision mechanisms. Learning an efficient visual classifier and constructing a robust visual representation in a large scale scenario are two main research issues. In this book, we present how to tackle these issues. Firstly, a novel approach is presented by using several local descriptors to improve the discriminative power of image representation. Secondly, the state-of-the-art SVMs are extended by building the balanced bagging classifiers with sampling strategy and parallelizing the training process with several multi-core computers. Thirdly, the binary stochastic gradient descent SVM is developed to the new multiclass SVM for efficiently classifying large image datasets into many classes. Finally, when the training data cannot fit into computer memory, the training task of SVM becomes more complicated to deal with. This challenge is addressed by an incremental learning method for both large scale linear and nonlinear SVMs