GPUSVM: a comprehensive CUDA based support vector machine package

Qi Li 1 , Raied Salman 1 , Erik Test 1 , Robert Strack 1  and Vojislav Kecman 1
  • 1 Virginia Commonwealth University, Richmond, VA, 23284, USA

Abstract

GPUSVM (Graphic Processing Unit Support Vector Machine) is a Computing Unified Device Architecture (CUDA) based Support Vector Machine (SVM) package. It is designed to offer an end-user a fully functional and user friendly SVM tool which utilizes the power of GPUs. The core package includes an efficient cross validation tool, a fast training tool and a predicting tool. In this article, we first introduce the background theory of how we build our parallel SVM solver using CUDA programming model. Then we compare our GPUSVM package with the popular state of the art Libsvm package on several well known datasets. The preliminary results have shown one to two orders of magnitude speed improvement in both training and predicting phases compared to Libsvm using our Tesla server.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] Abe S., Support vector machines for pattern classification advances in pattern recognition, Springer-Verlag, London, 2005

  • [2] Cao L.J., Keerthi S.S., Ong C.J., Zhang J.Q., Periyathamby U., Fu X.J., Lee H.P., Parallel sequential minimal optimization for the training of support vector machines, IEEE T. Neural. Network., 17(4), 1039–1049, 2006 http://dx.doi.org/10.1109/TNN.2006.875989

  • [3] Catanzaro B., Sundaram N., Keutzer K., Fast support vector machine training and classification on graphics processors, Proceedings of the 25th International Conference on Machine Learning, 104–111, 2008

  • [4] Collobert R., Bengio S., Bengio Y., A parallel mixture of SVMs for very large scale problems, Neural Comput., 14(5), 1105–1114, 2002 http://dx.doi.org/10.1162/089976602753633402

  • [5] Cortes C., Vapnik V., Support-vector networks, Mach. Learn., 20(3), 273–297, 1995

  • [6] Dong J.X., Krzyzak A., Suen C.Y., A fast parallel optimization for training support vector machine, In: Machine Learning and Data Mining in Pattern Recognition, Perner, P. and Rosenfeld, A. (Eds.), Lect. Notes Comput. Sc., 2734, 96–105, 2003

  • [7] Fan R.E., Chen P.H., Lin C.J., Working set selection using second order information for training support vector machines, J. Mach. Learn. Res., 6, 1889–1918, 2005

  • [8] Frank A., Asuncion A., UCI machine learning repository, Irvine, CA: University of California, School of Information and Computer Science, 2010

  • [9] Graf H.P., Cosatto E., Bottou L., Durdanovic I., Vapnik V., Parallel support vector machines: the cascade SVM, In: Adv. Neur. In., Weiss, T., Schölkopf, B. and Platt, J. (Eds.), MIT Press, Cambridge, MA, USA, 17, 521–528, 2005

  • [10] Herrero-Lopez S., Williams J.R., Sanchez A., Parallel multiclass classification using SVMs on GPUs, Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units, 2–11, 2010

  • [11] Hillis W.D., Steele Jr. G.L., Data parallel algorithms, Commun. ACM, 29, 1170–1183, 1986 http://dx.doi.org/10.1145/7902.7903

  • [12] Huang G.B., Mao K.Z., Siew C.K., Huang D.S., Fast modular network implementation for support vector machines, IEEE T. Neural. Network., 16(6), 1651–1663, 2005 http://dx.doi.org/10.1109/TNN.2005.857952

  • [13] Huang T.M., Kecman V., Kopriva I., Iterative single data algorithm for kernel machines from huge data sets: theory and performance, Stud. Comp. Intell., 17, 61–95, 2006 http://dx.doi.org/10.1007/3-540-31689-2_3

  • [14] Hull J., A database for handwritten text recognition research, IEEE T. Pattern Anal., 16(5), 550–554, 1994 http://dx.doi.org/10.1109/34.291440

  • [15] Joachims T., Making large-scale support vector machine learning practical, Schölkopf, B. Burges, C.J.C. and Smola, A.J. (Eds.), 169–184, MIT Press, Cambridge, MA, USA, 1999

  • [16] Keerthi S.S., Shevade S.K., Bhattacharyya C., Murthy K.R.K., Improvements to Platt’s SMO algorithm for SVM classifier design, Neural Comput., 13(3), 637–649, 2001 http://dx.doi.org/10.1162/089976601300014493

  • [17] Lecun Y., Bottou L., Bengio Y., Haffner P., Gradient-based learning applied to document recognition, Proceedings of the IEEE, 86(11), 2278–2324, 1998 http://dx.doi.org/10.1109/5.726791

  • [18] Li Q., Kecman V., Salman R., A chunking method for Euclidean distance matrix calculation on large dataset using multi-GPU, In: 2010 9th International Conference on Machine Learning and Applications (ICMLA), Draghici, S., Khoshgoftaar, T.M., Palade, V., Pedrycz, W., Wani, M.A. and Zhu, X. (Eds.), Washington D.C., US, 208–213, 2010

  • [19] Li Q., Salman R., Kecman V., An intelligent system for accelerating parallel SVM classification problems on large datasets using GPU, In: 10th International Conference on Intelligent Systems Design and Applications (ISDA), Hassanien, A.E., Abraham, A., Marcelloni, F., Hagras, H., Antonelli, M. and Hong, T.-P. (Eds.), Cairo, Egypt, 1131–1135, 2010

  • [20] Osuna E., Freund R., Girosi F., An improved training algorithm for support vector machines, In: Neural Networks for Signal Processing, Principe, J., Gile, L., Nelson, M. and Wilson, E. (Eds.), 7th Proceedings of the 1997 IEEE Workshop, 276–285, 1997

  • [21] Platt J.C., Fast training of support vector machines using sequential minimal optimization, In: Advances in kernel methods — support vector learning, Schölkopf, B. and Burges, C.J.C and Smola, A.J. (Eds.), 185–208, MIT Press, Cambridge, MA, USA, 1999

  • [22] Vapnik V.N., The nature of statistical learning theory, Springer Verlag, New York, 2000

  • [23] Zanghirati G., Zanni L., A parallel solver for large quadratic programs in training support vector machines, Parallel Comput., 29(4), 535–551, 2003 http://dx.doi.org/10.1016/S0167-8191(03)00021-8

OPEN ACCESS

Journal + Issues

Open Computer Science is an open access, peer-reviewed journal. The journal publishes research results in the following fields: algorithms and complexity theory, artificial intelligence, bioinformatics, networking and security systems,
programming languages, system and software engineering, and theoretical foundations of computer science.

Search