Image Compression Based on Block SVD Power Method

Abstract In recent years, the important and fast growth in the development and demand of multimedia products is contributing to an insufficiency in the bandwidth of devices and network storage memory. Consequently, the theory of data compression becomes more significant for reducing data redundancy in order to allow more transfer and storage of data. In this context, this paper addresses the problem of lossy image compression. Indeed, this new proposed method is based on the block singular value decomposition (SVD) power method that overcomes the disadvantages of MATLAB’s SVD function in order to make a lossy image compression. The experimental results show that the proposed algorithm has better compression performance compared with the existing compression algorithms that use MATLAB’s SVD function. In addition, the proposed approach is simple in terms of implementation and can provide different degrees of error resilience, which gives, in a short execution time, a better image compression.


Introduction
Singular value decomposition (SVD) is a generalization of the eigen-decomposition used to analyze rectangular matrices. SVD plays an important role in many applications, and it is the most useful tool of linear algebra with several applications including image compression [23]; mathematical models in economics, physical processes, and biological processes; data mining applications; search engines to rank documents in very large databases, including the Web; image processing applications; etc. In addition, the use of SVD in image compression has been widely studied [2,13,25,27]. In this paper, we will study SVD applied to image compression.
Image compression is a type of data compression that involves encoding information in images using fewer bits than the original image representation. The main idea of image compression is reducing the redundancy of the image and transferring data in an efficient form [15]. Image compression takes an important place in several domains, like web design. In fact, maximally reducing an image allows us to create websites faster and saves bandwidth for users; it also reduces the bandwidth of the servers, and thus saves time and money. When talking about compression, we generally take into account two aspects: image size in pixels and the degree of compression. There is also another aspect: pixel depth or bit cost to represent each pixel. The nature of the image also plays a significant role. The main goal of such a system is to reduce the storage quantity as much as possible while ensuring that the decoded image displayed in the monitor can be visually similar to the original image as much as it can be.
Our main goal in this work is to find an algorithm that can involve encoding information in images using fewer bits than the original image representation. Toward this end, we developed and tested our algorithm for image compression based on various and real images.
The rest of this paper is structured as follows. We briefly introduce a review of previous related works in Section 2. Section 3 deals with the algorithm of SVD function. Section 4 describes the proposed system for image compression. The performance evaluation of the proposed algorithm is reported in Section 5. Finally, conclusions are drawn in Section 6.

Related Work
In recent years, numerous image compression schemes and their applications in image processing have been proposed [18][19][20]. In this section, a brief review of some important contributions from the existing literature is presented.
In general, there are two approaches for image compression: lossy or lossless [16,26]. A lossless compression is a kind of image compression method that allows no loss of data, and which retains the full information needed to reconstruct the original image. This type of compression is also known as entropy coding because of the fact that a compressed signal is generally more random than the original one and the patterns are removed when a signal is compressed. Lossless compression can be very useful for exact reconstruction of images. The compression ratio provided by this kind of method is not sufficiently high to be truly used in image compression. Lossless image compression is particularly useful in image archiving as in the storage of legal or medical records. The lossless image compression methods include run-length coding, bit-plane coding, Huffman coding [1], LZW (Lempel-Ziv-Welch) codes, and entropy coding.
Lossy compression is another type of image compression technique in which the original signal cannot be exactly reconstructed from the compressed data. The reason behind this is that much of the detail in an image can be discarded without greatly changing the appearance of the image. In lossy image compression, even very fine details of the images can be lost, but ultimately, the image size is drastically reduced. Lossy image compressions are useful in many applications such as broadcast television, video conferencing, and facsimile transmission, in which a certain amount of error is an acceptable trade-off for increased compression performance. Among methods for lossy compression, we find fractal compression [14], transform coding, Fourier-related transform, discrete cosine transform [5,17], and wavelet transform.
Generally, SVD is a lossy compression technique that achieves compression by using a smaller rank to approximate the original matrix representing an image. Furthermore, lossy compression yields good compression ratio compared with lossless compression while lossless compression gives good-quality compressed images [10].
When we give the definition of lossless or lossy methods, it is necessary to clarify that near-lossless algorithms are theoretically lossless; however, they may suffer from numerical floating-point accuracy reconstruction issues.
According to the state of the art, several works suggested using the SVD with other compression methods or with variations of SVD [e.g. Shuffled Singular Value Decomposition (SSVD) [22], Joint Singular Value Decomposition (JSVD), and K-Singular Value Decomposition (KSVD)]. Awwal et al. [3] presented a new compression technique using SVD and the wavelet difference reduction (WDR). The WDR is used for further reduction. This technique has been tested with other techniques such as WDR and JPEG2000, and gives a better result than the other techniques. Furthermore, using WDR with SVD enhances the peak signal-to-noise ratio (PSNR) and compression ratio.
A technique based on wavelet-SVD, which uses a graph coloring technique in the quantization process, is presented in Ref. [21]. This technique worked well and enhanced the PSNR and compression ratio. The generated compression ratio by this work ranged between 50% and 60%, while the average PSNR ranged between 40 and 80 db.
Ranade et al. [22] suggested a variation of SVD-based image compression. This approach is a slight modification to the original SVD algorithm, which gives much better compression than the standard compression using the SVD method. In addition, it performs substantially better than the SVD method. Typically, for any given compression quality, this approach needs about 30% fewer singular values and vectors to be retained.
Bryt and Elad [6] suggested a method for compressing facial images using the KSVD algorithm, which is an algorithm for training over complete dictionaries that lead to sparse signal representations. This method was applied to grayscale images and shown to be most efficient, surpassing the JPEG2000 performance significantly. This work is extended in Ref. [7]. Doaa and Chadi [9] proposed block truncation coding (BTC), it is an image compression method proposed by . It is one of the simplest and easiest image compression algorithms, and also an efficient image coding method that has been adopted to obtain the statistical properties of a block in the compressed image.
The technique given by El Abbadi et al. [10] proposes to use SVD and moment preserving quantizer-BTC. Using this method, the input image is compressed by reducing the image matrix rank, by using the SVD process and then the result matrix compressed by using BTC.
The technique presented by El Asnaoui et al. [11] introduces two new approaches. The first one is an improvement of the BTC method that overcomes the disadvantages of the classical BTC, while the second one describes how to obtain a new rank of SVD method, which gives a better image compression. This method is extended in Ref. [12].
Following the same objective of image compression using SVD, the most important problem is which K rank to use for giving a better image compression. For this reason, El Asnaoui et al. [11] proposed a new technique that gives a maximum value for K, which provides better compression.
To the best of our knowledge, this is the first work to suggest an image compression method based on the block SVD power method.

Singular Value Decomposition (SVD)
As we have earlier mentioned, the main goal of studying the SVD of an image (matrix of m × n) is to create approximations of an image using the least amount of the terms of the diagonal matrix in the decomposition. This approximation of the matrix is the basis of image compression using SVD, as images can be viewed as matrices with each pixel being an element of a matrix.
In mathematics, the linear algebra of the SVD method of a matrix is an important tool for factorization of real or complex rectangular matrices.
Let us say we have a matrix A with m rows and n columns, with rank r and r ≤ n ≤ m. Then, A can be factorized into three matrices: U, ∑︀ , and V T such that [17] A = U ∑︀ V T , where U and V are orthogonal, ∑︀ is a diagonal matrix, and U ∑︀ V T is the closest rank K approximation to A (see Figure 1). The purpose of transforming matrix A into U ∑︀ V T is to approximate matrix A by using far fewer entries than in the original matrix. By using the rank of a matrix, we remove the redundant information (the dependent entries). SVD can be used to provide the best lower-rank approximation to matrix A and thus be used for image compression.
Then, there is a factorization of the form * r is the maximum rank of the matrix A such that r ≤ min(n, m).

SVD vs. Memory
First, we need to know how much memory an original image I requires.
For the image of m × n pixels in grayscale, the memory has to store m × n values, one value for each pixel: SVD is composed of three matrices: U, ∑︀ , and V. Originally, U is a matrix of m×m; however, we only want the first columns of K. Then, U M = m × K. Similarly, we only want the first K columns of V (which become the V T lines); thus, we only need to store V as an n × K matrix with values n × K, then: Finally, because we exploit the first K columns of U and V, we need only the first K singular values: ∑︀ M = K. Now, it is possible to calculate the total number of values needed to store this K rank. This rank was determined by using this formula [11]: where m and n are the size of original image.

Algorithm of SVD
The main idea of this section is to present two algorithms: the first one is MATLAB's SVD function, while the second one describes how to obtain a new SVD using the block SVD power method.

Block SVD Power Method [4]
The main goal in this section is to give a block iterative algorithm that computes the SVD. The idea is based on the technique used in the block power method. From a block-vector V (0) ∈ R m×s , we construct two blockvector sequences, V (k) ∈ R m×s and U (k) ∈ R n×s , which converge, respectively to the s first right and left singular vectors corresponding to singular values σ 1 ≥ σ 2 ≥ . . . ≥ σ s .

Input:
A matrix A∈ R n×m , a block-vector V = V (0) ∈ R m×s and a tolerance tol

Proposed Lossy Image Compression Technique
As it is mentioned in the related work, the SVD-based compression is lossy due to the nature of the process.
The contribution of this paper is the introduction of the concept of application of the block SVD power method to image compression, as the main idea of image compression is reducing the redundancy of the image and transferring data in an efficient form. As we have earlier mentioned, this work is the first one that proposes an application of the block SVD power method to image compression. Most of the methods focus on other methods and other variations of SVD. Moreover, our method is novel and efficient for solving our problem. It is general, and many other computer visions can benefit from using it. In this section, we propose our contribution in which we integrate the block SVD power method and adopt it to create an algorithm that compresses an image. Figure 2 shows the main pipeline of the proposed method.
When the SVD is applied to an image, it is not compressed, but the data take a form in which the first singular value has a great amount of image information. With this, we can use only a few singular values to represent the image with little differences from the original. The input image can be a color image with RGB color components or may be a grayscale image. Furthermore, for creating a new image with MATLAB's SVD function, as indicated in Figure 2, we use the following: Our contribution in this paper is to set up a new algorithm for image compression that overcomes some inconveniences encountered in existing methods that use MATLAB's SVD function. Our modification consists of computing the SVD for each component, in which the entries in image I are computed using the block SVD power method obtained by Bentbib and Kanber [4] instead of MATLAB's SVD function, and keeps the K rank determined by Eq. (1).
Thus, for the same compression, we have a better quality. We also provide a heuristic argument to justify our experimental finding.
In the next section, the experimental results are reported. The results clearly show the superiority of the proposed lossy image compression technique over MATLAB's SVD function and some different compression techniques in the state of the art.

Experimental Results
Our work is aimed at image compression. For this purpose, our experiments were performed on several images available on Windows 7 Professional and numerical examples. Simulations were done in MATLAB 2009a using a personal computer with Processor: Intel(R) Core (TM) 2 CPU T5200 @ 1.60 GHz, 1.60 GHz, 2Go RAM running on a Microsoft Windows 7 Professional (32-bit). In addition, the results and discussion of the proposed method are given in this section.

Parameters for Comparison
To evaluate the performance of the proposed method, the quality of the image is estimated using several quality measurement variables, like mean square error (MSE), root MSE (RMSE), and PSNR. These variables are signal fidelity metrics and do not measure how viewers perceive the visual quality of an image [3,21,22].

Compression Ratio
The degree of data reduction obtained by a compression method can be evaluated using the compression ratio (Q comp ), which is defined by the following formula:

MSE
The MSE for two m × n monochrome images I and J, where one of the images is considered noisy approximation of the other, is defined as follows:

RMSE
RMSE is one of many ways to quantify the difference between values implied by an estimator and the true value of the quantity being estimated. The use of RMSE is very common, and it makes an excellent general-purpose error metric for numerical predictions. This parameter is given as follows: A lower RMSE indicates that the model is better fitting.

PSNR
PSNR is measured in decibels (dB), and is only meaningful for data encoded in terms of bits per sample or bits per pixel. For example, an image with 8 bits per pixel contains integers from 0 to 255. PSNR is given by the following equation: A high PSNR value indicates that there is less visual degradation in the compressed image.

Numerical Examples
The results of the image compression depend strongly on the goodness of the algorithm to compress an image.
To illustrate it, first we checked if the chosen block SVD power method correctly detects the relative errors occurring when computing the singular values and CPU time. For this purpose, we did several tests where we chose some numerical examples.
We have compared and tested in this section the numerical results obtained by the algorithm [4] with MATLAB's SVD function. Toward this end, let A ∈ R n*m be a rectangular matrix defined as follows: A ∈ Q ∑︀ U T , where Q and U are random orthogonal matrices. We give below the relative errors occurring when computing the singular values and the CPU time. The started block-vector in the algorithm [4] is given by V = V(0) = eye(m, s) (Matlab notation). The results are given from the algorithm [4] after only at most K = 1 iteration. We have stopped the algorithm [4] whenever the error of the reduction, err = ‖AV − U ∑︀ ‖, is smaller than that achieved by MATLAB's SVD function (see Figures 3  and 4).

Image Compression
To test our method, we have developed a user interface. The method was applied to various and real images to demonstrate the performances of the proposed image compression algorithm.
This proposed algorithm is tested with many images, some of them are shown in Figures 5-10. Indeed, we used in this paper only two color images, Chrysanthème and Desert available on Windows Professional (32bit), and one in grayscale (see Figure 5). Furthermore, we have used Pepper and lena (see Figure 10). Figures 6,  7, 10, and 11 show the test images and the result of compressed images using MATLAB's SVD function [11] and the proposed compression method. From Figures 6, 7, 10, and 11, it is clear that the compressed images are perceptually similar to the original images.
We recall that our goal is to approximate an image (matrix of m × n) using the least amount of information. Thereby, to obtain a better quality of the compressed image using SVD, we use the K rank determined by El Asnaoui et al. [11] [Eq. (2)].
The compression ratio and the variation of PSNR with image rank when we apply MATLAB's SVD function [11] and the proposed compression method are showed in Figures 8, 9, and 12.

Application to Color Image
To evaluate the performance of the proposed method, we apply MATLAB's SVD function [11] and the new method to color image. After rank K = 438, we obtain (see

Application to Other Images
The proposed algorithm has also been tested with Pepper and lena images (see Figure 10). After K rank, we obtain the results shown in Table 1.

Application to Grayscale Image
In order to compare this performance, we also applied MATLAB's SVD function [11] and the new method to the grayscale image. After rank K = 548, we obtain (see Figure 11):

Comparison against Various Algorithms
To evaluate the robustness of our method, we have tested it with different compression techniques [7,[9][10][11].
The additional experimental results for three images are listed in Table 2.

Discussion
This study sets up a new algorithm for image compression that must be considered as an application of the block SVD power method to image compression. With our preliminary experimental analysis, observation, and evaluation, the proposed method achieves promising performance and provides a good PSNR. In addition, the proposed algorithm is compared with MATLAB's SVD function and other state-of-the-art algorithms.
The numerical examples given above show the efficiency of the new SVD [4] approach in computing the decomposition, error, and CPU time.
When applying the new method to image compression (see Figures 6,7,10,and 11), it is clear that the images compressed by two approaches are perceptually similar to the original images. However, the human visual response to image quality is insufficient.
To compare the performances of the proposed method, several values were used in this study to measure the quality of the compressed image. We will only discuss PSNR and MSE values, because they are used to compare the squared error between the original image and the reconstructed image. There is an inverse relationship between PSNR and MSE. Therefore, a higher PSNR value indicates higher quality of the image.
The above analysis shows the comparison when MATLAB's SVD function [11] and the proposed method are applied on real images. In these experiments, we used the K rank for different images. We see in this case that the compression ratio, PSNR, and other values of images varied when changing the rank of the   image during the SVD process, as shown in Figures 8, 9, and 12. Moreover, it is evident that the proposed method gives better performance compared to MATLAB's SVD function [11]. In addition, for MATLAB's SVD function [11], the value of K, which provides better PSNR value, is the maximum value of K = 438, while for the proposed method, a better compression ratio, PSNR, is provided from K = 150 for color images (see Figure 8). We can say that in this case, MATLAB's SVD function approximately presents one-third of the proposed method in terms of K rank. For Figure 9, the value of K that provides better PSNR value is the maximum value of K = 438, while for the proposed method, a better compression ratio, PSNR, is provided from K = 200.
Concerning the grayscale image tested (see Figure 11), it seems that the value of K that gives better PSNR value is the maximum value of K = 548, while for the proposed method, a better compression ratio, PSNR, is provided from K = 400 (see Figure 12).
We mainly compared the proposed algorithm with the other algorithms as illustrated in Table 2, because these algorithms are well known and are mainly using MATLAB's SVD function. Hence, we see that our proposed algorithm performs comparably to current state-of-the-art techniques and is able to produce a compressed image with better visual quality, as indicated by its PSNR.

Conclusion
We suggest in this work a novel lossy image compression technique by using the block SVD power method. This approach is simple in terms of implementation, and can be used to overcome the limitations of existing algorithms that use MATLAB's SVD function. The results obtained by the proposed technique are compared with those of several state-of-the-art image compression techniques, and indicated that the proposed approach might be considered as a solution for the development of image compression. Satisfactory compression of expected images is provided faster due to the lower number of iterations in the compression algorithm. Of course, MATLAB's SVD function is accurate. However, in numerical analysis, we always focus on improving results.

Future Works
We believe that this kind of block SVD power method opens the door to many other applications of this method in the future. For example, using this method for statistical applications to find relations between data, in the area of medical image denoising with different thresholding techniques associated with these multi-wavelets, implements a compression technique using neural network. It is also useful with other techniques in image restoration.
In addition, our future approach, which is defined in parallel with the Golub-Kahan method, will be to present a JSVD decomposition of a 2n × 2m rectangular matrix A based on the J-bidiagonal matrix computed by using symplectic reflectors. The obtained J-bidiagonal matrix is transformed to a diagonal matrix using symplectic Givens rotations. It allows us to compute the eigen-values of the skew-Hamiltonian matrix A J A.