Revisiting Weighted Stego-Image Steganalysis

Transcription

1 Revisiting Weighted Stego-Image Steganalysis Andrew D. Ker a and Rainer Böhme b a Oxford University Comuting Laboratory, Parks Road, Oxford OX 3QD, England; b Technische Universität Dresden, Institute for System Architecture, 0062 Dresden, Germany ABSTRACT This aer revisits the steganalysis method involving a Weighted Stego-Image (WS) for estimating LSB relacement ayload sizes in digital images. It suggests new WS estimators, ugrading the method s three comonents: cover ixel rediction, least-squares weighting, and bias correction. Wide-ranging exerimental results (over two million total attacks) based on images from multile sources and re-rocessing histories show that the new methods roduce greatly imroved accuracy, to the extent that they outerform even the best of the structural detectors, while avoiding their high comlexity. Furthermore, secialised WS estimators can be derived for detection of sequentially-laced ayload: they offer levels of accuracy orders of magnitude better than their cometitors. Keywords: Steganalysis, Weighted Stego-Image, LSB Relacement, Benchmarking, Sequential Embedding. INTRODUCTION One amongst many steganalysers for Least Significant Bit (LSB) relacement in digital images, the method involving a Weighted Stego-Image, commonly identified by the acronym WS, has received little attention beyond a simle extension to multile bit relacement. 2 This is robably because its erformance is weaker than the state-of-the-art detectors based on analysis of structure. 3 7 Consequently, WS has not rofited from as many refinements as the class of structural detectors. The standard WS method is not without advantages: it does not use the same ixel grou analysis as the structural detectors and its erformance is best recisely when the structural steganalysers are weakest (estimation of ayload size for near-maximal messages 8 ). Furthermore, unlike the structural detectors it retains its estimation accuracy when embedding changes are not distributed evenly over the cover. 9 The aim of this aer is to reconsider the WS method, first recaitulating the method of Ref. (Sect. 2) then offering imrovements to each comonent of the WS rocedure (Sect. 3), and finally extending the same techniques to roduce secialised detection and size estimation of sequentially- instead of randomly-located ayload (Sect. 4). The imroved methods are benchmarked thoroughly (Sect. 5), first to determine which of the novel variants is the best erformer, and then to comare with their leading ayload estimation cometitors. Imrovements to the basic WS method show notable erformance gains, outerforming the best structural detectors in a domain where they were reviously believed more reliable, and sequential WS-based ayload estimators will be demonstrated to be sectacularly accurate. 2. WEIGHTED STEGO-IMAGE DETECTORS We begin with a brief exosition of the WS method described in Ref., simlifying some of the terminology, followed by a short examination of its erformance when comared with structural steganalysis methods. The aim of the WS method is to estimate the size of ayload, ossibly zero, embedded by LSB relacement in a digital image. In this it is similar to most of the other sensitive detectors of LSB relacement. 3 7 Suose that a cover image consists of a vector of N samles (e.g. ixel intensities in a single-channel image) c = (c,..., c N ) and that a ayload of length M N bits is embedded by LSB relacement. We suose that the ayload Further author information: A. D. Ker: adk@comlab.ox.ac.uk, Telehone: R. Böhme: rainer.boehme@tu-dresden.de, Telehone:

2 is uncorrelated with the cover (if the embedding locations are generated seudorandomly, or if the ayload is comressed or encryted before embedding, this is likely to be the case). We write s = (s,..., s N ) for the stego image and, for α [0, ], s α for the real-valued sequence formed by taking a weighted average between the stego image and the stego image with every samle s LSB flied: s α i = αs i + ( α)s i, () where x denotes the nonnegative integer x with the least significant bit flied, x = x + ( ) x. The sequence s α is called the weighted stego-image and the key to the WS ayload estimator is Theorem of Ref., which demonstrates that the weighted stego image is closest to the cover image (if difference between the two vectors c and s α is measured using the Euclidean L 2 -norm) when α = M/2N. This corresonds to the exectation that roortion M/2N of the cover ixels are flied when embedding a ayload of length M. Of course, in steganalysis we do not have access to the cover as well as the stego object, so we cannot find the value of α which exactly minimizes distance from c, but the WS method attemts to estimate the cover image by filtering the stego image. We will write F(s) for the filtered stego image: in Ref. each cover samle is estimated by taking the mean of the surrounding four stego ixels. It is demonstrated that, for correctness of the WS method, each cover ixel estimate F(s) i must not deend on the corresonding stego ixel s i, only on other stego ixels. Thus we can describe the standard WS method. Given an image which might be a cover or stego image, the ayload size (ossibly zero) is estimated by finding α to minimize the distance D(α) = N ( ) s α 2. i F(s) i Differentiating D, it is shown that the roortionate ayload length = M/N is estimated by ˆ = 2 argmin D(α) = 2 α N N ( ) si F(s) i (si s i ). This can be comuted in O(N) oerations, i.e. the estimation comlexity is linear in the image size. In Ref., two enhancements are included. First, the distance between weighted stego and cover can itself be weighted so that ixels in noisy areas for which rediction of the cover is more difficult are given less weight than those in flatter areas. The distance minimized is w i (s α i F(s) i) 2. It is suggested that the weights should be roortional to /( + σ 2 i ), where σ2 i is the local variance of the 4 ixels surrounding stego ixel i. The estimator still has a closed form, now ˆ = 2 argmin α N ( ) w i s α 2 N i F(s) i = 2 w i (s i F(s) i )(s i s i ), (2) where the weight vector w is scaled so that w i =. (No weighting corresonds to w i = /N for each i.) Ref. demonstrates that this imroves accuracy in some test images. The estimate is still comuted in linear time. Second, some outliers are analysed and it is demonstrated that flat ixels (areas of uniformity) in the cover are to blame: such areas receive a high weighting, and furthermore they create a ositive bias in the estimator. A flat ixel correction term is derived, with the aim of reducing outliers. We will not reeat its derivation. 2.. Performance As a starting oint and motivation for the modifications resented in this aer, we include a summary of erformance, comaring the WS methods in Ref. with one of the earliest structural estimators,, 3 and the most recent refinement, Coules/ML. 7 For roortionate ayload size = 0, 0.05,...,, we simulated randomly-sread LSB relacement in a set of 3,000 scanned greyscale cover images and measured the accuracy

3 Samle Pairs Coules/ML WS unweighted WS weighted WS with flat ixel correction Figure. Mean absolute error of structural and WS estimators described in Ref., observed in 3,000 scanned greyscale images, as a function of the true embedding rate. The y-axis is on a log scale. Lower values denote better erformance. of the ensuing estimates. There is no uniformly best measure of estimator erformance, 8 absolute error (MAE) a reasonable overall measure in Fig.. but we dislay mean Observe that weighting usually imroves the erformance of the WS method, and that flat ixel correction only reduces erformance. The structural detectors aear generally suerior, but their erformance is weaker for larger ayloads. (This seems to be a feature of structural detectors, for reasons exlained in Ref. 0.) The weighted WS method is the most accurate estimator for roortionate ayloads of over about 75 % and its erformance is also quite good for very small ayloads, although not enough to match the very sensitive (and comutationally exensive) Coules/ML method. Therefore we conclude that the WS method does have a lace in steganalysis, and it is valuable to study it further. 3. IMPROVED WS DETECTORS We roose imrovements to all arts of the WS method the cover ixel estimation, the weighting factors for WS image fitting as well as a substantially more general bias correction to relace the flat ixel correction term from Ref.. Each comonent will be considered searately. 3.. Enhanced Pixel Predictors First, can we make a better redictor for the cover ixels than the very simle one in Ref.? Taking the average of four surrounding ixels amounts to comuting the convolution of the stego image with the two-dimensional filter in (3), so it is natural to consider other filters, for examle (4). However, this redictor based on eight neighbours has oor exerimental erformance (3) (4) (5) b a b a 0 a b a b (6) e d c d e d b a b d c a 0 a c d b a b d e d c d e (7) An imortant factor, not visible in the figure, is that the estimator can fail to roduce an estimate at all. Such failures were excluded from the MAE calculation, so the figure flatters the detector a little. However the occurrence of such failures is limited to large ayloads: 50 % of estimates fail for maximal ayloads =, but the rate reduces raidly for smaller ayloads. In all our tests, there were zero failures for 0.6 and only a 2 % failure rate at = 0.8.

4 In order to find a more accurate redictor, we first observed that the ideal filter should have symmetry, to reflect invariance of natural image roerties, and then tested those of the form (6) on a library of cover images. We found the values of a and b to minimize the L 2 distance between the redicted cover and the true cover: rounding the numbers slightly, the otimum mask of this shae aears to be given by (5). All four filters: (3), (4), (5), and also a 5 5 convolution filter determined using the same method (not dislayed) will be tested in Sect. 5. But the otimal redictor might deend on the articular image being analysed so, insired by the technique used to find a good fixed filter, we roose an adative method. For a given stego image, a filter of the form (6) can be alied to determine the values of a and b which best redicts its ixels; it is sensible to use a weighted least-squares regression here. For that image, this gives an otimal redictor of stego image ixels, and we exect that it is also a good redictor of cover image ixels because the LSB relacement rocess should cause errors of ±, in the ixels used for rediction as well as that redicted, aroximately equally often. Then, the standard (weighted or unweighted) WS method is alied with rediction by the learned filter. A similar technique can be alied to 5 5 filters, using the symmetrical attern in (7). With an adative filter the comutation time is aroximately doubled, as an initial ass through the image is required to determine the filter. WS based on adative 3 3 and 5 5 filters will be benchmarked in Sect Enhanced Weighting Factors Weighting the distance between stego and cover imroves the accuracy of the detector considerably. However, the choice of good weights is not very well understood and we offer only a heuristic imrovement. Ref. rooses that the weights should be of the form (8). w i + σ 2 i (8) w i 5 + σ 2 i (9) Our exeriments suggested that, although these weights roduce a low weighted MSE between redicted cover and true cover, they over-emhasise flatter areas in the image too much: in areas of low noise, the stego noise forms a significant art of the redictor error. Instead, w i should be moderated so that higher weights are reduced: weights of the form (9) give better ayload estimators, and will be benchmarked in Sect Bias Correction We have demonstrated that the flat ixel correction, derived in Ref., does not imrove erformance in (some) images. However systematic bias is a genuine feature of the WS method in some covers, and we seek a better way to correct it. Our analysis begins like that in Ref. but then diverges, roducing a bias correction term which is both simler and better justified than the original. To reason about bias, let us consider the exected ayload estimate under the (weighted) WS method: E[ˆ] = 2 N (si ) ] w i E[ F(s) i (si s i ) = 2 N ( w i E[ (s i c i ) + ( ) ( ) ) ] c i F(c) i + F(c)i F(s) i (s i s i ). (0) This breaks down into three arts: the true relative ayload size, bias due to inaccurate rediction, and bias arising from the filtered stego noise. In the first term of (0), E [ (s i c i )(s i s i ) ] = /2 because roortion /2 of c i are equal to s i and the rest are equal to s i, so the contribution of 2 w i E [ (s i c i )(s i s i ) ] is w i =, the correct estimate. In the (ci ) ] second term, E[ F(c) i (si s i ) = 0 if the redictor error (in covers) is uncorrelated with the arity of the stego ixels (a reasonable assumtion), so this contributes no bias. But in the third term (F(c)i ) ] (F(c ) ] E[ F(s) i (si s i ) = E[ s)i (si s i ) (assuming a linear filter) is only zero if the filtered added stego signal s c is uncorrelated with the corresonding stego ixel. This is false if there is arity co-occurrence between neighbours in the cover: suose that the ixel

5 c i is even, and so are many of its neighbours. Then c s will exect more negative values than ositive values near ixel i, so the same will be true for F(c s) i. To quantify this bias term, we imagine that it is the stego image which is fixed and the cover which was generated by randomly fliing roortion /2 of LSBs. (This disregards certain conditional robabilities in the structure of the cover, but that is not very significant.) Then we can simlify the exected bias, the third art of (0), to b = 2 w i (s i s i ) E [ F(c s) i ] = wi (s i s i )(F(s s) i ) () since the filter is linear, and (c s) i is (s s) i with robability /2 and zero otherwise. Given an initial estimate of, one can subtract the exected bias b to make the estimate more accurate. We will see, in Sect. 5, that this makes a substantial imrovement to the accuracy of the estimator in covers where there is strong arity co-occurrence between neighbouring ixels in the cover image. 4. SPECIALIZING THE WS METHOD FOR SEQUENTIAL EMBEDDING Most of the sensitive detectors for LSB relacement assume that the embedding changes are sread uniformly through the cover. When this is not the case, they fail. The WS method has two advantages here: first, it works with aroximately equal accuracy on sequential embedding as sread embedding; second, it can easily be adated to secialised and very highly sensitive detection of sequential embedding. Unlike in rior art, 9 we consider two tyes of sequential embedding: ayload overwriting LSBs at the start of the cover which we call initial sequential embedding, and ayload overwriting a sequence of LSBs starting at some other oint, arbitrary sequential embedding. 4.. Initial Sequential Embedding Consider the weighted stego image (). When the ayload is located in the first M samles of the stego object we should fix α = 2 for the first M samles and α = 0 for the rest. As with Theorem of Ref., the function m ( 2 (s ) 2 N i + s i ) c i + (s i c i ) 2 i=m+ is minimized in exectation at m = M. Following the WS method, we can estimate the cover image by filtering the stego image and finding m to minimize E(m) = m ( 2 (s ) 2 N ( ) 2. i + s i ) F(s) i + si F(s) i (2) i=m+ However, we cannot continue by differentiating E, since its derivative has no closed form and in any case the function can have multile minima. All we can do is find the location of the minimum by trying all the values of m. Naively, comuting the sum (2) for each m = 0,..., N would require O(N 2 ) oerations, but we can still achieve a linear time algorithm by noting that the linear recurrence e 0 = 0, e m = e m + ( 2 (s m + s m ) F(s) m ) 2 ( sm F(s) m ) 2 generates e m = E(m) N ( si F(s) i ) 2; the minimum term of em therefore gives the minimum of E(m), and we require only linear time to generate and examine the sequence e m. We can also aly the new ixel redictors and weighting (but not bias correction) suggested in Sect. 3 to imrove the accuracy of the estimate.

6 4.2. Arbitrary Sequential Embedding Finally, suose ayload embedded in consecutive samles L, L +,..., M. Again the WS method can be adated, fixing α = 2 for these samles and α = 0 elsewhere. We have that the function of l and m l m (s i c i ) 2 ( + 2 (s ) 2 N i + s i ) c i + (s i c i ) 2 i=l i=m+ is minimized at l = L, m = M. Similarly to initial sequence embedding, we can roceed to an estimator for L and M by finding the location of the minimum of l ( ) 2 m ( E(l, m) = si F(s) i + 2 (s ) 2 N ( ) 2, i + s i ) F(s) i + si F(s) i i=l i=m+ after redicting the cover in the usual way. However this aears to be an O(N 3 ), or at least O(N 2 ) roblem. In fact, it is ossible to achieve a linear time algorithm by considering the sequence e i = ( 2 (s i + s i ) F(s) i ) 2 ( si F(s) i ) 2, which is related to E by E(l, m) = m l e i + N( ) 2. si F(s) i Minimizing E is achieved by finding the subsequence of e,..., e N with minimum sum. Finding the minimum subsequence sum is a standard roblem with a simle linear time solution. Finally, we can again imrove the method by using our novel cover redictors, and weighting the distance calculations as in Subsect EXPERIMENTAL RESULTS Recent exerience with steganalysis benchmarking has drawn attention to the significance of cover set selection and the need for results to be relicated in images from different sources. Accordingly, the results in this aer are drawn from three different sets of cover images, to exclude the ossibility that any erformance imrovements are secific to one articular image source as well as roviding some comarability with rior research.. Our rimary set is a comletely new library of,600 never-comressed digital camera images, taken by the first author with the Minolta DiMAGE A camera in a raw format. Crucially, the raw images were extracted (using the Minolta DiMAGE Viewer version 2.37) directly to 2-bit greyscale bitmas, avoiding any colour filter array interolation. All image denoising was disabled. After slight croing, to avoid any ossibility of vignette artefacts, the images were all exactly ixels in size. This set of images is our gold standard as we controlled the entire acquisition and re-rocessing chain. This set will be referred to as RAW camera images throughout this aer. 2. A second set is of 3,000 images downloaded from the NRCS website. 2 Aarently scanned from film in full colour, these images vary slightly in size around aroximately ixels. We shall refer to this image set as scanned images. 3. A final database of images was sulied by the researchers at Binghamton University. Their full set was made u of ictures from many different digital cameras and numbers several thousand; we selected 040 images with the same size of ixels, in which sixteen different camera models are reresented. The images were sulied as 24-bit colour PNGs. We will call these images alternative RAW images. The results for RAW camera images and scanned images are broadly similar, but the alternative RAW images erformance rofile is markedly different. Surrised by these anomalous results, we guessed that the suosedly RAW alternative images had been subject to some tye of image rocessing oeration, erhas during the conversion from internal camera RAW format to PNG. Certainly the images are much less noisy than those in other sets. After a few attemts, we were able to mimic the behaviour of the alternative RAW images by All images derived from this set can be made available to other researchers on request.

7 alying a denoising filter to the rimary set of RAW camera images (Softwhile s DeNoise lug-in for Photosho CS, version.0 release 29). In the exeriments, we will concentrate on the two classes of cover image: those not subjected to denoising (the rimary RAW camera images and the scanned images) and those subject to denoising (the alternative RAW images and also the rimary RAW images after the denoise filter was alied). The two classes exhibit quite different behaviour, while behaviour within each class is similar. To standardize the size of the images, all were reduced to smaller images of width 640: in the case of the to , in the case of the other two sets to (the slight discreancy due to asect ratio). Exeriments were also erformed on the full-size images, but the results will not be reorted because of comarability across image sets. We used four different size reduction methods to control for ossible artefacts of common downsamling algorithms.. Downsamling using bilinear interolation (the standard bilinear resamling algorithm in Photosho CS). 2. Downsamling using bicubic interolation (ditto). 3. Downsamling using nearest neighbour interolation. Lacking an interolation filter, this roduces images more noisy than the above two methods. Noisy images are known to be articularly hard for steganalysis. 4. Croing random regions. The aim is to reserve the original ixel neighbourhood characteristics; a drawback is that the image content changes when it is croed. The steganalysis estimators aly to greyscale images, which are resumed to be of 8-bit recision. In the case of the rimary downsamling was erformed on the 2-bit images, with conversion to standard 8-bit greyscale after all other transformations. In the case of the scanned images, and also the alternative RAW images, downsamling was erformed on the colour originals, with conversion to greyscale (erformed by selecting the luminance comonent) after all other transformations. 5.. Performance of WS Estimators for Sread Embedding The modifications to the WS method, roosed in Sect. 3, give rise to very many variations. Estimators can be constructed using any of six ixel redictors (fixed filters (3), (4), (5), and one using 24 ixels, lus adative filters with 8 (6) and 24 (7)) tas, any of three weighting coefficients (no weights, standard weights (8), and the new moderated weights (9)), and with or without subtracting the estimated bias () derived in Subsect We would like to know which combinations leads to imroved erformance, and then to test the best of the WS estimators against their structural cometitors. It is imossible to dislay benchmarks for all 36 WS variants in every cover set. Instead, we will demonstrate that the otimal choices for redictor, weighting method, and bias correction, can be made one-by-one, and dislay enough exerimental results to justify each choice. First, let us select the cover rediction filter. In Fig. 2 we show how the (MAE) of the relative ayload estimator deends on the choice of filter. The results dislayed are for unweighted WS variants, without bias correction, in two of the cover sets. Although the denoised images have a clearly different rofile, in both cases the adative 24 ixel filter has the best erformance (gives the smallest errors). We observed similar behaviour with weighted variants, and when bias correction was included, in almost every cover set, and so from this oint onwards we will fix the cover redictor to be of this tye. Occasionally, in the scanned images, we did observe that the 8 ixel filter (5) roduced slightly better erformance, but the extent of the difference was not large enough to vary our choice of redictor. Second, we consider the choice of weight coefficients in (2). Figure 3 benchmarks versions of the WS estimators, using the chosen ixel redictor, with each of the weighting otions. Again we see very different results in RAW images and denoised images (and, again, comarable results were observed in scanned images, regardless of downsamling choices, and the alternative RAW images, resectively). For images that have not been denoised, the new moderated weights (9) give the best erformance, the standard weights (8) the next best, and the unweighted detector gives the worst erformance (largest errors). But in denoised images this is reversed. We conclude that we should use either moderated weights or no weights at all, and that choice will deend on the nature of the image under analysis.

8 ixel fixed (3) 8 ixel fixed (4) 8 ixel fixed (5) 24 ixel adative (7) 24 ixel fixed 8 ixel adative (6) croed ixel fixed (3) 8 ixel fixed (4) 8 ixel fixed (5) 24 ixel fixed 8 ixel adative (6) denoised, croed 24 ixel adative (7) Figure 2. Mean absolute error (log scale) of unweighted WS estimators with different ixel rediction filters, in RAW camera images and denoised versions. The adative 24 ixel filter gives the best results in these covers. The same figure also benchmarks estimators with and without the bias correction derived in Subsect In the RAW camera images the results are slightly in favour of bias correction for smaller ayloads but the effect is negligible. In the denoised images we see a very different icture, with bias correction making a substantial imrovement no matter which weighting method is chosen excet for ayloads greater than about 80 %. Similarly results were observed in the alternative RAW images. We conclude that bias correction should be used unless the uncorrected ayload estimate is over about 0.8. Thus we have reduced the WS variants to a shortlist of two: the 24 ixel adative filter, bias correction (disabled for initial estimates more than 0.8), and either no weights at all, or the new moderated weights (9). It seems that we should refer the unweighted version only for images which show signs of denoising no weights standard weights moderated weights no bias correction bias correction croed no weights standard weights moderated weights no bias correction bias correction denoised, croed Figure 3. Mean absolute error (log scale) of WS estimators with the adative 24 ixel filter, unweighted or weighted according to standard or moderated weights, and with or without a bias correction term. In almost all cover sets the moderated weights are suerior, but in denoised images unweighted WS has the better erformance. Bias correction makes but a small difference to ordinary images, but can give a large advantage on denoised images.

9 Coules/ML croed Imroved WS: 24 ixel adative redictor, unweighted Imroved WS: 24 ixel adative redictor, moderated weights Coules/ML NN interolation Imroved WS: 24 ixel adative redictor, unweighted Imroved WS: 24 ixel adative redictor, moderated weights Coules/ML bilinear interolation Imroved WS: 24 ixel adative redictor, unweighted Imroved WS: 24 ixel adative redictor, moderated weights Coules/ML bicubic interolation Imroved WS: 24 ixel adative redictor, unweighted Imroved WS: 24 ixel adative redictor, moderated weights Figure 4. Comarison of the best structural ayload estimators, standard WS, and the imroved WS estimators roosed here, in four different sets of cover images derived from the raw camera images. The WS variant with a 24 ixel adative filter, moderated weights, and bias correction for < 0.8, has the best overall erformance. We are now in a osition to benchmark the new WS variants, against their cometitors from the literature. The leading ayload estimators and detectors of LSB relacement in digital images are the structural detectors, which were rovided with a common framework in Ref. 5. They include estimators known as RS, 3, 3 -LSM, 4 Triles, 5 Quadrules, 6 and Coules/ML. 7 Performance of the first four is evaluated thoroughly in Ref. 8: it is demonstrated that, for never-comressed greyscale images, there is not a great deal of difference but the estimator is marginally most accurate, desite the more sohisticated methods used in -LSM and Triles. As demonstrated in Ref. 7 (and confirmed here), the Coules/ML estimator is more accurate than, but only for small ayloads. The same is true, to a lesser extent, of the Quadrules estimator. Since we consider only greyscale never-comressed images here, the leading cometitors for WS are and Coules/ML: the latter for small ayloads only, and the former in other situations. We will now comare their erformance, as well as standard WS as described in Ref., with the best two imroved WS estimators The same is not true if the cover images are colour, or have reviously been JPEG comressed; the results in Refs. 5 and 8 demonstrate that Triles is the most accurate estimator for ayloads less than about half the maximum, - LSM for ayloads between about half and three-quarters, and standard WS for ayloads near the maximum. The newer

10 Coules/ML Imroved WS: 24 ixel adative redictor, unweighted Imroved WS: 24 ixel adative redictor, moderated weights Alternative RAW images, croed Coules/ML Imroved WS: 24 ixel adative redictor, unweighted Imroved WS: 24 ixel adative redictor, moderated weights denoised, croed Figure 5. The relative erformance is different when the cover images have been subject to denoising. Results here from the alternative RAW images and the rimary set of RAW camera images which was subject to denoising before embedding. In this case, it is generally best to choose an unweighted WS estimator. For a small range of ayloads, the estimator remains most accurate, but the difference is very small. roosed here. Because Coules/ML erforms very oorly on large ayloads, and is orders of magnitude slower to comute than the others, we will only include its results for ayloads u to half maximum. We dislay results from the including the different downsamling otions, in Fig. 4. Observe that the detectors are all much less accurate in nearest-neighbour downsamled images: this interolation algorithm reduces neighbour correlation, causing the cover redictors to have larger errors. Aside from this difference, rather similar features are seen in each case: of the detectors in rior literature, is generally the best erformer but Coules/ML is better for very small ayloads and standard WS for very large ayloads. In almost all cases, however, the new WS method (with moderated weights) is the best erformer. The only excetion is that Coules/ML exhibits better erformance for zero ayloads (and one should balance this slight erformance advtange against the comutational costs). We erformed all the same exeriments with the scanned images. The results were quite similar so we will not dislay more charts, but we did observe that the nearest-neighbour downsamled set roduced slightly different results: the fixed 8-ixel filter (5) gave rather better results than the adative filter, and the Coules/ML erformance was substantially better than imroved WS, for a wider range of small ayloads. Now we turn to images which have been subject to denoising: they are dislayed in Fig. 5. The general erformance of the estimators is good (all error magnitudes are lower) which is to be exected for images where the cover can be redicted with high accuracy. The shae of the detectors accuracy is different to the other image sets, and it is the unweighted variant of the imroved WS estimator which erforms best. In these resects the results for denoised images is quite different to that for those not subject to denoising. In the alternative RAW images there is a small range of ayloads for which remained the best erformer, and a small range of ayloads for which the weighted WS variant was more accurate than unweighted. However the differences, in such situations, are not very large. Overall, the unweighted WS variant is suerior in denoised images. Finally, receiver oerating characteristics (ROC) curves lotted in Fig. 6 demonstrate that the lower error rates for our imroved WS estimators translate to better erformance (lower false ositive rate at any given detection rate) when the methods are emloyed as discriminator between covers and stego objects with very small embedding rates. Results are given for the RAW camera images in their croed versions with = 0.05 Quadrules and Coules/ML estimators do not alter this conclusion.

11 true ositives Coules/ML Imroved WS (weighted) true ositives Coules/ML Imroved WS (unweighted) false ositives (a) croed, = false ositives (b) denoised, croed, = 0.0 Figure 6. Receiver oerating characteristics (ROC): comarison of discriminatory ower of selected WS and structural estimators. Imroved WS variants outerform structural detectors by a fair margin. using moderated weights (left chart, comare MAEs in the to-left chart of Fig. 4) and their denoised and croed versions with = 0.0 using unweighted WS (right chart, comare MAEs in the right chart of Fig. 5) Sensitivity to Cover Proerties These exerimental results have demonstrated that the nature of the cover images can have a large effect on the erformance of the WS estimator. It is difficult to gauge whether such sensitivity is tyical in steganalysis, because it is common ractice in the literature to test against only one set of images. This could lead to misleading results: had we tested our WS imrovements on only the alternative images (and some other steganalysis literature does exactly this) then our conclusions about weighting would have been comletely different. An advantage of WS is its concetional and comutational simlicity which allows us to run extensive exeriments and dig a bit deeer into the deendencies between the comonents of the detection method and its erformance. As a starting oint, Tab. reorts the erformance differences of two WS variants across all image sets considered in this study for = 0.. We resent various measures of estimation erformance, most rominently MAE (as in all revious charts), mean error as a measure of bias (to see whether a method over- or underestimates systematically), and the inter-quartile range as a robust measure of variation. All measures show substantial erformance differences across image sets and re-rocessing methods. WS steganalysis is consistently least accurate for covers downsamled with nearest neighbour interolation. Other downsamling methods somewhat increase error rates for our RAW camera images relative to the croed images, but not consistently so for the scanned images. Comaring the WS variants, it becomes aarent that the weighted variant (dark bars) erforms considerably better in most image sets excet for the bias measure in interolated RAW camera images and for the alternative RAW images, where unweighted WS (bright bars) is slightly more accurate (by all measures). Hence we can see again that the alternative RAW set, as well as the images derived by denoising the rimary RAW camera images, give very different results to those with images not subject to denoising (cf. Fig. 5). The variance of erformance, between images sets, calls for extensive testing, with multile datasets. (It also makes it difficult to draw general conclusions about which method or variant erforms best, as every conclusion is conditional on a number of factors.) But no set of benchmarks will include the unimaginably high number of different image origins and re-rocessing histories found in the wild. Hence, it would be beneficial to identify roerties of images which influence the erformance differences, and search for ways to adat detection methods to the articular roerties of each image to be steganalysed.

12 Table. Performance differences across image sets. Summary statistics for embedding rate = 0.. Dark bars are WS estimates using the adative filter, moderated weights and bias correction; bright bars are WS estimates using the unweighted adative redictor and bias correction (bars for nearest neighbour images exceed the scale and are censored). Image set Mean absolute error Mean error (bias) Inter-quartile range RAW camera, bilinear RAW camera, bicubic RAW camera, n. n. RAW camera, croed scanner, bilinear scanner, bicubic scanner, n. n. scanner, croed alt. RAW, croed RAW camera, denoise Connection between Prediction Error and Estimator Performance A crucial art of the WS method is the filter F(s) which redicts cover ixels, so we will study the relation between the erformance of F(s) in redicting a known cover c and the secret message length estimation erformance. For each image we comute the root mean square error (RMSE) of the redictor RMSE = [ N w i ( F(c)i c i ) 2 ] 2 and lot them against the absolute estimation error ˆ for lain covers ( = 0). The RMSE calculation is weighted using the same weights as the WS method being studied. Selected results are dislayed in Fig. 7. Both axes are in logs, and units are intensity differences on the x-axis and fraction of the embedding caacity on the y-axis. As the oint clouds aear well-behaved we deem it justified to estimate regression lines, on log errors, to assess the sloe and strength of the relation. The R 2 goodness of fit is based on the ordinary least squares estimate (OLS) which we comlement, wary of misleading inference from outliers, with a robust regression using iterated least squares (IWLS) and the common Huber method. 4 In all cases, the two estimates are very close, so outliers are not an issue here. Comaring Fig. 7(a) and Fig. 7(b), we see that differences in the linear redictability of ixel intensities can exlain only about 0 % of the variation in the ayload estimation within a given image set and re-rocessing method. However, as bicubic and nearest neighbour covers lie on the same regression line, and the centre of mass for nearest neighbour images is shifted towards the uer-right corner (higher redictor RMSE, lower ayload estimation accuracy), we conclude that differences in linear redictability may well exlain much of the differences between image sets of different re-rocessing. We do not ossess enough different image sets to estimate the ortion of variation thus exlained. Fig. 7(c) dislays the relation for unweighted WS. The variation of the rediction error between images is higher than in the weighted case (cf. Fig. 7(a)), because weighting hels more for some ictures than for others. This exlains why R 2 is higher. Put another way, weighting imroves the redictor RMSE but the better cover estimate does not fully translate in better estimation erformance because factors other than linear redictability (e.g. the weights basis) also have influence. Fig. 7(d) comletes this analysis with data from the alternative RAW images. It is very visible that the higher detection erformance stems from much better cover redictability. Given the imortance of linear redictability for the detection erformance, one may ask where differences in linear redictability come from. One source is clearly re-rocessing with linear filters; another otion is saturation, which has been found as influencing factor is revious work. 8, 5 Saturated ixels often aear in

13 0 0 R 2 = R 2 = absolute estimation error absolute estimation error OLS estimate robust estimate (IWLS) 0 5 OLS estimate robust estimate (IWLS) root mean square redictor error (a) bicubic root mean square redictor error (b) nearest neighbour 0 0 R 2 = R 2 = absolute estimation error absolute estimation error OLS estimate robust estimate (IWLS) 0 5 OLS estimate robust estimate (IWLS) root mean square redictor error (c) bicubic, unweighted root mean square redictor error (d) alternative RAW images, croed Figure 7. Relation between redictor accuracy (measured in RMSE for the imroved redictor on cover images) and estimation erformance (absolute errors for = 0, no bias correction). Moderated weights unless otherwise stated. satial roximity; those regions are well redictable, its ixels get high weight, but do not fulfil some crucial assumtions for the correctness of the WS estimator (unlike other estimators, such as RS analysis, 3 which benefits from saturation 5 ). Fig. 8 lots the log of the share of saturated ixels against the log redictor RMSE. It is very visible that only few of the RAW camera images ossess significant ortions of saturated ixels, and outliers with low RMSE aear on both sides of the vertical line denoting a share of 5 % of saturated images. For illustration, we have also rinted the same chart for the croed scanned images, where saturation is more frequently observed, and one can see the trend towards lower RMSE well beyond the 5 % line. Nevertheless, saturation as an exlanatory factor can be ruled out for our set of RAW camera images. If redictability, as a result of different image acquisition or re-rocessing, influences the detection erformance between image sets, the question remains what measurable image roerties can exlain (the still high) erformance difference within a set of images from the same source. One indication that there are other factors is that, among the rediction methods offered in Sect. 3, the one that gives the lowest RMSE is rarely the best stego estimator: Tab. 2 shows that the adative redictor with standard weights roduces consistently better RMSE than moderated weights, but the latter has overall higher erformance when lugged into the WS method Parity Co-occurrence as Determinant for Estimation Performance Another roerty relevant to WS and orthogonal to linear redictability is the distribution of the arity of intensity values in the local neighbourhood of a ixel. We shall call this roerty arity co-occurrence and,

14 log % saturated ixels log rediction RMSE (a) nearest neighbour log % saturated ixels log rediction RMSE (b) scanned images, croed Figure 8. The share of saturated ixels (log ercentage) can hardly exlain the bulk of variation in redictor accuracy (log RMSE for the imroved redictor with moderated weights for = 0). Vertical lines corresond to 5 % saturation. to measure it, we count the number of ixels with equal arity among a centre ixel s eight neighbours. We aggregate mean arity co-occurrence by building the average across all (non-edge) ixels in an image. Going back to Fig. 7, one can see that most data oints are lotted as + but some as symbols. The latter corresond to the to 0 % quantile of mean arity co-occurrence in each image set. It is visible that for our cluster well above the regression lines, which indicates that high arity co-occurrence yields relatively oor detection erformance for a given level of linear redictability. Given our comments that arity co-occurrence can lead to bias, in Subsect. 3.3, this is to be exected. Note that 70 % of the alternative RAW images have an excetionally high level of arity co-occurrence: it is lausible that such a feature could arise from denoising oerations, which tend to smooth out image ixel values. This exlains why bias correction yields substantial erformance imrovement in the alternative RAW images (see for examle the right chart of Fig. 3). The distribution of average arity co-occurrence for selected image sets is deicted in Fig. 9. Again, it is visible that only our denoised RAW camera images (aroximately) match the rofile of the alternative RAW images Performance of WS Estimators for Sequential Embedding Now we consider sequential LSB relacement, where both initial and arbitrary sequences of samles have their LSBs relaced. It is known that structural methods do not work well in this case (random location of cover changes is essential for their correctness) but we will test the method anyway, as well as an old LSB relacement steganalysis method Chi-Square 6 which is adated for initial sequential embedding. Of the WS methods, we can aly the standard version (for which random location of cover changes is not a requirement) or our newer versions, as well the those adated secifically for sequential embedding. We do not include all the intermediate otions, focusing only on chi-square, structural, standard WS, sequential WS with the standard mask and no weights, and sequential WS with the 8-ixel mask (5) and imroved weights (9). (We did not test Table 2. Median cover redictor RMSE of image sets used in our exeriments Image set Predictor standard adative unweighted unweighted std. weights mod. weights RAW images, bilinear RAW images, bicubic RAW images, nearest neighbour RAW images, croed RAW images, denoised, croed alternative RAW images, croed

15 mean # of neighbour ixels with equal arity Cover images with saturation < 0.% RAW camera, croed scanner, croed alternative RAW, croed RAW camera, denoised+croed quantile Figure 9. Neighbour ixel arity statistics. The high arity co-occurrence in the alternative RAW images could be successfully dulicated by denoising the RAW camera images. Saturated images are excluded to avoid interference from different amounts of saturated images in the sets (saturated areas cause strong arity co-occurrence). the sequential WS method with adative redictors, because of minor comlications with training the mask at the image edges: sequential WS is so accurate that there is a significant enalty if the edges are not included.) Figure 0 dislays the MAE of roortionate ayload sizes for these detectors of initial sequence embedding. The chi-square estimator does not erform well excet for full ayload, does better than one might exect, similar to standard WS. The sequential WS estimator is at least an order of magnitude more accurate: curiously, the weighted version works better in both RAW and denoised images. Figure shows analogous results for arbitrary sequential embedding: the chi-square estimator is not immediately alicable here. Since there is less certainty about the location of the ayload we would exect that the accuracy is slightly lower, and that does turn out to be the case. But the sequential WS estimates of ayload size are still at least an order of magnitude more accurate than their cometitors. The method rovides exlicit estimates for the start and end locations of the ayload: in ractice, the error in size estimation is fairly evenly distributed between error estimating these two oints Sequential Chi Square Sequential WS: 4 ixel redictor, unweighted Sequential WS: 8 ixel redictor, moderated weights croed Sequential Chi Square Sequential WS: 4 ixel redictor, unweighted Sequential WS: 8 ixel redictor, moderated weights Alternative RAW images, croed Figure 0. Mean absolute error (log scale) of structural, sequential chi-square, standard WS, and sequential WS estimators, when the ayload is embedded as an initial sequence. Results for both raw cameras images (not subject to denoising) and the alternative raw images (which aear to have been denoised) are dislayed.

16 Sequential WS: 4 ixel redictor, unweighted Sequential WS: 8 ixel redictor, moderated weights croed Sequential WS: 4 ixel redictor, unweighted Sequential WS: 8 ixel redictor, moderated weights Alternative RAW images, croed Figure. Mean absolute error (log scale) of structural, standard WS, and sequential WS estimators, when the ayload is embedded as a segment with arbitrary start osition. Results for both raw cameras images (not subject to denoising) and the alternative raw images (which aear to have been denoised) are dislayed. These estimators are highly accurate: mean absolute relative error of the order of 0 4 corresonds to ayload size errors of around 30 bits, in an image of 0.3 megaixels. Such accuracy is literally unachievable in detection of sread embedding, because even a detector which detects every cover change erfectly cannot detect those ixel locations which convey ayload but were not changed. 6. CONCLUSIONS In a research effort aimed at exloring the behaviour of Weighted Stego-Image steganalysis under conditions where it has been known to work well (large ayloads 8 and sequential embedding 9 ), we created imrovements to all of the method s three comonents. We found that an ugraded WS method outerforms even the best structural detectors in domains where they were reviously believed more reliable. Extensive and robust exerimental results on a variety of image sources confirm our findings and make the re-discovered WS aroach the first choice quantitative detection method for LSB relacement in never-comressed cover images. More recisely, the WS variant using an adative 24 ixel filter with moderated weights has turned out to be the most accurate method in general, with the excetion that equal weights are referable if images are known to have been denoised. In either case, imroved bias correction is recommended for initial ayload estimates below 0.8. Fig. 2 illustrates these conclusions on the referred detection strategy for LSB relacement in form of a decision tree. We also erformed some exeriments in an attemt to identify the image roerties which are resonsible for the large observed erformance discreancies between different image sets. Predictor accuracy and cover arity co-occurrence are significant factors. Further, we have been able to modify the method for secialised detection of sequential LSB relacement, where it dislays a very high level of erformance, unmatched in the literature. It is imortant to mention that the WS estimators rely on the cover images being natural images not subject to lossy comression. If the covers had been stored as JPEGs, rior to satial-domain LSB relacement, the WS method would lose a lot of erformance. Some of the structural methods are not badly affected by this henomenon. Otimizing the WS method to secific cover tyes, such as revious JPEG comression or nearest neighbour downsamling, as well as further secialising the sequential detection to cases where embedding changes are introduced sarsely (e. g. through matrix embedding) are toics of further study. ACKNOWLEDGMENTS The first author is a Royal Society University Research Fellow. Thanks are due to Jessica Fridrich and Tomáš Pevný, who sulied the set of alternative RAW images.