Objective Assessment of Perceived Sharpness of Projection Displays with a Calibrated Camera

Transcription

1 Objective Assessment of Perceived Sharpness of Projection Displays with a Calibrated Camera Ping Zhao, Yao Cheng, Marius Pedersen Gjøvik University College, Norway ping.zhao@hig.no Abstract Sharpness is an important image quality attribute for projection displays. However, it has not been well studied for projection displays in the existing literature. In this paper, we conduct an experimental study of perceived sharpness on projection displays in a controlled environment. The basic idea is to simulate the optical blurring process with Gaussian filtering and apply them to selected natural images. We project these distorted images onto the screen, and we invite a group of human observers to give perceptual ratings. We also use a calibrated camera to capture them in order to measure the sharpness with eight state-of-art image quality metrics. The correlations between the objective and subjective results indicate that SSIM, FSIM and VIF metrics give excellent average performance. Keywords image quality, sharpness, camera calibration, projection display, image quality metric I. INTRODUCTION Nowadays, modern digital imaging with advanced technologies composes an essential part of our daily life. It is easy for consumers to capture what they see and record what they experience with portable imaging devices without professional training. Sharing stories with friends can be achieved by clicking a single button afterward. One common way to do this is via projection systems which are typically configured with high definition displays to visualize image reproductions. Comparing to other display technologies, projection systems have unique advantages like portability, flexibility for deployment, and large screens for sharing information to a crowd. In a few scenarios, multiple projections can be tiled up to produce large perceptual seamless images which visualize information to the target audiences [1], []. In recent years, there is an increasing general interest on embedding projectors into portable devices to further enhance the continuity experiences between mobile imaging devices and socialization over the Internet [3], [4]. The investigation [5] showed that users are willing to project content with other people around in social spaces, which indicates the projection system have a good potential to become more popular in the coming future. Hence, image quality assessment of projection displays becomes an increasingly interesting topic for both scientific research and industrial communities, because it defines a systematic approach to measure and evaluate the quality of image reproductions. Image quality is mainly evaluated with respect to the perception. The ultimate goal of image quality assessment is to correlate the objective results to the subjective results, and thereby eliminate the demand of human observers. In this context, image quality attributes are used. These are terms of perception, such as, but not limited to, lightness, contrast, color accuracy, sharpness, artifacts (including noises), and 15 Colour and Visual Computing Symposium (CVCS) /15/$31. c 15 IEEE physical properties (screen dimension, display resolution and refreshing rate, etc.). For the researches of imaging systems in various domains, the selection of the most important image quality attributes may have different priorities. For printing, Pedersen et al. [6] suggested that all these image quality attributes mentioned above are important; Johnson [7] specially remarked color accuracy, sharpness, and contrast for printing. For information displays, You et al. [8] and Lehtimaki et al. [9] pointed out noise, sharpness and perceived depth are priorities for stereoscopic imaging. However, for projection displays, limited works have been done so far. Thomas et al. [1] and Strand et al. [11] remarked lightness and color accuracy, while Majumder et al. [1], [13] indicated that lightness is more important than the color accuracy. With respect to the literature above, it is clear that despite of specific research domains and imaging technologies involved, sharpness is commonly recognized as an important image quality attribute for perceptual evaluation, and it is closely associated with other image quality attributes like lightness and contrast. Since sharpness defines the amount of details in image reproductions, it is commonly referred as the counterpart of blur which is one of the most typical image quality distortions. The human visual system has a remarkable capability to detect image blur without seeing the original image, but unfortunately the underlying mechanisms are not well understood [14], [15]. In this paper, we conduct an experimental study of perceived sharpness on projection displays in a controlled environment. The goal is to evaluate the performance of stateof-art image quality metrics measuring image sharpness, and determine their performance with respect to the correlations between perceived and measured sharpness. The results can be potentially used by manufacturers to improve their product setting without doing subjective investigations or the consumers to optimize their projection displays accordingly. The rest of this paper is organized as follows. First, in Section II, we present state-of-art image quality metrics used to measure image sharpness. Then, in Section III, we present the experimental setup. In Section IV, we demonstrate the subjective and objective results. Last, in Section V, conclusions and future works are presented. II. SHARPNESS METRICS Conventionally, sharpness was largely evaluated based on the definition of edges in images. The main idea is to locate edges in local regions, compute the quality scores in these regions at the detected edges, and pool them to generate a final score representing the global sharpness quality [16]. The edge features represent the quality of optical components in

2 an imaging system, so the measured edge responses can be used as an estimate of the modulation transfer function [17]. Many sharpness metrics are based on the clarity of details in the image reproductions. They are slightly more advanced than edge base metrics, because they are suitable to measure highly degraded images. Recently, there is also an increasingly interest on developing perceptual models to simulate human visual system on evaluating image sharpness. Nevertheless, sharpness metrics can be classified into two main categories depending on the availability of the original image. A. Full Reference Metrics The Structure Similarity Index (SSIM) [18] is commonly used to predict the degradation of structures in images. Although it was not originally developed to evaluate sharpness, it estimates the visibility of detail preservation which is implicitly associated with sharpness. Marziliano et al. [19] proposed two full reference based metrics to evaluate the sharpness of JPEG compressed images, not merely on ringing artifacts, but also on blurring. Zhang et al. [] proposed a full reference metric Feature Similarity based Index Metric (FSIM) to measure image sharpness. Firstly, they generated a local image quality map with phase congruence and image gradient magnitude as features, and then utilized the phase congruence information again as a weighting function to derive the final image quality score. Another commonly referred image quality metric Visual Information Fidelity (VIF) was proposed by Sheikh et al. [1]. This metric was derived from a statistical model for natural scenes, a model for image distortions, and a human visual system model in an information-theoretic setting. The Visual-Signal-to-Noise-Ratio (VSNR) was presented by Chandler et al. []. This metric uses low-level human visual system properties of contrast sensitivity and visual masking first via a wavelet-based model to determine if the distortions are below the threshold of visual detection. If the distortions are supra-threshold, the low-level human visual system property of perceived contrast and the mid-level human visual system property of global precedence are taken into account as an alternative measure of structural degradation. B. No Reference Metrics Caviedes et al. [3] developed a content independent no-reference sharpness metric based on the local frequency spectrum around edges, however this method has problems to predict image quality when artifacts become dominant. Maalouf et al. [4] defined a metric based on the eigenvalues of the wavelet-based multi-scale structure tensor to accumulate multi-scale gradient information of local regions. The structure tensor has the advantage to identify edges in spite of the presence of noises, so the metric is suitable to measure the sharpness of color edges. Cao et al. [5] introduced a metric which takes the advantage of anisotropic diffusion to build up a preliminary map of ringing artifacts and refined it by considering the property of ringing structure. The proposed metric was reported to work well for JPEG compressed images. Samira et al. [6] proposed a method to measure color differences to determine the sharpness in local regions. This method is good in the cases of which the color management is critical to the applications. Vu et al. [7] presented a block-based metric to quantify the local perceived sharpness within and across images. Both spectral and spatial properties of images are utilized to build up indexes for the standard deviation of the impulse response used in Gaussian blurring. Hassen et al. [15] developed a metric Local Phase Coherence based Sharpness Index (LPC-SI) to identify sharpness as strong local phase coherence in the complex wavelet transform domain. They incorporated this metric into a framework that allows for computation of local phrase coherence in arbitrary fractional scales. Leclaire et al. [8] introduced a metric S- Index which can be used to measure the sharpness in a probabilistic scene using the small variation of an image compared to that of certain associated random-phase fields. In addition, Narvekar et al. [9] presented a metric based on Cumulative Probability of Blur Detection (CPBD), which discretizes visual sharpness into several regions and for each region a distinct quality class or qualitative score is assigned. Then a training base method was proposed to determine the centroids of the quality classes for the assigned scores, and finally the index of image quality class is assigned as the measured image quality. Narvekar et al. [3] proposed a noreference metric based on a cumulative probability of blur detection. Comparing to the saliency-weighted foveal pooling based metric developed by Sadaka et al. [31], their metric does not require additional visual attention or salience maps. In the former case, the computational complexity is largely reduced. Besides, Ferzli et al. [3] derived from the measured justnoticeable blurs to develop a perceptual-based sharpness metric which is applied to 8x8 blocks instead of the entire image. The metric took into account the response of the human visual system to sharpness at different contrast levels. Wang et al. [33] proposed a metric to predict wavelet coefficients of local phase coherence structures across scale and space in a coarseto-fine manner. Another no-reference metric sharpness metric Just Noticeable Blur Metric (JNBM) proposed by Ferzli et al. [34] integrated the concept of just noticeable blur into a probability summation model. The metric was reported to be able to predict the relative amount of blurriness in images with different content. III. EXPERIMENTAL SETUP AND PROCEDURE In our experiment, we use a calibrated camera to capture all pixels on the projection screen in one shot, and the performance of eight selected state-of-art sharpness metrics are evaluated with respect to the perceptual ratings on the captured images which are registered with their original ones. The experiments take place in a controlled dark room, which simulates the condition of home theater. We use a portable three chip LCD projector SONY APL-AW15 (throw ratio: 1.5) to produce projections on a planar screen which is naturally hanging on the ceiling. The projector is put on a table placed in front of the projection screen about 3 meters away with respect to the throw ratio of the projector. A remote controlling laptop is connected to the projector via a HDMI cable in order to generate full screen projections which have resolution in pixels. On the screen, the dimension of projection area is approximately 1. in meters. We use a DLSR Nikon D61, which has an imaging resolution in pixels with a Sigma VR 4-15mmf/4G (VR off) lens to capture the projections. The camera is fixed on a tripod placed in front of the projection screen about 4 meters

3 Fig. 1. The selected test images from the Colourlab Image Database: Image quality, and they are Gaussian blurred to generate 6 levels of sharpness distortions. away. Pictures are taken remotely with a software without physically touching the camera. The pictures are saved in raw format and rendered with aliasing minimization and zipper elimination demosaicing algorithm [35] without automatic vignetting correction, brightness adjustment, gamma correction and noise reduction etc. We select 7 test images (Figure 1) from the Colourlab Image Database: Image Quality [36] to generate 6 levels of Gaussian blur with kernel size 11 and standard deviation,.5, 1, 1.5,, and 3 respectively. The selection criteria of the test images is established based on the coverage of different image features such as hue, saturation, lightness, contrast, skin color, sky, grass, size of neutral gray areas, color transition, fine details, and text presence etc. The cameras need to be calibrated in advance because they are known to have optical and electronic issues. Vignetting effect is an optical phenomenon which stands for the undesirable gradual intensity fall off from the image center to its external limits. It corrupts every picture taken from the cameras. In some cases, the camera sensors are not necessary to produce completely linear intensity responses. In this paper, we incorporate existing methods proposed in our previous researches [37] to eliminate the vignetting effect. The basic idea is to use a hazy sky as a closely uniform illuminant to create a vignetting mask from multiple rotated shots of the same scene, and apply the mask to the every picture we take subsequently. The camera settings are also optimized, so the sensor responses are linearized in all circumstances. Meanwhile, we incorporate the image registration algorithm [38] to register the captured projection with its original digital copy in order to apply full reference metrics. For reduced reference and no-reference metrics, we also apply the registered images, so the test images are identical to all metrics under evaluation. We calculate the objective sharpness using eight state-ofart image quality metrics, which are commonly referred the in existing literature: SSIM [18], VSNR [], VIF [1], FSIM [], LPC-SI [15], S-Index [8], CPBD [9], and JNBM [34]. IV. E XPERIMENTAL R ESULTS A. Subjective Results In the subjective experiment session, we invite 15 human observers (recommended by CIE [39] and ITU [4]) to give perceptual ratings to the projections of image distortions. Each observer is placed at exactly the same position as the camera. The viewing condition is similar to a home theater like environment where the room is completely dark and the visual angle from the projection boundaries to the principal axis of observation is approximately 15 degrees. The blurred images are displayed in a randomized order to observers, and each time only one image is displayed. The experiment is set up as a category judgment experiment. For each displayed image, the observers are asked to indicate the overall perceptual sharpness with a category label which stands for the rank between no blurring at all and completely blurred corresponding to the ratings numbers ranging from 1 to 9 respectively. After the experiment, the observers are asked to share their evaluation strategies on the image sharpness. The perceptual ratings collected in the experiment are scaled to generate [41] (Figure ). In this figure, it is clear that the rank order of perceived sharpness decreases as the blur level increases. However, the sharpness perception should not be simply fitted into a linear regression model, since for the average for test image 1, 4, 6 and 7 appear to have a flat region between the first and the second blur levels. Another observation is that the general tendency of mean for all test images are fairly similar and their value ranges are almost identical. The variation of on certain blur levels (the blur level 4 for test image 1) are slightly larger and observers have contrary arguments on the perceived sharpness (the blur level 1 for test image 5). This observation suggests that observers have closely the same perception on sharpness despite of image content. B. Objective Results The main purpose is to know which metric works the best for the captured images of projection displays. The performance of the image quality metrics are evaluated with respect to the Pearson correlation coefficients between the metric results and the mean of perceptual ratings (Figure 3a). It is clear that the correlations coefficients of objective and subjective results are high. In most cases, their values are above.9 and all metrics have a correlation above.7. S-Index and VSNR have relative poorer performance for test image ; S-Index, CPBD and JNBM have relative poorer performance for test image 4. Obviously, the performance of SSIM, LPC-SI, FSIM, JNBM and VIF have good performance on prediction in general. In this case, we generate a box plot of all metrics for the correlation coefficients for all test images (Figure 3b). In this figure, we can find that VIF has the best performance on prediction of sharpness in general. Both of its median and mean values are very high, and the variation of correlations is very compact. In other words, this metric has both good as well as stable performance despite of the content of test images. This observation can also be applied to SSIM and FSIM, both have slightly larger variations. Another interesting conclusion can be made based on comparing the difference in performance between the full reference metrics and no reference metrics. Among these metrics, the metric SSIM, FSIM, VSNR and VIF are full reference metrics, while the rest are no-reference metrics. Obviously, the average performance of full reference metrics is higher than

4 of Perceptual Ratings for All Test Images Test Image 1 Test Image Test Image 3 Test Image Test Image Test Image Test Image Fig.. The of perceptual ratings collected from 15 human observers based on 6 blur levels of 7 test images. The red dots stand for mean for each blur levels over all human observers, while the red bars stand for the median. The blue box stand for the 5% inner quantiles of, and the blue bars stand for 75% outer quantiles of. The red crosses stand for outliers with respect to the 75% outer quantiles. All plots are scaled to have the identical Z-score value range from -.5 to.5. the no-reference metrics, and the full reference metrics tend to give more stable outcomes with respect to the performance variance. Since the captured images in the experiments are all registered with their original ones, the full reference and no-reference metrics have the identical input captured images with the exactly the same dimension, content and optical degradation. In addition, there is no extra disturbance like the outer border of projection area or the background introduced in the captured images. Then the performance differences must root in the metrics themselves. One important purpose of incorporating metrics in image quality assessment is to maximize the prediction accuracy of image quality attributes. So, for image quality assessment of projection displays, we should always incorporate full reference metrics over noreference metrics in the first place since the original digital copies are available in most cases. C. Perceptual Evaluation Strategies After the subjective rating session, each human observer is asked to share his/her perceptual evaluation strategies on the image sharpness. In the experiment, almost all observers implicitly insist to focus on only a few specific areas to locate and observe the fine details assuming that the global sharpness is identical to the local areas. The areas without fine details are largely ignored by the human observers according to the experimental records. We rank these areas by the count of observers who really pay attentions to and put the top two for each test image in the Table I. From this table, we can see that the fine details that observers actually pay attention are located in the areas where the images are sharpest in the original images. When it comes to comparing two images, the observers are sensitive to the changes of these sharpest areas either due to the transition of blurred edges, lightness or colors in these areas. The changes of less sharp areas appear to be less visible to the human observers. This explains why the human observers largely ignore the unfocused background but only pay attention to the well focus foreground objects. In this case, for the design of a good image qualityf metric predicting sharpness, we should develop algorithms to separate well focused and non-focused areas, and take the advantage of the original images to discover the changes of lightness and colors in the sharpest areas. TABLE I. TOP TWO SALIENCY AREAS OF PERCEIVED SHARPNESS Test Image Saliency Area Count of Observers 1 Walls in the center 7 Chairs 4 Flower center 1 Flower staments 6 3 Green grass 11 River 1 4 Human face (highlight area) 7 Human face (shadow area) 4 5 Peacock feathers 5 Peacock eyes 5 6 Car body 8 Eaves 5 7 Texts 1 Bird 4 V. CONCLUSION In this paper, we conduct an experimental study of perceived sharpness on projection displays in a controlled en-

5 (a) (b) Fig. 3. The Pearson correlation coefficients between the objective and subjective results over all test images (a), and performance of sharpness metrics over all test images (b). In the experiment, we have 7 test images blurred with Gaussian filtering with kernel size 11 and standard deviation.5. We invite 15 human observers in the subjective experiments to give perceptual ratings. In (b), The red dots stand for mean Pearson correlation coefficient, while the red bars stand for the medians. The blue box and bars stand for the 5% inner and 75% outer quantiles of correlation values respectively. The red crosses stand for outliers. FR and NR indicate full and no reference metrics respectively. vironment. The perceptual results suggest that the perceived sharpness follows a nonlinear tendency pattern but its rank order remain the same as the unversed ranked blur levels. The correlations between the metric results and perceptual results indicate that the VIF metric perform well for most types of distorted natural images. However, SSIM, FSIM and VIF give excellent prediction performance on average in most cases, both terms of absolute mean values and variance of Pearson correlation coefficients. There is an indication that full reference metrics outperform no-reference metrics in our experimental environment. In the experiments, we ask human observers to share their perceptual evaluation strategies. Their ideas indicate that the human observers are only sensitive to the changes of lightness and colors in the sharpest areas in the original images. In the coming future, experiments should be done to investigate the impact of different viewing conditions. ACKNOWLEDGMENT We would like to acknowledge the contributions of a large number of volunteers in the subjective experiments. This work is part of the HyPerCept research program funded by the Research Council of Norway. REFERENCES [1] O. Bimber and A. Emmerling, Multifocal Projection: A Multiprojector Technique for Increasing Focal Depth., IEEE Transactions on

6 Visualization and Computer Graphics, vol. 1, no. 4, pp , 6. [] B. Sajadi, M. Lazarov, A. Majumder, and M. Gopi, Color Seamlessness in Multi-projector Displays Using Constrained Gamut Morphing, IEEE Transactions on Visualization and Computer Graphics, vol. 15, no. 6, pp , 9. [3] O. Bimber, A. Emmerling, and T. Klemmer, Embedded Entertainment with Smart Projectors, IEEE Computer, vol. 38, no. 1, pp , Jan. 5. [4] W. Zou and H. Xu, Colorimetric Color Reproduction Framework for Screen Relaxation of Projection Display, Displays, vol. 3, no. 5, pp , Dec. 11. [5] Yun Zhou, Tao Xu, Bertrand David, and René Chalon, Where is mobile projection interaction going? the past, present and future of the mobile projected interface, in Human-Computer Interaction. Applications and Services, pp Springer, 14. [6] M. Pedersen, N. Bonnier, J. Y. Hardeberg, and F. Albregtsen, Attributes of Image Quality for Color Prints, Journal of Electronic Imaging, vol. 19, no. 1, pp , Jan. 1. [7] G. M. Johnson, Measuring images: Differences, quality, and appearance, Ph.D. thesis, Rochester Institute of Technology, 3. [8] J. You, L. Xing, A. Perkis, and X. Wang, Perceptual Quality Assessment for Stereoscopic Images based on D Image Quality Metrics and Disparity Analysis, in International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ, USA, 1, pp [9] T. M. Lehtimaki, K. Saaskilahti, T. Pitkaaho, and T. J. Naughton, Evaluation of Perceived Quality Attributes of Digital Holograms Viewed with A Stereoscopic Display, in Euro-American Workshop on Information Optics, Helsinki, Finland, July 1, pp. 1 3, IEEE. [1] J. B. Thomas and A. M. Bakke, A Colorimetric Study of Spatial Uniformity in Projection Displays, in Computational Color Imaging, A. Trémeau, R. Schettini, and S. Tominaga, Eds., vol. 5646, pp Springer Berlin Heidelberg, 9. [11] M. Strand, J. Y. Hardeberg, and P. Nussbaum, Color Image Quality in Projection Displays: A Case Study, in Image Quality and System Performance II, Proceedings of SPIE, R. Rasmussen and Y. Miyake, Eds., San Jose, CA, USA, Jan. 5, pp , SPIE. [1] A. Majumder and R. Stevens, Perceptual Photometric Seamlessness in Projection-based Tiled Displays, ACM Transactions on Graphics, vol. 4, no. 1, pp , Jan. 5. [13] I. Juricevic, L. Land, A. Wilkins, and M. A. Webster, Visual Discomfort and Natural Image Statistics, Journal of Vision, vol. 9, no. 8, pp , Mar. 1. [14] V. L. Jaya and R. Gopikakumari, IEM : A New Image Enhancement Metric for Contrast and Sharpness Measurements, International Journal of Computer Applications, vol. 79, no. 9, pp. 1 9, 13. [15] R. Hassen, Z. Wang, and M. M. A. Salama, Image Sharpness Assessment Based on Local Phase Cohernce, IEEE Transactions on Image Processing, vol., no. 7, pp , 13. [16] K. Juha and H. Saarelma, Objective Quality Potential Measures of Natural Color Images, Journal of Imaging Science and Technology, vol. 4, no. 3, pp. 5 63, [17] D. Williams and P. D. Burns, Measuring and Managing Digital Image Sharpening, in Proceedings of IS&T Archiving Conference, Bern, Swizerland, 8, pp , SPIE. [18] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, Image Quality Assessment: from Error Visibility to Structural Similarity, IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 6 61, Apr. 4. [19] P. Marziliano, F. Dufaux, S. Winkler, and T. Ebrahimi, Perceptual Blur and Ringing Metrics: Application to JPEG, Signal Processing: Image Communication, vol. 19, no., pp , Feb. 4. [] L. Zhang, X. Mou, and D. Zhang, FSIM: A Feature Similarity Index for Image Quality Assessment., IEEE Transactions on Image Processing, vol., no. 8, pp , Aug. 11. [1] H. R. Sheikh and A. C. Bovik, Image Information and Visual Quality, IEEE Transactions on Image Processing, vol. 15, no., pp , Feb. 6. [] D. M. Chandler and S. S. Hemami, VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images, IEEE Transactions on Image Processing, vol. 16, no. 9, pp , Sept. 7. [3] J. Caviedes and F. Oberti, A New Sharpness Metric Based on Local Kurtosis, Edge and Energy Information, Signal Processing: Image Communication, vol. 19, no., pp , Feb. 4. [4] A. Maalouf and M.-C. Larabi, A No Reference Objective Color Image Sharpness Metric, in European Signal Processing Conference, Aalborg, Denmark, 1, pp [5] Z. Cao, Z. Wei, and G. Zhang, A No-Reference Sharpness Metric Based on Structured Ringing for JPEG Images, Advances in Optical Technologies, vol. 14, pp. 1 13, 14. [6] B. Samira and M. Lindsay, Colour Difference Metrics and Image Sharpness, in Color and Imaging Conference Final Program and Proceedings, Scottsdale, AZ, USA,, pp [7] C. T. Vu and D. M. Chandler, S3: A Spectral and Spatial Sharpness Measure, in International Conference on Advances in Multimedia, Colmar, France, July 9, pp , IEEE. [8] A. Leclaire and L. Moisan, No-reference Image Quality Assessment and Blind Deblurring with Sharpness Metrics Exploiting Fourier Phase Information, Journal of Mathematical Imaging and Vision, 14. [9] Niranjan D. Narvekar and Lina J. Karam, A No-Reference Image Blur Metric Based on the Cumulative Probability of Blur Detection, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, vol., no. 1, pp , 11. [3] N. D. Narvekar and L. J. Karam, A No-reference Perceptual Image Sharpness Metric based on A Cumulative Probability of Blur Detection, in International Workshop on Quality of Multimedia Experience, K. O. Egiazarian, S. S. Agaian, A. P. Gotchev, J. Recker, and G. Wang, Eds., San Diego, CA, USA, July 9, pp , IEEE. [31] N. G. Sadaka, L. J. Karam, R. Ferzli, and G. P. Abousleman, A No-reference Perceptual Image Sharpness Metric based on Saliencyweighted Foveal Pooling, in IEEE International Conference on Image Processing, San Diego, CA, USA, 8, pp , IEEE. [3] R. Ferzli and J. L. Karam, Human Visual System Based No-Reference Objective Image Sharpness Metric, in International Conference on Image Processing, Atlanta, GA USA, 6, pp , IEEE. [33] Z. Wang and E. P. Simoncelli, Local Phase Coherence and the Perception of Blur, in Neural Information Processing Systems, Whistler, Canada, 3, number December 3, pp. 9 11, MIT Press. [34] R. Ferzli and L. J. Karam, A No-reference Objective Image Sharpness Metric based on The Notion of Just Noticeable Blur (JNB), IEEE Transactions on Image Processing, vol. 18, no. 4, pp , Apr. 9. [35] E. Martinec and P. Lee, AMAZE Demosaicing Algorithm, 1. [36] X. Liu, M. Pedersen, and J. Y. Hardeberg, CID:IQ - A New Image Quaity Database, in International Conference on Image and Signal Processing, Abderrahim Elmoataz, Olivier Lezoray, Fathallah Nouboud, and Driss Mammass, Eds., Cherbourg, Normandy, France, 14, vol. 859 of Lecture Notes in Computer Science, pp. 193, Springer International Publishing. [37] P. Zhao, M. Pedersen, J. Thomas, and J. Y. Hardeberg, Perceptual Spatial Uniformity Assessment of Projection Displays with a Calibrated Camera, in Color and Imaging Conference, Boston, MA, USA, 14, pp , Society for Imaging Science and Technology. [38] P. Zhao, M. Pesersen, J. Y. Hardeberg, and J. B. Thomas, Image Registration for Quality Assessment of Projection Displays, in IEEE International Conference on Image Processing, Paris, France, 14, pp , IEEE. [39] CIE, Guidelines for the evaluation of gamut mapping algorithms, Technical Report ISBN: , CIE TC8-3, 156:4. [4] International Telecommunication Union, Recommendation itu-r bt.5-11: Methodology for the subjective assessment of the quality of television pictures, Technical report, International Telecommunication Union/ITU Radiocommunication Sector, 9. [41] P. G. Engeldrum, Psychometric Scaling: A Toolkit for Imaging Systems Development, Imcotek Pr,.