Problem Session 6. Computa(onal Imaging and Display EE 367 / CS 448I

Size: px

Start display at page:

Download "Problem Session 6. Computa(onal Imaging and Display EE 367 / CS 448I"

Kory Lewis
5 years ago
Views:

1 Problem Session 6 Computa(onal Imaging and Display EE 367 / CS 448I

2 Topics Photo- electron shot- noise SNR of an image with Poisson noise Wiener Richardson- Lucy Richardson- Lucy + TV prior Simula@on of a single pixel camera Least norm ADMM + TV prior ADMM + NLM

3 Photo- electron shot- noise average

4 Task 1: SNR You re asked to compare the SNR of two cameras: Consumer camera scmos At different modes: FluXer shuxer Burst Calculate the SNR and discuss your results.

5 SNR Flu4er shu4er: temporally modulated aperture paxern. Used, for example, for bexer deblurring (see Coded Exposure Photography: Mo5on Deblurring using Flu<ered Shu<er, Raskar et al.). The result is a single image. Burst: acquires mul@ple short- exposure images.

6 SNR Consumer camera: Lots of photons, moderate Gaussian noise. CMOS microscope sensor: photon- limited imaging, only a few photons. The sensor is cooled, so no Gaussian noise, but there is photo- electron shot- noise, which has a Poisson distribu@on.

7 SNR Both mean and variance

8 SNR For each of the four cases, calculate: The average number of photons (the signal) The standard of the noise Divide (signal / noise) Don t forget to describe your results and conclusions.

9 Task 2: Poisson

10 Poisson Note: as we saw in week 3 (using ADMM) will not work well. This is because the minimiza@on objec@ve (L2 norm) is op@mal for Gaussian noise, not Poisson noise. See notes! If you want to try ADMM for the Poisson case, complete the bonus ques@on (worth 25 points!).

11 Poisson Use 1 if images are normalized to 1

12 Poisson

13 Wiener Original With blurring and noise Wiener blurry

14 Poisson Noisy and blurry image Poisson process Image Blur kernel

15 Poisson Poisson PMF

16 Poisson (Using )

17 Poisson

18 Poisson (current x) Element- wise This is what you ll implement

19 Poisson Richardson- Lucy Very noisy!! What should we do? Add a regulariza@on term (prior)!

20 Poisson

21 Richardson- Lucy Original Richardson- Lucy Richardson- Lucy + TV prior

22 Poisson Not part of the HW

23 Poisson

24 Poisson

25 Task 3: Single pixel camera Instead of an image using N pixels, we record fewer (M) pixels by: 1. Mul@plying the scene with M random masks 2. Summing over all the unmasked pixels and recording a single- pixel measurement 3. Reconstruc@ng the scenes a) Least norm solu@on with conjugate gradients b) ADMM with TV prior c) ADMM with self- similarity prior

26 Single pixel camera

27 Single pixel camera

28 Single pixel camera: Least norm

29 The conjugate gradient method

30 Single pixel camera: Least norm Original Compression factor = 1 Compression factor = 2 Compression factor = 4 Compression factor = 8 Results are not great

31 From denoising to solving the single- pixel camera

From denoising to solving the single- pixel camera For a self- similarity prior, we can use non- local- means denoising: Here the argument is the variance, but no@ce that in some

32 From denoising to solving the single- pixel camera For a self- similarity prior, we can use non- local- means denoising: Here the argument is the variance, but no@ce that in some implementa@ons the argument will be the standard devia@on (This is more of a heuris@c, or we can define NLM to be the solu@on for the denoising problem with a self- similarity prior)

33 From denoising to solving the single- pixel camera

34 From denoising to solving the single- pixel camera

36 X update We can use an solver to compute this (see following slides).

37 Z update Looks exactly like the denoising We can use any Gaussian denoising algorithm (MAP for Gaussian noise) as the proximal operator! That s the big insight! We can use denoising algorithms for image reconstruc@on Total varia@on prior: The gradients are sparse This func@on matches the prior: z=dx, the gradients, can have high values, but low values go to zero Self similarity prior: Denoise with NLM The imag

38 Single pixel camera: ADMM + TV prior regulariza@on

39 Single pixel camera: ADMM + TV prior See notes for deriva@on!

40 Single pixel camera: ADMM + TV prior Compression factor = 1 Compression factor = 2

41 Single pixel camera: ADMM + TV prior

42 Single pixel camera: ADMM + TV prior

43 Hints for task 3 a, b Images are grayscale. 3 rd dimension can be used for the N masks. Create N random masks (use rand), where N=(# of pixels in image)/(compression) Define func@ons for A, A T, and AA T : A performs element- wise mul@plica@on of the 2D input x (which has N elements) with M 2D masks, and sums the results. The output is a vector size M. (b = Ax) A T mul@plies the M masks with the M values of the input and sums the results. The output is a 2D matrix with N elements. (A T b) AA T can be defined using the previous func@ons. Noise is added aoer applying the masks: For conjugate gradients, use 50 itera@ons and a tolerance of For ADMM, use the provided func@ons: opdx, opdtx, which work on non- vectorized images. You can also define a handle for D T D: What is the func@on handle for CG? No@ce reshaping the func@on to the correct dimensions! The residual for ADMM is,, plot the log residual

44 Single pixel camera: ADMM + NLM

45 Single pixel camera: ADMM + NLM

46 Comparison

47 Hints for task 3 c

48 Single pixel camera: d

49 Single pixel camera: ADMM + TV prior Compression factor Compression factor PSNR table

50 Single pixel camera: ADMM + NLM Compression factor Compression factor PSNR table

51 Have a nice weekend! And good luck with the homework!

Problem Set 3. Assigned: March 9, 2006 Due: March 23, (Optional) Multiple-Exposure HDR Images

Problem Set 3. Assigned: March 9, 2006 Due: March 23, (Optional) Multiple-Exposure HDR Images 6.098/6.882 Computational Photography 1 Problem Set 3 Assigned: March 9, 2006 Due: March 23, 2006 Problem 1 (Optional) Multiple-Exposure HDR Images Even though this problem is optional, we recommend you