Single Camera Calibra<on

Transcription

1 Laboratory of Image Processing Single Camera Calibra<on Pier Luigi Mazzeo

2 Prepare the Pa>ern, Camera and Images For best results, use between 10 and 20 images of the calibra<on pa>ern. The calibrator requires at least three images. Use uncompressed images or lossless compression formats such as PNG. The calibra<on pa>ern and the camera setup must sa<sfy a set ofrequirements to work with the calibrator The Camera Calibrator app uses a checkerboard pa>ern. A checkerboard pa>ern is a convenient calibra<on target. If you want to use a different pa>ern to extract key points, you can use the camera calibra<on MATLAB func<ons directly. You can print (from MATLAB) and use the checkerboard pa>ern provided. The checkerboard pa>ern you use must not be square. One side must contain an even number of squares and the other side must contain an odd number of squares. Therefore,the pa>ern contains two black corners along one side and two white corners on the opposite side. This criteria enables the app to determine the orienta<on of the pa>ern. The calibrator assigns the longer side to be the x-direc<on.

3 Check-board Pa>ern To prepare the checkerboard pa>ern: 1. A>ach the checkerboard printout to a flat surface. Imperfec<ons on the surface can affect the accuracy of the calibra<on. 2. Measure one side of the checkerboard square. You need this measurement for calibra<on. The size of the squares can vary depending on printer sexngs. 3 To improve the detec<on speed, set up the pa>ern with as li>le background clu>er as possible.

4 Camera Setup and images capture To properly calibrate your camera, follow these rules: Keep the pa>ern in focus, but do not use autofocus. Do not change zoom sexngs between images. Otherwise the focal length changes. Capture the images of the pa>ern at a distance roughly equal to the distance from your camera to the objects of interest. For example, if you plan to measure objects from 2 meters, keep your pa>ern approximately 2 meters from the camera. Place the checkerboard at an angle less than 45 degrees rela<ve to the camera plane. Do not modify the images. For example, do not crop them. Do not use autofocus or change the zoom between images. Capture the images of a checkerboard pa>ern at different orienta<ons rela<ve to the camera. Capture enough different images of the pa>ern so that you have covered as much of the image frame as possible. Lens distor<on increases radially from the center of the image and some<mes is not uniform across the image frame. To capture this lens distor<on, the pa>ern must appear close to the edges.

5 Checkboard capturing examples

6 Example Create a set of calibra<on images. images = imageset(fullfile(toolboxdir('vision'),'visiondata',... 'calibra<on','fisheye')); imagefilenames = images.imageloca<on; imshow(imagefilenames {1}) Detect the calibra<on pa>ern. [imagepoints, boardsize] = detectcheckerboardpoints(imagefilenames);

7 Example Generate the world coordinates of the corners of the squares. squaresizeinmm = 29; worldpoints = generatecheckerboardpoints(boardsize,squaresizeinmm); Calibrate the camera. params = es<matecameraparameters(imagepoints,worldpoints); Visualize the calibra:on accuracy. showreprojec<onerrors(params);

8 Example Plot detected and reprojected points. The third number indicates the image in which we reproject the points. figure; imshow(imagefilenames{1}); hold on; plot(imagepoints(:,1,1), imagepoints(:,2,1),'go'); plot(params.reprojectedpoints(:,1,1),params.reprojectedpoints(:,2,1),'r+'); legend('detected Points','ReprojectedPoints'); hold off;

9 ShowExtrinsics You can quickly discover obvious errors in your calibra<on by ploxng rela<ve loca<ons of the camera and the calibra<on pa>ern. Use the showextrinsics func<on to either plot the loca<ons of the calibra<on pa>ern in the camera's coordinate system, or the loca<ons of the camera in the pa>ern's coordinate system. Look for obvious problems, such as the pa>ern being behind the camera, or the camera being behind the pa>ern. Also check if a pa>ern is too far or too close to the camera. [params, ~, estimationerrors] = estimatecameraparameters(imagepoints, worldpoints); figure; showextrinsics(params, 'CameraCentric'); figure; showextrinsics(params, 'Pa>ernCentric');

10 Show extrinsic On this figure, the frame (Oc,Xc,Yc,Zc) is the camera reference frame. The red pyramid corresponds to the effec<ve field of view of the camera defined by the image plane.

11 World-centered view Every camera posi<on and orienta<on is represented by a green pyramid

12 Es<mate errors Es<ma<on errors represent the uncertainty of each es<mated parameter. The es<matecameraparameters func<on op<onally returns es<ma<onerrors output, containing the standard error corresponding to each es<mated camera parameter. The returned standard error sigma (in the same units as the corresponding parameter) can be used to calculate confidence intervals. For example +/ sigma corresponds to the 95% confidence interval. In other words, the probability that the actual value of a given parameter is within 1.96 sigma of its es<mate is 95%. displayerrors(es<ma<onerrors, params);

13 Interpre<ng Principal Point Es<ma<on error The principal point is the op<cal center of the camera, the point where the op<cal axis intersects the image plane. You can easily visualize and interpret the standard error of the es<mated principal point. Plot an ellipse around the es<mated principal point (cx,cy), whose radii are equal to 1.96 <mes the corresponding es<ma<on errors. The ellipse represents the uncertainty region, which contains the actual principal point with 95% probability. principalpoint = params.principalpoint; principalpointerror = es<ma<onerrors.intrinsicserrors.principalpointerror; fig = figure; ax = axes('parent', fig); imshow(imagefilenames{1},'ini<almagnifica<on', 60, 'Parent', ax); hold(ax, 'on'); plot(principalpoint(1), principalpoint(2), 'g+', 'Parent', ax);

14 Interpre<ng Principal Point Es<ma<on error halfrectsize = 1.96 * principalpointerror; rectangle('posi<on', [principalpoint-halfrectsize, 2 * halfrectsize],... 'Curvature', [1,1], 'EdgeColor', 'green', 'Parent', ax); legend('es<mated principal point'); <tle('principal Point Uncertainty'); hold(ax, 'off');

15 Interpre<ng transla<on vectors es<ma<on You can also visualize the standard errors of the transla<on vectors. Each transla<on vector represents the transla<on from the pa>ern's coordinate system into the camera's coordinate system. Equivalently, each transla<on vector represents the loca<on of the pa>ern's origin in the camera's coordinate system. You can plot the es<ma<on errors of the transla<on vectors as ellipsoids represen<ng uncertainty volumes for each pa>ern's loca<on at 95% confidence level.

16 Remove Distorsion from an image This example shows you how to use the cameraparameters object in a workflow to remove distor<on from an image. The example creates a cameraparameters object manually. In prac<ce, use the es<matecameraparameters or the cameracalibrator app to derive the object. Create a cameraparameters object manually. IntrinsicMatrix = [ ; ; ]; radialdistortion = [ ]; cameraparams = cameraparameters('intrinsicmatrix',intrinsicmatrix,... 'RadialDistortion',radialDistortion);

17 Remove Distorsion from an image Remove distorsion from image I = imread(fullfile(matlabroot,'toolbox','vision','visiondata',... 'calibration','fisheye','image01.jpg')); J = undistortimage(i,cameraparams); Display the original and undistorted images. figure; imshowpair(imresize(i,0.5), imresize(j,0.5), 'montage'); title('original Image (left) vs. Corrected Image (right)');

18 Measure planar objects %% Prepare Calibration Images % Create a cell array of file names of calibration images. numimages = 9; files = cell(1, numimages); for i = 1:numImages files{i} = fullfile(matlabroot, 'toolbox', 'vision', 'visiondata',... 'calibration', 'slr', sprintf('image%d.jpg', i)); end % Display one of the calibration images magnification = 25; figure; imshow(files{1}, 'InitialMagnification', magnification); title('one of the Calibration Images');

19 Measure planar objects %% Estimate Camera Parameters % Detect the checkerboard corners in the images. [imagepoints, boardsize] = detectcheckerboardpoints(files); % Generate the world coordinates of the checkerboard corners in the % pattern-centric coordinate system, with the upper-left corner at (0,0). squaresize = 29; % in millimeters worldpoints = generatecheckerboardpoints(boardsize, squaresize); % Calibrate the camera. cameraparams = estimatecameraparameters(imagepoints, worldpoints);

20 Measure planar objects Read the Image of Objects to Be Measured imorig = imread(fullfile(matlabroot, 'toolbox', 'vision', 'visiondata',... 'calibration', 'slr', 'image9.jpg')); figure; imshow(imorig, 'InitialMagnification', magnification); title('input Image');

21 Undistort the image Measure planar objects Use the cameraparameters object to remove lens distor<on from the image. This is necessary for accurate measurement. [im, neworigin] = undistortimage(imorig, cameraparams, 'OutputView', 'full'); figure; imshow(im, 'InitialMagnification', magnification); title('undistorted Image');

22 Measure planar objects Detect Coins boxes=[ ; ]; % Reduce the size of the image for display. scale = magnification / 100; imdetectedcoins = imresize(im, scale); % Insert labels for the coins. imdetectedcoins = insertobjectannotation(imdetectedcoins, 'rectangle',... scale * boxes, moneta'); figure; imshow(imdetectedcoins); title( Monete');

23 Measure planar objects Compute Extrinsics To map points in the image coordinates to points in the world coordinates we need to compute the rota:on and the transla:on of the camera rela:ve to the calibra:on pabern. Note that the extrinsics func:on assumes that there is no lens distor:on. In this case imagepoints have been detected in an image that has already been undistorted using undistortimage. % Detect the checkerboard. [imagepoints, boardsize] = detectcheckerboardpoints(im); % Compute rotation and translation of the camera. [R, t] = extrinsics(imagepoints, worldpoints, cameraparams);

24 Measure planar objects Measure the first coin To measure the first coin we convert the top-leh and the top-right corners of the bounding box into world coordinates. Then we compute the Euclidean distance between them in millimeters. Note that the actual diameter of a US penny is mm. box1 = double(boxes(1, :)); imagepoints1 = [box1(1:2);... box1(1) + box1(3), box1(2)]; % Get the world coordinates of the corners worldpoints1 = pointstoworld(cameraparams, R, t, imagepoints1); % Compute the diameter of the coin in millimeters. d = worldpoints1(2, :) - worldpoints1(1, :); diameterinmillimeters = hypot(d(1), d(2)); fprintf('measured diameter of one penny = %0.2f mm\n', diameterinmillimeters);

25 % Compute the distance to the camera. distancetocamera = norm(center1_world + t); fprintf('distance from the camera to the first penny = %0.2f mm \n',... distancetocamera); Measure planar objects Measure the distance to the first coin In addi<on to measuring the size of the coin, we can also measure how far away it is from the camera. Camera extrinsics R and t give us the transforma<on from the world coordinates into the camera coordinates. Thus the loca<on of the camera's op<cal center in the world coordinates is given by [0 0] * R' - t, which simplifies to -t. Then the distance from the camera center to any point p in the world coordinates is given by norm(p + t). % Compute the center of the first coin in the image. center1_image = box1(1:2) + box1(3:4)/2; % Convert to world coordinates. center1_world = pointstoworld(cameraparams, R, t, center1_image); % Remember to add the 0 z-coordinate. center1_world = [center1_world 0];