Primatte Hardware Assisted Keyer Plug-in Reference

Transcription

1 Primatte Hardware Assisted Keyer Plug-in Reference Release 2.0 Notice Quantel Limited accepts no responsibility for the accuracy of the information contained herein and reserves the right to change the contents without prior notice. This document does not form part of the product s Technical or Functional Specification and does not form part of any contractual agreement. This manual is a change controlled document. Please quote the revision status of this manual when reordering. This Document is Revision B. Copyright Quantel Ltd 2008 i

2 Notices Trade Marks Most of the product names mentioned in this manual are manufacturer trademarks and are used within this manual only for the purpose of identification. Quantel iq and Quantel QEffects are trademarks of Quantel Limited. Primatte is distributed and licensed by IMAGICA Corp. of America, Los Angeles, CA, USA Primatte was developed by IMAGICA Corp., Tokyo, Japan Primatte is a trademark of IMAGICA Corp., Tokyo, Japan ii

3 About This Manual The information for this manual was written by Scott Gross of Imagica Corp. of America: 3113 Woodleigh Lane Cameron Park, CA Phone: Fax: Mobile: URL: Skype: scottagross This manual was revised for Quantel by Barry Hicks of Video Design Software (VDS), and published by the Quantel Technical Publications Department. This is a change controlled document. Each page of this document is given an issue letter (shown at the bottom of each page with the drawing number and revision date) which represents the status of the page. Revision A on any page indicates that the page is the original. Any changes to any pages will raise the revision status of the document. When re-ordering, always quote the document type, the documents number and revision status along with the unit s serial number: B 10/08 E32514 Release 2.0 All pages iii

4 Contents Keying Using Primatte Hardware Assisted Plug-in Using the Primatte Hardware Assisted Plug-in 2 General Menu Operation 2 Selecting the Primatte Plug-in 3 Key Mode Selection 5 Select BG (Key) Color 6 Cleaning Foreground & Background 7 Removing Color Spill 9 Spill Process Functions 9 Fine Tuning the Composite 11 Make Foreground Transparent 12 Restore Detail 13 Spill Sampling Tools 13 Matte Sampling Tools 13 Detail Sampling Tools 13 Rendering the Whole Clip 13 The Primatte Algorithm 14 iv

5 Primatte Hardware Assisted Keyer Plug-in Reference Keying Using Primatte Hardware Assisted Plug-in 1

6 Using the Primatte Hardware Assisted Plug-in General Menu Operation Quantel offers two Primatte plug-ins, a Hardware Assisted Keyer Plug-in, (found under the Key menu in Effects), and a Software Keyer Plug-in, found under the QPlugin menu (also in Effects). The Hardware Assisted Keyer Plug-in, discussed in this reference manual, has faster performance, but fewer features and lower precision than the Software Keyer Plug-in. The Software Keyer Plug-in, discussed in a separate document, offers Primatte s full feature set and precision, but renders in software and thus may have slower performance. The screen shot below shows the Effects menu when the Quantel Keyer is enabled and the Primatte Hardware Assisted Keyer Plug-in is selected. When the Primatte Hardware Assisted Keyer plug-in is selected, the menu area at the bottom of the screen displays keying menus while the desktop provides a visual representation of the keying process being performed at the current frame position in the clip. The above screen shot shows a blue screen image, and this document uses the term blue screen throughout. However, the foreground image could just as easily utilize another key color, for example, green screen. See the Keyer chapter in the QEffects Application Reference for further details on general keyer operation. 2

7 Selecting the Primatte Hardware Assisted Keyer Plug-in To apply the Primatte Hardware Assisted Keyer Plug-in to the current layer, turn on the Key button (to the far left of the QEffects menu under Stretch ), then select the Primatte option from the blue scrolling list box to the right of the Dve button. If Primatte is not available as an option in the blue scrolling list box, please contact your Quantel representative about obtaining a Primatte license. Press reset followed by all if you need to reset the plug-in to its default state. During plug-in operation, you will see the composite displayed on the screen when the video button is on. When the key button is on, you will see the matte (alpha channel) generated by the keyer plug-in. You can use the undo and redo buttons to the lower left of the menus to undo and redo each step as you adjust your plug-in settings. 3

8 By default, the Primatte library does all internal calculations in integer mode using 16-bits per RGB component. Turn on the Floating Point box to do all internal calculations in floating point resolution. Floating point precision can produce higher quality images and is faster. You can also use the garbage button to enable and define a garbage matte, as shown below (see the Keyer chapter in the QEffects Application Reference for further details). And, use the rev button to invert the matte generated by the keyer plug-in. Garbage Matte controls (when garbage is on) 4

9 Key Mode Selection The basic functionality of the Primatte keyer plug-in is based around the group of mode selection boxes (Select BG Color, Clean BG Noise, Clean FG Noise, Spill +/-, Matte +/-, Detail +/-, Matte Sponge, Spill Sponge, Make FG Transparent, Restore Detail and Fine Tuning), and the image area of the desktop. Only one of the mode selection boxes can be on at once, and this determines how the plug-in operates when the user selects colors on the image. These different modes represent the different steps to be taken to produce a high quality composite image. The four main steps required to produce a key using the Primatte plug-in start with the Select BG Color button, which comes up highlighted as the default selection when the Primatte plug-in is first opened. 5

10 Select BG (Key) Color Turn on the Select BG Color box, then pick the key color from the image area using the pen (you will see a yellow box outlining the area you re picking as you drag, as shown in the screen shot below). When you release, you should see the composite, with keyed foreground based on the colors you ve picked. Be sure that the video button (below the timeline and to the right) is selected so you can see the composite. When picking colors, position the cursor in the key color of the image area near the foreground objects and sample the targeted background color. Hold down the pen and drag it within the image area to select and average the key color, or make a single pixel selection to sample a single color. When pen pressure is released, the compositing process will start. If the foreground shot was made under ideal lighting conditions, then 90-95% of the composite will be accomplished in this one step. TIP: If you dragged the cursor on the key color (for example, blue screen) area, Primatte averages the multi-pixel sample to get a single color to adjust to. Sometimes Primatte works best when only a single pixel is sampled instead of a range of pixels. The color selected at this point in the procedure is critical to the operation of the plug-in from this point forward. Should you encounter problems later in the process after having selected a range of key color shades, try Select BG Color again with a single dark key color pixel or single light key color pixel. TIP: If the foreground image has a shadow in it that you want to keep in the composite, do not select any of the darker pixels in the shadow, and the shadow will come along with the rest of the foreground image. Median Sampling The Median picking feature is useful to filter out noisy pixels when doing sampling and picking operations. For example, if you have a noisy image and pick a region to sample, then all the pixels in that region are used for the processing. Now, you can optionally ask Primatte to apply a 3x3 median filter on the sampled input pixels prior to performing the sampling operation. Turn on the Median Sampling box to enable this. 6

11 Cleaning Foreground & Background The second and third steps in using the Primatte plug-in require the Key (matte) to be displayed in the image area. Press on the key button below the timeline and to the right to display the matte. Press the Clean BG Noise box. If the key button is on, the image area will display the current key. If there are any white regions in the dark blue screen area, it is noise and should be removed. Move the cursor into these areas and sample these 'white noise' regions. When pen pressure is released, Primatte will process the data and eliminate the noise. Repeat this procedure as often as necessary to clear the noise from the background areas. Sometimes increasing the brightness of your monitor or changing the screen gamma allows you to see noise that would otherwise be invisible. Before Background noise removal After Background noise removal TIP: When clearing noise from around loose flying hair or any background/foreground transitional area, be careful not to select any of areas near the edge of the hair. Leave a little noise around the hair, as this can be cleaned up later using the Fine Tuning tool. TIP: Most pixels displayed as a dark color close to black in a key image will become transparent and virtually allow the background to be the final output in that area. Consequently, there is no need to eliminate all subtle noise in the blue screen portions of the image. In particular, if an attempt is made to meticulously remove noise around the foreground object, a smooth composite image is often difficult to generate. 7

12 If there are darker or gray regions in the middle of the mostly white foreground object, that is, if the key is not 100% in some portion of the targeted foreground, select the Clean FG Noise mode. Use the same techniques as with the Clean BG Noise mode, but this time, sample the darker pixels in the foreground area until that area is as white as possible. Again, stay away from the edges of the foreground object. Before Foreground Noise Removal After Foreground Noise Removal 8

13 Removing Color Spill The previous steps are necessary to create a clean 'matte' or 'key'. With this key, the foreground can be composited onto any background image. However, if there is 'spill' on the foreground object from light that was reflected off the original background, a final operation is necessary to remove that background spill to get a more natural looking composite. Before Spill Removal After Spill Removal After the first three steps mentioned on earlier pages, there may still be some blue fringing on areas around the hair or a blue tinge to the face or clothing caused by light reflecting from the physical chroma key background. There are two ways to remove the spill color in Primatte. The simple method is to select the Spill Sponge mode (ensure the video button is on so you can see the composite) and then sample the spill areas away. By just positioning the cursor over a bluish pixel on the foreground object and selecting it, the blue will disappear and be replaced by a more natural color. You can also drag on the screen to select a region of pixels containing spill. Note: This operation works on foreground 'color regions'. In the image above, samples should be made on the hair fringes, and also on the inside of the left arm below the sleeve. Spill Process functions: The Spill Process boxes, to the right of the Primatte plug-in menu, control which colors replace the foreground areas with spill. When Complement is on, Primatte replaces color spill with the background color's complementary color. For a spill (key) color of blue, this is usually a yellowish color. Complement is the default setting. 9

14 The Solid Replace function replaces color spill with a solid color, selected in the Palette menu (shown below). The replacement color in this mode is by default gray (R128, G128, B128). When Solid Replace mode is selected, the Palette menu is displayed to the right of the Spill Process buttons, allowing the user to select a color other than gray. Note: If the foreground object was a person wearing a red shirt and the user was having difficulty removing the blue spill from the shirt, he could use this feature and select a color close to the red shirt color to replace the spill. This sometimes results in a better edge around the foreground object. The Palette menu allows the selection of and displays the current Solid Replace color (by default, it is gray; R128, G128, B128). By clicking on an area of the color wheel, the user is selecting a color (other than the complement) to be used in replacing color spill. The Grey button positions the color selector cursor at the center of the palette color wheel, where shades of gray will be found. The gray scale value can be controlled from the gray scale bar (to the right of the color wheel). 10

15 Fine Tuning the Composite If the spilled color was not totally removed using the above procedure, a finetuning operation should follow for more subtle and sophisticated removal of the spilled background color. Select the Fine Tuning mode, then using the cursor, sample a small range of the pixels exhibiting color spill that you want to remove. When you lift the pen, three fine tuning number boxes will appear: Decolor, Clip Hi, and Clip Lo. At this time, Primatte will display the selected color in a color swatch box (just to the right of the number boxes). For most images, the Decolor function (which affects the Large Polyhedron, further details in the next section) is all that is required to remove any remaining color spill. Cursor movement on the Decolor control performs a color adjustment of the sampled color against the background. The more to the right the cursor moves, the less of the key color component (or spill) will be included. The more to the left the cursor moves, the closer the color component of the selected region will be to the original foreground image. If moving the control all the way to the right does not achieve the desired result, re-sample the color on the monitor image and again move the Decolor control to the right. These operations are additive and can be repeated as necessary to get the desired results. Note that when using the Decolor control in Fine Tuning mode to remove spill, spill colors will be replaced based on the setting of the Spill Process boxes (Complement and Solid Replace). TIP: It is better to make several small adjustments to the spill areas than a single large one. The two other Fine Tuning number boxes can be used in the same way for different key adjustments. The Clip Lo function controls the matte softness for the color that is closest to the background color. For example, you can recover lost rarefied smoke in the foreground by selecting the Fine Tuning operational mode, clicking on the area of the image where the smoke just starts to disappear and moving the Clip Lo control to the left. 11

16 Note that the Clip Lo control shrinks the small polyhedron and releases pixels that were close to the background color (the small polyhedron contains all the blue or green background colors and is described further in the next section). The Clip Lo Control is useful for restoring foreground image pixels that were lost in the original sample because they were so similar to the background color. The Clip Hi function controls the matte softness for the color that is closest to the foreground color. For example, if you have thick and opaque smoke in the foreground, you can make it semi-transparent by moving the Clip Hi control to the right after selecting the pixels in the Fine Tuning mode. Note that the Clip Hi control affects the Medium Polyhedron (further details in the next section) and adjusts the transparency of the matte against the sampled color. The more to the right this cursor is moved, the more transparent the foreground object becomes in the selected color region. TIP: If the foreground image changed color dramatically during the finetuning process, you can recover the original color by re-sampling an area of the off-color foreground image and moving the Decolor control slightly to the left. This may introduce some spill back into that color region. Again, use the Fine Tuning option to suppress the spill, but make smaller adjustments this time. If these final 'spill suppression' operations have changed the final compositing results, you may have to return to earlier operations to clean up the matte. If the Composite view looks good, but a 100% foreground area has become slightly transparent, you can clean those transparent areas up by using the Matte Sponge function. After selecting Matte Sponge, just click on the transparent pixels (gray on the white foreground object when viewing the key), and they will become 100% foreground (or white). All of the spillsuppression information will remain intact during this operation. Alternatively, using the key (matte) view, select Fine Tuning mode, then select those transparent areas in the image and move the Clip Hi control slightly to the left. This will move that color region from 0-99% foreground with spill suppression to 100% foreground with spill suppression and should solve the problem. Make Foreground Transparent When this mode is selected, the opaque foreground color region sampled in the image window becomes slightly translucent. This operation is useful for making foreground objects that are otherwise 100 percent covered with smoke or clouds visible. To use this mode, select the Make FG Transparent box then drag on the image and release to select the desired color region of interest in the foreground that you want to make transparent. 12

17 Restore Detail With this mode selected, the completely transparent background region sampled in the image window becomes translucent. This operation is useful for restoring lost hair details, thin wisps of smoke and the like. It shrinks the small polyhedron slightly. To use this mode, select the Restore Detail box, then drag on the image and release to select the color region of interest that you want to make translucent. Spill Sampling Tools (Spill + and Spill -) Using the Spill(+) and Spill(-) modes, you can gradually remove or recover the spill intensity on the foreground object by sampling the referenced color region repeatedly. The conventional Spill Sponge tool removes the spill component in a single action at one level and does not allow sampling the same pixel a second time. Even though just a small amount of spill needed to be removed, the Spill Sponge removed a preset amount without allowing any finer adjustment. The Matte Sampling Tools Effect of Spill(+/-) Repeatable Sampling The Matte(+) and Matte(-) modes are used to thicken or attenuate the matte information. If you want a thinner shadow on a foreground object, you can use the Matte(-) mode as many times as you like to make it more transparent. On the other hand, you can use the Matte(+) mode to make the matte thicker in that color region. The Detail Sampling Tools Effect of Matte(+/-) Repeatable Sampling The Detail(+) and Detail(-) modes are a refined version of Clean BG Noise and Restore Detail. For example, when you see some dilute noise in the backing area but don't want to remove it completely because it affects some fine detail in a different area, try using Detail(-). It will attenuate the noise gradually as multiple samples are made on the pixel. You should stop the sampling when important fine details start to disappear. 13 Effect of Detail (+/-) Repeatable Sampling Rendering the Whole Clip When the desired composite has been achieved, the whole clip needs to be processed before it can be used elsewhere. Select the Render box to start the rendering process. When complete, the resulting composite will be available in the library. See the Using QEffects chapter in the QEffects Application Reference for further details on rendering.

18 The Primatte Algorithm Explanation of how Primatte works... The Primatte chroma key algorithm is a sophisticated method of color space segmentation that can be easily explained to help a user achieve maximum effectiveness with the tool. Basically, Primatte segments all the colors in the foreground image into one of four separate categories. The result is a 'spill suppressed' foreground image and a matte which is used to apply the modified foreground to a suitable background. Primatte works in 3D RGB color space. Here is a visual representation of the Primatte algorithm after an image has been processed. By operating the Primatte interface, the user essentially creates three concentric, multi-faceted polyhedrons. These can be pictured as three globes (polyhedrons or polys), one within the other, which share a common center point. The creation of these polyhedrons separates all possible foreground colors into one of four regions; inside the small polyhedron (1), between the small and medium polyhedrons (2), between the medium and the large polyhedrons (3) and outside the large polyhedron (4). 14

19 The four regions created are described as follows: Region 1 (inside the small polyhedron) - This region contains all of the foreground image colors that are considered 100% background. These are the green or blue or whatever colors that were used as the backing (key) color of the foreground image. Region 2 (between the small and medium polyhedrons) - This region contains all the foreground colors that are at the edges of the foreground object(s), in glass, glass reflections, shadows, sheets of water and other transparent and semi-transparent color regions. These color regions also have spill suppression applied to them to remove color spill from the backing screen. Region 3 (between the medium and large polyhedrons) - This region contains all the foreground image colors that are 100% foreground but have spill suppression applied to them to remove color spill from the backing screen. Otherwise they are 100% solid foreground colors. Region 4 (outside the large polyhedron) - This region contains all the 100% foreground image colors that are not modified from the original foreground image. There is no spill suppression applied to these colors. 15

20 In the first step in using Primatte (Select BG Color), the user is asked to indicate the backing (key) color on the original foreground image. The sample should usually be taken from a 'medium shaded' area near the foreground object. By 'medium shaded' area, it is meant that if green is the backing color and the green area of the foreground image has many shades of green ranging from very pale green to almost black, a shade of green in-between these extreme ranges should be chosen. If good results are not obtained using this sample, Primatte should be reset and another sample taken using a slightly darker or lighter shade of green. The first sample of Primatte often determines the final result as the center point of all three polyhedrons is created based on this first sample. A single pixel may be selected or a range of pixels (rectangular sample). If a range of pixels is taken, the sample will be averaged to get a single color sample. This single pixel or averaged color sample then becomes the center of the small polyhedron. A few other shades around that color are included in the original small polyhedron. NOTE: It is recommended that a single pixel be selected as the first sample as you then have some idea where the center point of the polyhedrons is located. If a rectangular sample is made, you can only guess at the average color that ends up being the center point. You can get an idea how this sample affects the algorithm by resetting the Primatte plug-in, going to the Matte View ( key button on) and clicking around on the green or blue screen area while in the Select BG Color operation mode. You can immediately see the results of the initial settings of the polyhedrons in this way. After making a sample of the backing screen (key) color in the first step, the result is a small golf ballshaped poly as shown in the following image. The second step in using Primatte is to clean up the backing color area by adding additional shades of green or blue to the small poly. This second step (Clean BG Noise) is usually executed while viewing the black and white Matte (Key) View. Before BG Noise Removal After BG Noise Removal 16

21 While in the Clean BG Noise sampling mode, the user samples the white milky regions as shown in the left-hand image above. As the user samples these regions, they turn to black as shown in the right-hand image above. What is happening in the Primatte algorithm is that these new shades of green or blue (the white milky areas in the matte view) are added to the small poly. The left hand image below shows the new pixels sampled (white dots) in relation to the small poly, and the image next to it shows how the small poly extends outward to encompass the newly sampled colors into the small poly. The advantage of this technique is that the polyhedron distorts to enclose only the shades of blue or green that are in the backing screen. Other shades of blue or green around these colors are left undisturbed in the foreground. Other chroma keyers expand from a golf ball-sized shape to a baseball to a basketball to a beach ball. Since it expands in all directions, many shades of color are relegated to 100% background making it hard to get good edges around the foreground objects. Now that the user has created a small polyhedron, they must shape the medium and large polys. A default medium and large poly are both automatically created and are then modified based on the next couple of Primatte operations. The third Primatte step (Clean FG Noise) is to sample and eliminate gray areas in the 100% foreground area of the image. Before FG Noise Removal After FG Noise Removal 17

22 Again, the user makes several samples on the dark, grayish areas on the foreground object until it is solid white in color. Primatte is shaping the large polyhedron with each color region that is sampled. Care should be taken in both this and the previous steps to not sample too close to the edges of the foreground object. Getting too close to the foreground object's edges will result in hard edges around the foreground object. Primatte uses these samples to modify and shape the polys to the desired shape. At this point, the matte or key has been created and would allow the foreground objects to be composited into a new background image. If the user changes the display mode from black and white Matte View ( key button on) to color Composite View ( video button on), there is usually 'color spill' on the edges (and sometimes the center) of the foreground objects. When on the edges of the foreground object, this spill comes from where the edges of the foreground object blended into the backing color. If it is on the center of the foreground object, it usually results from reflected color from the backing screen. The next Primatte step, either Spill Sponge, Fine Tuning or Spill(-), can now be used to eliminate this spill color. Let's take a look at what is happening in the Primatte algorithm while this next step is performed. Here is what the various tools in Primatte do to the Polyhedrons when they are used: As you can see above, the Spill Sponge bulges the large polyhedron in the color region specified. A color region is specified by clicking on the image in a particular area with spill present. For example, if the user clicks on some spill on the cheek of a foreground person, Primatte goes to the section of the large polyhedron closest to that particular flesh tone and bulges the polyhedron there. As a result, the flesh tones move from outside the large poly to in-between the medium and large polys. This is Region 3 and, if you remember, is 100% foreground with spill suppression. As a result of the suppression, the spill is removed from that cheek color and all other shades of that color on the foreground. The user would then continue to sample 18

23 areas of the image where spill exists and each sample would remove spill from another color region. When all spill has been removed, the user should have a final composite. As a last step, the user should go back to the Matte View ( key button on) and make sure that gray, transparent areas have not appeared in the foreground area. If there are any, the Matte Sponge operation mode should be selected and those gray pixels should be sampled until they have all turned white again. The Matte Sponge and Spill Sponge tools bulge or dent the polyhedrons a preselected amount. If the desired results are not achieved or the results are too extreme for the image, a manual method can be applied. The user should choose the Fine Tuning sliders, select a color region of interest and then move the appropriate slider to get the desired results. For example, to remove spill, select a region of the composite image with spill on it. Move the Decolor (large poly) slider to the right a little bit - the large poly will bulge and the spill should disappear. Move it a little more, if necessary. Moving this slider to the right removes spill (moves the colors from outside the large poly to between the medium and large polyhedrons), and moving it to the left dents the large poly and moves that color region to outside the large poly. If the user samples a foreground object shadow and then moves the Clip Hi (medium poly) slider to the right, the shadow will become more transparent. This is useful for matching composited shadows to shadows on the plate photography. It can also be used to make clouds or smoke more transparent. If some foreground detail disappears during the composite, the user can select where the detail should be and then move the Clip Lo (small poly) slider to the left. This dents the small poly in that color region and releases the detail pixels from the small poly into the visible region between the small and medium polyhedrons. The Spill Sponge and Matte Sponge tools are 'shortcut tools' that automatically move the sliders a pre-selected amount as a timesaving step for the user. Other 'shortcut tools' include the Make FG Transparent tool and the Restore Detail tool. 19

24 These 'shortcut tools' are one-step operations where the user clicks on a color region of interest and Primatte performs a pre-calculated operation. Hopefully, most operations using Primatte would only require these tools, but the manual operation of the sliders is always an option. The Spill(-) tool bulges the large poly a small amount incrementally in the color region that is clicked on and the Spill(+) tool dents it a small amount with each click. The Matte(-) and Matte(+) tools do the same to the medium poly and the Detail(-) and Detail(+) do it to the small poly. 20

25 21

26 Index Clean BG Noise 7, 16, 17 Clean FG Noise 8, 17 Clip Hi 12, 19 Clip Lo 11, 19 Complement Replace 9 Composite View 18 Decolor 11, 19 Detail +, - 13, 20 Fine Tuning 11, 18, 19 Floating Point 4 Garbage Matte 4 Grey button 10 Hardware Assisted Plug-in 2 Key button 3 Make Foreground Transparent 12, 19 Matte +, - 13, 20 Matte Sponge 12, 19 Matte View 16, 18 Median Sampling 6 Palette Menu 10 Polyhedrons Redo 3 Reset 3 Restore Detail 13, 19 Select BG (Key) Color 6, 16 Software Plug-in 2 Solid Replace 10 Spill +, - 13, 18, 20 Spill Process 9 Spill Sponge 9, 18, 19 Undo 3 Video button 3 22