Processing Time Lapse Astro Images with RawTherapee Axel Mellinger Department of Physics Central Michigan University & Sunset Astronomical Society June 10, 2016 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 1 / 24 1/24
Outline 1 Introduction: RAW vs. JPEG 2 How Color is Recorded: Bayer Filter and Gamma 3 Recording Twilight Time-Lapse Videos 4 RawTherapee Demo 5 Exposure Value Smoothing and Video Processing Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 2 / 24 2/24
Introduction: RAW vs. JPEG JPEG (or JPG): A lossy compression format Nearly universal format for digital cameras Uses lossy compression (i. e. throws away image information) Reasonably small file sizes Example: Canon T2i (18 Megapixels): 5184 3456 2.5 4 MB (depending on quality settings) each color (red, green, blue) is represented by an 8-bit number 2 8 = } 2 2 {{... 2} = 256 numbers (from 0... 255) 8 factors total number of colors: 256 256 256 = 16,777,216 Now, this sounds like a huge number... but it can still be too small in certain situations (more later... ) Non-linear (doubling the light intensity does not double the brightness value in the JPEG file) Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 4 / 24 4/24
Introduction: RAW vs. JPEG Raw Format: Whatever the Camera Manufacturer Decides... Manufacturer- and camera-specific Canon:.cr2 Nikon:.nef Sony:.arw,.srf,.sr2 Often lossless compression Much larger file sizes. Example: Canon T2i (18 Megapixels): 5184 3456 20 30 MB Canon: each color (red, green, blue) is represented by a 14-bit number 2 14 = } 2 2 {{... 2} = 16,384 numbers 14 factors linear image format (doubling the light intensity doubling of the brightness value) But wait, there is more... Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 5 / 24 5/24
How Color is Recorded Bayer Filter Named after its inventor, Bryce Bayer of Eastman Kodak. Camera sensors are black and white only! For color images: Take 3 separate images through red, green and blue filters or Take single image through Bayer filter pattern Source: Wikipedia Twice as many green elements as red or blue to mimic the physiology of the human eye. Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 7 / 24 7/24
Photograph Through a Bayer Filter Original image Image credit: Anita Martinz / Wikipedia 8/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 8 / 24
Photograph Through a Bayer Filter Recorded by sensor Image credit: Anita Martinz / Wikipedia Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 8 / 24 8/24
Photograph Through a Bayer Filter Recorded by sensor (color coded for clarity) Image credit: Anita Martinz / Wikipedia Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 8 / 24 8/24
Photograph Through a Bayer Filter Recorded by sensor (this is what gets stored in a Raw file!) Image credit: Anita Martinz / Wikipedia 8/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 8 / 24
Demosaicing = De-Bayering Many algorithms See, e.g. Gunturk et al., IEEE Signal Processing Magazine 44, Jan. 2005 Problem areas: edges, repeated patterns Camera manufacturers use their own, proprietary algorithms Commercial software: Adobe Lightroom, Photoshop,... Free/open-source software: dcraw, RawTherapee, DarkTable,... Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 9 / 24 9/24
Gamma Correction From Raw to JPEG A long time ago... there were CRT monitors: brightness (voltage) 2.2... which needed a gamma-corrected signal 1 (max) 1 (max) monitor brightness linear JPEG RGB 0 0 1 (max) electric input signal 0 0 1 (max) linear brightness (Raw) Also convenient for compressing 0... 16384 into 0... 255 range 10/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 10 / 24
Raw to JPEG Conversion To-do list De-mosaicing Gamma correction... and additionally: White balance Exposure correction Tone mapping Noise suppression Sharpening... 11/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 11 / 24
Twilight Time-Lapse Videos Challenge: extreme changes in ambient lighting Equipment: Canon 6D with Rokinon 2.8/14 mm lens and Magic Lantern, motorized slider 900 images, 6 s intervall, spanning 90 minutes Exposure values: Start End Exposure time 1/60 s 5 s ISO 100 12,800 f-stop 2.8 2.8 ( ) ( ) 5 s 12 800 Decrease in lighting by factor = 38 400 1/60 s 100 13/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 13 / 24
Auto- Expose-to-the-Right (ETTR) Magic Lantern Feature: Push histogram to the right Idea: Maximize the amount of light on the sensor without causing saturation Maximize exposure time (up to a pre-set limit) Use the smallest ISO possible (to reduce noise) First, increase exposure time from 1/60 s to 5 s as twilight progresses Then, increase ISO from 100 to 12 800 Save images in Raw format 14/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 14 / 24
Basic Image Processing with RawTherapee Download: http://rawtherapee.com, available for Linux, Windows, Mac First Steps Exposure Correction (much easier in Raw!) White Balance (4800 K works well for most night shots) Noise Control Vignetting Correction (flat field optional) Lens Geometry Correction Sidecar Files RawTherapee works non-destructively Processing operations are written to a plain-text.pp3 file Next time the Raw image is loaded into RawTherapee, all settings specified in the.pp3 file are applied 16/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 16 / 24
RawTherapee 17/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 17 / 24
Basic Image Processing with RawTherapee Download: http://rawtherapee.com, available for Linux, Windows, Mac First Steps Exposure Correction (much easier in Raw!) White Balance (4800 K works well for most night shots) Noise Control Vignetting Correction (flat field optional) Lens Geometry Correction Sidecar Files RawTherapee works non-destructively Processing operations are written to a plain-text.pp3 file Next time the Raw image is loaded into RawTherapee, all settings specified in the.pp3 file are applied 18/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 18 / 24
Exposure Value Smoothing Exposure Value (ISO-corrected) EV = log 2 ( N 2 t 100 ) N... f-stop number t... exposure time (seconds) ISO ISO... ISO setting Examples: ( 2.8 2 f/2.8, 1/60 s, ISO 100: EV = log 2 1/60 s 100 ) = 8.91 9 ( 100 2.8 2 f/2.8, 5 s, ISO 100: EV = log 2 5 100 ) = 6.32 12 800 Larger EV brighter scene If EV increases by 1: light intensity doubles EV drops by (8.91 ( 6.32)) = 15.23 Test: 2 15.23 = 38 400 20/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 20 / 24
Exposure Value Smoothing Plot of EV vs. Time For each image: EV calculated from EXIF data (EXIF: exchangeable image file format; standard for including f-stop, ISO, etc. in JPEG or Raw image file) ExifTool demo But wait... there are steps in this plot! smoothing spline curve exposure value 8 6 4 2 0-2 -4-6 -8 0 100 200 300 400 500 600 700 800 900 1000 image number uncorrected EVs smoothed curve For each image, an individual.pp3 file is written. 21/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 21 / 24
From Raw Files to Video Automatic Raw JPEG/PNG conversion in RawTherapee After all.pp3 files have been generated: Launch RawTherapee Select images, send to queue Click Start processing, come back in a few hours... De-flickering Tools for video de-flickering: timelapse-deflicker: https://github.com/cyberang3l/timelapse-deflicker VirtualDub (www.virtualdub.org) with MSU deflicker filter Video Assembly (JPEG/PNG MP4) ffmpeg -f image2 -r 24 -start number 7040 -i IMG %04d.jpg -vcodec libx264 -s 2400x1600 -crf 18 out.mp4 Your favorite video editing program 22/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 22 / 24
Summary Image Recording Wide angle lens (Rokinon 14 mm f/2.8) Camera in Auto ETTR mode (Magic Lantern), Color Temp. 4800 K Exposure time 1/60 s 5 s, ISO 100 12 800 Save images in Raw (.CR2) format Image Processing Read exposure time & ISO from EXIF data Calculate and smooth exposure values Write EV correction to.pp3 files Run RawTherapee batch queue 23/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 23 / 24
Outlook 24/24 Axel Mellinger (Physics, CMU) Time Lapse Astro Image Processing June 10, 2016 24 / 24