Thermal Image Enhancement Using Convolutional Neural Network

Size: px
Start display at page:

Download "Thermal Image Enhancement Using Convolutional Neural Network"

Transcription

1 SEOUL Oct.7, 2016 Thermal Image Enhancement Using Convolutional Neural Network Visual Perception for Autonomous Driving During Day and Night Yukyung Choi Soonmin Hwang Namil Kim Jongchan Park In So Kweon

2 AGENDA Introduction to our Vision Introduction to Thermal Image Enhancement (TEN) The Period to Explore GPUs for Image Processing 2

3 Toward the Next Generation of Vision Technology Why do we use LWIR sensors? 3

4 4

5 KAIST Multi-spectral Recognition Dataset in day and night, IJRR under review. 5

6 KYLE BEAN - HISTORY OF MOBILE EVOLUTION, Motorola DynaTAC iphone 여주시립폰박물관 6

7 expensive heavy industrial cheap light commerciable Le Penseur 7

8 Our goal: Given a single modality, let s generate multispectral information. Time Invariance Spectral Image (LWIR) Transfer Visible-Spectral Context (Chromaticity, Depth, etc) (To be appear in IROS2016-Exhibition ) 8

9 What is the next generation vision sensor for use in any time of the day? Why do we enhance LWIR images? Visual information gives you a wider view than radars. - Angelova, IEEE Spectrum - 9

10 Thermal Infrared Camera Installation Dashboard BMW X5 (2014) Audi A8 (2014) Night Driving Trailer Mercedes Benz-S (2014) 10

11 Long wave infrared (LWIR) Visible (RGB) PM04 AM02 PM04 AM02 KAIST All-day dataset (FLIR A655sc, Point Grey Flea3) 8um~14um wavelength 400nm~700nm O light invariant X not enough texture/color enough But large diffraction distortion small small resolution large 11

12 Limitation #1: Resolution Image Enlargement (SR) Limitation #2: Diffraction Distortion (blur) Detail Enhancement (DDE) 320x240 (*LR) 640x480 ( HR) 640x480 (HR) 640x480 (Good Quality HR) *LR: low resolution, HR: high resolution Make HR image with LR image Improve HR image for better visibility and recognition performance 12

13 Image Enlargement (SR) Which type of image is useful in thermal image enhancing? Visible images? or Thermal images? or some other imaging spectra? Thermal image enhancing: when and where is it useful? Visibility? or Recognition performance? 13

14 (1) Which type of image is useful in thermal image enhancing? BLUR λ Visible um Short Wave IR um Middle Wave IR um Long Wave IR um Thermal Infrared Face Recognition A Biometric Identification Technique for Robust Security system, Refinements and New Ideas in Face Recognition. 14

15 (2) Thermal image enhancing: when and where is it useful? Input Bicubic Proposed GT Same or not? From Global-Local Face Upsampling Network, Arxiv (27 Apr 2016). 15

16 Image Enlargement : Architecture *LR Limitation #1: Resolution HR MSE (pixel loss) Feature Extraction Mapping Reconstruction (c in, c out,f,p) c in/out is the number of input/out channel, f is the size of filter, p is the size of padding. Bicubic : Interpolation Layer (preprocessing) Conv : Convolution Layer ReLU: Rectified Linear Unit Layer *LR: low resolution HR: high resolution 1) Shallow Network 2) MSE (pixel loss) 3) Bicubic Interpolation Thermal Image Enhancement Using Convolutional Neural Network, (To be appear in IROS2016) 16

17 Image Enlargement : Data Train Data [RGB] RGB 91 Dataset (gray channel) Limitation #1: Resolution (1) Which type of image is useful in thermal image enhancing? Test Data [LWIR] Multimodal Stereo Dataset [MWIR] Thermal Stereo Dataset [LWIR] Multimodal Stereo Dataset Training parameter Pre-training: 64 64, 91 patches Fine-tuning: 36 36, stride 6, 118,211 patches No data augmentation The size of batch : 128, Learning rate: (decreased by a factor 10 at every 30 epochs until 60 epochs) 17

18 Image Enlargement : (1) Which type of image is useful in thermal image enhancing? Comparison of performance in Visible and MWIR Bicubic Gray-TENet MWIR-TENet PSNR(dB) Limitation #1: Resolution TENet x2 RGB MWIR 18

19 Limitation #1: Resolution Image Enlargement : (2) when and where is it useful? Far Pedestrian detection result (a) (b) Visual odometry result GT: GPS/IMU data (c) [m] RGB LWIR E-LWIR AM11:00 [m] RGB LWIR E-LWIR AM01:00 (d) (e) (f) [m] [m] Detections are conducted on KAIST-RCV algorithm. [1] [1] Multispectral Pedestrian Detection: Benchmark Dataset and Baseline, CVPR2015 Trajectories are estimated by Andreas s algorithm. [2] [2] StereoScan: Dense 3D Reconstruction in Real-time, IV

20 Detail Enhancement : Architecture Limitation #2: Diffraction Distortion *HR R HR Feature Extraction Mapping Reconstruction (c in, c out,f,p) c in/out is the number of input/out channel, f is the size of filter, p is the size of padding. Conv : Convolution Layer ReLU: Rectified Linear Unit Layer BatchNorm: Batch Normalization Layer *HR: input image HR: enhanced image R: Residual image Patent Pending* (To be appear in KINPEX2016) 20

21 Detail Enhancement : Data Train Data [RGB] RGBT67 in KAIST all-day dataset (y channel) Limitation #2: Diffraction Distortion (1) Which type of image is useful in thermal image enhancing? Test Data [LWIR] RGBT67 in KAIST all-day dataset [LWIR] RGBT67 in KAIST all-day dataset Training parameter 64 64, stride 32, 136,800 patches Data Augmentation ( up-down flip, left-right flip ) The size of batch : 64, Learning rate:

22 Detail Enhancement : Data Limitation #2: Diffraction Distortion (1) Which type of image is useful in thermal image enhancing? Comparison of performance in Visible and LWIR *NIQE: Image distortion score [2] NIQE PSNR Input RGB Model LWIR Model [2] No Reference Image and Video Quality Assessment, SPL

23 RESULT VIDEO 23

24 It s time to explore GPUs for image processing 4th Industrial Revolution is just around the corner with GPUs 24

25 From NVIDIA 25

26 It s time to explore GPUs for image processing Image Enlargement Device Computatio nal Time Speed Up Frames per Sec CPU: i (3.30GHz) ms x GTX ms x Jetson TX ms x *test image : 320x240 (with caffe framework) I GTX 1080 Jetson TX1 26

27 Today s Summary Introduction of our vision Toward the Next Generation of Vision Technology Talk about Thermal Image Enhancement Network (TEN) Now let s explore GPUs for image processing! 27

28 SEOUL Oct.7, 2016 THANK YOU Thank you to all my coworkers Yukyung Choi Soonmin Hwang Namil Kim Jongchan Park In So Kweon

Convolutional Neural Network-Based Infrared Image Super Resolution Under Low Light Environment

Convolutional Neural Network-Based Infrared Image Super Resolution Under Low Light Environment Convolutional Neural Network-Based Infrared Super Resolution Under Low Light Environment Tae Young Han, Yong Jun Kim, Byung Cheol Song Department of Electronic Engineering Inha University Incheon, Republic

More information

DYNAMIC CONVOLUTIONAL NEURAL NETWORK FOR IMAGE SUPER- RESOLUTION

DYNAMIC CONVOLUTIONAL NEURAL NETWORK FOR IMAGE SUPER- RESOLUTION Journal of Advanced College of Engineering and Management, Vol. 3, 2017 DYNAMIC CONVOLUTIONAL NEURAL NETWORK FOR IMAGE SUPER- RESOLUTION Anil Bhujel 1, Dibakar Raj Pant 2 1 Ministry of Information and

More information

Multispectral Pedestrian Detection using Deep Fusion Convolutional Neural Networks

Multispectral Pedestrian Detection using Deep Fusion Convolutional Neural Networks Multispectral Pedestrian Detection using Deep Fusion Convolutional Neural Networks Jo rg Wagner1,2, Volker Fischer1, Michael Herman1 and Sven Behnke2 1- Robert Bosch GmbH - 70442 Stuttgart - Germany 2-

More information

360 Panorama Super-resolution using Deep Convolutional Networks

360 Panorama Super-resolution using Deep Convolutional Networks 360 Panorama Super-resolution using Deep Convolutional Networks Vida Fakour-Sevom 1,2, Esin Guldogan 1 and Joni-Kristian Kämäräinen 2 1 Nokia Technologies, Finland 2 Laboratory of Signal Processing, Tampere

More information

Research on Hand Gesture Recognition Using Convolutional Neural Network

Research on Hand Gesture Recognition Using Convolutional Neural Network Research on Hand Gesture Recognition Using Convolutional Neural Network Tian Zhaoyang a, Cheng Lee Lung b a Department of Electronic Engineering, City University of Hong Kong, Hong Kong, China E-mail address:

More information

LIGHT FIELD (LF) imaging [2] has recently come into

LIGHT FIELD (LF) imaging [2] has recently come into SUBMITTED TO IEEE SIGNAL PROCESSING LETTERS 1 Light Field Image Super-Resolution using Convolutional Neural Network Youngjin Yoon, Student Member, IEEE, Hae-Gon Jeon, Student Member, IEEE, Donggeun Yoo,

More information

23270: AUGMENTED REALITY FOR NAVIGATION AND INFORMATIONAL ADAS. Sergii Bykov Technical Lead Machine Learning 12 Oct 2017

23270: AUGMENTED REALITY FOR NAVIGATION AND INFORMATIONAL ADAS. Sergii Bykov Technical Lead Machine Learning 12 Oct 2017 23270: AUGMENTED REALITY FOR NAVIGATION AND INFORMATIONAL ADAS Sergii Bykov Technical Lead Machine Learning 12 Oct 2017 Product Vision Company Introduction Apostera GmbH with headquarter in Munich, was

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Deep Learning Barnabás Póczos Credits Many of the pictures, results, and other materials are taken from: Ruslan Salakhutdinov Joshua Bengio Geoffrey Hinton Yann LeCun 2

More information

Deep Learning. Dr. Johan Hagelbäck.

Deep Learning. Dr. Johan Hagelbäck. Deep Learning Dr. Johan Hagelbäck johan.hagelback@lnu.se http://aiguy.org Image Classification Image classification can be a difficult task Some of the challenges we have to face are: Viewpoint variation:

More information

Colorful Image Colorizations Supplementary Material

Colorful Image Colorizations Supplementary Material Colorful Image Colorizations Supplementary Material Richard Zhang, Phillip Isola, Alexei A. Efros {rich.zhang, isola, efros}@eecs.berkeley.edu University of California, Berkeley 1 Overview This document

More information

Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising

Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising Peng Liu University of Florida pliu1@ufl.edu Ruogu Fang University of Florida ruogu.fang@bme.ufl.edu arxiv:177.9135v1 [cs.cv]

More information

Deep Multispectral Semantic Scene Understanding of Forested Environments using Multimodal Fusion

Deep Multispectral Semantic Scene Understanding of Forested Environments using Multimodal Fusion Deep Multispectral Semantic Scene Understanding of Forested Environments using Multimodal Fusion Abhinav Valada, Gabriel L. Oliveira, Thomas Brox, and Wolfram Burgard Department of Computer Science, University

More information

Optimized Quality and Structure Using Adaptive Total Variation and MM Algorithm for Single Image Super-Resolution

Optimized Quality and Structure Using Adaptive Total Variation and MM Algorithm for Single Image Super-Resolution Optimized Quality and Structure Using Adaptive Total Variation and MM Algorithm for Single Image Super-Resolution 1 Shanta Patel, 2 Sanket Choudhary 1 Mtech. Scholar, 2 Assistant Professor, 1 Department

More information

Multi-Modal Spectral Image Super-Resolution

Multi-Modal Spectral Image Super-Resolution Multi-Modal Spectral Image Super-Resolution Fayez Lahoud, Ruofan Zhou, and Sabine Süsstrunk School of Computer and Communication Sciences École Polytechnique Fédérale de Lausanne {ruofan.zhou,fayez.lahoud,sabine.susstrunk}@epfl.ch

More information

Combination of Single Image Super Resolution and Digital Inpainting Algorithms Based on GANs for Robust Image Completion

Combination of Single Image Super Resolution and Digital Inpainting Algorithms Based on GANs for Robust Image Completion SERBIAN JOURNAL OF ELECTRICAL ENGINEERING Vol. 14, No. 3, October 2017, 379-386 UDC: 004.932.4+004.934.72 DOI: https://doi.org/10.2298/sjee1703379h Combination of Single Image Super Resolution and Digital

More information

Classification Accuracies of Malaria Infected Cells Using Deep Convolutional Neural Networks Based on Decompressed Images

Classification Accuracies of Malaria Infected Cells Using Deep Convolutional Neural Networks Based on Decompressed Images Classification Accuracies of Malaria Infected Cells Using Deep Convolutional Neural Networks Based on Decompressed Images Yuhang Dong, Zhuocheng Jiang, Hongda Shen, W. David Pan Dept. of Electrical & Computer

More information

A Neural Algorithm of Artistic Style (2015)

A Neural Algorithm of Artistic Style (2015) A Neural Algorithm of Artistic Style (2015) Leon A. Gatys, Alexander S. Ecker, Matthias Bethge Nancy Iskander (niskander@dgp.toronto.edu) Overview of Method Content: Global structure. Style: Colours; local

More information

Event-based Algorithms for Robust and High-speed Robotics

Event-based Algorithms for Robust and High-speed Robotics Event-based Algorithms for Robust and High-speed Robotics Davide Scaramuzza All my research on event-based vision is summarized on this page: http://rpg.ifi.uzh.ch/research_dvs.html Davide Scaramuzza University

More information

Visione per il veicolo Paolo Medici 2017/ Visual Perception

Visione per il veicolo Paolo Medici 2017/ Visual Perception Visione per il veicolo Paolo Medici 2017/2018 02 Visual Perception Today Sensor Suite for Autonomous Vehicle ADAS Hardware for ADAS Sensor Suite Which sensor do you know? Which sensor suite for Which algorithms

More information

New applications of Spectral Edge image fusion

New applications of Spectral Edge image fusion New applications of Spectral Edge image fusion Alex E. Hayes a,b, Roberto Montagna b, and Graham D. Finlayson a,b a Spectral Edge Ltd, Cambridge, UK. b University of East Anglia, Norwich, UK. ABSTRACT

More information

Color Constancy Using Standard Deviation of Color Channels

Color Constancy Using Standard Deviation of Color Channels 2010 International Conference on Pattern Recognition Color Constancy Using Standard Deviation of Color Channels Anustup Choudhury and Gérard Medioni Department of Computer Science University of Southern

More information

GESTURE RECOGNITION WITH 3D CNNS

GESTURE RECOGNITION WITH 3D CNNS April 4-7, 2016 Silicon Valley GESTURE RECOGNITION WITH 3D CNNS Pavlo Molchanov Xiaodong Yang Shalini Gupta Kihwan Kim Stephen Tyree Jan Kautz 4/6/2016 Motivation AGENDA Problem statement Selecting the

More information

IMAGE RESTORATION WITH NEURAL NETWORKS. Orazio Gallo Work with Hang Zhao, Iuri Frosio, Jan Kautz

IMAGE RESTORATION WITH NEURAL NETWORKS. Orazio Gallo Work with Hang Zhao, Iuri Frosio, Jan Kautz IMAGE RESTORATION WITH NEURAL NETWORKS Orazio Gallo Work with Hang Zhao, Iuri Frosio, Jan Kautz MOTIVATION The long path of images Bad Pixel Correction Black Level AF/AE Demosaic Denoise Lens Correction

More information

Semantic Segmentation on Resource Constrained Devices

Semantic Segmentation on Resource Constrained Devices Semantic Segmentation on Resource Constrained Devices Sachin Mehta University of Washington, Seattle In collaboration with Mohammad Rastegari, Anat Caspi, Linda Shapiro, and Hannaneh Hajishirzi Project

More information

Vehicle Color Recognition using Convolutional Neural Network

Vehicle Color Recognition using Convolutional Neural Network Vehicle Color Recognition using Convolutional Neural Network Reza Fuad Rachmadi and I Ketut Eddy Purnama Multimedia and Network Engineering Department, Institut Teknologi Sepuluh Nopember, Keputih Sukolilo,

More information

High Resolution Spectral Video Capture & Computational Photography Xun Cao ( 曹汛 )

High Resolution Spectral Video Capture & Computational Photography Xun Cao ( 曹汛 ) High Resolution Spectral Video Capture & Computational Photography Xun Cao ( 曹汛 ) School of Electronic Science & Engineering Nanjing University caoxun@nju.edu.cn Dec 30th, 2015 Computational Photography

More information

Simultaneous Capturing of RGB and Additional Band Images Using Hybrid Color Filter Array

Simultaneous Capturing of RGB and Additional Band Images Using Hybrid Color Filter Array Simultaneous Capturing of RGB and Additional Band Images Using Hybrid Color Filter Array Daisuke Kiku, Yusuke Monno, Masayuki Tanaka, and Masatoshi Okutomi Tokyo Institute of Technology ABSTRACT Extra

More information

The Art of Neural Nets

The Art of Neural Nets The Art of Neural Nets Marco Tavora marcotav65@gmail.com Preamble The challenge of recognizing artists given their paintings has been, for a long time, far beyond the capability of algorithms. Recent advances

More information

Fast Perceptual Image Enhancement

Fast Perceptual Image Enhancement Fast Perceptual Image Enhancement Etienne de Stoutz [0000 0001 5439 3290], Andrey Ignatov [0000 0003 4205 8748], Nikolay Kobyshev [0000 0001 6456 4946], Radu Timofte [0000 0002 1478 0402], and Luc Van

More information

arxiv: v1 [cs.ce] 9 Jan 2018

arxiv: v1 [cs.ce] 9 Jan 2018 Predict Forex Trend via Convolutional Neural Networks Yun-Cheng Tsai, 1 Jun-Hao Chen, 2 Jun-Jie Wang 3 arxiv:1801.03018v1 [cs.ce] 9 Jan 2018 1 Center for General Education 2,3 Department of Computer Science

More information

sensors Jin Kyu Kang, Hyung Gil Hong and Kang Ryoung Park *

sensors Jin Kyu Kang, Hyung Gil Hong and Kang Ryoung Park * sensors Article Pedestrian Detection Based on Adaptive Selection of Visible Light or Far-Infrared Light Camera Image by Fuzzy Inference System and Convolutional Neural Network-Based Verification Jin Kyu

More information

LANDMARK recognition is an important feature for

LANDMARK recognition is an important feature for 1 NU-LiteNet: Mobile Landmark Recognition using Convolutional Neural Networks Chakkrit Termritthikun, Surachet Kanprachar, Paisarn Muneesawang arxiv:1810.01074v1 [cs.cv] 2 Oct 2018 Abstract The growth

More information

Biologically Inspired Computation

Biologically Inspired Computation Biologically Inspired Computation Deep Learning & Convolutional Neural Networks Joe Marino biologically inspired computation biological intelligence flexible capable of detecting/ executing/reasoning about

More information

Guided Filtering Using Reflected IR Image for Improving Quality of Depth Image

Guided Filtering Using Reflected IR Image for Improving Quality of Depth Image Guided Filtering Using Reflected IR Image for Improving Quality of Depth Image Takahiro Hasegawa, Ryoji Tomizawa, Yuji Yamauchi, Takayoshi Yamashita and Hironobu Fujiyoshi Chubu University, 1200, Matsumoto-cho,

More information

New Techniques for Preserving Global Structure and Denoising with Low Information Loss in Single-Image Super-Resolution

New Techniques for Preserving Global Structure and Denoising with Low Information Loss in Single-Image Super-Resolution New Techniques for Preserving Global Structure and Denoising with Low Information Loss in Single-Image Super-Resolution Yijie Bei Alex Damian Shijia Hu Sachit Menon Nikhil Ravi Cynthia Rudin Duke University

More information

Imaging Process (review)

Imaging Process (review) Color Used heavily in human vision Color is a pixel property, making some recognition problems easy Visible spectrum for humans is 400nm (blue) to 700 nm (red) Machines can see much more; ex. X-rays, infrared,

More information

Wadehra Kartik, Kathpalia Mukul, Bahl Vasudha, International Journal of Advance Research, Ideas and Innovations in Technology

Wadehra Kartik, Kathpalia Mukul, Bahl Vasudha, International Journal of Advance Research, Ideas and Innovations in Technology ISSN: 2454-132X Impact factor: 4.295 (Volume 4, Issue 1) Available online at www.ijariit.com Hand Detection and Gesture Recognition in Real-Time Using Haar-Classification and Convolutional Neural Networks

More information

Image Manipulation Detection using Convolutional Neural Network

Image Manipulation Detection using Convolutional Neural Network Image Manipulation Detection using Convolutional Neural Network Dong-Hyun Kim 1 and Hae-Yeoun Lee 2,* 1 Graduate Student, 2 PhD, Professor 1,2 Department of Computer Software Engineering, Kumoh National

More information

Understanding Neural Networks : Part II

Understanding Neural Networks : Part II TensorFlow Workshop 2018 Understanding Neural Networks Part II : Convolutional Layers and Collaborative Filters Nick Winovich Department of Mathematics Purdue University July 2018 Outline 1 Convolutional

More information

Full Spectrum. Full Calibration. Full Testing. Collimated Optics, Software and Uniform Source Solutions

Full Spectrum. Full Calibration. Full Testing. Collimated Optics, Software and Uniform Source Solutions Full Spectrum. Full Calibration. Full Testing. Collimated Optics, Software and Uniform Source Solutions Combining the Expertise of Two Industry Leaders to Give You An Immense Range of Complete Electro-Optical

More information

Learning a Dilated Residual Network for SAR Image Despeckling

Learning a Dilated Residual Network for SAR Image Despeckling Learning a Dilated Residual Network for SAR Image Despeckling Qiang Zhang [1], Qiangqiang Yuan [1]*, Jie Li [3], Zhen Yang [2], Xiaoshuang Ma [4], Huanfeng Shen [2], Liangpei Zhang [5] [1] School of Geodesy

More information

Transformation to Artificial Intelligence with MATLAB Roy Lurie, PhD Vice President of Engineering MATLAB Products

Transformation to Artificial Intelligence with MATLAB Roy Lurie, PhD Vice President of Engineering MATLAB Products Transformation to Artificial Intelligence with MATLAB Roy Lurie, PhD Vice President of Engineering MATLAB Products 2018 The MathWorks, Inc. 1 A brief history of the automobile First Commercial Gas Car

More information

FOG REMOVAL ALGORITHM USING ANISOTROPIC DIFFUSION AND HISTOGRAM STRETCHING

FOG REMOVAL ALGORITHM USING ANISOTROPIC DIFFUSION AND HISTOGRAM STRETCHING FOG REMOVAL ALGORITHM USING DIFFUSION AND HISTOGRAM STRETCHING 1 G SAILAJA, 2 M SREEDHAR 1 PG STUDENT, 2 LECTURER 1 DEPARTMENT OF ECE 1 JNTU COLLEGE OF ENGINEERING (Autonomous), ANANTHAPURAMU-5152, ANDRAPRADESH,

More information

Digital images. Digital Image Processing Fundamentals. Digital images. Varieties of digital images. Dr. Edmund Lam. ELEC4245: Digital Image Processing

Digital images. Digital Image Processing Fundamentals. Digital images. Varieties of digital images. Dr. Edmund Lam. ELEC4245: Digital Image Processing Digital images Digital Image Processing Fundamentals Dr Edmund Lam Department of Electrical and Electronic Engineering The University of Hong Kong (a) Natural image (b) Document image ELEC4245: Digital

More information

SIMULATION-BASED MODEL CONTROL USING STATIC HAND GESTURES IN MATLAB

SIMULATION-BASED MODEL CONTROL USING STATIC HAND GESTURES IN MATLAB SIMULATION-BASED MODEL CONTROL USING STATIC HAND GESTURES IN MATLAB S. Kajan, J. Goga Institute of Robotics and Cybernetics, Faculty of Electrical Engineering and Information Technology, Slovak University

More information

Design and Testing of DWT based Image Fusion System using MATLAB Simulink

Design and Testing of DWT based Image Fusion System using MATLAB Simulink Design and Testing of DWT based Image Fusion System using MATLAB Simulink Ms. Sulochana T 1, Mr. Dilip Chandra E 2, Dr. S S Manvi 3, Mr. Imran Rasheed 4 M.Tech Scholar (VLSI Design And Embedded System),

More information

Hyperspectral Image Denoising using Superpixels of Mean Band

Hyperspectral Image Denoising using Superpixels of Mean Band Hyperspectral Image Denoising using Superpixels of Mean Band Letícia Cordeiro Stanford University lrsc@stanford.edu Abstract Denoising is an essential step in the hyperspectral image analysis process.

More information

Tiny ImageNet Challenge Investigating the Scaling of Inception Layers for Reduced Scale Classification Problems

Tiny ImageNet Challenge Investigating the Scaling of Inception Layers for Reduced Scale Classification Problems Tiny ImageNet Challenge Investigating the Scaling of Inception Layers for Reduced Scale Classification Problems Emeric Stéphane Boigné eboigne@stanford.edu Jan Felix Heyse heyse@stanford.edu Abstract Scaling

More information

arxiv: v1 [cs.cv] 17 Dec 2017

arxiv: v1 [cs.cv] 17 Dec 2017 Zero-Shot Super-Resolution using Deep Internal Learning Assaf Shocher Nadav Cohen Michal Irani Dept. of Computer Science and Applied Math, The Weizmann Institute of Science, Israel School of Mathematics,

More information

ADAS COMPUTER VISION AND AUGMENTED REALITY SOLUTION

ADAS COMPUTER VISION AND AUGMENTED REALITY SOLUTION ENGINEERING ENERGY TELECOM TRAVEL AND AVIATION SOFTWARE FINANCIAL SERVICES ADAS COMPUTER VISION AND AUGMENTED REALITY SOLUTION Sergii Bykov, Technical Lead TECHNOLOGY AUTOMOTIVE Product Vision Road To

More information

Université Laval Face Motion and Time-Lapse Video Database (UL-FMTV)

Université Laval Face Motion and Time-Lapse Video Database (UL-FMTV) 14 th Quantitative InfraRed Thermography Conference Université Laval Face Motion and Time-Lapse Video Database (UL-FMTV) by Reza Shoja Ghiass*, Hakim Bendada*, Xavier Maldague* *Computer Vision and Systems

More information

Convolutional Networks for Image Segmentation: U-Net 1, DeconvNet 2, and SegNet 3

Convolutional Networks for Image Segmentation: U-Net 1, DeconvNet 2, and SegNet 3 Convolutional Networks for Image Segmentation: U-Net 1, DeconvNet 2, and SegNet 3 1 Olaf Ronneberger, Philipp Fischer, Thomas Brox (Freiburg, Germany) 2 Hyeonwoo Noh, Seunghoon Hong, Bohyung Han (POSTECH,

More information

An Introduction to Convolutional Neural Networks. Alessandro Giusti Dalle Molle Institute for Artificial Intelligence Lugano, Switzerland

An Introduction to Convolutional Neural Networks. Alessandro Giusti Dalle Molle Institute for Artificial Intelligence Lugano, Switzerland An Introduction to Convolutional Neural Networks Alessandro Giusti Dalle Molle Institute for Artificial Intelligence Lugano, Switzerland Sources & Resources - Andrej Karpathy, CS231n http://cs231n.github.io/convolutional-networks/

More information

Demosaicing and Denoising on Simulated Light Field Images

Demosaicing and Denoising on Simulated Light Field Images Demosaicing and Denoising on Simulated Light Field Images Trisha Lian Stanford University tlian@stanford.edu Kyle Chiang Stanford University kchiang@stanford.edu Abstract Light field cameras use an array

More information

Enhanced DCT Interpolation for better 2D Image Up-sampling

Enhanced DCT Interpolation for better 2D Image Up-sampling Enhanced Interpolation for better 2D Image Up-sampling Aswathy S Raj MTech Student, Department of ECE Marian Engineering College, Kazhakuttam, Thiruvananthapuram, Kerala, India Reshmalakshmi C Assistant

More information

NU-Net: Deep Residual Wide Field of View Convolutional Neural Network for Semantic Segmentation

NU-Net: Deep Residual Wide Field of View Convolutional Neural Network for Semantic Segmentation NU-Net: Deep Residual Wide Field of View Convolutional Neural Network for Semantic Segmentation Mohamed Samy 1 Karim Amer 1 Kareem Eissa Mahmoud Shaker Mohamed ElHelw Center for Informatics Science Nile

More information

Continuous Gesture Recognition Fact Sheet

Continuous Gesture Recognition Fact Sheet Continuous Gesture Recognition Fact Sheet August 17, 2016 1 Team details Team name: ICT NHCI Team leader name: Xiujuan Chai Team leader address, phone number and email Address: No.6 Kexueyuan South Road

More information

arxiv: v1 [cs.cv] 19 Feb 2018

arxiv: v1 [cs.cv] 19 Feb 2018 Deep Residual Network for Joint Demosaicing and Super-Resolution Ruofan Zhou, Radhakrishna Achanta, Sabine Süsstrunk IC, EPFL {ruofan.zhou, radhakrishna.achanta, sabine.susstrunk}@epfl.ch arxiv:1802.06573v1

More information

Gradient-Based Correction of Chromatic Aberration in the Joint Acquisition of Color and Near-Infrared Images

Gradient-Based Correction of Chromatic Aberration in the Joint Acquisition of Color and Near-Infrared Images Gradient-Based Correction of Chromatic Aberration in the Joint Acquisition of Color and Near-Infrared Images Zahra Sadeghipoor a, Yue M. Lu b, and Sabine Süsstrunk a a School of Computer and Communication

More information

Multimedia Forensics

Multimedia Forensics Multimedia Forensics Using Mathematics and Machine Learning to Determine an Image's Source and Authenticity Matthew C. Stamm Multimedia & Information Security Lab (MISL) Department of Electrical and Computer

More information

On the Use of Fully Convolutional Networks on Evaluation of Infrared Breast Image Segmentations

On the Use of Fully Convolutional Networks on Evaluation of Infrared Breast Image Segmentations 17º WIM - Workshop de Informática Médica On the Use of Fully Convolutional Networks on Evaluation of Infrared Breast Image Segmentations Rafael H. C. de Melo, Aura Conci, Cristina Nader Vasconcelos Computer

More information

Infrared Variation Optimized Deep Convolutional Neural Network for Robust Automatic Ground Target Recognition

Infrared Variation Optimized Deep Convolutional Neural Network for Robust Automatic Ground Target Recognition Infrared Variation Optimized Deep Convolutional Neural Network for Robust Automatic Ground Target Recognition Sungho Kim Yeungnam University 280 Daehak-Ro, Gyeongsan, Gyeongbuk, 38541, Korea sunghokim@ynu.ac.kr

More information

MOST digital cameras contain sensor arrays covered. Learning Deep Convolutional Networks for Demosaicing. arxiv: v1 [cs.

MOST digital cameras contain sensor arrays covered. Learning Deep Convolutional Networks for Demosaicing. arxiv: v1 [cs. 1 Learning Deep Convolutional Networks for Demosaicing Nai-Sheng Syu, Yu-Sheng Chen, Yung-Yu Chuang arxiv:1802.03769v1 [cs.cv] 11 Feb 2018 Abstract This paper presents a comprehensive study of applying

More information

Lecture 19: Depth Cameras. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011)

Lecture 19: Depth Cameras. Kayvon Fatahalian CMU : Graphics and Imaging Architectures (Fall 2011) Lecture 19: Depth Cameras Kayvon Fatahalian CMU 15-869: Graphics and Imaging Architectures (Fall 2011) Continuing theme: computational photography Cheap cameras capture light, extensive processing produces

More information

ECC419 IMAGE PROCESSING

ECC419 IMAGE PROCESSING ECC419 IMAGE PROCESSING INTRODUCTION Image Processing Image processing is a subclass of signal processing concerned specifically with pictures. Digital Image Processing, process digital images by means

More information

Author(s) Corr, Philip J.; Silvestre, Guenole C.; Bleakley, Christopher J. The Irish Pattern Recognition & Classification Society

Author(s) Corr, Philip J.; Silvestre, Guenole C.; Bleakley, Christopher J. The Irish Pattern Recognition & Classification Society Provided by the author(s) and University College Dublin Library in accordance with publisher policies. Please cite the published version when available. Title Open Source Dataset and Deep Learning Models

More information

Lecture 11-1 CNN introduction. Sung Kim

Lecture 11-1 CNN introduction. Sung Kim Lecture 11-1 CNN introduction Sung Kim 'The only limit is your imagination' http://itchyi.squarespace.com/thelatest/2012/5/17/the-only-limit-is-your-imagination.html Lecture 7: Convolutional

More information

THE VISIONLAB TEAM engineers - 1 physicist. Feasibility study and prototyping Hardware benchmarking Open and closed source libraries

THE VISIONLAB TEAM engineers - 1 physicist. Feasibility study and prototyping Hardware benchmarking Open and closed source libraries VISIONLAB OPENING THE VISIONLAB TEAM 2018 6 engineers - 1 physicist Feasibility study and prototyping Hardware benchmarking Open and closed source libraries Deep learning frameworks GPU frameworks FPGA

More information

Learning to Predict Indoor Illumination from a Single Image. Chih-Hui Ho

Learning to Predict Indoor Illumination from a Single Image. Chih-Hui Ho Learning to Predict Indoor Illumination from a Single Image Chih-Hui Ho 1 Outline Introduction Method Overview LDR Panorama Light Source Detection Panorama Recentering Warp Learning From LDR Panoramas

More information

arxiv: v3 [cs.cv] 18 Dec 2018

arxiv: v3 [cs.cv] 18 Dec 2018 Video Colorization using CNNs and Keyframes extraction: An application in saving bandwidth Ankur Singh 1 Anurag Chanani 2 Harish Karnick 3 arxiv:1812.03858v3 [cs.cv] 18 Dec 2018 Abstract In this paper,

More information

Imaging with hyperspectral sensors: the right design for your application

Imaging with hyperspectral sensors: the right design for your application Imaging with hyperspectral sensors: the right design for your application Frederik Schönebeck Framos GmbH f.schoenebeck@framos.com June 29, 2017 Abstract In many vision applications the relevant information

More information

arxiv: v1 [cs.cv] 26 Jul 2017

arxiv: v1 [cs.cv] 26 Jul 2017 Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network Seonghyeon Nam Yonsei University shnnam@yonsei.ac.kr Seon Joo Kim Yonsei University seonjookim@yonsei.ac.kr arxiv:177.835v1 [cs.cv]

More information

ME 6406 MACHINE VISION. Georgia Institute of Technology

ME 6406 MACHINE VISION. Georgia Institute of Technology ME 6406 MACHINE VISION Georgia Institute of Technology Class Information Instructor Professor Kok-Meng Lee MARC 474 Office hours: Tues/Thurs 1:00-2:00 pm kokmeng.lee@me.gatech.edu (404)-894-7402 Class

More information

Convolutional neural networks

Convolutional neural networks Convolutional neural networks Themes Curriculum: Ch 9.1, 9.2 and http://cs231n.github.io/convolutionalnetworks/ The simple motivation and idea How it s done Receptive field Pooling Dilated convolutions

More information

APPENDIX 1 TEXTURE IMAGE DATABASES

APPENDIX 1 TEXTURE IMAGE DATABASES 167 APPENDIX 1 TEXTURE IMAGE DATABASES A 1.1 BRODATZ DATABASE The Brodatz's photo album is a well-known benchmark database for evaluating texture recognition algorithms. It contains 111 different texture

More information

Sensors and Sensing Cameras and Camera Calibration

Sensors and Sensing Cameras and Camera Calibration Sensors and Sensing Cameras and Camera Calibration Todor Stoyanov Mobile Robotics and Olfaction Lab Center for Applied Autonomous Sensor Systems Örebro University, Sweden todor.stoyanov@oru.se 20.11.2014

More information

Harmless screening of humans for the detection of concealed objects

Harmless screening of humans for the detection of concealed objects Safety and Security Engineering VI 215 Harmless screening of humans for the detection of concealed objects M. Kowalski, M. Kastek, M. Piszczek, M. Życzkowski & M. Szustakowski Military University of Technology,

More information

Visibility of Uncorrelated Image Noise

Visibility of Uncorrelated Image Noise Visibility of Uncorrelated Image Noise Jiajing Xu a, Reno Bowen b, Jing Wang c, and Joyce Farrell a a Dept. of Electrical Engineering, Stanford University, Stanford, CA. 94305 U.S.A. b Dept. of Psychology,

More information

Zero-Shot Super-Resolution using Deep Internal Learning

Zero-Shot Super-Resolution using Deep Internal Learning Zero-Shot Super-Resolution using Deep Internal Learning Assaf Shocher Nadav Cohen Michal Irani Dept. of Computer Science and Applied Math, The Weizmann Institute of Science, Israel School of Mathematics,

More information

Hand Gesture Recognition by Means of Region- Based Convolutional Neural Networks

Hand Gesture Recognition by Means of Region- Based Convolutional Neural Networks Contemporary Engineering Sciences, Vol. 10, 2017, no. 27, 1329-1342 HIKARI Ltd, www.m-hikari.com https://doi.org/10.12988/ces.2017.710154 Hand Gesture Recognition by Means of Region- Based Convolutional

More information

Can you tell a face from a HEVC bitstream?

Can you tell a face from a HEVC bitstream? Can you tell a face from a HEVC bitstream? Saeed Ranjbar Alvar, Hyomin Choi and Ivan V. Bajić School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada Email: {saeedr,chyomin, ibajic}@sfu.ca

More information

High Fidelity 3D Reconstruction

High Fidelity 3D Reconstruction High Fidelity 3D Reconstruction Adnan Ansar, California Institute of Technology KISS Workshop: Gazing at the Solar System June 17, 2014 Copyright 2014 California Institute of Technology. U.S. Government

More information

Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks

Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks Jiawei Zhang 1,2 Jinshan Pan 3 Jimmy Ren 2 Yibing Song 4 Linchao Bao 4 Rynson W.H. Lau 1 Ming-Hsuan Yang 5 1 Department of Computer

More information

AN INVESTIGATION INTO SALIENCY-BASED MARS ROI DETECTION

AN INVESTIGATION INTO SALIENCY-BASED MARS ROI DETECTION AN INVESTIGATION INTO SALIENCY-BASED MARS ROI DETECTION Lilan Pan and Dave Barnes Department of Computer Science, Aberystwyth University, UK ABSTRACT This paper reviews several bottom-up saliency algorithms.

More information

Microbolometers for Infrared Imaging and the 2012 Student Infrared Imaging Competition

Microbolometers for Infrared Imaging and the 2012 Student Infrared Imaging Competition Microbolometers for Infrared Imaging and the 2012 Student Infrared Imaging Competition George D Skidmore, PhD Principal Scientist DRS Technologies RSTA Group Competition Flyer 2 Passive Night Vision Technologies

More information

Bhanudas Sandbhor *, G. U. Kharat Department of Electronics and Telecommunication Sharadchandra Pawar College of Engineering, Otur, Pune, India

Bhanudas Sandbhor *, G. U. Kharat Department of Electronics and Telecommunication Sharadchandra Pawar College of Engineering, Otur, Pune, India Volume 5, Issue 5, MAY 2015 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com A Review on Underwater

More information

Convolutional Neural Networks: Real Time Emotion Recognition

Convolutional Neural Networks: Real Time Emotion Recognition Convolutional Neural Networks: Real Time Emotion Recognition Bruce Nguyen, William Truong, Harsha Yeddanapudy Motivation: Machine emotion recognition has long been a challenge and popular topic in the

More information

Super resolution with Epitomes

Super resolution with Epitomes Super resolution with Epitomes Aaron Brown University of Wisconsin Madison, WI Abstract Techniques exist for aligning and stitching photos of a scene and for interpolating image data to generate higher

More information

International Journal of Advance Engineering and Research Development CONTRAST ENHANCEMENT OF IMAGES USING IMAGE FUSION BASED ON LAPLACIAN PYRAMID

International Journal of Advance Engineering and Research Development CONTRAST ENHANCEMENT OF IMAGES USING IMAGE FUSION BASED ON LAPLACIAN PYRAMID Scientific Journal of Impact Factor(SJIF): 3.134 e-issn(o): 2348-4470 p-issn(p): 2348-6406 International Journal of Advance Engineering and Research Development Volume 2,Issue 7, July -2015 CONTRAST ENHANCEMENT

More information

A Vision Based Hand Gesture Recognition System using Convolutional Neural Networks

A Vision Based Hand Gesture Recognition System using Convolutional Neural Networks A Vision Based Hand Gesture Recognition System using Convolutional Neural Networks Simran Shah 1, Ami Kotia 2, Kausha Nisar 3, Aneri Udeshi 4, Prof. Pramila. M. Chawan 5 1,2,3,4U.G. Students, Department

More information

Enhanced LWIR NUC Using an Uncooled Microbolometer Camera

Enhanced LWIR NUC Using an Uncooled Microbolometer Camera Enhanced LWIR NUC Using an Uncooled Microbolometer Camera Joe LaVeigne a, Greg Franks a, Kevin Sparkman a, Marcus Prewarski a, Brian Nehring a a Santa Barbara Infrared, Inc., 30 S. Calle Cesar Chavez,

More information

Bettina Selig. Centre for Image Analysis. Swedish University of Agricultural Sciences Uppsala University

Bettina Selig. Centre for Image Analysis. Swedish University of Agricultural Sciences Uppsala University 2011-10-26 Bettina Selig Centre for Image Analysis Swedish University of Agricultural Sciences Uppsala University 2 Electromagnetic Radiation Illumination - Reflection - Detection The Human Eye Digital

More information

DIGITAL IMAGING. Handbook of. Wiley VOL 1: IMAGE CAPTURE AND STORAGE. Editor-in- Chief

DIGITAL IMAGING. Handbook of. Wiley VOL 1: IMAGE CAPTURE AND STORAGE. Editor-in- Chief Handbook of DIGITAL IMAGING VOL 1: IMAGE CAPTURE AND STORAGE Editor-in- Chief Adjunct Professor of Physics at the Portland State University, Oregon, USA Previously with Eastman Kodak; University of Rochester,

More information

Séminaire Voiture Autonome: Technologies, Enjeux et Applications February , Paris (France) Asprom UIMM Cap Tronic

Séminaire Voiture Autonome: Technologies, Enjeux et Applications February , Paris (France) Asprom UIMM Cap Tronic Embedded Perception & Risk Assessment for next Cars Generation Christian LAUGIER, Research Director at Inria Chroma Team & IRT Nanolec Christian.laugier@inria.fr Contributions from Mathias Perrollaz, Christopher

More information

Lecture 23 Deep Learning: Segmentation

Lecture 23 Deep Learning: Segmentation Lecture 23 Deep Learning: Segmentation COS 429: Computer Vision Thanks: most of these slides shamelessly adapted from Stanford CS231n: Convolutional Neural Networks for Visual Recognition Fei-Fei Li, Andrej

More information

Remote Sensing. The following figure is grey scale display of SPOT Panchromatic without stretching.

Remote Sensing. The following figure is grey scale display of SPOT Panchromatic without stretching. Remote Sensing Objectives This unit will briefly explain display of remote sensing image, geometric correction, spatial enhancement, spectral enhancement and classification of remote sensing image. At

More information

Coursework 2. MLP Lecture 7 Convolutional Networks 1

Coursework 2. MLP Lecture 7 Convolutional Networks 1 Coursework 2 MLP Lecture 7 Convolutional Networks 1 Coursework 2 - Overview and Objectives Overview: Use a selection of the techniques covered in the course so far to train accurate multi-layer networks

More information

GPU ACCELERATED DEEP LEARNING WITH CUDNN

GPU ACCELERATED DEEP LEARNING WITH CUDNN GPU ACCELERATED DEEP LEARNING WITH CUDNN Larry Brown Ph.D. March 2015 AGENDA 1 Introducing cudnn and GPUs 2 Deep Learning Context 3 cudnn V2 4 Using cudnn 2 Introducing cudnn and GPUs 3 HOW GPU ACCELERATION

More information

Efficient Construction of SIFT Multi-Scale Image Pyramids for Embedded Robot Vision

Efficient Construction of SIFT Multi-Scale Image Pyramids for Embedded Robot Vision Efficient Construction of SIFT Multi-Scale Image Pyramids for Embedded Robot Vision Peter Andreas Entschev and Hugo Vieira Neto Graduate School of Electrical Engineering and Applied Computer Science Federal

More information

International Journal of Innovative Research in Engineering Science and Technology APRIL 2018 ISSN X

International Journal of Innovative Research in Engineering Science and Technology APRIL 2018 ISSN X HIGH DYNAMIC RANGE OF MULTISPECTRAL ACQUISITION USING SPATIAL IMAGES 1 M.Kavitha, M.Tech., 2 N.Kannan, M.E., and 3 S.Dharanya, M.E., 1 Assistant Professor/ CSE, Dhirajlal Gandhi College of Technology,

More information