Where Tegra meets Titan! Prof Tom Drummond!

Transcription

1 Where Tegra meets Titan! Prof Tom Drummond!

2 Computer vision is easy!! But first a diversion to 10 th Century Persia!!!!!!!! and the first recorded game of chess!

3 The rice and the chessboard!

4 The rice and the chessboard!

5 The rice and the chessboard!

6 The rice and the chessboard!

7 The rice and the chessboard!

8 The rice and the chessboard! First half of the chessboard: 100 tons of rice

9 The rice and the chessboard! First half of the chessboard: 100 tons of rice Second half of the chessboard: 400 billion tons of rice = 1000 years of production And the moral of the story is

10 The transistor and the chessboard!

11 The transistor and the chessboard! 1974: Intel 8080 (6,000 transistors) 1978: Intel 8086 (29,000 transistors) 1982: Intel (134,000 transistors) 1993 Intel Pen:um (3,000,000 transistors) 2004 P4 Intel Presco> (125,000,000 transistors)

12 The transistor and the chessboard! 2004: Nvidia NV40 (222,000,000 transistors) 2006: Nvidia G80 (484,000,000 transistors) 2008: Nvidia GT200 (1,400,000,000 transistors) 2010: Nvidia GF104 (1,900,000,000 transistors) 2012: Nvidia GK104 (3,540,000,000 transistors) 2015: Nvidia GM200 (8,000,000,000 transistors)? How many on the last square? 1974: Intel 8080 (6,000 transistors) 1978: Intel 8086 (29,000 transistors) 1982: Intel (134,000 transistors) 1993 Intel Pen:um (3,000,000 transistors) 2004 P4 Intel Presco> (125,000,000 transistors) This notebook > 2 trillion transistors

13 Can run Mooreʼs law backwards! Q: According to Moore s law, when was there just one transistor? A: 1948

14 Can run Mooreʼs law backwards! Q: According to Moore s law, when was there just one transistor? A: 1948 In Nov 1947, Bardeen, Bra>ain and Shockley a>ached two gold contacts to a crystal of germanium

15 Power! Mooreʼs law gives us increasing compute power! BUT! With great power comes great!

16 Mooreʼs Law is not always our friend!! Even with GPUs, compute on mobile devices is limited Can t put a K40 on a Quadrotor!

17 Mooreʼs Law is not always our friend!! Even with GPUs, compute on mobile devices is limited But a TX1 fits just fine! (Stereolabs TX1 enabled drone)

18 ACRV! The Australian Research Council Centre of Excellence for Robo:c Vision $25.5M over 7 years 13 Chief Inves:gators in 4 Universi:es 16 Research Fellows ~50 PhD students Research into: Seman:cs (deep learning) Robust vision (all weathers) Vision and Ac:on (closing the loop) Algorithms and Architecture (constrained resources)

19 Distributed Robotic Vision! Simplest method is to just partition the problem somewhere, giving some tasks to the mobile and some to the server! mobile server

20 Distributed Robotic Vision! But often this isnʼt the best solution!e.g. latency introduced by the network may be a problem! Many interesting solutions not like this, e.g:! Obtain sensor data Extract summary informa:on Compute accurate solu:on Compute approximate solu:on Compute approximate solu:on Compare Calculate output Update local model Bring correc:on up to date Calculate and send correc:on

21 Distributed Robotic Vision! Want to create solutions to enable robotics in a distributed sensing and compute environment! K40 K40 K40 CPU TX1 TX1 K40 K40 K40 K40 CPU TX1 K40

22 Distributed Localisation Service! CCTV1 Build Image Pyramid Extract landmarks Build Descriptors Index Match CCTV2 Build Image Pyramid Extract landmarks Build Descriptors Index Match Robot Build Image Pyramid Extract landmarks Build Descriptors Compute 1 Compute Robot pose

23 Distributed Localisation Service! ==3031== NVPROF is profiling process 3031, command:./computeorb 1! Frame# 1! Elapsed time : ms! Frame Elapsed time : ms! numcorners: 28304, nmsnumcorners: 5073! ==3031== Profiling application:./computeorb 1! ==3031== Profiling result:! Time(%) Time Calls Avg Min Max Name! 57.18% ms ms ms ms OrbDescriptors( )! 30.57% ms ms ms ms ( )! 4.29% us us us us fastcorner( )!! 4.00% us us us us harris( )! 1.46% us us us us NMS( )! 0.73% us us us us cleansweep( )! Speedup over CPU* implementation is 4-5X!! * Intel Core2 Quad

24 Sub-pixel localisation! Camera 1 Find landmarks Extract image patch Compute sub- pixel correspondence on many subsequent frames Compute 1 Compute matrix Camera 2 Find landmarks Extract image patch Compute sub- pixel correspondence on many subsequent frames Timing Results:!!!(µs/keypoint) Inverse Additive!!!672 Inverse Compositional!367 Ours!!!!!7!

25 Approximate Nearest Neighbor! Big data in high dimensional spaces Given a query point, find the nearest reference point Solu:on: FANNG (Fast Approximate Nearest Neighbor 2016 Can serve 1.2M queries/second at 90% recall in a database of 1M reference points in 128D space on Titan X

26 Approximate Nearest Neighbor! CUDA implementa:on requires a short priority queue BUT int array[30]; // very slow global memory! Solu:on is to treat a warp as a single unit with array spread over the warp in a single register: int array; // there are 32 of these in a warp!...! // find the first entry in array that is > thresh! int pq = ffs( ballot(array > thresh));!...!!

27 Approximate Nearest Neighbor! Want to keep the array sorted when we insert a new value, discarding the largest value thread: array: new_value: (each thread sees this value) ship value: =max(new_value,array) shuffle: Write new value if less than array array: