Compound Object Detection Using Region Co-occurrence Statistics

Size: px

Start display at page:

Download "Compound Object Detection Using Region Co-occurrence Statistics"

Sarah Rosamond Cameron
5 years ago
Views:

1 Compound Object Detection Using Region Co-occurrence Statistics Selim Aksoy 1 Krzysztof Koperski 2 Carsten Tusk 2 Giovanni Marchisio 2 1 Department of Computer Engineering, Bilkent University, Ankara, Turkey 2 DigitalGlobe, Inc., Research and Development, Longmont, CO, USA IIM 2014 IIM 2014 c 2014, Aksoy et al. (Bilkent University) 1 / 31

2 Introduction Conventional object detection techniques expect the objects of interest to appear as homogeneous regions. This expectation may be satisfied for some objects such as buildings, roads and trees that have relatively homogeneous spectral content and consistent shape. However, the detection of more complex objects such as schools, power plants, shopping malls is still very difficult due to their heterogeneous content in VHR images. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 2 / 31

3 Related work Edge-based model for detecting airports Texture model for detecting orchards Texture model for detecting golf courses and harbors Two-level classification for detecting high schools Bag-of-words model for detecting nuclear plants, coal power plants, and airports Latent topic discovery using spatial signatures IIM 2014 c 2014, Aksoy et al. (Bilkent University) 3 / 31

4 Outline 1 Data set 2 Region segmentation 3 Region occurrence histogram 4 Region co-occurrence histogram 5 Learning and classification 6 Experiments 7 Conclusions IIM 2014 c 2014, Aksoy et al. (Bilkent University) 4 / 31

5 Data set I Multispectral WorldView-2 image of King County, Washington, USA I 28, 920 9, 804 pixels and 2 m spatial resolution I Reference data obtained from Open Street Map IIM 2014 c 2014, Aksoy et al. (Bilkent University) 5 / 31

6 Data set Table 1: Summary of the reference data. The number of positive examples (N) and the sizes (height width) of the smallest, average, and largest reference object bounding boxes are shown. Object N Smallest Average Largest School Park , 044 1, 346 Residential , 653 2, 272 Retail Power IIM 2014 c 2014, Aksoy et al. (Bilkent University) 6 / 31

7 Region segmentation Our segmentation is based on pixel classification. A large number of spectral and textural features were used with a random forest classifier to classify each pixel into one of 16 classes: water, swimming pool, shrub, barren, sand, tidal flat, wetland, dry grass, pasture hay, cultivated crop, green grass, roof, road, shadow, tree, clearcut. The initial land cover classification was post-processed and was converted into a region segmentation. We also added a building layer obtained from the City of Seattle GIS database as the 17 th class. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 7 / 31

8 Region segmentation IIM 2014 c 2014, Aksoy et al. (Bilkent University) 8 / 31

9 Region occurrence histogram Histogram of the regions within a window Quantization of region sizes using a Gamma fit Length: 17 q where q is the number of quantization levels IIM 2014 c 2014, Aksoy et al. (Bilkent University) 9 / 31

10 Region occurrence histogram 0.05 x Histogram Histogram Area Area (a) Gamma fit (b) Quantized size Figure 1: Quantization of the region sizes using a Gamma fit. The quantization levels (red lines) are selected according to the cumulative probabilities computed from the estimated Gamma distribution (green curve). IIM 2014 c 2014, Aksoy et al. (Bilkent University) 10 / 31

11 Region co-occurrence histogram Co-occurrence histogram of region pairs within a window Quantization of region sizes using a Gamma fit Count only the pairs that are closer than a distance threshold Length: (17 q)(17 q + 1)/2 where q is the number of quantization levels IIM 2014 c 2014, Aksoy et al. (Bilkent University) 11 / 31

12 Learning and classification We pose the object detection task in a binary setting where a given image window is classified as containing a target object of interest or not. The binary classification is performed using logistic regression that is a discriminative classifier that assumes a parametric form for the posterior probability and directly estimates its parameters from the training data. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 12 / 31

13 Learning and classification Let x R d denote the feature vector and y { 1, 1} denote the corresponding binary class variable. The logistic model has the form p(y = 1 x; w, w 0 ) = exp( w T x w 0 ) where w is the weight vector and w 0 is the intercept. Given a test sample x, the discriminative logistic regression classifier chooses the positive class 1 if p(y = 1 x; w, w 0 ) > 0.5, or equivalently, w T x + w 0 > 0. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 13 / 31

14 Learning and classification Logistic regression Given labeled data {(x i, y i )} n i=1, the maximum likelihood estimates of w and w 0 can be found by solving min w,w 0 n i=1 ( log 1 + exp ( y i (w T x i + w 0 ) )). IIM 2014 c 2014, Aksoy et al. (Bilkent University) 14 / 31

15 Learning and classification When the number of training examples (n) is not large enough compared to the number of features (d), the logistic regression classifier tends to suffer from over-fitting. The l 1 -regularized formulation is useful due to its tendency to prefer solutions with fewer nonzero parameter values. Logistic regression with l 1 -regularization min w,w 0 n log i=1 ( 1 + exp ( y i (w T x i + w 0 ) )) + λ w 1 IIM 2014 c 2014, Aksoy et al. (Bilkent University) 15 / 31

16 Experiments We used 5-fold cross-validation to train the classifier and evaluate its accuracy. Number of size quantization levels: 1, 5, 10. Maximum distance threshold: 5, 20, 50. We randomly sampled 2, 000 windows of pixels to construct the set of negative examples. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 16 / 31

17 Experiments Quantitative performance evaluation was done by using ROC curves based on different thresholds on the posterior. The area under the ROC curve was used as the criterion to compare different parameter settings. Qualitative performance evaluation was performed via visual inspection of the classification maps computed using pixel sliding windows with 10 pixel increments. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 17 / 31

18 Experiments Table 2: Summary of the classification results using 5-fold cross-validation. The number of size quantization levels (SQL) and the maximum distance threshold (MDT) that resulted in the best performance are shown together with the corresponding area under the ROC curve (AUC, as mean ± std). Object SQL MDT AUC School ± Park ± Residential ± Retail ± Power ± IIM 2014 c 2014, Aksoy et al. (Bilkent University) 18 / 31

19 Experiments TPR school 20, 5, ± TPR park 20, 5, ± TPR residential 50, 5, ± FPR (a) School 1 retail 20, 5, ± FPR (b) Park 1 power 5, 10, ± FPR (c) Residential TPR 0.5 TPR FPR FPR (d) Retail (e) Power Figure 2: ROC curves for different object classes with the best parameter settings. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 19 / 31

20 School IIM 2014 c 2014, Aksoy et al. (Bilkent University) 20 / 31

21 School IIM 2014 c 2014, Aksoy et al. (Bilkent University) 20 / 31

22 School IIM 2014 c 2014, Aksoy et al. (Bilkent University) 20 / 31

23 School Figure 3: School detection results in a 1, 760 2, 717 pixel residential neighborhood. There are 12 schools in the reference data (red overlay), and the algorithm detected 16 schools (green overlay) among which 8 are true positives. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 21 / 31

24 School water swimmingpool shrub otherbarren sand tidalflat wetland drygrass pasturehay cultivatedcrop greengrass roof road shadow tree clearcut building 1,1 1,2 1,3 1,4 1,5 2,1 2,2 2,3 2,4 2,5 3,1 3,2 3,3 3,4 3,5 4,1 4,2 4,3 4,4 4,5 5,1 5,2 5,3 5,4 5,5 6,1 6,2 6,3 6,4 6,5 7,1 7,2 7,3 7,4 7,5 8,1 8,2 8,3 8,4 8,5 9,1 9,2 9,3 9,4 9,5 10,1 10,2 10,3 10,4 10,5 11,1 11,2 11,3 11,4 11,5 12,1 12,2 12,3 12,4 12,5 13,1 13,2 13,3 13,4 13,5 14,1 14,2 14,3 14,4 14,5 15,1 15,2 15,3 15,4 15,5 16,1 16,2 16,3 16,4 16,5 17,1 17,2 17,3 17,4 17,5 11,513,5 17,5 17,4 12,515,5 8,2 17,5 8,2 12,5 4,2 14,2 4,2 11,5 7,5 8,3 8,5 14,2 7,3 14,2 8,3 12,5 8,5 14,1 5,2 8,2 8,5 14,3 11,514,3 1,1 11,5 8,1 17,4 1,1 5,2 8,4 17,2 4,5 14,2 4,1 12,1 4,2 17,5 4,4 7,5 8,2 17,4 4,2 12,1 7,5 17,4 8,5 13,4 11,417,5 7,5 12,5 3,2 12,5 5,1 17,4 6,2 8,2 6,2 11,1 4,5 12,5 9,2 14,3 4,1 14,1 7,2 17,5 8,4 13,3 4,5 11,2 4,5 15,3 11,515,5 4,5 11,1 4,2 17,4 4,4 4,5 4,5 12,1 8,5 15,2 4,2 14,4 8,3 14,4 9,3 14,4 4,5 14,1 9,5 14,3 4,2 9,4 9,4 12,4 9,4 12,5 9,4 13,4 IIM 2014 c 2014, Aksoy et al. (Bilkent University) 22 / 31

25 Park IIM 2014 c 2014, Aksoy et al. (Bilkent University) 23 / 31

26 Park IIM 2014 c 2014, Aksoy et al. (Bilkent University) 23 / 31

27 Park IIM 2014 c 2014, Aksoy et al. (Bilkent University) 23 / 31

28 Park water 1,1 1,2 1,3 1,4 1,5 swimmingpool 2,1 2,2 2,3 2,4 2,5 shrub 3,1 3,2 3,3 3,4 3,5 otherbarren 4,1 4,2 4,3 4,4 4,5 sand 5,1 5,2 5,3 5,4 5,5 tidalflat 6,1 6,2 6,3 6,4 6,5 wetland 7,1 7,2 7,3 7,4 7,5 drygrass 8,1 8,2 8,3 8,4 8,5 pasturehay 9,1 9,2 9,3 9,4 9,5 cultivatedcrop 10,1 10,2 10,3 10,4 10,5 greengrass 11,1 11,2 11,3 11,4 11,5 roof 12,1 12,2 12,3 12,4 12,5 road 13,1 13,2 13,3 13,4 13,5 shadow 14,1 14,2 14,3 14,4 14,5 tree 15,1 15,2 15,3 15,4 15,5 clearcut 16,1 16,2 16,3 16,4 16,5 building 17,1 17,2 17,3 17,4 17,5 1,515,5 8,511,1 15,515,5 11,3 1,21,2 11,4 1,58,2 1,515,1 4,211,5 11,17,4 5,213,3 7,512,2 4,214,2 1,53,5 8,517,4 8,211,5 11,512,1 4,211,1 4,311,5 11,515,5 7,212,3 1,48,3 4,511,4 8,314,4 8,311,4 5,211,2 8,19,3 1,211,1 3,5 8,317,4 5,111,3 4,48,3 7,213,2 11,512,2 7,417,4 4,29,3 9,311,1 1,211,3 1,512,5 8,49,4 8,211,3 1,18,1 1,31,5 8,411,2 8,19,4 8,413,1 3,216,4 11,16,4 15,416,4 11,513,3 3,213,2 5,215,2 9,217,1 4,412,1 7,58,2 7,311,5 5,213,1 1,110,2 10,211,2 1,24,4 3,24,4 3,211,1 4,414,2 4,27,2 7,312,1 4,215,3 4,25,1 7,38,2 8,28,5 5,18,5 1,27,3 1,211,4 11,513,1 1,511,5 4,415,2 1,213,3 8,417,5 8,217,4 11,314,5 14,515,3 1,18,5 7,18,2 1,17,5 11,415,2 11,415,4 11,214,3 2,2 2,24,1 2,28,2 2,28,3 2,211,1 2,212,2 2,212,3 2,213,2 2,213,5 2,214,1 2,214,4 2,215,2 2,215,5 2,217,2 1,411,5 1,1 IIM 2014 c 2014, Aksoy et al. (Bilkent University) 24 / 31

29 Residential IIM 2014 c 2014, Aksoy et al. (Bilkent University) 25 / 31

30 Residential IIM 2014 c 2014, Aksoy et al. (Bilkent University) 25 / 31

31 Residential IIM 2014 c 2014, Aksoy et al. (Bilkent University) 25 / 31

32 Residential water 1,1 1,2 1,3 1,4 1,5 swimmingpool 2,1 2,2 2,3 2,4 2,5 shrub 3,1 3,2 3,3 3,4 3,5 otherbarren 4,1 4,2 4,3 4,4 4,5 sand 5,1 5,2 5,3 5,4 5,5 tidalflat 6,1 6,2 6,3 6,4 6,5 wetland 7,1 7,2 7,3 7,4 7,5 drygrass 8,1 8,2 8,3 8,4 8,5 pasturehay 9,1 9,2 9,3 9,4 9,5 cultivatedcrop 10,1 10,2 10,3 10,4 10,5 greengrass 11,1 11,2 11,3 11,4 11,5 roof 12,1 12,2 12,3 12,4 12,5 road 13,1 13,2 13,3 13,4 13,5 shadow 14,1 14,2 14,3 14,4 14,5 tree 15,1 15,2 15,3 15,4 15,5 clearcut 16,1 16,2 16,3 16,4 16,5 building 17,1 17,2 17,3 17,4 17,5 13,513,5 15,515,5 13,17,1 13,313,5 13,417,3 15,517,1 13,514,2 13,514,1 13,514,3 11,314,1 15,16,4 17,1 8,314,1 15,15,5 13,313,4 8,411,3 13,417,5 13,413,4 13,515,5 8,111,3 7,417,2 13,213,4 12,415,1 1,115,3 15,415,5 14,315,2 3,215,3 8,317,3 12,414,4 4,213,3 12,313,2 14,15,5 14,315,1 8,312,1 11,12,5 13,17,3 11,412,5 13,213,3 12,513,3 1,58,1 13,317,4 13,13,2 11,417,2 4,313,4 12,212,5 8,317,1 12,212,3 1,114,3 14,315,3 1,213,4 12,315,5 8,414,3 14,315,4 1,11,1 1,115,4 5,117,1 12,314,2 11,217,5 11,311,5 7,214,1 13,217,3 1,113,4 7,212,3 1,412,2 12,416,1 12,516,1 15,216,1 11,517,3 1,215,4 1,215,3 12,515,5 8,314,5 8,514,5 7,28,1 14,14,1 14,17,3 13,15,1 1,413,3 12,413,1 13,414,5 1,115,2 12,217,5 12,515,2 8,315,1 13,413,5 14,17,1 13,313,3 14,17,2 3,214,1 3,213,4 13,16,4 3,214,2 14,217,2 7,414,2 17,17,1 11,311,3 1,515,1 13,213,2 17,17,5 1,514,4 11,314,2 IIM 2014 c 2014, Aksoy et al. (Bilkent University) 26 / 31

33 Retail IIM 2014 c 2014, Aksoy et al. (Bilkent University) 27 / 31

34 Retail IIM 2014 c 2014, Aksoy et al. (Bilkent University) 27 / 31

35 Retail IIM 2014 c 2014, Aksoy et al. (Bilkent University) 27 / 31

36 Retail water swimmingpool shrub otherbarren sand tidalflat wetland drygrass pasturehay cultivatedcrop greengrass roof road shadow tree clearcut building 1,1 1,2 1,3 1,4 1,5 2,1 2,2 2,3 2,4 2,5 3,1 3,2 3,3 3,4 3,5 4,1 4,2 4,3 4,4 4,5 5,1 5,2 5,3 5,4 5,5 6,1 6,2 6,3 6,4 6,5 7,1 7,2 7,3 7,4 7,5 8,1 8,2 8,3 8,4 8,5 9,1 9,2 9,3 9,4 9,5 10,1 10,2 10,3 10,4 10,5 11,1 11,2 11,3 11,4 11,5 12,1 12,2 12,3 12,4 12,5 13,1 13,2 13,3 13,4 13,5 14,1 14,2 14,3 14,4 14,5 15,1 15,2 15,3 15,4 15,5 16,1 16,2 16,3 16,4 16,5 17,1 17,2 17,3 17,4 17,5 12,1 12,5 12,2 12,5 12,1 17,5 12,3 15,5 15,5 17,5 1,1 12,4 17,2 17,5 12,3 14,1 14,3 15,1 11,1 12,3 12,1 12,4 12,3 17,5 12,3 14,2 8,3 12,4 12,4 15,1 12,4 12,4 13,1 17,3 12,3 13,3 1,1 12,5 4,1 12,3 12,3 15,1 12,1 17,3 12,1 14,3 14,2 17,5 IIM 2014 c 2014, Aksoy et al. (Bilkent University) 28 / 31

37 Power IIM 2014 c 2014, Aksoy et al. (Bilkent University) 29 / 31

38 Power IIM 2014 c 2014, Aksoy et al. (Bilkent University) 29 / 31

39 Power IIM 2014 c 2014, Aksoy et al. (Bilkent University) 29 / 31

40 Power water swimmingpool shrub otherbarren sand tidalflat wetland drygrass pasturehay cultivatedcrop greengrass roof road shadow tree clearcut building 1,1 1,2 1,3 1,4 1,5 1,6 1,7 1,8 1,9 1,10 2,1 2,2 2,3 2,4 2,5 2,6 2,7 2,8 2,9 2,10 3,1 3,2 3,3 3,4 3,5 3,6 3,7 3,8 3,9 3,10 4,1 4,2 4,3 4,4 4,5 4,6 4,7 4,8 4,9 4,10 5,1 5,2 5,3 5,4 5,5 5,6 5,7 5,8 5,9 5,10 6,1 6,2 6,3 6,4 6,5 6,6 6,7 6,8 6,9 6,10 7,1 7,2 7,3 7,4 7,5 7,6 7,7 7,8 7,9 7,10 8,1 8,2 8,3 8,4 8,5 8,6 8,7 8,8 8,9 8,10 9,1 9,2 9,3 9,4 9,5 9,6 9,7 9,8 9,9 9,10 10,1 10,2 10,3 10,4 10,5 10,6 10,7 10,8 10,910,10 11,1 11,2 11,3 11,4 11,5 11,6 11,7 11,8 11,911,10 12,1 12,2 12,3 12,4 12,5 12,6 12,7 12,8 12,912,10 13,1 13,2 13,3 13,4 13,5 13,6 13,7 13,8 13,913,10 14,1 14,2 14,3 14,4 14,5 14,6 14,7 14,8 14,914,10 15,1 15,2 15,3 15,4 15,5 15,6 15,7 15,8 15,915,10 16,1 16,2 16,3 16,4 16,5 16,6 16,7 16,8 16,916,10 17,1 17,2 17,3 17,4 17,5 17,6 17,7 17,8 17,917,10 8,2 11,6 4,10 14,7 4,5 13,10 12,1 12,6 6,5 4,6 6,5 4,1 4,7 4,1 13,8 4,1 17,10 IIM 2014 c 2014, Aksoy et al. (Bilkent University) 30 / 31

41 Conclusions We described an algorithm for the detection of heterogeneous objects in very high spatial resolution images. We used occurrence and co-occurrence histograms of regions in image windows as features for a logistic regression classifier. Experiments were performed using shape data from the Open Street Map. Future work includes more detailed evaluation of different parameters and object classes. IIM 2014 c 2014, Aksoy et al. (Bilkent University) 31 / 31

Detection of Compound Structures in Very High Spatial Resolution Images

Detection of Compound Structures in Very High Spatial Resolution Images Selim Aksoy Department of Computer Engineering Bilkent University Bilkent, 06800, Ankara, Turkey saksoy@cs.bilkent.edu.tr Joint work