Statistics and Computing. Series Editors: J. Chambers D. Hand

Transcription

1 Statistics and Computing Series Editors: J. Chambers D. Hand W. Härdle

2 Statistics and Computing Brusco/Stahl: Branch-and-Bound Applications in Combinatorial Data Analysis. Dalgaard: Introductory Statistics with R. Gentle: Elements of Computational Statistics. Gentle: Numerical Linear Algebra for Applications in Statistics. Gentle: Random Number Generation and Monte Carlo Methods, 2nd Edition. Härdle/Klinke/Turlach: XploRe: An Interactive Statistical Computing Environment. Krause/Olson: The Basics of S-PLUS, 4th Edition. Lange: Numerical Analysis for Statisticians. Lemmon/Schafer: Developing Statistical Software in Fortran 95 Loader: Local Regression and Likelihood. Ó Ruanaidh/Fitzgerald: Numerical Bayesian Methods Applied to Signal Processing. Pannatier: VARIOWIN: Software for Spatial Data Analysis in 2D. Pinheiro/Bates: Mixed-Effects Models in S and S-PLUS. Venables/Ripley: Modern Applied Statistics with S, 4th Edition. Venables/Ripley: S Programming. Wilkinson: The Grammar of Graphics.

3 Michael J. Brusco Stephanie Stahl Branch-and-Bound Applications in Combinatorial Data Analysis

4 Michael J. Brusco Department of Marketing College of Business Florida State University Tallahassee, FL USA Stephanie Stahl 2352 Hampshire Way Tallahassee, FL USA Series Editors: J. Chambers Bell Labs, Lucent Technologies 600 Mountain Avenue Murray Hill, NJ USA D. Hand Department of Mathematics South Kensington Campus Imperial College, London London SW7 2AZ United Kingdom W. Härdle Institut für Statistik und Ökonometrie Humboldt-Universität zu Berlin Spandauer Str. 1 D Berlin Germany Library of Congress Control Number: ISBN-10: ISBN-13: Printed on acid-free paper Springer Science+Business Media, Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc., 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed in the United States of America. (EB) springeronline.com

5 For Cobol Lipshitz and Snobol Gentiment

6 Preface This monograph focuses on the application of the solution strategy known as branch-and-bound to problems of combinatorial data analysis. Combinatorial data analysis problems typically require either the selection of a subset of objects from a larger (master) set, the grouping of a collection of objects into mutually exclusive and exhaustive subsets, or the sequencing of objects. To obtain verifiably optimal solutions for this class of problems, we must evaluate (either explicitly or implicitly) all feasible solutions. Unfortunately, the number of feasible solutions for problems of combinatorial data analysis grows exponentially with problem size. For this reason, the explicit enumeration and evaluation of all solutions is computationally infeasible for all but the smallest problems. The branch-and-bound solution method is one type of partial enumeration solution strategy that enables some combinatorial data analysis problems to be solved optimally without explicitly enumerating all feasible solutions. To understand the operation of a branch-and-bound algorithm, we distinguish complete solutions from partial solutions. A complete solution is one for which a feasible solution to the optimization problem has been produced (e.g., all objects are assigned to a group, or all objects are assigned a sequence position). A partial solution is an incomplete solution (e.g., some objects are not assigned to a group, or some objects are not assigned a sequence position). During the execution of a branch-andbound algorithm, solutions are gradually constructed and are, therefore, only partially completed at most stages. If we can determine that a partial solution cannot possibly lead to an optimal solution, then that partial solution and all possible complete solutions stemming from the partial solution can be eliminated from further consideration. This elimination of partial solutions and, simultaneously, all of the complete and partial solutions that could be generated from them is the cornerstone of the branch-and-bound method. In the worst case, a branch-and-bound algorithm could require the complete enumeration of all feasible solutions. For this reason, we note from the outset that we will reach a point where branch-and-bound is not computationally feasible, and other solution approaches are required.

7 viii Preface We develop and describe a variety of applications of the branch-andbound paradigm to combinatorial data analysis. Part I of the monograph (Chapters 2 through 6) focuses on applications for partitioning a set of objects based on various criteria. Part II (Chapters 7 through 11) describes branch-and-bound approaches for seriation of a collection of objects. Part III (Chapters 12 through 14) addresses the plausibility of branch-and-bound methods for variable selection in multivariate data analysis, particularly focusing on cluster analysis and regression. Our development of this monograph was largely inspired by the recent monograph by Hubert, Arabie, and Meulman (2001) titled, Combinatorial Data Analysis: Optimization by Dynamic Programming. Like the branch-and-bound method, dynamic programming is a partial enumeration strategy that can produce optimal solutions for certain classes of combinatorial data analysis problems that would be insurmountable for exhaustive enumeration approaches. Many of the problems tackled in Parts I and II are also addressed in the Hubert et al. (2001) monograph, which would make an excellent companion reading for our monograph. At the end of many of the chapters, we identify available computer programs for implementing the relevant branch-and-bound procedures. This software is offered (without warranty) free of charge, and both the source code and executable programs are available. This code should enable users to reproduce results reported in the chapters, and also to implement the programs for their own data sets. We are deeply indebted to Phipps Arabie, J. Douglas Carroll, and Lawrence Hubert, whose enthusiasm and encouragement were the principal motivating factors for our pursuing this area of research. We also thank J. Dennis Cradit, who has frequently collaborated with us on related problems of combinatorial data analysis. Michael J. Brusco Stephanie Stahl January, 2005

8 Contents Preface vii 1 Introduction Background Branch-and-Bound A Brief History Components of a Branch-and-Bound Model An Outline of the Monograph Module 1: Cluster Analysis Partitioning Module 2: Seriation Module 3: Variable selection Layout for Nonintroductory Chapters I Cluster Analysis Partitioning 13 2 An Introduction to Branch-and-Bound Methods for Partitioning Partitioning Indices A Branch-and-Bound Paradigm for Partitioning Algorithm Notation Steps of the Algorithm Algorithm Description Minimum-Diameter Partitioning Overview The INITIALIZE Step The PARTIAL SOLUTION EVALUATION Step A Numerical Example Application to a Larger Data Set An Alternative Diameter Criterion Strengths and Limitations Available Software... 39

9 x Contents 4 Minimum Within-Cluster Sums of Dissimilarities Partitioning Overview The INITIALIZE Step The PARTIAL SOLUTION EVALUATION Step A Numerical Example Application to a Larger Data Set Strengths and Limitations of the Within-Cluster Sums Criterion Available Software Minimum Within-Cluster Sums of Squares Partitioning The Relevance of Criterion (2.3) The INITIALIZE Step The PARTIAL SOLUTION EVALUATION Step A Numerical Example Application to a Larger Data Set Strengths and Limitations of the Standardized Within-Cluster Sum of Dissimilarities Criterion Available Software Multiobjective Partitioning Multiobjective Problems in Cluster Analysis Partitioning of an Object Set Using Multiple Bases Partitioning of Objects in a Single Data Set Using Multiple Criteria Strengths and Limitations Available Software II Seriation 89 7 Introduction to the Branch-and-Bound Paradigm for Seriation Background A General Branch-and-Bound Paradigm for Seriation Seriation Maximization of a Dominance Index Introduction to the Dominance Index Fathoming Tests for Optimizing the Dominance Index Determining an Initial Lower Bound The Adjacency Test The Bound Test Demonstrating the Iterative Process EXAMPLES Extracting and Ordering a Subset

10 Contents xi Tournament Matrices Maximum Dominance Index vs. Perfect Dominance for Subsets Strengths and Limitations Available Software Seriation Maximization of Gradient Indices Introduction to the Gradient Indices Fathoming Tests for Optimizing Gradient Indices The Initial Lower Bounds for Gradient Indices The Adjacency Test for Gradient Indices The Bound Test for Gradient Indices EXAMPLE An Archaeological Exploration Strengths and Limitations Available Software Seriation Unidimensional Scaling Introduction to Unidimensional Scaling Fathoming Tests for Optimal Unidimensional Scaling Determining an Initial Lower Bound Testing for Symmetry in Optimal Unidimensional Scaling Adjacency Test Expanded to Interchange Test Bound Test Demonstrating the Iterative Process EXAMPLE Can You Hear Me Now? Strengths and Limitations Available Software Seriation Multiobjective Seriation Introduction to Multiobjective Seriation Efficient Solutions Maximizing the Dominance Index for Multiple Asymmetric Proximity Matrices UDS for Multiple Symmetric Dissimilarity Matrices Comparing Gradient Indices for a Symmetric Dissimilarity Matrix Multiple Matrices with Multiple Criteria Strengths and Limitations

11 xii Contents III Variable Selection Introduction to Branch-and-Bound Methods for Variable Selection Background Variable Selection for Cluster Analysis True Variables and Masking Variables A Branch-and-Bound Approach to Variable Selection A Numerical Example Strengths, Limitations, and Extensions Variable Selection for Regression Analysis Regression Analysis A Numerical Example Application to a Larger Data Set Strengths, Limitations, and Extensions Available Software Appendix A: General Branch-and-Bound Algorithm for Partitioning 203 Appendix B: General Branch-and-Bound Algorithm Using Forward Branching for Optimal Seriation Procedures 205 References 209 Index 219