Benjamin Van Roy. Professor of Electrical Engineering, of Management Science and Engineering and, by courtesy, of Computer Science.

Transcription

1 Professor of Electrical Engineering, of Management Science and Engineering and, by courtesy, of Computer Science CONTACT INFORMATION Administrator Andrea Kuduk - Administrative Associate Bio akuduk@stanford.edu Tel (650) BIO is a Professor of Electrical Engineering, Management Science and Engineering, and, by courtesy, Computer Science, at Stanford University, where he has served on the faculty since His research focuses on understanding how an agent interacting with a poorly understood environment can learn over time to make effective decisions. He is interested in questions concerning what is possible or impossible as well as how to design efficient learning algorithms that achieve the possible. His research contributes to the fields of reinforcement learning, online optimization, and approximate dynamic programming, and offers means to addressing central problems of artificial intelligence. He has graduated fifteen doctoral students, published over forty articles in peer-reviewed journals, and been listed as an inventor in over a dozen patents. He has served on the editorial boards of Machine Learning, Mathematics of Operations Research, and Operations Research, for which he has also served as editor of the Financial Engineering Area. He has also founded and/or led research programs at several technology companies, including Unica (acquired by IBM), Enuvis (acquired by SiRF), and Morgan Stanley. He received the SB in Computer Science and Engineering and the SM and PhD in Electrical Engineering and Computer Science, all from MIT. He has been a recipient of the MIT George C. Newton Undergraduate Laboratory Project Award, the MIT Morris J. Levin Memorial Master's Thesis Award, the MIT George M. Sprowls Doctoral Dissertation Award, the National Science Foundation CAREER Award, the Stanford Tau Beta Pi Award for Excellence in Undergraduate Teaching, and the Management Science and Engineering Department's Graduate Teaching Award. He is an INFORMS Fellow and has been a Frederick E. Terman Fellow and a David Morgenthaler II Faculty Scholar. He has held visiting positions as the Wolfgang and Helga Gaul Visiting Professor at the University of Karlsruhe and as the Chin Sophonpanich Foundation Professor and the InTouch Professor at Chulalongkorn University. ACADEMIC APPOINTMENTS Professor, Electrical Engineering Professor, Management Science and Engineering Member, Bio-X Page 1 of 9

2 PROGRAM AFFILIATIONS Institute for Computational and Mathematical Engineering (ICME) PROFESSIONAL EDUCATION BS, Massachusetts Institute of Technology, Computer Science and Engineering (1993) MS, Massachusetts Institute of Technology, Electrical Engineering and Computer Science (1995) PhD, Massachusetts Institute of Technology, Electrical Engineering and Computer Science (1998) Teaching COURSES Introduction to Optimization: ENGR 62, MS&E 111, MS&E 211 (Spr) Reinforcement Learning: MS&E 338 (Spr) Advanced Topics in Information Science and Technology: MS&E 338 (Win) Dynamic Programming and Stochastic Control: MS&E 351 (Aut) Introduction to Optimization: ENGR 62, MS&E 111 (Aut) Advanced Topics in Information Science and Technology: MS&E 338 (Win) Dynamic Programming and Stochastic Control: MS&E 351 (Aut) STANFORD ADVISEES Doctoral Dissertation Reader (AC) Hongseok Namkoong Doctoral Dissertation Advisor (AC) Maria Dimakopoulou, Xiuyuan Lu Master's Program Advisor Nahri Ahn, Stephone Christian, Christopher Spears Publications PUBLICATIONS Learning to Optimize Via Posterior Sampling. Russo, D., Roy, B., Van Reputation Markets Yan, X., Directed Principal Component Analysis. Kao, Y., H., Roy, B., Van Adaptive Execution: Exploration and Learning of Price Impact. Park, B., Roy, B., Van Page 2 of 9

3 Adaptive Execution: Exploration and Learning of Price Impact OPERATIONS RESEARCH Park, B., 2015; 63 (5): Learning to Optimize via Posterior Sampling MATHEMATICS OF OPERATIONS RESEARCH Russo, D., 2014; 39 (4): Directed Principal Component Analysis OPERATIONS RESEARCH Kao, Y., 2014; 62 (4): Learning a factor model via regularized PCA MACHINE LEARNING Kao, Y., 2013; 91 (3): Eluder Dimension and the Sample Complexity of Optimistic Exploration to appear in NIPS. Russo, D., Roy, B., Van 2013 Efficient Exploration and Value Function Generalization in Deterministic Systems abridged version to appear in NIPS. Wen, Z., Roy, B., Van 2013 (More) Efficient Reinforcement Learning via Posterior Sampling to appear in NIPS. Osband, I., Russo, D., Roy, B., Van 2013 Strategic execution in the presence of an uninformed arbitrageur JOURNAL OF FINANCIAL MARKETS Moallemi, C. C., Park, B., 2012; 15 (4): Intermediated Blind Portfolio Auctions MANAGEMENT SCIENCE Padilla, M., 2012; 58 (9): Portfolio selection with qualitative input JOURNAL OF BANKING & FINANCE Chiarawongse, A., Kiatsupaibul, S., Tirapat, S., 2012; 36 (2): Efficient Reinforcement Learning for High Dimensional Linear Systems Advances in Neural Information Processing Systems 25 Ibrahimi, M., Javanmard, A., Roy, B., Van MIT Press.2012 Approximate Dynamic Programming for Optimizing Oil Production Chapter 25 in Reinforcement Learning and Approximate Dynamic Programming for Feedback Control Wen, Z., Durlofsky, L., J., Roy, B., Van, Aziz, K. edited by Lewis, F., L., Liu, D. Wiley-IEEE Press.2012 Industry dynamics: Foundations for models with an infinite number of firms JOURNAL OF ECONOMIC THEORY Weintraub, G. Y., Benkard, C. L., 2011; 146 (5): Resource Allocation via Message Passing INFORMS JOURNAL ON COMPUTING 2011; 23 (2): Control of Diffusions via Linear Programming in Stochastic Programming: The State of the Art, in Honor of George B. Dantzig Han, J., Page 3 of 9

4 Springer.2011: Use of Approximate Dynamic Programming for Production Optimization Wen, Z., Durlofsky, L., J., Roy, B., Van, Aziz, K Manipulation Robustness of Collaborative Filtering MANAGEMENT SCIENCE, Yan, X. 2010; 56 (11): Investment and Market Structure in Industries with Congestion OPERATIONS RESEARCH Johari, R., Weintraub, G. Y., 2010; 58 (5): On Regression-Based Stopping Times DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS 2010; 20 (3): Computational Methods for Oblivious Equilibrium OPERATIONS RESEARCH Weintraub, G. Y., Benkard, C. L., 2010; 58 (4): Universal Reinforcement Learning IEEE TRANSACTIONS ON INFORMATION THEORY Farias, V. F.,, Weissman, T. 2010; 56 (5): Convergence of Min-Sum Message-Passing for Convex Optimization IEEE TRANSACTIONS ON INFORMATION THEORY 2010; 56 (4): Dynamic Pricing with a Prior on Market Response OPERATIONS RESEARCH Farias, V. F., 2010; 58 (1): Convergence of the Min-Sum Algorithm for Convex Optimization IEEE Transactions on Information Theory Moallemi, C., C., Roy, B., Van 2010; 56 (4): Convergence of Min-Sum Message Passing for Quadratic Optimization IEEE TRANSACTIONS ON INFORMATION THEORY 2009; 55 (5): Directed Regression Advances in Neural Information Processing Systems 22 Kao, Y., H., MIT Press.2009: Manipulation-Resistant Collaborative Filtering Systems Roy, B., Van, Yan, X Markov Perfect Industry Dynamics With Many Firms ECONOMETRICA Weintraub, G. Y., Benkard, C. L., 2008; 76 (6): Capacity of the trapdoor channel with feedback IEEE International Symposium on Information Theory Permuter, H., Cuff, P.,, Weissman, T. IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC.2008: Approximate and Data-Driven Dynamic Programming for Queueing Networks Moallemi, C., C., Kumar, S., Page 4 of 9

5 2008 A short proof of optimality for the MIN cache replacement algorithm INFORMATION PROCESSING LETTERS 2007; 102 (2-3): Managing the quality of a resource with stock and flow controls JOURNAL OF PUBLIC ECONOMICS Keohane, N.,, Zeckhauser, R. 2007; 91 (3-4): Capacity and zero-error capacity of the chemical channel with feedback IEEE International Symposium on Information Theory Permuter, H., Cuff, P.,, Weissman, T. IEEE.2007: An Approximate Dynamic Programming Approach to Network Revenue Management Farias, V., F., 2007 Consensus propagation 19th Annual Conference on Neural Information Processing Systems (NIPS 05) IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC.2006: A cost-shaping linear program for average-cost approximate dynamic programming with performance guarantees MATHEMATICS OF OPERATIONS RESEARCH 2006; 31 (3): Performance loss bounds for approximate value iteration with state aggregation MATHEMATICS OF OPERATIONS RESEARCH 2006; 31 (2): A generalized Kalman filter for fixed point approximation and efficient temporal-difference learning DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS Choi, D., 2006; 16 (2): Approximation algorithms for dynamic resource allocation OPERATIONS RESEARCH LETTERS Farias, V. F., 2006; 34 (2): A nonparametric approach to multiproduct pricing OPERATIONS RESEARCH Rusmevichientong, P.,, Glynn, P. W. 2006; 54 (1): Tetris: A Study of Randomized Constraint Sampling in Probabilistic and Randomized Methods for Design Under Uncertainty Farias, V., F., edited by Calafiore, G., Dabbene, F. Springer-Verlag.2006 Opportunities and Challenges in Using Online Preference Data for Vehicle Pricing: A Case Study at General Motors Journal of Revenue and Pricing Management Rusmevichientong, P., Salisbury, J., A., Truss, L., T., Roy, B., Van, Glynn, P., W. 2006; 5 (1): Consensus Propagation Advances in Neural Information Processing Systems 18 Moallemi, C., C., MIT Press.2006 TD(0) Leads to Better Policies than Approximate Value Iteration Advances in Neural Information Processing Systems 18 Roy, B., Van MIT Press.2006 Page 5 of 9

6 Oblivious Equilibrium: A Mean Field Approximation for Large Scale Dynamic Games Advances in Neural Information Processing Systems 18 Weintraub, G., Y., Benkard, C., L., Roy, B., Van 2006 A Generalized Kalman Filter for Fixed Point Approximation and Efficient Temporal-Difference Learning Discrete Event Dynamic Systems Choi, D., S., 2006; 16 (2) A Non-Parametric Approach to Multi-Product Pricing Operations Research Rusmevichientong, P.,, Glynn, P., W. 2006; 54 (1): An approximate dynamic programming approach to decentralized control of stochastic systems Workshop on Control of Uncertain Systems Cogill, R., Rotkowitz, M.,, Lall, S. SPRINGER-VERLAG BERLIN.2006: A universal scheme for learning IEEE International Symposium on Information Theory and Its Applications Farias, V. F.,, Weissman, T. IEEE.2005: Solitaire: Man Versus Machine Advances in Neural Information Processing Systems 17 Yan, X., Diaconis, P., Rusmevichientong, P., Roy, B., Van MIT Press.2005 A Linear Program for Bellman Error Minimization with Performance Guarantees Advances in Neural Information Processing Systems 17 de Farias, D., P., MIT Press.2005 On constraint sampling in the linear programming approach to approximate dynamic programming MATHEMATICS OF OPERATIONS RESEARCH 2004; 29 (3): Making eigenvector-based reputation systems robust to collusion 3rd International Workshop on Algorithms and Models for the Web-Graph Zhang, H., Goel, A., Govindan, R., Mason, K., SPRINGER-VERLAG BERLIN.2004: Approximate Dynamic Programming for High-Dimensional Dynamic Resource Allocation Problems in Handbook of Learning and Approximate Dynamic Programming Powell, W., B., edited by Si, J., Barto, A., G., Powell, W., B. Wiley-IEEE Press, Hoboken, NJ.2004: An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems Cogill, R., Rotkowitz, M., Roy, B., Van, Lall, S Distributed optimization in adaptive networks 17th Annual Conference on Neural Information Processing Systems (NIPS) M I T PRESS.2004: The linear programming approach to approximate dynamic programming OPERATIONS RESEARCH 2003; 51 (6): Decentralized decision-making in a large team with local information GAMES AND ECONOMIC BEHAVIOR Rusmevichientong, P., 2003; 43 (2): On constraint sampling in the linear programming approach to approximate linear programming 42nd IEEE Conference on Decision and Control Page 6 of 9

7 IEEE.2003: Decentralized Protocols for Optimization of Sensor Networks Moallemi, C., C., 2003 Approximate Linear Programming for Average-Cost Dynamic Programming Advances in Neural Information Processing Systems 15 de Farias, D., P., MIT Press.2003 Improving Eigenvector-Based Reputation Systems Against Collusion Workshop on Algorithms and Models for the Web Graph Zhang, H., Goel, A., Govindan, R., Mason, K., Roy, B., Van 2003 Book Review: Self-Learning Control of Finite Markov Chains, by A. S. Poznyak, K. Najim, and E. Gomez-Ramirez Automatica Roy, B., Van 2003; 39 (2): On average versus discounted reward temporal-difference learning MACHINE LEARNING 2002; 49 (2-3): Approximate dynamic programming via linear programming 15th Annual Conference on Neural Information Processing Systems (NIPS) M I T PRESS.2002: Algorithms for GPS Operation Indoors and Downtown GPS Solutions Agarwal, N., Basch, J., Beckmann, P., Bharti, P., Bloebaum, S., Casadei, S., 2002; 6 (3): Regression methods for pricing complex American-Style options IEEE TRANSACTIONS ON NEURAL NETWORKS 2001; 12 (4): An Analysis of Belief Propagation on the Turbo Decoding Graph with Gaussian Densities IEEE Transactions on Information Theory Rusmevichientong, P., 2001; 47 (2): Neuro-Dynamic Programming: Overview and Recent Trends in Handbook of Markov Decision Processes: Methods and Applications edited by Feinberg, E., Shwartz, A. Kluwer.2001 A Generalized Kalman Filter for Fixed Point Approximation and Efficient Temporal-Difference Learning Choi, D., S., 2001 A Tractable POMDP for a Class of Sequencing Problems Rusmevichientong, P., 2001 On the existence of fixed points for approximate value iteration and temporal-difference learning JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS 2000; 105 (3): The optimal harvesting of environmental bads 39th IEEE Conference on Decision and Control Keohane, N.,, Zeckhauser, R. IEEE.2000: Page 7 of 9

8 Fixed Points for Approximate Value Iteration and Temporal-Difference Learning de Farias, D., P., 2000 Approximate value iteration with randomized policies 39th IEEE Conference on Decision and Control IEEE.2000: Temporal-difference learning and applications in finance Computational Finance 1999 Conference M I T PRESS.2000: Approximate value iteration and temporal-difference learning Symposium on Adaptive Systems for Signal Processing, Communications, and Control (AS- SPCC) IEEE.2000: An analysis of turbo decoding with Gaussian densities 13th Annual Conference on Neural Information Processing Systems (NIPS) Rusmevichientong, P., M I T PRESS.2000: Average cost temporal-difference learning AUTOMATICA 1999; 35 (11): Optimal stopping of Markov processes: Hilbert space theory, approximation algorithms, and an application to pricing high-dimensional financial derivatives IEEE TRANSACTIONS ON AUTOMATIC CONTROL 1999; 44 (10): Optimal Stopping of Markov Processes: Hilbert Space Theory, Approximation Algorithms, and an Application to Pricing High-Dimensional Financial Derivatives IEEE Transactions on Automatic Control Tsitsiklis, J., N., 1999; 44 (10): An analysis of temporal-difference learning with function approximation IEEE TRANSACTIONS ON AUTOMATIC CONTROL Tsitsiklis, J. N., VANROY, B. 1997; 42 (5): Neuro-dynamic programming overview and a case study in optimal stopping 36th IEEE Conference on Decision and Control IEEE.1997: Overview of Neuro-Dynamic Programming and a Case Study in Optimal Stopping Tsitsiklis, J., N., 1997 A Neuro-Dynamic Programming Approach to Retailer Inventory Management, Bertsekas, D., P., Lee, Y., Tsitsiklis, J., N Solving Data Mining Problems Through Pattern Recognition Prentice-Hall Kennedy, R., Lee, Y., Roy, B., Van, Reed, C., Lippman, R A neuro-dynamic programming approach to retailer inventory management 36th IEEE Conference on Decision and Control, Bertsekas, D. P., Lee, Y., Tsitsiklis, J. N. IEEE.1997: Analysis of temporal-difference learning with function approximation 10th Annual Conference on Neural Information Processing Systems (NIPS) Page 8 of 9

9 Tsitsiklis, J. N., VANROY, B. M I T PRESS.1997: Average cost temporal-difference learning 36th IEEE Conference on Decision and Control IEEE.1997: Approximate solutions to optimal stopping problems 10th Annual Conference on Neural Information Processing Systems (NIPS) Tsitsiklis, J. N., VANROY, B. M I T PRESS.1997: Feature-based methods for large scale dynamic programming MACHINE LEARNING Tsitsiklis, J. N., VANROY, B. 1996; 22 (1-3): Stable linear approximations to dynamic programming for stochastic control problems with local transitions 9th Annual Conference on Neural Information Processing Systems (NIPS) VANROY, B., Tsitsiklis, J. N. M I T PRESS.1996: Feature-based methods for large scale dynamic programming 34th IEEE Conference on Decision and Control Tsitsiklis, J. N., VANROY, B. IEEE.1995: Solving Pattern Recognition Problems Unica Kennedy, R., Lee, Y., Reed, C., 1995 Page 9 of 9