Branislav Kveton is a machine learning scientist at Adobe Research in San Jose. I was at Technicolor's Research Center from 2011 to 2014, and at Intel Research from 2006 to 2011. Before 2006, I was a graduate student in the Intelligent Systems Program at the University of Pittsburgh. My advisor was Milos Hauskrecht. My e-mail is bkveton@yahoo.com.
Research I propose, analyze, and apply algorithms that learn incrementally, run in real time, and converge to near optimal solutions as the number of training examples increases. Most of my recent work is focused on online learning of structured problems, such as graphs, submodularity, matroids, polymatroids, and reinforcement learning.

Practical problems are often so massive that even low-order polynomial-time solutions are not practical. Fortunately, many optimization problems can be solved greedily, either optimally or suboptimally with guarantees. Two popular examples of such problems are finding the maximum of a modular function on a matroid and finding the maximum of a submodular function subject to a cardinality constraint. Recently, I proposed several algorithms for solving this kind of problems when the model of the problem is initially unknown / imperfect, and is learned by interacting repeatedly with the environment. These algorithms can solve many interesting real-world problems, such as learning near-optimal preference elicitation policies from eliciting preferences, and learning optimal policies for network routing from repeated rerouting.

Kernel-based reinforcement learning (RL) on representative states is an algorithm for batch-mode non-parametric RL with continuous state variables. The algorithm does not need the model of the MDP or a parametric approximation of its value function to solve the problem. Our approach consists of two steps. First, we cover the state space of the problem using cover trees and discover k representative states. Second, we summarize the dynamics of the problem in these states and solve it by policy iteration. Our algorithm is fast, learns nearly optimal policies, has only one tunable parameter, and its solutions are easy to interpret. In practice, it outperforms fitted Q iteration (FQI), both in the quality of solutions and their computation time. The following videos show how our method approximates the optimal value functions on the pendulum:


and mountain car problems:


A MATLAB implementation of our algorithm is available for downloading.

Online semi-supervised learning on quantized graphs is an algorithm for real-time learning without explicit feedback. The algorithm iteratively builds a compact representation of the world and updates it on-the-fly with unlabeled data. The algorithm is biased through labeled examples, which are provided offline. We proved regret bounds on the predictions of the algorithm and applied it to a challenging problem, learning a robust face recognizer of a person from a single training image:

Work in progress Branislav Kveton, Zheng Wen, Azin Ashkan, and Michal Valko. Learning to Act Greedily: Polymatroid Semi-Bandits.
[Paper]
Recent work Branislav Kveton, Hung Bui, Mohammad Ghavamzadeh, Georgios Theocharous, S. Muthukrishnan, and Siqi Sun. Graphical Model Sketch. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, Riva del Garda, Italy, September 2016.
[Paper]
Branislav Kveton and Shlomo Berkovsky. Minimal Interaction Content Discovery in Recommender Systems. ACM Transactions on Interactive Intelligent Systems 6, pages 15:1-15:25, July 2016.
[Paper]
Suvash Sedhain, Hung Bui, Jaya Kawale, Nikos Vlassis, Branislav Kveton, Aditya Menon, Trung Bui, and Scott Sanner. Practical Linear Models for Large-Scale One-Class Collaborative Filtering. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York City, New York, July 2016.
[Paper]
Shi Zong, Hao Ni, Kenny Sung, Nan Rosemary Ke, Zheng Wen, and Branislav Kveton. Cascading Bandits for Large-Scale Recommendation Problems. In Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence, Jersey City, New Jersey, June 2016.
[Paper]
Sumeet Katariya, Branislav Kveton, Csaba Szepesvari, and Zheng Wen. DCM Bandits: Learning to Rank with Multiple Clicks. In Proceedings of the 33rd International Conference on Machine Learning, New York City, New York, June 2016.
[Paper]
Branislav Kveton, Zheng Wen, Azin Ashkan, and Csaba Szepesvari. Combinatorial Cascading Bandits. In Advances in Neural Information Processing Systems 28, December 2015.
[Paper]
Jaya Kawale, Hung Bui, Branislav Kveton, Long Tran-Thanh, and Sanjay Chawla. Efficient Thompson Sampling for Online Matrix-Factorization Recommendation. In Advances in Neural Information Processing Systems 28, December 2015.
[Paper]
Branislav Kveton, Csaba Szepesvari, Zheng Wen, and Azin Ashkan. Cascading Bandits: Learning to Rank in the Cascade Model. In Proceedings of the 32nd International Conference on Machine Learning, Lille, France, July 2015.
[Paper]
Zheng Wen, Branislav Kveton, and Azin Ashkan. Efficient Learning in Large-Scale Combinatorial Semi-Bandits. In Proceedings of the 32nd International Conference on Machine Learning, Lille, France, July 2015.
[Paper]
Azin Ashkan, Branislav Kveton, Shlomo Berkovsky, and Zheng Wen. Optimal Greedy Diversity for Recommendation. In Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, July 2015.
[Paper]
Branislav Kveton, Zheng Wen, Azin Ashkan, and Csaba Szepesvari. Tight Regret Bounds for Stochastic Combinatorial Semi-Bandits. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, San Diego, California, May 2015.
[Paper]
Branislav Kveton and Shlomo Berkovsky. Minimal Interaction Search in Recommender Systems. In Proceedings of the 20th ACM Conference on Intelligent User Interfaces, Atlanta, Georgia, March 2015.
[Paper]
Active and online learning Victor Gabillon, Branislav Kveton, Zheng Wen, Brian Eriksson, and S. Muthukrishnan. Large-Scale Optimistic Adaptive Submodularity. In Proceedings of the 28th AAAI Conference on Artificial Intelligence, Quebec City, Canada, July 2014.
[Paper]
Branislav Kveton, Zheng Wen, Azin Ashkan, and Hoda Eydgahi. Matroid Bandits: Practical Large-Scale Combinatorial Bandits. In Proceedings of AAAI Workshop on Sequential Decision-Making with Big Data, Quebec City, Canada, July 2014.
[Paper]
Branislav Kveton, Zheng Wen, Azin Ashkan, Hoda Eydgahi, and Brian Eriksson. Matroid Bandits: Fast Combinatorial Optimization with Learning. In Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, Quebec City, Canada, July 2014.
[Paper]
Michal Valko, Remi Munos, Branislav Kveton, and Tomas Kocak. Spectral Bandits for Smooth Graph Functions. In Proceedings of the 31st International Conference on Machine Learning, Beijing, China, June 2014.
[Paper]
Victor Gabillon, Branislav Kveton, Zheng Wen, Brian Eriksson, and S. Muthukrishnan. Adaptive Submodular Maximization in Bandit Setting. In Advances in Neural Information Processing Systems 26, pages 2697-2705, December 2013.
[Paper]
Sandilya Bhamidipati, Branislav Kveton, and S. Muthukrishnan. Minimal Interaction Search: Multi-Way Search with Item Categories. In Proceedings of AAAI Workshop on Intelligent Techniques for Web Personalization and Recommendation, Bellevue, Washington, July 2013.
Zheng Wen, Branislav Kveton, Brian Eriksson, and Sandilya Bhamidipati. Sequential Bayesian Search. In Proceedings of the 30th International Conference on Machine Learning, Atlanta, Georgia, June 2013.
[Paper] [Appendix]
Branislav Kveton and Michal Valko. Learning from a Single Labeled Face and a Stream of Unlabeled Data. In Proceedings of the 10th IEEE International Conference on Automatic Face and Gesture Recognition, Shanghai, China, April 2013.
[Paper] [BibTeX]
Zheng Wen, Branislav Kveton, and Sandilya Bhamidipati. Learning to Discover: A Bayesian Approach. Presented at NIPS Workshop on Bayesian Optimization and Decision Making, Lake Tahoe, Nevada, December 2012.
[Paper]
Stephane Caron, Branislav Kveton, Marc Lelarge, and Smriti Bhagat. Leveraging Side Observations in Stochastic Bandits. In Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, Catalina Island, California, August 2012.
[Paper] [BibTeX]
Avneesh Saluja, Frank Mokaya, Mariano Phielipp, and Branislav Kveton. Automatic Identity Inference for Smart TVs. In Proceedings of AAAI Workshop on Lifelong Learning from Sensorimotor Experience, San Francisco, California, August 2011.
[Paper] [BibTeX]
Michal Valko, Branislav Kveton, Ling Huang, and Daniel Ting. Online Semi-Supervised Learning on Quantized Graphs. In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, Catalina Island, California, July 2010.
[Paper] [BibTeX]
Qian Zhu, Branislav Kveton, Lily Mummert, and Padmanabhan Pillai. Automatic Tuning of Interactive Perception Applications. In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, Catalina Island, California, July 2010.
[Paper] [BibTeX]
Mariano Phielipp, Branislav Kveton, Magdiel Galan, Richard Lee, and Jeffrey Hightower. Fast, Accurate, and Practical Identity Inference using TV Remote Controls. In Proceedings of the 24th AAAI Conference on Artificial Intelligence, Atlanta, Georgia, July 2010.
[Paper] [BibTeX]
Branislav Kveton, Michal Valko, Matthai Philipose, and Ling Huang. Online Semi-Supervised Perception: Real-Time Learning without Explicit Feedback. In Proceedings of the 4th IEEE Online Learning for Computer Vision Workshop, San Francisco, California, June 2010. Best paper award.
[Paper]
Real-Time Adaptive Face Recognition. Presented as a demo at the 23rd Annual Conference on Neural Information Processing Systems, Whistler, Canada, December 2009.
Branislav Kveton, Jia Yuan Yu, Georgios Theocharous, and Shie Mannor. Online Learning with Expert Advice and Finite-Horizon Constraints. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pages 331-336, Chicago, Illinois, July 2008.
[Paper] [BibTeX]
Branislav Kveton, Jia Yuan Yu, Georgios Theocharous, and Shie Mannor. A Lazy Approach to Online Learning with Constraints. In Proceedings of the 10th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, Florida, January 2008.
[Paper]
Online Learning with Constraints for Systems. Presented at NIPS Workshop on Statistical Learning Techniques for Solving Systems Problems, Whistler, Canada, December 2007.
Branislav Kveton, Prashant Gandhi, Georgios Theocharous, Shie Mannor, Barbara Rosario, and Nilesh Shah. Adaptive Timeout Policies for Fast Fine-Grained Power Management. In Proceedings of the 22nd AAAI Conference on Artificial Intelligence, pages 1795-1800, Vancouver, Canada, July 2007.
[Paper] [BibTeX]
Reinforcement learning and MDPs Branislav Kveton and Georgios Theocharous. Structured Kernel-Based Reinforcement Learning. In Proceedings of the 27th AAAI Conference on Artificial Intelligence, Bellevue, Washington, July 2013.
[Paper] [BibTeX]
Branislav Kveton and Georgios Theocharous. Kernel-Based Reinforcement Learning on Representative States. In Proceedings of the 26th AAAI Conference on Artificial Intelligence, Toronto, Canada, July 2012.
[Paper] [BibTeX]
Branislav Kveton and Milos Hauskrecht. Partitioned Linear Programming Approximations for MDPs. In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, pages 341-348, Helsinki, Finland, July 2008.
[Paper] [BibTeX]
Branislav Kveton. Planning in Hybrid Structured Stochastic Domains. PhD thesis, University of Pittsburgh, December 2006.
[Paper]
Branislav Kveton, Milos Hauskrecht, and Carlos Guestrin. Solving Factored MDPs with Hybrid State and Action Variables. Journal of Artificial Intelligence Research 27, pages 153-201, October 2006.
[Paper] [BibTeX]
Branislav Kveton and Milos Hauskrecht. Learning Basis Functions in Hybrid Domains. In Proceedings of the 21st National Conference on Artificial Intelligence, pages 1161-1166, Boston, Massachusetts, July 2006.
[Paper] [BibTeX]
Branislav Kveton and Milos Hauskrecht. Solving Factored MDPs with Exponential-Family Transition Models. In Proceedings of the 16th International Conference on Automated Planning and Scheduling, pages 114-120, The English Lake District, Cumbria, United Kingdom, June 2006.
[Paper] [BibTeX]
On the Smoothness of Linear Value Function Approximations. Presented at ICAPS Doctoral Consortium, The English Lake District, Cumbria, United Kingdom, June 2006.
Milos Hauskrecht and Branislav Kveton. Approximate Linear Programming for Solving Hybrid Factored MDPs. In Proceedings of the 9th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, Florida, January 2006.
[Paper]
Branislav Kveton and Milos Hauskrecht. An MCMC Approach to Solving Hybrid Factored MDPs. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, pages 1346-1351, Edinburgh, Scotland, August 2005.
[Paper] [BibTeX]
Carlos Guestrin, Milos Hauskrecht, and Branislav Kveton. Solving Factored MDPs with Continuous and Discrete Variables. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, pages 235-242, Banff, Canada, July 2004.
[Paper] [BibTeX]
Carlos Guestrin, Milos Hauskrecht, and Branislav Kveton. Solving Factored MDPs with Continuous and Discrete Variables. In Proceedings of AAAI Workshop on Learning and Planning in Markov Processes - Advances and Challenges, pages 19-24, San Jose, California, July 2004.
Branislav Kveton and Milos Hauskrecht. Heuristic Refinements of Approximate Linear Programming for Factored Continuous-State Markov Decision Processes. In Proceedings of the 14th International Conference on Automated Planning and Scheduling, pages 306-314, Whistler, Canada, June 2004.
[Paper] [BibTeX]
Heuristic Refinements of Approximate Linear Programming for Factored Continuous-State Markov Decision Processes. Presented at ICAPS Doctoral Consortium, Whistler, Canada, June 2004.
Milos Hauskrecht and Branislav Kveton. Linear Program Approximations for Factored Continuous-State Markov Decision Processes. In Advances in Neural Information Processing Systems 16, pages 895-902, June 2004.
[Paper] [BibTeX]
Linear Program Approximations for Factored Continuous-State Markov Decision Processes. Presented at NIPS Workshop on Planning for the Real World: The Promises and Challenges of Dealing with Uncertainty, Whistler, Canada, December 2003.
Anomaly detection Michal Valko, Branislav Kveton, Hamed Valizadegan, Gregory Cooper, and Milos Hauskrecht. Conditional Anomaly Detection with Soft Harmonic Functions. In Proceedings of the 11th IEEE International Conference on Data Mining, Vancouver, Canada, December 2011.
[Paper] [BibTeX]
Michal Valko, Hamed Valizadegan, Branislav Kveton, Gregory Cooper, and Milos Hauskrecht. Conditional Anomaly Detection using Soft Harmonic Functions: An Application to Clinical Alerting. In Proceedings of ICML Workshop on Machine Learning for Global Challenges, Bellevue, Washington, June 2011.
[Paper]
Milos Hauskrecht, Michal Valko, Branislav Kveton, Shyam Visweswaram, and Gregory Cooper. Evidence-Based Anomaly Detection in Clinical Domains. In Proceedings of the 2007 American Medical Informatics Association Annual Symposium, Chicago, Illinois, November 2007.
Denver Dash, Branislav Kveton, John Mark Agosta, Eve Schooler, Jaideep Chandrashekar, Abraham Bachrach, and Alex Newman. When Gossip is Good: Distributed Probabilistic Inference for Detection of Slow Network Intrusions. In Proceedings of the 21st National Conference on Artificial Intelligence, pages 1115-1122, Boston, Massachusetts, July 2006.
[Paper] [BibTeX]
John Mark Agosta, Abraham Bachrach, Denver Dash, Branislav Kveton, Alex Newman, and Eve Schooler. Distributed Detection and Inference for Enterprise Networks. Presented at NIPS Workshop on Intelligence Beyond the Desktop, Whistler, Canada, December 2005.
John Mark Agosta, Abraham Bachrach, Denver Dash, Branislav Kveton, Alex Newman, and Eve Schooler. Distributed Inference to Detect a Network Attack. Presented at the 4th Adaptive and Resilient Computing Security Workshop, Santa Fe, New Mexico, November 2005.
Branislav Kveton and Denver Dash. Automatic Excursion Detection in Manufacturing: Preliminary Results. In Proceedings of the 18th International Florida Artificial Intelligence Research Society Conference, pages 375-380, Clearwater Beach, Florida, May 2005.
[Paper] [BibTeX]
General machine learning Salman Salamatian, Nadia Fawaz, Branislav Kveton, and Nina Taft. SPPM: Sparse Privacy Preserving Mappings. In Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, Quebec City, Canada, July 2014.
[Paper]
Salman Salamatian, Amy Zhang, Flavio du Pin Calmon, Sandilya Bhamidipati, Nadia Fawaz, Branislav Kveton, Pedro Oliveira, and Nina Taft. How to Hide the Elephant - or the Donkey - in the Room: Practical Privacy Against Statistical Inference for Large Data. In Proceedings of the 1st IEEE Global Conference on Signal and Information Processing, Austin, Texas, December 2013.
Diana Joumblatt, Jaideep Chandrashekar, Branislav Kveton, Nina Taft, and Renata Teixeira. Predicting User Dissatisfaction with Internet Application Performance at End-Hosts. In Proceedings of the 32nd IEEE International Conference on Computer Communications, Turin, Italy, April 2013.
[Paper] [BibTeX]
Tianxi Li, Branislav Kveton, Yu Wu, and Ashwin Kashyap. Incorporating Metadata into Dynamic Topic Analysis. In Proceedings of the 9th Bayesian Modeling Applications Workshop, Catalina Island, California, August 2012.
[Paper] [BibTeX]
Jennifer Healey, Georgios Theocharous, and Branislav Kveton. Does My Driving Scare You? Presented at the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Pittsburgh, Pennsylvania, November 2010.
Georgios Theocharous, Jennifer Healey, and Branislav Kveton. User Adaptive Lane Deviation Warnings. Presented at the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Pittsburgh, Pennsylvania, November 2010.
Branislav Kveton, Michal Valko, Ali Rahimi, and Ling Huang. Semi-Supervised Learning with Max-Margin Graph Cuts. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Sardinia, Italy, May 2010.
[Paper] [BibTeX]
Keng-hao Chang, Jeffrey Hightower, and Branislav Kveton. Inferring Identity using Accelerometers in Television Remote Controls. In Proceedings of the 7th International Conference on Pervasive Computing, pages 151-167, Nara, Japan, May 2009.
[Paper] [BibTeX]
Georgios Theocharous, Shie Mannor, Nilesh Shah, Prashant Gandhi, Branislav Kveton, Sajid Siddiqi, and Chih-Han Yu. Machine Learning for Adaptive Power Management. Intel Technology Journal 10(4), November 2006.
[Paper]
Ondrej Turza and Branislav Kveton. Decision Support Automatization in HR Management. In Proceedings of the 2nd International Conference on Human Potential Management in the Corporate Environment, pages 177-182, Zilina, Slovak Republic, March 2005.
Branislav Kveton. Estimation of the Critical Value of Oriented Percolation in 1 + 1 Dimensions. Master's thesis, Comenius University, July 2001.