Abhimanyu Dubey

Bio | News | Research | Publications | CV

Ph.D. Student
Human Dynamics Laboratory
Media Lab and Institute for Data, Systems and Society
Massachusetts Institute of Technology

dubeya[at]mit.edu | GitHub | Twitter | Google Scholar | LinkedIn


I am a 4th-year Ph.D. student in the Human Dynamics group at MIT, advised by Professor Alex Pentland. My research interests are in robust and cooperative (federated) machine learning, including problems in multi-agent decision-making and transfer learning. Prior to this, I received a master's degree in Computer Science and bachelor's degree in Electrical Engineering at IIT Delhi, where I was advised by Professor Sumeet Agarwal. I've also spent time as a research intern at Facebook AI, and was a post-baccalaureate fellow at the Department of Economics at Harvard, under Professor Ed Glaeser. My research has been supported by a Snap Research Scholarship (2019) and an Emerging Worlds Fellowship (2017).

I am also part of the organizing committee of the Machine Learning across MIT (MLxMIT) initiative. In my spare time, I enjoy road biking and high fantasy fiction. In another life, I was a nescient wordsmith; you can find some of my writing here and here.


Publications

Provably Learning Pareto-Optimal Policies in Low-Rank Cooperative Markov Games
Abhimanyu Dubey and Alex Pentland
Preprint 2021
Preliminary version in ICML 2021 Workshop on Reinforcement Learning Theory
Abstract | Technical Report

We study cooperative multi-agent reinforcement learning in episodic Markov games with n agents. While the global state and action spaces typically grow exponentially in n in this setting, in this paper, we introduce a novel framework that combines function approximation and a graphical dependence structure that restricts the decision space to O(dn) for each agent, where d is the ambient dimensionality of the problem. We present a multi-agent value iteration algorithm that, under mild assumptions, provably recovers the set of Pareto-optimal policies, with finitesample guarantees on the incurred regret. Furthermore, we demonstrate that our algorithm is no-regret even when there are only O(1) episodes with communication, providing a scalable and provably no-regret algorithm for multi-agent reinforcement learning with function approximation. Our work provides a tractable approach to multi-agent decision-making that is provably efficient and amenable to large-scale collaborative systems.
Hide Abstract

Adaptive Methods for Real-World Domain Generalization
Abhimanyu Dubey, Vignesh Ramanathan, Alex Pentland and Dhruv Mahajan
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021 (oral)
Abstract | Technical Report | Poster | Dataset | Code

Invariant approaches have been remarkably successful in tackling the problem of domain generalization, where the objective is to perform inference on data distributions different from those used in training. In our work, we investigate whether it is possible to leverage domain information from the unseen test samples themselves. We propose a domain-adaptive approach consisting of two steps: a) we first learn a discriminative domain embedding from unsupervised training examples, and b) use this domain embedding as supplementary information to build a domain-adaptive model, that takes both the input as well as its domain into account while making predictions. For unseen domains, our method simply uses few unlabelled test examples to construct the domain embedding. This enables adaptive classification on any unseen domain. Our approach achieves state-of-the-art performance on various domain generalization benchmarks. In addition, we introduce the first real-world, large-scale domain generalization benchmark, Geo-YFCC, containing 1.1M samples over 40 training, 7 validation and 15 test domains, orders of magnitude larger than prior work. We show that the existing approaches either do not scale to this dataset or underperform compared to the simple baseline of training a model on the union of data from all training domains. In contrast, our approach achieves a significant improvement.
Hide Abstract

No-Regret Algorithms for Private Gaussian Process Bandit Optimization
Abhimanyu Dubey
International Conference on Artificial Intelligence and Statistics (AIStats), 2021
Abstract | Technical Report

The widespread proliferation of data-driven decision-making has ushered in a recent interest in the design of privacy-preserving algorithms. In this paper, we consider the ubiquitous problem of gaussian process (GP) bandit optimization from the lens of privacy-preserving computation. We propose a solution for differentially private GP bandit optimization that combines a uniform kernel approximator with random perturbations, providing a generic framework to create differentially-private (DP) Gaussian process bandit algorithms. For two specific DP settings - joint and local differential privacy, we provide algorithms based on efficient quadrature Fourier feature approximators, that are computationally efficient and provably no-regret for a class of stationary kernel functions. In contrast to previous work, our algorithms maintain differential privacy throughout the optimization procedure and critically do not rely on the sample path for prediction, making them scalable and straightforward to release as well.
Hide Abstract

Differentially-Private Federated Linear Bandits
Abhimanyu Dubey and Alex Pentland
Neural Information Processing Systems (NeurIPS), 2020 (spotlight)
Abstract | Technical Report | Proceedings | Slides

The rapid proliferation of decentralized learning systems mandates the need for differentially-private cooperative learning. In this paper, we study this in context of the contextual linear bandit: we consider a collection of agents cooperating to solve a common contextual bandit, while ensuring that their communication remains private. For this problem, we devise FEDUCB, a multiagent private algorithm for both centralized and decentralized (peer-to-peer) federated learning. We provide a rigorous technical analysis of its utility in terms of regret, improving several results in cooperative bandit learning, and provide rigorous privacy guarantees as well. Our algorithms provide competitive performance both in terms of pseudoregret bounds and empirical benchmark performance in various multi-agent settings.
Hide Abstract

Kernel Methods for Cooperative Multi-Agent Contextual Bandits
Abhimanyu Dubey and Alex Pentland
International Conference on Machine Learning (ICML), 2020
Abstract | Technical Report | Proceedings | Video | Slides

Cooperative multi-agent decision making involves a group of agents cooperatively solving learning problems while communicating over a network with delays. In this paper, we consider the kernelised contextual bandit problem, where the reward obtained by an agent is an arbitrary linear function of the contexts’ images in the related reproducing kernel Hilbert space (RKHS), and a group of agents must cooperate to collectively solve their unique decision problems. For this problem, we propose COOP-KERNELUCB, an algorithm that provides near-optimal bounds on the per-agent regret, and is both computationally and communicatively efficient. For special cases of the cooperative problem, we also provide variants of COOP-KERNELUCB that provides optimal peragent regret. In addition, our algorithm generalizes several existing results in the multi-agent bandit setting. Finally, on a series of both synthetic and real-world multi-agent network benchmarks, we demonstrate that our algorithm significantly outperforms existing benchmarks.
Hide Abstract

Cooperative Multi-Agent Bandits with Heavy Tails
Abhimanyu Dubey and Alex Pentland
International Conference on Machine Learning (ICML), 2020
Abstract | Technical Report | Proceedings | Video | Slides

We study the heavy-tailed stochastic bandit problem in the cooperative multi-agent setting, where a group of agents interact with a common bandit problem, while communicating on a network with delays. Existing algorithms for the stochastic bandit in this setting utilize confidence intervals arising from an averaging-based communication protocol known as running consensus, that does not lend itself to robust estimation for heavy-tailed settings. We propose MP-UCB, a decentralized multi-agent algorithm for the cooperative stochastic bandit that incorporates robust estimation with a message-passing protocol. We prove optimal regret bounds for MP-UCB for several problem settings, and also demonstrate its superiority to existing methods. Furthermore, we establish the first lower bounds for the cooperative bandit problem, in addition to providing efficient algorithms for robust bandit estimation of location.
Hide Abstract

Private and Byzantine-Proof Cooperative Decision-Making
Abhimanyu Dubey and Alex Pentland
Autonomous Agents and MultiAgent Systems (AAMAS), 2020
Abstract | Technical Report | Proceedings

The cooperative bandit problem is a multi-agent decision problem involving a group of agents that interact simultaneously with a multi-armed bandit, while communicating over a network with delays. The central idea in this problem is to design algorithms that can efficiently leverage communication to obtain improvements over acting in isolation. In this paper, we investigate the stochastic bandit problem under two settings - (a) when the agents wish to make their communication private with respect to the action sequence, and (b) when the agents can be byzantine, i.e., they provide (stochastically) incorrect information. For both these problem settings, we provide upper-confidence bound algorithms that obtain optimal regret while being (a) differentially-private and (b) tolerant to byzantine agents. Our decentralized algorithms require no information about the network of connectivity between agents, making them scalable to large dynamic systems. We test our algorithms on a competitive benchmark of random graphs and demonstrate their superior performance with respect to existing robust algorithms. We hope that our work serves as an important step towards creating distributed decision-making systems that maintain privacy.
Hide Abstract

Leveraging Agent Communication Topology in Deep Reinforcement Learning
Dhaval Adjodah, Dan Calacci, Abhimanyu Dubey, Peter Krafft, Esteban Moro and Alex Pentland
Autonomous Agents and MultiAgent Systems (AAMAS), 2020
Abstract | Technical Report | Extended Abstract | Video | Code

A common technique to improve learning performance in deep reinforcement learning (DRL) and many other machine learning algorithms is to run multiple learning agents in parallel. A neglected component in the development of these algorithms has been how best to arrange the learning agents involved to improve distributed search. Here we draw upon results from the networked optimization literatures suggesting that arranging learning agents in communication networks other than fully connected topologies (the implicit way agents are commonly arranged in) can improve learning. We explore the relative performance of four popular families of graphs and observe that one such family (Erdos-Renyi random graphs) empirically outperforms the de facto fully-connected communication topology across several DRL benchmark tasks. Additionally, we observe that 1000 learning agents arranged in an Erdos-Renyi graph can perform as well as 3000 agents arranged in the standard fully-connected topology, showing the large learning improvement possible when carefully designing the topology over which agents communicate. We complement these empirical results with a theoretical investigation of why our alternate topologies perform better. Overall, our work suggests that distributed machine learning algorithms could be made more effective if the communication topology between learning agents was optimized.
Hide Abstract

Thompson Sampling on Symmetric α-Stable Bandits
Abhimanyu Dubey and Alex Pentland
International Joint Conference on Artificial Intelligence (IJCAI), 2019
Abstract | Technical Report | Proceedings | Slides

Thompson Sampling provides an efficient technique to introduce prior knowledge in the multi-armed bandit problem, along with providing remarkable empirical performance. In this paper, we revisit the Thompson Sampling algorithm under rewards drawn from α-stable distributions, which are a class of heavy-tailed probability distributions utilized in finance and economics, in problems such as modeling stock prices and human behavior. We present an efficient framework for α-stable posterior inference, which leads to two algorithms for Thompson Sampling in this setting. We prove finite-time regret bounds for both algorithms, and demonstrate through a series of experiments the stronger performance of Thompson Sampling in this setting. With our results, we provide an exposition of α-stable distributions in sequential decision-making, and enable sequential Bayesian inference in applications from diverse fields in finance and complex systems that operate on heavy-tailed features.
Hide Abstract

Defense Against Adversarial Attacks using Web-Scale Nearest Neighbor Search
Abhimanyu Dubey, Laurens van der Maaten, Zeki Yalniz, Yixuan Li and Dhruv Mahajan
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019 (Oral)
Abstract | Technical Report | Proceedings | Slides

A plethora of recent work has shown that convolutional networks are not robust to adversarial images: images that are created by perturbing a sample from the data distribution as to maximize the loss on the perturbed example. In this work, we hypothesize that adversarial perturbations move the image away from the image manifold in the sense that there exists no physical process that could have produced the adversarial image. This hypothesis suggests that a successful defense mechanism against adversarial images should aim to project the images back onto the image manifold. We study such defense mechanisms, which approximate the projection onto the unknown image manifold by a nearest-neighbor search against a web-scale image database containing tens of billions of images. Empirical evaluations of this defense strategy on ImageNet suggest that it is very effective in attack settings in which the adversary does not have access to the image database. We also propose two novel attack methods to break nearest-neighbor defenses, and demonstrate conditions under which nearest-neighbor defense fails. We perform a series of ablation experiments, which suggest that there is a trade-off between robustness and accuracy in our defenses, that a large image database (with hundreds of millions of images) is crucial to get good performance, and that careful construction the image database is important to be robust against attacks tailored to circumvent our defenses.
Hide Abstract

Maximum-Entropy Fine-Grained Classification
Abhimanyu Dubey, Otkrist Gupta, Ramesh Raskar and Nikhil Naik
Neural Information Processing Systems (NeurIPS), 2018
Abstract | Technical Report | Proceedings | Slides

Fine-Grained Visual Classification (FGVC) is an important computer vision problem that involves small diversity within the different classes, and often requires expert annotators to collect data. Utilizing this notion of small visual diversity, we revisit Maximum-Entropy learning in the context of fine-grained classification, and provide a training routine that maximizes the entropy of the output probability distribution for training convolutional neural networks on FGVC tasks. We provide a theoretical as well as empirical justification of our approach, and achieve state-of-the-art performance across a variety of classification tasks in FGVC, that can potentially be extended to any fine-tuning task. Our method is robust to different hyperparameter values, amount of training data and amount of training label noise and can hence be a valuable tool in many similar problems.
Hide Abstract

Pairwise Confusion for Fine-Grained Visual Classification
Abhimanyu Dubey, Otkrist Gupta, Pei Guo, Ramesh Raskar, Ryan Farrell and Nikhil Naik
European Conference on Computer Vision (ECCV), 2018
Abstract | Technical Report | Proceedings | Code

Fine-Grained Visual Classification (FGVC) datasets contain small sample sizes, along with significant intra-class variation and inter-class similarity. While prior work has addressed intra-class variation using localization and segmentation techniques, inter-class similarity may also affect feature learning and reduce classification performance. In this work, we address this problem using a novel optimization procedure for the end-to-end neural network training on FGVC tasks. Our procedure, called Pairwise Confusion (PC) reduces overfitting by intentionally {introducing confusion} in the activations. With PC regularization, we obtain state-of-the-art performance on six of the most widely-used FGVC datasets and demonstrate improved localization ability. {PC} is easy to implement, does not need excessive hyperparameter tuning during training, and does not add significant overhead during test time.
Hide Abstract

Coreset-Based Neural Network Compression
Abhimanyu Dubey, Moitreya Chatterjee and Narendra Ahuja
European Conference on Computer Vision (ECCV), 2018
Abstract | Technical Report | Proceedings | Code

We propose a novel convolutional neural network (CNN) compression algorithm based on coreset representations of filters. We exploit the redundancies extant in the space of CNN weights and neuronal activations (across samples) in order to obtain compression. Our method requires no retraining, is easy to implement, and obtains state-of-the-art compression performance across a wide variety of CNN architectures. Coupled with quantization and Huffman coding, we create networks that provide AlexNet-like accuracy, with a memory footprint that is 832Ă— smaller than the original AlexNet, while also introducing significant reductions in inference time as well. Additionally these compressed networks when fine-tuned, successfully generalize to other domains as well.
Hide Abstract

TuringBox: An Experimental Platform for the Evaluation of AI Systems
Ziv Epstein, Blakeley Hoffman Payne, Judy Hanwen Shen, Casey Jisoo Hong, Bjarke Felbo, Abhimanyu Dubey, Matthew Groh, Nick Obradovich, Manuel Cebrian and Iyad Rahwan
International Joint Conference on Artificial Intelligence (IJCAI), 2018
Abstract | Technical Report | Proceedings

We introduce TuringBox, a two-sided platform to study the behavior of artificial intelligence systems via empirical black-box testing. On one side of the platform, machine learning contributors upload existing and novel algorithms to be studied scientifically by others. On the other side, machine learning examiners develop and post machine intelligence tasks designed to evaluate and characterize the outputs of algorithms. We discuss the merits of black-box testing, outline the architecture of such a platform, and describe two interactive case studies of algorithmic auditing on the platform.
Hide Abstract

Sparse Matching for Embedding Image Macros
Abhimanyu Dubey, Esteban Moro, Manuel Cebrian and Iyad Rahwan
World Wide Web Conference (WWW), 2018
Abstract | Technical Report | Proceedings | Slides

The analysis of the creation, mutation, and propagation of social media content on the Internet is an essential problem in computational social science, affecting areas ranging from marketing to political mobilization. A first step towards understanding the evolution of images online is the analysis of rapidly modifying and propagating memetic imagery or `memes'. However, a pitfall in proceeding with such an investigation is the current incapability to produce a robust semantic space for such imagery, capable of understanding differences in Image Macros. In this study, we provide a first step in the systematic study of image evolution on the Internet, by proposing an algorithm based on sparse representations and deep learning to decouple various types of content in such images and produce a rich semantic embedding. We demonstrate the benefits of our approach on a variety of tasks pertaining to memes and Image Macros, such as image clustering, image retrieval, topic prediction and virality prediction, surpassing the existing methods on each. In addition to its utility on quantitative tasks, our method opens up the possibility of obtaining the first large-scale understanding of the evolution and propagation of memetic imagery.
Hide Abstract

Modeling Image Virality with Pairwise Spatial Transformer Networks
Abhimanyu Dubey and Sumeet Agarwal
ACM Multimedia Conference (MM), 2017
Abstract | Technical Report | Proceedings

The study of virality and information diffusion online is a topic gaining traction rapidly in the computational social sciences. Computer vision and social network analysis research have also focused on understanding the impact of content and information diffusion in making content viral, with prior approaches not performing significantly well as other traditional classification tasks. In this paper, we present a novel pairwise reformulation of the virality prediction problem as an attribute prediction task and develop a novel algorithm to model image virality on online media using a pairwise neural network. Our model provides significant insights into the features that are responsible for promoting virality and surpasses the existing state-of-the-art by a 12% average improvement in prediction. We also investigate the effect of external category supervision on relative attribute prediction and observe an increase in prediction accuracy for the same across several attribute learning datasets.
Hide Abstract

Deep Learning the City: Quantifying Global Urban Perception
Abhimanyu Dubey, Nikhil Naik, Devi Parikh, Ramesh Raskar and Cesar Hidalgo
European Conference on Computer Vision (ECCV), 2016
Abstract | Technical Report | Proceedings | Code

Computer vision methods that quantify the perception of urban environment are increasingly being used to study the relationship between a city's physical appearance and the behavior and health of its residents. Yet, the throughput of current methods is too limited to quantify the perception of cities across the world. To tackle this challenge, we introduce a new crowdsourced dataset containing 110,988 images from 56 cities, and 1,170,000 pairwise comparisons provided by 81,630 online volunteers along six perceptual attributes: safe, lively, boring, wealthy, depressing, and beautiful. Using this data, we train a Siamese-like convolutional neural architecture, which learns from a joint classification and ranking loss, to predict human judgments of pairwise image comparisons. Our results show that crowdsourcing combined with neural networks can produce urban perception data at the global scale.
Hide Abstract