From optimizing engagement to measuring value

FAccT

Most recommendation engines today are based on predicting user engagement, e.g., predicting whether a user will click on an item or not. However, there is potentially a large gap between engagement signals and the desired notion of "value" worth optimizing for. 

Authors:
Smitha Milli, Luca Belli, Moritz Hardt

Publication date: 7/30/2021

Tuning Word2vec for Large Scale Recommendation Systems

RecSys

Word2vec is a powerful machine learning tool that emerged from Natural Language Processing (NLP) and is now applied in multiple domains, including recommender systems, forecasting, and network analysis. As Word2vec is often used off the shelf, we address the question of whether the default hyperparameters are suitable for recommender systems.

Authors:
Benjamin P. Chamberlain, Emanuele Rossi, Dan Shiebler, Suvash Sedhain, Michael M. Bronstein

Publication date: 2020

Deep Bayesian Bandits: Exploring in Online Personalized Recommendations

RecSys

In this work, we formulate a display advertising recommender as a contextual bandit and implement exploration techniques that require sampling from the posterior distribution of click-through-rates in a computationally tractable manner.

Authors:
Dalin Guo, Sofia Ira Ktena, Pranay Kumar Myana, Ferenc Huszár, Wenzhe Shi, Alykhan Tejani, Michael Kneier, Sourav Das

Publication date: 2020

Model Size Reduction Using Frequency Based Double Hashing for Recommender Systems

RecSys

In this paper, we propose a hybrid hashing method to combine frequency hashing and double hashing techniques for model size reduction, without compromising performance.

Authors:
Caojin Zhang, Yicun Liu, Yuanpu Xie, Sofia Ira Ktena, Alykhan Tejani, Akshay Gupta, Pranay Kumar Myana, Deepak Dilipkumar, Suvadip Paul, Ikuhiro Ihara, Prasang Upadhyaya, Ferenc Huszár, Wenzhe Shi

Publication date: 2020

Doubly Robust Joint Learning for Recommendation on Data Missing Not at Random

ICML

First, we propose an estimator that integrates the imputed errors and propensities in a doubly robust way to obtain unbiased performance estimation and alleviate the effect of the propensity variance. Based on this estimator, we propose joint learning of rating prediction and error imputation to achieve good performance guarantees. 

Authors:
Xiaojie Wang, Rui Zhang, Yu Sun, Jianzhong Qi

Publication date: 2019

A Correlation Maximization Approach for Cross Domain Co-Embeddings

arXiv

In this paper, we introduce ImplicitCE, an algorithm for recommending items to new users during their sign-up flow. 

Authors:
Dan Shiebler

Publication date: 2018

RecService: Distributed Real-Time Graph Processing at Twitter-Embeddings

HotCloud

We present RecService, a distributed real-time graph processing engine that drives billions of recommendations on Twitter. 

Authors:
Ajeet Grewal, Jerry Jiang, Gary Lam, Tristan Jung, Lohith Vuddemarri, Quannan Li, Aaditya Landge, Jimmy Lin

Publication date: 2018

A Meta-Learning Perspective on Cold-Start Recommendations for Items

NeurIPS

In this paper, we present a meta-learning strategy to address item cold-start when new items arrive continuously. We propose two deep neural network architectures that implement our meta-learning strategy.

Authors:
Manasi Vartak, Arvind Thiagarajan, Conrado Miranda, Jeshua Bratman, Hugo Larochelle

Publication date: 2017

GraphJet: Real-Time Content Recommendations at Twitter

HotCloud

This paper presents GraphJet, a new graph-based system for generating content recommendations at Twitter.

Authors:
Aneesh Sharma, Jerry Jiang, Praveen Bommannavar, Brian Larson, Jimmy Lin

Publication date: 2016

The Effect of Recommendations on Network Structure

WWW

Here we examine the aggregate effects of such recommendations on network structure, focusing on whether these recommendations increase the popularity of niche users or, conversely, those who are already popular. 

Authors:
Jessica Su, Aneesh Sharma, Sharad Goel

Publication date: 2016

Real-time twitter recommendation: online motif detection in large dynamic graphs

VLDB Endorsement

We describe a production Twitter system for generating relevant, personalized, and timely recommendations based on observing the temporally-correlated actions of each user’s followings. The system currently serves millions of recommendations daily to tens of millions of mobile users.

Authors:
Pankaj Gupta, Venu Satuluri, Ajeet Grewal, Siva Gurumurthy, Volodymyr Zhabiuk, Quannan Li, Jimmy Lin

Publication date: 2014

An Experimental Study of Structural Diversity in Social Networks

ICWSM

We investigate the role of structural diversity on retention by conducting a large-scale randomized controlled study on the Twitter platform. 

Authors:
Jessica Su, Krishna Kamath, Aneesh Sharma, Johan Ugander, Sharad Goel

Publication date: 2019

What is the Value of Experimentation and Measurement?

ICDM

We tackle this problem by analyzing how, by decreasing estimation uncertainty, E&M platforms allow for better prioritization. We quantify this benefit in terms of expected relative improvement in the performance of all new propositions and provide guidance for how much an E&M capability is worth and when organizations should invest in one.

Authors:
Bryan Liu, Ben Chamberlain

Publication date: 2019

Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction

RecSys

The focus of this paper is to identify the best combination of loss functions and models that enable large-scale learning from a continuous stream of data in the presence of delayed labels. In this work, we compare 5 different loss functions, 3 of them applied to this problem for the first time.

Authors:
Sofia Ira Ktena, Alykhan Tejani, Lucas Theis, Pranay Kumar Myana, Deepak Dilipkumar, Ferenc Huszar, Steven Yoo, Wenzhe Shi

Publication date: 2019

Detecting Strong Ties Using Network Motifs

WWW

In this work, we demonstrate via experiments on Twitter data that using only such structural network features is sufficient for detecting strong ties with high precision. These structural network features are obtained from the presence and frequency of small network motifs on combined strong and weak ties.

Authors:
Rahmtin Rotabi, Krishna Kamath, Jon Kleinberg, Aneesh Sharma

Publication date: 2017

Cascades: a view from audience

WWW

Cascades on social and information networks have been a tremendously popular subject of study in the past decade, and there is a considerable literature on phenomena such as diffusion mechanisms, virality, cascade prediction, and peer network effects. Against the backdrop of this research, a basic question has received comparatively little attention: how desirable are cascades on a social media platform from the point of view of users?

Authors:
Rahmtin Rotabi, Krishna Kamath, Jon Kleinberg, Aneesh Sharma

Publication date: 2017

Understanding over-squashing and bottlenecks on graphs via curvature

ICLR

We provide a precise description of the over-squashing phenomenon in GNNs and analyze how it arises from bottlenecks in the graph.

Authors:

Jake Topping, Francesco Di Giovanni, Benjamin Paul Chamberlain, Xiaowen Dong, Michael M. Bronstein

Publication date: 3/16/2022

GRAND: Graph Neural Diffusion

ICML

We present Graph Neural Diffusion (GRAND) that approaches deep learning on graphs as a continuous diffusion process and treats Graph Neural Networks (GNNs) as discretisations of an underlying PDE.

Authors:
Ben Chamberlain, James Rowbottom, Maria Gorinova, Stefan Webb, Emanuele Rossi, Michael M. Bronstein

Publication date: 7/30/2021

Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks

ICML

The pairwise interaction paradigm of graph machine learning has predominantly governed the modelling of relational systems. However, graphs alone cannot capture the multi-level interactions present in many complex systems and the expressive power of such schemes was proven to be limited. 

Authors:
Cristian Bodnar, Fabrizio Frasca, Yu Guang Wang, Nina Otter, Guido Montúfar, Pietro Liò, Michael Bronstein

Publication date: 7/30/2021

Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020

PMLR

This paper presents the results and insights from the black-box optimization (BBO) challenge at NeurIPS 2020 which ran from July-October, 2020. 

Authors:
Ryan Turner, David Eriksson, Michael McCourt, Juha Kiili, Eero Laaksonen, Zhen Xu, Isabelle Guyon

Publication date: 7/30/2021

Geometric Deep Learning

Book Preview

While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications.. 

Authors:
Michael Bronstein, Joan Bruna, Taco Cohen, Petar Veličković

Publication date: 7/30/2021

Fast geometric learning with symbolic matrices

NeurIPS

Geometric methods rely on tensors that can be encoded using a symbolic formula and data arrays, such as kernel and distance matrices. We present an extension for standard machine learning frameworks that provides comprehensive support for this abstraction on CPUs and GPUs: our toolbox combines a versatile, transparent user interface with fast runtimes and low memory usage. 

Authors:
Jean Feydy, Joan Alexis Glaunès, Benjamin Charlier, Michael M Bronstein

Publication date: 2020

SIGN scalable graph neural network architecture

ICML GRL+ Workshop, WWW

In this paper we propose a new, efficient and scalable graph deep learning architecture which sidesteps the need for graph sampling by using graph convolutional filters of different size that are amenable to efficient precomputation, allowing extremely fast training and inference.

Authors:
Fabrizio Frasca, Emanuele Rossi, Davide Eynard, Ben Chamberlain, Michael Bronstein, Federico Monti

Publication date: 2020

Temporal Graph Networks for deep learning on dynamic graphs

WWW

In this paper, we present Temporal Graph Networks (TGNs), a generic, efficient framework for deep learning on dynamic graphs represented as sequences of timed events. 

Authors:
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, Michael Bronstein

Publication date: 2020

Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting

arXiv

We propose "Graph Substructure Networks" (GSN), a topologically-aware message passing scheme based on substructure encoding. We theoretically analyse the expressive power of our architecture, showing that it is strictly more expressive than the WL test, and provide sufficient conditions for universality. 

Authors:
Giorgos Bouritsas, Fabrizio Frasca, Stefanos Zafeiriou, Michael M. Bronstein

Publication date: 2020

Differentiable Graph Module

arXiv

In this paper, we introduce Differentiable Graph Module (DGM), a learnable function predicting the edge probability in the graph relevant for the task, that can be combined with convolutional graph neural network layers and trained in an end-to-end fashion.

Authors:
Anees Kazi, Luca Cosmo, Nassir Navab, Michael Bronstein

Publication date: 2020

Transferability of Spectral Graph Convolutional Neural Networks

arXiv

This paper focuses on spectral graph convolutional neural networks (ConvNets), where filters are defined as elementwise multiplication in the frequency domain of a graph.

Authors:
Ron Levie, Michael M. Bronstein, Gitta Kutyniok

Publication date: 2019

Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation

ICCV

In this paper, we focus on 3D deformable shapes that share a common topological structure, such as human faces and bodies. 

Authors:
Giorgos Bouritsas, Sergiy Bokhnyak, Stylianos Ploumpis, Michael M. Bronstein, Stefanos Zafeiriou

Publication date: 2019

Single Image 3D Hand Reconstruction with Mesh Convolutions

BMVC

We evaluate the quality of the mesh reconstructions produced by the decoder on a new dataset and show latent space interpolation results.

Authors:
Dominik Kulon, Haoyang Wang, Riza Alp Güler, Michael M. Bronstein, Stefanos Zafeiriou

Publication date: 2019

ncRNA Classification with Graph Convolutional Networks

KDD DGL Workshop

We improve on the stateof-the-art for this task with a graph convolutional network model which achieves an accuracy of 85.73% and an F1-score of 85.61% over 13 classes. 

Authors:
Emanuele Rossi, Federico Monti, Pietro Liò, Michael Bronstein

Publication date: 2019

Fake News Detection on Social Media using Geometric Deep Learning

ICLR RLG Workshop

In this paper, we show a novel automatic fake news detection model based on geometric deep learning.

Authors:
Federico Monti, Fabrizio Frasca, Davide Eynard, Damon Mannion, Michael Bronstein

Publication date: 2019

Faster gaze prediction with dense networks and Fisher pruning

arXiv

We first present a simple yet principled greedy pruning method which we call Fisher pruning. Through a combination of knowledge distillation and Fisher pruning, we obtain much more runtime-efficient architectures for saliency prediction, achieving a 10x speedup for the same AUC performance as a state of the art network on the CAT2000 dataset.

Authors:
Lucas Theis, Iryna Korshunova, Alykhan Tejani, Ferenc Huszár

Publication date: 2018

Frame Interpolation with Multi-Scale Deep Loss Functions and Generative Adversarial Networks

arXiv

To maximise the efficiency of our network, we propose a novel multiscale residual estimation module where the predicted flow and synthesised frame are constructed in a coarse-to-fine fashion.

Authors:
Joost van Amersfoort, Wenzhe Shi, Alejandro Acosta, Francisco Massa, Johannes Totz, Zehan Wang, Jose Caballero

Publication date: 2018

KDGAN: Knowledge Distillation with Generative Adversarial Networks

NeurIPS

We propose a three-player game named KDGAN consisting of a classifier, a teacher, and a discriminator. The classifier and the teacher learn from each other via distillation losses and are adversarially trained against the discriminator via adversarial losses. 

Authors:
Xiaojie Wang, Rui Zhang, Yu Sun, Jianzhong Qi

Publication date: 2018

Realgraph: User interaction prediction at Twitter

User Engagement Optimization

In this work, we provide a framework to compute relationship strength for ties based on directed interactions between users. The proposed framework, called RealGraph, produces a directed and weighted graph where the nodes are Twitter users, and the edges are labeled with interactions between a directed pair of users.

Authors:
Krishna Kamath, Aneesh Sharma, Dong Wang, Zhijun Yin

Publication date: 2014

Algorithmic Amplification of Politics on Twitter

Our results reveal a remarkably consistent trend: In 6 out of 7 countries studied, the mainstream political right enjoys higher algorithmic amplification than the mainstream political left. Consistent with this overall trend, our second set of findings studying the U.S. media landscape revealed that algorithmic amplification favours right-leaning news sources.

Authors:

Ferenc Huszár, Sofia Ira Ktena, Conor O’Brien, Luca Belli, Andrew Schlaikjera, and Moritz Hardt

Publication date: October 2021

Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency

CSWM

Twitter uses machine learning to crop images, where crops are centered around the part predicted to be the most salient. In fall 2020, Twitter users raised concerns that the automated image cropping system on Twitter favored light-skinned over dark-skinned individuals, as well as concerns that the system favored cropping woman's bodies instead of their heads. In order to address these concerns, we conduct an extensive analysis using formalized group fairness metrics.

Authors:
Kyra Yee, Uthaipon Tantipongpipat, Shubhanshu Mishra

Paper received an Impact Recognition award at CSCW 2021.

Publication date: 5/18/2021

Lessons Learned Addressing Dataset Bias in Model-Based Candidate Generation at Twitter

ISR-KDD workshop

We first explore the dynamics of the dataset bias problem and then demonstrate how to use random sampling techniques to mitigate it. Finally, in a novel application of fine-tuning, we show performance gains when applying our candidate generation system to Twitter's home timeline.

Authors:
Alim Virani, Jay Baxter, Dan Shiebler, Philip Gautier, Shivam Verma, Yan Xia, Apoorv Sharma, Sumit Binnani, Linlin Chen, Chenguang Yu

Publication date: 2020

Assessing Demographic Bias in Named Entity Recognition

KG-BIAS workshop

In this work, we assess the bias in various Named Entity Recognition (NER) systems for English across different demographic groups with synthetically generated corpora

Authors:
Shubhanshu Mishra, Sijun He, Luca Belli

Publication date: 2020

Isotonic Regression Adjustment for Variance Reduction

CODE@MIT

This paper presents a novel approach for variance reduction in A/B experiments that is based on isotonic regression, which works as well as more complicated methods but is computationally much more efficient.

Authors:
Ryan Turner, Umashanthi Pavalanathan, Stefan Webb, Nils Hammerla, Brent Cohn, Anthony Fu

Publication date: 5/11/2021

Context-Uncertainty-Aware Chatbot Action Selection via Parameterized Auxiliary Reinforcement Learning

PAKDD

We propose a context-uncertainty-aware chatbot and a reinforcement learning (RL) model to train the chatbot. The proposed model is named Parameterized Auxiliary Asynchronous Advantage Actor Critic (PA4C)

Authors:
Chuandong Yin, Rui Zhang, Jianzhong Qi, Yu Sun, Tenglun Tan

Publication date: 2018

Lossy Image Compression with Compressive Autoencoders

ICLR RLG Workshop

We propose a new approach to the problem of optimizing autoencoders for lossy image compression. 

Authors:
Lucas Theis, Wenzhe Shi, Andrew Cunningham, Ferenc Huszár

Publication date: 2017

Fast Face-swap Using Convolutional Neural Networks

ICCV

By combining neural networks with simple pre- and post-processing steps, we aim at making face swap work in real-time with no input from the user. 

Authors:
Iryna Korshunova, Wenzhe Shi, Joni Dambre, Lucas Theis

Publication date: 2017

Variational Inference using Implicit Distributions

arXiv

This paper provides a unifying review of existing algorithms establishing connections between variational autoencoders, adversarially learned inference, operator VI, GAN-based image reconstruction, and more. Secondly, the paper provides a framework for building new algorithms: depending on the way the variational bound is expressed we introduce prior-contrastive and jointcontrastive methods, and show practical inference algorithms based on either density ratio estimation or denoising.

Authors:
Ferenc Huszár

Publication date: 2017

Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resize

arXiv

A note on sub-pixel convolution, resize convolution and convolution resize

Authors:
Andrew Aitken, Christian Ledig, Lucas Theis, Jose Caballero, Zehan Wang, Wenzhe Shi

Publication date: 2017

Amortised MAP Inference for Image Super-resolution

ICLR

We propose three methods to solve this optimisation problem: (1) Generative Adversarial Networks (GAN) (2) denoiser-guided SR which backpropagates gradient-estimates from denoising to train the network, and (3) a baseline method using a maximum-likelihoodtrained image prior. Our experiments show that the GAN based approach performs best on real image data. Lastly, we establish a connection between GANs and amortised variational inference as in e. g. variational autoencoders.

Authors:
Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi, Ferenc Huszár

Publication date: 2017

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

CVPR

In this paper, we present SRGAN, a generative adversarial network (GAN) for image superresolution (SR). To our knowledge, it is the first framework capable of inferring photo-realistic natural images for 4× upscaling factors. 

Authors:
Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, Wenzhe Shi

Publication date: 2017

Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation

CVPR

In this paper, we introduce spatio-temporal subpixel convolution networks that effectively exploit temporal redundancies and improve reconstruction accuracy while maintaining real-time speed. Specifically, we discuss the use of early fusion, slow fusion and 3D convolutions for the joint processing of multiple consecutive video frames. We also propose a novel joint motion compensation and video super-resolution algorithm that is orders of magnitude more efficient than competing methods, relying on a fast multi-resolution spatial transformer module that is endto-end trainable. Authors:

Jose Caballero, Christian Ledig, Andrew Aitken, Alejandro Acosta, Johannes Totz, Zehan Wang, Wenzhe Shi

Publication date: 2017

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

CVPR

In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space.

Authors:
Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, Zehan Wang

Publication date: 2016