← volver

Charla: «Effects of Interference on Marketplace Experiments and Decision-Making», Gabriel Weintraub, Stanford University

La charla se desarrollará en SALA DE CONSEJO PISO 4, BEAUCHEF 851, SANTIAGO

INSCRIPCION AQUI

Abstract

Platforms rely on experiments (A/B tests) to aid decision-making. However, prior work has shown that
in marketplace experiments, interactions between users can create interference effects that lead to biased
estimates of the treatment effect. We develop mathematical models to capture these interference
effects and study the biases that arise. We show that the magnitude of the treatment effect bias depends on
the level of supply and demand imbalance in the platform. Building on these insights, we propose a novel
class of experimental designs using “two-sided randomization” (TSR) that reduces bias across wide
ranges of market imbalance.
We then consider the effect of interference on the resulting platform decisions. We show that a second
type of bias arises that also impacts decisions.
Specifically, interference also leads to biased estimates of the standard error, which can result in confidence
intervals that are too wide or too narrow, causing the platform to be under or over-confident in their
decisions. We show that there are interaction effects between the standard error and treatment effect biases
that impact the quality of decisions. Finally, we outline future work to assess the impacts on decisions made in
the Airbnb marketplace.