This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Saar-Tsechansky, M. Articles by Provost, F. Search for Related Content

Decision-Centric Active Learning of Binary-Outcome Models

Red McCombs School of Business, University of Texas at Austin, Austin, Texas 78712
Leonard N. Stern School of Business, New York University, 44 West Fourth Street, New York, New York 10012
maytal{at}mail.utexas.edu
fprovost{at}stern.nyu.edu

It can be expensive to acquire the data required for businesses to employ data-driven predictive modeling—for example, to model consumer preferences to optimize targeting. Prior research has introduced "active-learning" policies for identifying data that are particularly useful for model induction, with the goal of decreasing the statistical error for a given acquisition cost (error-centric approaches). However, predictive models are used as part of a decision-making process, and costly improvements in model accuracy do not always result in better decisions. This paper introduces a new approach for active data acquisition that specifically targets decision making. The new decision-centric approach departs from traditional active learning by placing emphasis on acquisitions that are more likely to affect decision making. We describe two different types of decision-centric techniques. Next, using direct-marketing data, we compare various data-acquisition techniques. We demonstrate that strategies for reducing statistical error can be wasteful in a decision-making context, and show that one decision-centric technique in particular can improve targeting decisions significantly. We also show that this method is robust in the face of decreasing quality of utility estimations, eventually converging to uniform random sampling, and that it can be extended to situations where different data acquisitions have different costs. The results suggest that businesses should consider modifying their strategies for acquiring information through normal business transactions. For example, a firm such as Amazon.com that models consumer preferences for customized marketing may accelerate learning by proactively offering recommendations—not merely to induce immediate sales, but for improving recommendations in the future.

Key Words: decision-support systems; active learning; classifier induction; decision making; predictive modeling
History: This paper was received on April 11, 2005.

This article has been cited by other articles:

				A. Bensoussan, R. Mookerjee, V. Mookerjee, and W. T. Yue Maintaining Diagnostic Knowledge-Based Systems: A Control-Theoretic Approach Management Science, February 1, 2009; 55(2): 294 - 310. [Abstract] [PDF]