CAOSS: Computational and Online Social Science

Friday, October 12, 2012

Altschul Auditorium
Columbia University
417 International Affairs Building
New York City

With an explosion of data on every aspect of our everyday existence—from what we buy, to where we travel, to who we know—we are able to observe human behavior with granularity largely thought impossible just a decade ago. The growth of such online activity has further facilitated the design of web-based experiments, enhancing both the scale and efficiency of traditional methods. Together these advances have created an unprecedented opportunity to address longstanding questions in the social sciences, ranging from sociology to political science to economics and beyond.

The inaugural 2012 workshop on Computational and Online Social Science (CAOSS, pronounced “chaos”) aims to bring together diverse research communities that work at the intersection of computer science and social sciences to build a lasting foundation for this emerging discipline.

As space is limited, we ask that you please complete the free registration form if you plan to attend.

Please contact us if you have any additional questions.

Steering Committee

CAOSS will be held at the Altschul Auditorium at Columbia University on October 12th, 2012. The workshop consists of 5 plenary talks by invited speakers along with shorter contributed talks and a poster session.

As space is limited, we ask that you please complete the free registration form if you plan to attend.

8:50am-9:00am Welcome
9:00am-9:50am Measuring and Propagating Influence in Networks
Sinan Aral, NYU Stern

Measuring influence and finding influential people in social networks is now all the rage. But, true estimates of influence are fraught with statistical difficulties which naïve scoring methods cannot address. So, how can we robustly measure influence and identify influentials in networks? Whether in the spread of disease, the diffusion of information, the propagation of social contagions, the effectiveness of viral marketing, or the magnitude of peer effects in a variety of settings, a key problem is understanding whether and when the statistical relationships we observe can be interpreted causally. Sinan will review what we know and where work might go with respect to identifying causal peer influence in social networks and the importance of causal inference for understanding the spread of products, political views and public health behaviors through society. He will provide examples from large scale observational and experimental studies in online social media networks and describe a new project to spread HIV testing using peer to peer influence and mobile messaging in South Africa, the subject of which is the basis for a new documentary film entitled "The Social Cure."

10:00am-10:50am Inferring Causality in Observational Data about Social Networks
David Jensen, University of Massachusetts Amherst

Over the past decade, realization has been growing about a fruitful synthesis between machine learning and social science. One area of particularly high potential is the connection between large data sets and the desire to understand the deep causal structure of social systems. Over the past several decades, computer scientists and others have developed theoretical infrastructure to formally express causal models and to reason about the connections between a given causal model and its observable consequences in data. This work has resulted in highly effective algorithms for learning causal models that are consistent with a given set of observational data, often allowing strong inferences about the direction and size of specific causal dependencies. Unfortunately, most of this theoretical infrastructure assumes that data records are statistically independent and identically distributed, although many of the most interesting social science problems concern interacting sets of heterogeneous people, places, and things. I will discuss recent progress in extending the formal theories of causal inference and discovery to data about the behavior of social systems. I will also identify several key challenges that remain unsolved.

11:00am-11:30am Break
11:30am-12:20pm How users evaluate each other in social media
Jure Leskovec, Stanford University

In a variety of domains, mechanisms for evaluation allow one user to say whether he or she trusts another user, or likes the content they produced, or wants to confer special levels of authority or responsibility on them. We investigate a number of fundamental ways in which user and item characteristics affect the evaluations in online settings. For example, evaluations are not unidimensional but include multiple aspects that all together contribute to user’s overall rating. We investigate methods for modeling attitudes and attributes from online reviews that help us better understand user’s individual preferences. We also examine how to create a composite description of evaluations that accurately reflects some type of cumulative opinion of a community. Natural applications of these investigations include predicting the evaluation outcomes based on user characteristics and to estimate the chance of a favorable overall evaluation from a group knowing only the attributes of the group's members, but not their expressed opinions.

12:30pm-1:45pm Lunch & Poster Session
Wein Hall
1:45pm-2:45pm Short talks
  • A caution against convenient data: assessing bias in psychological studies of online data
    Seth Flaxman, Carnegie Mellon University

    Social science research based on Twitter and other massive online datasets has taken off in recent years. A large collection of tweets is an exciting source of information with which to investigate basic psychological questions. Golder and Macy (2011) estimated diurnal and seasonal mood patterns across cultures using Twitter data. Even just a few years ago, Daniel Kahneman and his colleagues (see Krueger et al. 2009) needed to run costly and time-consuming lab experiments and telephone surveys to answer similar questions. But convenient datasets must be analyzed with care to avoid bias. Beyond the obvious question of representativeness, other factors may introduce bias. As a case study, we compare Golder and Macy (2011)’s estimates of diurnal mood patterns to patterns we estimate from a novel experience sampling (ES) dataset from paid study participants (n = 2041) collected through an iPhone application. The most notable discrepancy between the two datasets is that the Twitter dataset shows mood as a “U”-shape, with good mood in the morning deteriorating to a valley from noon until 5pm, and then increasing until midnight. In contrast, the ES dataset shows the opposite, an inverted “U”-shape, with the lowest mood at waking, increasing to a maximum plateau at noon, and then decreasing until midnight, a pattern much more in line with previous research. We investigate this discrepancy and discuss various possible explanations for the striking contrast. Using ES data on concurrent self-reports of activities and emotions we estimate a basic confounding variable, asking whether mood is independent of the likelihood of a user to use a social networking site. We also look for sub-populations of our dataset that resemble the Twitter users in diurnal patterns, and investigate the reliability of sentiment analysis on tweets. While datasets like Twitter are appealing, properly avoiding bias is a challenge. Meanwhile, classically designed studies can achieve much larger sample sizes than in the past because of the widespread availability of smartphones. Fortunately, this is not an either/or proposition: methods for big datasets are helpful in analyzing the ES dataset we discuss, and the ES dataset gives insight into sources of bias in the Twitter dataset.

  • An internet experiment on bargaining in networks
    Yashodhan Kanoria, Microsoft Research

    Exchange networks model the behavior of a set of players who need to reach pairwise agreements for mutual benefit, as in the labor market, the housing market and the 'market' for social relationships. We describe internet-based experiments on bargaining in networks, that are the largest such experiments to date. Our results include some of the first insights into the dynamics of bargaining.

  • Giving cascades in crowd-funded marketplaces
    Rem Koning & Jacob Model, Stanford University

    While past research clearly demonstrates how social influence leads to the emergence of self-fulfilling prophecies in cultural markets it remains an open question if, and to what degree, this process occurs in other market types. Using a web-based natural field experiment on the philanthropic fundraising platform DonorsChoose.Org we are able to causally identify the effects of social influence in crowdfunding markets. Specifically we find that self-reinforcing cycles do occur within crowdfunded marketplaces but that not all forms of social influence have an equal impact on the behavior of subsequent donors. Beyond demonstrating the role of social influence in non-cultural markets our work sheds light on the under-studied dynamics of the rapidly growing crowdfunding industry.

  • Testing Behavioural Economics in the Wild with online and eCommerce Data
    Debajyoti Ray, California Institute of Technology

    The explosion of data from eCommerce and social media provides an unprecedented opportunity to empirically test theories in Behavioural Economics. In this short talk, I’ll discuss a couple of theories that have been well-documented in lab and field experiments, namely Prospect Theory and Projection Bias, and present some of our recent results from eCommerce datasets: 1) Using a large eCommerce transactions dataset, we test predictions of Prospect theory: the propensity of consumers to be loss-averse with respect to a reference point. As opposed to standard economic theory explanations such as stockpiling, prospect theory predicts that consumers’ purchases of substitute products would be influenced by whether or not an item is on sale. 2) Projection bias is the propensity for consumers to make errors in predicting their future consumption, when influenced by current, temporary events such as weather and social trends. We test this effect using local weather data at the time of purchases of durable goods in our eCommerce dataset. Theoretical models from Behavioural Economics provide a principled framework for analyzing and interpreting human behavior from massive datasets. The empirical estimation of these effects has real-world economic prescriptions, for the pricing and promotions strategies of eCommerce retailers.

  • Improving Spatial Models of Political Ideology by Incorporating Social Network Data
    John Myles White, Princeton University

    Social scientists are now debating the usefulness of Big Data for the future of their field(s). In our recent work, we have developed a proof of concept study of one way in which novel sources of data about human behavior might enrich social science research: we exploit social network data gathered from Twitter to improve our capacity to measure the political ideology of the Members of the 110th-112th sessions of the United States Congress. We demonstrate that we can extend the Ideal Points model of political ideology, which traditionally uses only voting records, by incorporating data from social networks into our analysis. This extension allows us to estimate the ideology of members of both Houses of the United States Congress on a common scale even though the voting records for the two Houses are strictly disjoint. We argue that multimodal modeling approaches, in which heterogenous data sets are analyzed in aggregate, provide one mechanism for the social sciences to leverage the kinds of large-scale, naturalistic data being generated by the Internet’s ubiquitous data generating processes.

  • The Groupon Effect on Yelp Ratings: A Root Cause Analysis
    Georgios Zervas, Yale University

    Daily deals sites such as Groupon offer deeply discounted goods and services to tens of millions of customers through geographically targeted daily e-mail marketing campaigns. In our prior work we observed that a negative side effect for merchants using Groupons is that, on average, their Yelp ratings decline significantly. However, this previous work was essentially observational, rather than explanatory. In this work, we rigorously consider and evaluate various hypotheses about underlying consumer and merchant behavior in order to understand this phenomenon, which we dub the Groupon effect. We use statistical analysis and mathematical modeling, leveraging a dataset we collected spanning tens of thousands of daily deals and over 7 million Yelp reviews. In particular, we investigate hypotheses such as whether Groupon subscribers are more critical than their peers, or whether some fraction of Groupon merchants provide significantly worse service to customers using Groupons. We suggest an additional novel hypothesis: reviews from Groupon subscribers are lower on average because such reviews correspond to real, unbiased customers, while the body of reviews on Yelp contain some fraction of reviews from biased or even potentially fake sources. Although we focus on a specific question, our work provides broad insights into both consumer and merchant behavior within the daily deals marketplace.

2:45pm-3:00pm Break
3:00pm-3:50pm "Which Half is Wasted?": Controlled Experiments to Measure Online-Advertising Effectiveness
David Reiley, Google

The department-store retailer John Wanamaker famously stated, “Half the money I spend on advertising is wasted—I just don’t know which half.” Compared with the measurement of advertising effectiveness in traditional media, online advertisers and publishers have considerable data advantages, including individual-level data on advertising exposures, clicks, searches, and other online user behaviors. However, as I shall discuss in this talk, the science of advertising effectiveness requires more than just quantity of data - even more important is the quality of the data. In particular, in many cases, using various statistical techniques with observational data leads to incorrect measurements. To measure the true causal effects, we run controlled experiments that suppress advertising to a control group, much like the placebo in a drug trial. With experiments to determine the ground truth, we can show that in many circumstances, observational-data techniques rely on identifying assumptions that prove to be incorrect, and they produce estimates differing wildly from the truth. Despite increases in data availability, Wanamaker's complaint remains just as true for online advertising as it was for print advertising a century ago.

In this talk, I will discuss recent advances in running randomized experiments online, measuring the impact of online display advertising on consumer behavior. Interesting results include the measurable effects of online advertising on offline transactions, the impact on viewers who do not click the ads, the surprisingly large effects of frequency of exposure, and the heterogeneity of advertising effectiveness across users in different demographic groups or geographic locations. I also show that sample sizes of a million or more customers may be necessary to get enough precision for statistical significance of economically important effects - so we have just reached the cusp of being able to measure effects precisely with present technology. (By comparison, previous controlled experiments using split-cable TV systems, with sample sizes in the mere thousands, have lacked statistical power to measure precise effects for a given campaign.) As I show with several examples that establish the ground truth using controlled experiments, the bias in observational studies can be extremely large, over-or-underestimating the true causal effects by an order of magnitude. I will discuss the (implicit or explicit) modeling assumptions made by researchers using observational data, and identify several reasons why these assumptions are violated in practice. I will also discuss future directions in using experiments to measure advertising effectiveness.

4:00pm-4:50pm The Virtual Lab
Duncan Watts, Microsoft Research

Crowdsourcing sites like Amazon's Mechanical Turk are increasingly being used by researchers to construct "virtual labs" in which they can conduct behavioral experiments. In this talk, I describe some recent experiments that showcase the advantages of virtual over traditional physical labs, as well as some of the limitations. I then discuss how this relatively new experimental capability may unfold in the near future, along with some implications for social and behavioral science.