Anthropology and Big Data

The following is a slightly edited cross-posting from anthrodesign.

As I read this thread, I find myself wondering if some of the critical remarks about big data aren't a bit behind the curve. I am thinking in particular of Sam's remark that,

Without some working theory of what social behaviour is, we have nothing but reams of meaningless data.

There is, of course, a sense in which this statement is obviously true. I suspect, however, that those who read it will imagine some degree of theoretical sophistication that only an anthropologist could provide. The fact of the matter is that relatively commonplace rules of thumb may result in delivering the value that businesses are looking for. One example is the proposition that someone who has already purchased a product is more likely to purchase one similar to it than a randomly selected prospect. This is the basic rule behind Amazon's you-might-also-be-interested-in suggestions. It is also the basic rule of thumb for political and charitable fundraisers, the reason why one donation leads to solicitations for repeated or similar donations.

I recall a bit of history. Back in the early 1990s, I was working on the Coca-Cola account for a Japanese advertising agency. I recall talking to someone about how the business was changing. Then the issue was data from point-of-sale (POS) systems. In "the good old days," the agency would pitch the client using research that was often conducted months, even years, before Coca-Cola made marketing decisions for the coming fiscal year. Then, when Coca-Cola had made its decisions, a big annual meeting was held at which Coca-Cola revealed its marketing plans to the bottlers. The plans would be executed starting a few weeks later. POS data created a totally different world. Convenience store (CVS) chains, in particular, were now able to track product sales day-by-day, and data collected months or years previously was hopelessly out of date. Suddenly the world in which researchers developed complex theories, basically good stories, to sell annual marketing strategies was a thing of the past. Now everybody, at the bottlers, at Coca-Cola, at the agency, was scrambling to keep up with the latest POS data and agile response became more important than carefully constructed theories.

Now the Internet has shifted data collection from day-by-day through POS systems to second-by-second data collected on every online transaction. And that isn't all that has changed. In Bursts: The Hidden Pattern Behind Everything We Do, Albert-Laszlo Barabasi suggests that the proposition that people are predictable en masse but not as individuals is, in fact, nearly totally backwards. As individuals we are mostly creatures of habit, and now that data on our habits can be collected on an individual level, we are very highly predictable in what we do. Stories that explain the means and medians revealed by normal curves are pointless in a world where the behavior or people on the tails of the curve can be predicted as easily and precisely as that of the "average" person.

Does this mean that theory is dead? Perhaps not. But theories developed for the world in which the aim was to explain the norm, and the norm was the average of some population? They are increasingly dead in the water. One suspects that Tom Kelley has it right in The Ten Faces of Innovation when he describes the anthropologist as a person of infinite curiosity and meticulous note taking who notices things that other people miss and says nothing at all about theory. The question is whether the anthropologist will be equipped to notice things that other people miss IN big data or will be left standing on the sidelines moaning about the fact that what we used to think was valuable isn't seen that way any more.

Might be worth thinking about.

Comment

You need to be a member of Open Anthropology Cooperative to add comments!

Comment by John McCreery on October 25, 2012 at 1:01am

Jason, I am looking forward to that white paper. FYI, my current research involves the following steps. The topic is the world of top advertising creatives in Tokyo. I am

Starting with archival data—credits from the annuals that document the finalists and winners of an advertising contest that has been held every year since 1963. At the moment, I am working on data from six contests (1981, 1986, 1991, 1996, 2001, 2006). Data from 2011 is now being added to a database that already includes over 4000 ads and 9000 creators linked by 34000+ roles (copywriter, creative director, etc.)
Using results from the social network analysis to focus desk research—reading what has been written by and about key figures in the networks (at this point the top 0.046%). Lots to do here since the annuals from which the network data are taken are only a fraction of the output of an active trade press that covers the advertising industry in Japan.
Using previous industry connections to arrange interviews with key figures, to show them what I have discovered and hear what they have to say about it.

In sum the basic design is Network Analysis > Historical Research>Ethnographic Interviews. Plus, of course, the personal perspective I bring to the project from going on three decades working in and around the industry in question and the opportunity to combine interpretive insights with quantitative correlations between the output from the Network Analysis and other statistical information, e.g., fluctuations in GDP and ad spend broken down by medium, that sort of thing.

Should you want to learn more, I have put presentations about the project up on SlideShare. Any feedback you might be able to offer will be much appreciated.

Comment by Jason Hare on October 24, 2012 at 7:45pm: I am currently working on a white paper detailing an approach to Open Data policies from a social sciences perspective. In preparation for this I have gathered some information on the resurgence of cultural anthropology and some of the data modeling that is going on within the anthropology community. The white paper will also detail the plan for the City of Raleigh's Open Data program to effectively mine data and analyze patterns to create visualizations for public consumption (not info-graphics). From that we may be able to recognize patterns of behavior in response to city programs. We can at least start to tackle issues when we see programs fail and emulate those that are successful. Thank you for your patience.

Comment by Keith Hart on October 18, 2012 at 9:21pm: By all means do that whenever, Jason and I will feature it.

Comment by Jason Hare on October 18, 2012 at 5:51pm: Will do John- I am hosted a hackathon Saturday in Durham NC where we will be discussing inferring user behavior from Google Analytics data- we have millions if interactions to use for pattering over time. Perhaps I can post a blog article that breaks up some of my comments into some assertions supported from the data we have collected?

Comment by John McCreery on October 18, 2012 at 4:14pm: Jason, please do follow up with that more complete analysis you mention. I look forward to reading it.

Comment by Jason Hare on October 18, 2012 at 3:11pm: While I hear the arguments on both sides Big Data can show a shift in behavior and this can affect design choices as it relates to the whole user experience. Folks are correct in asserting that a lot of what was important to anthropologists in the past may not be seen as that important today. I think though we have a very important contribution to make in understanding big data and in the task of implementing open data and creating transparency in government.

Back to big data- in analyzing seven years of click tracking data for a public sector website, i noticed a shift in the use of referral traffic to a specific page within this website versus a browse from the homepage starting point. Indeed when I did a visitor path analysis i noticed over seven years a precipitous drop in "browse from the homepage" versus landing on a particular page from a Google search. Several other municipal web managers noticed this. We managers coined this a "drop in assurance level" on the part of web visitors. What we believe the data is telling us is that the user experience no longer started from the home page of a destination site but rather starts with a search engine results page. based on this conclusion (I can post the data sets that led to these conclusions- not just one aspect) municipal sites changed their design to be more search-centric. I would like to write a blog post to explain more of what I asserted here in this response.

To summarize, anthropologists are needed to make sense of the behavior patterns that can be mined from Big Data. Anthropologists are needed to understand the culture of an organization that is about to implement open data programs and anthropologists are needed to understand how citizens affected by both Big Data and Open data view themselves in relation to their government and to their sense of place. I will be happy to followup with a more complete analysis. Thank you for letting me respond with my few brief comments and assertions.

Comment by John McCreery on June 20, 2012 at 5:09am: Thanks, Huon. Personally, I'm not yet ready to write off old-fashioned American four-fields or British social anthropology. It seems to me that once we give up our literally blasphemous desire to know everything, all at once, and focus on more modest aims, there is lot still to be done, trying to identify and explain why relatively stable aspects of custom, habit, belief, ritual, etc., take the forms they do, are resistant to change, and sometimes change dramatically.

But, returning to big data, here is my latest contribution to the Anthrodesign thread with which this topic began.

>>>>

I don't know about the rest of us, but for me this has been an extremely interesting thread. Makes me realize how rarely online discussions involve experience so thoughtfully presented. I note in particular,

Arvind Venkataramani

Big data - in the sense discussed above - can occupy an important space between narrative accounts and experimental measurements. It would be a mistake to treat it just as statistical evidence, as you point out - its rhetorical impact would be much more if it was humanized into a narrative - made small, as it were.
You also speak of design skills not being widely available: this, precisely, is the strategic importance of big data - that it can enable forms of discourse that bridge the worlds of usability, user experience, design research, and product management; that it can enable design decisions by people who might not otherwise be 'qualified'. [1]

Whitney Quesenbery

On several projects, I have been challenged to help make sense out of data, and to make connections between what we are seeing in analytics and what we see in design research.
One of the more interesting experiments in this sort of triangulation was done by the person in charge of web analytics at a university. She took our personas and rich descriptions of the process of moving from "thinking about attending" and "being a student."  Using profiles based on the personas, she was able to filter the datastream down to a (relatively) small number of cases, and look at their navigation patterns over time. The picture the data showed was similar to what we heard directly. The visuals were quite interesting, and when we matched them to the narrative from several participants, it was pretty powerful, especially in talking to some of the people who mistrusted the small numbers in the qualitative work.
To go back to my original question, however, all of these uses of data are (nominally) anonymous aggregations.
What set me on this thread was that I'm starting to see data collection tools that  allow you to collect the sort of detail you might see if observing someone using the device or site - and in a context that makes it non-anonymous. And, it does this without informing the person from whom the data is collected. ClickTale, for example, is explicitly aimed at being able to reproduce a session of use.

Norman Stolzoff

Brick and mortar retailers, such as the C-Stores you describe, are still using POS data to plan for the norm.  These data streams do not explain themselves--provide us with the why behind the what.  They don't explain the behavior or the motivations for it.  This is not to say these data streams aren't useful, but they can be misleading.

In a study I did for a CPG client they had scanner data that showed that grocery basket size was down--this was right after the Recession of 2008 hit.  What they didn't know was why.  We shopped nation-wide with consumers and what we found was that many had started to shop at multiple stores--cherry picking the best deals from each store.  Their overall basket hadn't shrunk it was now being split up between retailers.  The POS data didn't have an explanation for this, the strategies the shoppers were using to hunt for deals, etc.
When the digital wallet becomes ubiquitous more of the loop will be closed.  This is the NEXT BIG FRONTIER / Battlefield for big data and where privacy concerns should be focused.  Who will control this data?  Right now a particular retailer has a snapshot of some of my purchases (if I use my loyalty card), but they aren't close to having a picture of my complete consumption pattern.  Wal-mart does not provide their data to others--and that's 1/3 or all purchasing in the grocery channel, for example.

Returning to Barabasi, I note that what he is writing about is the near future that Norman anticipates, one which raises in acute form the issues with which Whitney began the thread. Here in Japan a smartphone is becoming not only a digital wallet but a digital ticket for public transportation (trains, subways, buses) as well. Add the increasing ubiquity of CCTV surveillance cameras, the use of the digital wallet to pay taxi fares, and rapidly improving biometric identification technology. I have translated material about new airport security systems with cameras able to capture retinal scans from distances of a couple of meters, smoothing traffic flow by doing the scans as people stand in line instead of their having to stop and stare at a camera at the counter. Put all this together with network integration and big data processing tools....

/div>

Comment by Huon Wardle on June 19, 2012 at 2:17pm: I agree John. It has been obvious for a long time that the normative style of anthropology is morbid if not totally dead. As such, few practising anthropologists work in that genre any longer; or at least only heuristically for partial elements of comparison and contrast. You have pointed elsewhere to how, for some, network theory or network analysis can be useful for framing interpretations. The shifts in question have also resulted, in part, in the growth and enriching of life history methods and modes of interpretation. Life histories have become more reflexive and collaborative; they ask more open ended social questions than when Radin wrote his masterpiece about Crashing Thunder.

What you are pointing to is that the individual is a relatively still point in a swirling medium. After the long history of bureaucratic effects, and the emphases on collective consciousness, individuals turn out to be important after all. It should follow that if singular persons are, through their self-habituation, more predictable than the various latterday crowds, then this offers a potentially extremely valuable specialist role for the anthropologist: anthropologists are the ones who can enter into these rich individual worlds. My own experience as an ethnographer has been that knowing a few individuals closely over time has been of far more value than attempting to make collective or normative claims about 'a community' or 'a society'.

Speaking for myself I am a creature of habit like the next person: I have continued to buy products via Amazon since I learnt that they don't pay any tax to speak of. However, the 'personalised' advertising directed at me electronically is always a sad disappointment - I wish that they thought of something that would interest me, but sadly it is always 'fun half price at the Dundee multiplex' (because my computer sits on a desk relatively near to Dundee). Electronic personalisation is, on the one hand, a reflexive feature of the current phase of modernity that should be explored by anthropologists. On the other hand, it is an opportunity, because the mathematised approximations to what it means to be a human being are well short of how anyone would recognise themselves. Anthropologists are well placed to show, analogue rather than digital, individuality.

In Bursts: The Hidden Pattern Behind Everything We Do, Albert-Laszlo Barabasi suggests that the proposition that people are predictable en masse but not as individuals is, in fact, nearly totally backwards. As individuals we are mostly creatures of habit, and now that data on our habits can be collected on an individual level, we are very highly predictable in what we do. Stories that explain the means and medians revealed by normal curves are pointless in a world where the behavior or people on the tails of the curve can be predicted as easily and precisely as that of the "average" person.

RSS

Welcome to
Open Anthropology Cooperative

Sign In

Translate

OAC Press

@OpenAnthCoop

Tweet #OpenAnthroCoop

Events

Badges | Report an Issue | Terms of Service