Pros and cons of self-reported data when doing mobile/digital ethnography​​​

rESEARCH

Sun

Blog

As a market researcher who is a cultural anthropologist and psychologist specializing in qualitative studies, I conduct single projects as well as journey work that involve a multiplicity of research events with the same set of respondents and ideally at the opportune time and place of authentic context. If the team has inclination and budget, a combination of other- and self-reported processes such as focus groups or IDIs, live if possible, then the choice of one or more ethnographic approaches, can be optimum. In our observer teams, we conduct authentic ethnography live at the time of the behavior where possible, but also employ digital ethnography to get at longer periods of time like weeks or months of ongoing moments in a day, evening, and weekend when our team is not there. Mobile and digital ethnography gives us a 360 view of a subject, their household, and life, or at least a close approximation. But mobile and digital are self-reported. I wonder about all that self-reported data when the study is conducted completely online or with mobile, when there is no live trained observer there to observe what else may be happening? I just came from a compelling qualitative research conference last week. It was sponsored by my qualitative org, the metro-New York chapter of the QRCA. There was a well-organized, innovative set of workshops on a Friday afternoon in a great facility at Columbus Circle, New York City, in which new processes of mobile ethnography were being explored. I received good ideas and references for new mobile platforms.

 

Still, I have to ask my readers: How do we feel about only self-reported data in which there is no trained observer on hand at the time of the behavior or in-situ situation, and where we as anthropologists are left analyzing weeks of intense data that has been handed to us by the respondent? Even if our subjects’ textual accounts are said to be at the perfect time of the behavior happening, with accompanying video or photos, I’m left with a bit of puzzlement. When a study only uses mobile or digital ethnography, could there be something major lost without even knowing or having the ability to find out what that missing piece is? I think I want to do a paper on it for a research conference. For now, I’ll examine the pros and cons of self-reported data through online approaches reported by the participant on mobile, laptop, and tablet. Pros. On the plus side, there are major benefits. We can collect good, sometimes deep, sometimes surface data quickly, without having to be in that location, and because we see their accompanying video or photos, it looks as if it is collected in the optimum context. This saves on time, effort, and travel expense even though the research team needs to pay for the mobile ethnographic platform and the time for us to monitor, analyze, moderate, and log on continually should be factored in. Digital-mobile is relatively easy especially with new and exciting mobile platforms. I took a look at dscout ( at that QRCA conference, listened to the invited speaker demonstrate its power, and want to consider it the next time we design a new study involving retrieval of mobile ethnographic data. On sensitive topics, subjects may feel more comfortable answering anonymously with mobile than if I were there with a team. (Or, they may not, since silence and anonymity affect inner subjectivities, projections, and assumptions as biasing.) With mobile, there could be less of a certain type of bias coming from me and the team, who—even if we dress down and stay relaxed and research-y in our attitudes—still look, communicate, dress, and interact with our participants in a particular way that can be unfamiliar to them. No matter how likable and expert we are, we remain outsiders in a new family; outsider bias is inevitable; we can counter it but not totally eliminate it. This is one reason why classic anthropological research is often done with an anthropologist in the field living with subjects for a year or more; over lengthy time, the village, segment, or culture gets used to her.

 

When mobile respondents are compensated well and motivated to keep going with daily answers, and especially are not given overwhelming, time-consuming tasks—we try to consciously keep the experience intriguing, easy, quick, and fun to do for them—we find much authenticity, spontaneity of behavior, and often good video and photos that can be analyzed and positioned in report. So, in most journey work for qualitative projects, it makes sense to work with mobile; mobile is a perfect fit in the contemporary mobile environment that has pervaded every culture and every demographic- psychographic segment. Cons. On the down side, some things I can’t help but keep in mind are what might form a paper or talk at a research conference with a little more thought and research. In mobile, the data can be biased by the respondent him or herself, particularly since consciously and usually unconsciously, most people respond and report their data in a way that is either socially acceptable or is unconsciously communicated to make a particular point (to the self or outside world), which could be positive, negative, or put the subject in a particular light. The world of psychological motivations is infinite; some people want to look good to others, while others might want to look victimized or different than they are. When a trained anthropologist and psychologist who does observational work is present, the observer will notice other elements as outliers happening around that household, in-situ situation, or behavior that are unconscious within the landscape experienced by the subject; the subject is completely unaware of these dynamics because they experience them all the time. It’s like interviewing a fish about how he feels about the water…there’s no extraordinary insight when this is all the fish knows. The subject simply won’t mention something that may be completely familiar, even though it might be new and important to us. The ethnographer’s perspective on even the most ordinary experiences, if there live, could make all the difference in the findings. There are peaks and valleys in a continuum of participants’ interest in a mobile study period.

 

Subjects are usually enthusiastic about participating in digital ethnography in the beginning but a wearout effect occurs as time goes on, especially if the same or similar task is being asked to be reported on daily or multiple times a day. For personal care topics, snacking, measurement of blood sugar, taking meds, making dinner on a regular evening, child care, and many other actions that repeat for weeks on end, the subject may run out of things to report or have little of interest to describe. Without an outside observer skilled at intervening based upon observed body language and subtle emotions, subjects may simply repeat surface or already-reported motivations because they have nothing new to say. Subjects may experience demand characteristics such as memory burden, confusion, fatigue, or distraction by life, family members…or get involved in other tasks that they find more important and skimp on their mobile involvement with the journey. If the question on that day’s task for mobile is not relevant or asked correctly, the participant might answer in emotionless one-liners, drop out, or find the topic boring and inconsequential even if it’s of evolving relevance to the research. Of course, when designing and executing a qualitative ethnographic study in which multiple approaches are possible, it’s optimum to conduct both, i.e., both live observation and digital/mobile ethnography. This gives us the best of all possible worlds. But, when the research is designed as only mobile-digital and only self-reported, we may need to keep our eyes open, to factor in respondent bias, have more client conversations about what’s coming across, and look carefully at the data for subtexts to figure out where we can analyze differently and more openly. The analysis may need to take greater leaps of faith in terms of psychological, archetypal, or behavioral implications even if not reported as such by subjects. Finally, I wonder about safe mobile, i.e., whether incentivizing subjects to use mobile causes them to report a behavior when they shouldn’t text or photograph their activity, even if it’s within the right context.​​

SUN

rESEARCH