“Successful Qualitative Research” webinar recording with extended Q&A

Last week, Dr. Kristi Jackson of Queri presented a webinar on how qualitative researchers can be flexible with the scope, focus and perspective of their studies for more compelling research. The recorded webinar is available above and the slides, along with a list of resources, may be found at the bottom of this post.

Though Kristi was unable to answer all of the audience’s questions in the webinar’s lively Q&A session, she provided answers to the remaining questions here on SAGE Connection.

Advice for starting as a novice

I am a doctoral student just beginning proposal writing.  Can you identify one or two potential pitfalls that either doctoral students or beginning qualitative researchers make when embarking on their first qualitative study?

  1. Trying to do too much, or trying to make the study to big in scope. Keep it focused as you start (it’s likely to get bigger on its own!)
  2. Failing to make a logical argument between the research questions, methods, & theories. Ask your committee members to point you to one or two proposals that they found to be clear and well-bounded. It could help give you some helpful ideas. In general, I recommend writing these (questions, methods, theories) up in separate sections, but on one of your later re-writes, see if you can both foreshadow some of the language you’ll be using later, and weave in prior language as you progress to help give it continuity.

Remember, when in doubt, your committee is always right. So spend some time asking them what they think, preferably in person over a relaxed conversation instead of email.

When embarking on the journey of a new qualitative research project, if you’re considering both interview and survey, which do you recommend as a starting point?

This depends on the research question and the setting and the relationship between these data types and your participants. Interviews can help clarify the important survey questions to ask (or help you craft the survey altogether), but surveys can be helpful ways of generating ideas for interviews (what issues are raised, what questions are unanswered, how to these findings point to the “how” and “why” questions). When in doubt, do a pilot study of both to see which order leverages the most relevant data.

What would be your advice to a qualitative novice – there’s so much information out there, where should I start?

Start with your interests and the problems you want to investigate. Read literature about those problems and carefully think through the strategies researchers used to conduct the research. Start with something real instead of something abstract (and consider getting Richard’s Handling Qualitative Data!) If the methods seem vague when you read existing research literature (there’s sometimes not much room to write about the methods), reach out to the author(s). They’re often quite willing to talk through their approaches and to send you supplemental materials from their studies (interview guides, etc.).  Dissertations are often great places to get more detail about the research process and methods, so consider hunting for a few of those, and if you find something you want to emulate, ask a mentor to have a look and give you his/her opinion.

The relationship between qualitative & quantitative

What are your thoughts on the balance between quantitative and qualitative research? Do you think one can be a good qualitative researcher if one’s main angle is quantitative?

ABSOLUTELY. There’s a great figure (1.1) on p. 4 of Bernard and Ryan (2010) that shows how you can use qualitative data to conduct quantitative research, and quantitative data to do qualitative research. The data types and the analysis strategies aren’t necessarily the same. One of the reasons I spent so much time thinking through closeness and distance in qualitative research is that I think this can be a metaphor that cuts through the qual/quant divide. Instead of thinking of subjectivity versus objectivity, we can see all research along a complex set of continuums of closeness and distance (and remember, as soon as you get “close” to one thing, you get “far” from another, so it’s all relative). If I want to take a more “distanced” approach (what most people think of as “objective”), I’m really just moving away from one cadre of angles toward another. The same thing is true for getting “close.”  Furthermore, as a researcher, articulating what you are getting close to and far from – and for what reasons – can be extremely helpful for you, the participants, and your audience. This seems to be a goal that both qualitative and quantitative researchers can pursue. You might also consider looking at: Howe, Kenneth R. (2003).  Closing Methodological Divides. Boston: Kluwer Academic Publishers.

Coding

Can you give some more examples of ways to code rather than theme?

I use qualitative research for evaluation. Do you have any specific suggestions for how to move beyond thematic coding?

How do you determine the boundaries with coding in terms of sub-codes. Can there be too many sub-codes? When do you know if you are going too far with the sub-codes?

How do we avoid falling into that trap of only using pre-defined definitions for codes? I am asking this because I am working with a team that wants to continue coding at a micro level while we are on a tight time frame. So, there is a tradeoff of time versus coding at a level where we will get meaning.

First: Take a look at these two resources, they’ll give you tons of ideas:

Johnny Saldaña (2015). Thinking Qualitatively: Methods of Mind. Thousand Oaks: Sage

Johnny Saldaña (2012). The Coding Manual for Qualitative Researchers. (2nd Ed). Thousand Oaks: Sage

Secondly, beware of anyone who tells you that there are “rules” for coding. Coding is pointless unless it is taking you somewhere. Deconstructing data and putting it into piles is not usually the goal (unless you’re doing a really basic data-distillation strategy on materials like open ended comments from feedback forms from a conference). The utility of coding usually has to do with what you want to do AFTER you create the piles, and these goals vary widely. I’ve certainly seen some researchers start to do similar things with their codes and their process of coding over time (in different projects), but it’s because they start to craft and fine-tune the way they like to handle data; this isn’t a horrible thing – sometimes I think that approaches to coding are as much about the personality and frame of mind of the researcher as they are about methodologies! There’s an interaction between them.

Thirdly: one of the reasons I really like NVivo, Dedoose, and the other QDA software is because they have strategies for looking at relationships among codes, so you can play with constellations (or, co-occurrences) of codes. I usually find that whether researchers are using QDAS or not, a “pilot” of a subset of the data to get a feel for how you’ll be handling/coding (and where that work is eventually taking you) is WELL WORTH the effort.

Finally: DON’T forget to memo (and in NVivo learn how to do “See Also Links” in your memos) so that you’re not relying on coding as your only source of tracking your thoughts. When I help people with their projects and I see they have lots of Memos/journals about their ideas (instead of just piles of codes) I am much more likely to find good coding as well.

 Software

From your experience, could you tell which software is more useful to analyze a qualitative data from interviews?

I think the top options on the market are very similar things, and that you should beware of people who are INSISTENT that one is SO MUCH BETTER than another. I often use the analogy of writing a book. If I am a novelist, it’s like debating whether to use Mac or PC to write my book. There’s probably an argument that can be made about the preference, but IT’S PROBABLY THE LEAST OF MY WORRIES! You just need to leverage the tools properly (get training, work with a mentor, understand the qualitative methods you are using, etc.). Something people often overlook is the relevance of what your colleagues are using. I’d never encourage someone to switch to option A if all their peers were using option B and they were happy with option B. Being able to trouble-shoot with colleagues can be really helpful, so find out what they’re using.

Do you have to use software to analyze a qualitative data? What if I want to go the old school way, is there any disadvantage to it, apart from been labeled “old school”?

If you are a young researcher in a setting where you don’t have much power and you have to be careful about your moves, think about this in the context of how you’ll be received/treated (sometimes the use of QDA software is viewed well, and sometimes it isn’t). I’ve seen strange things happen (around both loving and hating the use of QDAS), especially in dissertation committees. So, investigate the preferences of those with power in your local context, particularly if you are a new qualitative researcher among some heavy-hitters. Otherwise, focus on YOUR goals and YOUR preference, and don’t worry about being called “old school” or a “luddite” or a “technophile” or an “NVivo groupie”. No matter what you do, someone will get arrogant with you about your choice. Researchers can get that way (it’s my biggest gripe about researchers of all kinds). I do think that the LARGER the project gets, the more QDA software tends to help (though I use it in projects of all sizes because I’m comfortable with it).

I would NEVER judge the merits of a project just based on whether someone did or didn’t use software, but I’ve seen some researchers really leverage the tools elegantly in order to do things that might have been avoided if it weren’t for some of the capabilities. I’ve copied a portion of my dissertation below, in the event you want to reference the special features of iterativity and flexibility afforded by QDA software. BUT, use your own, good judgment about whether or not to use QDAS. 

This is from page 12-13 of my dissertation, Qualitative Methods, Transparency, and Qualitative Data Analysis Software: Toward an Understanding of Transparency in Motion:

It has been over a decade since Richards (2002) articulated the increasingly relevant role software could play in qualitative methods. “The image of qualitative computing as only coding remains even in recent literature. . .” but software is now “far more varied,” allowing for new methodological approaches (p. 426). One such approach is “systems closure,” which allows researchers to keep search results and researcher ideas inside the database and to use them in subsequent searches or investigations. In earlier versions of the software, the database was “open” because it pushed the results outside of the database for review. Richards argued that the introduction of “systems closure” gave researchers the ability to pursue additional questions efficiently, accurately, and iteratively. Pursuits they might intentionally or unintentionally avoid if not for the recursive capabilities of software (Tesch, 1990). The example I turn to momentarily about the relationship tool in NVivo will demonstrate several turns of interpretation that are supported by systems closure, iterativity, and flexibility.

As Miles and Huberman (1994) argued, the flexible, recursive and iterative capabilities of software (promoted by characteristics such as systems closure) provided unprecedented opportunities to challenge researcher conceptualizations. In support of this claim, Garcia-Horta and Guerra-Ramos (2009) detailed their use of MAXQDA and NVivo in two different research projects, concluding that, among other things, the programs helped push researchers past the powerful and often unwarranted influence of first impressions of the data. Lyn and Tom Richards (1994) agreed, and stated that as they began developing NUD*IST, their analysis “became far surer, with provision for constant interrogation of themes. The processes of building and interrogating themes gave an impression of constant working at theory built up and peeled back in onion skin layers” (p. 164). Lyn and Tom Richards (1994) also stated that one of their primary goals during the early development of NUD*IST and NVivo was to encourage researchers to consider alternative explanations for patterns in the data.  

Boundaries

You mentioned some boundaries or general rules around being flexible?  Can you specify?

I’m about to conduct a focus group study. How can I use closeness in a focus group study and remain objective?

Can you share how you handle the outsider restraint that is potential in qualitative research?

How much flexibility is too much? When are you being too disciplined, too free in your approach?

I am using case study (longitudinal questionnaire, interview, learning analytics, observations). I wrote up my results because I thought it would be useful to “get it out of the way” now I am having trouble “telling the story” — Do I need distance?

I realize I did not address, at all, the boundaries of “what’s acceptable” when adopting a flexible posture as you move through the research process. But, there’s a reason I pointed to improvisational jazz, improvisational comedy, and improvisational conversations. Frankly, you usually don’t know the acceptable range of movement until you do it with others and watch how they react and ask them good questions. The boundaries are socially constructed and fluid. I don’t think that improvisational jazz or improvisational comedy can be taught in the abstract. They’re taught in the “doing”.  The same is true for qualitative research. Furthermore, different musicians and different comedians have different rules around this range of motion. Like any craft, you’ll benefit from finding an expert you respect. Study with him/her, ask questions, challenge, say, “what if . . .” and get ready for diverse opinions.

If you’re working on a dissertation, strap on your seatbelt. You’ll be learning about your boundaries and the boundaries of your committee/chair. This could be the biggest, long-lasting lesson of your dissertation; figuring out the boundaries (and, more specifically, tuning your skills about HOW to figure out the boundaries). While you’re analyzing data, try hard to understand why the committee will/won’t allow certain movement. Understanding their rationale is key to finding your way in the qualitative research journey. In general, I think they’ll be happy just to hear/understand that you are thinking about and playing with these boundaries; for a dissertation, use caution if your goal is to “get it done”.

If you’ve set aside your sense of humor amidst the seriousness of academic life, go get it. Laughing at yourself and at the process (at appropriate times) can help nudge you through the tough spots while you figure out this strange, flexible world.

Specific Questions

I had done a qualitative analysis and submitted the result as an abstract to a conference. One of my reviewers said it was not so insightful. What should get revised before resubmitting?

There’s no way of knowing from the distance that I have on your situation. If you can get to the reviewer and ask for examples of other work that show “insightful” research, do it. The peer review process is probably the best we have, but it is fraught with problems. Though it doesn’t feel like it when you read the reviews, think of the reviewers as other passengers on your train, with a moderate degree of disinterest. You don’t know if you like them, respect them or have anything in common. BUT, they can really influence how much you’ll enjoy this train ride. When the ride is over, you can talk to people who know and respect you and they’ll get you grounded again in what’s important to you.  Friendly clarifying nudges back to the editor/reviewer are warranted when you just don’t get what you should do next if you want to get published.

Thank you for saying we need to give ourselves a break and perhaps let our “not-so-great” interviews allow us to re-work the interview guide. Is there an issue with the analysis if the guide or the focus of the interviews morphs?

This is a question that can be debated. My take on it is that if you have a strong reason to believe that you’ll get better data if your questions morph, then there’s good reason to do so. You are trying to honor the participant’s and their time (it’s unethical to interview them just to cross them off the list!).  BUT, some reviewers/advisors/mentors take greater issue with this than others.  Like most other qualitative research decisions, think through things carefully, take time to understand alternative views, and then make your argument for why you adjusted things if you did. Oftentimes, a reviewer or a committee member just wants a good rationale for the adjustment.

How do you transcend above the obvious after you’ve done an analysis and discovered what might be already known or common sense knowledge?

Add to what’s already known by pushing towards the “why” and “how” questions that may be unique in this setting. Or, to shake things up. Find out what the prior literature/research says should be happening, and ISN’T in your setting. Perhaps looking at the ‘absent’ will help you get to some interesting conclusions? This is another part of “moving” and flexibility that I didn’t address in the talk.

I’m stuck with the concept of thematic vs content analysis. Are they similar or different? What are your thoughts on it?

Both are fairly focused on finding textual patterns, for instance. I think the following distinction makes sense:

https://books.google.com/books?hl=en&lr=&id=SHiUvmKzuFwC&oi=fnd&pg=PA56&dq=thematic+versus+content&ots=JnUOubZE5N&sig=-JpDQSlWmu7xp0vSE7ObI-xl5Oo#v=onepage&q&f=false

I worry that both of these approaches tend to emphasize data distillation over data expansion. So, it might be worth it to bring those two concepts into the mix and see where they take you.

For reference: what we answered in the webinar:

  • I’m stuck in a Thematic Analysis rut (stick w/ what I know!) hoping this #SAGEtalks will get me thinking broader – any suggestions?
  • How can I keep my composure during an interview? Sometimes a wow or oh wants to pop out.  How does this influence the interviewees responses?
  • I am working now with my dissertation, I am a teacher and my participants are school principals. In this case how can I be far?
  • Closeness – does the concept of near/far apply to data too? Transcribing services are available, but transcribing own data = deeper immersion #SAGEtalks
  • My research is related to the pedagogical aspects of journalistic ethics. My study entails interviews with a representative of the National Council for the Training of Journalists, the head of the journalist department at Sheffield Hallam University, a senior lecturer on the undergraduate and postgraduate journalism programs and a student who has recently completed the taught segment of her MA in Sport Journalism is this in line with the Insider/Outsider model discussed?
  • What are your “go-to” explanations for validating qualitative data/analysis to research team members who are strictly quantitatively minded?
  • Any suggestions for working with mixed-methods team members who are less comfortable with flexibility and induction?
  • What if I ask one question during an interview and the respondent wants to talk about other things, what do I do? Should I let them go ahead or stop them?
  • Regarding elite interview, some professionals escape from answering and this diverts the agenda of research. What do you advise?
  • I’m always concerned that my pre-existing ideas will bias either the collection or analysis of data. What do you do to help prevent self-fulfilling prophesy?
  • Any tips related to closeness and distance when you have countertransference with your data? For example I’m a mom of a child with disability doing disability research. I tend to over-compensate in interviews for distance.
  • How can I take break from my data while we are working with our dissertation?
  • Can you recommend any good references for the distill/expand processes?
  • Is there literature on the concept you used earlier about “dropped out” or “pushed out?”

 

Resources:

https://www.youtube.com/watch?v=LwwqlsSH4Hw

(90 second motivational video)

Metaphors in everyday speech

George Lakoff and Mark Johnson (1980). Metaphors We Live By. Chicago: The University of Chicago Press.

George Lakoff and Rafael E. Núñez (2002). Where Mathematics Comes From. New York: Basic Books.

Metaphors in qualitative research (see table below – this is not an exhaustive list, just pointers to places to start. In some instances I provide specific pages, chapters, or brief descriptions):

  • Participant language: Listening to the metaphoric language that tells us more about what is going on in the social context.
  • Communicating findings: Using metaphor to convey meaning to outsiders/readers.
  • Watching yourself and describing your practice: Examples of the qualitative research journey and metaphors that some scholars have used to describe this journey.

 

Participant language Communicating  findings Watching yourself and describing your practice
Miles, Matthew B., Huberman, A. Michael, and Saldaña, Johnny. (2014). Qualitative Data Analysis: A Methods Sourcebook. Thousand Oaks: Sage Publications. p. 280 p. 333
Richardson (1998). Writing: A method of inquiry. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of Qualitative Research (2nd ed., pp. 923-948). Thousand Oaks: Sage. x x
Lofland, J. & Lofland, L.H. (1984). Analyzing Social Settings. Belmont: Wadsworth Publishing company. p. 122-123
Coffey, Amanda & Atkinson, Paul (1996). Making Sense of Qualitative Data. Thousand Oaks: Sage Publications. Chapter 4: Meanings and metaphors
Denzin, Norman  K. & Lincoln, Yvonna S. (1997).  The discipline and practice of qualitative research In N. K. Denzin & Y. S. Lincoln (Eds.), The Landscape of Qualitative Resarch. Thousand Oaks: Sage. p. 3 Qualitative researchers as ‘bricoleurs’
Hammersley, M. (1999). Not bricolage but boatbuilding: Exploring two metaphors for thinking about ethnography. Journal of Contemporary  Ethnography.28(5), 574-585. Qualitative researchers as ‘boatbuilders’
Kamberelis, George, and Dimitriadis, Greg (2005). Qualitative Inquiry: Approaches to Language and Literacy Research. New York: Teachers College Press. p. 156: Researcher as geneologist
Quinn, Naomi (1996). Culture and contradiction: The case of Americans reasoning about marriage.Ethos 24:391–425. Marriage is like the Rock of Gibraltar
Janesick, Valerie J. (1994) The Dance of Qualitative Research Design: metaphor, methodolatry and meaning. In Norman K. Denzin, & Yvonna S. Lincoln (Eds.), Handbook of qualitative research (pp.209-219). Sage: London. x
Goffman, Erving (1959). The Presentation of Self in Everyday Life.  (or, just about anything by Goffman!) Front Stage and back stage.
Saldaña, Johnny (2015). Thinking Qualitatively: Methods of Mind. Thousand Oaks: Sage Chapter 7: Thinking Artistically
Corbin, Juliet & Strauss, Anselm. (2008). Basics of Qualitative Research 3e. Thousand Oaks: Sage. p. 83
Fred Davis (1973). The Martian and the convert: Ontological polarities in social research. (Urban Life and Culture). x

 

Closeness and distance with Qualitative Data Analysis Software

Lyn Richards (1998). Closeness to the data: The changing goals of qualitative data handling (Qualitative Health Research).

Linda Gilbert (2002) Going the distance: ‘Closeness’ in qualitative data analysis software (International Journal of Social Research Methodology).

Closeness and Distance with Transcription

Mishler, E. G. (1991). Representing discourse: the rhetoric of transcription. Journal of Narrative and Life History, 1(4), 255-280.

Ochs, E. (1979). Transcription as theory. In E. Ochs & B. Shieffelin (Eds.), Developmental pragmatics (pp. 43-72). New York: Academic Press.

 “Go to” Resources from my Bookshelf

Bazeley, Pat (2013). Qualitative Data Analysis: Practical Strategies. London: Sage.

Charmaz, Kathy (2007). Constructing Grounded Theory. Thousand Oaks: Sage.

Creswell, John W. (2013). Qualitative Inquiry & Research Design: Choosing Among Five Approaches. Thousand Oaks, Sage.

Given, L. (2008). The Sage Encycolopedia of Qualitative Research Methods. Thousand Oaks: Sage.

Goodall, H.L. Jr. (2008). Writing Qualitative Inquiry: Self, Stories, and Academic Life. Walnut Creek: Left Coast Press

Howe, Kenneth R. (2003).  Closing Methodological Divides. Boston: Kluwer Academic Publishers.

Janesick, Valerie (2015). Contemplative Qualitative Inquiry: Practicing the Zen of Research. Walnut Creek: Left Coast Press.

Leavy, Patricia (2015) Method Meets Art: Arts Based Research Practice. New York: Guilford Press.

Maxwell, Joseph (2013). Qualitative Research Design: an Interactive Approach. Thousand Oaks: Sage

Patton, M. Q. (2002). Qualitative Research and Evaluation Methods (3rd ed.). Thousand Oaks: Sage

Richards, L. (2014). Handling Qualitative Data. (3rd Ed.) London: Sage.

Johnny Saldaña (2015). Thinking Qualitatively: Methods of Mind. Thousand Oaks: Sage

Johnny Saldaña (2012). The Coding Manual for Qualitative Researchers. (2nd Ed). Thousand Oaks: Sage

Schwandt, Thomas (2015). Dictionary of Qualitative Research. (4th Ed.) Thousand Oaks: Sage.

Silverman, D. (2010). Doing Qualitative Research (3rd. Ed.). London: Sage.

Smith, Linda Tuhiwai (2004). Decolonizing Methodologies: Research and Indigenous Peoples. New York: Zed Books.

Validity/Reliability and alternatives in Qualitative Research

Kirk, J. & Miller, Marc L. (1986). Reliability and Validity in Qualitative Research. Newbury Park: Sage Publications.

Lincoln, YS. & Guba, EG. (1985). Naturalistic Inquiry. Newbury Park, CA: Sage Publications.

Miles, Matthew B., Huberman, A. Michael, & Saldaña, Johnny  (2014). Qualitative Data Analysis: A Methods Sourcebook. Thousand Oaks: Sage.

Flexibility in Qualitative Research

Leavy, P. (2014). A brief statement on the public and future of qualitative research. In Oxford Handbook of Qualitative Research (p. 724).  Oxford: Oxford Library of Psychology.

Padget, D. K. (2008). Qualitative Methods in Social Work Research. Thousand Oaks: Sage.

Wickson, F., Carew, A. L. & Russell, A. W. (2006). Transdisicplinary research: Characteristics, quandaries and quality. Futures, 38, 1046-1059.

Students: Push-outs and drop-outs

http://educationvoters.org/wp-content/uploads/2012/11/White-paper-discipline-010612.pdf

http://thenotebook.org/april-2010/102374/dropped-out-no-pushed-out

http://www.nytimes.com/2003/07/31/nyregion/to-cut-failure-rate-schools-shed-students.html

 

     
This entry was posted in SAGE Connection, Tips for You. Bookmark the permalink.

Leave a Reply