On Wednesday, October 29, SAGE authors Gregg Van Ryzin and Dahlia Remler shared their experiences and approach to teaching Research Methods to students in a one-hour webinar. Check it out above and feel free to click through the slides below. Because we were unable to address all the questions during the webinar, Gregg and Dahlia took some time to answer them for SAGE Connection. Here is what they had to say:
Q: Where does single subject research come into play? I am in special education research.
Gregg: I cover single-subject studies in the context of case studies, which is part of my class on qualitative research. Case studies can involve an individual person (such as a clinical case study), even though case studies more often involve groups, organizations, or political jurisdictions.
Q: Gregg noted he handed out abstracts in class, is this on the day of discussion or prior day?
Gregg: I hand out abstracts on the day of discussion, as a way to get students to practice quick identification and assessment of a study. The students also read a full study in advance of class, so they can consider the details of at least one study in more depth. But the abstracts add to our class discussion by showing the range of topics and fields that rely on a given method (such as surveys, randomized experiments, or quasi experiments).
Q: Your presentation seems to me to focus on (a) research matters and (b) research methods matters. Do you see a distinction between these topics and a research methods course focused primarily on how to do research?
Dahlia: I do think there is a difference. Research refers to the questions and the (ostensible) findings. I know of research courses in many professional fields that teach nothing about the methods but just tell them the published results so that they can learn “best practice.” Unfortunately, students have no way to judge the quality of the research and whether they can really trust the findings. Nor can they assess the generalizability of the results. Methods, to me, is how the research is done, and therefore how convincing it is, the caveats and applicability.
Q: Do you have students do any actual mini experiments as part of the course?
Gregg: Yes, I very much like doing this as it really works well to illustrate the logic of an experiment – plus it is often fun for students. I sometimes use a version of an anchoring experiment, in which people are asked to guess the age at which Mahatma Gandhi died, given differing anchor-primes. The problem is often sample size, if you don’t have a large class. But I save the data from previous semesters and then add in the new responses.
Q: How do you teach action and evaluation research? Do any of you work with pre- or in-service teachers?
Gregg: In my MPA program, the required research methods course focuses on the application of research methods to evaluating programs and policies. So in a sense, it is a course about evaluation research. My program also has an elective course on performance measurement and evaluation, which covers more specialized topics but is taught less regularly. I should add that Dahlia and I use lots of program evaluation examples throughout our book, especially in the part on causal research.
Q: I have a co-faculty member completing his dissertation. What is the best method to share this with him?
Dahlia: You can share the link to this site, which provides a recording of the webinar, the slides and the answers to these questions. If the faculty member is going to be teaching a research methods course in the future, he or she should be able to get an examination copy of our book from SAGE.
Q: How important is it in your teaching of research methods to teach one of the statistics packages out there, like SPSS or R? I ask with an eye on the job market: is Methods a more helpful job preparing course with or without the exposure to a stats package
Dahlia: We teach a statistical package in our statistics course. I feel that working with real data in a statistics package is very valuable for really getting data analysis and statistics. And certainly a statistical package is a valuable qualification for many kinds of jobs. In my research methods course (following statistics), we do some data analysis with the statistical package. But it is not the main point of the course. I really think that the importance depends on the course goals—and the kinds of jobs students will get. For people who will be managers, they only need simple data analysis, say descriptive statistics in Excel. But they need to be a skeptical and effective CONSUMER of research and analysis. For those students, learning to do their own regression or cross-tabs analysis in a package might not be as important as things like how to write a good survey, being aware of response bias in surveys, conducting qualitative research or thinking hard about which control variables are needed in a particular study.
Q: In my discipline, we struggle with students reading an abstract and then going straight to the discussion without analyzing/evaluating anything in between. Using abstracts as an in-class activity might reinforce this behavior. Any other suggestions?
Dahlia: An important part of the training is to insist that they focus on the methods as described in the abstract—and also ask important questions about the methods that are not in the abstract. We wouldn’t want to use an abstract that did not describe the methods at all. We also have them read at least one full study each week, so the abstracts are for added examples and discussion purposes in class.
Q: It appears that when teaching research methods, quantitative aspects of the course outweigh the qualitative aspects. How can we make certain that we provide a very balanced assessment of research methods from both quantitative and qualitative perspectives?
Dahlia: For us balance means using whatever approach or approaches are best suited to the research question and context. That can mean an exploratory qualitative approach, followed by a quantitative approach, followed by a qualitative and so on. And it may be different researchers at the different stages. Or a given study may have missed methods. In my experience in applied research, such as health policy, there is no great conflict between qualitative and quantitative, not like in some social science disciplines.
Q: Are purposeful sampling methods discussed?
Dahlia: Yes, we discuss purposeful sampling in both the qualitative methods chapter and the sampling chapter.
Q: How does IRB work for assigning the surveys in class?
Gregg: At my university, actually collecting data would require IRB approval. So I have them design the survey, including identifying a population and sampling strategy as well as crafting a complete questionnaire. But they don’t actually collect data.
Q: Is this a case study or phenomenological study etc… (slide 12)?
Dahlia: The study used in-depth interviews.
Q: What do you think about teaching statistics and research methods in an integrated course rather than as separate courses?
Dahlia: In my opinion, the best pedagogical approach is an integrated approach. To be more precise, I would ideally order things roughly the way they are in our book, with the foundations of descriptive research (such as measurement and survey methods) before most of statistics, then the bulk of statistics, then the causal research methods, which often rely on statistics. But it is very hard for most institutions (including my own) to swing that for practical reasons.
Q: Where do you get the research articles/abstracts that you use to show students what research can do? I use academic journals but most articles are too difficult for students to comprehend.
Dahlia: I find articles from my own research, from conferences, from media that focus on research (e.g., Upshot at the New York Times), from policy discussions, from people I follow on twitter, emails from journals—all the usual ways you find out about research. I just keep my eye out for what works for my students. While I do want simpler research for them on average, I also use some studies where they may find large parts too advanced or technicaland then guide them about what to focus on.
Q: Just to clarify: you appear to be using ‘path model’ in a broad sense to refer to any model and not just SEM type models?
Gregg: Yes, we use path model (or path diagram) in a broad sense to refer to a visual diagram showing causal relationships among variables.
Q: I’m an academic librarian, and I’m often give one-shot instructional sessions for Research Methods students. I usually just teach basic database searching.
Gregg: The courses we teach (and our book) are mainly about doing primary (original) research.
Q: If stats and methods are not taught as an integrate course, then stats should come AFTER the methods course, not before. Without a context for how stats are used, most students cannot learn how they are tools!
Dahlia: I totally agree about integrating and about students needing a context for how stats are used. See the answer to #17. However, if one has to pick between before and after for practical reasons, then I prefer before, despite the problem you describe. I know programs that teach statistics after methods and the quantitative methods are very theoretical. Almost no real studies can be read and it is hard to get a feeling for real quantitative research without statistics. The better approach, in my opinion, is to try to put as much context and as many real world examples as possible into the stats course. But you are totally right about the problem.
Q: How are qualitative research methods approached? I noticed that there was a chapter focused specifically on these methods. Which types of qualitative research methods are explored in this chapter? Are they dealt with outside of this chapter…if so how?
Gregg: We cover a range of qualitative methods in Chapter 3, including interviews, participant observation, focus groups, case study research, and qualitative observation. We also cover coding and qualitative data analysis. In most chapters on traditionally quantitative topics (like surveys or experiments), we discuss how qualitative methods often play a role in such studies as well.
Q: I teach fully online courses, and it is possible to teach research methods when the professor sets the tone by providing main posts that guide the students.
Gregg: Yes, good point. I’ve just started teaching research methods online for the first time, and it can be done with proper guidance, as you suggest. Lots of good examples that students can practice with on their own are also important.