I’ve been doing research for a long time now, mainly quantitative survey-based research and analysis. In grad school I studied Psychology and Education, specializing in research methodology and validity/psychometrics, a focus on the research itself to ensure conclusions aligned with hypotheses and alternative explanations were controlled. In a nutshell, did some piece of research actually measure what it intended, and could the conclusions be justified based on the study design. It was a very theoretical and idealistic time in my life, and I felt empowered with sharp tools to make precision cuts in the fabric of falsehood, or something like that.
As I became a researcher in the real world, I discovered that research is conducted for a variety of reasons, (even sometimes for political reasons, gasp!) and study design is driven more by budget than a requirement for perfect accuracy. Corners are cut, design flaws are justified, and conclusions are often drawn in a vacuum. Sharp tools are desired but not always necessary. My idealism dulled, eventually turning to cynicism.
Then I learned about the Genealogical Proof Standard (GPS). I’d been a hobbyist genealogist for years but last year decided I wanted to learn the craft from experts, and what I gained was a new appreciation for all research, qualitative and quantitative.
The field of Genealogy, like many in the social sciences, has worked for years to establish standards for what constitutes good research. In this case, what steps has a genealogist taken to establish proof that their conclusions about ancestors are real and defensible. On the surface, the GPS is similar to construct validation in psychology – my inner idealist felt like it had come home!
There are five basic elements to the GPS that must be met to prove a conclusion in Genealogy:(1)
• Reasonably exhaustive research
• Complete, accurate citations to the source or sources of each information item
• Tests—through processes of analysis and correlation—of all sources, information items, and evidence
• Resolution of conflicts among evidence items
• A soundly reasoned, coherently written conclusion
This seems logical and simple, right? As social science researchers don’t we usually do these things? The most subtle and powerful aspect of these standards, though, is the absolute requirement for all five to be met to prove a conclusion. If you skip even one of these steps, no dice. You have to do complete and thorough research or it’s not done. What beautiful music to my ears.
Did you do reasonably exhaustive research? Do have a clearly stated research question or hypothesis? Did you read all the relevant background literature to educate yourself, or just do a quick Google search and use what was readily available? Did you incorporate all the relevant and expected sources, or just the low-hanging fruit?
Did you analyze your sources to make sure you were using the highest quality, primary, representative information? Did you critique or rank any of your sources, some being more credible than others?
What about conflicts – if your study revealed something unusual, how did you resolve it? Did you quickly draw a conclusion based on that one result, or did you compare the results to similar studies testing samples from similar populations under similar contexts? Did you eliminate or control as many sources of systematic error as possible?
Did you write a conclusion that showed, carefully and logically, how the results of your work supported or didn’t support your hypothesis, and why, including recommendations for future, better-designed studies?
Introduction to the GPS brought back the magic of research to me. Sure, in Psychology and Education there are textbooks full of dictates for conducting solid, valid studies, but I’d never seen these ideas so clearly and succinctly codified before the GPS. I highly recommend studying research methodology outside your own domain once in a while; social science researchers need to keep their tools sharp, after all.
(1) Board for Certification of Genealogists, Genealogical Standards, 50th Anniversary Edition (Nashville & New York: Ancestry Imprint, Turner Publishing, 2014), 1-3.