Generally, a research question can be answered with a minimum of three qualified sources, depending on the question.

Pubmed and Google Scholar are good places to begin research. Unfortunately, the full text is not always available. If you’re in college, you have access to databases with studies that offer full texts.

When researching, consider carefully the terms used. Too specific, or too broad, can yield little to no relevant information.

A good quality study has three essential aspects. These factors aren’t usually hard to find.

  1. Recent publication
    1. Science and technology advances fairly rapidly
    2. Studies must be fewer than ten years old
    3. Found at the top of the study, around the title
  2. Peer review
    1. There needs to be at least one other expert in the relevant field to validate the study
    2. It cannot be anyone who helpd the research in question
    3. Sometimes hard to find. If there’s a “submitted” date and an “accepted” date, that can mean peer-review
  3. Lack of a conflict of interest
    1. The author(s) of the study cannot gain money or power as a result of the conclusion
    2. Found at the top and bottom of the study

For experiments:

  1. Randomized
    1. Participants must be randomely assigned to either the intervention or the placebo
  2. Double-blind
    1. Neither participants nor researchers must know which participants are given the intervention, and which are given the placebo

For some questions, an internet search will do. There’s no definite rule for when, but here are a few possible reasons…

  1. Not enough information is found by extensively searches
    1. Not enough research on the topic
    2. Needing the full text, but not having access to it
  2. Trying to find individual experiences
    1. To validate these, they should appear in the press
  3. The answer is widely agreed upon by laymen (everyday people)
    1. For example, that Earth revolves around the Sun
  4. Studies on the matter significantly violate the rules of being statistically rigorous (the quality of a study)
    1. Perhaps, a conflict of interest

Important: Studies, of course, cite information from other studies. Sometimes I use such information. So this data isn’t the center focus of the study I’m looking at, but it’s a reflection of the statistical rigor of the study I’m analyzing. I don’t consider this “wrong”. I think of it as akin to trusting a proven friend or family member. Similarly, we don’t need to prove the physics equation, F=MA, in order to use it.

Here we have a plethora of more examples:

1. The absolute first thing to do before searching, is changing the publishing date to within ten years of the present. Why do this? If a study isn’t fairly recent, it may not apply to current circumstances.

For example,

Does marijuana use cause, or bring out, schizophrenia?

Our research carries us to the conclusion that marijuana does not lead to psychosis. But unless our data is recent, the study is useless.

Why?

Because we never researched how the composition of marijuana has varied over time. It’s only been in the past 10 or so years that levels of THC in marijuana have skyrocketed – tripled. At the same time, levels of CBD, which largely counteracts THC, have decreased to a mere smidgen of what they once were. This is according to respected news outlets such as Forbes, Livescience, and The New York Times; it’s also all over the internet. Remember: if a scientific question is widely agreed on, there’s usually no need to search for studies on it.

End example

2. Next of the necessities is peer review. It’s not enough that the authors are intelligent, respected, dedicated and experienced. Peer review gains value because it’s another expert’s outsider perspective. It’s an editor of sorts. To spot a peer-reviewed paper, (usually) there’s a “submitted” and an “accepted” date, found near to the title.

3. Finally: no conflict of interest. Allowing full text access for free means that the authors can’t make much money off study. So some make money by benefiting from the conclusions of a study, creating a conflict of interest. Overall, conflict of interest significantly compromises objectivity nature of research. Potential conflicts of interest are declared at the bottom of the study n question.

For example,

A Vyvanse study, published in 2013, exploring new indications. One of them is for the negative symptoms of schizophrenia, which no medication is presently approved for. The study is recent, peer-reviewed, has several authors, and the full test is free. But half of the authors, including the two lead authors, are well-connected to Shire, the corporation that has Vyvanse on patent. This study isn’t statistically valid enough to use.

End example

The next rule applies to experiments.

Experiments must be randomized, and double-blind. Randomization is when participants are assigned to either the experimental group, or the control group, at random. Many experiments do this through a random number generator. Next, no one knows who’s receiving placebo, and who’s being given the intervention; this is called “double-blind”. So neither the researchers, nor the participants, know until the study is peer reviewed and published. This information may be difficult to find out if only the abstract of a study is available.

  1. Randomized, double-blind procedure
    1. Neither the participants, nor the researchers, know who’s being given placebo, and who’s being given the intervention
    2.  Intervention
      1. An active treatment given to a group in a study
    3.  Placebo
      1. A fake treatment that is administered such that the participant cannot tell the difference

Not always necessary, but greatly increases the validity of what you’re trying to prove!

There are also three traits of a study that aren’t always needed, but which make a study stronger:

  1. Access to full-text
    1. The entire study, not just the abstract
    2. The study ought to be finely analyzed
    3. There is usually useful information in the methods or results that don’t make it to the abstract
  2. Systematic review
    1. A lot of relevant information is taken as a whole
    2. Trends are discovered
    3. Moves to a meta-analysis when there are enough similar studies on a topic
  3. Meta-analysis
    1. A lot of relevant studies are taken through statistical tests, creating one meta-study
    2. No field work conducted
    3. Studies must be similar enough to each other
    4. Oftentimes conducted after a systematic review

1. Try to access the full text! Full text access means that the study can be well looked over. Sometimes, there are problems that can only be spotted in the full text. Unfortunately, many of us can’t access databases. This means that we are blocked from looking at the full text of many studies. College students, professors, and experts, usually have database access. But, let’s assume we don’t. Sometimes, a full text can be found by searching for the title of the desired paper on google. If only the abstract is accessible, it may be okay to use, depending on how well it fulfills the other rules.

2. A systematic review takes into account all of the studies that have been conducted on a topic. It’s more general than meta-analyses, allowing more leeway between study design, and isn’t run through tests. A general opinion is reached.

3. The most useful type of study is a meta-analysis. It includes many studies on a fairly similar topic, conducted in a similar manner. Unlike systematic reviews, meta-analyses run statistical tests to reach a quantitative (having to do with numbers), and more specific, conclusion.

Important: sometimes, the rules of evaluating a study as statistically rigorous enough to be included, may not strictly apply. For instance, say I wanted information on the adverse health consequences of taking adderall in doses that it’s frequently prescribed. After looking through study after study on methamphetamine abuse, I find: Amphetamine Treatment Similar to That Used in the Treatment of Adult Attention-Deficit/Hyperactivity Disorder Damages Dopaminergic Nerve Endings in the Striatum of Adult Nonhuman Primates. It’s the sole study on the very specific matter of my research. But it was conducted in 2005, 11 years ago. We also might also say that Dr Ricaurte, the head of the study, was trying to clear his name after his scandalous study in the past. But the study was well-done and comprehensive.

Both are relatively minor imperfections. It can be used.

Sources: https://www.researchgate.net, Dr. Laura Wray-Lake, https://research.collegeboard.org, http://www.statsdirect.com, http://blog.minitab.com, http://stattrek.com, Dr. John Bezirganian, Ben Komor, http://libguides.usc.edu/writingguide/abstract, https://explorable.com