WikiJournalClub:Interpreting the medical literature
Welcome to the Wiki Journal Club guide for interpretation of medical literature!
Basics
The easiest way to understand clinical trials is in understanding the components of a Population Intervention Comparison Outcome (PICO) question
- Population
- The group being studied
- Individuals with unstable angina
- Intervention
- A change being made part of a the study group
- Giving aspirin 324 mg by mouth daily
- Comparison
- The rest of the study group that doesn't get the intervention
- Giving placebo by mouth daily
- Outcome
- The end point(s) being measured
- Rates of acute myocardial infarction or death at 12 weeks
We can combine these together to get a PICO question!
- Question : What is the PICO question you come up with here? (Hint: this is the PICO question for the VA Cooperative Study!)
- Answer:
Population | In individuals with unstable angina, how does... |
---|---|
Intervention | giving aspirin 324 mg by mouth daily... |
Comparison | compared to placebo by mouth daily... |
Outcome | affect rates of acute myocardial infarction and death at 12 weeks? |
Trial or Study Types
Randomized Control Trial
Randomization of a population to receive an intervention (often a new medication) or no intervention (often placebo or old medication. Follow forward to find outcomes!
Observational Trials
May be prospective or retrospective. Types of randomized control trials:
- Cohort - may be prospective or retrospective, helps estimate relative risk
- Given an exposure, what are the outcomes in the intervention group versus the comparison group?
- Case-control - retrospective, helps estimate an odds ratio
- Given an outcome, what are the odds that people with an disease (cases) had an exposure versus people without the disease (controls)
Meta Analysis
Combines data from multiple studies, increasing the sample size and improving the ability to identify the size of an effect Problems:
- No prospective control of variation in the individual studies
- Publication bias - the data available is more likely to have a positive outcome than unpublished data
Statistics
Basics
We are looking for an association between a test and a state.
Let's imagine you are sitting on the grass at a T intersection with a 3-way stop. Cars only ever come down one road. When they approach a stop sign, they can either go straight or turn left. Most of the time when they are turning left, they will turn on their left blinker. Most of the time they will go straight, they don't have on their left blinker. You are wondering about the association between a blinker and turning. You can state it like this:
- Test
- The blinker is either on or off
- State
- The driver will turn or not
We can make this into a Bayesian Four Square table with all of the possible outcomes (remember to put the state on the top!):
State (turning) | |||
Turning | Not turning | ||
---|---|---|---|
Test (blinker) | On | A | B |
Off | C | D |
And we can substitute pluses and minuses for the states and tests:
State (turning) | |||
+ | - | ||
---|---|---|---|
Test (blinker) | + | A | B |
- | C | D |
There are four possibilities:
- A. The car has its left blinker on and is turning left -- True Positive (TP)
- B. The care has its blinker on and doesn't turn -- False Positive (FP)
- C. The car doesn't have its blinker on and turns -- False Negative (FN)
- D. The car doesn't have its blinker on and doesn't turn -- True Negative (TN)
We can rewrite the chart to look like this:
State (turning) | |||
+ | - | ||
---|---|---|---|
Test (blinker) | + | TP | FP |
- | FN | TN |
You can define two things pertinent to the test:
- Sensitivity - Positive In Disease (PID)
- =TP/(TP + FN)
- The chance that a positive state will have a positive test
- How likely a person with their blinker on will turn
- Specificity - Negative In Health (NIH)
- =TN/(TN + FP)
- The chance that a negative state will have a negative test
- How likely a person with their blinker off will not turn
You can define two things pertinent to the state:
- Positive predictive value (PPV)
- =TP/(TP + FP)
- The chance that a positive test will be associated with a positive state
- How likely a turning car will have on their blinker
- Negative predictive value (NPV)
- =TN/(TN + FN)
- The chance that a negative test will be associated with a negative state
- How likely a non-turning car will not have on their blinker
Let's say that you sit at this corner and count the next 1,000 cars to drive through it and write down whether they turn and whether they had their blinker on. You get the following results:
- 60 people turned, 40 of those had their blinkers on.
- 40 people didn't turn, 30 of those did not have their blinkers on.
Let's fill in that table, first the TP and TN (the people correctly using their blinkers):
State (turning) | |||
+ | - | ||
---|---|---|---|
Test (blinker) | + | 40 | FP |
- | FN | 30 |
Now let's fill in the FP and FN:
State (turning) | |||
+ | - | ||
---|---|---|---|
Test (blinker) | + | 40 | 10 |
- | 20 | 30 |
Statistical Significance
An attempt to prove or disprove the null hypothesis
- Type I error (alpha error) - Being wrong about an intervention having an effect on the outcome because of random chance
- Type II error (beta error) - Being wrong about an intervention not having an effect on the outcome because of random chance
Magnitude of Effect
- Odds ratio
- ...
- Hazard ratio
- ...
- Relative risk
- ...
- Absolute risk
- ...
- Number needed to treat (NNT)
- ...
- Number needed to harm (NNH)
- ...
Validity
- Internal
- Is the outcome caused by the intervention? Poorly designed trials may have a positive outcome but lack internal validity
- Type 1 error
- External
- Generalizable - Sure you showed an effect, but is that effect useful to others?
- Reproducibile - Sure you had an effect, but is that effect reproducible?
Comparison of Interventions
See Piaggio, g et al JAMA 2006:295;1152-1160
- Superiority
- New treatment better than old
- Non-inferiority
- New treatment not worse than old
Types of Analyses
- Intention to treat
- All study participants evaluated based on initial assignment, regardless on if they dropped out, died, or didn't stick to the assigned group
Handling drop-outs
- Last observation carried forward
- Count them in the worst-case scenario (e.g. death)
- Multiple imputation
- If they all did well
- If they all died
- If they all had outcomes the same as the non-withdrawals
- Per protocol
- Only evaluate participants who stuck with the assigned treatment group. Used to demonstrate efficacy.
Bias introduced:
- More withdrawals because of the result of a drug?
- Underlying medical issues causing people to drop out?
- Specific characteristics of those who dropped out and how they would respond to a drug?
- Kaplan-Meier Curve
- X-axis with time, Y-axis with events, used to demonstrate time until an event.
Endpoints
Study endpoints are classified as primary or secondary endpoints, efficacy or adverse outcome endpoints, etc. Usually the most frequently remembered are the primary efficacy endpoints, since the design and statistical analysis of studies are constructed around these endpoints primarily. There are loads of caveats with interpreting endpoints, and a primer on the endpoints used in HF trials covers many of the most commonly encountered pitfalls:
References
Adams, Michael. Guide to Interpreting Medical Literature. Georgetown University Hospital General Internal Medicine Course. Lecture: 8 May 2012