Assessment of Web-Based Consumer Reviews as a Resource for Drug Performance Some health websites provide a public forum for consumers to post ratings and reviews on drugs. Drug reviews are easily accessible and comprehensible, unlike clinical trials and published literature. Because the public increasingly uses the Internet as a source of medical information, it is important to know whether such information is reliable. This analysis shows that Web-based consumer drug ratings and reviews can be used as a resource to compare drug performance. We analyzed >100,000 drug reviews from the health website WebMD. Statistical analysis identified 427 drug pairs from 24 conditions for which two drugs treating the same condition had significantly and substantially different satisfaction ratings. Scientific literature compared to online findings. Nearly two-thirds of the online drug trends were supported by published literature. The discrepancies between scientific literature and findings online were further examined to obtain more insights into the usability of Web-based consumer-generated reviews. We discovered that (1) drugs with FDA black box warnings or used off-label were rated poorly in Web-based reviews, (2) drugs with addictive properties were rated higher than their counterparts in Web-based reviews, and (3) second-line or alternative drugs were rated higher. In addition, Web-based ratings indicated drug delivery problems. Web-based reviews can be used to inform patients’ drug choices, with certain caveats. Conclusions: Web-based reviews can be viewed as an orthogonal source of information for consumers, physicians, and drug manufacturers to assess the performance of a drug. However, one should be cautious to rely solely on consumer reviews as ratings can be strongly influenced by the consumer experience.

1. Can social media tell us which
drug is better?
Assessment of Web-Based Consumer Reviews as a Resource for Drug Performance
J Med Internet Res 2015;17(8):e211

2. Can we use online reviews?
Easy to understand & access
Large numbers
– 6% have shared drug
experiences online
– Millions of health experiences
online

3. Limited Access to Drug Comparisons
Publications
1. Funded by drug companies
2. Shown to be biased
3. Hard to critique based on the
way it’s written
FDA Drug Trials
1. Studies are funded by drug
companies themselves
2. Biased self-reporting

4. An online WebMD Drug Review

5. Compare drugs that treat the same condition
– Controls for the heterogeneous populations
– Assume similar populations being treated

6. Agreeing with the Literature
Condition: Joint damage causing pain
“The overall 6-month effectiveness rates <…>59.8% for nabumetone, 67.6% for meloxicam.”
Clin Drug Investig. 2004;24(2):89-101
“Median time to a good or excellent PGART was significantly shorter with rofecoxib (52 hours)
than nabumetone (100 hours, P = 0.001)”
J Clin Rheumatol. 2006 Feb;12(1):17-25

7. 427 drug pairs with significant and substantially different ratings
First Drug
(online rating)
Second Drug
(online rating)
Deduced online trend
Amlodipine (2.5) Felodipine (3.2) Amlodipine < Felodipine
Amlodipine (2.5) Telmisartan (3.1) Amlodipine < Telmisartan
Search scientific literature
Search terms: Condition, Drug A vs. Drug B
e.g. Hypertension, Amlodipine vs. Felodipine
See whether literature agrees with online trend

8. Online ratings supported 62-68% when rely on
literature evidence
62
(48/77)
64
(28/44)
68
(15/22)
* 71.4 (55/77)
* 75 (33/44)
* 81.8 (18/22)
0
10
20
30
40
50
60
70
80
90
100
0.5 0.75 1
%validatedwithscientific
support
Difference between drug ratings
Scientific Literature
Scientific
Literature+FDA Labels

9. Where is there more support for online ratings?
FDA label carries
warnings

10. FDA Warnings
Amiodarone had the LOWEST online ratings. Many complain about side effects.
Amiodarone
#Reviews/total
#reviews
Complaints
13.9% (23/166) “kill, died, death, dead”
But the FDA does not recommend amiodarone to treat atrial fibrillation.
Scientific Publication says “Amiodarone is effective compared to flecainide”
Arch Intern Med. 1995 Sep 25;155(17):1885-91

11. Online data supported 71-82%
when FDA labels added as support
*62
(48/77)
*64
(28/44)
68
(15/22)
* 71
(55/77)
* 75 (33/44)
*82
(18/22)
0
10
20
30
40
50
60
70
80
90
100
0.5 0.75 1
%validatedwithscientific
support
Difference between drug ratings
Scientific Literature
Scientific Literature+FDA
Labels
*significant

12. Validating Results
• 71% of online comparisons agree with literature and
FDA labels
• 82% for comparisons with at least 1 point rating
difference

13. Reviewer biases and insights
Reviewers give higher ratings to:
– Addictive drugs
– Alternative treatments

14. Addictive Drugs Rate Higher
Condition: Muscle spasm
Carisoprodol and cyclobenzaprine not significantly different
Clin Ther. 2004 Sep;26(9):1355-67.
Drug Rating % reviews with the word
“addict”
Carisoprodol 4.35 9.2
cyclobenzaprine 3.33 0
It's a great medication,
but can easily become
dangerously addicting.
so far this has been the best
medication to help give me
almost complete relief. Just be
careful using it, it is addictive.

15. Alternative Choice of Treatment
(the road less travelled)
Second-line treatments rate
higher.
Is it truly a better drug?
Or a preselected population
who are happy to have found
this drug because standard
therapy did not work?

16. Example: Underactive Thyroid
Thyroxine (synthetic thyroid hormone)
“official” approved drug
Online rating 2.22
Natural thyroid extract
Online rating 3.92

17. When the Generic is Better than the Brand-Name
• Pro-Air brand rated 2 stars lower than generic
• “Green”, environmentally-friendly design, cost more $
Online reviewers complained about “inhaler”
Generic had a counter with number of doses remaining, Pro-Air did not
Pro-Air later released new design

18. Conclusion
• Online drug reviews are useful, with caveats
Reviewers give higher ratings to
– Addictive drugs
– Alternative treatments
• Gain new insights like bad inhaler design