Wikipedia talk:Identifying reliable sources (medicine)

Uptodate.com

There has been mention previously about the us of Uptodate.com as a secondary source. Wondering what people thought are about structuring references to this source? I have started siting both Uptodate and the primary research article so people without access can look at the primary source.

Doc James (talk) 15:31, 16 July 2008 (UTC)

UpToDate is a good source, but the best sections on UpToDate heavily reference the actual medical literature. As much as possible, we should use UpToDate as a guide to which literature is relevant and how to weight it,but citing UpToDate directly is a bit iffier since it's essentially one expert's opinion. That said, the site has an excellent reputation for accuracy and is a widely used professional resource, so it's not a matter of accuracy, more of how we approach things. MastCell ^Talk 15:55, 16 July 2008 (UTC)

I think UpToDate is a good resource, but heavily restricted even to many editors here, making it hard to access.

Jmh649 has started this discussion specifically because UpToDate is one of the secondary sources that cites PMID 814892. Now if we could find another secondary source that cites that paper, then I think this source would be preferable to UpToDate. JFW | T@lk 17:16, 16 July 2008 (UTC)

Or you could just use that primary source in good faith, and refer anyone who questions you to uptodate.

Should we set up a template, like for {{eMedicine}} ? --Steven Fruitsmaak (Reply) 17:25, 16 July 2008 (UTC)

Just one data point. I'd never looked into uptodate.com, so I thought I'd check it by looking at its coverage of the epidemiology of autism, an area I've worked on in Wikipedia. Uptodate.com's first sentence on the subject is "The prevalence of ASD in the United States and other countries has increased since the 1970s and particularly since the late 1990s"[1]. But this claim isn't true, and the source they cite (Williams et al. 2006, PMID 15863467) doesn't make the claim. What the source says is that the reported prevalence has increased, but it is not known whether the actual prevalence has increased; it could be that the changes are due to changes in diagnostic criteria or other factors. Admittedly this is just one data point, but still, wow, that's a pretty basic error. Accordingly:

• I'd like to second MastCell's suggestion to use uptodate.com as a source of plausible citations rather than to cite uptodate.com directly.
• If there is some dispute about whether to cite PMID 814892 (a 30-year-old letter to the editor?) I wouldn't rely on uptodate.com to resolve it.

Eubulides (talk) 17:46, 16 July 2008 (UTC)

Following up on my own comment: I see that later text in uptodate.com does make the point that autism's bureaucratic prevalence is not the same as its true prevalence, so I was too critical: it's more a major editorial error in uptodate.com than an actual content error. And it is just one data point. Still…. Eubulides (talk) 17:51, 16 July 2008 (UTC)

That actual reference from Uptodate JFW is referring to is: Lee L, Kumar S, Leong LC (1994). "The impact of five-month basic military training on the body weight and body fat of 197 moderately to severely obese Singaporean males aged 17 to 19 years". Int. J. Obes. Relat. Metab. Disord. 18 (2): 105–9. PMID 8148923. {{cite journal}}: Unknown parameter |month= ignored (help)CS1 maint: multiple names: authors list (link) Doc James (talk) 20:31, 16 July 2008 (UTC)

Mea culpa, dropped a digit. I have refactored your comment to include the citation rather than the entire abstract. A primary research study in 197 subjects remains problematic. JFW | T@lk 21:12, 16 July 2008 (UTC)

In follow up I would like to mention that the article from Uptodate was used to emphasis a point made by the systematic review. That the amount of exercise has an effect on the degree of weight lose. It was not used to "debunk" the systematic review. See page on obesity.

Doc James (talk) 15:25, 17 July 2008 (UTC)

uptodate is peer reviewed - here is what they say about there process:

Editorial policy

Authors

All topics in UpToDate are written by the listed authors in conjunction with a deputy editor. Authors are identified as experts by the Editors-in-Chief, our editorial staff, and the participating societies. Exceptions are guidelines from major societies, which are added to UpToDate in their original form. All material is originally prepared by the contributing author(s) whose name(s) and affiliation(s) appear in the upper left corner of each topic. This material is reviewed extensively by our physician editors and peer reviewers for accuracy and completeness of the literature search, and for consistency with all aspects of the editorial policy.

Doc James (talk) 17:53, 20 July 2008 (UTC)

There is no doubt that UpToDate is peer-reviewed, but it remains expert opinion in review form with an uncertain selection process. The content is also not permanent and the same URL may lead to content that is liable to change. If there is a recent review available in a "real" journal this is vastly preferable over UpToDate.

What does the panel think about using UpToDate as a "hidden" supportive source. For instance, a primary research study is cited on UpToDate, thereby confirming its relevance, so the editor who includes that reference makes a comment in the reference code (not visible to the general reader) that the reference is cited by UpToDate? Thereby we satisfy the demands of this policy without overburdening the reader with a resource they are unlikely to be able to access even by interlibrary loan? JFW | T@lk 07:47, 21 July 2008 (UTC)

Hidden sources give me the willies. Wikipedia should be open about its sources, even if the sources themselves are closed. Eubulides (talk) 07:58, 21 July 2008 (UTC)

I don't understand how this is really "hidden". We are actually citing the primary source, and justify its importance under the "reviews preferred" aspect of WP:MEDRS by saying that it was cited by UpToDate. So long as you then actually read the primary source, to meet WP:CITE#SAYWHEREYOUGOTIT, then I have no problem with this. WhatamIdoing (talk) 20:48, 22 July 2008 (UTC)

Sources the ordinary reader can't see are hidden to the reader. If there's a reliable review, and the reason we're citing the primary source is that the reliable review cited it, then we typically should be citing the reliable review and not the primary source. This is true regardless of whether the review is published in uptodate.com or elsewhere. There are some exceptions to this (for example, notable primary sources worthy of discussion directly in the article text), but the exceptions shouldn't be common; and even then, it's better to cite recent reviews in addition to citing the primary source directly. Eubulides (talk) 23:03, 22 July 2008 (UTC)

RSN that may interest some editors here

I have revived a question at RSN on pseudonymous authors of "letters to the editor" in otherwise reliable sources. It may interest some editors here, and as the only editor who supports this source is demanding responses from additional editors to demonstrate "consensus", I am looking for editors who are willing to read it and respond.

Note that I don't have a vested interest in the outcome; I just want it very thoroughly settled one way or the other. WhatamIdoing (talk) 18:06, 4 August 2008 (UTC)

Process for guideline status?

Does anyone know what the proper procedure is to have this adopted as a guideline? WhatamIdoing (talk) 17:27, 5 August 2008 (UTC)

Huge can of worms and pet peeve extraordinaire alert. There isn't one, there should be, there is no coordination, but some editors are vehemently opposed to any efforts to centralize and coordinate MoS. In terms of history, we (MEDMOS) were actually the only WikiProject to be subjected to a process. In the olden days, Radiant! tagged our page (historical, I think), and we had to jump through hoops to get added to MoS. We posted to several dozen other Projects, the Village Pump, etc. to garner widespread consensus for our guidelines. To my knowledge, no other MoS page has ever had to do that. When I proposed something similar at MoS, it was rejected. Summary: we have a mess at MoS where a few editors can put together a page and call it a guideline with no mechanism for gathering broad consensus. And I'm supposed to make sure FAs are MoS-compliant, per WP:WIAFA crit 2, even though guidelines often contradict each other. Back when I proposed this, someone started a MoS WikiProject, which was promptly ignored. I could find all of these links if I had to, but it would take me a very long time. Ugh. SandyGeorgia (Talk) 17:32, 5 August 2008 (UTC)

By the way, the place to get our reliable sources text included would be at WP:RS, but it was already moved out of there, back during the old WP:ATT debacle, so that is yet another can of worms alert. SandyGeorgia (Talk) 17:34, 5 August 2008 (UTC)

I'm sorry things are such a mess. So, Sandy, what would make you (and therefore the FA process) happy? Would you like me to declare this a guideline on the grounds that the editors of this page largely agree that other people ought to follow our excellent advice? Would you like me to pretend that such a thought never crossed my mind? Would you like me to invent an "RfG" process for you? WhatamIdoing (talk) 17:51, 5 August 2008 (UTC)

Should we make this a guideline?

I find the arguements for making this a guideline compelling. Whilst some sensible concerns raised, overall the WP:MEDRS acts as a additional help to the main WP:RS, giving pointers & issues that may apply in biomedical topics. Of course any specific review article might be found wanting, this can also occassionally apply to primary source research papers which may be questioned with authors & journals later ammending or even rarely retracting. Clearly therefore one can not be absolutist, and so this is a guideline and not a policy with header instruction that it be "treated with common sense and the occasional exception". In general therefore RS prefers secondary sources to primary in outlining a topic, but MEDRS seeks to give a little greater creadance and advice on citing the primary sources for the origin of items of knowledge.

As RS states "In science, single studies usually are considered tentative evidence that can change in the light of further scientific research. How reliable a single study is considered depends on the field, with studies relating to very complex and not entirely-understood fields, such as medicine, being less definitive. If single studies in such fields are used, care should be taken to respect their limits, and not to give undue weight to their results. Meta-analysis and systematic reviews, which combine the results of multiple studies, are preferred (where they exist)."

I see no direct objection above that common sense is required in looking at how any one source (review article, primary research or indeed a "standard" textbook) compares to the bulk of other sources - for WP:UNDUE weight needs to be considered for views that do not follow the majority viewpoint if we are to maintain WP:NPOV (in essence a review article or text book that presents a view not held by the majority is in a minority whatever type of source it is). That said, a citable-reference in a reliable-publication-source that tries to summarise an overall topic and its current research-base, is likely to assesss the importance or otherwise of individual primary sources better than any editor (who so summarising themselves would be at risk of WP:Synthesis which is original research).

Therefore I promote this MEDRS from proposal to an accepted guideline in light of satisfactory consensus above. This does not of course preclude the guidence to be discussed below and, hopefully, improved further :-) David Ruben ^Talk 00:53, 1 September 2008 (UTC)

As an editor, I have had a 1st go at adding a {{nutshell}} header - feel free to discuss and tweak that David Ruben ^Talk 00:53, 1 September 2008 (UTC)

Thanks David. I think this is a major step forward. I hope we can settle some final disagreements (below) pretty soon now. JFW | T@lk 05:26, 1 September 2008 (UTC)

Systematic reviews and primary sources

This edit removed the following sentence from Wikipedia:WikiProject Medicine/Reliable sources #Article type without discussion:

A systematic review is both a primary and secondary source—it summarizes other papers but it does so in order to research the field and possibly come to a novel conclusion.

with the comment "this is really not true -- a good systematic review basically summarizes previous findings only. Looser reviews do the more primary work." Possibly-controversial edits like this should be discussed on the talk page first, so I took the liberty of reverting the change and moving the text here, for discussion. Eubulides (talk) 02:20, 15 August 2008 (UTC)

I actually support removal of this sentence. In the vast majority of cases, systematic reviews make no new observations at all. Non-systematic (or "narrative") reviews occasionally reflect expert experience without an evidence base, but they remain reviews in their editorial sense. JFW | T@lk 02:22, 15 August 2008 (UTC)

I honestly didn't think that removing this sentence could possibly be controversial, as it seemed to be superflous in addition to being false. I thought about simply cutting the word systematic from the sentence, but I figured that might be controversial! II | (t - c) 02:42, 15 August 2008 (UTC)

I also support the removal of that sentence since it differs from the definition of the term in the article at systematic review and from the use of the term in the lead paragraph of this page. --Jack-A-Roe (talk) 02:57, 15 August 2008 (UTC)

The sentence in question has been present in nearly its current form since the talk page was created in 2006. It was added in the very first change to the article, by Colin, the original author of the page, here. I'll leave a note on Colin's talk page to ask him to weigh in on this issue. Eubulides (talk) 05:31, 15 August 2008 (UTC)

Feel free to correct my mistakes -- I'm just an amateur. I bow to the folk with training in this subject! I wrote that a while ago and might have worded it differently now. I've always hoped, in fact, that someone with training/qualifications in this area would rewrite much of MEDRS, while keeping those aspects that are unique to WP. Re-reading Greenhalgh's How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses), I can see how I thought the "novel conclusion = primary" might occur with such papers. If you guys think that is rare then clearly the sentence isn't useful. Here's my impression: A narrative review by an expert might be likely to have a conclusion that the expert already "knew" before he even started working on the paper. He's set out to describe rather than systematically discover the current situation. A systematic review is an objective exercise in examining the published research and the results might surprise the author(s). Such a review might gain access to the raw data from published trials, and other less accessible input (see the "Checklist of data sources for a systematic review" in Greenhalgh's paper). If the review contains a meta-analysis or other synthesis of results, then arguably it produces its own conclusion (even if this is similar to and adds weight to what was "known" already). The publication of a Cochrane review can make the newspapers, so it must be saying something we haven't heard before or with as much conviction. Still, I agree that it may be simpler to just consider such reviews to be excellent secondary sources. Colin°^Talk 18:33, 15 August 2008 (UTC)

Thanks for following up. In rereading the article it seems clear to me that something needs to be said here about systematic reviews, as systematic reviews are mentioned in the lead and therefore need to be in the body as well. How about if we remove the phrase about primary and secondary source, along with a newly-redundant phrase, as follows:

A systematic review is both a primary and secondary source—it summarizes other papers but it does so in order to research the field and possibly come to a novel conclusion.

Eubulides (talk) 20:09, 15 August 2008 (UTC)

Eubulides, edit away and expand if you can. Please, the more you can add to this proto-guideline the better IMO. Colin°^Talk 20:21, 15 August 2008 (UTC)

OK, thanks. For now the consensus seems to be that this change is OK, so I put it in. Expanding will take more work.... Eubulides (talk) 18:57, 22 August 2008 (UTC)

I support the direction you seem to be taking with your edits above. In my mind systematic reviews are not given their due. They should not be lumped in with primary sources, or with summary reviews of the literature that frequently precede the presenting of original research in a journal article. The latter is often self-selected to support the hypothesis the researcher is evaluating, but I frequently see these reviews (often in abstract form) used as a secondary source. —Mattisse (Talk) 20:32, 15 August 2008 (UTC)

Hmm. I don't oppose the addition of that sentence, but I think it is bit superflous. Systematic review is wikilinked in the lead. We don't really need define it in MEDRS. Incidentally, Mattisse, I actually previously supported emphasizing systematic reviews, but if I recall correctly JFW wasn't really happy about that(not sure what Eubulides thoughts on the matter were). Then again, the page on systematic review already states that these are the standard in evidence-based medicine, which works just as well. Actual medicine guidelines > MEDRS in my mind. I actually think modularity in these articles is best -- keep the things which are strictly MEDRS here, and fork off tangents and definitions elsewhere. And, no offense, but that sentence is a "duh" sentence. It's a "duh" statement that in most cases, reviews will synthesize the literature and come to their own, possibly novel, probably not novel, conclusion. If you want to add that sentence, you're going to have to at least strike the word systematic from it, Eubulides. II | (t - c) 19:07, 16 August 2008 (UTC)

I have never opposed the use of systematic reviews, and indeed I feel that we should place much emphasis on them. But they tend to be relatively narrow in scope and broad in methodological thoroughness. Hence, "narrative reviews" are great for broad sketches, while systematic reviews are better at answering specific questions. JFW | T@lk 21:56, 17 August 2008 (UTC)

• There seems to be some confusion here, as the proposal is to remove some text, not to add it.
• Removing the entire systematic-review sentence would be unwise, as the containing paragraph discusses all sorts of article types, and discussing other article types while omitting mention of systematic reviews (a preferred source) would be very odd.
• It's not a "duh" statement to define a systematic review; we can't expect all Wikipedia editors to know what it is. The paragraph already defines general reviews, and it's useful for it to make a point about systematic reviews and what makes them different. If you prefer a different definition for "systematic review", please propose something.

Eubulides (talk) 23:12, 16 August 2008 (UTC)

I don't think it needs to be defined, and I don't think they come to novel conclusions. Generally a systematic review will have one of two conclusions: 1) Not enough evidence, 2) Enough evidence, some consistent result. The paper pointed by Colin has good summary of them, and I think it would be nice to actually reference that paper in this article. A good definition would note that a systematic review uses a predefined, replicable methodology to select all articles from a certain time period, with predefined inclusion criteria. II | (t - c) 01:15, 18 August 2008 (UTC)

WP:MEDRS #Article type defines/discusses general reviews (which are mentioned in the lead); for consistency shouldn't it also define/discuss systematic reviews (also mentioned in the lead)? It would be odd for the lead to mention a topic not discussed in the body. The current (and proposed) wording doesn't say that systematic reviews always come to novel conclusions, only that they "possibly" come to novel conclusions. Again, if you could propose specific wording to replace the proposal above, that would help drive the discussion forward more quickly. Eubulides (talk) 19:42, 18 August 2008 (UTC)

Although general (journalistic or narrative) reviews are a good introduction to a topic, they should be used with a consideration of the sources cited by the author. When citing information on the effectiveness of certain therapies, consider looking for a systematic review. [Alternative: Systematic reviews are the considered to be the gold standard in evidence-based medicine in regard to complex medical questions.] A systematic review uses a predefined, replicable methodology to select all articles from a certain time period in order to answer a specific question. This it thought to reduce bias and increase thoroughness, since articles which do not support the researcher's preconceived conclusion must be included if they fit the inclusion criteria.

— [2]

II | (t - c) 20:06, 18 August 2008 (UTC)

I assume this is intended to replace the last 3 sentences of WP:MEDRS #Article type? If so, the proposed replacement has two major problems. First, it disses general reviews far too much. General reviews, when done well, are better sources for Wikipedia than systematic reviews are, since they match Wikipedia's encyclopedic aims better. Second, WP:MEDRS #Article type is descriptive: it does not (and is not intended to) give advice. Advice is in WP:MEDRS #Some definitions and basics and proposed changes to advice should be made there. Eubulides (talk) 20:13, 18 August 2008 (UTC)

Sorry, it is intended to replace the last sentence only. I tweaked it slightly. General reviews are only as good as the sources (or perhaps the credentials) of the author. If the review cites only one primary study for a claim, its reliability is not much better than that primary study. I don't understand how general reviews are more encyclopedic. They are certainly not more reliable than systematic reviews. As far as "advice", I think that's your opinion. This page is a guide, and I think it should read as such. But if you prefer, the second sentence can be changed to "" Any other advice? II | (t - c) 20:28, 18 August 2008 (UTC)

(Undent.) As discussed previously, editorial judgement is advised when selecting sources and citing their contents. But secondary sources will generally trump primary sources for most citations, and I don't think we should try to codify for exceptions. I dispute the claim that systematic reviews can answer complex medical questions; in the vast majority of cases, the data is so insufficient or contradictory as to allow for the drawing of definite conclusions about complex medical questions. Back to you, ImperfectlyInformed. JFW | T@lk 20:50, 18 August 2008 (UTC)

Yes, and here we go again. I'm not codifying for exceptions. I'm just pointing out that some attention should be paid to the sources cited by a review. That's probably the most important part of editorially analyzing reviews. As far as systematic reviews actually answering the questions, perhaps you're right. But they are considered to have the best chance at answering these types of contentious questions, and they are the best at putting the question into proper context, since they attempt to include all context. II | (t - c) 21:17, 18 August 2008 (UTC)

I think we're actually much in agreement regarding the place of various types of publications. In a world full of evidence, systematic reviews (especially of the Cochrane flavour) would answer any question. JFW | T@lk 21:31, 18 August 2008 (UTC)

agreed. Systematic reviews of this sort are in fact the goldstandard and the very model of an authoritative secondary/tertiary source. ; the Cochrane series are not the only ones--there are also other formal consensus statements issued by appropriate groups. When they don'ta gree, which is not unheard of, w need only present the different ones equally. I wish other fields were as well organized for such things. DGG (talk) 04:14, 19 August 2008 (UTC)

I agree with Eubulides and JFW that the proposed text disses general reviews too much. I don't really know what the "should be used with a consideration of the sources cited by the author" is supposed to add. Many editors won't have the ability/experience/training to judge whether the review's sources are balanced/good (and so just have to trust that a review published in a good journal is probably good). Those that are capable of judging the quality of a review probably don't need to take advice on this matter from Wikipedia! I wonder if II you are thinking too much about arguing about the efficacy of therapies rather than trying to write a rounded encyclopaedic article on a subject. A good general review provides far more than just an "introduction to a topic". In the ketogenic diet, I was able to source whole paragraphs to narrative review sources such as Clinical Aspects of the Ketogenic Diet. I used two systematic reviews: Ketogenic Diet for the Treatment of Refractory Epilepsy in Children: A Systematic Review of Efficacy and Ketogenic diet for epilepsy (the former funded by US insurance companies in 2000 and the latter from the Cochrane Database of Systematic Reviews 2003). Both answered only one question, "does it work", and both said "weakly yes, but there are no randomised controlled trials" and are dated since such a high quality trial was finally published in 2008 (PMID 18456557). Both those systematic reviews read much like a primary research paper (background, methods, results, discussion, conclusion) and are so focussed that they aren't any use for the rest of the WP article.

Guidelines

I agree with DGG about the "other formal consensus statements issued by appropriate groups". Currently, MEDRS doesn't mention clinical guidelines. Two excellent sources I have used are the guidelines for Scotland and England and Wales. I know less about the US but I have seen consensus statements begin published (self or in journals) by various medical professional bodies and by some of the more respectable charities. Could we add some mention of this to MEDRS? Colin°^Talk 08:28, 19 August 2008 (UTC)

Guidelines tend to be limited in geographical scope. NICE does not pretend that its guidelines are valid outside England and Wales, and therefore not everything recommended by that body is therefore necessarily representative of practice in the rest of the world. An example would be the risk stratification algorithm for warfarin in people with atrial fibrillation; this is a novel system not used elsewhere. Similarly, its guidelines on computed tomography of the brain after head injury may differ substantially from similar guidelines elsewhere. As a result, I prefer to use reviews in preference over guidelines; that said, they can be excellent sources of information (e.g. the British Society of Gastroenterology guideline at coeliac disease). JFW | T@lk 18:49, 19 August 2008 (UTC)

• I like the idea of mentioning guidelines, but agree with JFW that the area's a bit tricky. In Chiropractic I've had to deal with the problem that there are three competing groups of guidelines for treatments used by chiropractors: one by the majority "mixer" chiropractors, another by the minority "straight" chiropractors, and another being the mainstream medical guidelines. The three groups differ markedly in their results. Even if we restrict ourself to mainstream medical guidelines developed by national governmental organizations, guidelines in different countries disagree markedly; see Murphy et al. 2006 (PMID 16949948).
• My experience is that guidelines tend to be less useful than reviews as sources to Wikipedia articles than reviews are, not necessarily because they are lower quality, but because their goal (advice about treatment) is more distant from Wikipedia's goal (encyclopedic coverage) than the goal of reviews (broad and balanced coverage of a topic). Though of course guidelines often have good reviews within them.

Eubulides (talk) 19:19, 19 August 2008 (UTC)

Guidelines must be carefully evaluated. An institution like the Mayo Clinic may offer "guidelines" meant for the general public without any attempt to source or valid the claims. —Mattisse (Talk) 20:03, 19 August 2008 (UTC)

Again, editorial judgement is advised. The NICE guidelines, too, are sometimes expert/experience based rather than supported by evidence because such evidence is often lacking. Also, Matisse, we are talking about professional guidelines here, adherence to which is often mandatory and a yardstick by which professional performance is meausured, as opposed to resources for the general public that cannot legitimately be called "guidelines" but rather "advice" or "tips". JFW | T@lk 20:19, 19 August 2008 (UTC)

Well, psychology articles are notoriously awful, but these sorts of references are often used in articles on topics in psychology. I would like to see a firm way of discouraging it. —Mattisse (Talk) 20:23, 19 August 2008 (UTC)

Sounds reasonable. JFW | T@lk 20:35, 19 August 2008 (UTC)

I have to agree with matisse, some of the guidelines I have seen put out by otherwise reputable bodies (stocked with professionals) have been pretty poor at times. Maybe a good caveat WRT secondary sources is be especially careful if one finds two or more sec. sources putting forth substantially variant positions/findings/information etc. Cheers, Casliber (talk · contribs) 22:29, 22 August 2008 (UTC)

Funny one I found - in australia (and apparently America), screening tools are fait accompli these days and promoted as such, yet the data seems to tell a different story..actually there was another BMJ paper which was even more cautious. Cheers, Casliber (talk · contribs) 22:34, 22 August 2008 (UTC)

Conflicting advice?

Hi - I apologize for making a few late edits to this proposed guideline, but I trust I'll be reverted if they're out of line. In any case, I wanted to raise what looks like conflicting advice. We emphasize the importance of secondary sources as the basis for medical articles - great. But in the section headed "In science, avoid citing the popular press", we tell people to cite specific researchers' papers rather than newspaper coverage of them. I understand where this is coming from, but we seem to be telling people to go to primary sources in preference to a good popular-press account, which seems a bit iffy. I'd prefer to re-title the section to "Use the popular press with caution" rather than "Avoid the popular press", and perhaps to indicate that popular press can be useful to adduce the sociological and cultural aspects of a disease, drug, etc. For instance, (quality) popular-press accounts are pretty much essential to a full understanding of the context around Lyme disease or Vioxx. "Avoid" just seems to strong. Thoughts? MastCell ^Talk 20:28, 27 August 2008 (UTC)

I'd agree with that. There are sometimes relevant stories covered in the op. press...I just need to think of an example now....maybe a medication taken in a notable court case, or erson diagnosed with a particular condition. etc. Cheers, Casliber (talk · contribs) 20:37, 27 August 2008 (UTC)

I agree also. The indiscriminate use of primary sources (small n, a few case histories, etc.) even from PMID to "prove" a general point is more of a problem, in my opinion, then using reputable popular press references which can be presented for what they are, a popular press review. —Mattisse (Talk) 21:09, 27 August 2008 (UTC)

Press reports are usually unsuitable as secondary sources. Here in the UK every rat study on beta amyloid is reported in the Daily Wail with huge headlines: "Dementia cure round the corner". JFW | T@lk 22:56, 27 August 2008 (UTC)

Well, the Daily Mail is certainly not a great source. Who was it that said they classify every substance known to man as either causing or curing cancer? :) I do think there's a role for high-quality popular press, like Scientific American or New Scientist (though I still haven't forgiven the latter for their inexcusable DCA hype, it's out of character). The New York Times also tends to have decent to excellent science and health coverage. MastCell ^Talk 23:05, 27 August 2008 (UTC)

Agree that whoever was behind the DCA garbage was hopefully and unceremoniously sacked. In other words, it does again boil down to editorial judgement on which popular press source is to be regarded as reliable. JFW | T@lk 23:18, 27 August 2008 (UTC)

It might be useful to list a few examples of legitimate uses for a popular media, such as reports on court cases, as Casliber mentions. Business/marketing issues, information about drug approvals, all sorts of scandals -- there's a lot of not-directly-scientific stuff that goes into some medicine-related articles. WhatamIdoing (talk) 00:14, 29 August 2008 (UTC)

Like MastCell, I'd like to re-title this section. I think the main point to make here is that it's better to cite a study directly (and correctly) than to cite a mass media summary of a study. More links in the chain = more opportunities for errors. Also, quite a lot of what we need in basic medical articles isn't going to be available in a good newspaper. (When was the last time you saw a good anatomy article in The London Times?)

Perhaps more importantly, I think it should be expanded to add one sentence about things that mainstream media does better than scientific journals, such as the non-scientific end of things. It's not really "don't cite the popular press" -- it's really "don't cite the popular press for medical/scientific details. WhatamIdoing (talk) 00:51, 31 August 2008 (UTC)

Popular press and primary sources

This edit altered WP:MEDRS #Some definitions and basics to say that most articles and press releases are primary sources in medicine. My experience is just the opposite: in medicine the primary sources are the research articles, and the popular press and press releases comment on the research articles. I reverted the change, and perhaps we can discuss the situation further here. Eubulides (talk) 01:40, 29 August 2008 (UTC)

Press releases are usually issued by companies or organizations soon after the publication of a scholarly article or even before such an article. They are usually a shortened or popularized version of the scholarly article without analysis, deeper interpretation or review of other relevant sources, which is a necessary pre-requisite for it being a secondary source. For example, these press-releases (like most press releases) are clearly primary sources: [3], [4], [5], [6] Please provide more evidence of your belief that press-releases are secondary sources. Paul Gene (talk) 02:11, 29 August 2008 (UTC)

A press release is a press release: it is what it is. We don't need to say anything specific about them on this page; that is for WP:V and WP:RS. SandyGeorgia (Talk) 02:13, 29 August 2008 (UTC)

Sandy, do you suggest deleting the mention of press releases from the definition of secondary sources? I would support that. Paul Gene (talk) 02:42, 29 August 2008 (UTC)

No, I think we've got it just right as is. We're in line with WP:PSTS and anything beyond that should be dealt with on main policy pages. SandyGeorgia (Talk) 02:51, 29 August 2008 (UTC)

Sandy, press releases are not mentioned in WP:PSTS. Insofar as the definition of secondary sources on WP:PSTS goes: "Secondary sources are accounts at least one step removed from an event.[3] Secondary sources may draw on primary sources and other secondary sources to create a general overview; or to make analytic or synthetic claims.[4][5]", most press releases do not fit it. Please see the examples of typical press-releases I gave above. Please provide more evidence of your belief that press-releases are secondary sources. Paul Gene (talk) 11:00, 29 August 2008 (UTC)

Eubulides, your opinion that news articles constitute secondary sources goes against the prevailing expert consensus. According to this definition,[7] "In science, secondary sources are those which simplify the process of finding and evaluating the primary literature. They tend to be works, which repackage, reorganise, reinterpret, summarise, index or otherwise “add value” to the new information reported in the primary literature." Most of the articles in the popular press tend to simply report the news, with the information usually taken directly from a press release or an abstract. In addition, they are contemporaneous with the object/phenomenon, which is one of the distinctive characteristics of a primary source.("A secondary source is a work that interprets or analyzes an event or phenomenon well after the fact. It is generally at least one step removed from the event." [8]) This source [9] directly lists mass media as a primary source: Examples of primary sources: ... Mass media ... The products of the mass media can be primary source documents if they were produced at the time of the events or phenomena in question. Examples are: newspaper and magazine articles". Another source [10] also states Some examples of primary sources include:...Contemporary magazine and newspaper articles." I was not able to find any support for your argument that, unlike in historical sciences, in medicine the news articles are secondary and not primary sources. Paul Gene (talk) 02:45, 29 August 2008 (UTC)

Wiki's definitions are at WP:PSTS; we are in line with them, which is where we should be. SandyGeorgia (Talk) 02:51, 29 August 2008 (UTC)

In other fields, e.g., political reporting, newspapers are indeed primary sources. For example, to report Barack Obama's position on the educational vouchers, Political positions of Barack Obama cites a Chicago Tribune article about a 2004 debate; this was a primary source on that debate. In contrast, in medicine, typically refereed journal articles are the primary sources, and press releases and/or newspapers report about the journal articles, adding commentary about their significance; these are secondary sources. For example, Morrow et al. 2008 (PMID 18621663) published in Science a study about homozygosity mapping to identify shared genes in autism, and that same day a story about the study appeared in The Times. The Times story was a secondary source: it commented about the study, with quotes like "This publication a big event in the world of autism research", without directly accessing the original data that the study reported. There may be occasions where a newspaper would be a primary source on a medical topic, but these would be rare, I'd think; not worth mentioning or emphasizing here. Eubulides (talk) 03:27, 29 August 2008 (UTC)

Sandy and Eubilides, you are not in line with the definition of secondary sources at WP:PSTS: "Secondary sources are accounts at least one step removed from an event.[3] Secondary sources may draw on primary sources and other secondary sources to create a general overview; or to make analytic or synthetic claims.[4][5]" As I already mentioned, most of the stories in pop press are news articles, they are contemporary and thus are not sufficiently removed from the event to count as secondary sources. They generally do not provide analysis based on several other primary sources or create an overview of the topic. Many news articles in the political area often make comments similar to the ones made in the science Times story you are referring to. For example, in another news story http://www.timesonline.co.uk/tol/news/politics/article4615384.ece] Times comments: " The commission’s push for a national register, however, could be more controversial.", without directly accessing the original commission's report. Political reporting in the pop press in that respect is not any different from scientific reporting. Nevertheless, as the thrust of the press reports is on the news they are firmly considered to be primary sources. In order to re-define the pop-press reports of scientific news as secondary sources you have to provide the evidence that they are sufficiently different from other reporting. You have to back that up with the quotations showing that scientific reports in pop-press are considered by experts to be secondary sources, otherwise you are engaging into original research. In what I was able to find so far on the expert definitions of secondary sources as applied to science, the pop-press reports appear to be firmly ruled out (see the definitions in my previous post). Paul Gene (talk) 11:00, 29 August 2008 (UTC)

Whether we regard press releases as primary or secondary isn't the most important thing we can say about them. The vital point is that they are a biased source of information written at lay level which may not contain enough contextual information or balance of viewpoints in order to write a good encyclopaedic article. If the press office for hospitals/research labs are writing about newly published work, they are often effectively writing the journalist's article for them, so would be considered secondary sources. If they are reporting on unpublished data, then they are more like primary sources. If they contain quotations from the researchers, as though they had been interviewed about it, then the are also more like primary sources for that detail. Rather than trying to divide all published material cleanly into primary and secondary, it may be better to avoid the issue with certain sources, and concentrate on their strengths and weaknesses without classification. Colin°^Talk 14:04, 29 August 2008 (UTC)

I agree with Colin's points. I looked at WP:MEDRS and its text currently does what it says: it does not say that the popular press or press releases are primary sources, or secondary sources. Perhaps some minor change to WP:MEDRS would help clarify this issue (to avoid further questions along these lines), but offhand I don't see what change that would be. Eubulides (talk) 18:26, 29 August 2008 (UTC)

I agree with Sandy: WP:MEDRS's version of PRIMARY just needs to line up with the normal WP:PRIMARY, while providing some useful information about how it might apply to a medicine-related article.

Press releases are difficult to assess in the PSTS model. A press release from the FDA about fining the American Red Cross Blood Services for long-standing cGMP problems is probably a primary source. If the American Red Cross Blood Services issues a press release whining about it, then their press release is a primary source for their response, but probably a secondary source for the fact of the fine. If a third organization issues a press release to chide the FDA for dragging their feet, then the third press release is a primary source for their opinions, but definitely a secondary source for the fact of the fine.

None of that changes the fact that none of these are issued by independent third-party sources with reputations for fact-checking (something that is mentioned in WP:V because it's a relevant legal standard). WhatamIdoing (talk) 19:46, 29 August 2008 (UTC)

• Sandy, what is your problem with deleting the questionable paragraph and just leaving it up to WP:PRIMARY to deal with? I thought that was essentially what you suggested. WhatamIdoing also agrees that press-releases are difficult to assess in the PSTS model. Colin also says that "Whether we regard press releases as primary or secondary isn't the most important thing we can say about them.". Eubulides says "I looked at WP:MEDRS and its text currently does what it says: it does not say that the popular press or press releases are primary sources, or secondary sources." Remember, we should not engage into original research and re-write conventional definitions of secondary and primary sources. No examples of expert support of the questionable definitions has been given in this discussion in spite my repeated requests and questions.Paul Gene (talk) 00:53, 30 August 2008 (UTC)

The text you deleted has so far, been questioned only by you. It is stable text, and has been in the article in its current form for over a year. It is not a definition; it is advice from experienced Wikipedia editors. It does not say that press releases or the popular press are secondary sources; it says that the secondary source material in press releases and the popular press tends to be less useful than the other secondary sources mentioned (e.g., refereed articles in medical journals). This is widespread consensus among experienced editors of medical articles. Certainly I find it to be true: for example, this article in the Washington Post is far less useful than this article in the Annual Review of Public Health; they are both dated 2007 and they both talk about the epidemiology of autism, but the medical-journal review article is way, waay, waaay more informative and reliable. Eubulides (talk) 01:58, 30 August 2008 (UTC)

Eubulides,

(1) It is not true that Paul Gene is the only one who questions (actually, objects) to that text. I also object to it.

(2) I've known Ed Edelson for many years, since he was science reporter for the New York Daily News, and I'm familiar with his work. I think this Washington Post story on the rise in autism is a good, useful story. It's a good report of what experts said at a meeting, and it gets a range of authoritative views. You have not expressed any objection to it on the merits. You have merely used it to support your personal opinion that you don't like newspaper medical stories.

You don't define "useful." The Washington Post story is free on line. The Annual Reviews article is not. Unless Wikipedia editors have a subscription to Annual Reviews, they can't access it. That violates WP:VERIFY. A source that is inaccessable and cannot be verified is not useful.

You have provided no evidence that newspaper articles are less "useful" than academic reviews. In fact, you've provided evidence against it. I propose, based on your own comparison of Washington Post to Annual Reviews, that newspapers do provide useful material, and therefore we should delete the incorrect text as Paul Gene did.Nbauman (talk) 18:10, 30 August 2008 (UTC)

Your claim that offline and non-free sources fail WP:V is nonsense. There is not one word in that policy that says we cannot cite journals that would require either a paid subscription or a trip to the library. WhatamIdoing (talk) 00:23, 31 August 2008 (UTC)

The question is whether newspapers are "useful". Eubulides doesn't define the term "useful", and he has not cited anything in the Washington Post story that is wrong, nor given any reason why it or similar articles should not be used in a WP article. I think it's clearly useful, in the ordinary sense of the term. I would like to see Eubulides's specific criticism of the article. So far, he hasn't given us one.

I agree that peer-review subscription-only articles should be used, and are usually more reliable, although that has the problems of WP:LINKS Sites requiring registration. The question is whether newspapers should also be used.

I would like to see a good, logical argument for why newspapers should not be used. So far, nobody has given us one. Nbauman (talk)

While we're wishing, I'd like to know how many times WP:EL has to state The subject of this guideline is external links that are not citations of article sources and This guideline does not apply to inline citations, which appear in the "References" or "Notes" section and A site that requires registration or a subscription should not be linked unless...[it] is being used as an inline reference (and so forth) before editors will realize that it does not apply to references. May I encourage you to read both paragraphs in the section you linked? WhatamIdoing (talk) 20:56, 31 August 2008 (UTC)

Newspapers

Journalists, writing in the popular press, and corporate press releases tend to provide less useful secondary source material. Such material may be appropriate for inclusion in some contexts.

As a journalist, I disagree with that statement, and I agree with Paul gene that "This defintion is questionable and OR. You have to present evidence in its favor before including it.

I'd like to know why somebody made deletions in the following paragraph. I didn't see anything about this in the history or my watch list, so I didn't realize it had been deleted.[11]. This is a WP:RS authoritiative academic review of the quality of newspapers, as distinct from an editor's personal opinion.

Newspapers

The quality of newspaper coverage of medicine ranges from excellent to irresponsible, and they should be verified like any other sources. Even peer-reviewed journals like the New England Journal of Medicine cite articles in newspapers like the New York Times and Wall Street Journal. Some tabloids, like the New York Daily News, have a reputation for careful fact-checking and knowledgeable reporters; others do not. Newspapers should be judged on the facts, not on prejudices. Some academic organizations that evaluate news coverage are Health News Review (U.S.), Media Doctor (Australia), and Media Doctor (Canada). The British Medical Journal reviews U.K. media coverage.

Nbauman (talk) 01:24, 30 August 2008 (UTC)

The material in question was deleted in this edit with the comment "keep it simple", a sentiment I tend to agree with. This should be a guideline, not an academic paper studded with citations. Besides, the longer text is self-contradictory: "Newspapers should be judged on the facts, not on prejudices." collides with "Some tabloids, like the New York Daily News, have a reputation...." Let's just keep it simple and say "quality newspapers". Eubulides (talk) 01:58, 30 August 2008 (UTC)

We've been round this debate already. See the archive. Newspapers simply aren't designed to be source material for medical or science articles in an encyclopaedia. I really struggle to understand how you think they are. And for anyone who thinks the UK press should be used as a source for medical facts I suggest you read today's article by Ben Goldacre in The Guardian. Colin°^Talk 12:56, 30 August 2008 (UTC)

Wikipedia should be simple, but we must follow WP rules of verifiability, WP:RS and WP:NPOV.

The statements about newspapers in the present articles are WP:OR, personal opinion, unsourced and not verifiable. The deleted text is an academic and professional consensus, supported by verifiable WP:RS and academic research and authority.

We did have a debate about this before. Your argument was to cite selected examples of newspaper stories that, in your opinion, were wrong in some way. My argument was to cite international academic studies such as Health News Review, which you deleted.

In Wikipedia, the personal opinion of one or a few editors can't prevail against verifiable WP:RS and WP:NPOV.

Whether you like it or not, Wikipedia editors use newspaper articles extensively to write and source WP articles. If you read the daily newspapers, and also the medical journals, as I do, you can see that the medical facts in newspaper stories are often more accurate than the facts in some journal articles, as the medical journals themselves often admit in retrospect (for example the NEJM articles on VIOXX). So if you accept major peer-review articles as reliable sources, you must accept their conclusions that newspapers can be reliable.

Since Wikipedia editors do use newspapers, this guideline should give them specific guidance about how to evaluate newspaper stories -- a vague statement about "quality newspapers" is useless, since it merely transfers the argument to whether a given newspaper is "quality." Why not tell them how to use WP:RS to decide for themselves whether a newspaper is quality?

Our responsibility here is to elaborate on WP:RS#News_organizations for medicine. Medicine is a specialty, with specialized rules, and particularly complicated information sources. We should give them useful, helpful guidance, from available academic sources, to decide for themselves, not just tell them to follow the authority of "our" peer-reviewed literature.

Wikipedians use newspapers as sources for WP articles because most people don't have access to the academic, subscription-only sources that you presumably do. This guideline could be harmful if pedantic editors use this as an excuse to delete good newspaper stories as sources, and replace them with subscription-only sources that ordinary readers can't verify, which would violate WP:VERIFY.

To address Colin, I would like to know how you support the claim that newspapers "aren't designed to be source material for medical or science articles in an encyclopedia." I work for newspapers, and we are aware that we have many readers who use our stories for many purposes. I also know people who worked for encyclopedias and reference books. They would certainly accept a Wall Street Journal or New York Times story for fact-checking purposes.

As for Ben Goldacre's story in the Guardian, first I would point out that you're contradicting yourself. You say that newspapers are not a source for medical facts. But you are using Goldacre's story yourself as a source for medical facts.

Second, if Goldacre is correct, the debunked link between MMR and autism was published in peer-reviewed journals, including review articles. So some newspaper stories (including Goldacre's) were more reliable than peer-reviewed journals.

Third, Goldacre says that the anti-MMR lobby targeted general journalists, rather than health or science correspondents (such as himself). This supports the conclusions of Health News Review, which you deleted.

But the fundamental problem is that you're simply citing a single example (published in a newspaper at that). You need more than a data point of 1 to prove your point. If you really believe what you're saying, then go to the peer-reviewed academic communications literature, as I did, and see what they say.

In short, the statements about newspapers in this entry are personal opinion, and wrong. You are deleting authoritiative, WP:RS, useful information that will help editors evaluate the reliability of newspapers for themselves. Nbauman (talk) 15:20, 30 August 2008 (UTC)

Contrary to the apparent beliefs of two editors, NPOV and NOR apply to articles, not guidelines created for internal use. Guidelines are not required to cite their sources, to find independent, third-party publications to support their claims, or to fairly present anything at all. They are, indeed, permitted to stand strongly against deprecated practices, to oppose things deemed undesirable, to announce good practices, and to base themselves entirely on original research. (Just the other week, WP:LAYOUT was counting up the relative proportion of articles using different headings for references in an effort to make a decision, which is clearly "original research".)

When a newspaper story reports a publication, then a responsible science editor will double-check the newspaper reporter's understanding by looking up the actual publication instead of blithely assuming that every single reporter always gets every single scientific fact correct the first time. (Nbauman, do you assert that you have never published a single factual error?) WhatamIdoing (talk) 00:35, 31 August 2008 (UTC)

• Somehow this discussion has overspilled from the previous topic. Thank you, WhatamIdoing for raising a good argument. This is the kind of discussion I would like to have in this space. Paul Gene (talk) 11:49, 31 August 2008 (UTC)
• And I concede you this point. It is then my original research (which I of course believe is better ;) ) vs. Eubulides's. It is then my and Nbauman's opinion vs. yours, Eubulides's, Sandy's and other editors. Then, we have reasonable people like Colin, who would not mind not having newspapers and press releases mentioned in the definition of secondary sources. It is also a poor style to include something which is often a primary source (as I believe everybody agrees) into the definition of the secondary source. I would suggest as a compromise option either deleting or moving this sentence from the definition part to Avoid citing popular press, which it anyways duplicates. Paul Gene (talk) 11:49, 31 August 2008 (UTC)

I just realized from the other discussion that this is a guideline, and should not be changed without discussion and consensus in Talk. The text about Health News Reviews was added by consensus and has been in the guideline for over a year without objection.

The burden of proof is not upon me to gain a consensus that the text about Health News Reviews should be restored; the burden of proof is upon Colin to gain a consensus that it should be removed.

I am restoring the text. If anyone wants to delete it, please discuss your reasons here and get consensus. Nbauman (talk)

I have made an edit regarding press releases that should keep Paul happy. I've moved mention of it down to the Popular press section.

As for Nbauman's comment: This isn't a guideline (yet) and there is no rule on WP that all edits to guidelines require discussion and consensus in Talk. Nbauman's text was originally added against consensus (as can be see on the archive) and was much discussed already. Life is too short to repeat all that, I'm afraid. Nbauman continues to misunderstand the word "reliable" and continues to cite the current-affair/investigative journalism example of Vioxx. Nbauman provided the example of a good pop press article, not me (Bell Curve). It was then promptly destroyed as a good source for WP. Goldacre was cited not for his medical facts, but his article on newspapers. Is it so hard to understand that newspapers are good for some things and not others? I don't think the English Literature wikiproject has this trouble, or the Mathematics wikiproject, etc, etc. Other WP:MED editors, feel free to remove the offending text. Colin°^Talk 14:24, 31 August 2008 (UTC)

Nbauman, I would be most happy if WP cited newspapers for their medical facts with precisely the same frequency as "the New England Journal of Medicine cite[s] articles in newspapers like the New York Times and Wall Street Journal.". That would improve the quality of our medical articles immensely. Colin°^Talk 15:23, 31 August 2008 (UTC)

No consensus for text about newspapers

I went back and read the records, and did not find support for Nbauman's claim "The text about Health News Reviews was added by consensus". Nbauman's original edit, which added this material in April 2007, was clearly controversial at the time. Newspapers, the first thread about it, had commentary by three editors, only one of whom (Nbauman) favored the change. Avoid citing the popular press had commentary by the same three editors, with the same results. Not ready for guidelines (May 2007), the only thread that explicitly mentions Health News Review, has commentary by four editors; again, only one (Nbauman) favored inclusion of such material. What appears to have happened is that a single editor tirelessly advocated the change and installed it without consensus.

Given all this history, along with the discussion above, I'm inclined to revert the controversial edit. The text in that edit sticks out like a sore thumb: it's argumentative, it gives a mistaken impression to unwary readers that the New York Daily News is in the same ballpark as the New England Journal of Medicine for reliability, and basically it encourages Wikipedia editors to substitute newspaper articles for medical-journal articles if they think the newspaper is reliable. Eubulides (talk) 19:17, 31 August 2008 (UTC)

Proposed replacement for Newspapers

I am not opposed to mentioning Health News Review, but it needs to be placed in context better. There are specialized journals such as Journal of Health Communication and Health Communication, but the best coverage of how media covers tends to appear all over the place, in journals ranging from the American Journal of Disaster Medicine (e.g., see Gionis et al. 2007, PMID 18488831) to the American Journal of Public Health (e.g., see Wenger et al. 2001, PMID 11211641). So, after reverting, I propose replacing the contents of WP:MEDRS #Newspapers with the following text.

The quality of newspaper coverage of medicine ranges from excellent to irresponsible. Quality newspapers can make a good source for social, biographical, current-affairs and historical information in a medical article. However, medical facts and figures should be sourced to medical journal articles, perhaps with an additional citation to a free newspaper source for the benefit of readers who lack access to medical journals; one way to do this is with the "laysummary=" parameter of the Cite journal template. Sources for evaluating health-care media coverage include Health News Review, Health Communication, and the Journal of Health Communication; the best reviews of media coverage often appear in more-general sources such as the American Journal of Public Health and The Guardian.

Further comments are welcome, of course. Eubulides (talk) 19:17, 31 August 2008 (UTC)

(An updated version of this draft is in #Proposed replacement for Newspapers 2 below). Eubulides (talk) 08:46, 1 September 2008 (UTC)

First, clarify "more-general." I read or have read several of the major peer reviewed journals, and they all have occasional articles about the media, and while useful, they only cover one specific topic at a time, and do it irregularly. Health News Review and its sister journals, which are also peer-reviewed, cover the media regularly. In that sense they are more-general sources. I don't understand what you mean by saying that AJPH and The Guardian are "more-general."

Second, I don't mind mentioning The Guardian, which I read sometimes, but (1) do they have much analysis of medical news besides Goldacre? (2) The Guardian isn't peer-reviewed, like Health News Review, and to me that makes a big difference. Nbauman (talk) 20:37, 31 August 2008 (UTC)

• I changed "more-general" to "other" in #Proposed replacement for Newspapers 2 below.
• I mentioned The Guardian only because it came up in earlier discussion here. It's nice to mention a non-U.S. source, but if there's a preference to mention some other source that'd be fine too.
• I'd rather mention at least one non-peer-reviewed source in the list; some of the best work in this area (such as Goldacre's) is not peer-reviewed.

Eubulides (talk) 08:46, 1 September 2008 (UTC)

The statement "medical facts and figures should be sourced to medical journal articles" is too restrictive and not required given the reset of this proto-guideline. I think the lay summary suggestion is good, though the text needs a little work. Even readers who have access may wish a easy-to-digest version as WP is written for the general reader. In fact, this is one area where a press release might be useful (see discussion below).

I'm not sure what the "sources evaluating health-care media coverage" has to do with WP:MEDRS. By all means summarise their conclusions in this guideline. To answer Nbauman, Goldacre's remit in the Guardian is the discussion of bad science, usually wrt medicine. This often discusses the bad reporting of bad science in newspapers. His work is accessible and may be an eye-opener for some, but it is certainly an opinion-piece rather than a work of scholarly analysis. I'm not aware that any UK newspaper has a qualified full time science or medical reporter let alone anyone capable of editing such articles. Even good quality medical articles are mediocre sources for WP -- see the Bell Curve example in the archive. A good quality lay medical article can be well worth reading, in order to work out how to discuss the topic in lay terms. Colin°^Talk 20:55, 31 August 2008 (UTC)

• In #Proposed replacement for Newspapers 2 below, I changed "medical facts and figures should be sourced to medical journal articles" to "They should not be used as a sole source for medical facts and figures", which is nearly what is there now (with the addition of a "sole").
• Now that you mention it, the "sources evaluating health-care media coverage" stuff seems to be more relevant to WP:MEDRS #Assess the quality of evidence available than to WP:MEDRS #Newspapers. I revised the proposal to summarize their results in WP:MEDRS #Newspapers, and to move the "sources evaluating health-care medica coverage]] stuff to the other section.

Eubulides (talk) 08:46, 1 September 2008 (UTC)

In general I like this approach. However, I'd change However, medical facts and figures should be sourced to medical journal articles, perhaps with an additional citation to a free newspaper source to be less specific.

Medical facts can be properly sourced not just to journal articles, but also to college-level (and higher) textbooks and other reputable reference works (i.e., Gray's Anatomy). This expansion is particularly important for facts involving basic science.

Similarly, newspapers are not the only possible source of open-access information: there are many highly reputable health-related websites that might be at least as appropriate (*.cdc.gov, for example). Additionally, I'm happy to see good, open-access media articles listed in the ==External links==, which is another way to put lay-language information into the hands of interested readers. WhatamIdoing (talk) 21:16, 31 August 2008 (UTC)

#Proposed replacement for Newspapers 2 below changes the "medical facts and figures" text along the lines you suggest. I prefer to reserve External links for websites and the like, and to cite open-access media articles as usual with citations, except with a URL so that the reader can easily see that the media articles are freely readable (but surely this is a style issue?). Eubulides (talk) 08:46, 1 September 2008 (UTC)

Proposed replacement for Newspapers 2 etc.

With all the above in mind, here is a revised proposal for replacement for WP:MEDRS #Newspapers:

The quality of newspaper coverage of medicine ranges from excellent to irresponsible. Quality newspapers can make a good source for social, biographical, current-affairs and historical information in a medical article. However, most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits, and should not be used as a sole source for medical facts and figures. One possibility is to cite a higher-quality source along with a more-accessible newspaper source, for example with the "laysummary=" parameter of the Cite journal template.

And here is proposed additional text to be appended to the last paragraph of WP:MEDRS #Assess the quality of evidence available:

Sources for evaluating health-care media coverage include Health News Review, Health Communication, and the Journal of Health Communication; the best reviews of media coverage often appear in other sources such as the American Journal of Public Health, Columbia Journalism Review, and The Guardian.

(Please see a proposed further revision below.) Eubulides (talk) 21:02, 1 September 2008 (UTC)

(This latter text wikilinks to a Health News Review article I just created.) Again, comments are welcome.

Eubulides (talk) 08:46, 1 September 2008 (UTC)

I think this addresses all the issues. Great writing! Paul Gene (talk) 09:17, 1 September 2008 (UTC)

(1) Along with Health News Review (U.S.), I included Media Doctor (Australia), and Media Doctor (Canada), to make it less U.S.-centric.

(2) I don't understand why you cite Health News Reviews, and then say that "the best reviews of media coverage" often appear in other sources. That says that HNR is not publishing "the best," while the others are. HNR's only job is to regularly evaluate health news, by a structured peer review process that includes medical doctors and journalists. I think that evaluation which is peer-reviewed is better than evaluation that is not peer reviewed. Do you disagree? If anything, HNR is "the best"; I wouldn't single it out but I would like to know why you single out the others as "the best."

As far as I know, the only reporting on health coverage in Columbia Journalism Review is by Trudy Lieberman (which is very good, but not peer reviewed). Can you cite anything else? The only reporting on health coverage in The Guardian is by Goldacre (which is not peer reviewed). Can you cite anything else? Nbauman (talk) 16:02, 1 September 2008 (UTC)

• We don't have room to list all such sources. One example of each major type should be enough. Come to think of it, this should be worded more clearly; please see below.
• The definition of "the best" depends on what kind of review one is looking for. Goldacre's column, cited above, is very high in quality, but as far as I know broad-scope coverage like that is not published in Health News Review.
• Although Health News Review has a team that produces high-quality reviews, these reviews are not themselves peer-reviewed, as far as I can see. This is in contrast to peer-reviewed publications such as the Journal of Health Communication and the American Journal of Public Health. That being said, I agree that "the best" doesn't need to appear here.
• With the above in mind, here's an updated version of the "Sources for" sentence.

Sources for evaluating health-care media coverage include review websites such as Health News Review and specialized academic journals such as the Journal of Health Communication; high-quality reviews can also appear in sources as various as the American Journal of Public Health, the Columbia Journalism Review, and The Guardian.

Eubulides (talk) 21:02, 1 September 2008 (UTC)

If space is a consideration and we don't have room for the Australian and Canadian reviews, then we should shorten it to:

Sources for evaluating health-care media coverage include review websites such as Health News Review and specialized academic journals such as the Journal of Health Communication; reviews can also appear in the American Journal of Public Health, the Columbia Journalism Review, The Guardian, and others.

Nbauman (talk) 04:22, 2 September 2008 (UTC)

Yes, that'd be fine. Eubulides (talk) 05:16, 2 September 2008 (UTC)

With respect to the following:

Quality newspapers can make a good source for social, biographical, current-affairs and historical information in a medical article. However, most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits, and should not be used as a sole source for medical facts and figures.

I'd like to know the evidence you have to support the statement that "most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits."

I think we all agree that 'some news articles do discuss exactly those issues, and some do not. For example, Health News Reviews evaluated the New York Times article on HPV vaccines, and it covered all those issues and more. [12] Health News Reviews found many more articles that met those issues.

On the evidence of Health News Reviews and other sources, we can say that some news articles fail to discuss those things, but I want to know the evidence for saying most.

As I understand it, this is based on the personal opinion of several editors. Is it based on anything else? Nbauman (talk) 17:02, 2 September 2008 (UTC)

That was not based on my (or other Wikipedia editors') opinion. It was based on Schwitzer 2008 (PMID 18507496), a source cited in the Health News Review article. That source's summary says "journalists usually fail to discuss costs, the quality of the evidence, the existence of alternative options, and the absolute magnitude of potential benefits and harms." Eubulides (talk) 17:31, 2 September 2008 (UTC)

Additionally, that statement appears to line up with everyone's experience (except perhaps for our professional journalist's) and this guideline is not an encyclopedia article. We do not actually have to provide reliable sources to support statements that we all agree on. The newspaper-seller's apparent horror of absolute risks drives me nuts. "Risk tripled!" the headline trumpets -- but never mentions that it's from 1 in 30,000 to 1 in 10,000. WhatamIdoing (talk) 17:58, 2 September 2008 (UTC)

Very good. That's a good source, open source, and an important point. I'm glad I pressed you on it. I think we should include the source. It doesn't lengthen the body of the text significantly to insert a footnote, and it increases the reliability and authority infinitely.

WhatamIdoing, could you or somebody else help me out? Could you link to the WP rule that says that factual statements in guidelines are exceptions to WP:RS? Nbauman (talk) 19:10, 2 September 2008 (UTC)

Content policies and guidelines apply to article space (i.e. the encyclopaedia). Read any one of them and you'll see the word "articles" repeated again and again. The policy and guideline pages, along with talk pages, user and project pages are all part of the machinery but not the product. The overriding rule for policy and guidelines is that the text is achieved through consensus. Colin°^Talk 19:40, 2 September 2008 (UTC)

Thank you, Colin. Can you link to an official statement in Wikipedia that says that? Nbauman (talk) 22:59, 2 September 2008 (UTC)

I can: First sentence, second paragraph of WP:RS: "Wikipedia articles should use reliable, third-party, published sources." Or let us have the third sentence of the second paragraph at WP:V: "Jointly, these policies determine the type and quality of material that is acceptable in Wikipedia articles."

So it appears that RS and V apply solely to articles. If you want to be WP:POINTy about it, let me suggest that you take it up at Wikipedia:Village pump (policy). I suggest that you start by telling them that you think that the provisions of WP:V itself need to meet WP:V. WhatamIdoing (talk) 00:19, 3 September 2008 (UTC)

WhatamIdoing and Colin, I believe that when WP:RS refers to "articles", that includes guidelines and policy statements insofar as the guidelines and policy statements refer to facts.

You believe WP:RS excludes guidelines and policy statements.

However, neither you nor anybody else can cite with a link any WP guideline or policy statement that explicitly supports your belief.

Is that correct? I just wanted to get that on the record. Thanks. Nbauman (talk) 09:10, 3 September 2008 (UTC)

• "Policy change comes from three sources: 1.Documenting actual good practices and seeking consensus that the documentation truly reflects them." Wikipedia:Policies and guidelines#Sources of Wikipedia policy Paul Gene (talk) 09:56, 3 September 2008 (UTC)

Nbauman, WP's content guidelines and policies explicitly apply to articles and don't need an explicit exclusion for the other parts of WP (which aren't articles) in much the same way that the Highway Code doesn't contain an exclusion for aircraft or boats. Colin°^Talk 12:20, 3 September 2008 (UTC)

Colin, my question was, is your contention that facts in guidelines do not require WP:RS (1) based on your own personal determination of what is common sense, or is it (2) based on an explicit WP rule that you can cite with a link. Your answer is therefore (1). Nbauman (talk) 16:59, 3 September 2008 (UTC)

For the record, I draw your attention to Wikipedia:Verifiability WP:V: "This page in a nutshell: Material challenged or likely to be challenged, and all quotations, must be attributed to a reliable, published source."

It doesn't distinguish between articles, guidelines, or anything else on WP, it just says "material."

So I think WP:V clearly states that any material challenged must be attributed to a reliable, published source -- including factual material incorporated into guidelines. Nbauman (talk) 18:11, 4 September 2008 (UTC)

This interpretation of WP:V is contradicted by material that is present in WP:V itself. For example, WP:V says "In general, the most reliable sources are peer-reviewed journals and books published in university presses". This is a factual claim, that is likely to be challenged using exactly the sort of challenges that we're seeing for WP:MEDRS. But WP:V does not cite a source for this claim. And the claim is not an isolated example: there are several such claims in WP:V, and none of them cite a source. Similiarly, WP:RS (which is a generalization of WP:MEDRS) cites zero sources for its several claims. Eubulides (talk) 19:30, 4 September 2008 (UTC)

WP:V's lead text says "Wikipedia:Verifiability is one of Wikipedia's core content policies. The others are Wikipedia:No original research and Wikipedia:Neutral point of view. Jointly, these policies determine the type and quality of material that is acceptable in Wikipedia articles." Note that last word. This is beyond tiresome. Colin°^Talk 20:01, 4 September 2008 (UTC)

I'm not convinced. I think the statement, that peer-reviewed journals and university presses are reliable, is so generally accepted that it does not require a WP:RS. However, the statement that newspapers usually have those flaws is not generally accepted, so it requires a source.

I don't want to argue about it forever. However, I do want to insert the source into that statement about newspapers. I think that's a reasonable compromise. One more footnote won't detract from the article. Nbauman (talk) 02:32, 6 September 2008 (UTC)

OK, there seems to be some consensus for the revised proposed text, as far as it goes, so I installed it. One minor wording change as I installed it: I changed "and should not be used" to "and news articles should not be used", to prevent a plausible misreading of the phrase to mean "and most news articles should not be used", a meaning which differed from the previous meaning and was not the intent. Eubulides (talk) 07:32, 3 September 2008 (UTC)

I have to throw in a few pennies, here. I think that anywhere on Wikipedia a statement is presented as fact, the factuality of that statement should be open to question, and if necessary those questions ought to be met with proper sourcing. even in policy and guidelines... in this case (it seems to me) Nbauman is objecting to what seems like a prejudicial statement against newspapers being presented as fact, and I'm not convinced he's wrong - wp:RS#News_organizations would seem to be on his side. I'm not sure that I would personally disagree with you that some newspaper articles on medicine are poor quality, mind you, but I can't really see how we can make that judgement as editors - on a case by case basis, or in total - without violating wp:SYN. I think this passage ought to be reverted while we discuss the matter a bit more. --Ludwigs2 00:16, 6 September 2008 (UTC)

I think we can reach consensus by adding the citation to support this claim: "However, most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits.[13]" I don't think we have consensus otherwise. Without the citation, it sounds like some editor's personal opinion and cranky anti-media bias. Nbauman (talk) 02:24, 6 September 2008 (UTC)

If you want to add a citation to back up what has already been agreed by the consensus of WP editors, then that is fine. I have no problem with that. The point we are making is that such citations are not required and WP could, if it wants to, collectively disagree with outside opinions (e.g., that lay people can make a useful contribution to a scholarly encyclopaedia :-) Colin°^Talk 10:32, 6 September 2008 (UTC)

I'd still be happier

I'd till be happier if this section were not so anti-newspaper. a newspaper is a reliable source, and it should always be assumed that a newspaper article has done appropriate fact checking to present a credible viewpoint. the objection that they don't present certain scientific facts is really more a concern for people doing primary research; it should not matter to a tertiariy source like wikipedia, and excluding newspaper articles on that basis is no justified. let me see if I can make a corrective edit. --Ludwigs2 21:47, 7 September 2008 (UTC)

Unfortunately the change you installed (without previous discussion) had serious problems. Here's what the change was:

However, most many news articles fail to discuss important scientific issues such as evidence quality, costs, and risks versus benefits, and a news article should not be used as a sole source for a medical fact or figure. One possibility is to cite a higher-quality source along with a more-accessible newspaper source, and should generally be supplemented with academic secondary sources, for example with the "laysummary=" parameter of the Cite journal template.

Some problems with this:

• It changes "most" to "many". But "most" is accurate; see Schwitzer 2008 (PMID 18507496). It should not be watered down to "many".
• It removes mention of quality, costs, and risks versus benefits, but these are important and relevant issues, which Schwitzer mentions.
• It removes longstanding advice in this article that news articles should not be sources for medical facts and figures.
• It has things backwards, by suggesting that newspapers be the main source, with refereed journal articles being supplements. The more-reliable sources should be the main sources.

There is one change, the addition of the word scientific, which is not a serious problem, though I don't see why it's needed here. For now, because of the serious problems mentioned above, I reverted the change. Eubulides (talk) 23:16, 7 September 2008 (UTC)

it was a first draft, and I didn't expect it to be perfect. the main problem I have here is that you all seem to have unilaterally decided that [wp:RS#News_organizations]] does not apply to medicine articles. you seem to have decided this because you are (unfairly) equating newspaper articles in toto to poor quality newspaper articles. It might be advisable to do this on a case by case basis - possibly news articles require more careful vetting - but writing them off in advance as poor-quality sources is clearly against established policy. If you don't like my changes, that's cool, but how do we modify this passage to keep it in line with policy? --Ludwigs2 23:46, 7 September 2008 (UTC)

WP:RS is a guideline; it's not policy. The WP:MEDRS advice to not source medical facts and figures from newspapers has been in the page for a long time and seems to represent a working consensus here. It is appropriate to have a stricter guideline for biomedical topics. It's probably better to discuss drafts like that here first, before installing them; perhaps you could draft a proposed improvement with the above comments in mind? Eubulides (talk) 00:11, 8 September 2008 (UTC)

I've not seen a newspaper account of Tourette syndrome yet that was correct. Not one. No reason to prefer them when we have peer-reviewed sources. After viewing Ludwigs2's editing to WP:NPOV, I'm concerned about the effort to diminish NPOV and high quality sources. SandyGeorgia (Talk) 00:13, 8 September 2008 (UTC)

don't make this personal, Sandy. if you want to debate the value of particular edits, that's fine, but if you want to criticize my understanding or intent then you're way off-base (because - frankly - you really don't have the faintest idea what's going through my mind; if you did, you wouldn't be so quick to cast aspersions). let's try to keep things on track, ok? --Ludwigs2 20:28, 8 September 2008 (UTC)

WP:RS used to explicitly warn against using the popular press as a source for science. The remnants of that advice can be found in Wikipedia:Reliable source examples, which is just an essay. Regardless, both WP:V and WP:RS place an emphasis on scholarly works as sources. A quality newspaper is a vital reliable source for "news" (current affairs) and serves as a historical record of such events. It should no more be considered a reliable source for medical facts, than my Radio Times is a reliable source on Jane Austen. Colin°^Talk 08:47, 8 September 2008 (UTC)

I don't know what RS used to say, and I'm not trying to rest this discussion on what RS currently says (because, yes, policies change over time). to my mind, what reliable sourcing means is that sources provide informations which we can trust to be factually correct depictions of common knowledge in a field. academic journals and other scholarly sources are generally factually accurate, but they often lean towards primary research (and thus are not reflections of 'common knowledge'). newspapers reflect common knowledge well, but are not as factually accurate (in part because they are not peer reviewed, and in part because they focus on different facts than scientists do). do you think that's a bad assessment? --Ludwigs2 20:41, 8 September 2008 (UTC)

Again, WP:RS is a guideline, not policy. In my experience, the "common knowledge" about medical facts and figures that newspapers express but academic sources do not would be more-accurately characterized as "folk theories", or "speculation". For example, popular coverage on the causes of autism focuses heavily on environmental theories, which are are popular among non-experts but are quite speculative; in contrast, popular coverage tends to deemphasize genetic theories, which have a strong consensus among experts (see Singh et al. 2007, PMID 17237806). Eubulides (talk) 21:20, 8 September 2008 (UTC)

Controversies

I haven't been following the discussion very closely, so correct me if I've overlooked something. I assume the RS we're talking about are mostly concerned with dealing with scientific facts, in which case we want to use the best sources available, such as peer reviewed research, etc. Well, medical subjects, which (unfortunately) includes alternative medicine and other forms of quackery and health fraud, involves much controversy and there are numerous commentaries from mainstream sources, but such commentaries don't usually appear in peer-reviewed research. Such commentaries will also deal with the medical politics, turf wars, etc., which are also matters not usually dealt with in peer-reviewed research. Wikipedia's NPOV policy requires that we tell the whole story, and thus these controversies will need to be mentioned in medical and scientific articles. We will need to use non-peer-reviewed commentaries. Have you been dealing with this issue at all here? -- Fyslee / talk 19:50, 31 August 2008 (UTC)

So far the proposals have not explicitly addressed that situation; instead, they give generic advice that ideally would apply to alternative-medicine topics just as well as to mainstream-medicine. In Chiropractic I've found quite a few sources about medical politics etc. in peer-reviewed articles. That may be a special case, but at any rate the current guidelines don't advise against using non-peer-reviewed commentaries in that case, they merely advise not to use the popular press for medical facts and figures. So, although WP:MEDRS doesn't offer much guidance in your situation, it doesn't suggest not using the popular press either. Is that good enough for now? Eubulides (talk) 08:46, 1 September 2008 (UTC)

Press releases

Since I've used and fact-checked press releases extensively, and occasionally written press releases, I know the mechanics of writing a press release, and I can contribute some facts to the discussion.

Press releases represent the institution that issues the press release, and they are as reliable as that institution.

Peer-reviewed journals like BMJ, Lancet, JAMA, Science and Nature issue press releases, which are approved by both the editors of the journal and the authors of the papers. I think we can agree that those PRs are as reliable as the journals.

Universities issue press releases, and when they are about scientific subjects, usually in peer-reviewed journals, they are reviewed by the investigators who did the work, so they are as reliable as the investigator and usually as reliable as the journal article itself.

Book publishers issue press releases. Since they are reviewed by the author, they are as reliable as the author. (So saying "Press releases are accurate" is like saying "Books are accurate.")

Major pharmaceutical companies issue press releases, which are reviewed by their legal departments; they have to meet many legal requirements including FDA labeling restrictions and securities laws requirements. Corporations would be vulnerable to legal actions and damages if they knowingly printed inaccurate information, so they are usually literally accurate, although they can also be promotional and are not as objective as peer-reviewed journals. They're works of advocacy, which is not the same as the whole truth. If the pharmaceutical company wants to tell the whole truth, which they sometimes do, they'll give the benefits and the limitations of a new drug.

Finally, anybody can issue a press release. Any nut with a 50kg drum of DHEA and a pill press can put out a press release touting the benefits of DHEA. So it's hard to generalize about press releases and they have to be taken on their merits.

I see press releases, and then see the stories that are written about them in the newspapers and elsewhere. First, the wire services take about 10% of the medical press releases and run them pretty much as is. (Science Daily seems to be entirely compiled from press releases.) Then, if a story is particularly important or controversial, they'll do some fact checking and get comments (as Edward Edelson checked the meeting presentations in that Washington Post story.) Major newspapers like the New York Times and Guardian treat press releases that same way.

Sometimes the news sources get it wrong and the original press release is more accurate. Sometimes the news sources add useful perspectives to the story. Many science reporters have PhDs and MDs (for example Robert Altman of the New York Times) so their stories can often be more reliable and accurate than the original peer-reviewed journal article.

The main function of a press release is to explain the scientific publication in language that a non-specialist reader can understand. (Wikipedia is also written for a non-specialist audience.) Scientific journals have certain conventions which make them hard to understand, such as using technical language and putting the conclusion at the end. Press releases explain technical terms (as WP should) and put the conclusion at the beginning. They usually know that laymen will frequently misinterpret certain things from journal articles, so press releases should (but don't always) articulate those points clearly, for example that a study has only been done in vitro.

Significantly, press releases are designed to be copied, so under the copyright laws "fair use" would be very broadly interpreted.

Furthermore, they often contain the significant content of a subscription-only journal article, in a form that is reliable and less subject to misinterpretation than the free abstract and sometimes the article itself.

For these reasons, I think that press releases from reliable sources are a good WP:RS for Wikipedia, in addition to the link to the PubMed or original article. Press releases are open access, and you can check them to make sure an editor didn't misunderstand or distort the original article.

That's why I also think that press releases that have gone through an additional layer of fact-checking, and published as news stories in a reliable newspaper, are also good WP:RS, in addition to peer-reviewed sources.

Are press releases about journal articles primary or secondary sources, for Wikipedia purposes? I don't know. The main problem with primary sources is that they are subject to misinterpretation, and using them is WP:OR. Press releases are less subject to misinterpretation, because they are designed to be read by non-specialists and designed to flag common misunderstandings. So they do have that advantage and serve that purpose of secondary sources. Nbauman (talk) 20:09, 31 August 2008 (UTC)

All this proto-guideline currently says is that press releases "can be a biased source". They are saying "look at what we've done" or "listen to what we have to say on X". As you say, some press releases are effectively self-published sources that any crank could create. Others have a degree of editorial (or, at least, legal) oversight. Regardless, they are about as far from independent and impartial as you can get. So we need to treat them with caution. Is there anything in this guideline wrt press releases that you disagree with. I've suggested above that they may be useful as lay-summary versions of scholarly or inaccessible sources. Do you want to suggest some extra text on PRs? If so, please make it brief as the above posting was verging on TLDR. Colin°^Talk 21:07, 31 August 2008 (UTC)

(BTW) Using primary sources isn't necessarily WP:OR. For more details, see WP:OR! Colin°^Talk 21:21, 31 August 2008 (UTC)

Again, the fact that a press release is an open-access source does not make it a reliable source. Those two qualities are entirely unrelated. WhatamIdoing (talk) 21:08, 31 August 2008 (UTC)

"Universities issue press releases, and when they are about scientific subjects, usually in peer-reviewed journals, they are reviewed by the investigators who did the work, so they are as reliable as the investigator and usually as reliable as the journal article itself. " They do not misrepresent the facts of the work, but they may well misrepresent the significance. They tend to tout the importance of the study without saying just what it is or exactly where it has been published. They are not intrinsically unreliable, but I';d never ever use one for a matter of opinion about the work. The claims in the journal article are peer-reviewed. The claims in the press release are whatever the author of the article pleases. DGG (talk) 08:12, 1 September 2008 (UTC)

My general experience is that a press release for a scientific study is not as good as the study it promotes. Here's an example: Autism says "A 2008 study found that compared to their peers, autistic boys have significantly thinner bones if on casein-free diets" and cites Hediger et al. 2008 (PMID 17879151). But the NIH press release for that study doesn't bother to say specifically whether that difference in thinness was statistically significant. To justify the word "significant" in Autism, one really must cite the research study, not the press release. And this is one of the better press releases I've seen; it was carefully and conservatively written and was obviously vetted by the researchers. Eubulides (talk) 08:46, 1 September 2008 (UTC)

I agree with Eubulides that a press release of a scientific study (from academia) often exaggerates its significance and skips on finer details. On the other hand, I agree with NBauman that corporate press releases are generally of a better quality as they avoid making any exaggerated claims for legal reasons and strictly stick to the facts for the same reasons. They often precede a peer reviewed article and may be the only information source of the new developments for quite some time. It is a bit of a paradox but corporate press releases are generally reliable and often useful sources. In regards to style, "Such articles may be based uncritically on a press release, which can be a biased source." sounds awkward, does not add any content, and can easily be deleted. I wonder if NBauman can come up with a short sentence on press releases in general to replace it. Paul Gene (talk) 09:45, 1 September 2008 (UTC)

Guideline name & location

Ok I've acted as an admin and promoted from a proposed guideline to an actual guideline (RfC closed and my thoughts above).

A quick question though, as now a guideline of wikipedia, should it remain in this a single project space or be promoted in its name & location, so loosing the subpage "/" ? Hence from Wikipedia:WikiProject Medicine/Reliable sources to (current word order might suggest Wikipedia:Medical reliable sources, but I think might be better by using the standard disambiguation convention of brackets) Wikipedia:Reliable sources (Medicine) ? David Ruben ^Talk 01:09, 1 September 2008 (UTC)

This is good news! Excellent work, everyone. I think we should stick with the standard disambiguation format. -- Fyslee / talk 01:58, 1 September 2008 (UTC)

MEDMOS is at Wikipedia:Manual of Style (medicine-related articles). Perhaps this should use a parallel name. WhatamIdoing (talk) 06:11, 1 September 2008 (UTC)

Agree with WhatamIdoing on David's second suggestion. Also, this page needs to link back to MEDMOS somewhere. Colin°^Talk 10:32, 1 September 2008 (UTC)

To keep to bracketed disambiguation style and in keeping with MEDMOS approach, I've moved this wikiproject subpage from Wikipedia talk:WikiProject Medicine/Reliable sources to Wikipedia:Reliable sources (medicine-related articles). We can (at this early stage) rename, but I think I prefer WhatamIdoing's suggestion to my previosu "Wikipedia:Reliable sources (Medicine)" which seems over-terse in structure. David Ruben ^Talk 23:57, 1 September 2008 (UTC)

Colin, is this what you meant for "link back to MEDMOS somewhere" ? David Ruben ^Talk 00:12, 2 September 2008 (UTC)

Just as Wikipedia talk:Reliable sources not general place to discuss specific sources (vs the overall policy), should this guideline talk page have a similar top tag-box to some form of /Noticeboard. We can either elect to point to Wikipedia:Reliable sources/Noticeboard or create (but then have to personally watch ourselves) Wikipedia:Reliable sources (medicine-related articles)/Noticeboard ? Feel noticeboard creep if we are not careful, so I think I would prefer the former existing RS/Noticeboard... David Ruben ^Talk 00:19, 2 September 2008 (UTC)

To be honest, I don't expect to get too many questions. The usual RSN is probably good enough, or people can leave a message at WT:MED (or WT:PHARMA, or other relevant project). WhatamIdoing (talk) 03:27, 2 September 2008 (UTC)

Thanks, with no further direct comments on this specific point, I've added[14] a top banner to this talk page, pointing discussion of specific examples (vs overall guideline discussion) to RS/N (as per WP:RS's own talk page) as well as options of WT:MED & WT:PHARM. Likewise in a similar style to the last line of WP:RS's leadin, to WP:MEDRS's leadin - rephrase at will. David Ruben ^Talk 22:33, 8 September 2008 (UTC)

---

I left a message on David's page asking for the precedents he followed to promote this useful but extremely raw essay over the objections of significant minority of editors (3 vs 14). Paul Gene (talk) 09:59, 5 September 2008 (UTC)

David gave a full explanation of his reasons further up at #Should we make this a guideline?. He specifically addresses your and Una's issue with secondary sources, and in common with most other editors finds WP:MEDRS to be in alignment with existing policies and guidelines in this regard. The other "oppose" is an outlier and can be ignored. Since both objects wrt primary vs secondary are not actionable without bringing this guideline into conflict with established policy, there seems little reason for them to block the adoption of this guideline. I'm sure this guideline can be improved and will be by collaboration and consensus. Lets work to improve it and move on. Colin°^Talk 12:16, 5 September 2008 (UTC)

Assess the quality of evidence available

I'd like to see WP:MEDRS#Assess the quality of evidence available expanded along the lines of #Use up-to-date evidence.

The primary point that I'd like to add is that if you don't have good quality evidence for a scientific or medical fact, then the Wikipedia article probably doesn't need to mention that fact. I'm sure you can all imagine a prototypical "poor evidence" scenario: an unblinded, nonrandomized, unverified, retrospective, unvalidated internet-based survey of a nonrepresentative sample of people that "proves" something.

This guideline (I think) would be a reasonable place to provide the basic advice that editors need: It's often best to omit low-quality evidence entirely, even if it means that the article is shorter or "incomplete".

What do you think? WhatamIdoing (talk) 05:42, 5 September 2008 (UTC)

That sounds like a rules creep to me. There are many cases when nonrandomized retrospective studies are the only sources, for example, for the rare drug effects. Another case is when such a study generates a considerable controversy and press coverage, and as such should be reflected. So, I do not see how this addition would be helpful to the editors. The overarching principles of editorial discretion and reaching consensus via discussion are sufficient to handle the situations described by What. Paul Gene (talk) 09:47, 5 September 2008 (UTC)

This assumes that most of medicine's decision making is founded on evidence-based medicine; it isn't. While it is useful for WP editors to have knowledge of what makes good-quality primary evidence, judging that primary evidence is really the job of someone writing a review (literature or systematic or whatever). Such a review may also make statements based on consensus opinion of experts or even from personal experience (esp. for the rare stuff that will never get a RCT). We can base our articles on such statements, even if the evidence behind them is lacking. There's also a place for opinion and supposition, if directly attributed. Colin°^Talk 10:11, 5 September 2008 (UTC)

I'd hate to have to write an article on surgery or emergency medicine or any of dozens of other medical fields if I were limited to citing only randomized controlled trials. See, for example, Vincent 2004 (PMID 15302748) "in intensive care medicine the situation is a little different, with RCT evidence frequently lacking" and Jeppsson & Thorlacius 2005 (PMID 18333189) "There are several reasons for this lack of RCTs in surgery." Eubulides (talk) 16:45, 5 September 2008 (UTC)

I agree with Paul Gene about rules creep.

Evidence-based medicine is not identical to randomized controlled trials. There are many reasons why RCTs are impossible, and doctors who follow EBM acknowledge that and use the best available evidence. For example, there is no RCT to demonstrate the harmful effects of cigarettes. And there is always the period from the time of the innovation to the time of the definitive RCT, when people have to make decisions based on the best evidence. I refer you again to that "Clinical decisions" article in NEJM about skin infections. Sometimes, when you diagnose the disease specifically enough, there is no good evidence.

I think our job should be to explain how strong the evidence is -- where it is along the path to definitive evidence. To do that might require a bit of background on EBM. And it might require some clinical detail such as whether something is supported by prospective or retrospective trials, etc. Nbauman (talk)

Reorganize newspaper-related text

As I understand it, the two sections WP:MEDRS #Use caution in citing the popular press and WP:MEDRS #Newspapers are intended to have two different functions. The former is supposed to provide basic advice about whether and how to cite the popular press, and the latter is supposed to describe the popular press's coverage of medicine-related articles. However, in reading the two sections, it appears that these two functions are not clearly separated, and material that should be in one section is put into the other (and vice versa), which is confusing. So I propose the following replacements for these two sections, with strikeouts indicating the old version and italics the new. These changes are intended to merely move text around, not to make any significant insertions or deletions.

---

Use caution in citing the popular press

The popular press is generally not a reliable source for science and medicine articles. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new experimental medicine as the "discovery of the cure" of a disease, or an every-day substance as the "cause" of some dreaded disease. Newspapers and magazines frequently publish articles about scientific results before those results have been peer-reviewed or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source. They also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms. A news article should not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible newspaper source, for example with the "laysummary=" parameter of the Cite journal template.

On the other hand, high-quality popular press sources can be very useful in describing the sociocultural aspects of medicine, disease, and health. Common sense and the general guidelines presented in the verifiability policy and general reliable sources guideline should be considered in determining whether a popular press source is suitable for these purposes.

---

Newspapers

The quality of newspaper coverage of medicine ranges from excellent to irresponsible. Quality newspapers can make a good source for social, biographical, current-affairs and historical information in a medical article. However, most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits,. and a news article should not be used as a sole source for a medical fact or figure. One possibility is to cite a higher-quality source along with a more-accessible newspaper source, for example with the "laysummary=" parameter of the Cite journal template. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new experimental medicine as the "discovery of the cure" of a disease, or an every-day substance as the "cause" of some dreaded disease. Newspapers and magazines frequently publish articles about scientific results before those results have been peer-reviewed or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source. They also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms.

---

(end of proposal) Eubulides (talk) 17:30, 8 September 2008 (UTC)

I favour just merging the "popular press" section into one section later on (i.e., keeping it out of "Definitions and basics"). The "Periodicals" section could be split into two: "Scholarly journals" and "Popular press". The note about popular science magazines could be incorporated into the latter, along with the text on newspapers and other popular press issues. Colin°^Talk 18:06, 8 September 2008 (UTC)

OK, thanks, a revised draft is in #Popular press reorg 2 below. Eubulides (talk) 21:22, 8 September 2008 (UTC)

I think this still fails to recognize the distinction between press sources. The NYT for example is generally considered reliable for mecial coverage to the same extent as most science oriented news services. This is a little tricky though, as at present I am not aware of any other English language newspaper I would say the same about--most certainly I would no longer say this of the Guardian. DGG (talk) 18:14, 8 September 2008 (UTC)

The issue is not just reliability wrt medical facts, but also the suitability of the text as a scholarly source for an encyclopaedia. Can you think of a situation were, writing an FA-quality article, you would choose to cite the NYT rather than seek out a more authoritative source on medical information (excluding current affairs)? I assume you are referring to NYT articles rather than the NYT's online health guide, whose text is taken from A.D.A.M., Inc. Colin°^Talk 20:07, 8 September 2008 (UTC)

The NYT is quite good, but even it has problems at times. Its quality ranges from Elizabeth Rosenthal's fine piece on cancer vaccines last month (HNR review) to Andrew Pollack's relatively-mediocre piece on OraVerse in May (HNR review). And if you read the reviews you'll see that even's Rosenthal's piece missed an important aspect of the HPV vaccine issue; this is something that would have been caught in a good peer-reviewed journal. Eubulides (talk) 21:22, 8 September 2008 (UTC)

Yes I can think of such an occasion when the researchers have a strong conflict of interest. For example, a suicide of a healthy volunteer in the duloxetine trial (see Duloxetine#Suicidality) has only been described in a popular press. In this case, the conflict of interests prevented the normal flow of scientific information. If it did not happen in the Lilly labs, there would have been a case study in a scientific journal. As the things stand now, the popular press is the only source of such information. Paul Gene (talk) 11:20, 9 September 2008 (UTC)

Popular press reorg 2

OK, here's a revised proposal in the light of Colin's suggestions above:

1. Remove WP:MEDRS #Use caution in citing the popular press. Almost all its text will be moved to a new Popular press section as described below.
2. Replace WP:MEDRS #Popular science and WP:MEDRS #Newspapers with a new Popular press section shown below. This section's text is taken from the current WP:MEDRS #Use caution in citing the popular press section, with additions in italics and deletions struck out:

Popular press

The popular press is generally not a reliable source for science and medicine information in articles. Most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new experimental medicine as the "discovery of the cure" of a disease, or an every-day substance as the "cause" of some dreaded disease. Newspapers and magazines frequently publish articles about scientific results before those results have been peer-reviewed or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source. They also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms.

A news article should not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible popular source, for example with the "laysummary=" parameter of the Cite journal template.

On the other hand, the high-quality popular press sources can be very useful in describing the sociocultural aspects of medicine, disease, and health can be a good source for social, biographical, current-affairs and historical information in a medical article. For example, popular science magazines such as New Scientist and Scientific American are not peer reviewed but sometimes feature articles that explain medical subjects in plain English. As the quality of press coverage of medicine ranges from excellent to irresponsible, cCcommon sense and the general guidelines presented in the verifiability policy and general reliable sources guideline should be considered in determining whether a popular press source is suitable for these purposes.

---

(end of proposal) Eubulides (talk) 17:30, 8 September 2008 (UTC)

• The popular press is generally not a reliable source for science and medicine articles doesn't seem quite right. Perhaps The popular press is generally not a reliable source for science and medicine information might be more accurate. An article that is primarily on a scientific or medical topic may include many facts that are not themselves scientific or medical in nature, such as the sales figures for a prescription drug. WhatamIdoing (talk) 05:23, 9 September 2008 (UTC)
  • Let's combine these to say "information in articles". After all, a Ladybird book can be a reliable source of information, but certain not suitable as a source on WP. Colin°^Talk 07:24, 9 September 2008 (UTC)

• OK, I added the "information in", in the draft above. Eubulides (talk) 16:49, 9 September 2008 (UTC)

I've been following this discussion for a while, with a mixture of reactions. I agree with WhatamIdoing that "information" would be better than "articles" in the above, but I suspect that the more central issue is that: The popular press is not >as< reliable as is the original research articles. In some cases, the original research article is more accurate (my own scientific findings are very frequently misinterpreted by the press), but the original research article is oftentimes too technical for most WP editors to interpret themselves.
— James Cantor (talk) (formerly, MarionTheLibrarian) 12:06, 9 September 2008 (UTC)

• My own experience is that the original research is almost invariably more accurate, and all too often a lot more accurate, than the news stories about it. If that's the case, we should be citing the original research, instead of relying on the news story. A news story can be helpful for establishing the nontechnical context, but for the facts and figures themselves it's almost never a better source.
• I agree that the original research article is often too technical for most WP editors. My experience, however, is that when that happens, the article is too technical for science reporters as well. One case in point is Abrahams & Geschwind 2008 (PMID 18414403), arguably the best current review of the genetics of autism. This is a source that I suspect no professional news reporter can read and understand; at least, if there is such a reporter, I don't know who it would be. In autism, science reporters tend to go off and write stories on hypothesized environmental influences, in part because that's stuff they do understand.

Eubulides (talk) 16:49, 9 September 2008 (UTC)

What Schwitzer says

This proposed section misstates the source:

The popular press is generally not a reliable source for science and medicine articles. Most news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits.

Schweitzer studied newspaper and TV news stories, not the popular press in general. His summary says nothing about popular science magazines, such as Scientific American, New Scientist, WebMD, etc.

I'm all for criticizing the press, but I also believe in objective research and having a solid scientific basis for your research.

I object to any statements about the popular press that simply reflects the personal opinions, prejudices or WP:OR of the editors.

There are lots of evaluations of the accuracy of popular scientific magazines. If you believe in peer review and the scientific method, you should read that literature and use that as the basis of your statements about the popular press.

For example, the standard reference book used by librarians to evaluate periodicals is Magazines for Libraries, which used to be edited by Bill Katz. The book has several chapters, at least one of which deals with scientific magazines, written by a librarian with a scientific degree, and I believe the chapter itself is peer-reviewed. (I have a copy filed away somewhere.) A statement about popular science books cited to Magazines for Libraries is reliable and meaningful. A statement based on the personal opinions of a WP editor (or even a majority of editors) is unreliable and meaningless.

Broad statements about the "popular press" or "popular scientific press" are also WP:WEASEL. How do you define "popular scientific press"? Is Scientific American or New Scientist the popular scientific press (and unreliable)? What about WebMD (which is reviewed by doctors)? What about IEEE Spectrum? Or the news sections of Science (which is often more reliable than the peer-reviewed articles they report on)?

Terms like "generally" are WP:WEASEL. They're using vagueness as a substitute for evidence. ("Let's compromise and say 'generally'.") Schwitzer doesn't say "generally," he says (based on research) that a majority didn't meet his predefined standards. Nbauman (talk) 15:36, 9 September 2008 (UTC)

• The citation supports a claim about "news articles", not "popular press".
• The proposed text clearly states that it includes Scientific American and New Scientist among the popular press.
• As Colin wrote, this is a guideline not an article, and it doesn't need to cite sources to support its advice. If this sort of citation-quibbling is going to be a problem, I propose that we simply remove the citation to Schwitzer 2008. The text doesn't need that citation to reflect the working consensus of Wikipedia editors in this area.

Eubulides (talk) 16:49, 9 September 2008 (UTC)