MELANOMA INFORMATION ON THE INTERNET: OFTEN INCOMPLETE OR INACCURATE. A PUBLIC HEALTH OPPORTUNITY?

Participants: C.K. Bichakjian, T. Johnson, J.M. Hall, J.L. Schwartz, T.S. Wang, J.S. Biermann

Keywords: melanoma, internet use, patient information

Introduction

Patients faced with life-threatening disease often turn to many sources for medical information about their disease, including, more recently, the Internet.

The incidence of melanoma is rapidly rising, and it is now the seventh most common cancer in the United States. Unlike many other malignancies, there are clearly identified prevention strategies such as use of sunscreen, midday sun avoidance, and protective hats and clothing, which can reduce the risk of melanoma. Risk factors for melanoma such as sunburn, fair skin and hair, family history of melanoma, and nevi similarly have been described. However, the use of protective strategies and the identification of persons at high risk require education both of patients and families of those with melanoma, as well as of the general public. The Internet is one such potential educator.

In this study, we evaluated the completeness and accuracy of medical information on melanoma from Internet sites that were retrieved by the use of search engines.

Materials and Methods

We retrieved information regarding the relative frequency of use of search engines, and selected the six search engines that were reportedly used most frequently. Additionally, we selected two medical search engines.

Using a method modified from Biermann et al,¹ we used each search engine to retrieve the first 30 URLs searching the term "melanoma". The Web site of each accessible URL generated by the search engines was evaluated using a 35 point checklist (see Table 1)

Reliability of ratings. To determine whether the rating system would produce valid and reliable data, the first 35 articles were independently evaluated by all four raters and the level of agreement across all rater pairs for each of the thirty-five evaluated factors was computed using Cohen’s kappa. (The 35 factors that were evaluated are listed in Table 1.)

Descriptive statistics. Ratings from a single rater for each article were used in the primary analysis. For the 35 articles that were evaluated by all four raters, the ratings included in the analysis were randomly selected from the four sets of ratings. The remaining 39 articles each had only a single set of ratings, all of which were included in the analysis. Descriptive statistics were used to determine the number and percentage of sites that mentioned each of the 35 factors included in the rating system.

Results

The eight search engines collectively referred to a total of 74 unique sites that were accessible.

Reliability of ratings. The mean kappa for all variables and all rater pairs for which a kappa could be calculated was 0.818. This represents excellent overall inter-rater agreement. Only 6 of the 35 factors showed marginal to poor agreement (kappa <70).

Descriptive statistics. Of the 35 factors that comprised the rating system (see Table 1), only 8 factors were present on at least half of the evaluated sites. The definition of melanoma occurred the most frequently, with 46 sites (62%) including an accurate definition. Prognosis was the second most frequently included factor, with a mention on 44 sites (59%).

Factors that were mentioned on fewer than a quarter of the evaluated sites included Epiluminescence (4 sites, 5%); Pregnancy/Hormones (7 sites, 9%); Alternative Medicine (10 sites, 14%), Mucosal/Vaginal Melanoma (12 sites, 16%); Work-Up (15 sites, 20%); Solicitation to buy products (15 sites, 20%); Risk Factors — Personal History (16 sites, 22%); Follow-Up (17 sites, 23%); Treatment — Sentinel Lymph Node Biopsy (17 sites, 23%); and Diagnosis — Other Histological Parameters (17 sites, 23%).

Factors related to prevention were present on only about a third of the evaluated sites, with Sunscreen mentioned on 27 sites (36%), Hat/Clothing mentioned on 25 sites (34%), and Midday Sun Avoidance mentioned on 23 sites (31%). Risk factors were mentioned on fewer than half the sites, including sunburn (35 sites, 47%) personal history (16 sites, 22%), normal nevi (24 sites, 32%) dysplastic nevi (25 sites, 34%).

We noted a total of 25 inaccuracies in 18 (25%) of the 74 Web sites evaluated. Melanoma incidence was the factor most frequently rated as present but inaccurate, with 8 sites (11%) misreporting melanoma incidence. Work-up was the second most frequently occurring present but inaccurate factor, with 4 sites (5%) so rated.

Discussion

Medical information retrieved on the search term "melanoma" frequently contained inaccuracies, and was likely to lack information on risk factors and prevention. Oncologists can help patients by recommending comprehensive and accurate Web sites for patient review, by working to create accurate and thorough Web material, and by educating patients and the public about the variability of Web content both in accuracy and completeness.

References

[1] Biermann JS, Golladay GJ, Greenfield MLVH, Baker LH: Evaluation of Cancer Information on the Internet. Cancer 86(3):381-390, 1999.

Table 1. Number Of Websites Mentioning Specified Factors