The internet now plays a key role for many parents of children with autism. Following the diagnosis of their child a significant proportion of parents visit autism-related sites to look for information about intervention (Chowdhury, Drummond, Fleming, & Neitfeld, 2002; Mackintosh, Myers, & Goin-Kochel, 2005; Sabo & Lorenzen, 2008; Stephenson, Carter, & Kemp, 2012).

There is not only a plethora of autism interventions, but a wealth of online information about them readily available for parents to access quickly and easily (Miller, Schreck, Mulick, & Butter, 2012). Parents of children with autism have been found to be more likely to obtain information from websites than consult other parents or health professionals (Mackintosh, et al., 2005).

With so many parents accessing information via the internet there are anxieties about the quality of this information given that content is so varied (Sabo & Lorenzen, 2008) and could be inaccurate, misguided, out of date, or be produced by authors who have particular motives or biases (Chowdhury, et al., 2002; Stephenson, et al., 2012).

There are also other complexities with online information because it can change rapidly: if parents were to enter the same keyword ‘autism’ into a search engine on different days, it is likely to yield different results (Reichow, Naples, Steinhoff, Halpern, & Volkmar, 2012).

The main concern is that parents could make decisions about interventions for their child based on online information which is false, biased, or has no sound basis in empirical evidence (Grant, Rodger, & Hoffmann, 2014). This study therefore evaluated the quality of treatment information on 20 autism-related websites over two days in 2013.

Method

The authors carried out the evaluation over two days and went through a series of stages:

Selection and filtering

• The authors selected 10 keywords which parents of newly diagnosed children with autism may use and entered these systematically into 3 major worldwide search engines (Google, Yahoo, and MSN). Different types of media such as video, advertisements and news items were not included.
• They extracted the top 20 website hits for each of the 10 key words. These websites were entered into an excel spreadsheet.
• The authors ranked the websites in descending order according to the number of times they appeared within that spreadsheet, creating a list of top 20 websites.
• It was these top 20 websites were then part of the evaluation.

Evaluation

• The 20 websites that appeared the most were evaluated by two independent reviewers. They used the DISCERN tool (Charnock, Shepperd, Needham, & Gann, 1999), which is designed to evaluate written health material that provides information on treatment options within three main areas: 1) reliability/dependability of information, 2) quality of information on choices of treatment, and 3) the overall quality of a website. Questions are rated on a Likert 1-5 scale (1=information did not meet all the criteria/ 5= all criteria met), with an overall total score of 80. Scores were then reviewed by the two reviewers and any discrepancies (where there were scores that were more than a point apart) were then discussed and altered in agreement. A mean overall score was produced for each website and each individual item on the DISCERN tool.
• Reviewers then explored the references/sources used to support the website content, grouping them into one of three categories: 1) no references or sources, 2) some references or sources but with no empirical research evidence, 3) research references provided.

Findings

• The overall mean DISCERN score for websites was 46.5 out of a maxiumum of 80. The highest scoring site had a mean of 67.5, and the lowest 25. Based on an interpretation of the DISCERN total score (Hargrave, Hargrave, & Bouffet, 2006) the overall quality was rated as ‘Fair’ (scores ranging from 39-50).
• For individual items of the DISCERN tool, the highest mean score for all 20 sites was item 7 (Does it provide details of additional sources of support and information?), and the lowest were items 11 (does it describe the risks of each treatment?) and 12 (does it describe what would happen if no treatment is used?)
• Further analysis of the source of intervention information on the websites revealed that just over a quarter (6/20) of the websites provided references or sources when describing the intervention options. 4 of these sites were in the top searched results and featured in the top 10 of the websites when ranked by DISCERN scores.
• Among the other 14 sites, 8 provided references or sources that were not of an empirical, research nature, and 6 provided no references or sources to support any of the information about the interventions on the website.

Conclusion

Overall the evaluation reported that the quality of the information provided on the 20 autism-related sites was rated to be ‘Fair’ (range scores 39-50). It also revealed that there are some significant aspects of intervention information that scored poorly on the DISCERN tool, namely the risks of treatment and what happens if no treatment is pursued.

The authors debate some of the possible solutions in regard to the results. Content managers of autism-related websites could more effectively match website aims to content, and ensure that the references, or lack of references, are made more apparent to readers.

They also suggest that website developers and managers consider using the DISCERN tool to evaluate their own content, given that it is freely available to use and requires little training.

The authors contend that controlling for what is presented on websites alone is not the solution, and indeed this is a complex and onerous task. They refer to the idea that service-providers could ‘filter’ information (Chowdhury, et al., 2002) and provide parents with an up-to-date list of reputable sites, although they recognise that this in itself would be difficult given the fast changing nature of the internet and the time and resource it would take to do this.

They describe the need for parents or consumers of intervention information to be supported in determining the quality of the information on websites for themselves. This could be assisted by health professionals.

Overall, they maintain that when considering an intervention, a discussion with a health professional should always be considered above all other forms of consultation.

Strengths and limitations

This is an interesting and timely paper which fills a research gap and highlights some of the issues related to the quality of intervention information available on the internet.

Parents of children with disabilities are likely to turn to the internet for answers both before and after the diagnosis, and therefore what is readily available to parents rightly needs to be trustworthy and evidence-based.

The findings of this paper revealed some ethical concerns (such as a lack of mention of intervention risk or alternative pathways) when evaluating website intervention information, which are crucial to tackle if we are to help parents or consumers to make informed choices.

The authors discuss their evaluation procedure in an orderly way. This included in-depth information about their selection of search engines, keyword terms, website scoring and ranking, and how certain decisions were made (for example the selection of the key terms).

The moderation of scores by two independent reviewers who discussed conflicting scores ensured consistency. They also mention some of the difficulties in conducting this type of internet-based research, such as websites frequently changing and the issues with websites and PDF documents being housed within sites which are not always owned by the page owner.

I would have perhaps liked to have known a little more about how these issues were dealt with because it would aid future researchers who either try to replicate or expand on this type of study.

The sample size of websites evaluated is notably small (n=20), however given the large amount of hits generated from the 10 key terms in 3 search engines search it would take a significant amount of time and resource to evaluate a larger number. However it is perhaps also the case that few internet searches continue past the first 20 results.

Furthermore, as mentioned in the paper, content on the internet changes rapidly and thus it may only be possible to take a snapshot of content rather than conduct a larger and more longitudinal analysis. The findings may also be bound to that context, as while the 3 most used search engines in the world were used in the study, the authors noted that there would be some Australian bias because MSN and Yahoo are both powered by Bing and default to the Australian network.

Furthermore an individual’s own computer search history will influence the hits that are generated.

The paper clearly breaks down the results into different levels providing the overall mean for: all websites, individual websites and individual items, as well as a separate analysis of the number of references. While this may be dictated by the tool, it seems that the results of all the websites could have been interrogated a little further.

Furthermore in Table 2, item 6 (Is it balanced and unbiased?) scored a mean of 3.4 out of a possible 5 and when compared to other items was the third highest item. When we are thinking about the quality of information, it appears that an important aspect such as this did not fair too badly.

There is also a question of whether the DISCERN tool could include an item which looks at whether evidence provided is of an empirical nature, rather than relying on ordinal forms of analysis (research sources / sources (no research) / no research or sources).

The authors discuss in great detail the implications of their findings and I think this is one of its greatest strengths because it has practical relevance. They justifiably advocate a need for a multifocal approach to addressing this issue which requires reforming autism websites but also enabling and supporting parents and consumers to be more critical of what they read on websites.

Arguably, they should also be helped to recognise that even empirical research studies could be misinterpreted (either unintentionally or deliberately) on websites, for example by not including the author’s caveats or limitations.

The paper advocates further research in this area that explores what websites are most desirable and valuable to parents. I agree that this would be useful moving forward and would tell us more about why parents visits particular sites and how important the quality of information may be for different purposes.

It is also worth noting the positive aspects of autism-related websites which may provide parents with instant information which they can access and process at their own leisure, as well as share information with other parents through an online community.

Summary

The evaluation has revealed that 20 websites did not meet the criteria for quality health information, with many failing to cite evidence for the interventions discussed on their site. Improved content with information that is rigorous, trustworthy, and grounded in quality empirical research evidence is essential to assist parents in making informed decisions about their child’s future. It is extremely important to make parents and consumers of autism information aware that the content on websites varies massively and to keep this in mind when searching.

They should also be reminded of the primacy of professional agency information. Moving forward there also needs to be some strategy where approved sources of autism-related information can feature as the top search links within the top worldwide search engines.

Links

Primary paper

Grant, N., Rodger, S. and Hoffmann, T. (2015), Evaluation of Autism-Related Health Information on the Web. Journal of Applied Research in Intellectual Disabilities, 28: 276–282. [abstract]

Other references

Charnock, D., Shepperd, S., Needham, G., & Gann, R. (1999). DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. Journal of epidemiology and community health, 53(2), 105-111

Chowdhury, J., Drummond, J., Fleming, D., & Neitfeld, S. (2002). Content analysis of online autism specific sites. Journal on Developmental Disabilities, 9(2), 157-165

Grant, N., Rodger, S., & Hoffmann, T. (2014). Evaluation of Autism-Related Health Information on the Web. Journal of Applied Research in Intellectual Disabilities.

Hargrave, D. R., Hargrave, U. A., & Bouffet, E. (2006). Quality of health information on the Internet in pediatric neuro-oncology. Neuro-oncology, 8(2), 175-182.

Mackintosh, V. H., Myers, B. J., & Goin-Kochel, R. P. (2005). Sources of information and support used by parents of children with autism spectrum disorders. Journal on Developmental Disabilities, 12(1), 41-51.

Miller, V. A., Schreck, K. A., Mulick, J. A., & Butter, E. (2012). Factors related to parents’ choices of treatments for their children with autism spectrum disorders. Research in Autism Spectrum Disorders, 6(1), 87-95

Reichow, B., Naples, A., Steinhoff, T., Halpern, J., & Volkmar, F. (2012). Brief Report: Consistency of Search Engine Rankings for Autism Websites. Journal of Autism and Developmental Disorders, 42(6), 1275-1279.

Sabo, R. M., & Lorenzen, J. M. (2008). Consumer health web sites for parents of children with autism. Journal of Consumer Health on the Internet, 12(1), 37-49.

Stephenson, J., Carter, M., & Kemp, C. (2012). Quality of the information on educational and therapy interventions provided on the web sites of national autism associations. Research in Autism Spectrum Disorders, 6(1), 11-18.