Search This Blog

Friday, June 20, 2014

Should We Believe the Centers for Disease Control and Prevention’s Autism spectrum Disorder Prevalence Estimates?

From Sage Journals' Autism

July, 2014 Vol. 18 No. 5  Pages 482-484

In March 2014, the Centers for Disease Control and Prevention (CDC) released their most recent estimate of the prevalence of autism spectrum disorder (ASD) among children aged 8 years (CDC, 2014). Their startling finding was that 1 in 68 children met criteria for ASD in 2010, compared with 1 in 88 in 2008 (CDC, 2012), 1 in 110 using combined data from 2006 and 2004 (CDC, 2009), and 1 in 150 in 2002 (CDC, 2007).

This dramatic increase has raised many concerns regarding possible causes, with much attention given to toxic exposure in the perinatal period (Rodier, 2011) and broadening of the diagnostic criteria (Newschaffer et al., 2007). While these and other hypotheses certainly are worthy of further exploration, we believe that this apparent increase should raise as many concerns about the study methods themselves as it does about other reasons for the observed change in prevalence.

The CDC surveillance studies have performed an important service for policy makers, system administrators, and advocates. Prior to the first surveillance study that used data from 2000, the only consistent count of the number of children with autism in the United States came from the report each state provided to the US Department of Education of the number of children served in the autism category of special education (Shattuck, 2006; Shattuck and Grosse, 2007). This count did not start until 1990, when autism became a separate disability category under US special education law.


These types of administrative counts are fraught with problems. They are subject to the idiosyncrasies of state reporting systems, there is no validation of diagnosis, and the potential for missed cases is substantial.

CDC-sponsored surveillance studies of ASD responded to the limitations of administrative data by putting in place a cost-efficient, population-based protocol for estimating ASD prevalence. The surveillance protocol is based on one the CDC developed for use in the Atlanta, Georgia, metropolitan area (Yeargin-Allsopp et al., 2003). In the original study using this protocol with data from 1996, the medical and education records of all children aged 3–10 years in a 5-county area were examined, looking for any indication in the records that children may have had ASD, either because they had that diagnosis in their record or because there were enough other “red flags” that suggested its presence. An expert clinician then reviewed the records of all children with suspected ASD and made a case determination based on those records. That study found that 1 in 294 children met criteria for ASD, and that prevalence peaked at age 8, which is why CDC surveillance studies since then have estimated prevalence in 8-year-olds.


Of great importance, 98% of children with an ASD diagnosis in their medical records, 99% of children in the autism category of special education, and 100% of children with both, were classified as autism cases. That is, there were almost no false positives. Yet, the study clinicians did not directly observe a single child to validate their record review process. On the other hand, 18% of children meeting research criteria for autism did not have that diagnosis in their records.

This study was rightly lauded as a landmark in estimating the prevalence of autism. Its methods became the backbone of the multi-site studies the CDC then funded to continue its important surveillance work. In 2007, a report was published from six sites that used the same methods as the CDC had in Atlanta, but this time examining the records of children who were 8 years old in 2000 (CDC, 2007). They found that 1 in 150 children met criteria for ASD, almost twice the prevalence found in the original study, but almost exactly the same as was observed when the CDC conducted a door-to-door prevalence study in Brick Township, New Jersey, in which they clinically assessed a large portion of the population of children in person (Bertrand et al., 2001).


In this 6-site study, prevalence across sites varied from 1 in 222 to 1 in 101. The boy:girl ratio varied twofold across sites. The prevalence of intellectual disability among children with ASD ranged from 40% to 62%.

The study expanded to 14 sites for the surveillance of children who were 8 years old in 2008 (CDC, 2012). In that study, which found an overall prevalence of 1 in 88, prevalence across sites ranged from 1 in 208 to 1 in 47. Racial and ethnic differences in prevalence were observed, with traditionally underserved minority children less likely to meet criteria for a diagnosis. Only 38% of children had a documented intellectual disability, although this ranged from 13% to 54% across sites.

The most recent study, examining records of children who were 8 years old in 2010, relied on data from 11 sites (CDC, 2014). The range of prevalence across sites was similar to the prior study, ranging from 1 in 46 in New Jersey to 1 in 175 in Alabama. Racial and ethnic prevalence differences remained. This time, only 31% of children had a documented intellectual disability, ranging from 18% to 37% across sites.

To those who read these studies carefully, the cross-site variation should be as startling and alarming as the apparent increase in prevalence over time. Why should ASD respect geo-political boundaries? Could it be that there are characteristics of the physical environment in New Jersey that result in a greater prevalence of autism than in Alabama? Why would the proportion of children with ASD and intellectual disability vary so greatly across sites? Why would one observe racial and ethnic disparities that differed by site?

The discussion sections of these papers and subsequent commentaries suggested that local policies, resources and awareness may drive observed differences in prevalence. We agree these are likely sources of these observed differences. But if they are, then these studies are not measuring true prevalence and should not be advertised as such. They instead measure the extent to which clinicians and educators test for and document the symptoms of autism, regardless of whether the practitioner ultimately assigns that diagnosis.


For example, as awareness of the importance of social ability and its precursors increases (in large part due to increased awareness of autism), clinicians may be more likely to make notations about eye contact or joint attention. They may note related impairments in the chart. Even if they do not ultimately diagnose the child with an ASD, the expert clinical reviewer at the surveillance site may still think there is enough information to warrant a diagnosis. In other words, in a number of cases, the community clinician has not assigned a diagnosis, but the CDC clinician is overriding this decision based on a chart review.

It is interesting to note that 80% of children in the most recent study had a previous diagnosis or classification of autism, almost exactly the same as in the original study conducted in Atlanta. Based on that statistic, community practitioners were no better in 2010 in identifying cases than they were in 1996. That is, in both years, they identified only 4 out of 5 CDC-determined cases.

In a “true” prevalence study, the information a child has in their clinical or educational record is irrelevant. Researchers identify some population or population-based sample and clinically assess individuals in person to determine the presence of ASD. The CDC did not rely on this in-person strategy, presumably because of the high costs. The result, however, is that the data they have collected may be uninterpretable as it relates to prevalence. Simply put, without direct assessments of children, we will not know the extent to which the CDC-determined “cases” include false positives, or the extent to which children who it was determined do not have autism are really false negatives.


Social impairments and repetitive behaviors are present in many other childhood psychiatric disorders and developmental disabilities (Casey et al., 2013). The flaws in this methodology certainly could explain the great variation in prevalence, clinical presentation, and racial disparities by site.

Tracking ASD is no easy task. In addition to the changes in diagnostic criteria, ASD is clinically complex and has no established biomarkers (Lai et al., 2014). The CDC surveillance studies have resulted in rich datasets from which much important research has been published regarding disparities in diagnosis (Giarelli et al., 2010;Mandell et al., 2009), age of diagnosis (Shattuck et al., 2009), and clinical presentation (Maenner et al., 2013). We question, however, whether they should be used any longer to provide meaningful estimates of prevalence. In fact, we believe it is a mistake to do so.

References


 Bertrand J, Mars A, Boyle C, et al. (2001) Prevalence of autism in a United States population: the Brick Township, New Jersey, investigation. Pediatrics 108(5):1155–1161.
Abstract/FREE Full Text

Casey B, Craddock N, Cuthbert B, et al. (2013) DSM-5 and RDoC: progress in psychiatry research? Nature Reviews Neuroscience 14(11): 810–814.
CrossRefMedlineOrder article via InfotrieveSearch Google Scholar
CDC (2007) Surveillance summaries. Morbidity & Mortality Weekly Report (MMWR) 56(SS-1): 1–28.

CDC (2009) Prevalence of autism spectrum disorders: autism and developmental disabilities monitoring network, United States, 2006. MMWR Surveillance Summaries 58(10): 1–20.

CDC (2012) Prevalence of autism spectrum disorders: autism and developmental disabilities monitoring network, 14 sites, United States, 2008. MMWR Surveillance Summaries 61(3): 1–19.

CDC (2014) Prevalence of autism spectrum disorders among children aged 8 years: autism and developmental disabilities monitoring network, 11 sites, United States, 2010. MMWR Surveillance Summaries 63(2): 1–22. Web of Science
Giarelli E, Wiggins L, Rice C, et al. (2010) Sex differences in the evaluation and diagnosis of autism spectrum disorders among children. Disability and Health Journal 3(2): 107–116. CrossRefWeb of ScienceSearch Google Scholar
Lai M, Lombardo M, Baron-Cohen S (2014) Autism. The Lancet 383(9920):896–910. Search Google Scholar

Maenner M, Schieve L, Rice C, et al. (2013) Frequency and pattern of documented diagnostic features and the age of autism identification. Journal of the American Academy of Child and Adolescent Psychiatry 52(4): 401–413. CrossRefMedlineOrder article via InfotrieveSearch Google Scholar

Mandell D, Wiggins L, Carpenter L, et al. (2009) Racial/ethnic disparities in the identification of children with autism spectrum disorders (Article). American Journal of Public Health 99(3): 493–498. CrossRefMedlineOrder article via InfotrieveWeb of ScienceSearch Google Scholar
Newschaffer C, Croen L, Daniels J, et al. (2007) The epidemiology of autism spectrum disorders (Review). Annual Review of Public Health 28: 235–258. CrossRefMedlineOrder article via InfotrieveWeb of ScienceSearch Google Scholar

Rodier P (2011) Environmental exposures that increase the risk of autism spectrum disorders. In: Amaral D, Dawson G, Geschwind D (eds) Autism Spectrum Disorders. New York: Oxford University Press, pp. 863–874. Search Google Scholar

Shattuck P (2006) The contribution of diagnostic substitution to the growing administrative prevalence of autism in US special education. Pediatrics 117(4):1028–1037. Abstract/FREE Full Text

Shattuck P, Grosse S (2007) Issues related to the diagnosis and treatment of autism spectrum disorders. Mental Retardation and Developmental Disabilities Research Reviews 13: 129–135. CrossRefMedlineOrder article via InfotrieveWeb of ScienceSearch Google Scholar

Shattuck P, Durkin M, Maenner M, et al. (2009) Timing of identification among children with an autism spectrum disorder: findings from a population-based surveillance study. Journal of the American Academy of Child and Adolescent Psychiatry 48(5): 474–483. CrossRefMedlineOrder article via InfotrieveWeb of ScienceSearch Google Scholar

Yeargin-Allsopp M, Rice C, Karapurkar T, et al. (2003) Prevalence of autism in a US metropolitan area. JAMA 289(1): 49–55. CrossRefMedline

About the Authors

Luc Lecavalier, Ph.D. graduated from the Université du Québec à Montréal in 2001 with a Ph.D. in Psychology. He is interested in diagnosis and treatment of behavioral and psychiatric problems in individuals with intellectual disability and/or autism spectrum disorders. He has authored or co-authored more than sixty scholarly publications. In 2008, he was recipient of the Early Career Awards given by the American Association on Intellectual and Developmental Disabilities (AAIDD) and Division 33 of the American Psychological Association (APA). Dr. Lecavalier is Associate Editor of Autism: International Journal of Research and Practice and is a member of several Editorial Boards including the Journal of Autism and Developmental Disorders. He is Director of the Child Unit of Behavior Support Services at the Nisonger Center of the Ohio State University.

David S. Mandell, Sc.D., University of Pennsylvania, Associate Professor of Psychiatry and Pediatrics, directs the Center for Mental Health Policy and Services Research. Dr. Mandell is also Associate Director of the Center for Autism Research at The Children’s Hospital of Philadelphia. The goal of his research is to improve the quality of care individuals with autism receive in their communities. This research is of two types. The first examines, at the state and national level, the effects of different strategies to organize, finance and deliver services on service use patterns and outcomes. The second consists of experimental studies designed to determine the best strategies to successfully implement proven-efficacious practices in community settings. Dr. Mandell holds a bachelor of arts in psychology from Columbia University and a doctorate of science from the Johns Hopkins School of Hygiene and Public Health.

No comments:

Post a Comment