Validation of the short version of the dimensional inventory for child development assessment

Abstract

Objectives

There is a critical need to monitor the development of children around the world, and in Brazil, this need is substantial since there is a paucity of assessment tools. This study aimed to describe the design and provide evidence of reliability and validity for the short version of the Dimensional Inventory for Child Development Assessment (IDADI-short).

Methods

A sample of 1,865 biological mothers of children aged 4–72 months (M = 34.8, SD = 20.20) completed the IDADI to assess Cognitive, socio-emotional, Expressive, and Receptive Language and Communication, Fine and Gross Motor, and Adaptive Behavior development. The psychometric properties of a total of 118 subscales of IDADI were obtained and the IDADI-short age-specific scores were correlated with the original inventory, and criteria variables such as neurodevelopment diagnosis, socioeconomic status, and sex.

Results

Item Response Theory analysis, Cronbach's Alpha, and McDonald’s Omega indicated excellent internal consistency and optimal participant discrimination after minor alterations. IDADI-short scores were strongly associated with the original inventory, with high sensibility and specificity precision for developmental delays. Significant associations with relevant criteria variables were also observed.

Conclusion

Findings support the use of IDADI-short as a parental measure of young children’s development.

Introduction

Developmental researchers have long recognized the complex and multidimensional perspectives needed to study human developmental phenomena.¹ Child development makes the investigation more complex as changes are observed at a surprising rate.² This phenomenon is characterized by remarkable and rapid bio-behavioral changes³ wherein the fast physical and neural development, social relations, and the environment interact favoring great advances in the child's capacity during this sensitive period of development.^4,5

Early healthy development is an important determinant of long-term mental and physical outcomes. There is substantial evidence supporting the importance of early identification of developmental delays and intervention for children’s developmental health and well-being.^6,7 This phenomenon is related to the impact that early life experiences and genetic dispositions have on cognitive, social, behavioral, and emotional development.^4,8 As such, it is essential to assess the development of children with and without developmental disabilities, as evidence has shown that “early intervention might be even more important for children with developmental disabilities than for typically developing children, because learning is cumulative, and barriers to healthy development early in life may impede development in subsequent stages”.²

In early childhood, negative outcomes might be mitigated when delays or development problems are detected and treated early.^7,8 Unfortunately, studies have shown that primary care professionals detect less than one-third of delays, and it is estimated that clinical assessments can detect only 30% of children with delays before they reach school age.⁹ Accordingly, the use of screeners of developmental milestones has been increasingly recommended as an important resource for improving the quality of evaluations in public health services.¹⁰ Although health professionals emphasize the screening of young children’s development, there is a paucity of standardized assessment tools with normative data for the Brazilian context.¹¹ The scarcity is more critical when considering the population of children below three years of age living in Brazil.¹¹ Recently, the third edition of the Bayley Scale of Infant Development (BSID)¹² started to be commercialized, but there are some concerns regarding the adaptation process such as the use of outdated North-American norms, and the lack of a cultural adaptation study. The high cost associated with this tool and the requirement of highly trained professionals is another limitation that has precluded its use on a large scale. The under detection of children with developmental delays is thus a particularly critical concern in Brazil.

Given the paucity of instruments designed for use in the Brazilian context, there is a clear demand for an instrument that can benefit both the research and clinical communities. For this, the Dimensional Inventory for Child Development Assessment (IDADI; Inventário Dimensional de Avaliação do Desenvolvimento Infantil) was developed by Silva, de Mendonça Filho, and Bandeira.¹³ An extensive, multi-phased process was taken to develop items for the IDADI, wherein IDADI’s items were either inspired from existing instruments and modified for the parental report, or newly written specifically for the Brazilian context. This included content scrutiny by 18 subject matter experts, semantic analysis using a four-session focus group with seven mothers of low socioeconomic status, which were purposefully selected to check item comprehension, and a pilot study with 12 mothers who answered the full set of IDADI items.¹⁴ IDADI also underwent various studies to obtain validity and reliability evidence.^15,16

IDADI administration takes around 30−40 min, and psychometric analyses indicated that the 435 items cover a great continuum of developmental abilities, indicating that IDADI surveys more than just developmental milestones. Due to the lengthy testing time, the need for a short version that could favor the identification of children who need special attention is crucial for effective monitoring in primary care settings. Thus, the objective of the present study was to empirically derive a short version of the Dimensional Inventory for Child Development Assessment (IDADI-B) and gather reliability and validity evidence.

Method

Participants and procedures

A convenience sample of 1865 biological mothers of children aged 4–72 months (M = 34.8, SD = 20.20) took part in the study. Approximately one-third of participants were recruited from public and private institutions of education, university clinical service centers, and health service centers (public, private, and philanthropic sectors) located across different regions of Brazil. These participants completed paper-and-pencil versions of the IDADI. The remaining participants (66.8%) were recruited via the internet using a media release coordinated by the research team. Internet participants completed an online questionnaire on the SurveyMonkey® platform. Only cases with complete data from the Sociodemographic and Clinical Questionnaire of the online data collection were considered. Both types of data collection included the same set of items and instruments. Mothers and their children were eligible for the study if the mother and child lived together and had daily contact.

Most participants were from the South region of Brazil (67.7%), followed by the Southeast (22.5%), Northeast (4.7%), Midwest (3%), and North (1.6%). The majority of mothers (79%) self-identified as white, 15.4% brown, 4% black, 1.5% yellow (i.e., East Asian), and 0.2% indigenous. Children’s sex distribution was approximately equal with a slight majority identified as male (51.1%). A small number of cases (approximately 5%) reported a developmental diagnosis (2% autism spectrum disorder, 0.6% Down syndrome, 0.5% cerebral palsy, 0.5% developmental coordination disorder, 0.2% intellectual disability, and 0.8% had another type of diagnosis). In terms of economic status, 37.8% of the women had a family income greater than five times the minimum wage and only 4.4% had an income of less than one minimum wage per month. The majority (69.0%) of mothers were involved in paid work (Table 1).

Table 1.

Sample demographics.

Age group (months)	Female			Male			Total
	N	M	SD	N	M	SD	N	M	SD
4 m - 5 m	32	4.87	0.5	47	5	0.5	79	4.95	0.5
6 m - 8m	64	7.56	0.86	72	7.43	0.8	136	7.48	0.83
9 m - 11 m	48	10.4	0.84	45	10.35	0.82	93	10.38	0.83
12 m - 14 m	61	13.64	0.89	43	13.46	0.85	104	13.56	0.87
15 m - 17m	52	16.33	0.9	48	16.46	0.91	100	16.4	0.9
18 m - 20m	40	19.26	0.81	47	19.62	0.93	87	19.46	0.89
21 m - 23 m	38	22.83	0.75	41	22.51	0.92	79	22.67	0.85
24 m - 26 m	46	25.55	0.87	45	25.71	0.81	91	25.62	0.84
27 m - 29 m	35	28.41	0.84	36	28.74	0.84	71	28.57	0.85
30 m - 32 m	31	31.59	0.91	40	31.49	0.56	71	31.53	0.73
33 m - 35 m	30	34.71	0.82	44	34.52	0.91	74	34.59	0.87
36 m - 41m	69	39.09	1.89	85	39.07	1.55	154	39.07	1.71
42 m - 47 m	75	44.68	1.61	78	44.89	1.76	153	44.78	1.69
48 m - 53 m	64	50.94	1.77	71	50.8	1.65	135	50.85	1.71
54 m–59 m	71	56.89	1.66	71	57.45	1.63	142	57.17	1.66
60 m–65 m	72	62.6	1.64	79	63.34	1.7	151	62.97	1.72
66 m–72 m	58	69.25	1.74	74	69.45	1.89	132	69.36	1.82
Maternal education	N	%		N	%		N	%
Illiterate	0	0		2	0		2	0.1
Elementary school	4	1		13	1		17	1
Less than high school	33	4		33	3		66	4
Highschool	184	20		173	18		357	20
Incomplete undergraduate	115	13		121	13		236	13
Undergraduate degree	189	22		223	23		412	22
Graduate degree	358	41		398	41		756	41
Family income (monthly wage)	N	%		N	%		N	%
Less than 1 MW	38	3		26	3		64	3
1 to 2 MW	117	15		148	15		275	15
2 to 3 MW	116	12		104	11		220	12
3 to 4 MW	87	10		97	10		184	10
4 to 5 MW	90	11		117	12		207	11
5 to 10 MW	201	22		212	22		413	22
10 to 15 MW	123	14		131	14		254	14
15 MW or more	107	13		125	13		232	13

MN, Brazilian monthly minimum wage.

MN during the data collection window was 937.00 Brazilian Reais, which was equivalent to 297.19 USD.

All participants provided informed consent, which complied with the Brazilian National Health Council guideline 466/12 as well as earlier guidelines. The study was approved by the research ethics committee of the Institute of Psychology from the Federal University of Rio Grande do Sul (UFRGS; CAAE: 45991815.5.0000.5334).

Instruments

Dimensional inventory for child development assessment

The IDADI is a multidimensional inventory that assesses Cognitive, Communication, and Language (Expressive and Receptive), Motor (Gross and Fine), Socio-emotional, and Adaptive Behavior. The answer alternatives for each item are given as follows: 0 = not yet, 1 = sometimes and 2 = yes. Items of each developmental domain are divided across 17 age-bands (with exception of the Adaptive Behavior domain that covers 16 age-bands), yielding a total of 118 subscales with a varying number of items depending on the domain and the age-band considered. The detailed item information for each domain and age-group is available in Supplementary Tables S8-S14. The items of each original subscale were selected according to their respective levels of difficulty and precision, which were defined by Item Response Theory (IRT) analyses.

Sociodemographic (SES) and clinical questionnaire

This questionnaire aimed to provide data on variables identified by the literature as being related to the children’s development. Examples of the information that were collected via mother report include: 1) family demographics (i.e., social, cultural, economic, parental education), 2) mother’s health during pregnancy, 3) pregnancy and birth conditions, and the child’s early development, and 4) child’s environment and interactions with caregivers.

Data analysis plan

Intending to empirically derive a short version of IDADI, data analysis consisted of two stages. First, psychometric characteristics of each of the 17 age group subscales from each IDADI domain were investigated using the rating scale model by Andrich,¹⁷ an extension of the 1-parameter (Rasch), IRT model.¹⁸ The purpose of this step was to select the seven most informative items considering the total variation of the latent trait for each age group. The information function is a quantification of how well an individual item and the item set (test or subscales) estimate the person location parameters.¹⁸ Thus, it provides an indication of the items that can estimate child development with the least error. The estimation of the cumulative item information considered the whole range of abilities of each age group. The choice of seven items per age group/domain was based on the average number of items used in gold-standard screening instruments.^19,20

Second, an IRT analysis with the selected items was performed to investigate the fit of the items to the short format. The infit mean-square, a statistic with an approximate chi-square distribution that indicates the discrepancy between the model-predicted probabilities and empirical responses, was used to determine the item fit to the measurement model.²¹ Items were considered adequate if infit values were between 0.5 and 1.5.²¹ We also considered Cronbach’s Alpha and McDonald’s Omega²² to assess evidence of subscale-level reliability. To account for the categorical nature of the item responses, reliability was estimated from the polychoric correlation. Reliability was considered adequate if values were above 0.7.²³ If the criteria for infit and reliability were not met, the eighth-most informative item replaced the item with fit problems or with less contribution to the subscale's reliability. If the criteria adopted were still not met, the next item was selected.

Correlations between IDADI-B and its original version were computed, as well as sensitivity and specificity receiver operating characteristic (RoC) curves for developmental delay classifications. This allowed comparing the true positive rate (sensitivity) and true negative rate (specificity) of IDADI-B screening potential in relation to the original inventory. We standardized IDADI-B scores using means and standard deviations of the sample of children without developmental diagnosis or health problems. Children screened positive for development delay if an IDADI-B z-score below -1 SD was obtained. Multiple regression analyses were performed to gather additional validity evidence based on diagnosis, maternal education, family income, and sex.

Results

The seven domains and 17 age-groups of IDADI resulted in the analysis of 118 subscales. Data from some questionnaires were analyzed in a single run because the items on these questionnaires are exactly the same.

Subscales of the Cognitive domain presented a Cronbach's Alpha and McDonald's Omega that varied from 0.75 to 0.93, and 0.85 to 0.97 respectively, indicating good to excellent consistency values (Tables 2 and S1). Moreover, all items presented an adequate fit to the IRT Rasch model. Table 2 shows the descriptive statistics for the infit values of the most informative items for each age group.

Table 2.

Summary of item fit and reliability of IDADI-B subscales.

Domain	Infit		Alpha		Omega
	M	SD	M	SD	M	SD
COG	0.86	0.14	0.85	0.06	0.91	0.03
SE	0.86	0.16	0.86	0.06	0.91	0.04
MA	0.87	0.16	0.84	0.06	0.91	0.04
MF	0.88	0.12	0.80	0.06	0.88	0.04
CLE	0.87	0.21	0.93	0.07	0.96	0.03
CLR	0.89	0.14	0.89	0.07	0.94	0.04
CA	0.87	0.12	0.77	0.06	0.87	0.04

COG, Cognitive; SE, Socioemotional; CA, Adaptive Behavior; CLE, Expressive Communication and Language; CLR, Receptive Communication and Language; MF, Fine Motor; MA, Gross Motor.

The Socio-emotional domain presented one subscale (4 m–5 m) with an Alpha value below 0.70 and had two items replaced (SE17 by SE16, and SE21 by SE24). After modifications, the resulting Alpha and Omega reliabilities ranged from 0.76 to 0.94, and 0.81 to 0.97 (Table 2 and S2). Item SE50 had misfit for the subscales 30 m – 32 m, 33 m–35 m, 36 m–47 m, and 60 m–72 m and was replaced by item SE64. Items SE54 and SE57 were replaced by SE71 on the 60 m–65 m and 66 m–72 m subscales (Table S9). After alterations, the estimated psychometric parameters were considered acceptable.

All items from the Gross Motor subscales were adequate in terms of infit (Table 2 and S3). Reliabilities were adequate and ranged from 0.73 to 0.93 for Alpha and 0.82 to 0.94 for Omega estimates. The same pattern was observed for the items of the Fine Motor subscales in which reliabilities varied from 0.73 to 0.90 and 0.81 to 0.93 respectively, and all items fit adequately to the IRT model (Table S4).

One subscale (9 m – 11 m) from the Expressive Communication and Language domain had low Alpha reliability and had the CLE3 item replaced by CLE9. The 48 m – 59 m subscale had one item with misfit (CLE40 was replaced by CLE44). The 60 m – 65 m and 66 m – 72 m subscales had the CLE44 item replaced due to Alpha reliability and CLE40 due to misfit, both items were replaced by the CLE41 item (Tables 2 and S12). For the Receptive Communication and Language domains, all subscales had good to excellent reliabilities (α variation: 0.70 to 0.97; ɷ variation: 0.83 to 0.99). One item (CLR30) from the 48 m – 53 m, 60 m – 65 m, and 66 m – 72 m subscales had misfit and was replaced by item CLR31.

Three subscales – 18 m–23 m, 42 m–53 m, and 54 m–59 m – from the Adaptive Behavior domain presented misfitting items that were replaced by the next most informative items (CA16, CA21, and CA23 were replaced by CA15; CA32 by CA36; and CA59 was replaced by CA54). The 6 m–11 m subscale had the CA3 item replaced by C11 due to low reliability. After alterations, Alpha values ranged from 0.70 to 0.91 while Omega values ranged from 0.78 to 0.96 (Tables 2 and S14).

A sample of items from the final version of IDADI-B is available in the supplementary Table S16. Free access to all IDADI-B content is available at the following link: https://vol.vetoreditora.com.br/idadi_breve.

A strong association was observed between the short and original versions of IDADI’s domains. Correlations varied from 0.75 to 0.89 with an average of 0.84 (Table 3). The precision for development delay screening was high with a sensitivity that varied from 0.90 to 0.97 and specificity that ranged from 0.87 to 0.93. The ROC area under the curve was highly significant with values that varied from 0.95 to 0.97.

Table 3.

Correlations and precision of developmental delay screening of IDADI-B in relation to IDADI.

		IDADI							ROC Curve
		COG	SE	MA	MF	CLE	CLR	CA	Sensitivity	Specificity	AUC
IDADI Short	COG	0.86	0.53	0.40	0.53	0.55	0.54	0.49	0.91	0.87	0.96
	SE		0.75	0.39	0.51	0.68	0.62	0.61	0.90	0.87	0.95
	MA			0.86	0.50	0.33	0.38	0.49	0.97	0.88	0.97
	MF				0.89	0.44	0.51	0.56	0.94	0.88	0.97
	CLE					0.83	0.59	0.47	0.94	0.93	0.97
	CLR						0.85	0.54	0.92	0.90	0.96
	CA							0.84	0.91	0.87	0.95

COG, Cognitive; SE, Socioemotional; CA, Adaptive Behavior; CLE, Expressive Communication and Language; CLR, Receptive Communication and Language; MF, Fine Motor; MA, Gross Motor.

All correlations and ROC curves were significant at p < 0.001 level.

Table 4 shows the regression models considering developmental diagnosis, maternal education, family income, and sex of the child, simultaneously. The diagnostic predictor showed significant negative associations in all domains of IDADI-B with the largest effect sizes. Maternal education was positively associated with five domains (Cognitive, Receptive Communication and Receptive Language, Fine Motor, and Adaptive Behavior). Income was positively associated with Cognitive, Fine Motor, and Receptive Communication and Language scores. Girls had higher levels of development in all domains of IDADI-B except for the Gross Motor domain.

Table 4.

Regression results of IDADI-B domains and criteria variables.

Domain	Predictors
	Diagnosis			Maternal Education			Family Income			Child’s sex
	β	t	p	β	t	p	β	t	p	β	t	p
COG	−1.59	−14.12	< 0.001	0.04	2.90	< 0.001	0.04	2.54	< 0.01	−0.14	−2.69	< 0.001
SE	−2.15	−17.19	< 0.001	0.04	2.11	0.04	0.00	−0.13	0.89	−0.32	−5.59	< 0.001
MA	−1.67	−14.42	< 0.001	0.03	1.74	0.08	−0.01	−0.46	0.65	0.09	1.75	0.080
MF	−1.39	−12.36	< 0.001	0.04	2.39	0.02	0.04	2.61	0.01	−0.26	−5.10	< 0.001
CLE	−2.89	−20.67	< 0.001	0.03	1.70	0.09	0.03	1.75	0.08	−0.33	−5.22	< 0.001
CLR	−2.26	−16.72	< 0.001	0.04	2.23	0.03	0.07	3.83	< 0.001	−0.21	−3.42	< 0.001
CA	−1.27	−10.38	< 0.001	0.04	2.56	0.01	−0.01	−0.64	0.523	−0.27	−4.98	< 0.001

COG, Cognitive; SE, Socioemotional; MA, Gross Motor; MF, Fine Motor; CLE, Expressive Communication and Language; CLR, Receptive Communication and Language; CA, Adaptive Behavior.

The predictor child’s sex had the female group as reference category.

Discussion

This study used a rigorous quantitative approach when contrasted with the methods that have been used by other comparable assessments, to develop an empirically-based short version of the Dimensional Inventory for Child Development Assessment, IDADI-B. A large sample from different regions of Brazil was collected to investigate its psychometric properties and gather validity evidence with variables commonly associated with developmental outcomes. The results provided initial evidence of internal consistency and validity to support the interpretation and the free use of IDADI-B scores in research and clinical practice settings.

Psychometric analysis indicated that most items had a good fit for the IRT model. The use of IRT made it possible to evaluate the psychometric characteristics of IDADI-B that cannot be obtained using Classical Test Theory (CTT).²⁴ It allowed us to evaluate the adequacy of an item to a one-dimensional logistic model while modeling the probability of responses of the endorsed categories. As well, the item information curves made it possible to verify the precision of each item along the latent continuum under investigation.¹⁸ The fact that more than 95% of the 300 selected items had infit values within the acceptable range of 0.5–1.5 provided additional psychometric support, indicating that the items in the final version of IDADI-B fit well to the construct under investigation.

The finding that some items which demonstrated good fit in the original inventory but misfit in the short version has been observed in similar studies using the Ages and Stages Questionnaires (ASQ-3).²⁰ A hypothesis raised was that an item may present problems in a specific age-group, but when considered in a wider ability range (multiple age-groups simultaneously) the residuals might be diluted. As the objective of this work was to validate individual subscales for each age group, only items with satisfactory adjustment were considered. It’s worth mentioning that CTT’s reliability estimates varied from adequate to excellent, also supporting the use of the scale using this measurement approach.²³

The strong correlation with the original inventory and the high sensitivity and specificity for developmental delays highlight the good quality of the empirically-based item selection. Although not perfect, a comparison of IDADI-B scores with the original inventory point to a strong convergence using only a half to a third of IDADI’s items which ensures time efficiency. Such results are similar to the validation of Bayley's developmental screening scale, which obtained high correlations with the original task-based scale.²⁴

IDADI-B scores were also associated with several external criteria variables. Diagnosis was the variable most strongly associated with child development. Such results suggest the potential to identify children with developmental delays caused by neurodevelopmental disorders, one of the main objectives of the inventory. Although it is not characteristic for screeners to provide specific diagnoses, the results highlight the importance of monitoring the development of children, as the screening tools are highly sensitive to the identification of neurodevelopmental disorders such as Autism,²⁵ disruptive behavior,²⁶ and motor problems.²⁷ Maternal education was associated with five IDADI-B domains and family income with three domains replicating the frequently observed effects of socioeconomic indicators on child development outcomes.^16,28 Differences between children's sex were also observed in other instruments such as the BSID and the ASQ,^12,20 stressing the importance of considering the specificities between the sexes when dealing with development in early childhood.

Despite the potential benefit of using IDADI-B, some limitations should be addressed. The use of a new sample using the IDADI-B is needed to verify whether the results observed in the present study are replicable.²³ The test-retest analysis would constitute an important indicator of the degree of consistency of development estimates over a brief period of time. Validity evidence based on theoretically similar instruments such as the Denver Developmental Screening Test,²⁹ ASQ-BR,²⁰ or the Survey of Wellbeing of Young Children³⁰ would also support the utility of the inventory. It is worth mentioning that, currently, IDADI is the only instrument specifically developed in Brazil. The ASQ-BR was validated only in the context of public day-cares from the state of Rio de Janeiro,²⁰ while the BSID commercial version was only translated to Brazilian-Portuguese, and the norms of interpretation are based on an outdated American sample collected from years 2000 to 2002.¹² IDADI, on the other hand, benefitted from a thorough content scrutiny of experienced professionals, semantic analysis of Brazilian mothers, and empirical validation data from all five regions of the country,^14,15 thus making it an important tool for this population.

Taken together, these findings suggest that IDADI-B is a practical and psychometrically sound instrument for assessing the development of young children. The inventory is expected to improve the early detection rates of children with signs of developmental delays and facilitate referrals for interventions. This need is crucial since the earlier the referral, the greater the benefits, and chances are higher of mitigating the negative consequences of a disorder due to the greater brain plasticity in early childhood.²⁸ Moreover, culturally appropriate measures of child development provide the means of evaluating the effectiveness of new interventions and the quality of health and educational programs.^7,10 Screening instruments identify children who need more individualized attention, taking into account under- or over-classification errors.² It is known that transient qualitative aspects also characterize child development,¹ and, therefore, pathologies need to be examined considering the environmental-relational and neuroplastic aspects of this sensitive period of development.^4,7,28 Hence, IDADI-B does not aim to be a long-term development predictor, on the contrary, it aims to provide a valid and reliable assessment of the child at the moment of screening. Professionals who use it should not draw any linear deterministic conclusions of future development, but use it as one indicator of the need for future intervention, paying special attention to the individualized development process of each child. It is expected that the identification of adversities in child development through IDADI-B, freely available at https://vol.vetoreditora.com.br/idadi_breve, can benefit functional and structural aspects of child development opening the venue for intervention strategies implemented quickly to reduce risk factors and promote social protective factors for children and their families.

Conflicts of interest

de Mendonça Filho, Silva, and Bandeira developed, validated, and have all rights reserved for IDADI and IDADI-Short.

Acknowledgments

This research was supported by the University of Nebraska-Lincoln through its UNL/Brazil Collaboration Pilot Impact Grant; Fundação de Amparo a Pesquisa do Rio Grande do Sul (FAPERGS); and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES). Opinions expressed herein are those of the authors and do not reflect the position of the funders. IDADI-B is freely available at https://vol.vetoreditora.com.br/idadi_breve. We thank the children and their families who participated in the study and generously contributed their time and effort.