Validity and sustainability of a phonological assessment tool for Bulgarian preschool children: cross-sectional study

Abstract

Phonology is a widely discussed topic that is studied in detail by linguists, psycholinguists, philologists, pedagogues, and last, but not least, by speech and language pathologists. The assessment methods do inform us about the phonological processes and phonological delays in details by comparing the results with those of a normative sample and offer an opportunity to gradate the severity of the disorder, which is individual and can vary significantly in individual children. The aim of the study is to present evidence of the validity and reliability of a phonological assessment tool developed to assess the phonological inventory of Bulgarian preschool children. For this purpose, an author’s test was developed and conducted among 382 Bulgarian children.

Results The present study provided evidence of validity and reliability (internal consistency) for this phonological assessment tool, confirming the reliability of its three subtests.

Conclusions The present findings provide strong evidence of the validity and reliability of this phonological assessment tool.

Background

Phonetics and phonology — essence

Language

The topic of language and the language system has been comprehensively discussed and studied in the speech therapy literature, due to the fact that language disorders are the most common pathology in childhood. Lahey [19] defined language as a code by which ideas about the world are expressed through a conventional system of random communication signals. Silva et al. [33] define language as a system of symbols for communication that allows this to happen between individuals in an unrestricted and highly structured way. Through interaction between symbols with specific meanings, one’s emotions and thoughts can be expressed through gestures and written or verbal speech.

These definitions given above have a common idea and describe the same processes: (a) language is realized by using the same communication system by members of the community; (b) language is a system, and this system is based on rules which are the subject of linguistics; (c) language consists of signs, which participants in communication use to convey a particular meaning; and (d) language is realized by using one and the same communication system by members of the community.

Phonetics and phonology

Phonetics and phonology are two different concepts that define and distinguish two different sides: on one hand — speech and on the other — the linguistic behavior of an individual. Both phonetics and phonology are concerned with the description of speech sounds, sound sequences, and sound patterns that occur in speech production. Hegde [13] describes the main distinction between phonology and phonetics, noting that phonology deals with abstract rules and knowledge that govern the production of speech sounds, while phonetic rules are based on the physiology and acoustics of speech,therefore, they are empirically observable and measurable. Phonology can be divided into two levels — segmental and suprasegmental, and Ingram [15] mentions two main types of processes that affect the segmental level — assimilation and substitution. At the suprasegmental level, processes related to syllable structure affect consonant elision, cluster reduction, and unstressed syllable elision.

Phonological development

The use of various processes is typical for children in infancy, but these should decrease when growing up. Data from studies of phonological processes vary for a specific age norm.
Stein and Fabus [16] and ASHA [2] define phonological processes as follows:

• Syllable structure processes that occur when the syllable structure of the target word is changed by reducing, deleting, or expanding one or more sounds in the syllable.

• Substitution processes: One sound class affects another sound class in which phonemes are changed by changing the place or mode of articulation [3].

• Assimilation processes: A sound or sound families that change to resemble another sound in the word. He notes that these are the most common phonological processes, but individual differences can be considered.

According to Stoyanova [32], children follow universal patterns related to the simplification and changes of the sound composition of words in the process of speaking. She describes the order of appearance of the segments in the Bulgarian language, being guided by the universal trends described in scientific sources, as well as by the phonological processes in early language acquisition. The author describes the processes of substitution, elision, epenthesis, metathesis, reduplication, etc. and specifies that several phonological processes can be combined in the production of a given word.

Phonological disorder

ASHA [2] defines the term “speech sound disorders” as difficulties related to the perception, motor activity, or phonological representation of speech sounds or speech segments — including phonotactic rules. Disorders of speech sounds are divided into two categories — organic disorders and functional disorders. Organic speech sound disorders are due to a motor/neurological, structural, or sensory cause. Waring and Knight [38] suggest that phonological disorders can be divided into five groups according to their assessment:

• 1. Repetitive processes — Typical patterns of errors in younger children that persist to an age when they should have disappeared

• 2. Chronological mismatch — Uneven speech development in which earlier patterns occur alongside features of later language development

• 3. Unusual processes — Use of rare or atypical error patterns

• 4. Systematic sound preferences — Overuse of one sound for a large set of target consonants.

• 5. Variable process use — Different uses for the same target consonant

According to Ingram [14], phonological disorders are as follows: (1) phonological delay — children show phonological patterns of younger, normally developing children and have a vocabulary corresponding to their phonological level,
(2) developmentally different phonological skills — children have mastered a relatively large vocabulary but express words using patterns typical of those used in the earliest stages of speech development; (3) socially influenced phonological patterns — children use an unusual phonological pattern due to awareness of their speech difficulties and try to apply all the measures they are capable of to achieve improvement; and (4) delay in the development of the supralarynx — children with advanced voice development do not learn consonant sounds in their typical sequence according to Waring and Knight [38].

Diagnosis of phonological processes

There are currently three international systems that classify speech sound disorders in childhood and define diagnostic criteria. These are the following: Diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5) (American Psychiatric Association (APA) [1], International Classification of Diseases-10 (ICD-10), World Health Organization [41], and the International Classification of Functioning, Disability and Health (ICF) [39, 40], including the next version for children and youth International Classification of Functioning, Disability and Health Children and Youth version (ICF-CY) [41]. Boycheva and Simonska [5] study the tools that speech therapists in Bulgaria use in their practice, related to the diagnosis of articulation and phonological disorders: “Test for the study of phonological development of children” [17], “Protocol for the assessment of the articulation of children in preschool and primary school age” [35], and “Exploring Articulation and Phonology in Childhood” [12]. Deleva [4] approved a test for the study of articulation and phonology on a sample of 80 children.
Shtereva [29,30,31] explores phonological awareness and provide a diagnostics for bulgarian preschoolers. López-Escribano et al. [22] also investigated rapid serial naming as a predictor of reading processes. Padareva-Ilieva [26] describes the incorrect articulation of the late ontogenetic sound [l] and the consequences that lead to writing errors. The author presents a number of studies related to the perception of soft consonants and their acoustic base in relation to different dialects [24,25,26,27]. Tsenova [36] offers a diagnostic protocol with basic procedures for the study of communication disorders in children and adults.

Differential diagnosis

Dodd [8] proposed a differential diagnosis model for functional disorders of spoken sounds. The system she developed through much research in the field and by extracting normative data is as follows. The types of subgroups she describes are as follows: articulation disorder, phonological delay, consistent phonological disorder, inconsistent phonological disorder, and infantile apraxia.
Kirk and Vigeland [18] and Eisenberg and Hitchcock [10] describe a test for evaluating articulation and phonological processes in children, which provides a differential diagnosis of articulation and phonological disorders — Diagnostic Evaluation of Articulation and Phonology (DEAP).

The classification that brings together the diagnostic and therapeutic aims of paraclinical professions is the ICF — International Classification of Functioning, Disability and Health, of the World Health Organization and its subsequent version for children and youth ICF-CY [41]. Simonska [34] presents the children’s version of the ICF, which was developed for children and adolescents in order to be applied in the diagnosis and therapy of communication disorders. The classification is based on the interaction of the medical and social problem, which consider the disability as a problem of the person and as a socially created one. Cunningham et al. [7] specify that the ICF-CY builds on the original ICF and provides a biopsychosocial framework addressing health issues that are specific to infants, toddlers, children, and adolescents.

Methods

Ethics approval and consent to participate

This research is conducted after the approval of the Research Ethics Committee (REC) of South-West University “Neofit Rilski” with protocol no. 2103–2/31 March 2021 and permission no. 2103–2/31 March 2021.

Before conducting the study, a declaration of informed consent was signed for each child by a parent and/or guardian. The parents were provided with information about the purpose of the study, how it will be conducted, and the tasks that are included in the research form. Each parent received an information sheet to familiarize themselves with.

After conducting the experiment, every parent who had expressed their desire to receive individual feedback on their child’s performance was informed in a timely manner, which was assessed as a huge plus and an opportunity for feedback from a specialist speech therapist.

Each child admitted to the study was invited to a quiet and peaceful place where in the form of a game, it was explained to him exactly what was required of them. The instructions to the children were clearly given so that they were fully aware of what was required of them and completely calm and willing to respond to the tasks set.

Characteristics of participants

The total number of children included in the analysis of the results was 382 (three hundred and eighty-two) between the ages of 3 and 7 years: 206 of them are girls, and 176 are boys. All of them were divided into four age groups: 3–4 years — 59 children (31 of them are boys, and 28 are girls), 4–5 years old — 75 children (35 boys and 40 girls), 5–6 years — 140 children (57 boys and 83 girls), and 6–7 years old — 108 children (53 boys and 55 girls). This age group covers all children who attend kindergarten and belong to the preschool age group. All of them attend kindergarten, and their mother language is Bulgarian. Children with disabilities and deficits were not admitted to the experiment, so they have no intellectual, neurological, sensory, and emotional-behavioral disorders, and there is no information about depriving factors of the environment. The children included were not diagnosed with a language disorder and were not working with a speech and language therapist at the time of the survey. This study included all children who were administered the test over a period of time. The groups are homogeneous, comparing them by gender, which also allows analyses in this direction.

The survey was conducted on the territory of Blagoevgrad and the capital of Bulgaria — Sofia, at the university practice center for speech and language therapy at South-West University “Neofit Rilski,” and in two state kindergartens.

Aim, design, and setting of the study

The concept of the scientific research is aimed at preparing an algorithm for the assessment of phonological processes in children. Data on the results of diagnostics of phonological processes in this age group are missing for Bulgarian children.

For the purpose of the research, an author’s test was created to analyze the phonological development of preschool children. The main goal of the methodology is to check the phonological processes as well as to compare the child’s phonological performance in the different subtests. It consists of nouns, represented by picture stimuls, whose phonemes are chosen in such positions to provide information about the child’s speech patterns. Due to the different lengths and syllabic structure of the stimuls, this test can detect all typical and atypical processes in preschool children.

The methodology provides information about the child naming, repeatition abilities, and generating one’s own speech by describing 15 pictures.

Scientific methodology can also be used to detect changes in the phonological system during therapy, that is, to re-evaluate after the initiation of therapy. In addition, the test makes it possible to make an articulation assessment of the child during the assessment.

The applied scientific methodology also collects the following:

• Personal data about the child — age, gender, address, date of completion of the test, and start and end of the test and information about the kindergarten the child attends.

• Information on established diagnoses and diseases (if the child has any) to establish whether there is a primary disorder that would cause the phonological disorders

• Subjective evaluation by the speech and language therapist, about the expressive skills of the child

• Statements to be noted by the speech and language therapist and findings during the examination of the specific child

• The tasks from Subtest 1, Subtest 2, and Subtest 3

“Subtest 1 — nouns” covers 18 different consonant sounds /r, l, s, h, ts, sh, g, h, k, d, f, c, p, b, m, n, t, and d/ in different positions and syllabic variants.

“Subtest 1 contains 96 pictures representing high-frequency words. Each card contains only one noun, so that the child can concentrate on what we want him to name specifically. Picture stimuli contained words of varying length, syllabic structure, and spanned specific phoneme segments and consonant clusters. (The instruction given to the children for this task is ‘What is this?.’” If necessary, cues are used to retrieve the target word from the child’s expressive vocabulary.
The possibilities for describing the answers that the child gave are through the following options: pronounces correctly,pronounces incorrectly, does not know the word repeats the word correctly,and repeats the word incorrectly.

“Subtest 2 — sentence repetition”: In this task, children have to repeat sentences that vary in length and complexity. The sentences are 10 in number, the minimum number of syllables in a sentence is 7, and the maximum is 13. Sentence 1 contains 7 syllables, and sentence 10 contains 13 syllables, but the syllable structure gets more complicated. The purpose of the task is to measure the maximum repeated phrase by the child, but the essential idea in Subtest 2 is to analyze the phonological errors made by the child at the word level, as well as to measure the number of correctly repeated syllables. To identify incorrectly pronounced words, they are transcribed. The possibilities for describing the answers that the child gave are through the following options: repeats correctly or repeats incorrectly. The instruction given to the children for this task is as follows: “I will read you a sentence, listen carefully and repeat after me.”

“Subtest 3 — generating sentences” contains 15 pictures with actions for children to generate their own statements. The task elicits well-defined target words to provide information about phonological errors by time of generating one’s own speech, as well as orienting researchers to the peculiarities of the child’s spontaneous speech production. The target words analyzed are 31 in number: swim, sea, wash, hands, food, baby, milk, hug, bear, talk, phone, sick, bed, read, book, play, playground, clothes, shop, bathe, bath, kitten, box, water, flowers, celebrate, birthday, skate, sled, clean, and vacuum cleaner.

The instruction given to the children for this task is as follows: “What is happening in the picture? Use all the words you can think of and tell me.”

The form and pictorial material through which the children were examined are the same for all ages. Through a preliminary, pilot study, the images used for the paintings were selected. This was necessary to ensure that the proposed picture stimulus corresponded to a specific word, and that the children would not use other synonyms when naming it.

Statistical data processing

For the purposes of this empirical study, the following statistical procedures were used in SPSS:

• Descriptive statistics

• Crosstabs

• Correlation analysis

• Analysis of variance

• Comparison of one or two groups with a t-test

Results

Discriminant analysis

In order to determine the children’s phonological assessment (based on the number of phonological errors they have made), a discriminant analysis was conducted. The independent variables are age, substitution, elision, epenthesis/sound, prosthesis, depalatalization, reduplication, assimilation, dissimilation, metathesis/sound, vowel harmonization, iotation, vowel sound reduction, overall reconstruction of phonological appearance, non-segmented phrases, sonorization, desonorization (disambiguation), syllabic elision, syllabic metatheses, syllabic boundary consonant elision, and syllabic cluster consonant elision. Wilks’ Lambda was statistically significant: λ = 0.149, χ² = 699.833, p < 0.001, and η = 0.685, indicating that the model with these independent variables was capable of statistically significantly discriminating the five groups of children.

The discriminant model explained 81.9% of the variance in the dependent variable. The classification results showed that the model successfully predicted 79.7% of excellent performing children, 66.2% of good performing children, 45.7% of normal performing children, 71.1% of acceptable performing children, and 84.6% of children below the norm. The overall reliability of the classification is 70.2%.

That is, according to this model, whose results represent a high percentage of credibility, it would be possible to predict almost 80% of a sample of subjects with excellent performance and to detect and predict almost 85% of a sample of children who would perform below norm. The abovementioned values are very high, as shown by the average of all five scales that determine the phonological assessment.

Sustainability of results through repeated measurement “test–retest”

The most intuitive way to check the reliability of any measurement is to repeat it again and compare the two results. This way of checking the reliability is called the reliability of the results by repeated measurement of the scale. The present test assesses phonological features in the speech of children between 3 and 7 years of age. Twenty-five days after the administration of the test, a re-test measurement was carried out with the same test with 30 children who had been administered before. The retest was conducted in the same conditions and with the same speech and language therapist who performed the initial examination. Then the results were compared.

The correlation between the two interval variables “Number of phonological errors for Subtest 1 test and re-test” is statistically significant and high positive rtt = 0.988 (sig. = 0.000). Therefore, the test (scale) is considered reliable. The correlation between the two interval variables “Number of phonological errors for Subtest 2—test and re-test” is statistically significant and high positive rtt = 0.802 (sig. = 0.000). Therefore, the test (scale) is considered reliable.

The correlation between the two interval variables “Number of phonological errors for Subtest 3 test and re-test” is statistically significant and high positive rtt = 0.945 (sig. = 0.000). Therefore, the test (scale) is considered reliable.

The correlation between the two interval variables “Number of phonological errors for Whole test—test and re-test” is statistically significant and high positive rtt = 0.988 (sig. = 0.000). Therefore, the test (scale) is considered reliable.

Testing the measurement reliability of Subtest 1 with Cronbach’s alpha

In order to examine whether the 96 words in Subtest 1 form a reliable scale, Cronbach’s coefficient alpha was measured. The total number of valid observations is 382, and there are no observations with missing data. In this case are 77 out of a total of 96 words in the test, and 19 of the items (words) are excluded from the analysis because they had 0 variance (std. deviation = 0), and all children performed without a single phonological error. The words excluded from the analysis were as follows: duck, baby, table, nose, knife, road, fire, hat, mouse, yellow, frog, bear, tea, king, thread, fish, crab, basket, and kitten. Alpha α is equal to α = 0.805 because there are no large differences between the average and the variance and is reported as α = 0.81, which is an indicator of good reliability of the measurement of Subtest 1, and after the additional analyses, it was proved that the scale in Subtest 1 is correctly worded, and each item contributes to its reliability.

Testing the measurement reliability of Subtest 2 with Cronbach’s alpha

In order to examine whether the 10 sentences in Subtest 2 form a reliable scale, Cronbach’s coefficient alpha was measured.

Out of the total number of 382 observations, 377 are valid, with 5 observations missing data.

Alpha α is equal to α = 0.688 because there are no large differences between the average and the variance and is reported as α = 0.69, which is an indicator of acceptable to good measurement reliability of Subtest 2. The scale is considered to be correctly constructed, and that each item contributes to its reliability if there is no value greater than 0.699. We can conclude that the scale is formulated correctly, and each item contributes to its reliability. The entire scale in Subtest 2 is correctly formulated, and each item contributes to its reliability.

Testing the measurement reliability of Subtest 3 with Cronbach’s alpha

In order to examine whether the 15 sentences in Subtest 3 form a reliable scale, Cronbach’s coefficient alpha was measured.

The total number of observations was 382, of which 364 are valid and 18 observations are missing data. The number of items included in the analysis are 10 out of a total of 15 sentences in the test, as 5 of the items (sentences) are excluded from the analysis. Situations that are excluded from the analysis are as follows: “bath and bathtub,” “kitten and box,” “celebrate a birthday,” and “skating and sledding.” They have zero variance (std. deviation = 0), and all children performed without a single phonological error. The sentence “clean, vacuum cleaner” is also not included in the analysis, but the reason for its exclusion is different: 50% of the observations made a phonological error, and this significantly distinguishes it from the rest of the sentences (the correlation with the rest of the sentences tends to zero).

Alpha α is equal to α = 0.760 because there are no large differences between the arithmetic mean and the variance and is reported as α = 0.76, which is an indicator of good measurement reliability of Subtest 3. We can conclude that the scale in Subtest 3 is correctly formulated, and each item contributes to its reliability.

Discussion

Based on the results obtained from the research and their analysis, the following conclusions can be formed:

• 1. The chosen methodology for conducting the scientific research is applicable to children between the ages of 3 and 7. It enables to collect information about the phonological development and the processes the child uses in the process of language development.

• 2. Statistical analysis proved high reliability of the scientific methodology.

• 3. The scientific research methodology can be used for phonological assessment in addition to articulatory one.

• 4. The scientific methodology does not prove the existence of a statistically significant correlation between articulatory and phonological errors.

• 5. The objectivity of the study was confirmed after applying the retest.

• 6. The scientific methodology established the following:

  • 18.6% of 3–4 children have phonological disorder.

  • 13.3% of children 4–5 have phonological disorder.

  • 12.9% of children 5–6 have phonological disorder.

  • 12% of children 6–7 have phonological disorder.

  • 13.6% is the average percentage of children who, according to scientific methodology, have a phonological disorder.

After conducting the experiment and analyzing the results, it was shown that for the study of phonology in children, the results obtained from the picture noun-naming task were informative enough. The sentence repetition task proved to be easy for children of all age groups, and the information obtained from this task did not provide definitive data on their phonological development. The sentence generation task, in addition to providing data on phonological processes, was sufficiently informative about the child’s syntactic and pragmatic skills, the length of his utterances, and the prosodics of his speech.

Yancheva and Mitova [42] studied the prevalence of specific language disorder in Bulgarian children and found that 5.89% of the sample had such a disorder, with 4.64% of them having a phonological disorder. Eadie et al. [9] summarize that speech sound disorders occur in a population of preschool children, and their prevalence is between 3 and 6%, with the accuracy of children’s production increasing with age and the number of error patterns decreasing. Eadie et al. [9] claimed that in an Austrian cohort of 4-year-old children, the frequency of speech disorders was 3.4%. According to Ferreira et al. [11], phonological disorder occurs between children aged 3:0 and 8:11 years and is associated with the linguistic organization of speech sounds. Ceron et al. [6] conducted a phonological assessment of children between 3:0 and 8:11 years and found that 15.26% of the sample demonstrated a phonological disorder.

Tresoldi et al. [37] provide information on the prevalence of children with speech sound disorders between 2 and 25% of the population aged 5 to 7 years. Lee [21] reported that the incidence of speech sound disorders in young children was between 8 and 9% and noted that approximately 5% of these persisted to first grade. Patah and Takiuchi [28] found a prevalence of phonological disorders of 8.27% in 7-year-old children. Law et al. [20] reported that in the UK, the prevalence of voice disorders varied from 4.6 to 24.6%. According to Macrae [23], speech sound disorders in 3–4-year-old American children range from 15 to 16%. Law et al. [20] identified phonological disorder as one of the most common communication disorders in young children, with prevalence estimates ranging from 7 to 11% for 5 years old.

Conclusions

The author’s abstract can draw the following conclusions based on the results obtained from the scientific research and their analysis:

• 1. The chosen methodology for conducting the scientific research is applicable to children aged between 3 and 7 years. It enables the collection of information about phonological development and the processes that the child uses in the process of language development.

• 2. The statistical analysis proved high reliability of the scientific methodology.

Recommendations could be noted regardless of the fact that the statistical data indicate a high reliability of the results of the chosen methodology, and the need for more investigated people is in the foreground. This will allow the standardization of this tool, which is needed and could be useful in speech therapy practice for prevention, diagnosis, therapy, and retesting in the therapy process.

A direction for future studies could include data from ongoing therapy sessions, which would allow the scientific experiment to be used to measure progress over a longer period of time.