Literacy Measurement: A Holistic Framework

Literacy Outcomes

This section focuses on the skills and sub-skills that make up reading literacy, and is organized by constrained skills and unconstrained skills. Constrained skills are those that need to be fully mastered in order to support literacy acquistion. For these skills, there is a ceiling beyond which further learning is not necessary for literacy improvement.  Unconstrained skills, by contrast, are those that continue to develop over the course of a lifetime. For these skills, there is no ceiling as all further leanring contributes to improved literacy (1).
Make sure to have a look at our helpful framing question here.

Constrained Skills

Constrained skills are those that need to be fully mastered in order to support literacy acquistion. For these skills, there is a ceiling beyond which further learning is not necessary for literacy improvement.  

Concepts of print

This dimension refers to the awareness of how print works-- the knowledge of what print is, that it has meaning, and broadly, that there exists a relationship between print and speech.
Why is this important?
The knowledge that print conveys a message, and generally of how print conveys a message is an essential foundation for the development of reading and writing skills.

Phonological awareness

Phonological awareness is the ability to recognize and work with sounds in spoken language. Phonemic awareness is a sub-skill of this, and involves the manipulation of individual sounds within spoken words (phonemes).
Why is this important?
Being able to distinguish and manipulate the sounds within spoken words is linked to learning necessary print-sound relationships, which is critical to reading and spelling success.

Alphabetic principle

Alphabetic principle refers to the understanding that letters and letter patterns represent the sounds of spoken language.
Why is this important?
Children's knowledge of letter names, shapes and the sounds they represent is a predictor of their success in learning to read. Having this basis of knowledge helps emerging and early readers remember written words and begin to treat words as sequences of letters.
Considerations & Limitations
As the name of this dimension implies, alphabetic principle is particularly important for alphabetic languages (e.g. English), and is closely tied to phonics. However, the alphabetic principle does not underlie other types of writing systems, such as logographic or syllabic languages. Even though these systems generally do contain some phonemic elements, testing knowledge of alphabetic principle may not be appropriate for non-alphabetic languages.

Reading fluency

Reading fluency refers to the ability to read text quickly, accurately, and (when done out loud) with expression.
Why is this important?
The ability to read fluently is linked to the ability to comprehend what one reads.  Reading with sufficient speed, in particular, is necessary for freeing up working memory to comprehend the content being communicated.
Considerations & Limitations
Reading speed (usually measured in correct words per minute) is relatively straightforward to measure, and this (combined with its demonstrated relationship to reading comprehension), can become the focus of measurement efforts for reading fluency, particularly when the resources available for assessment are limited.  However, accuracy and expression are important as well, and should be emphasized in the training of assessment personnell and teachers.  

Additionally, words per minute benchmarks depend on the language of assessment.  Languages with longer words (more letters per word) require lower benchmarks for correct words per minute than those with shorter words.  Again, an understanding of the language of instruction is important with regards to this particular dimension.

Unconstrained Skills

Unconstrained skills, by contrast, are those that continue to develop over the course of a lifetime. For these skills, there is no ceiling as all further learning contributes to improved literacy (1).

Oral language skills

Oral language skills refer to the ability to understand and use a broad, rich vocabulary to generate and clearly express relevant ideas, think aloud and to demonstrate comprehension, as well as the ability to respond appropriately to others.  This includes expressive language skills (speaking), defined as the ability to construct sentences that apply the syntax of the language, and to produce coherent, extended text.  It also includes receptive language skills (listening/oral comprehension), which refers to the ability to understand what one hears, and, at more advanced states, to critically reflect on, interpret and evaluate this material.
Why is this important?
Oral language skills (in the relevant language) are a necessary prerequisite for the development of reading and writing skills.  Listening comprehension has a strong relationship with reading comprehension.  Listening exercises have been shown to be an effective technique for introducing increasingly challenging language, vocabulary, and topics to emerging and early readers. Despite their demonstrated importance, the development of oral language skills are overlooked in many classroom settings.
Considerations & Limitations
This becomes particularly complicated (but perhaps even more important) to assess in multilingual environments, and/or environments where the language of instruction differs from students' mother tongues.  Oral literacy (in the language of instruction) is the bedrock of reading and writing with meaning in the school language. This is an educational access/inclusion issue. Many children sit in classrooms where they cannot access the education the teacher is providing because their school language skills are too limited.  Students need to speak the language of instruction clearly and well (they need several thousand words) to develop phonological awareness that is attuned to all the variations in sounds in words in this language and to then use phonics to write meaningful words. Students need to develop their listening comprehension skills, in the school language, especially in understanding and negotiating the far more complex syntax and ideas that are used in written text, before they are ready to read with understanding in that language. Students must be able to generate and express meaningful ideas orally in the school language before they can produce coherent writing that conveys meaning.


Vocabulary

Vocabulary is the knowledge of the words necessary to communicate effectively.
Why is this important?
Research shows that vocabulary is correlated with reading comprehension, indicating that developing a comprehensive knowledge of words is essential for learning to read.

Reading habits & love of reading

This dimension refer to individual motivation to read, enjoyment of reading, positive attitudes towards reading, and practices around establishing a reading habit.
Why is this important?
Reading motivation is correlated with literacy outcomes.  This represents a mutually-reinforcing cycle, whereby students who enjoy reading continue to practice their reading skills, get better at it, and continue to read more.  Nonetheless, the instruction and measurement of literacy progress still tends to focus largely on the cognitive processes of reading.  Reading habits and motivation to read can be harder to measure, but are an important component of lifelong literacy.

Reading comprehension

Reading comprehension is the ability to understand what one reads, and, particularly in more advanced levels, to evaluate and reflect on what one has read.
Why is this important?
The ability to understand what one reads is necessary for later using reading as a mechanism for learning (to move from "learning to read" to "reading to learn").  It is also a necessary component of functional literacy for adults, as being able to comprehens what one reads is necessary for using written information and acting on it. In order to fully understand what one reads, one has to not only sound out the letters and words, but also "read between the lines."  While evaluation and reflection skills start developing early, they become increasingly important as time goes on, particularly for students in secondary school.

Functional literacy

Functional literacy refers to the development of reading and writing skills that are adequate to manage daily living and employment tasks.
Why is this important?
Functional literacy is important for adult and advanced readers in particular, as it emphasizes literacy as a tool to consume information and engage with society.  Functional literacy, in particular, emphasizes the importance of literacy with regards to individual empowerment and agency.

Writing, composition & spelling

The entire writing process, from letter formation, spelling, and sentence creation to generating and organizing ideas, etc.
Why is this important?
Research shows that reading, writing and spelling skills are closely linked. Spelling reflects the underlying word-sound relationships of language, and has been shown to aid in the development of reading comprehension skills. Likewise, many of the skills that are involved in writing, such as grammar and spelling, reinforce reading skills.
Considerations & Limitations
The time needed to master spelling will depend largely on the language type (alphabetic vs. logographic, for example), and in alphabetic languages, on the transparency of the orthography (that is, how closely sounds and letters map to each other).  Alphabetic languages with less-transparent orthographies may require more time to learn proper spelling, because letters correspond to multiple sounds.
References:
(1) Paris, Scott G. (2005). "Reinterpreting the development of reading skills." Reading Research Quarterly 40(2).

Framing Questions

Who is the population you are working with?

What ages are they?  What languages do they speak? What are the hurdles they face with regards to developing reading skills?  Do they have any special needs (related to learning disabilities, sensory disabilities or otherwise)?  As measurement is not a one-size-fits-all activity, understanding the population you are working with will help you adjust your measurement strategy accordingly.

What does success look like for your program?

Does it look like primary school students achieving national reading fluency benchmarks in your country? Does it look like adults improving their reading abilities to ultimately secure higher paying work?  Or maybe your program is focusing on instilling a love of reading in secondary school students? Keeping the ultimate aim of your program in mind will help you design a measurement strategy that is in line with your programmatic goals, not just your instructional or pedagogical approach.

What learning outcomes are you trying to cultivate, with what sort of programmatic approach?

Your measurement of literacy outcomes is linked to the programmatic approach you are using.  For example, some of the dimensions here are closely-related to a phonics-based approach.  If the program does not use a phonics-based approach to teaching reading, you might need to adapt how you measure.

What is the linguistic context of the program?

Do most learners speak the language of instruction, or are they learning in a second or third language? What type of language is being used in the program (alphabetic, syllabic, etc.)?  In general, much of the research around literacy outcomes comes from English-speaking context or alphabetic language contexts.  This is particularly important as you review the "limitations and considerations" column of this framework, as the outcomes listed here may have more or less importance depending on the language of instruction and assessment.a

Download our handy worksheet to guide you through these questions.
want more updates?

Subscribe to our Newsletter

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.