Recently, in a small bistro in the middle of Sao Paulo, Brazil, I encountered one of the best communicators I have ever met. A waitress in the restaurant, Paula, was able to manage the orders for a large group of people from a range of different countries, none of whom spoke any Portuguese. She clarified items on the menu which was in Brazilian Portuguese, knew who wanted steak and who wanted fish, who wanted ice in their drinks, dressing on the side, or a separate bill at the end of the meal.
The interesting thing is that Paula did this without a word of English – and without the use of Google translate or any other device. She had a great talent in connecting with people, deciphering what it was they were wanting from the menu, and clarifying when she wasn’t sure. She did this using a range of different strategies – hand gestures, pointing to other diners’ food, selecting words that she thought we might understand (like common brand names), and skilfully manipulating intonation to carry across meaning - and she had an overall willingness and determination to achieve mutual understanding, get the food on our table, and get paid at the end of the night. In the process, she even taught us some useful words in Portuguese.
Of course, our conversation did not venture into more abstract areas – we were in a context rich environment where talk was limited to only a few possible trajectories, many of which were predictable to all parties in the conversation. It is unlikely that discussion would have progressed much further outside of this context.
This experience made me think that, as teachers and testers, we spend a great deal of time and effort focusing on the linguistic aspects of language – vocabulary, grammar, pronunciation – yet successful communication comprises much more than this. The making of meaning using language is more than just sounds and words and sentences, it is how these are connected to the context in which the speech act is happening, how the speakers and listeners engage in an unspoken agreement to work towards a shared understanding, and the desire to achieve a communicative goal. Recognising this means that, as educators and assessors, we need to reconceptualise the construct we are measuring as a much broader ability than just linguistic building blocks.
This presents challenges. While teachers would do well to raise awareness of the broader strategies involved in achieving comprehensibility, there is little impetus to do this if tests are focused only on lexical range, grammatical complexity, or ‘native-like’ pronunciation. We will continue to meet test takers who can deliver rehearsed monologues about complex ideas but cannot engage in basic conversation with their fellow university classmates. In effect, we are producing one-dimensional communicators if we continue with a narrow focus on language elements only.
So, if tests are the drivers of teaching and learning, and those, in turn, shape the kind of communicators we are producing, how do we add depth to the tests so that our learners are more multifaceted (and successful) communicators? Linguistic elements are, by and large, easy to count and evaluate (especially by machines). Effective communication is much more nuanced and complex – and less objective, a thorn in the side of standardised testing. There are no easy answers, but the richness of human communication demands that we find ways to test all the dimensions that contribute to making oneself comprehensible, rather than opting for a flat comic-character version of communication because it is easier or more amenable to quantitative evaluation by machines or humans. Holistic testing should take more than just the linguistic building blocks of communication into account.
Interested in participating in the discussion about holistic language testing? Join us at New Directions, 27-29 October 2023.