Listen on Apple Podcasts Listen on Spotify
This Week’s Episode: How to Use Dynamic Assessment When Evaluating Culturally & Linguistically Diverse Students
This month I had the pleasure of chatting with Kallie Knight (@kknighttherapy), a school-based SLP, regarding evaluations for culturally and linguistically diverse students.
So far this month we discuss why different types of evaluations are important and different things to consider when evaluating CLD students, tips on how to take good language samples for these students, and last week we talked about non-word repetition and how it is a non-biased form of evaluation for CLD students.
This week we get to talk about the juiciest part, dynamic assessments, and how important it is when you contrast it with norm-referenced tests.
Kallie’s reel shares a real-life example of what happens when you take the time to do a little more research in your evaluation process. Be curious and ask questions.
Key Takeaways + Topics Covered
Why Use Dynamic Assessment?
> DA contrasts with norm-referenced test (static measure, you get it right it’s right, you get it wrong it’s wrong, it doesn’t matter what their experience is)
> Construct validity (are we measuring what we want to measure)
How to use Dynamic Assessments
>DA is the complete opposite of norm-referenced/static tests–it’s DYNAMIC
1.test 2.teach 3.retest
>>mediated learning experience
>>check the student’s “modifiability” (how well do they respond to instruction)
>>was it difficult to teach? or was it easy to teach?
>> vocabulary is experience-based (e.g., ice cream sandwich)
Additional Links
Kallie Knight: @kknighttherapy
ASHA Leaders Project: Non-Word Repetition Module
SLP Now Membership
Bilinguistics
Gathercole et al 1994
In this Pod Series
4/5/22: Evaluations for Culturally & Linguistically Diverse Students: The Why
4/12/22: How to Use Language Samples When Evaluating Culturally & Linguistically Diverse Students
4/19/22: How to Use Non-Word Repetition Tasks When Evaluating Culturally & Linguistically Diverse Students
4/26/22: How to Use Dynamic Assessment When Evaluating Culturally & Linguistically Diverse Students
Subscribe & Review in iTunes
Are you subscribed to the podcast? If you’re not, subscribe today to get the latest episodes sent directly to you! Click here to make your listening experience auto-magic and as easy as possible.
Bonus points if you leave us a review over on iTunes → Those reviews help other SLPs find the podcast, and I love reading your feedback! Just click here to review, select “Ratings and Reviews,” “Write a Review,” and let me know what your favorite part of the podcast is.
Thanks so much!
Transcript
Speaker 1: Hello there and welcome to the SLP Now Podcast, where we share practical therapy tips and ideas for busy speech language pathologists. Grab your favorite beverage and sit back as we dive into this week's episode. Now we get to into maybe the juiciest most fun part dynamic assessment. For SLP who aren't familiar with dynamic assessment, do you want to give us a quick overview of what it is?
Kallie: Yes. And I think understanding not only what it is, but how important it is when you contrast it with norm-referenced tests, which is what we usually use. And if you listen to that first episode in this series, you'll hear a little bit about that. But I'm going to compare first, a couple things that I didn't say in that first intro series so that I can compare it to dynamic assessment and show you why it's so gold, just gold. So with norm-referenced tests, thereKallie's static measures.
:
I'm giving it to the kid and that's it. If you get it right, you get it right. If you get it wrong, you get it wrong. Doesn't matter if you don't even understand the language. Doesn't matter if for example, if you've never seen a wagon and it's a vocabulary test and they're like, "What's this?" And you're like, "A chair with wheels? No, wrong. It's a wagon." And it's like, well, I've never seen a wagon before, but it doesn't matter. Like it's static. It's just right, wrong, right, wrong. So that static test is not taking into account any of these biases.
Kallie: It's just, what have you experienced in your life up to this point? And that is what I'm testing. And when we look at construct validity, which is another psychometric principle that we're just saying, "Are you measuring what you want to measure? And are you doing it effectively?" Even logically, you're going to say, "No." And so these are some of the things that we have with those kinds of measures. But when we look at things like dynamic assessment, dynamic assessment is different because it is the complete opposite of static assessment. It is dynamic.
Kallie: There are quite a few different teaching protocols, but the one that most frequently used takes a test teach retest procedure. So you do test them just like you would in a norm-referenced test, but instead of stopping there, you do a little teaching. They call it an mediated learning experience, but you give them a little bit of instruction. You check there what they call modifiability in the research you see how well they respond to that instruction. And at the same time you're also paying attention as an examiner. Like, "Am I pulling my hair out trying to teach this to this kid, or is it fairly easy? Are they transferring the things that I'm teaching them."
Kallie: And then at the end you do a post test and then you see essentially, how did they perform after that mediated learning experience? And so obviously even if you're looking at the construct and idea of a dynamic assessment, it aligns better construct validity wise with what you're measuring. I want to measure a language disorder, which really is a language learning, or language acquisition disorder. And what is a dynamic assessment measuring language learning or the ability to learn, or respond to intervention in a short amount of time.
Speaker 1: Yeah. Perfect. And I think it's pretty clear why that's important, but how could that give us different results for culturally and linguistically diverse students?
Kallie: That in my head, I feel like that's like, duh, but at the same time, I was like, but it's not. So let's use the example of, so vocabulary is so experience based, regardless of whether where you come from, if you have not experienced or heard a vocabulary word, you're not going to know it and you shouldn't just magically know it because it developmentally pops into your head. So say you would take a norm-referenced test. What is this? It's a wagon. You didn't know what it was. Well, what if you did something like, oh, you ever seen a wagon, something like that. Talk about the features of a wagon.
Kallie: "This is called a wagon. You pull it," and then you might wait a little bit and then pull it back up and be like, "We just learned about this thing. What was this?" And they'll be like, "Oh," and they might be able to not know the exact word, but they'll be like, "It was the thing and you pull it and it's a toy, it's a wagon." So that would show, oh, with a tiny, tiny bit of instruction. It's like, "Oh, I got this." And I think that's the important thing about dynamic assessment too, is like with a norm-referenced test, the purpose of a norm-referenced test is just to say, is it typical, or is it a disorder? It does not tell you difference, or disorder.
Kallie: The difference there with a dynamic assessment is we're saying, if I give you that experience that you probably didn't come to the table with, can you suddenly do this task? And that helps you delineate, is it a difference, or a disorder? Because if you're giving that instruction and they're still having a really hard time with it's probably a disorder. If you're giving them that context, or that instruction that they've never received and they respond to it readily, it was likely an exposure issue.
Speaker 1: Perfect. This is a little off topic, but I'm an English language learner too. And when you were talking about the wagon example, it reminded me of, I was at a friend's house and the mom asked me if I wanted an ice cream sandwich and that was not something that was in my vocabulary. So I tried to infer and I was like, ice cream with bread does not sound good. But I saw her give the other kids an ice cream sandwich, and then I learned it and it was just like, I hadn't had that where I was from. So if they don't have exposure to it, how can we expect it to just magically pop up in their head?
Speaker 1: A lot of kids, if it is a difference than just having that little bit of exposure, and then I'm remembering some experiences, I've made mistakes, I've qualified kids that probably didn't need to be qualified. I did the evaluation, they got the scores they needed to, to qualify, but then we'd start treatment. And they'd just like zoom through everything. And maybe I'm the most epic therapist, but it probably more likely was that I didn't incorporate that to dynamic assessment.
Speaker 1: And yeah, maybe they'll zoom through therapy quickly because it is a difference, but in terms of our caseload, why you do that? And then also in terms of the student, that has an impact on them. It'll take at least some time for them to be exited and then return to the general education classroom. So I think this is a really, really awesome tool to include in our evaluations to make sure that, that doesn't happen.
Kallie: And to your point, just because I'm all about being real. I have also qualified people that same way because on the day, or days that they came in for their evaluation, the data that I got in that snapshot did align with what I would call a disorder. However, you only get so much time with them, oftentimes, especially in an initial evaluation. And so guess what? Everyone's going to make mistakes because you can only operate off of the information you have.
Kallie: So I always tell people like, don't beat yourself up. Even if you did everything right, there are going to be when we realize, wow, they do respond to instruction and they do it readily and guess what? They probably didn't have a disorder, but I didn't know that given the data I had and I did a good evaluation. So just to your point, I just want people to know even well seasoned clinicians probably make a mistake every now and then, because you just don't know how a kid is going to perform on a given day. So just to your point.
Speaker 1: Yeah. And I think we're constantly learning and growing so we can look back and maybe the student just happened to be sick those few days. And that's just when we pulled them and I know my language isn't as great when I'm sick. Yeah. So there's things that we can try and take away. Like maybe we don't really have control over, like the student's health, we definitely don't have control over and we are really same free time. So maybe that's not a situation that we could have changed.
Speaker 1: And I think like trying to pull from other resources as well, like looking at classroom data and the parent report, teacher report and all of that will help remediate that. But we can look at those examples and we don't have to beat ourselves up, but we can say, "Is there anything I could have done?" And sometimes there isn't anything and sometimes they're like, "Oh, maybe I could have pulled a little bit more, or tried a different schedule," but always learning opportunities. But we also have to just know that we're doing our best.
Kallie: Exactly.
Speaker 1: Give ourselves some grace. Okay. So because you mentioned using dynamic assessment in conjunction with a norm-referenced test, what does that look like for you?
Kallie: So I'm going to give a little bit of a backup real quick so I can compare two things. Dynamic assessment in and of itself is like a teaching procedure. It's a set of principles. It's not like I'm going to Google dynamic assessment and there's going to be one procedure and I'm going to print it out. And it's going to walk me through what I need to say and do, no. Dynamic assessment is just a procedure for evaluating. So that being understood, you can apply that anywhere.
Kallie: And I can tell you for example, how you can apply that to a norm-referenced test, but there are other dynamic assessments that clinicians have realized, you know what? While some people do like to just take the principals and go, lots of us, SLPs, just want to be told what to do. So they've made protocols that are more broken down and systematic and say, this is what you say, and this is what you do. So we've got those two classes where you can either take the principles and apply it anywhere for example, to a norm-referenced test, or any therapeutic opportunity, or you can access a dynamic assessment that has more structure.
Kallie: And we can talk about both, but I'll talk about the norm-referenced test example first, just because you asked about it. So for example, I really don't like the self at all and its psychometrics are very poor, but sometimes I just have to use it because that's what I have access to. And so I'm going to give another vocabulary example, maybe like the wagon, or you could do this for syntax too, but say I was doing like, I was just doing a bilingual evaluation the other day and we were doing the word class, I think, where you give them four words and you say, which two of these words go together?
Kallie: This kid, you got to a certain point and it was like, he didn't know any of the words. He heard one of them that sounded similar to something he knew. And so he'd do his best guess. And of course he got it wrong and it's probably words that other kids his age probably were familiar with. But I was like, "I don't know that this is a straight disorder. This could totally just be language experience." And so I would straight up ask him, I score the standardized test exactly per standardization. However, I do go back to test items and I'll be like, "Okay, we're going to do some instruction. Are there any of these four words that you don't know what they are?"
Kallie: And usually two or three of them he'd be like, "I don't know what those are. I have never heard those in my life." And then you just give a little bit of instruction on what each of those words mean, and then you can reask it. And then he was always like, "Oh, my goodness, oh, it's these two." But he would've never known because he'd never heard the words before. And another plug this isn't just for kids who are culturally and linguistically diverse. This is just a shortcoming of all norm-referenced tests. It's just somehow in your life, you have never accessed to no certain vocabulary been experienced to it. Every vocabulary static test is going to be biased in that way.
Kallie: But that's just an example of showing how a little bit of instruction resulted in immediate aha like, "Oh, I know what the answer is, it's this." Whereas likely I've done the same thing with a kid who has a more severe disorder and it's just like headlights, or deer in the headlights, or they try to guess to were, or sometimes they can't even remember the words to give back to you. So that's just an example of, you can go back to test items and you can do a little bit of instruction and then retest. So it's again, you're just stopping in a test item and doing test, which is giving it to them, teach a little bit of something and then retest them and see if it was easy for them to just get it, or if it's like, "I'm still not getting this at all."
Speaker 1: Yeah. Perfect. And then do you typically do the teach and retest in the same moment, or do you space them out?
Kallie: Truthfully in this probably isn't perfect for standardization purposes, but I usually will stop in the moment and do it just because it's natural. And since the standardized score isn't going to be valid for me anyway, if I'm breaking standardization ever so slightly to give these little teaching experiences, in my clinical opinion, the benefits outweigh any damage that, that could do. So I do stop and it's so quick. All you have to do is be like, "Are there any of these words you don't know?" And most of the time they're like, "Yeah. I have never heard of those." And then you instruct.
Kallie: And then what I do is I have a column and, if I've done a repetition and I'll put a little R, or I'll put dynamic assessment, DA and then I'll put plus or minus. With the short dynamic assessment, did they get it? So when I'm done with my norm-referenced test, when I'm writing it up, I report everything I say, "Interpret with caution." And then really I have a lot of information where it just says, "In nine out of 11 opportunities, dynamic assessment was employed and in nine out of 11 opportunities, it resulted in immediate accurate response." And so when you look at things like that, it's like, well, it's very likely that this was an exposure issue and not a disorder.
Kallie: And things like that, like I said, it only takes an extra 30 seconds to two minutes per item that you want to do it on and you get a lot of extra information. And the benefit on top of that especially as a school-based SLP is A, I can see if they have a disorder or not. And B, I see what accommodations and support they need to be successful. So I already know what recommendations I'm going to write.
Speaker 1: Yeah. That's perfect. I love the example with a vocabulary and with a norm-referenced test, do you ever do dynamic assessment with the language sample trying to teach a grammatical construct? I feel like that might be a little harder to generalize in the moment, but-
Kallie: I mean, it depends. I feel like things and like I said, it tends to be kids who speak Spanish that I'm evaluating. So I can always go back to a language sample later. Like I'll mark in the norm-referenced test that they missed this so that I can be sure watching it in a language sample to see. So I do have that added benefit, but something else that you could do even in a syntactic portion in the self, for example, irregular past tense verbs, I promise that the vast majority of kids are not going to tell you your irregular past tense verbs, or irregular plurals, or something like that.
Kallie: And so you can do a quick thing and be like, you know what? In English, we have really weird words when we call lots of kid, or what is it? Child and children that they have in the self. And they'll always say, "Childs." I'll be like, "We're weird in English. We don't just add an S, sometimes we change the whole word. It's children for a lot of kids." And they're like, "What?" Yeah. So there's one child, or lots of children and they'll be like, "Oh, okay, lots of children." And then you might talk about it and then come back a little bit later and be like, "Okay, you remember the weird word in English when we talk about a lot of different kids?" And they'll be like, "Children," or they'll just be like, "I don't know."
Kallie: So, I mean, you can do that with syntactic things too. Really the big thing with dynamic assessment is, it's working off principles of zone of proximal development. So I mean, whatever you want to do, take them with whatever task it is, take them to those zone of proximal development, see what they can do by themselves. But with you do a little bit of teaching and see if they acquire it. It really doesn't even matter what the task is.
Speaker 1: Yeah. Perfect. I love it. And we know how to teach all of these things like where you target them day in and day out. So we can just use our typical strategies, see if it helps the student and go from there.
Kallie: Exactly.
Speaker 1: I love it. I love the easy... Well, it may not be easy to implement at first, but it makes sense.
Kallie: It's one of those things where I think it's uncomfortable able to not have something structured when we're so used to having a lot of structure as SLP to take something that the principles are always consistent, but you have to use your clinical intuition and your clinical expertise to implement them. And I think that's a little bit scary to some people and can be hard to master, but once you do it just comes to you. Like you don't even think about it. And some people do it and don't even know. So you might as well just take the data on it. And that's really the information you need for your report anyway.
Speaker 1: Perfect. And I love the example that you gave of how you report that. I might just have to add that in the show notes for easy reference. I feel like that made it more structured in my head too. Like the strategy of writing down, whether you gave a repetition, or you did some dynamic assessment and how the student responded and then just reporting the score without the support, and then also adding in what worked. I think that's so helpful, if any receiving SLP would read that report, I feel like it'd be so helpful to figure out how to navigate. And even for the teachers, which strategies are helpful.
Kallie: I was going to say, I do that for all my students, whether they're culturally linguistically diverse or not, just because there are some kids where on the standardized test, there's no repetition that's allowed, but then you just think, what would happen if they just got one repetition? I was like, you don't need specialty design instruction from a speech therapist for a teacher to repeat things one time, if one repetition results in an accurate response the majority of the time. And so even for typically developing kids all do dynamic assess, well, typically developing and kids who maybe even have a disorder, but are mainstream.
Kallie: You still get the same information because I mean, now you know exactly what accommodations they need, whether they're going to be in sped or whether you're saying, "I'm not going to qualify, even though the standardized score shows lower because I know that they can still perform well given these supports that don't require specialty design instruction."
Speaker 1: Yeah. Perfect. Well, this was so incredibly helpful. Thank you, [Kallie 00:18:59] for breaking this down for us. And I hope everyone walked away with at least one new strategy, or perspective that they can use with their students. So yeah, that's a wrap. Thanks for listening to the SLP Now Podcast. If you enjoyed this episode, please share with your SLP friends and don't forget to subscribe to the podcast to get the latest episode sent directly to you. See you next time.
Sign up to receive email updates
Enter your name and email address below and I'll send you periodic updates about the podcast.
It was a very concise and good review for me. (I have been practicing for over 40 years and am a bilingual therapist). It was particularly helpful for a new practicing therapist in the field of evaluating children who’s primary language is not English. Still I did enjoy the exchange.