Determining the Response in RTI

29 April 2008, 11:00 AM EDT

Read more about John Hintze
Read more about Matt Burns
Read more about Pamela Radford

Transcript

ken howell:
While information about typical/average growth rates accumulate with the increased use of progress-monitoring, these don’t give much information about what should be expected from interventions designed to correct important performance-progress deficits. And what information does exist seems less than “ambitious” to teachers. How can we determine behavioral criteria for acceptable progress in content areas (independent of tier level, instructional approach and/or the typical growth of other students)?
John Hintze:
This is an excellent question and one that I hope the research and practice community can come together to try to address. We know very little about the "dose/response" index for various educational interventions and until we have better estimates our recommendations for intensity and length of intervention are best guesses.
John Humphries:
My question pertains to disability determination using RtI data. Please comment on the idea of using a trend line slope minus 1 SEM as a crtieria for inadequate response.
Matt Burns:
Evaluating slope can only be one aspect of the eligibility model. Moreover, the 1 SEM is the variability, but what is the standard? Certainly computing 1 SEM is a psychometrically defensible approach, but the criterion from which it is subtracted matters more. The mean slope could be used, but I think a better idea is to use the St. Croix River Education District model and compute the score at each benchmark that predicts end-of-the-year success (i.e., scoring proficiently on state or accoutability tests), then determine the slope of learning that best represents those three benchmark scores. That resulting slope would be the criterion and 1 SEM could be subtracted from it to determine the "cut score" for slope. Again, this should be considered along with the level of performance. The research on this approach is scant, but has suggested a lack of ethnic and gender bias, and earlier access to intense interventions (as compared to more traditional models).

I also suggest focusing on adequate response as a means for SLD identification as well.
John Hintze:
Something like that might be viable, however, you need to make sure that (a) the difficulty of the progress monitoring material is suitable for the student (otherwise you might not see much growth due to the difficulty of the progress monitoring material as opposed to lack of student growth), and (b) the length of time (or # of data points) is sufficient to get a reliable estimate of slope and SEM.
James Heider, Ed.S.:
How does the current RtI approach differ from the Precision Teaching approach of Ogden Lindsley in the 1970s at Kansas Univ.?
John Hintze:
For us things that are similar to precision teaching are commonly used for students in Tier 3. Tier 2 students usually receive more generalized interventions. The frequency of progress monitoring may be less than compared to precision teaching. Also, the type of assessments used to assess response to intervention are developed using a general outcome measurement model where PT usually uses a specific subskill mastery measurement approach. Finally, RTI uses equal interval graphing where PT uses semi-log approches for displaying data.
Matt Burns:
I love this question because you are right on! Precision teachers have been doing much of this for 30+ years. The difference is the systematic resource allocation. Precision teaching uses data to design and monitor instruction and interventions with standard celeration charts. Thus, an aspect of PT exists across the three tiers. However, RTI uses schoolwide data to allocate resources where they will do the most good. Thus, RTI is more systemic than PT.
Beverly:
Can progress monitoring tools be used for universal screening?
John Hintze:
My feeling is that universal screening - or benchmarking - is a form of progress monitoring. Basically, you're monitoring the progress of all students 3 times a year. Yes, we use the same type of assessment materials for universal screening as we do for more frequent (e.g., weekly) progress monitoring.
Ed Huth:
How do you set weekly rate improvement goals for students who are well below grade level and will be progress monitored using below grade level assessments?
John Hintze:
For these students we generally recommend using what is referred to "intra-individual" goal setting. To do this you first calculate the student's current rate of progress using at least 8 data points (i.e., slope). Once you have the slope for those data, mulitply the slope by 1.5 (this is a constant). Mulitply this product by the number of weeks of intervention or the number of weeks in the school year (whichever you are setting your goal for) and then add this product to the baseline average of the first 3 points of the data series. This will give you a goal for expected growth at some point in the future based on the student's current growth to date. It would also be important to make sure that the difficulty level of the progress monitoring material is such that it would allow for growth to be observed. For many of these students monitoring them in material that is equivalent to their grade may be too challanging and may not allow for growth to be evidenced.
Matt Burns:
My suggestion is to use grade-level probes for placement decisions. It is good practice to determine if the intervention is working with instructional level probes. However, if these data are to be used for placement decisions (e.g., tier I to tier II, tier III to special education), then some standardization and ability to compare to a norm group are needed.
Sungja Collins:
Is it possible to have interventions when a student is on Tier I of the process? We have been told yes and we have been told no.
John Hintze:
For us the Tier 1 intervention is the core curriculum. Now, having said that the core curriculum might have features of what you might refer to as interventions. For example, in one school that we have worked with all students get both the core curriculum and Read Naturally in Tier 1. The reason for this is that that principal felt that in order to get 80% of her students meeting benchmarks standards that this approach would be necessary. The core curriculum in this sense is whatever it takes you to get 80% of students meeting benchmark standards. In this case, however, Read Naturally is considered part of their core and NOT a separate intervention in Tier 1.
Pamela Radford:
The three-tiered model is not about whether students receive interventions or not. Instead, it is a framework to define the level of educational need so the appropriate level and intensity of resources can be allocated for the purpose of improving student performance. If a student’s educational need is defined as Tier 1, the student’s educational is not significant enough to warrant the allocation of additional resources, such as Tier 2 interventions (e.g., small group instruction 30 minutes daily). Instead the educational need may be met through differentiation within the core curriculum. If the student’s educational need is defined as Tier 2 (e.g., significantly discrepant from peers or expected level of performance), then additional resources should be allocated (such as supplemental interventions) to address the educational need. So, to answer your question, yes, a student in Tier 1 can receive an intervention, however, that intervention may be defined as differentiation of the core curriculum.
Matt Burns:
Think of RtI as a resource allocation model. Good teaching involves differentiating instruction based on student needs. That is not really "an intervention," but just good teaching. If the intervention that the student needs can be best delivered by a classroom teacher as part of the regular school day, then certainly an intervention can occur within Tier I. If the child's needs dictate that an intervention that requires more resources than can reasonably be dedicated within Tier I, then the intervention should occur in Tiers II or III.
Tom Trocco:
How do you determine of the Core is meeting the needs of 80% of your students, and do you begin Tiering progress monitoring if it doesn't?
Matt Burns:
I suggest comparing class medians to the 25th percentile for the national norming (Hasbrouck and Tindal, 2005 - available for free at www.readnaturally.com). If the median for a class is less than the 25th percentile, you have what Amanda VanDerHeyden calls a classwide problem, which suggest the curriculum is not meeting the needs of the student.
John Hintze:
We get an estimate of this by seeing what percentage of our students are meeting benchmark targets. If 80% or greater of the students are consistently meeting benchmark targets across the year then we feel comfortable that the core curriculum is meeting student needs. If less than 80% of the students are meeting benchmark targets then it is time to perhaps thinking about changing and/or altering the core program to get students to these benchmark levels. You might scroll down to see a related answer below.
Jessica Watt:
Can you speak to any research that has been done looking at the implementation of RTI at the secondary level? Also, has any information been compiled regarding utilizing RTI in response to behavioral difficulties?
John Hintze:
Perhaps someone else can address secondary level applications. If not, I know that Mark Shinn who is at National Louis University has been doing some implementation work at the secondary level. As for behavioral applications - Positive Behavior Support (PBS) is very much in keeping with a multiple-gating RTI process. There are also some commercially available systems that use a process similar to RTI for making data-based decisions.
Pamela Radford:
Research is limited at the secondary level, however, there are many resources available regarding implementation at the secondary level. Here are a few: Dr. Mark Shinn's (National Louis University) website(under downloads, select the folder titled presentation and handouts and visit the secondary problem solving folder). The RtI Action Network website has posted information, resources and research related to high school implementation. For information regarding behavior, visit the Positive Behavioral Interventions and Supports website.
Matt Burns:
There is not much. There is research examining measures to use for progress monitoring and universal screening (John can speak to that better than I can), and there is some regarding effective interentions (see the recent special issue of Learning Disabilities Research and Practice edited by Carolyn Denton and Sharon Vaughn), but there is little examining the system as a whole. Below are a few references that examine RTI for secondary schools.

Gable, R. A., Manning, M. L., Hendrickson, J. M., & Rogan, J. P. (1997). A secondary student instructional support team. High School journal, 81 (1), 22-27.

Windram, H., Scierka, B., & Silberglitt, B. (2007). Response to intervention at the secondary
level: A description of two districts’ models of implementation. Communique, 35 (5), 43-45.

Burns, M. K. (2008). Response to intervention at the secondary level. Principal Leadership: High School Edition, 8 (7), 12-15.


Sandy Beherns, M.Ed:
How many research-based interventions do you recommend using for problem solving? 2-3?
Matt Burns:
I don't think there is an answer for this. It is probably more a policy question. More important is the number of data points. My suggestion is to collect at least 6 to 8 data points for making a change, to not take a kid from Tier II to Tier III (or Tier III to special education) unless at least two interventions are tried and at least 10 data points are collected.
Sandy Beherns, M.Ed:
Is it necessary to use a universal screening tool with ALL students (each grade level)?
If Tier 1 includes everyone (i.e., core curriculum), should a student at risk be getting 90 minutes of reading istruction at his Level or togeher with his peers (whole group insruction)?
John Hintze:
I believe that it is best to monitor all students periodically (at least 3 times a year). In doing so you are essentially progress monitoring everyone periodically and making sure that they are maintaining adequate progress throughout the year. For us, at-risk students (or those in Tier 2) receive the core curriculum plus supplemental instruction/intervention at Tier 2.
Pamela Radford:
It is recommended to assess ALL students at each grade level for the purpose of identifying at-risk students. However, I want to distinguish between universal screening and benchmarking. The purpose of universal screening is to identify students who are at-risk. So screening may occur 1 time per year (but can occur more frequently). For example, a school may screen middle school students at the beginning of the year to identify students who may at-risk for academic failure. Whereas, if a school decides to benchmark students, that would entail collecting data 3-4 times per year to evaluate the effectiveness of the core curriculum, to inform instruction (e.g., plan for differentiation of the core curriculum) AND to identify students who are at-risk. The primary purpose of benchmarking is to monitor and ensure student progress to improve educational outcomes. The difference is in the frequency in which data is collected and the types of decisions that can be made with that data.

For the second part of your question, it will depend on how discrepanct the student is from his/her peers. Students whose educational need is defined as Tier 2 should continue to receive core instruction at his/her grade level. For students who are signficantly discrepanct from peers (e.g., 3-4 grade levels below), core instruction may occur closer to his/her instructional level.
Matt Burns:
Question 1 - I think it is a good idea to screen all students if for no other reason than normative purposes.

Question 2 - Yes, students in Tiers II and III should get 90 minutes of core reading instruction in addition to the 30 minutes of tiered intervention. The interventions are supplemental, not instead of instruction.
Kim:
In my area, many schools claim they are using an RTI system...yet, in reality they are screening kids and putting data on shelves. Is there a way to accredit schools for doing this right as well as a special sort of credential for coordinators who actually know what they are doing?
Matt Burns:
A failure to consume the data that are collected is a very common difficulty. I don't know of any accrediting system and I hate to even suggest one. However, there are implementation checklists for RTI that explicitly target things like data consumption, data management teams etc. that can be used. I apologize for plugging my book, but Kim Gibbons has one that we published in our book (published by Routledge) and I'm sure there are others.
John Hintze:
To the best of my knowledge there is no accrediting body at the moment. However, I would bet that if you mentioned this to NCATE that they would do it (ha, only kidding!).
Laura Driscoll:
How do you see testing (psychological evaluations) to determine eligibility for special education services fitting into the RTI framework?
John Hintze:
For SLD, I see more typical psychometric psychological evaluations used to rule out other disability categories that might better explain a child's learning problems. For example, cognitive and adaptive behavior assessments could rule-out MR as a primary contributor to a child's learning problems.
Matt Burns:
That depends on what you mean by a psychological evaluation. In my opinion, the evaluation team examines RTI data and decides if any and what additional data are needed. Thus, a psychological (and comprehensive) evaluation could very well be just the student response data plus parent and teacher interviews with observations (for example). The referring questions drives the assessment used. If the team decides for whatever reason that an IQ test is needed, then those data are collected. However, in practice I think rarely is an IQ test or a measure of cognitive processing needed.
Lori Hensold:
Do you see any essential differences in the implementation of RtI in rural versus urban areas?
Matt Burns:
I'm not really sure and am submitting several grants right now to look at this very issue. I think the bigger question is impoverished or not. To be honest, I think it is easier to do RtI in a Title 1 school than other schools because they have more resources.
John Hintze:
Procedurally I would not see differences in the way that the model is used in rural vs. urban areas.
Ingrid Oxaal:
How would you evaluate the response of English language learners in an RTI framework?
Matt Burns:
Several studies have recently come out (or will be soon) that show similar psychometric properties for CBM measures among ELL as English speakers. Thus, I suggest using the same process. Moreover, English speaking status has not differentiated student response (according to research by Mike Vanderwood at University of California Riverside). In other words, students with little English have responded well to interventions.
Victoria S. Pando:
If a student is significantly below grade level i.e (5th grade reading 1st grade) but is responding to intervention--can you still make special ed. referral?
Matt Burns:
A special referral is always an option. However, if the student is making sufficient progress within grade-level assessment probes, then he or she would likely not meet an RtI eligibilty criteria. That is unless the intervention is so intensive that it cannot be implemented for a long-term without special education resources.
John Hintze:
I think that I've addressed this in another answer. If you believe that the student requires long-term intervention to maintain that level of progress (and that without such long-term support their progress would be slowed) you might consider such. I don't know that I would call it a referral as much as I would say that the RTI data are signaling the possible need for special education (i.e., I do not see the referral process as something different).
Ruth Poage-Gaines:
If a student is in an intensive support, i.e., Tier 2 or 3, situation and clearly needs that much support to be successful, should special education be considered? After how long, e.g., 2, 3 months?
Pamela Radford:
An eligibility decision within an RtI Model includes three essential components:
1.Level of discrepancy (what is the student’s educational need?)
2.Educational progress (what is the student’s rate of improvement with the current level of intervention)?
3.Instructional needs (what skills need to be taught, how are they taught, what resources are needed to deliver the instructional needs, etc.)
The purpose of assessment is to assist in intervention development. If an effective intervention is found to be successful in reducing the gap with peers and the level of resources needed to maintain the instructional needs is beyond general education resources, then special education may be considered. The decision to determine whether or not to consider special education should not be time focused but rather focused on the essential components. For more information regarding eligibility download the Illinois FSDS Entitlement Committee Recommendations document from the Illinois ASPIRE north website.
John Hintze:
If a student is in Tier 3 (or Tier 4 for those using a 4 tiered system) and it appears that over a course of 3 months that they require this level of support to maintain their progress then you might start to think that the student's learning needs may be of result of a disability and that their needs require special education services.
Matt Burns:
There are two ways a student can be found eligible for special education. First, research based interventions were attempted and the sufficient number of data points suggest that the intervention was not successful. Thus, a more intense intervention is needed, which may be special education. Second, the intervention was successful, but the amount of resources needed to do it are so intense that the intervention cannot occur without the resources of special education.

Remember, RTI is not the process of putting kids in tiers etc. before considering special education, it is the process of examining student response data that already exist. In other words, special education assessments do not involve trying Tier II for 8 weeks and then Tier III for 8 weeks. The eligibility decision-making process is examining the student data that were collected as the tiered interventions were implemented as part of the schoolwide data-based decision-making, which would require 30 school days or less!
James Cressey:
I have a screening question... I'm curious about how to approach the "average range" on a CBM measure like NWF for example. If a student is around the 50th percentile, should we consider them to be at risk, or no?
Matt Burns:
50th percentile on a national norm? If it is a local norm, I first suggest that you compare the classwide data to a national norm (e.g., compare to the 25th percentile on the Hasbrouck and Tindal 2005 data - see www.readnaturally.com for the data). If the median is below a national criterion, then individual student data are difficult to interpret.
John Hintze:
We generally consider student below the 25th percentile as possibly at-risk. We progress monitor those students for a brief time to confirm/disconfirm risk status.
Victoria S. Pando:
How many data points should be collected before determining if a student has mastered a specified skill or if intervention must be adjusted?
Matt Burns:
That depends on the conditions. Ted Christ here at the University of Minnesota has found that if optimal assessment conditions occur, then estimates of slope become acceptably reliable at about 6 to 8 data points. The more realistic conditions require more like 10 to 12 data points.
Pamela Radford:
The recommendation for how many data points are needed to make a decision range anywhere from 3-10 data points. In general, the more data and the greater the stability of those data, the more confidence to make educational decision(s). Greater variability in data indicates the need for additional data to make educational decision(s). When making data based decisions, there shoudld be a convergence of data in evaluating intervention effectiveness from a variety of sources (e.g., progress monitoring graph, classroom performance, work samples, anecdotal observations, etc.). The several sources of data should lead to the same conclusion (e.g., the studen has mastered the skill).
John Hintze:
Well, as you would probably expect the more data points that you collect the more reliable the estimate of trend and decisions made. As a rule, we generally recommend collecting at least 8 data points prior to making decisions.
Victoria S. Pando:
How do you determine when to progress monitor out of grade level and which tools to use for progress monitoring?
Matt Burns:
I suggest monitoring at grade level for any decision. However, if a child is reading two grade levels below his or her current grade, using instructional level (or one grade above the instructional level) may provide INSTRUCTIONALLY useful data.
John Hintze:
We use a form of survey-level assessment to determine the level at which students should be progress monitored. Basically, we sample back in the area of concern until we find a level that will optimize growth when it occurs. You can use commonly accepted performance standards to do this (see Ed Shapiro's CBA book for these) or benchmark standards to determine the best level of progress monitoring. We would generally only do this for students in Tier 3 (or 4) and not for students in Tier 2 since the assumption is their goal is to get back into Tier 1.
ken howell:
In response to Hintze's "I would see increasing time and/or pre-correction as interventions. For me, anything that is done to alter the course and/or direction of learning would be considered an intervention" I agree (as, I'm sure, does your physics teacher). So, doesn't this call into question the RTI standard protocol approaches which asign specific instructional techniques to different intervention levels?
John Hintze:
As you might suspect, we do not use a standard protocol. I know that this has been offered by many as one of two ways to go in Tier 2, but we have not seen it that way. For Tier 2 we generally use "manualized" or "commercially" available intervention that are based in science (notice I didn't say scientifically-based) that teachers can implement with moderate levels of coaching. Tier 3 interventions are usually more intensive and sometimes are more individualized (here we would be using the problem-solving model as mentioned by NASDE).
Matt Burns:
That is why standard protocol approaches can only be used as a part of RtI. I suggest Tier I being good quality core instruction and curriculum; Tier II being effecient small-group standard protocol interventions directed at targeted deficits (e.g., phonics deficits get Road to the Code and fluency deficits get Read Naturally); Tier III is what most people would call problem solving - intense problem anaysis to design interventions for individual kids.
Deb Kroehler:
How do you handle adverse environmental impact on a fourth grade student with chronic school changes and unstable home situations that have occurred for many years?
The need for intensive academic support to make up for lost academic time may require the highest tier. Would the environmental rule/out preclude special education?
Matt Burns:
The environment currently rules out/preclude special education. I think that the classwide issues need to be addressed first before moving to Tiers II and III. The Peer Assisted Learning Strategies (developed by folks at U of Minnesota, Vanderbilt U, and U of Kansas) have been shown to do that even in impoverished urban settings. Ideally, special eduction would be an intensive intervention that would likely be appropriate for a kid like that, but I doubt most schools are ready for that.
Mindy Moody:
Our district does not allow us to modify the grades. It is unfortunate that children are being failed because they do not meet district standards when they are making progress. What should we do about grade retentions?
Matt Burns:
Grade retetion is such an ineffecive practice that it should be almost nonexistent. Refer them to the research and many meta-anlyses pointing this out. It probably won't help, but at least you've done your job.
Lisa HS:
Do you think there is a danger that we settle for too LITTLE response/progress in some of our interventions under RTI as it is currently being implemented? (I think this may go back to Ken Howell's original question about ambitiousness)
John Hintze:
Progress and/or response for us is always determined by benchmark standards for students in Tiers 1 and 2. For students in Tier 3, the goals may be individualized.
Victoria S. Pando:
How can you determine effectiveness of an intervention so that your school can decide if it should be continued or not?
Matt Burns:
Compare the student's data to a post-intervention level criterion and a slope criterion. If the student is making sufficient progress with the slope of growth, but remains below the desired reading level, then the intervention should be continued.
John Hintze:
We use progress monitoring to determine intervention effectiveness.
Lisa HS:
Isn't there a role for problem solving and still calling it Tier 2? I have respect for standard protocol as an initial starting place, but if a grade level team comes up with differentiated groups, or even a "tweak" in instruction for just one student (e.g., splitting a 30 min intervention into 15 min in th AM and 15 in th PM instead of one chunk or adding in another instructional component) I would not say that those interventions are Tier 3... it depends on the intensity and in these cases it may no longer be standard protocol but it also has not nec. increased significantly in intensity... just a problem solving shift in focus or arrangment...
Matt Burns:
The entire process is problem solving. What differentiates tier II and III is the level of problem analysis needed to identify the intervention and the intensity of the resulting intervention. If the team can determine an intervention with relatively low level analysis (e.g., fluency versus a phonic deficit), and it can be delivered in a small group format (or a "tweaked" small group format), then it is tier II.
John Hintze:
Yes, problem-solving as you refer to it could fit into into Tier 2. This might happen for example as part of an instructional support team and would be keeping with the NASDE concept of problem-solving approaches.
Linda:
Whst does a balanced assessment system look like in a district that has a well established RTI model in place?
Pamela Radford:
A balance assessment system is one that has assessments defined for various purposes. That is, screening (to identify at-risk students), diagnostic (to identify why the student may be struggling), progress monitoring (to evaluate if the intervention is working) and program evaluation (how is the system doing overall to improve achievement). Within an RtI model, ideally the screening and progress monitoring tool should be the same in order to determine level of discrepancy and rate of improvement.
John Hintze:
I'm not sure what you mean by a balanced system, but what I would imagine is a system that is equipped to do good assessments of a wide variety of common and more specialized problems of childhood. Other assessment types may include contemporary approaches to social-emotional & behavioral assessment, assessment for determining MR, language impairment, ASD, etc. As for balanced, my bias is toward a multimethod-multisource assessment where corroborating information is observed across methods and sources.
Matt Burns:
Please tell me what you mean by a "balanced assessment system." I suggest using CBM data plus whatever group tests are used (MAP from the NWEA for example).
Bethe:
Related to Linda's question about an assessment system, would you advocate using CBMs across tiers or along with more traditional CBA's? Several in our district are concerned about using fluency measures in that teachers may discount the emphasis on balanced literacy in lieu of drill & practice or focus on speed reading.
Matt Burns:
I'm sorry, I don't know what you mean by a "more traditional CBA." Does that refer to the assessments that come with curricula?

Fluency correlates highly with comprehension and prosody, but you can assess fluency in 1 to 3 minutes. A good comprehension assessment requires 20 minutes per kid. Thus, using CBM will accurately identify kids with problems, who can then be assessed with more in depth tools such as measures of comprehension. This is a resource allocation system. So we have to use data that can identify who needs more help while using as few of the resources we are trying to allocate as possible.
John Hintze:
We use CBMs across all Tiers for progress monitoring.
Kelly:
Please clarify the answer to disability determination using RtI data (John Humphries). My colleagues and I use AIMSweb. We are interested in the use of cut scores for Tier II and III interventions as well as disabilty determination. However, the question about slope minus 1 SEM is confusing to us.
Matt Burns:
Be very careful using progress towards goals (aimlines) for eligibility purposes. Those decision are often unreliable, espcially given that AIMSweb (a very good program) draws its aimlines from the first data point rather than the median of multiple baseline points.

I suggest computing a numerical slope instead and comparing that number to a standard. If the numerical slope falls one standard error of measure below the standard, then the data reliably indicate that he or she is not making sufficient progress. See Lynn Fuchs (2003) article in LDRP or my 2005 article in Journal of School Psychology. Ted Christ also published something about this in the NASP Communique if you are a school psychologist and have access to the Communique.
John Hintze:
I you are using AIMSweb I think what this would mean is that a student's growth is 1 SEM unit or lower than the ROI that corresponds to the 50th percentile. AIMSweb doesn't provide the SEM so you couldn't really do this.
Thea Scott:
Would tier 2 interventions continue if a student is moved to tier 3?
John Hintze:
If a student is moved to Tier 3 then it can be assumed that they are under-responding to the Tier 2 interventions. In this case we wouldn't recommend continuiing the Tier 2 intervention. The way we implement the model Tier 2 and 3 interventions are characteristically differet.
Pamela Radford:
Potentially, the same Tier 2 intervention may stay the same if the student moves to Tier 3. What changes is the instructional time, number of repetitions to mastery, size of group, the amount of content covered, etc.
Matt Burns:
Probably not for that student. It could be that the team decides to continue tier II interventions with an additional supplement, but I would be hesitant to consider that Tier III.
Diane Herrick:
My district has 97,000+ students and nearly 100 elementary schools. I'm concerned with consistency across schools regarding training, universal screenings, progress monitoring procedures & tools, and data management & analysis. How can we best support our schools as we begin RTI in reading?
John Hintze:
At the risk of sounding trite I really think that you have to set up adminitrative supports to marshall an RTI effort that large. Such might call for a totally separate RTI unit for the whole district if your district really wants to do it correct.
Matt Burns:
WOW! You should be concerned. I wish I could tell you how to best support them, but that is unique to your system. I suggest using a standardize measure (e.g., AIMSweb), providing sufficient training and time to consume the data. Good luck and I'm so thankful you are tackling this important issue.
Victoria S. Pando:
When we Progress monitor Tier III or IV students with grade level ORF, it indicates they're not making sufficient progress. Should they be monitored out of grade?
Matt Burns:
I suggest no. Important decisions should be made with grade-level probes. However, you may wish to assess a different skill. If the intervention is addressing phonics, then ORF might not tell the whole tale. Use NWF or LSF too.
John Hintze:
Yes, they definitely should be monitored our of grade level. Grade level monitoring may be too difficult and you might prematurely or inappropriately terminate an otherwise effective intervention because the PM level is too difficult to show progress. See my response previously regarding sampling-back in the curriculum or doing a survey-level assessment.
Tom Trocco:
Once you have identified the students to be placed in Tier 2 intervention, what steps do you take to assign them to groups, especially for upper elementary students for whom you might only have ORF data? And have you seen success in cross-class groupings?
Matt Burns:
A VERY important question. I suggest using the areas of the National Reading Panel. Once they are identified as a struggling reader, assess their fluency (done), their phonics (e.g., NWF or letter-sound fluency) and their phonemic awareness (e.g., ISF or PSF). Use those data to identify the core of the problem and place the kids in cross-grade groups based on the data.
John Hintze:
You would usually do some further assessment and/or use other assessment data. For example, we have used the information provided by "Sidewalks" (the Tier 2/3 intervention program that comes with SF reading program), the Wilson pretest, etc. Once we have found students with similar needs, we would then group across classes.
Kristie Long:
Is there a difference in the readabilty of a MAZE presented in sentence format and a MAZE presented in paragraph format?
Matt Burns:
I'm sorry, but I don't have a good answer. John would be the better person to answer this. However, my preference is always contextual reading. For example, I do not like how the Woodcock Johnson assesses reading comprehension with sentences. Thus, my preference would be paragraph reading, but I don't know of any study that examines this.
John Hintze:
We have found that students' score do not differ statistically in the two formats that you note. This has led us to believe that the MAZE taps into syntatic features of comprehension. Because of this we use another assessment in addition to the MAZE that addresses what we feel are semantic features of comprehension.
ken howell:
I'm with Lisa HS in regards to problem-solving and modifications at the 2nd tier. Isn't it best to find a solution or solutions closest to one which can be employed in a GenEd context (to rule out or in need for Sped)? There are plenty of GenEd teachers who make, and are willing to make, adjustments to instruction.
Matt Burns:
If it is a modification made in a classroom, then it is Tier I. If RTI is to be successful, we have to make it easy and effective. Thus, we start with efficiency and work to effectiveness. I don't think any answer suggested the need to use special education.
John Hintze:
Our Tier 2 interventions are provided in general education.
Mindy Moody:
Where would you suggest starting in a large urban district? There are so many things that need to be done, where do you start?
Pamela Radford:
First, it is important that leadership teams (both building and district level teams) are educatated on the essential components of RtI. Next, the leadership teams should conduct a self-assessment of the essential components (assessment system, instructional practices, problem solving process) that are in place at the building level and district level. There are many self-assessment tools available.
Matt Burns:
I am sympathetic to your comment. Start with data. Collect the data and use them to identify and remediate the classwide problems that Amanda VanDerHeyden discusses while also identifying kids who need immediate and intense help. When in doubt, start with data. It may not be the answer you are looking for, but it is likely the best place to start.
John Hintze:
I think that before you start in any district that a good needs assessment should be done. The two critical pieces that we have found that need to be in place before starting is (a) interventions (i.e., having a sound core curriculum and a menu of interventions that we can call on immediately for Tier 2 and 3), and (b) an assessment system (i.e., a system that can be used for both universal benchmark screening and more frequent progress monitoring). From a football perspective you need to have both your offense (e.g., the core curriculum and progress monitoring) and defense (Tier 2 and 3 interventions) in place and your playbook ready (e.g., you decision making system) before you can begin implementation. If you have only one but not the other you end up looking like the Baltimore Ravens or the Colts. If you have neither say hello to the Dolphins.
Kelly:
Dr. Burns,
Referring to your response to Laura Driscoll's question about goal setting, how do you determine goals for Tier 2 and Tier 3?
Matt Burns:
There are many potential ways. I suggest following the model that the St. Croix River Education District uses. They conducted a simple correlational analysis to determine what scores at the three benchmark assessments predict that students will pass the state accountability test. For example, in second grade a fall score of 43 words correct per minute would predict with 80% to 90% accuracy that the student would pass the state test. The scores for winter and spring would be 72 and 90. Those could serve as level goals for any tier. Then, they computed the number of weeks that occured between those benchmark assessments and determined that a slope of growth of 1.31 words per minute per week was neccessary to improve from 43 to 72 to 90 words in the alloted time. Thus, that would be the criterion for rate of growth for tiers II and III. A kid who was reading 55 words in the winter and progressing at .50 words/minute per week would be moved to the more intense tier. A kid reading 50 words per minute but with a slope of growth of 1.40 would not.
Laura Driscoll:
When determining goals for students is it better to set a standard goal (on benchmark assessment) for all students in Tier I, then individualize the goals (on progress monitoring probes) for Tier 2 and 3? Or use a standard goal for all Tier I and II, and individualized for Tier III? Or some other variation?
Matt Burns:
It is probably best to set standard goals for level at all three tiers and for growth rates in Tiers II and III.
Pamela Radford:
The goal for Tier 1 should be for all students to reach a predetermined criteria to ensure success. At Tier 2, I would recommend starting with a group goal and assess whether or not the goal is feasible based on the discrepancy level of individual students. At Tier 3, set individual goals.
John Hintze:
We use standard goals for both Tiers 1 and 2. The goals for Tier 2 students are essentially Tier 1 because our goal is to get them back into Tier 1. For students in Tier 3 (or 4) we often have to set more individualized goals because they have demonstrated difficulty in meeting the goals of Tiers 1 and 2. Here we use the intra-indivual approach referenced in another answer.
Loraine Lawler:
Would John Hintze please elaborate on other comprehension assessments besides MAZE? We are working on this in our schools who use RtI and progress monitor using ORF and Retell fluency with DIBELS.
John Hintze:
We have been exploring the use of something called the "Sentence Verification Technique(SVT)." Google that and you should find it. It was developed by a colleague of ours in Educational Psychology - Mike Royer - here at UMass.
ken howell:
Does anyone remember the continuum of service? Looking at this (and other) discussions I'm thinking the idea of Tiers is promoting too much concern about which intervention belongs where (as opposed to which fits a student's need). Catagorical thinking is still catagorical thinking.
John Hintze:
While we might get stuck sometimes into "Tier" think, in many ways RTI is a throwback to Phyllis Deno's "cascade of services." Just my two cents.
Matt Burns:
You make an important point! We have to be careful to talk about interventions rather than kids. We should say the intervention was not successful rather than the saying the kid did not make adequate response. That is why I oppose use of the word "responsiveness." This is not a measure of a child's ability to respond.

The Deno continuum of service was a special education model. RTI, obviously, cuts across special and general education. However, I agree that it is not the intervention that drives the tier, but the level of problem analysis needed and the intensity of the intervention, which is consistent with the continuum of service thinking.
Bethe:
I think a balanced assessment system incorporates screening, benchmarking, progress monitoring, and outcome measures. Teachers need tools to inform their instruction. They also need a way to assess overall reading, writing, and mathematical problem-solving skills. Would you advocate using CBMs exclusively or combine them with other types of assessments? If so, what other assessments would you include?
Matt Burns:
O.K. - I got it. I wasn't sure if that was using the term as in "balanced instruction." Thanks for clarifying.

CBM is the most effecient manner to get the same data. For example, the DRA is commonly used and beloved, but it takes 15 minutes per kid. I strongly suggest not dedicating that level of resource to assessment for the purposes of resource allocation. Many schools use the MAP test from NWEA. Those would be good data to include because it is a good test that is administered in a group format. However, I think any assessment system has to include CBM for screening and progress monitoring.
Barbara Countz:
We have found that the integrity of the implementaion of the intervention is an issue for some teachers who just want the students out of the classroom. How is that being addressed?
Pamela Radford:
Before implementing another intervention, ongoing integrity checks of the intervention should be assessed. The purpose is to help students be successful by identifying what works, rather than what was or was not implemented. In my experience, when interventions are not implemented with integrity it is because implementers are asked to implement interventions with little training and ongoing support. RtI is about providing better tools, training and support to ALL staff. We need to move away from blaming staff for not implementing interventions and shift to how can we support successful implementation. A basic philosophical belief of RtI is that everyone is responsible for improving outcomes for all the students.
Matt Burns:
It usually isn't, but it needs to be. Make implementation integrity a part of the process. Every intervention plan should include an assessment of the treatment fidelity.
Laura Driscoll:
Can you talk about any resources or research regarding RTI for math in the elementary grades?
Matt Burns:
Math is still catching up to reading. Here are some references for research regarding concepts of math RTI. Of course the National Math Panel Report is out.

Beatty, R., & Moss, J. (2007). Teaching the meaning of the equal sign to children with learning disabilities: moving from concrete to abstractions. In W. G. Martin, M. E. Strutchens, & P. C. Elliott (Eds.) The learning of mathematics: Sixty-ninth yearbook (p. 27-42). Reston, VA: National Council of Teachers of Mathematics.

Burns, M. K., VanDerHeyden, A. M., & Jiban, C. (2006). Assessing the instructional level for mathematics: A comparison of methods. School Psychology Review, 35, 401-418.

Cheisa, M., & Robertson, A. (2000). Precision teaching and fluency training: Making math easier for pupils and teachers. Educational Psychology in Practice, 16, 297-310.

Christ, T. J., & Vining, O. (2006). Curriculum-based measurement procedures to develop multiple-skill mathematics computation probes: Evaluation of random and stratified stimulus-set arrangements. School Psychology Review, 35, 387-400.

Christ, T. J., Johnson-Gros, K. N., & Hintze, J. M. (2005). An examination of alternate assessment durations when assessing multiple-skill computational fluency: The generalizability and dependability of curriculum-based outcomes within the context of educational decisions. Psychology in the Schools, 42, 615-622.


Fuchs, L. S., Fuchs, D., Compton, D. L., Bryant, J. D., Hamlett, C. L., & Seethaler, P. M. (2007). Mathematics screening and progress monitoring at first grade: Implications for responsiveness to intervention. Exceptional Children, 73, 311-330.

Fuchs, L. S., Fuchs, D., Hamlett, C. L.,, & Stecker, P. M. (1991). Effects of curriculum-based measurement and consultation on teacher planning and student achievement in mathematics operations. American Educational Research Journal, 28, 617-641.

Fuchs, L. S., Fuchs, D., & Karns, K. (2001). Enhancing kindergarteners’ mathematical development: Effects of peer-assisted learning strategies. Elementary School Journal
101, 495-510.

Gersten, R. & Chard, D. (1999). Number sense: Rethinking arithmetic instruction for students with mathematical disabilities. The Journal of Special Education, 33, 18-28.
Helwig, R., Anderson, L., & Tindal, G. (2002). Using a concept-grounded curriculum-based measure in mathematics to predict statewide test scores for middle schools students with LD. The Journal of Special Education, 36, 102-112.

Hintze, J. M., Christ, T. J., & Keller, L. A. (2002). The generalizability of CBM survey-level mathematics assessments: Just how many samples do we need? School Psychology Review, 31, 514-528.

Hofmeister, A. M. (2004). Education Reform in Mathematics: A history ignored? Journal of Direct Instruction, 4(1), 5-11.

Rivera, D. M., & Bryant, B. R. (1992). Mathematics instruction for students with special needs. Intervention in School & Clinic, 28, 71-86.
VanDerHeyden, A. M., & Burns, M. K. (In press). Examination of the utility of various measures of mathematics proficiency. Assessment for Effective Intervention.

VanDerHeyden, A. M., & Burns, M. K. (2005). Using curriculum-based assessment and curriculum-based measurement to guide elementary mathematics instruction: Effect on individual and group accountability scores. Assessment for Effective Intervention 30 (3), 15-29.
John Hintze:
We monitor in either mathematics calculations, applications, or both. AIMSweb comes with math calculation probes. You might also look into the MBSP (Monitoring Basic Skills Probes) available from Pro-Ed. If your district has the money, you might look into something like "Yearly Progress Pro," available from McGraw-Hill Digital Learning.
Mindy Moody:
If a student is in an intensive support; i.e. Tier II or III and is clearly making progress would you still give the students failing grades on their report card?
John Hintze:
For elementary school students I see the RTI process and the classroom grades as distinct. What RTI will give you insight into is the extent to which a child is making growth in the basic academic areas of literacy, mathematics, spelling, and written expression. They might not map on to the content that grades are issued for. So I see the two as different.
Pamela Radford:
Grades and progress monitoring data are providing different types of information about the student's performance. A student may have the skill, but because of a lack of motivation or organizational skills, receive poor grades. So I am not sure why you would want to have one reliant on the other?
Matt Burns:
That is difficult to answer because report cards are a local issue. In my opinion, grades should be based on how much a student has learned. Bloom's classic work showed that grades are based much more on what students knew before instruction occured than on how much they learned.
Jane:
What is the best way to show teachers that RTI is not another add-on and actually is continuous improvement?
Pamela Radford:
Essential with implementing an RtI model is that we abandon "old" practices. RtI is about utilizing our resources more efficiently and effectively, which means we don't keep on doing the same things over and over again just because we have always done them that way. To be more effective in teaching, assessment, and teaming practices, it is essential to identify roles and responsibilities that can be eliminated. One example is streamlining assessment practices by eliminating assessments in which the data is not utilized to make decisions or eliminate redundancies in assessments that collect the same information.
Matt Burns:
Several things - first, help them as much as you can. For example, have a team collect data rather than have the classroom teacher. Second, get the data to the teacher immediately, preferably within 2 days after they are collected. Third, make it part of their regular routine including grade-level team meetings. Finally, constantly show them data so they can see student progress.
Mindy Moody:
How do we monitor treatment fidelity?
John Hintze:
Depeding on the Tier and/or situation we may use a combination of systematic direct observation or descriptive observation.
Pamela Radford:
There are many different tools available to monitor treatment integrity. The instructional planning form (University of Oregon) is a great tool for many trades. Not only does it provide a tool to document interventions, but it can be used to assess implementation integrity through observation. Also, many instructional programs have checklists to assess integrity.
Matt Burns:
With permanent product. Every intervention results in something being made (e.g., worksheets, flash cards, student writing, etc). During the team meeting in which the intervention is designed, be sure to identify what permanent product will result and how it can be used to monitor the implementation integrity. It is not a perfect answer, but according to George Noell at LSU, it works very well.
ken howell:
What is an intervention? Does it really need to be a totally new technique; or can we just 'fine tune'. For example, by increasing the time or adding pre-correction?
Pamela Radford:
A quality intervention includes three essential components:
1)Proactive strategies that set the student up for success (e.g., level of reading material, environmental structure), 2) teaching strategies (what is taught and how), and 3) motivational strategies (how positive and corrective feedback are provided). To identify interventions to improve student performance, we first need to look at why the current intervention is not working. It may be as simple as "fine tuning" the intervention by adding more instructional time or pre-correction procedures. In my experience you are more likely to have greater fidelity of the implementation when interventions are "fine tuned" rather than starting from "scratch." However, in some cases the old intervention needs to be thrown out completely and a new technique implemented.
John Hintze:
I would see increasing time and/or pre-correction as interventions. For me, anything that is done to alter the course and/or direction of learning would be considered an intervention (my highschool physics teacher would be proud of that answer!).
Victoria S. Pando:
Is is appropriate to progress monitor older students with CBM measures like NWF and PSF, if they are struggling readers?
Matt Burns:
I suggest monitoring them in the skill being taught. If you are doing a phonics intervention with a strugglng third grader, then perhaps NWF or letter-sound fluency might be appropriate. However, you should also monitor them with grade-level ORF.
John Hintze:
I think that it would depend on how old the "older students" were.
Pamela Radford:
It depends on what type of decision you are trying to make with the data. If you want to see how the student is doing overall, you want to use a general outcome measure, such as R-CBM. As a general rule of thumb, monitor using R-CBM if students are able to read 10 words read correct or more on a reading passage. And use the NWF and PSF as more diagnostic information since they are designed as mastery monitoring tools. (i.e., measures a specific skill rather than global performance)