How does standards-based grading affect standardized test scores?

A teacher from a school in Iowa recently contacted me with an inquiry,

Is there any research indicating students going through a standards based grading system show improved standardized test scores  or college readiness compared to students that go through a traditional grading school?

Context: What we know about educational research

This is an excellent question that comes up often when educators and their school systems are considering a shift towards standards-based grading. Based upon my knowledge of research methodology, it would be challenging (but not impossible) to conduct a study truly isolating standards-based grading as a single variable influencing a school’s student achievement data. In other words, we know that in any educational study, limitations exist (i.e. fidelity of implementation, additional initiatives simultaneously assisting or hindering test scores, the alignment between state standards taught and standards assessed, and any placebo effect).  Finally, any experienced educational researcher will readily admit the challenge of generalizing study results outside of the original context.

How does standards-based grading affect standardized test scores?

I scoured the literature as part of my recent dissertation and found only a few quantitative studies that consider this type of question (How does SBG affect standardized test scores?), some of which look at an entire school and others that look at a smaller section of a school, such as an individual classroom. Each has clearly stated the study limitations.  Here’s a quick copy/paste from my dissertation:

The impact of standards-based grading on external achievement measures is mixed. While some studies indicate a significant statistical difference in one or more content areas (e.g. Haptonstall, 2010; Pollio & Hochbein, 2015), others have not (e.g. Rosales, 2013; Welsh, D’Agostino, & Kaniskan, 2013).

To add additional clarity, I’ve added more detail below related to each of the aforementioned citations.

Haptonstall (dissertation): From the abstract of the study, emphasis mine:

This study examined the correlation between the grades a student earns in his or her classroom and the scores that each student earned on the Colorado Student Assessment Program tests, in Reading, Writing, Math, and Science. The study also examined the mean scores of varying sub-groups to determine if certain sub-groups demonstrated higher means, dependent of the school districts that they were enrolled. While all the school districts that participated in the study showed a significant level of correlation between grades and test scores, Roaring Fork School District Re-1, using a standards- based grading model demonstrated both higher correlations and higher mean scores and grades across, the overall population and sub-groups.

Pollio & Hochbein: From the abstract of the study:

Results indicated that the rate of students earning an A or B in a course and passing the state test approximately doubled when utilizing standards-based grading practices. In addition, results indicated that standards-based grading practices identified more predictive and valid assessment of at-risk students’ attainment of subject knowledge.

Rosales: From the dissertation study, emphasis mine:

This study seeks to determine whether standards-based grading has the same effect on students at the high school level (grades 9-12) by comparing end-of-course test scores and posttest scores of Algebra 2 students enrolled in a standards-based graded classroom with to those enrolled in a traditionally-graded classroom. Two teachers each taught two classes of Algebra 2 and graded one class using standards-based grading and one class using traditional grading methods. Students at both the honors level and the regular level of mathematics were included in the study.
Honors students performed better than regular students on both assessments, but no significant difference was found between the performance of traditionally-graded students and the students who were graded with standards-based grading.  The results of this study indicate that standards-based grading may offer improved methods of communication between teachers, parents, and students and may give students a new perception of learning.

Welsh, et al.: From the abstract, emphasis mine:

Standards-based progress reports (SBPRs) require teachers to grade students using the performance levels reported by state tests and are an increasingly popular report card format. They may help to increase teacher familiarity with state standards, encourage teachers to exclude nonacademic factors from grades, and/or improve communication with parents. The current study examines the SBPR grade–state test score correspondence observed across 2 years in 125 third and fifth grade classrooms located in one school district to examine the degree of consistency between grades and state test results. It also examines the grading practices of a subset of 37 teachers to determine whether there is an association between teacher appraisal style and convergence rates. A moderate degree of grade–test score convergence was observed using three agreement estimates (coefficient kappa, tau-b correlations, and classroom-level mean differences between grades and test scores). In addition, only small amounts of grade–test score convergence were observed between teachers; a much greater proportion of variance lay within classrooms and subjects.

Given inconclusive evidence, why would a teacher or school decide to embark upon standards-based grading?

Here’s what I’ve found: First, the absence of research supporting traditional grading practices is concerning. Too many times, stakeholders enter these types of conversations assuming traditional grading practices are some type of research-proven paradigm when in fact, schools inherited the traditional grading system over one hundred years ago (Cureton, 1971).  Next, beyond the limited quantitative studies related to standards-based grading available right now (and summarized above), classroom teachers can do a better job aligning what we teach with our assessments, a fundamental tenet of grading reform.  In other words, due to a dearth of evidence, educators should strongly consider pragmatic solutions.  For more on these pragmatic themes and their connection to scholarly literature, I encourage readers to consider, “What does the research say about standards-based grading?” a research primer I co-wrote with Dr. Tom Buckmiller several years ago.  Finally, standards-based grading provides students and parents with more useful information about current levels of work (“proficient understanding of Pythagorean’s Theorem”) when compared to traditional grading practices (“85% on the Chapter 3 Test”).

In summary, the impact of standards-based grading on external achievement measures is mixed. With limited quantitative studies available to base this conclusion upon, educators should consider pragmatic benefits of standards-based grading practices and urge educational researchers to respond to this question with more detail in the future.

Works Cited

Cureton, L. W. (1971). The history of grading practices. NCME Measurement in Education, 2(4), 1-8.

Haptonstall, K. G. (2010). An analysis of the correlation between standards-based, non standards-based grading systems and achievement as measured by the Colorado Student Assessment Program (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (UMI No. 3397087)

Pollio, M. & Hochbein, C. (2015). The association between standards-based grading and standardized test scores as an element of a high school model reform. Teachers College Record, 117(11), 1-28.

Rosales, R. B. (2013). The effects of standards-based grading on student performance in Algebra 2 (Doctoral dissertation). Retrieved from http://digitalcommons.wku.edu/diss/53/

Welsh, M. E., D’Agostino, J. V., & Kaniskan, B. (2013). Grading as a reform effort: Do standards-based grades converge with test scores? Educational Measurement: Issues and Practice, 32(2), 26-36. doi:10.1111/emip.12009

Doctoral program: 36 month update [graduated!]

Nearly three years ago, I started a doctorate in education program at the University of West Georgia.  Today, I am happy to share all coursework and dissertation requirements are finished and as a result, I recently walked across the stage!

(Dissertation chair, Matt Varga, and me before commencement)

Although participating in the ceremony is optional, it provided a sense of closure and personal accomplishment for me.

What is next?

  1. Business as usual as a district administrator.
  2. Teaching a weekend curriculum course for aspiring school administrators this summer.
  3. Continue working with my chair to publish portions of my dissertation research in one or more peer-reviewed journals.

Previous doctoral program updates:

So, you want to become a curriculum lead?

I wrote the column below for Iowa ASCD’s “The Source,” an e-newsletter for state ASCD affiliate members.

So, you want to become a curriculum lead?

Around this time of the year, I usually receive an email or two from a teacher who is interested in transitioning to a more defined curriculum role.  Whether it’s a classroom teacher who recently completed a masters program or a veteran educator thinking about a teacher leadership role created by TLC funds, desiring to have an impact beyond the classroom can be an exciting, yet uncertain time.  In my experience and observations, several different roles exist involving curriculum leadership.  I will unpack each one in the following paragraphs.

Instructional Coaches

The specific responsibilities of an instructional coach often vary by district or building, however these educators are charged with assisting adults improve their practice as educators.  Sometimes, instructional coaches work alongside individual teachers to provide instructional feedback.  Other times instructional coaches work with groups of teachers (i.e. data teams or collaborative learning teams).  Jim Knight has a nice write-up elaborating on the role of instructional coaches.  Finally, an instructional coach job description may include more formal curriculum leadership roles within a building or district, such as serving on curriculum materials review committees.

Some instructional coaches are content-neutral (i.e. the “elementary instructional coach”) whereas others are content-specific (i.e. the “K-8 math instructional coach.”)  With an increase in teacher leadership dollars the past three years, many Iowa public schools now have funds to employ instructional coaches and other similar roles.  Iowa’s teacher leadership funds come with a few stipulations – teachers assigned to a leadership role must have at least three years of teaching experience and at least one year of experience in the school district (Source: Iowa Department of Education)

AEA Curriculum Consultants

Many area education agencies (AEAs) in Iowa employ math, reading, science, and technology consultants. These positions are content-specific and frequently require a masters degree.  AEA curriculum consultants are often assigned several school districts and provide support to individual teachers or entire departments, as agreed upon by the AEA and district.  AEA curriculum consultants also lead workshops at the agency that teachers across the AEA can participate.  Teachers who desire to make an impact beyond a single school building and enjoy making connections between and across districts may enjoy an AEA curriculum consultant role.  Pre-requisites for this role include a strong content knowledge, pedagogical content knowledge and experience leading professional learning.  In addition, AEA curriculum consultants are often invited to participate on state level task forces and committees when new standards and state initiatives are rolled out in Des Moines.

Curriculum Directors/Coordinators

For the past six years, I have served as a district curriculum director for over 100 teachers and approximately 1,500 students.  In some smaller school districts, the curriculum director responsibilities may be assigned to a principal. Still other small districts, share a curriculum director.  Larger districts may employ several curriculum directors or coordinators who specialize in one or more areas (i.e. “Middle School Director of Teaching & Learning” or “Science Curriculum Coordinator”).  In my role, the curriculum director title comes with many additional responsibilities including directing the ELL, Title 1, gifted education, special education, and teacher leadership programs and overseeing the media services and technology departments.  I complete a number of state and federal reports; facilitate curriculum materials review and purchasing processes; attend meetings to keep abreast of new state and federal initiatives; manage the curriculum and teacher leadership budgets; and serve as the primary contact for Department of Education accreditation.  Often times, curriculum directors are on extended contracts requiring an PK-12 administrative endorsement, however some schools may honor a masters degree in curriculum or a related field.

Concluding Thoughts

Serving as a curriculum lead can be a rewarding experience.  Any step away from working with the same students each day/week deserves careful thought.  Hopefully, this column provides an overview of several possible curriculum leadership roles.  We’re fortunate in Iowa each public school opening must be posted on TeachIowa.gov, so we have a one stop shop for curriculum leadership openings throughout the year.  Are you wondering if your experiences might be preparing you for a curriculum leadership role?  It would be my pleasure to review and provide feedback on your resume.  Happy job hunting!

Three Lessons for Schools Shifting Their Grading

I was recently invited to write a brief article for School Administrator, an AASA (The School Superintendents’ Association) publication.  “Three Lessons for Schools Shifting Their Grading” was embedded within a feature article about standards-based grading, (a well written one, I might add) by Ken O’Connor.

The full article is available via the AASA digital magazine link here.

Doctoral program: 30 month update

Nearly two and a half years ago, I embarked on a third graduate program.  A little over six months ago, I provided another update.  Today, I am happy to share the light is very bright at the end of the tunnel:  I recently defended my dissertation!  If you’re not familiar with the process, a final dissertation defense is when a doctoral candidate provides a verbal and visual overview of the study to the committee (comprised of university professors).  Following the presentation, the committee asks questions for the purpose of better understanding the study’s results, conclusions and implications.  Finally, the committee deliberates and communicates one of several possible outcomes: pass without revisions (rare), pass with minor revisions (often), fail (with opportunity to try again), fail.

What is next?

  1. I have several small revisions to make and will then submit my dissertation to an online database (ProQuest) for permanent storage and retrieval.
  2. Complete one more class next semester (noted below)
  3. Graduate in May 2017!

semester_countdown

Standards-based grading: Frequently asked questions about reassessment

The email questions continue to roll in this school year.  A math teacher from California gave me permission to share some of our recent communication:

If there is unlimited number of reassessment, how do you manage students who want to re-assess till they attain “got it” and eventually the school year is ending?

Our teachers usually have a “final reassessment deadline.”  For example, if the semester ends on December 20 and grades are due on December 23, a teacher might communicate a final reassessment deadline of December 18.  In the ideal world, students would be allowed to re-assess forever (think: 31 year old re-assessing his high school government coursework!), however the realities of our current educational paradigm (180 days, bells, student schedules, etc.) force us to create arbitrary deadlines, even in a standards-based grading system. It would be very challenging for a teacher to allow reassessments to be turned in up until the last minute grades are due to the school, therefore some type of buffer should be established to allow for the teacher to score and enter the final batch of reassessments.  In general, as long as the final reassessment deadline is communicated early and often, students and parents understand the need for the teacher to grade the reassessments and get them entered into the grade book before the school-imposed grade deadline.
If the student can choose to reassess just one standard, doesn’t it tell the student what concept or skill is required to solve the problem given and makes learning segmented? For the reassessment problem, should it be similar to the previous assessment/practice or should a novel problem be given? 
You’ve identified one of the biggest critiques I have read/heard of standards-based grading — narrowing down learning into finite concepts rather than seeing math as a larger body of ideas in which life rarely tells us which formula to use.  With this in mind, any assessment in a standards-based grading or traditional grading classroom is only as good as its author.  In other words, any test could be written with a very low cognitive complexity or a very high level or rigor.  I have seen standards-based grading tests that break down the concepts into fine-grained concepts and I have seen others that expect students to climb Bloom’s taxonomy.  I believe a starting point is thinking about re-assessments at a similar level of cognitive complexity as the original assessment.  Sometimes the expected level of rigor is described in the standards themselves.  For example:

CCSS.MATH.CONTENT.HSG.GMD.B.4
Identify the shapes of two-dimensional cross-sections of three-dimensional objects, and identify three-dimensional objects generated by rotations of two-dimensional objects.

“Identify” may give the assessment author a glimpse into the expected level of Bloom’s intended by the writer of the standards, in this case the Common Core State Standards. 
Does any of this make sense?

Standards-based grading: Integrating homework completion with reassessment

A teacher from a nearby school district recently emailed me with a few questions about standards-based grading in preparation for the upcoming school.  He gave me permission to share some of our dialogue in this public space.  This is the second of two question and answer posts about homework as practice, rather than merely point accumulator, in standards-based grading. (The first one is here)

Q: I have heard of some teachers that will not allow retakes on assessments unless the homework has been completed. What are your thoughts?

A: Great question! I have seen this work in two different ways, each with pros and cons.

The first perspective goes something like “Students may not complete a reassessment unless they completed the homework prior to the initial assessment.”

Pros: May motivate students to do the homework prior to the first assessment; may help students do better on the initial assessment (because they did their homework).  Cons: May motivate students to copy homework and play this part of the “game of school.”  Only allowing students an opportunity to reassess if they completed homework seems to go against the idea students learn at different rates/paces.  What if the reason the student didn’t complete the homework initially is because he/she had no idea about the concepts?  Why require the student to do a slew of problems incorrectly?

The second perspective is the one I tend to favor, “Students may complete a reassessment only if they completed the homework sometime (doesn’t matter if it it was before or after initial assessment) and complete additional re-learning steps.”

Pros: Better honors that students learn at different rates/paces; may also be a better chance students are likely to ask better questions when completing the homework “Okay, I obviously didn’t get this idea as you saw on the test, Mr. H….could you help me understand it on the homework?”  Re-assessment should be meaningful for students while not overburdening the instructor.  Cons: It’s not a silver bullet!  Students may still game the system by copying from a friend…but there’s less incentive to do so, because if they copy and do not learn, they will not likely do any better on the reassessment.

How do you integrate homework completion and the reassessment process within standards-based grading?  Feel free to leave your experiences in the comments below.

Standards-based grading: Motivating students to complete homework/practice without points

A teacher from a nearby school district recently emailed me with a few questions about standards-based grading in preparation for the upcoming school.  He gave me permission to share some of our dialogue in this public space.  This is the first of two question and answer posts about homework as practice, rather than merely point accumulator, in standards-based grading.

Q: When homework is no longer completed for points or a grade in a standards-based grading environment, I could see some students reluctant to submit homework assignments. Do you have any advice on convincing students that homework is still a required, essential part even if they aren’t getting a score for it? What about those kids who just don’t turn in their work?

A: Let’s be honest for a moment – there’s no silver bullet for motivating kids to do homework! When we think about the traditional paradigm, students do not often value practice/homework, even when it is worth points. Evidence of this theory includes students across the country in high schools with traditional grading who do not turn in their homework assignments and students who turn in copied homework assignments.

Here is my biggest bit of advice: In traditional grading, when a student does not turn in an assignment, we give them a zero, but do not talk with them about it. We assume they’re going to “learn” from the zero. Instead, I tried something different in my later years as a high school teacher.  It went something like this…

Class, the purpose of homework in this class is practice.  Although homework is not worth any points, it is still very important.  The purpose of homework is practice!  Just like you practice for a small group speech contest, vocal music solo or volleyball game, practicing is important in math class.  Turn to your neighbor and share with them two reasons you believe practicing in math class is important and one reason why you might be tempted to not complete homework in this class….Okay, who can share with me a summary of their partner discussion?  How many of you talked about the importance of practice in doing well on tests?….

Then, when a student (inevitably) decides to not turn in an assignment a few days later, have a good ole fashioned one-on-one, sit down conversation with him/her.

Why did you decide not to do this assignment?

and

How well do you think you’ll do on the next assessment?

I’ve found this to be a relationship building exercise and gives me a better idea how I can help the student. Sometimes students are going through tough times at home. Other times, they’re lost conceptually and don’t want to admit it.

This is not a cure-all type of solution, but one that I and other educators using standards-based grading practices have found to be helpful.  In what ways have you motivated students to complete homework when it is worth zero points?  Leave your ideas in the comments below.

 

Doctoral program: 24 month update

Nearly two years ago, I embarked on a third graduate program.  Recently, I defended my dissertation proposal.  This is a big time milestone!  If you’re not familiar with the process of a doctoral program, the dissertation proposal is a draft of the first three chapters reviewed by the committee in which they give a doctoral student the “green light” to continue.  Chapter one is an overview of the study.  Chapter two is a literature review covering related studies and describing how the dissertation fills a “gap” in the literature.  Chapter three is the methodology or to rephrase it, who, what, when and why the researcher plans to collect and analyze data in order to answer each carefully crafted research question.

What is next?

  1. Upon receiving approval from the university’s institutional research board (think: government-required committee ensuring research on human subjects is completed with the appropriate precautions and participant consent) within the next month, I will begin collecting data this summer.
  2. Write chapter 4 (results)!  This will involve a lot of tables with very little original thinking.
  3. Write chapter 5 (discussion and recommendations)!  My committee chair suggested this could be the hardest chapter to write outside of the literature review, because it involves making a lot of connections between the results and previous studies.
  4. Defend the dissertation.  This a final time for the committee to ask questions following significant review of all five chapters.
  5. Complete a few more classes (noted below) and walk across the stage in Spring 2017 to be hooded!

semester_countdown

All of this aligns with the ambitious timeline I described in the 18 months update.

Help from a few friends

None of this could be possible without the support of two critical friends in the program, Bipul and Steve, who have provided feedback encouragement along the way.  I also owe a big thank you to Mrs. T. for allowing me to spend 3-4 hours per week holed up in the local library at night working on dissertation drafts.  Completing a doctoral program while working full-time has proven to be a much more challenging task than the first two graduate programs, but with help from a few friends, it has been doable.

Look for another update in a semester or two!