--- title: National Study Shows Significant Results on an Unprecedented Scale description: Independent education research firm WestEd recently published the largest ever national study evaluating a math edtech program. image: https://2024.mindresearch.org/hubfs/Blog%20header-%20WestEd.png --- [![](https://2024.mindresearch.org/hs-fs/hubfs/MIND-logo-2018.jpg?width=1701&height=623&name=MIND-logo-2018.jpg)](http://www.mindresearch.org/) [![mind logo](https://2024.mindresearch.org/hubfs/graphics/logos/MIND-logos-color2.svg)](https://www.mindeducation.org/) [![mind white logo](https://2024.mindresearch.org/hubfs/graphics/logos/MIND-logos-white2.svg)](https://www.mindeducation.org/) - [Our Approach](https://2024.mindresearch.org/our-approach) - [Why MIND?](https://2024.mindresearch.org/why-math-education) - [Our Research](https://2024.mindresearch.org/our-research) - [Our Patents](https://2024.mindresearch.org/patents) - [Our Impact](https://2024.mindresearch.org/impact) - [Our Partners](https://2024.mindresearch.org/partners) - [Partner Stories](https://2024.mindresearch.org/partner-stories) - Our Programs - [ST Math](http://www.stmath.com/) - [InsightMath](http://www.insightmath.com/) - [MathMINDs](https://www.mindresearch.org/mathminds) - [Life at MIND](https://2024.mindresearch.org/careers) - Resources - [Blog](https://2024.mindresearch.org/blog) - [Math Discourse](https://2024.mindresearch.org/puzzletalks) - [Podcast](https://2024.mindresearch.org/edtech-podcast) - [News](https://2024.mindresearch.org/news) - [Activities](https://2024.mindresearch.org/stem-resources) - [Donate](https://store.mindresearch.org/pages/donate) **Student privacy****:** learn more about our commitment to keeping students safe [Read More](https://www.mindresearch.org/privacy-policy) ![MIND-on-Math_Blog-Header-5](https://2024.mindresearch.org/hs-fs/hubfs/graphics/headers/_blog-headers/MIND-on-Math_Blog-Header-5.png?width=2000&height=667&name=MIND-on-Math_Blog-Header-5.png "MIND-on-Math_Blog-Header-5") Category Community Teaching & Learning Technology & Games Research & Outcomes Neuroscience Celebrations & Successes [![Subscribe](https://no-cache.hubspot.com/cta/default/237516/a35fb57d-2960-4554-a0f3-e7bb5946fc70.png)](https://cta-redirect.hubspot.com/cta/redirect/237516/a35fb57d-2960-4554-a0f3-e7bb5946fc70) [Research & Outcomes](https://2024.mindresearch.org/blog/topic/research-outcomes) # National Study Shows Significant Results on an Unprecedented Scale By [Andrew Coulson](https://2024.mindresearch.org/blog/author/andrew-coulson) - - - - - [mailto:?subject=National%20Study%20Shows%20Significant%20Results%20on%20an%20Unprecedented%20Scale&body=National%20Study%20Shows%20Significant%20Results%20on%20an%20Unprecedented%20Scale%20https://2024.mindresearch.org/blog/national-study-significant-math-results](mailto:?subject=National%20Study%20Shows%20Significant%20Results%20on%20an%20Unprecedented%20Scale&body=National%20Study%20Shows%20Significant%20Results%20on%20an%20Unprecedented%20Scale%20https://2024.mindresearch.org/blog/national-study-significant-math-results) - Independent education research firm WestEd recently published the largest ever national study evaluating a math edtech program. The study found significant results at schools that used [ST Math](https://www.stmath.com/), a PreK-8 visual instructional program that builds a deep conceptual understanding of math through rigorous learning and creative problem solving. For schools that implemented ST Math consistently, the results were especially significant. See an overview of the study in the following infographic: [![WestEd-Study-Infographic_EN-180_web-01.png](https://2024.mindresearch.org/hs-fs/hubfs/Blog/Images/WestEd-Study-Infographic_EN-180_web-01.png?width=650&height=840&name=WestEd-Study-Infographic_EN-180_web-01.png)](https://2024.mindresearch.org/hubfs/Operations/Collateral/Research%20and%20Results/WestEd-Study-Infographic-2018.pdf) [*Download the Infographic*](https://2024.mindresearch.org/hubfs/Operations/Collateral/Research%20and%20Results/WestEd-Study-Infographic-2018.pdf) #### What are the Key Findings? Two outcomes were evaluated: average math scale scores and the proportion of students who were proficient or above in math. For both measures, grades that consistently implemented ST Math improved significantly more than similar grades that didn’t use the program. For example, a school that initially ranked at the 50th percentile in its state and then consistently implemented ST Math, would, on average, get a statistically adjusted boost of 14 percentile points in statewide ranking. The effect size of all grades combined equaled **0.36**, surpassing the [Federal What Works Clearinghouse criteria](https://ies.ed.gov/ncee/wwc/Handbooks) for “substantively important” effect. This means, generally speaking, that using ST Math made a meaningful difference in schools’ math achievement. #### Which states were included in the study? The 14 states in the high fidelity group were California, Colorado, Connecticut, Florida, Georgia, Iowa, Michigan, Missouri, New Jersey, Nevada, New York, Texas, Virginia, Wisconsin. #### How was it possible to compare different state tests? Since every state has its own standardized test, comparing test scores across state lines is like comparing apples to oranges. To normalize them, the researchers z-scored each school, grade by grade, within the state – essentially providing a statewide ranking. They then looked at how each grade at the school scored within their state before and after using ST Math, and compared it to how similar schools’ scores changed within their state during the same timeframe. With all 14 states’ math tests transformed to the same basis (statewide z-score) it was possible to aggregate and compare data across any state or assessment. #### Which grades and schools were included? Rather than selecting specific districts or a single state, this study examined *all* grades 3, 4 and 5 at all schools using ST Math in 2015-16 in 36 states. Schools had to have used ST Math for 1 to 3 years and state grade-average standardized test scores had to be complete and available for those years. #### What does it mean to consistently implement ST Math? Schools were included in this group if at least 85% of students in the grade used ST Math, and if those students covered, on average, at least half of their ST Math grade-level content. As a general guideline, most students can complete their grade-level program if they use ST Math for about 20 minutes every weekday, or 90 minutes/week either at home or at school. The report also separately evaluated all 80,000 ST Math students regardless of implementation and found a significant effect size of 0.17. #### How was the control group created? Like other rigorous studies, WestEd carefully matched the treatment group (which used ST Math) and a similar control group (which did not use ST Math). To create the control group, WestEd statisticians matched each ST Math grade at a school to the most similar grade at another school in the same state based on size, ethnic composition, percent free and reduced lunch, and baseline year test performance. Control schools were not filtered based on math programs being used.   [Read the full report from WestEd.](https://www.wested.org/resources/cross-state-evaluation-of-mind-research-institutes-spatial-temporal-math/) #### EdTech Evaluation Resources: - [Program Evaluation Rubric](http://blog.mindresearch.org/blog/edtech-evaluation-rubric) - [Federal What Works Clearinghouse Handbook](http://blog.mindresearch.org/blog/edtech-evaluation-rubric) - [The Need to Hold Programs Accountable for Long-term Impact](http://blog.mindresearch.org/blog/what-if-you-held-your-math-program-accountable-for-long-lasting-impact) ![Andrew Coulson](https://2024.mindresearch.org/hubfs/2018_S2/meet-the-team/andrew-coulson-bio-185230-edited.jpg) ### About the Author Andrew R. Coulson, Chief Data Science Officer at MIND Research Institute, chairs MIND Education's Science Research Advisory Board and drives and manages all research at MIND enterprises. This includes all student outcomes evaluations, usage evaluations, research datasets, and research partnering with grants-makers, NGOs, and universities. With a background in high-tech manufacturing engineering, he brings expertise in process engineering, product reliability, quality assurance, and technology transfer to edtech. Coulson holds a bachelor's and master's degree in physics from UCLA. ### Comment #### [Educator Perspectives: Taking Math Culture to the Next Level](https://2024.mindresearch.org/blog/math-personalized-learning-culture) [Assessment and Adaptation in Personalized Learning](https://2024.mindresearch.org/blog/adaptation-personalized-learning) ### Search Result By: ![feature image](https://2024.mindresearch.org/hubfs/Blog/Images/Inside-our-Minds-logo-teal-800x800.png) [Featured Podcast](https://2024.mindresearch.org/blog/topic/featured-podcast) ### [Rethinking Attendance in Edtech](https://2024.mindresearch.org/blog/rethinking-attendance-in-edtech) [Listen Now](https://2024.mindresearch.org/blog/rethinking-attendance-in-edtech) ##### LATEST PODCAST EPISODES ###### BY MIND RESEARCH INSTITUTE - [Rethinking Attendance in Edtech](https://2024.mindresearch.org/blog/rethinking-attendance-in-edtech) - [People, Process, and Penguins: Designing Math Experiences](https://2024.mindresearch.org/blog/people-process-penguins) - [Bridging STEM Gaps with STEMconnector CEO Jo Webber](https://2024.mindresearch.org/blog/bridging-stem-gaps-with-stemconnector-ceo-jo-webber) - [The EdTech Genome Project: Talking Implementation with Elizabeth Birie](https://2024.mindresearch.org/blog/edtech-genome-project) - [From Meeting the Moment to Foundational Change](https://2024.mindresearch.org/blog/from-meeting-the-moment-to-foundational-change) ##### [See all podcasts](https://www.mindresearch.org/edtech-podcast) ## Interested in Contributing? [![Read Our Blog Guidelines](https://no-cache.hubspot.com/cta/default/237516/a5ccc223-4d16-4ac9-b049-55259b5fa0fc.png)](https://cta-redirect.hubspot.com/cta/redirect/237516/a5ccc223-4d16-4ac9-b049-55259b5fa0fc) ## Join Our Newsletter - [Speakers](https://2024.mindresearch.org/speakers) - [Partners](https://2024.mindresearch.org/partners) - [Patents](https://2024.mindresearch.org/patents) - [Resources](http://blog.mindresearch.org/resources) - [Equity](https://2024.mindresearch.org/equity) - [Accessibility](https://www.mindresearch.org/accessibility) - [Blog](https://2024.mindresearch.org/blog) - [Math Discourse](https://2024.mindresearch.org/math-puzzle-talks) - [Podcast](https://2024.mindresearch.org/edtech-podcast) - [News](https://2024.mindresearch.org/news) - [ST Math](http://STMath.com) - [MathMINDs](https://2024.mindresearch.org/mathminds) - [Careers](https://2024.mindresearch.org/careers) - [MIND Marketplace](https://store.mindresearch.org/) - [FAQs](https://2024.mindresearch.org/faq) - [Contact Us](https://2024.mindresearch.org/contact-us) - [Site Map](https://2024.mindresearch.org/site-map) [![DONATE](https://no-cache.hubspot.com/cta/default/237516/c729ff2c-0758-4634-bf2d-46c915ce5cba.png)](https://cta-redirect.hubspot.com/cta/redirect/237516/c729ff2c-0758-4634-bf2d-46c915ce5cba) - [*facebook*](https://www.facebook.com/MindEduORG) - [*twitter*](https://x.com/MINDEduOrg) - [*youtube*](https://www.youtube.com/user/mindresearch/) - [*pinterest*](https://www.instagram.com/mindeducationorg/) - [*linkedin*](https://www.linkedin.com/company/mind-research-institute/) [![DONATE](https://no-cache.hubspot.com/cta/default/237516/c729ff2c-0758-4634-bf2d-46c915ce5cba.png)](https://cta-redirect.hubspot.com/cta/redirect/237516/c729ff2c-0758-4634-bf2d-46c915ce5cba) Copyright © 2024 MIND Education®. All rights reserved. [Terms of Use](https://2024.mindresearch.org/terms-of-use) | [Privacy Policy](https://2024.mindresearch.org/privacy-policy) Search: Jump to: Category Community Jump to: Most Recent Most Popular Most Recent Most Popular