Validity Evidence for Measurement in Mathematics Education (VM²Ed), is a Level 3, track 1 proposal seeking to advance fundamental knowledge in Research on STEM Learning and Learning Environments. In 2017, the Validity Evidence for Measurement in Mathematics Education (VM²Ed) National Science Foundation-funded conference brought together 40 educational researchers with expertise in mathematics education, psychometrics, and applied measurement to construct a shared understanding regarding validity within mathematics education contexts. We seek to extend work from the first grant, to contribute knowledge and resources about the validation and validity evidence of quantitative assessments in mathematics education contexts in ways that are potentially transformative for those working in K-20 mathematics education. We propose to research validity evidence for instruments used in mathematics education by hosting a conference, conducting syntheses, developing a repository, and scaling up our work.

This project is supported by the National Science Foundation under Grant No. DRL 1920619 awarded to North Carolina State University and Grant No. DRL 1920621 awarded to Bowling Green State University. Any opinions, findings, and conclusions or recommendations expressed herein are those of the principal investigators and do not necessarily reflect the views of the National Science Foundation.

Project Research Questions

Research Question 1: How might validity evidence related to quantitative assessments used in mathematics education research be categorized and described?

Research Question 2: What validity evidence exists for quantitative instruments used in mathematics education scholarship since 2000?

Research Question 3: How do users draw upon the repository to inform their professional work?

Research Question 4: What are conference attendees’ perceptions of the repository and associated training resources?

Project Goals

Goal 1: Conference. Develop a framework for categorizing and describing validity evidence associated with quantitative assessments for math education contexts. (BLUE)

Goal 1A. Determine a framework to apply when reviewing assessments.

Goal 1B. Determine features of validity evidence linked with assessments that will fuel the repository.

Goal 2: Synthesis. Synthesize, describe, and categorize math education instruments using available published materials alongside their validity evidence. (RED)

Goal 2A. Organize and oversee teams of scholars for diverse instruments to support the synthesis work.

Goal 2B. Synthesize, categorize, and describe the quality of quantitative instruments using the framework designed through goal 1.

Goal 2C: Review results from syntheses to move them to the repository and for dissemination.

Goal 3: Repository. Develop and implement a repository of quantitative instruments and their validity evidence for math education contexts. (PURPLE)

Goal 3A: Build repository within a university-sponsored website that allows free access to all scholars and practitioners interested in math education.

Goal 3B: Maintain and update repository annually.

Goal 3C: Train scholars interested in being peer-facilitators to function as curators.

Goal 4: Scale Up. Train scholars how to effectively use the repository. (ORANGE)

Goal 4A: Create materials and online training that facilitate learning for a variety of scholars and practitioners in the math education space (via face-to-face and online trainings).

Project Research Plan

Our project timeline (Figure 1) is guided by the four research questions and associated project goals.