Narrative

Using Item Analysis software to Examine Students' Progress

Introduction and Purpose:

This grant proposal requests funding for a statistical software to analyze items in classroom and standardized assessments as well as allow students the possibility to conduct research using the software.

All undergraduate education students in the College of Education are required to take the course of EDF 3430, Measurement and Evaluation in Education. These education majors are called pre-service teachers.  As part of the course curriculum, students must familiarize with measurement concepts and be introduced with data generated by classroom tests and standardized-test. Among the skills they are introduced to the concept conducting item analysis on test items.

Per academic year, there are approximately 10 sections of this course, with an estimate of 780 students enrolling per year across all campuses. As the lead instructor of EDF 3430, I am planning to advance item analysis assignments across all sections to allow undergraduate students to be exposed to item analysis software and to provide them with cutting-edge skills so they could understand the process of writing well-founded assessments and the technical decisions that need to be involved related to psychometrics. Additionally, the software would allow students interested in pursuing research initiatives analyze their data and have a deeper understanding of item functioning.

The current teacher assessment knowledge requires teachers to use multiple assessment formats. Brookhart (2011) suggests that teachers should have the skills to analyze classroom questions, test items, and performance assessment task to determine the specific knowledge and thinking skills required for students as well as articulate interpretation of assessments.

Item Analysis Software

Using the STATA software, they would be able to see Rasch models function and the purpose of conducting such analysis. Rash models analyze categorical data, such as answers to a history or reading assessment as a function of the balance between (a) the respondent's abilities, attitudes or personality traits and (b) the item difficulty. The proposal request the purchase of two types of software license to allow students to analyze data with instructional purposes and with research purposes: 50-user Stata/MP 13 (2-core) Lab and 2-user Stata/MP 13 (2-core) Network.         

Method of Implementation

The software will become an integral module to pre-service teachers enrolled in EDF 3430 Measurement and Assessment class. During the first year, selected sections of EDF 3430 will be exposed to the item analysis software. Attempts will be made to access large-scale data to demonstrate item analyses. During the second year, a pilot study will be designed to examine the impact of learning with technology and their measurement knowledge. Outcomes may be share through professional meetings and refereed publications.             

Alignment of this Project with FAU strategic Plan (2012-2017):

FAU Goal I: Enrich the educational experience

This project will enhance the experiential aspect of understanding how scores are made up of and will expose pre-service teachers to new assessment technology resulting in a more meaningful integration of a variety of assessment formats needed in their future classrooms.

 FAU Goal II: Inspire research, scholarship and creative activity

This project will provide an opportunity to use research tools among undergraduate students as well as may create venues for students who may wish to follow a measurement and psychometrics path.

 References

Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30(1), 3-12.

Facilities

None needed.

Hardware Requirements

There are not hardware requirements.

Software Requirements

The purchase of a software for data analysis. The proposed software to be funded is STATA.

Stata statistical software is a complete, integrated statistical software package that provides everything you need for data analysis, data management, and graphics.  http://www.stata.com/why-use-stata/

Unlike the current statistical software used , SPSS, this software allows RASCH models analysis.

Personnel Costs

None associated.

Other Costs

None associated.

Timeline

Upon receipt of funding implementation to start Fall 2015 implementation.

Sustainability

This license is perpetual. Therefore, not additional license renewals will be requested.

Resource Matching

None.

Implementing Organization

Department of Educational Leadership and  Research Methodology, College of Education with help of OIT as needed,