# Research plans

Two sources of project evaluation are planed: consultation with the Advisory Board and Evaluation Research that extends that conducted as part of CCLI I funding. Advisory Board members for this project will provide consultation and feedback on project goals, and a summary report of meetings with them will be included in reporting about the project to NSF.

**Evaluation Research**

Evaluation Research for this project will be conducted by K. Ann Renninger, Professor and Chair of the Educational Studies Department at Swarthmore College. There are two foci of this work: First, evaluation of student learning from work on image pages (writing math, programming, and development of curricular materials) and ability to work with feedback will be studied in relation to student interest in mathematics, mathematical beliefs, and feelings of self-efficacy about working with mathematics content. In these studies, archived information about student work on the project will be analyzed to inform feedback practices. Second, study of readers’ work with the mathematics and the images of the Math Images pages and what they learn will be undertaken. Data from this research will be used to inform: (a) the refinement and development of a rubric for Math Images page development, and (b) information provided to instructors and project personnel about students’ needs as learners, in particular information about the forms of feedback that support development of their thinking about mathematics.

**Expected measurable outcomes**

Evaluation research will provide both formative and summative feedback to the project about student learning from working with Math Images II projects and users’ reading of Math Images pages. Data about students will describe the value of scaling and extending undergraduate participation in the collaborative, inquiry-oriented, project-based context of working with Math Images II. Studies of students’ learning will focus on their work with feedback, learning of mathematics from work with Math Images pages, and learning from writing mathematics. These studies will assess whether project participation enables participants to:

- develop and/or deepen their interest for mathematics;

- increase in their feelings of self-efficacy about working with mathematics;

- develop mathematical beliefs that are process-oriented;

- broaden and deepen their knowledge of mathematics content;

- strengthen their ability to communicate and work with mathematics;

- deepen their understanding of math concepts with which they work as part of page development, programming, and/or lesson construction;

- develop their abilities to write mathematics;

- develop their abilities to ask questions and make use of feedback; and

- persevere to work on mathematics that is challenging.

In addition, data from the evaluation research will inform understanding of how users work with the mathematics and images, information that will be used in the development of a rubric for guiding Math Images page construction, as well as in assessment of changes in students’ abilities based on project participation. Two studies of reader behavior (Reading Study 1 and Reading Study 2) will assess: What features of pages attract readers’ attention and how does the context under which pages are read (e.g. as stand-alone material required for a course, as supplemental course reading for topics covered in other lectures as texts, as recreation) affect reader behavior? Questions about reader learning include: What do readers learn from working with pages (e.g. basic definitions understood in layman's terms, sophisticated lines of mathematical reasoning and their relationship to particular problems posed by a page)? How do different types of text (e.g. motivational or historical context for page content, plain English overviews preceding mathematical derivations, detailed explanations of images) or images (e.g. interactive applets exploring major principles, captioned images explaining algebraic manipulation) differ in their facilitation of understanding page content? Differences among readers based on demographics, motivation and experience with mathematics will be tracked for each question.

Data will also be collected for a study of longer-term effects to be undertaken as part of a CCLI Type 3 project. Questions to be addressed include whether work with Math Images pages affects the academic course selections and performance of participating students over time, e.g., Do students’ grades in project-related courses change following project participation? Do students take different courses than they originally planned to take prior to their work with the project? What are the demographic and motivational characteristics of students who continue to be involved in responding to Math Images wiki discussions of pages after their project.