Summary of research findings from Math Images 1
Overview of Research Reports
Data collected during the Type 1 project is summarized in the three reports that follow. These address the participating students’ experience and learning, description of work to assess open-ended mathematics writing, and findings from think-aloud work to understand how readers of the image pages work with images and text.
Report 1: Participating Undergraduates’ Experience and Learning in the Math Images Project Summer Program
The Math Images Project summer program was created with two purposes in mind: creating content for the Math Images site and providing undergraduate students with an engaging, informative summer experience. This paper reports and evaluates data from their experience based on the analysis of student interviews and surveys and the web pages they produced. Various positive outcomes are reported, such as a self-perceived increase in mathematics writing ability that is supported by the analysis of their writing, and increased confidence in their ability to learn mathematics. Analysis for student writing points towards a need for more pointed feedback to improve the mathematical accuracy of student work. Follow up interviews in the coming years should be conducted to understand the long term impacts the summer program has for participants.
This paper describes the benefits of the summer program for undergraduates. Findings from pre and post surveys and interviews, as well as the students’ work is presented. Surveys and interviews were designed to evaluate students’ interest and background in mathematics and related fields. Students’ pages on the Math Images site were evaluated to show change both over the course of the feedback process and over the summer overall. Two groups of students are described: those who entered the program with a strong, active interest in mathematics or a related field, and those whose interest had diminished during the course of their first year of college. Both groups of students reported having strong interests in mathematics prior to their first year of college.
Students in the active interest group reported that their interest in mathematics or a related field stayed the same or increased over the summer. Students in the diminished interest group expressed that they may minor in mathematics or a related field, and described how the summer had helped them focus their academic interests, even if that focusing was not towards mathematics.
We found that all students reported an increase in their ability to write mathematics, which was supported by the direct evaluations of their pages. All students also reported significantly increased confidence in reading mathematics, talking with professors, or writing mathematics. Students indicated that having multiple sources of feedback, including staff and faculty, were important to their writing process.
Over the course of the feedback process, students’ pages tended to improve most in areas that were more related to the writing style and layout. They were least likely to improve in mathematical accuracy, and so our work indicates methods for providing feedback relating to the mathematics content of the page need improvement.
Overall, our research indicates that the summer program provided students with a positive experience that fostered the development of various skills, particularly those involved with writing mathematics.
In the future, we would like to interview these students at later dates to determine what they eventually decide to major in and ask if they believe the summer program had any impact on their choice of major. It will also be informative to conduct similar studies of students who participate in the program in the future to see if the trends in the present study continue. Following all participants for several years to determine their eventual majors will be essential to understanding the program’s impact on students.
Report 2: Evaluating Undergraduates’ Open-ended Mathematics Writing
The literature on students’ mathematics writing focuses primarily on students’ writing for specific assignments in classroom contexts; to date, a framework for evaluating students’ open-ended mathematics writing has not been presented in the literature. This paper is designed to fill that void.
In the summer of 2009, undergraduate students worked to create content for the Math Images site. In examining their work, a concrete method of analyzing each student written page was needed, and so the Math Image Wiki Page Coding Scheme (MWS) was developed. Based on the work of developing the MWS, this paper describes the challenges in evaluating open-ended mathematics writing.
In the summer of 2009, undergraduate students worked to create content for the Math Images site. In examining their work, a concrete method of analyzing each student written page was needed, and so the Math Image Wiki Page Coding Scheme (MWS) was developed. Based on the work of developing the MWS, this paper describes the challenges in evaluating open-ended mathematics writing. Since the literature on evaluating mathematics is focused on how to evaluate specific classroom assignments, there is a need for a new framework for evaluating mathematics writing that falls outside of this narrow scope. This paper is designed to provide an example of such a framework.
Four types of mathematics writing that can arise from open-ended assignments are identified and described using examples from the Math Images site. The first type of writing is “Proofs and Solutions to Problems,” which can be evaluated using methods familiar to many mathematics teachers. This writing should state the problem and provide a context for the problem. One fundamental evaluation issue is whether or not the piece of writing provides a thorough, rigorous, and accurate solution or proof for a given problem. A particularly good piece of writing in this style provides a thorough, relevant description of the problem, and describes all steps of a solution in clear, concise writing. An example of this style of writing on the Math Images site is the page on the Three Cottages Problem.
The second type is writing on “Central Topics” which are defined to be areas of mathematics that have many branches. Pieces of this type of writing cannot and do not cover the entire topic by definition—they merely outline some of the most important ideas. Writing in this style does not just state definitions, but should also give the reader a sense of the mathematical uses and significance of the topic. As such, this writing also often mirrors the writing style found in mathematics textbooks, and frequently includes examples or applications of the topic. Two examples of writing in this style on the Math Images site are Vector Fields and Probability Distributions.
“Special topics” writing describes areas of mathematics that are highly specific and do not have many branches. These topics are often used as tools for visualizing certain mathematical concepts, for providing insight on a particular problem, or to provide examples of mathematical objects that have certain properties. Writing about special topics should provide a well-elaborated explanation of topic in relation to the few contexts in which it is relevant. Most of the pages on the Math Images site that fall into this category describe fractals, such as the Henon Attractor and Lévy's C-curve.
“Applications of Central Topics” writing focuses on one specific use of a general topic in mathematics. Writing that falls into this category tends to describe applications outside of the realm of pure mathematics or very specific uses within mathematics. Two examples of this on the Math Images site are Newton's Basin and Parabolic Reflector.
These four types of mathematics writing are not mutually exclusive—a single piece of writing can fall into more than one category. An example is the Problem of Apollonius page. At its core, this page provides a history of and solution to the problem, though it also focuses in on a special case of the problem that gives rise to the Apollonian Gasket, a fractal. Thus this page is both a solution to a problem and a page on a special topic.
Each of these four types of writing poses different challenges for evaluation; these challenges are identified and discussed.
The concept of audience is explored as a possible complication in evaluating student mathematics writing. As traditional mathematics writing is written only for a teacher, Web 2.0 tools such as wikis and blogs pose new challenges for evaluation. Mathematics writing posted on the internet must take into account its public nature, and so must its evaluation.
The MWS is presented as a method of evaluating the four different types of mathematics writing with features that take into account the issue of audience. The coding scheme evaluates pages in six different areas: “Context,” “Connectedness,” “Examples, Calculations, Applications and Proofs,” “Mathematical Correctness, Accuracy of Language and Precision of Language,” “Prose Quality,” and “Layout/Aesthetic.” The coding scheme is flexible enough in each area to work with all four types of mathematics writing, and the theme of writing for a public audience is woven throughout all of the areas.
While this coding scheme is designed specifically for mathematics wiki pages, it has the potential to be adapted to other contexts where open-ended mathematics writing and/or mathematics writing for a public audience are evaluated.
As the Math Images Project continues to develop, new pages may arise that fit entirely outside of the proposed four types of mathematics writing. It also would be beneficial to analyze open ended student writing in other contexts, such as senior projects or classroom assignments, and see if the patterns described here hold for a broader range of mathematics writing.
Report 3: Summary Findings Regarding the Reading of Images and Text
This summary reports on preliminary findings from a pilot study and follow-up target study in which undergraduates with a wide range of competence in and attitude towards mathematics did «think alouds» as they read pages that were created this summer. An observer/researcher asked participants to describe their engagement with different types of page content: non-technical text, technical text, equations, images, interactive applets, etc. The pages are designed to provide an intriguing educational experience for readers by explaining the mathematics behind interesting images, and conversely, by using images to help make mathematics content interesting and accessible.
The data from this study suggest that the primary trigger of interest for readers of all levels was textual descriptions of surprising ideas or of connections between mathematical concepts and the world outside of mathematics. The primary value of images appears to be that they play a large role in sustaining reader engagement by providing access to varied representations.
The data further suggest significant differences in level of engagement as a function of the participant's mathematical experience. More experienced students (those who have taken math beyond single-variable calculus as well as those who are majoring or considering majoring in a hard science) were likely to be interested in technical information that developed mathematical arguments and ideas, while less experienced students often limited their engagement to non-technical, less mathematical text.
Participants were recruited for study participation with the understanding that all ranges of math ability or experience were welcome. Each participant was assigned three Math Image pages from a set of five whose content represented a wide range of mathematical difficulty.
In both the pilot and the target study, students were asked to read and think aloud about their reactions to the pages that they were assigned. Following their work with each page, the students were asked questions about their comfort, understanding, and interest for the page. They were also asked to provide a three-sentence summary of its content. After working with all three pages, participants were asked a series of individualized, in-depth open-ended questions that addressed their approach to and perceived needs for working with Math Image pages.
Roles of different types of content in triggering and sustaining interest and engagement. The most effective trigger for interest appeared to be text describing either connections with the world outside of math or ideas that are surprising or counter-intuitive. Such text was especially effective if it was presented near the beginning of a page. For less experienced students, non-technical text describing relationships between mathematical and non-mathematical ideas were especially important. One such student, who was assigned to read a page on the golden ratio, said afterwards that she was most engaged by the page's introduction, which describes how the golden ratio was “once thought to possess inherent beauty.” While reading that section of the page, she had remarked, "The first couple sentences were cool, because beauty is so subjective for me, so knowing there could be mathematical reasons for things being beautiful is cool."
More experienced students also found intriguing ideas or applications to be a more effective trigger for interest than images were, but participants in this subgroup was more drawn to explicitly quantitative ideas. One student commented repeatedly on her “fascination” with the idea that the Koch Snowflake could have infinite perimeter and finite area, later explaining, "I find it really interesting that some series blow up while others converge." This student, however, who was also assigned the page on the golden ratio, was dismissive of comments about the ratio's cultural significance and its connection to beauty. More experienced students were mixed in their reactions to descriptions of applications and connections to the world outside of math. The student quoted above was also assigned the page on the golden ratio. She reacted to the page's description of the “inherent beauty” associated with the ratio, by commenting, “Really? I doubt that!” While some students in this subgroup did not react as negatively to non-technical material as this student did, her experience accurately reflects the general trend.
While text describing engaging ideas was central to generating interest, the inclusion of page content that provided multiple representations, especially images and interactive tools, was critical to sustaining reader engagement. Comparing images, text, and, for more experienced students, equations, allowed readers to generate and test hypotheses. This in turn allowed them to engage with content that they may have otherwise found approachable, and to read pages less passively than they may have with only textual representation. One less experienced student described how images in a page on Koch's Snowflake scaffolded his understanding of equations deriving the fractal's area. He explained, “What the picture did was let me say, 'This one ninth [from one of the equations]? What the heck [does this come from]? Oh, ok, [I see the] one ninth'. It's also nice how you can see the twelve [that appears in another equation]. It's three groups of four here.”
Depth of reader engagement. Data from the think-alouds suggest a strong correlation between readers' background in mathematics and their overall level of engagement with the pages. Less experienced students often engaged with and remembered only non-technical ideas. For example, students in this group who were assigned the Koch Snowflake page tended to focus their summaries on the page's passing references to connections betweens fractals and biology or geology, rather than on the mathematical properties of the snowflake. These participants were especially unlikely to spend much time working with equations. Many of them expressed the sense that they would not be able to understand equations. These predictions often became self-fulfilling as students gave up on equation without first using the same tools, such as rereading or comparing multiple representations, that they would apply to difficult text. Even participants who felt they could understand the equations usually expected that equations would not provide new understanding that they would find meaningful. When these students were asked to read equations they had planned on skipping, this prediction was often accurate.
More experienced students generally engaged with the mathematical thrust of pages. Students in this group summarized the Koch Snowflake page by describing the construction process and/or its infinite perimeter and finite area. The extent to which participants in this subgroup engaged with mathematical details and with equations varied. Some students in this group saw equations as a primary vehicle for understanding the development of the concepts in a page and were only likely to skip equations if they thought that the mathematical details were fairly routine, and thus did not warrant a closer reading. Students who did not make extensive use of equations and other mathematical details were still likely to appreciate the value of understanding the outline of mathematical arguments.
Directions for future researchd
This was a hypothesis-generating study, and as such most of the findings should be confirmed with more controlled testing. For example, a controlled study in which one group of participants reads original pages, while another group reads the same pages with images removed, would be helpful in confirming observations from this study on the role images play in promoting sustained interest.
Finally, the current study does not address reader learning. While understanding what qualities of a page promote learning is of central importance on its own to making the Math Images site educationally valuable, it also affects how findings on interest and engagement should be used and interpreted. Studies of interest for non-mathematical text, for example, have suggested that seductive details (text or especially images that readers may find interesting but that do not contribute to the development of important material) can derail the learning process by grabbing reader attention away from central, relevant ideas. Similarly, while this study has identified features of pages that stimulate reader interest and engagement, their contribution towards learning of content should be evaluated before any conclusions are reached about the ways these features should or should not be generally incorporated into page design.