The impact of diagnosing teacher-made erroneous solutions, with the aid of on-line principle based feedback, on preparation for future learning

so conceptual understanding will indeed develop: Elicitation of students’ ideas, Adding scientifically acceptable or non-acceptable ideas through discussions with peers, teachers and textual sources, Developing Criteria to distinguish ideas and Sorting out one's ideas. The materialization of KI processes depends upon student's inclination to self-repair (Chi, 2000), i.e., to recognize and resolve conflicts between their possibly flawed mental model and the scientifically acceptable model. In the terminology suggested by Schwartz, Bransford and Sears (2005) having such inclination and ability to self-repair optimally prepares students for future learning as they have developed along two dimensions: efficiency (i.e. retrieving and applying scientific concepts and principles to solve accurately physics problems) as well as innovation (i.e. rearranging one's thinking to handle new types of problems or information). This study examines the hypothesis that a yearlong trouble-shooting (TS) intervention would enhance students' inclination and ability to self-repair (i.e. development along the innovation dimension), and as a result, improves their conceptual understanding (i.e. development along the efficiency dimension). The TS intervention was also compared to a more traditional self-diagnosis1 (SD) intervention. The TS intervention comprised of several computerized TS activities that guide students to diagnose fictitious erroneous solutions with principle based prompts, and provide feedback on the diagnosis in shape of exemplary diagnosis that students are asked to compare to their own diagnosis (Yerushalmi et al., 2013). Similarly, the SD interventions comprised of several computerized activities in which students solve problem (similar to the problem dealt with in the corresponding TS activities), and are asked to diagnose their solution in light of an exemplary solution. The TS and the SD activities considered conceptual problems reported in the literature as eliciting conceptual difficulties in geometrical optics. 1 Self Diagnosis Activity (SD): First, students perform a problem solving task; requiring them to solve a physics problem, then they are presented with feedback in shape of exemplary solution. Students are required to process the feedback and analyze their work in light of it. iii The study maintained ecological validity, being carried out in 10th grade classrooms that operate under the time constraints of teachers and students in high-stakes testing settings. The sample analyzed included 67 students. One class that followed the troubleshooting (TS) intervention included 30 students and the other class followed the selfdiagnosis (SD) intervention included 37 students. Both classes were taught by the same teacher in the same school. Students worked in pairs on eight computerized activities covering four consecutive topics in geometrical optics, allowing examining the effect of extended engagement on the learning outcomes. The comparison between TS and SD intervention was examined be means of posttest that considered three aspects: a) diagnostic performance; b) conceptual understanding; and c) self-repair. Students performance was normalized by means of a pre-test in which students performed a troubleshooting task (diagnosing fictitious erroneous solution) followed by a comparison stage between students' own diagnosis and an exemplary diagnosis. The detailed analysis of change processes along the yearlong TS intervention considered two aspects: a) The development of diagnostic performance; and b) the materialization of knowledge integration processes. Conceptual understanding (efficiency) was assessed by means of the direct transfer methodology (Schwartz and Martin, 2004) - performance on problems that are isomorphic to the intervention problems that were administered twice, after the 4th and 8 th activities. Self-repair (i.e. innovation) was assessed in the post test by means of the double transfer methodology; students solving a core problem, than study a learning resource - diagnosing fictitious erroneous solution to the problem aided by exemplary solution and apply their learning to solve transfer problems. The main findings of the dissertation are: The effect of the TS vs. SD intervention on diagnostic performance We examined the gap between students natural diagnostic performance (diagnosing fictitious erroneous solution), measured in the pre-test, and their diagnosis performance in the post-test (diagnosing fictitious erroneous solution aided by an exemplary solution). It was found that the trouble-shooting intervention group improved significantly in their diagnostic performance, while the self-diagnosis group did not. In particular, the trouble-shooting group improved in explaining the nature of the difference between the fictitious student view and the scientifically accepted view. The iv post-test took place in the school year following the year that the intervention took place, suggesting that the impact of the trouble-shooting intervention was maintained over time. The effect of the TS vs. SD activities on conceptual understanding The effect on conceptual understanding (ie. Efficiency) was examined by means of direct transfer problems administered twice; after the 4th and 8th activity. It was found that Students' performance in the transfer problems improved along the year in both groups, with no significant difference between the groups. Namely, both interventions had similar effect on efficiency. Comparing the effect of the TS vs. SD intervention on self-repair The effect on self-repair (ie. Innovation) was examined by means of double transfer methodology within the post-test. Students solved a physics problem (core problem) on their own, than studied a learning resource - diagnosing an erroneous solution aided by exemplary solution. Self-repair was examined by means of subsequent transfer problems. It was expected that students' self-repair performance would also correlate with their diagnostic performance. For both groups, there was no evidence of self-repair process. This finding is explained in the way the post-test was implemented: The teacher had his students enter the post-test very knowledgeable about the topic, in contrast to the post test design. Thus they performed well on the core problem, and there was little room for improvement. As a result in both intervention groups the post-test could not and did not detect Innovation: using the learning resource (diagnostic task aided by an exemplary diagnosis) to refine conceptual understanding. The change processes along the yearlong TS intervention: a) The development of diagnostic performance As mentioned earlier, the trouble-shooting intervention consisted of eight activities as a mean to develop the students’ diagnosis performance. The trouble-shooting group engaged in three types of diagnosis activities: Within the diagnosis stage in the pre-test (diagnosing an erroneous solution), eight trouble-shooting activities (two activities per topic), and in post-test diagnosis (diagnosing an erroneous solution aided by an exemplary solution). It was found that the intervention had an accumulative effect. Students’ natural diagnostic performance prior to the intervention was found to be low (measured prior to the intervention in the diagnostic stage in the pre-test). In particular, v students rarely provided explanation to the nature of the discrepancy between misused of scientific concepts and principles within the mistaken solution and the scientifically accepted ones. The significant changes in students’ diagnosis performance developed gradually: Students’ diagnosis performance in the first activities (1st and 3rd activity) did not differ significantly from their natural performance in the diagnostic stage in pre-test. Students’ diagnosis performance improved significantly towards the end of the yearlong intervention (7th activity out of eight, as well as the diagnostic stage in the post-test). While the development in diagnosis capability was slow and required exposure to several activities, it was maintained after a long time (the post-test took place in the following school year), manifesting significant improvement in students diagnostic performance with respect to their initial performance. In particular, they improved in explaining the nature of the difference between the fictitious student view and the scientifically accepted view. b) The materialization of knowledge integration processes while students engage in the trouble-shooting activities. As mentioned earlier, trouble-shooting activity stages were designed to trigger Knowledge Integration (KI) processes: Elicitation of students’ ideas, Adding scientifically acceptable or non-acceptable ideas, Developing Criteria to distinguish ideas and Sorting out one's ideas (Linn & Eylon, 2006). All stages of the troubleshooting activity elicited students' ideas and added information. The diagnosis stage (2nd stage) served to identify students' diagnostic performance level when entering the activity in terms of their ability to develop criteria: Low - students who merely pointed to mistake but did not explain it; Moderate – those who also presented a criteria they have developed - articulated the principle that was misused, high – Students who not only presented the principle, but also explained how they distinguish correct from an incorrect use of the principle/concept. The mirroring and the comparison stages (3rd and 4th stages) provide an opportunity to materialize the processes of developing criteria and sorting-out. In the mirroring stage of the TS activity I perceived materialization of the KI process of developing criteria if students' recognize the principle/concept that according to the exemplary diagnosis was not used, or misapplied within the erroneous solution, and articulate the explanation vi provided in the exemplary diagnosis for the difference between the students view and the scientifically accepted view. In the comparison stage of the TS activity (comparing the students' diagnosis and the exemplary diagnosis) I perceived materialization of the KI process of developing criteria and sorting out when the students’ evaluation to their diagnosis matched the researcher evaluation. We identified four profiles characterizing the KI processes that materialized in students' responses in the mirroring and comparison stages; (1: materialization of all the expected KI process in both the mirroring and comparison stages; 2: materialization of the KI process only in the mirroring stage; 3: partial materialization of the KI process in the mirroring stage and complete materialization in the comparison stage; 4: partial materialization of the KI processes in the mirroring stage). It was found that the percentage of students with low diagnostic level has decreased along the intervention. It was also found that in the end of the yearlong intervention the majority of students populated mainly the high level profiles 1 & 2 in contrast with the starting point where they populated all four profiles. These findings align with the findings regarding the development of diagnostic performance during the troubleshooting intervention. It was found that regardless of students' diagnostic level when entering the activity, students who were sorted in profiles 1 and 2 succeeded in solving the subsequent transfer problems. Namely, the extent to which the "developing criteria" materialized in the mirroring stage was later expressed in students' performance on the transfer problems. Therefore, the TS mirroring and comparison stages are essential in order to guide students to develop criteria that allow them to distinguish between scientifically acceptable and non-acceptable ideas. The activity design indeed achieves its goals, encouraging development of similarly designed activities in additional physics topics. When implementing these activities in classrooms one needs to consider the slow development in student's diagnosis capability, implying that there is a need to engage students in several trouble-shooting activities, where the mirroring stage occurs again and again, in order for their action plans (i.e. the strategic steps that they perceive as needed to accomplish a task Wertsch (1984)) to approach the expert's one.