Dr. Julie Libarkin, Dept. of Geological Sciences, Ohio University

Over 5000 students from more than fifty universities and colleges nationwide participated in a study aimed at developing an assessment instrument for entry-level geoscience courses. Short, open-ended questionnaires from 1000 students and interviews with 200 students provided insight into ideas about the Earth held by entry-level students, and these ideas drove the development of test questions and answers. The test was created in two phases: a small, 29-item test was piloted in Fall 2002 and evaluated using item response theory and qualitative validation techniques. Based upon the success of this initial testing, a second set of 45 questions was created for piloting in Fall 2003. Our experiences with the Geoscience Concept Test indicate that 1) assessment questions created in direct response to student interviews are particularly useful in large-scale testing of student ideas; 2) a variety of qualitative and quantitative measures should be used when creating assessment instruments to ensure validity of the test design and application; and 3) many alternative ideas about the Earth are difficult to modify, as evidenced by small change between pre- and post-test scores nationwide.