This page describes research related to the Geoscience Concept Inventory and Critical Thinking.
THE GEOSCIENCE CONCEPT INVENTORY
A valid and reliable assessment instrument designed for diagnosis of alternative conceptions and assessment of learning in entry-level earth science courses
The Geoscience Concept Inventory (GCI) is a multiple-choice assessment instrument for use in the Earth sciences classroom. The GCI v.1.0 consisted of 69 validated questions that could be selected by an instructor to create a customized 15-question GCI subtest for use in their course. These test items cover topics related to general physical geology concepts, as well as underlying fundamental ideas in physics and chemistry, such as gravity and radioactivity, that are integral to understanding the conceptual Earth. Each question has gone through rigorous reliability and validation studies. Over TWENTY colleagues have contributed new questions to the item bank, bringing the number of available, high quality questions to almost 200.
We built the the GCI using the most rigorous methodologies available, including scale development theory, grounded theory, and item response theory (IRT). To ensure inventory validity we incorporated a mixed methods approach using advanced psychometric techniques not commonly used in developing content-specific assessment instruments. We conducted ~75 interviews with college students, collected nearly 1000 open-ended questionnaires, grounded test content in these qualitative data, and piloted test items at over 40 institutions nationwide, with ~5000 student participants.
In brief, the development of the GCI involved interviewing students, collecting open-ended questionnaires, generating test items based upon student responses, soliciting external review of items by both scientists and educators, pilot testing of items, analysis of items via standard factor analysis and item response theory, “Think Aloud” interviews with students during test piloting, revision, re-piloting, and re-analysis of items iteratively. Although time consuming, the resulting statistical rigor of the items on an IRT scale suggest that the methods we have used constitute highly valid practice for assessment test development.
THE CRITICAL THINKING SURVEY
A valid and reliable assessment instrument designed to measure critical thinking ability.
The Critical Thinking Survey is a new assessment tool designed to measure how well an individual can engage in critical thinking. Critical thinking relates to the ability to objectively analyze information without judgement. Critical thinking allows people to set aside biases, form rational opinions, make decisions based on evidence, and building explanations for observations. All disciplines, from science to art, require people to engage in critical thinking.
A subset of questions on this survey were piloted with the general public and students enrolled in a non-majors science courses. Expansion of the question set and piloting with incoming college freshmen is occurring in Summer 2014. Validity and reliability analyses thus far indicate that the test is a good measure of critical thinking ability, with increases in scores in response to instruction directly targeting critical thinking.