Cognitive Taxonomy of Items Based on Capability- Complexity Model Case Study: differential calculi test items

Document Type : Original Article

Authors

1 کارشناس ارشد پژوهشی مرکز تحقیقات، ارزشیابی، اعتبارسنجی و تضمین کیفیت آموزش عالی سازمان سنجش آموزش کشور

2 faculty member of science educational department of tarbiat modares university

Abstract
Developing hard items with complicate calculation algorithms cause to in appropriate trends such as teaching to the test and test preparation strategies in examinees. Having exact educational standards and applying taxonomy models, however, will help to obtain fine evaluation of educational programs, contents and students achievement.  In this manuscript, a small sample of differential calculi multiple choice items test has been studied in behavioral objectives framework which is based on combining the Bloom and Web approaches; the goal was to illustration of capability- complexity approach in taxonomy of test items. Twenty items of a 30 items test battery which administered on a 3409 sample of students has been classified by capability-complexity approach. Outcomes of 3 items have been reported in this article.  Results had shown all the test items measuring procedural knowledge at level three. 92% of items were abstract, and simultaneously recalling multiple calculation algorithms require for responding correctly to each item. Tasks with one calculation algorithm and lower complexity had better statistical parameters such as scalability and their item information functions yielded much more information about examinees capability.

Keywords


سیف، علی‌اکبر. (1385). روش‌های اندازه‌گیری و ارزشیابی آموزشی. تهران: دوران.
Airasian, P. W., & Miranda, H. (2002). The role of assessment in the revised taxonomy. Theory into practice41(4), 249-254.
Anderson, L. W. (Ed.), Krathwohl, D. R. (Ed.), Airasian, P.W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M.C. (2001).A taxonomy for learning, teaching, and assessing: A revision of Bloom's Taxonomy of Educational objectives (complete edition). New York: longman
Bazemore, M., Kramer, M. B., Gallagher, M. P., Englehart, T. P. & Brown, R. A. (2008). The North Carolina Mathematics Tests. Associate State Superintendent Office of Innovation and School Transformation. 6301 Mail Service Center. Raleigh, NC 27699-6301
Bloom, B. S., Hastings, J. T. & Madaus, G. F. (1971). Handbook on the Formative and Summative Evaluation of Student Learning. New York: McGraw-Hill.
Bloom, B. S. (1956). Taxonomy of educational objectives, handbook 1: Cognitive domain. New York: McKay.
Confrey, J. (1990). A review of the research on student conceptions in mathematics, science, and programming. In C. B. Cazden (Ed.), Review of research in education: Vol. 16 (pp. 3–56).Washington, DC: American Educational Research Association.
Florida Department of Education. (2009). Draft fcat Mathematics Test Item Specifications Grades 3–5.The Administrator Office of Assessment Florida Department of Education Tallahassee, Florida 32399-0400
Florida Department of Education. (FCAT). (2008). Cognitive Complexity Classification of Fcat Test Items. Florida Comprehensive Assessment Test (FCAT).
Florida Department of Education. (2012). Cognitive Complexity Classification of the 2012-13 Statewide Assessment Test Items. Office of Assessment available at: http://www.fldoe.org/core/fileparse.php/3/urlt/cognitivecomplexity.pdf
Gagné, E. D, Yekovich, C. W., & Yekovich, F. R. (1993). The cognitive psychology of school learning. New York: Harper Collins.
GED Testing Service. (2012).Webb's Depth of Knowledge: Transiting to the 2014 GED Test. American Council on Education (ACE)
Hess, K. (2005). Introduction to Depth of Knowledge (DOK) - Based on Norman Webb’s Model. Kentucky Department of Education. Center for Assessment/ NCIEA, 2005.
Hess, K., Jones, B. S., Carlock, D., & Walkup, J. R. (2009). Cognitive rigor: Blending the strengths of Bloom’s Taxonomy and Webb’s Depth of Knowledge to enhance classroom-level processes. ERIC: ED517804 NYC Department of Education. Introduction to Depth of Knowledge [Video].
Kentucky Department of Education (2007). Support materials for core content for assessment version 4.1: Mathematics. Retrieved from:
King l., & Razzouk R. (2014). Content Complexity-Florida Standards: Definitions. Florida State University
Krathwohl, D. R. (2002). a Revision of Bloom's Taxonomy: An Overview. Theory into Practice 41 no4 2002. http://coe.ohio-state.edu/.
Linn, R. (1989). Has Item Response Theory Increase the Validity of Achievement test scores? University of Colorado, UCLA Center for Research on Evaluation, Standards, and Student Testing. CSE Technical Report 302
Marzano, R. J. & Kendall, J. S. (2006). The New Taxonomy of Educational Objectives. Publisher: Corwin; 2nd edition (December 20, 2006). ISBN-10: 1412936292
Matthews, B. (2010). Developing Higher Order Thinking Questions Based on Webb’s DOK and FCAT Content Complexity Presented by Brenda Matthews, District Literacy Coach November 22 & 23, 2010
Oosterhof, A., Rohani, F., Sanfilippo, C., Stillwell, P. & Hawkins, K. (2008). The Capabilities-Complexity Model. Center for Advancement of Learning and Assessment. Florida State University, Tallahassee, FL.www.cala.fsu.edu
Raths, J. (2002). Improving instruction. Theory into Practice41(4), 233-237. College of education, The Ohio State University
Rothman, R. (2002). Statewide Testing Programs, In: Encyclopedia of education (Vol. 8) edited by James W Guthrie, New York: Macmillan Reference USA.
Sforza, D. (2014). The Inclusion of Cognitive Complexity: A Content Analysis of New Jersey's Current and Past Intended Curriculum. Seton Hall University Dissertations and Theses (ETDs). Paper 2020.
Thompson, T. (2011). An Analysis of Higher-Order Thinking on Algebra I End-of Course Tests. International Journal for Mathematics Teaching & Learning.
Webb, N. L. (2005). Web Alignment Tool. 24 July 2005. Wisconsin Center of Educational Research. University of Wisconsin-Madison. <http://www.wcer.wisc.edu/WAT/index.aspx>.
Webb, N. L. (2002). Depth-of-knowledge levels for four content areas. Language Arts. March 28.
Webb, N. M., Herman, J. L., & Webb, N. L. (2007). Alignment of mathematics state‐level standards and assessments: The role of reviewer agreement.  Educational Measurement: Issues and Practice26(2), 17-29.
Zimowski, M., Muraki, E., Mislevy, R. J., & Bock, R. D. (2003). BILOG-MG 3: Item analysis and test scoring with binary logistic models. Chicago, IL: Scientific Software.