1
2
3
4
DOK is... Focused on ways in which students interact with content standards and assessment items and tasks. It focuses on how deeply a student has to know the content in order to respond. DOK is determined by looking at the standard or the assessment item, not student work or the student s ability. It describes the kind of thinking involved in the task, not whether it will be completed correctly. DOK is reflected in the standard and/or the assessment item or task. In other words, does the cognitive demand of the assessment items match the stated learning expectations? DOK guides assessment item development and increases rigor on assessments by illustrating the complexity of mental processing that is needed to answer items or solve tasks. If students cannot perform at the standard s DOK level, they have not mastered the standard. 5
Bloom s Taxonomy looks at the activities and the questioning used in a lesson, in other words, the type of thinking. DOK refers to how you assess students and the work they are doing. DOK is not about difficulty, it s about complexity. Like Bloom s Taxonomy, Webb's theories are based on research about student thinking to extend student learning. While Bloom's Taxonomy relies on the verb, Webb's DOK extends beyond the verb and into the thinking process to expand student learning. Please review the slide to see how the levels of Bloom s Taxonomy correlate with the four levels of Webb s DOK. 6
Webb s Depth of Knowledge explores how deeply students have to know and understand content, called complexity of thinking, in order to be successful on an item or task. Some general distinctions of the four levels of DOK include Level 1, Recall/Reproduction, requiring recall or recognition of facts or using simple skills and abilities. Level 2, Skills and Concepts, requiring engagement in some mental processing beyond recalling or reproducing a response. Level 3, Strategic Thinking, requiring complex and abstract thinking through reasoning, analyzing, evaluating, and/or synthesizing information. Level 4, Extended Thinking, requiring the use of higher order thinking skills and multiple sources. The distinguishing factor for DOK Level 4 would be the use of an assessment performance task which uses a product to show that the cognitive demands have been met. 7
DOK requires looking at the assessment item and standard in order to determine the level. In other words, how deeply do you have to know the content to be successful. DOK Level 1, Requires Verbatim recall, Simple understanding of a word or phrase, Shallow understanding of text, Applying a basic mathematical formula; and Remembrance of who, what, when, and where DOK Level 2, Entails applying skills and concepts, Explanations of how and why, Comprehending and processing portions of a text, while main ideas are stressed. DOK Level 2 May apply DOK level 1 skills, but not in a complex way. DOK Level 3, Cognitive reasoning and Justifying how and why from DOK level 2 is a must for this level, along with Going beyond the text to explain, generalize and connect ideas. Since DOK Level 3 Proposes and evaluates solutions, more than one correct response is often possible. At DOK Level 4, the Cognitive demands should be high and the work should be very complex. It calls for even more complex reasoning than DOK Level 3 along with the addition of planning, Investigating, and/or developing that will most likely require an Extended period of time. Tasks and items include Taking material from one content area and applying it to another. DOK Level 4 calls for solving New situations with deep awareness and great creativity. 8
It is time for a reflection on Defining DOK. Based on our learning thus far, please answer the following questions on page 2 of the Participant s Guide: 9
The purpose at DOK Level 1 is to ensure students can solve a basic problem or task. When learning a concept at this level, students should be able to identify facts, terms, properties of objects, and other attributes. DOK Level 1 involves recalling simple routine procedures and formulas. Students either know the answer or not. DOK Level 1 activities require one right answer and one step. 10
Please review the DOK Level 1 activities on the slide. These activities are DOK Level 1 because they involve recall and the response is automatic. DOK Level 1 activities require students to demonstrate a rote response, follow a set of procedures, or perform simple calculations. Answering a DOK Level 1 item can involve following a well known formula. Simple skills and abilities characterize DOK Level 1. 11
The content knowledge or process of DOK Level 2 is more complex than in DOK Level 1. The difference between DOK Level 1 and DOK Level 2 is DOK Level 2 allows students to act on information in some method and understand the relationship between pieces of information. DOK Level 2 is Moving beyond simple definitions. Students at DOK Level 2 find the main idea, infer, and predict outcomes based on verbal and non-verbal clues. A DOK Level 2 item requires students to explain why something might happen. DOK Level 2 involves formulating routine problems and deciding how to solve them. This often requires comprehension and application where one concept is applied and then a decision is made before going on and applying a second concept. DOK Level 2 goes beyond basic recall by acting on information in some method and understanding the relationship between pieces of information. For instance, students will interpret and organize data with simple graphs. With DOK Level 2, there is one right answer and usually two or more steps are involved with decision points along the way. 12
Please review the DOK Level 2 activities on the slide. These activities are DOK Level 2 because they are more complex and require students to engage in mental processing and reasoning beyond a habitual response. DOK Level 2 includes the engagement of processing beyond recalling or reproducing a response. Students decide how to approach the problem with more than one mental or cognitive process or step. When asked to summarize, students should determine what is relevant and discern fact from opinion with DOK Level 2. 13
An assessment item that has more than one possible answer and requires students to justify the response they give would most likely be a DOK Level 3. Complex, strategic thinking becomes the focus at DOK Level 3. DOK Level 3 analyzes similarities and differences in issues and problems; explains, and generalizes or connects events or ideas using supporting evidence from a text or other sources. DOK Level 3 relies more on applying knowledge or skills in real world situations with predictable outcomes. DOK Level 3 requires students to move toward a deeper level of understanding beyond describing how and why to justifying the how and why through application and evidence. There might be several right answers and multiple methods for arriving at a DOK Level 3 answer. The complexity does not result only from the fact that there could be multiple answers, but because the multi-step task usually requires students to explain their thinking. 14
Please review the DOK Level 3 activities on the slide. DOK Level 3 activities necessitate higher cognitive demands than the previous two levels. The cognitive demands at DOK Level 3 are complex and abstract. At DOK Level 3 students are providing evidentiary support and reasoning for conclusions they draw. DOK Level 3 requires deep understanding as exhibited through planning and more demanding cognitive reasoning. 15
DOK Level 4 combines the complex reasoning of DOK Level 3 with the addition of planning, discovery, and development. DOK Level 4 consists of complex authentic problems with a high cognitive demand. At DOK Level 4, students conduct investigations, identify a real-world problem, plan a course of action, enact that plan, and make decisions based on collected data. DOK Level 4, therefore, usually requires strategic thinking over an extended period of time due to the research needed. Please note, extending the length of an activity does not necessarily create rigor, and extended time alone is not the distinguishing factor of a DOK Level 4 item. Connecting multiple content areas across disciplines and sources to create unique solutions and associate ideas refers to DOK Level 4. DOK Level 4 performance tasks get students to analyze and synthesize information from multiple sources and examine and explain alternative perspectives. Therefore, there are usually multiple solutions and steps. 16
Please review the DOK Level 4 activities on the slide. DOK Level 4 includes those tasks in which students must demonstrate reasoning, planning, and developing connections within and beyond a content area. This activity requires complex reasoning, planning, and thinking generally over extended periods of time for an investigation. Students are expected to make connections and select or devise one approach among many alternatives on how the situation can be solved. 17
DOK Level 3 and DOK Level 4 both have complex items that require reasoning, yet there are distinct differences in each level. An item or task that requires students to apply problem-solving criteria based on one source in order to determine the best solution would be a DOK Level 3. DOK Level 3 items and tasks allow for the understanding of one source, text, data set, or investigation. The addition of an investigation to gather data from multiple sources to use as evidence of the problem, and adding an implementation plan would make this same task a DOK Level 4. DOK Level 4 uses multiple texts usually across multiple concepts or disciplines to create a final solution or product. Remember, the investigation of multiple sources requires extended time, which is often a distinguishing factor of DOK Level 4. 18
It is time to reflect on Describing the Characteristics of DOK. Based on our learning so far, please answer the following questions on page 2 of the Participant s Guide. 19
It is important not to confuse DOK with difficulty. Difficulty is a reference to how complicated an item seems to be and how many students can answer a question correctly. DOK is about intended outcome, not difficulty. The intended student learning outcome, such as adding or subtracting numbers, is the focus of the depth of understanding or DOK. DOK is a reference to the complexity of mental processing, cognitive complexity, that must occur to answer a question, perform a task, or generate a product. 20
DOK provides a measurement of both the rigor and cognitive complexity, or thinking, required by an assessment item or task, or a standard. For instance, Many students would consider a Difficult question to be as follows: Name all of the presidents of the United States beginning with George Washington. Although some would say this is a hard question, this question is a DOK Level 1 item because the cognitive demand only calls for recall and reproduction. A Complex thinking question for most students would be as follows: After a study on the American Revolution, research & explain reasons why George Washington did not want to be king of the United States AND why he agreed to be a candidate for president. Be sure to answer ALL parts of the task. Use details from your references to support your answer. Answer with complete sentences and use correct punctuation and grammar. This question is a DOK Level 4 because it calls for complex reasoning, detailed research and investigation, and synthesizing data from several sources. 21
Let s do an activity to illustrate how DOK is about cognitive complexity, and not difficulty. As you recall, difficulty refers to how many students can answer a question correctly. Cognitive complexity is the mental processing required to complete an item or task. Based on this activity, it is clear how DOK focuses on complexity of thinking, and not how complicated an item seems to be. Note: Pneumo/noultra/microscopic/sili/covolcano/co/niosis is a 45-letter long word and it is one of the longest words found in dictionaries. According to the eighth edition of Webster dictionary, it means, pneumoconiosis disease caused by inhaling small particles of quartzite. This is the scientific name for a coal miner s disease, which is particularly caused by breathing in particles of siliceous volcanic dust. It is the lung disease that miners in Africa came down with from getting silicon silvers in their lungs. 22
The context of the item or task must be considered to determine the DOK level, not just looking at the verb chosen. 23
DOK is not verb dependent. It is based on the cognitive complexity of the standard. Each of the sample content standards on this slide use the same verb: Identify. Look at the different expectations at each DOK level. Make a mental note of the differences in each standard. The first example, DOK Level 1, requires students to simply recall information. In the second example, DOK Level 2, students must process information in order to determine the facts. This requires cognitive processing beyond basic recall. In the third example, DOK Level 3, students are required to analyze and justify by providing supporting evidence. This means they must first have a deep understanding of the argument and then be able to analyze it for a specific purpose. In the final example, DOK Level 4, students must take material from one content area and apply it to another. They are using multiple texts to identify interrelationships. 24
Review each of the statements individually. Assign each item a DOK Level of 1, 2, or 3. Record answers on Page 3 of the Participant s Guide. 25
Let s review the answers for the DOK and Context of Verb Activity. Examining the provided sample items the DOK level for the items are: 1. DOK Level 3 for the first item 2. DOK Level 1 for the second item 3. DOK Level 2 for the third item 26
It is time for a reflection on cognitive complexity and DOK. Based on our learning so far, please answer the following questions on page 4 of the Participant s Guide: 27
DOK levels can be cumulative because each level builds on the previous level. With DOK, the highest level of cognitive processing for a standard is called the ceiling. The ceiling is also the highest DOK level at which a standard can be assessed for item development. It represents the highest level of knowledge that students are expected to demonstrate on an assessment. Review the sample standard and DOK information on the slide. Determining the ceiling helps to ensure the learning goes as deep as the standard needs it to be. For instance, the standard on the slide is an example of a DOK Level 3 ceiling. The expectation expressed in this standard is that students will not only solve a two-step linear equation, a well-known formula, but will verify the results and interpret the solution. This will require students to do some fairly complex reasoning in order to interpret the solution, depending on the context. If students were only required to solve linear equations and verify their solutions, then the expectation would be DOK Level 2. Please note, based on the SLO expectations, assessment items should be written to the highest DOK level provided in the standard. 28
Let s complete an activity identifying the ceiling, the highest level which a standard can be assessed for item development. 29
Let s review the answers for the DOK Ceiling Activity. Examining the provided sample standard: DOK Level 1 is Identify various forms of government; DOK Level 2 is Compare and contrast various forms of government; and DOK Level 3 is Evaluate forms of government based on criteria. Therefore, the standard s ceiling is DOK Level 3. Remember, assessment items should be written to the highest DOK level provided in the standard. 30
Depth of Knowledge supports and enhances the SLO assessment development and implementation processes at the LEA because: DOK measures rigor and cognitive complexity. DOK requires looking at the standard, assessment item, or task in order to determine the DOK level. In order to work at the higher levels, students must have first mastered content at the previous levels. A greater DOK level requires greater conceptual understanding and cognitive processing by the students. Therefore, on average, students who reach greater DOK levels regularly should have increased student achievement. DOK ensures standard and level of complexity required by that standard match the assessment items. For a quality assessment, the DOK level of the item should be at or above the DOK level of the standard. DOK is an alignment method developed to examine the consistency between the cognitive demands of standards and the cognitive demands of assessments. If students cannot perform at the standard s DOK level, they have not mastered the standard. 31
It is time to reflect on how DOK and SLOs are connected. Based on our learning so far, please answer the following questions on page 4 of the Participant s Guide: Define DOK ceiling and its importance. Why is it important to measure student performance at the DOK ceiling level? Why is DOK an essential part of the SLO assessment development process? 32
The Teacher and Leader Effectiveness Division at the Georgia Department of Education aims to provide resources and support for teachers and leaders in the successful implementation of SLOs. All resources are designed to develop knowledge, skills, and behaviors to improve teacher and principal practice and effectiveness leading to increased student achievement. 33
34