Kingsborough Community College Assessment 

he first step of developing a course is first determining the Course Learning Outcomes (CLOs): what students will know and be able to do as a result of completing the course.

In addition to being an important part of the assessment process, CLOs can be a valuable tool for both instructors and students. Clearly-defined CLOs can:

• • Provide clear information to students about a course before they register for it and help the student identify how the course will contribute to their development in the program.

  • Help guide student learning by assisting students in focusing on the most important aspects of the course.

  • Assist instructors in prioritizing content in a course. This can be particularly helpful in unusual circumstances- like the COVID-19 pandemic- in determining how to revise or condense course materials.

Clearly-defined CLOs are objectives that are specific, observable, and measurable. This allows the CLOs to be assessed in an objective way, and for students to know what goals they are aiming for. Let's take, for example, the learning outcome “Students will know the scientific method”. This raises a few questions. First: What will students know about the scientific method? The names of each step? What each step is? How it can be applied to a particular situation? Second: What do we want students to be able to do with this knowledge? Perhaps they just need to identify the different steps, or maybe describe each one? Or explain how it is being used in a particular situation? Further defining this CLO will help. We can do this by using this process:

1. 1. Consider the extent of knowledge students should have. For example: Do students need to have a basic understanding of the topic? Or to explain how it is being used in a study they have read about? Or do they need to be able to use it to design a study of their own?

   2. Choose an appropriate verb to describe the outcome. (Rather than saying “know” or “understand”, which are unspecific and also hard to observe or measure, try a word like “Describe” or “Critique” or “Apply”) ’Bloom’s Taxonomy of Verbs is a great resource for identifying the right verb for the right level of learning.

Using these ideas, we can change “Students will know the scientific method” to “Students will be able to explain how the scientific method is used in psychological research”.

A few other tips for developing CLOs:

1. 1. Try to limit the number of CLOs to between 4 and 6. This may seem counterintuitive considering that the goal is to make CLOs that are specific, but remember that the CLOs are here to illustrate the most important aspects of the course. So, rather than using the CLOs to create an exhaustive list of topics that will be covered in the course, think instead of the key priorities of the course. If there are 4-6 things every single student in the class should know or be able to do, what would they be?

   2. Make sure that the CLOs will work year after year. Though CLOs aren’t set in stone, it’s best to avoid changing them year after year. While developing them, think about what differences the course might have year over year. Or, if the course has already run for a few (or many) semesters, think about what areas tend to change. Are there specific topics that get skipped over or added? Are there new readings or assignments? If any of these changes will mean the course no longer meets a CLO, the CLO may need to be reframed.

   3. Be sure to consider both knowledge and skills. It can be easy to only focus on one, but many classes will include both. Think about what knowledge students will have, but also any tools that they will learn to use or processes that they will develop in order to complete tasks.

   4. Align CLOs with the Program Learning Outcomes (PLOs). Each CLO should ideally align with at least one PLO. This may help identify CLOs that can be dropped- if it doesn’t align with a PLO it may not be right to consider it a CLO. Keep in mind that one purpose of the CLOs is to demonstrate to students how this course will contribute to their progress in the program, not just as a standalone course.

Once the CLOs are developed, the next step is to identify opportunities to assess each one in the course. As a goal, there should be at least one assignment in the class that allows students the opportunity to demonstrate whether or not they have achieved each learning outcome. Opportunities of assignments may include:

• • Midterms or exams (multiple choice or open-ended)

  • Essays

  • Research reports

  • Lab reports

  • Discussion board forums

  • Presentations (group or individual)

  • In-class discussions

  • Quizzes

  • Homework assignments

When determining whether an assignment is appropriate for assessing a particular CLO, it’s important to consider two things:

1. 1. Whether the assignment is formative (assessing student progress as they develop their skills over the course of the semester) or summative (assessing where students end up by the end of the course). Looking at formative assessments (e.g., quizzes, homework problems, or other smaller assignments) is crucial for monitoring student progress and providing feedback; however, for program-level assessment or for end-of-semester reflection, summative assignments may give a better sense of whether students are meeting the course outcomes.

   2. Whether the assignment is targeting the CLO at the appropriate level. Some types of assignments are well-suited for some learning outcomes than others. For example, multiple choice exams are great for measuring whether students have acquired the ability to identify information, but aren’t always great at demonstrating whether a student can apply their knowledge to a new situation.

Rubrics are a helpful tool in conducting assessment and, just like Course Learning Outcomes, provide a great service to both instructors and students. They can:

• • Speed up the grading process by assisting the instructor in determining the grade and organizing feedback

  • Ensure equity of assessment by ensuring that each student is being evaluated using the same metrics

  • Allow the instructor or program to get a big picture idea of students’ strengths and areas for improvement.

  • Communicate expectations to students, ensuring that they understand what is expected of them and can meet all the requirements.

  • Illustrate to external stakeholders how students are assessed and the standards that students are held to.

As indicated above, rubrics can be used to assist in grading student work, but can also be used for assessing SLOs alongside the instructor’s normal grading process. For example, an instructor could create a rubric for an assignment that aligns with the CLOs. Or, a program assessing a PLO or Pathways LO could develop a more in-depth rubric to assess a specific PLO and ask instructors to complete it alongside their normal grading process.

A well-designed rubric consists of a list of criteria (the aspects of the work that are being graded) and for each criterion, a description of each level of quality. In short, a rubric will spell out the specific knowledge or skills the student is expected to demonstrate in the assignment, and what different levels of mastery will look like.

Depending on the LO being assessed, there may be already-created rubrics available, such as the AAC&U VALUE Rubrics, which were developed to assess gen ed criteria. For assessing FLEX Core course, the ’CUNY’s FLEX Core rubric may be helpful.  

To get a complete picture of student learning, it is best to try to incorporate both quantitative and qualitative data.

Quantitative: Using quantitative results can help answer the crucial question, “What percentage of students are achieving the learning outcome?” Though looking at average scores or grades can provide some information, thinking about the distribution of student outcomes can provide more a more valuable and detailed understanding of student performance. Take a moment to think about the difference in these two statements:

• Students received an average score of 78% on the assessment.

• 78% of students received a Meets or Exceeds Expectations rating on the relevant criterion.

With the first statement,  the reader is left wondering whether the average student mastered the LO at a 78% level, or whether some students mastered the LO at a high level and others did not achieve mastery at all. However, with the second the reader can see that a good number of students met the program’s expectations for achievement of the LO, but that 22% of students did not. The first statement may lead to the conclusion that student performance is okay, as students are all receiving passing scores; but the second statement illustrates that there is a part of the student population who need additional support to meet the LO.

Qualitative: Though quantitative results are valuable, they cannot give the full picture on their own. As instructors are assessing student work, they will likely come up with insights that cannot be represented on a rubric, no matter how detailed. Thus, it is important to not limit the analysis of results to just scores on a rubric. Excerpts from student work that highlight strengths or areas of confusion or difficulty or observations from instructors on common questions or issues can help contextualize the quantitative feedback.