Overview:
This Spring 2022 semester, the Computer Science Assessment Committee developed rubrics to support assessment of their department’s program learning outcomes. Mrs. Rebecca Cunningham, Assistant Professor and Undergraduate Program Director, reached out to the Office of Assessment & Institutional Effectiveness for assistance in creating "a sustainable assessment process that can be replicated (and refined) by the other programs in the department".
Background:
The computer science program is accredited by the Computing Accreditation Commission of ABET. Recently, the program went through reaccreditation and received feedback concerning their data collection and assessment process. The existing process included informal data collection and analysis. Creating an assessment committee and formalized process was necessary.
New Process:
Outcomes:
To support a stronger assessment process for the computer science program, and other departmental programs seeking ABET accreditation in the near future, an assessment committee was formed. The committee reviewed the ABET program learning outcomes, which are shared across computing programs, and determined to also adopt one program-specific outcome for each computing program.
Program Learning Outcomes for the Computer Science:
Measurement:
The committee then identified three subcomponents (ABET refers to these as “performance indicators”) of each learning outcome with five levels of achievement (Exemplary [4], Satisfactory [3], Developing [2], Unsatisfactory [1], and Not Present [0]). These were given descriptors, as shown in the Figure 1.
Figure 1 Learning Outcome 1 - Analyze a complex computing problem and apply principles of computing and other relevant disciplines to identify solution.
Data Collection:
Mrs. Cunningham recognized the need for a sustainable method to collect rubric data that could easily be replicated each semester and by all computing programs. The OAIE assisted by creating scoring rubrics in an Excel spreadsheet that features a tally sheet of all student’s results by outcome, subcomponent, overall score, and cohort averages. Figure 2 shows the Learning Outcome 1 scoring rubric and Figure 3 shows the outcome criteria and results display.
Figure 2 Outcome 1 Scoring Rubric in Excel format
Figure 3 Outcome Criteria and Results example
The committee crafted a curriculum map to identify the classes, outcomes, and levels (introductory, reinforced, mastery), that each outcome shall be assessed. See Figure 4.
Figure 4 CS Curriculum Map
A Spring 2022 semester table identifies courses, faculty, outcomes, and types of activities for assessment. See Figure 5.
Figure 5 Excerpt from Spring 2022 CS Assessment Schedule
Mrs. Cunningham created a video walk-through of the new process including the file collection process (rubrics and evidence) and a demonstration of how the rubrics should be used. Each rubric spreadsheet creates a tally of that course section’s results, broken down by outcome and subcomponents.
Data Analysis, Interpretation, and Planning for Improvement
All course section’s rubric data will be combined into one master spreadsheet which will enable data analysis by outcome, subcomponent, student major, course number, section, semester, etcetera. The committee will monitor the data collection process and review results at the end of each semester.
To establish a performance baseline, the computer science program will continue data collection on all six program learning outcomes through Fall 2023. At that point, they will analyze all the data, make plans for improvements to the two lowest scoring outcomes, and set a new schedule that rotates learning outcomes every two or three years.
Committee members are interested in what the findings may show and working together to determine if changes are needed to improve student learning.
