1. What is assessment and why is it necessary?
Simply put, assessment is ensuring that your students learn what you expect them to learn and making necessary changes to strengthen your program. The key question is how you know if they actually learn these things. For many years, college faculty assumed that if students took the right kinds of courses and made good grades, then they must have learned the material. But many people, especially students, no longer accept such assumptions. You probably don’t either. How many times have you asked yourself, “how did that student ever graduate?” You know the student has not learned what you think a graduate from your program should know, but the student made the necessary grades to pass.
Assessment is also a collaborative effort that begins when faculty in your department decide what things they expect their majors to have learned when they finish your program. But it is also a systematic process of determining how students learn these things, and if not, deciding what changes are necessary to improve their learning.
Assessment at Arkansas Tech enables faculty and staff to answer important questions asked by students, parents, employers, accrediting bodies, and legislators about what our college degree is worth. Everyone benefits when ATU builds a process that helps academic programs and courses systematically improve.
2. If this is assessment, what is it not?
Assessment is not about collecting data, repeatedly testing students, telling faculty what they must teach, or making all faculty conform to one model of collegiate instruction. Assessment is not always quantifiable, nor is it contradictory to the goals of liberal learning that have been valued for centuries. Above all, assessment is not about evaluating the abilities of a single professor or student. In fact, assessment is really not about individuals at all. It is evaluating the larger dimensions of your program and, in many cases, how they relate to the goals of the university as a whole.
3. Who does assessment?
Everybody does assessment in some form or fashion. In higher education, everybody from the administration to the students perform assessment. Our goal is to ensure that assessment is a collaborative effort between faculty, staff, and administration. It should never be the case that someone becomes the “assessment guru” in your department, nor should it the sole responsibility of the department head. Assessment only works properly when as many faculty are involved as possible.
4. How do I begin an assessment plan for our program?
The first thing is for your faculty to determine what they believe students in their program should learn. This differs, obviously, from discipline to discipline, but it can also differ among the various schools of the university. For example, your department may have only one major, and all the faculty teach for that major. Still, your faculty also has unique strengths that separate them from a similar department on another campus. Faculty should always keep their abilities in mind as well as the needs of their discipline when determining what they want their students to learn.
After faculty decide what they want their majors to learn, then a decision must be made about ensuring their goals are met. This involves measurement, but such measurements need not always be quantitative. It is impossible to say what you will learn from these measurements. If students easily meet program goals, perhaps they are too low. If students fail to learn some key information about your discipline, then perhaps courses should be changed. Regardless, goals do matter and meeting those goals matter more than simply going through the steps. Too often, department heads are told that “measuring” and reporting “measurements” are what really matter. Being obsessed with the process of assessment is never a substitute for improving your program.
5. How do I encourage faculty to participate in assessment?
Knowledge of the purpose and goals of assessment usually encourage faculty to participate in assessment. The main way we can encourage is if we respond to the results and show them that their participation is making a difference. The best way department heads can encourage faculty is by setting aside a separate time in each department meeting where assessment issues are discussed. Chances are you do this already and call this point in your meeting by a different name. When faculty discuss their students and their programs, assessment is taking place.
6. What do we assess and what kinds of information do we collect?
Assessment at ATU is designed to answer many questions including:
Basically, you should assess the focus of the program on its stated aims and outcomes. Also, assess the degree to which students succeed in meeting the objectives of the program.
7. What kinds of things make for good assessment measures, and how many do you need?
Third Party observation, self-administered questionnaire, or interviews with students, faculty, and/or staff.
External assessment instruments (Praxis exams, licensure exams, etc.), departmental constructed pre- and post-tests, alumni surveys, placement rate of graduates, employer surveys, portfolio evaluation
Direct measures most often allow the data to be reported in quantifiable terms and is, therefore, easily interpreted. Indeed, one measure may be enough. When indirect measures are utilized, one may need to gather data through several instruments to fully analyze the objective and strengthen one’s conclusions.
8. What is the difference between direct and indirect measurements?
Direct measures assess student performance of identified learning outcomes, such as mastery of a lifelong skill. They require standards of performance. Examples of direct assessments are: pre/post test; course-embedded questions; standardized exams; portfolio evaluation; videotape/audiotape of performance; capstone course evaluation.
Indirect measures assess opinions or thoughts about student knowledge, skills, attitudes, learning experiences, and perceptions. Examples of indirect measures are: student surveys about instruction; focus groups; alumni surveys; employer surveys.
9. Why are course grades not an effective measure of student learning?
This is perhaps the most often asked question about assessment. Course grades do not give us a clear measure on how to improve student learning since many other factors go into why the student may or may not be learning. Faculty often include different criteria when determining course grades. Discussion grades, attendance, and extra credit often skew the final grade. Even the organization of tests will make a difference. For example, an instructor may set as a theme of the course one of the goals for the majors. The same instructor might construct exams whereby students choose which essays to answer. If the student chooses not to answer the question related to the goal, then the test grade does not reflect whether or not the student learned that particular goal.
Having said this, please remember that some course grades are very effective measures of student learning, especially capstone courses and internships. That is, when an entire course is designed to determine how much a graduate of a program has learned, then the final course grade is acceptable.
Grades are still essential for determining student learning. Faculty in a department may wish to use grades earned on specific tests or on questions/exercises within tests as well as other kinds of grades earned within a course. This will require faculty to be alert to the key goals of their program so that they know what information will be useful to determining if students are meeting those goals.
10. Must all of our goals be measurable and quantifiable? I thought we were a liberal arts university?
Measurement can be either quantitative or qualitative. Neither is more important than the other. Either may be employed in assessment. The assessment process merely helps us in continuously improving our programs. Constant vigilance in examining our goals and our methods of reaching them, and allowing faculty discourse, as well as student input, regarding our program’s direction is a healthy and appropriate endeavor for a liberal arts university. The assessment plan is an example of faculty governance which ensures that program decisions are research-based rather than arbitrary.
11. Can’t we just resubmit the same plan each year?
Each year the departments benefit from assessment in some way, whether it is from changes made as a result of their own assessments or from those made as a result of a past year’s assessments. Some changes can be made immediately. Other changes will take more time. The short answer to the question is, “no, of course not.” Nobody denies that good assessment can reveal consistent trends. If faculty are tempted to produce the same report each year with only minor modifications, then it could mean that you and your faculty are overly concerned with the process of assessment rather than using it in a meaningful way. Always remember, your assessment report is not your goal. You should be ensuring students learn what you want them to learn.
12. Should we consistently meet our entire student learning goals each year?
Maybe. However, if every goal is met consistently from year to year, it may mean that the goals are set too low.
Purposefully setting goals that are easily met defeats the purpose for having an Assessment Plan. On the other hand, you do not want to set goals that are impossible to reach. It is expected that sometimes you will meet some or all of your goals and sometimes you will not meet all of them.
13. How does our program assessment plan relate to the four column report provided to the Assessment Committee each October 1?
The 4 column template is characterized by 2 things: First it is a temporary effort to provide a framework by which programs can be assessed. Second, it is merely a summary of a particular program’s assessment. The template should never replace a comprehensive assessment effort, nor should department heads and faculty solely focus their energies on completing the form. The form was designed to collect a sample of your efforts, not everything that you do.
14. What assessment resources are offered to us by the office of institutional research and career services?
Institutional Research gives assessment, retention, enrollment statistics, student semester credit hours statistics, degrees awarded, demographics (common data), FERPA info., and faculty statistics.
Career Services has most of the records about ATU students’ employment.
15. What resources can the University Assessment Committee offer to us?
Knowledge, experience, and examples of assessment. We also offer annual grants of up to $5,000 to cover innovative assessment projects other than what you normally do.
16. What if we don’t have time for assessment?
Everybody has time for assessment. Assessment is essential for a department to stay focused on their mission and the progress of their program. It is necessary to make assessment a top priority in order for a department to know if they are consistently educating each culture group of students correctly.
17. Where does general education fit in all this?
Currently, the general education goals are being reviewed by each department’s faculty members. The Curriculum Committee is charged with a review of the suggested revisions. At this time, we’re unsure if significant changes will be made.
The general education goals are presently assessed with the CAAP exam. A more extensive assessment may be needed in the future, but those decisions won’t be made until after the current review has been completed.
Since every student does not take the same group of general educations courses, it is necessary that every program determine when, where, and how each ATU general education goal is being met by their own students in their own courses.