When Arkansas Tech commenced the HLC reaccreditation process in the Fall of 2018, departments across campus began building their evidence documents. The Math Department at Arkansas Tech was no exception. Kristi Spittler-Brown assisted in building a 70-page department assessment evidence document, which covered the overall mathematics success of the 2018-2019 academic year for the Russellville campus, Ozark campus, and Concurrent enrollers, as well as pre/post analysis, and remedial and gen-ed comparative analysis.
The depth of assessment the Math Department has gone through is what is noticeably impressive. Across Russellville, Ozark, and concurrent enrollment, the Math Department has broken down assessment across several factors, including:
The Math Department also implemented a Pre vs Post test for MATH 1113, MATH 1003, and MATH 0903. Across these courses, summary statistics covering mean, standard deviation, confidence interval, and many other statistics provided insight into their assessment work. Additionally, each test was broken down into a percentage of correct and incorrect responses, listing what each question covered. The report concluded with success rates for Remedial and General Education Mathematics courses from Fall 2016-2019 and recommendation for improvements.
Closing the Loop:
Although 2020 had many challenges due to the pandemic, the Math department was still able to make a few changes based on assessment data. First, the post test data was gathered and assessed, looking specifically at the performance of each exam question. The Math Department Curriculum Committee examined these questions, rewriting them or making changes in education strategy to ensure a more consistent exam performance.
Next, there were topics that students still struggled with at the end of the semester. Different strategies were implemented into the program to improve the performance and retention of these topics. Unfortunately, recent semesters were impacted by the pandemic. Implementation was not ideal and data gathered was not considered normal. However, the department will be meeting prior to the fall 2021 semester to measure successful implementation fidelity of their strategies.
Finally, the department used student enrollment data to reach the decision to drop Intermediate Algebra. In addition to that, the Math Department will also only offer corequisite college algebra for STEM track students who need the course for remediation and college algebra.
Arkansas Tech’s Study Abroad program has kept up to date with their yearly assessment, as well as recently completing their CAS review. Gabriele Haulmark, Study Abroad Coordinator, has dedicated Study Abroad’s Mission to “increasing ATU’s presence and effective participation in the nation and the world.” The program partners with National Student Exchange and Global Endeavors to offer students a chance to broaden perspectives and experience history and culture firsthand.
Study Abroad uses two forms of assessment data. In addition to pre and post surveys (mostly rating scales), short answers from the surveys and student blog posts are used to measure their learning outcomes. These outcomes, created in 2017 by the Study Abroad Committee with the Study Abroad Coordinator, consist of:
- Increase Intercultural Competence
- Interact appropriately and effectively within a variety of cultures
- Develop global citizenship
- Adapt effectively to cross-cultural settings
- Develop Global Awareness
- Develop an awareness of social, political and economic state of host culture
- Increase awareness of housing, transportation, employment, education and healthcare systems of host cultures
- Increase marketability for professional employment
- Personal Growth and Development
- Self Confidence
- Respect for cultural differences between student’s culture and host culture
- Second Language Acquisition
- Improve listening, speaking, writing, and reading abilities in a second language and its dialects
Questions and blog-posting directions have been crafted to align with the learning outcomes. For example:
- Does your host culture have a different concept of time or space than you are used to? (Outcomes 1 and 3)
- What's your favorite food you've tried so far? (Outcome 2)
- What have you accomplished while abroad that makes you proud? (Outcome 3)
Haulmark finds the most satisfying and rewarding part of the assessment process to be reading the short answer and blog posts. The following excerpts of student responses and posts show alignment to the learning outcomes:
So far, I’ve learned a great deal about life in Graz and how people go about their routines. For example, a huge adjustment I’ve had to make is that stores and markets close extremely early (no Walmart people!). People here tend to run all of their errands before 6:00pm. This has been a big change, because you have to be prepared to get everything you need during the day.
For when I didn’t have class, I had arranged a small teaching assistant job for the duration of my stay in Austria at a primary school not far from Graz. I met with the professor at the beginning of February, and we had arranged for me to come to the school every Friday beginning on March 6th. Anywho, the teachers were excited and nervous to practice English with me, and the students looked at me as though I was magical, which really boosted my self-esteem. We played English games, sang songs, read books together, and learned dances all in the span of four hours. And I loved every second of it.
However, there is no doubt in my mind that I will return to Austria in the future. The life I had made for myself there completely on my own is something I’m very proud of! I’ve gained new independence and confidence in myself and with others. I can’t wait to see where that helps me with wherever life takes me. I am thankful again for the opportunity I had to go and the connections I’ve made all over the world.
On the 6th of February, I started German lessons. As I am sure to no one’s surprise by my intro, I started at the very bottom of the German knowledge pyramid. A1/1. This class is extremely fast paced as it is a full semester of German in only three weeks and very overwhelming at times as we are in class for three hours a day every day. However, I feel mildly more confident in my German capabilities and I have made it an effort to read all signs aloud to practice more and to repeat the announcement voice on public transport.
Closing the Loop:
The first change Study Abroad made based on assessment findings was after the first year of surveys. The original surveys were taken with pencil and paper. The lack of efficiency was fixed with the surveys being taken online. This allowed the data to be analyzed easier and more efficiently.
The next issue that Study Abroad is planning to address is the lack of conclusive and useful data. This issue stems from students failing to complete both surveys (pre and post). Thus, there is no way to compare results. Haulmark will explore options to ensure the completion of both surveys.
Next, Study Abroad noticed that the survey questions did not go into enough depth to ensure that outcomes were reached. It was especially apparent for long-term study abroad students. Special questions were implemented for these long-term students to answer in the form of blogs or reports. Study Abroad will continue to evolve their assessment practices to improve the program and ensure student learning outcomes are reached.
Arkansas Tech’s College of Business recently implemented a new assessment process. This process helps the college meet the standards for assurance of learning as required by the Association to Advance Collegiate Schools of Business (AACSB), the accreditor for the college. A key feature of this new process was the development of rubrics by the COB faculty to assess the primary learning objectives of the college.
Dr. Hunter and Dr. Huang attended two AACSB seminars focused solely on the assurance of learning process. As a result of this training and because the college revised its mission and vision statement in Spring 2018, the learning goals and related objectives were revised by the curriculum/AOL committee to ensure the goals were consistent with the mission and were then submitted to the COB faculty for its approval. At a faculty meeting in September 2018, the undergraduate business faculty were divided into groups to develop appropriate rubrics for each of the relevant learning goals. Each group began the process by asking each group member to list as many items as possible on individual “sticky” notes that they felt should be considered for the rubric. After that step was completed, the group collected all notes relating to a single topic (for example, correct punctuation for the writing rubric); then, the group developed subheadings and assigned the various ideas to a subheading. Dr. Hunter then created a rough draft of each rubric that was analyzed, discussed, and revised by the COB’s curriculum/AOL committee. Once the committee approved the rubrics, they were distributed to the entire COB faculty. After some additional discussion and minor revisions, the rubrics were approved by the COB faculty. The graduate rubrics were developed by the graduate faculty using the same process.
The College of Business undergraduate faculty developed goals/objectives and then the related rubrics to assess oral communications, written communications, ethics, technology, and professionalism. The graduate faculty created goals/objectives and then rubrics to assess ethics, oral and written communications, and technology. The decision was made to implement the assessment plan gradually. The COB began by using the ethics and oral communications (UG) and Technology (G) rubrics at the end of the Fall 2018 semester; during Spring 2019, data was collected using the written and oral communications rubrics (UG) and the ethics and oral communications rubrics (G). In Fall 2019, the COB used the professionalism, written, and oral communications rubrics (UG) and the written communication rubric (G) to collect data.
Findings and Interventions:
With the introduction of the rubrics, it was found that the students consistently struggled with two items on the oral communications rubric at both the formative and summative level for undergraduates and the summative level for graduate students. The first issue was the students failed to use a significant portion of the allotted time or ran over their allotted time. The second finding was that students did not comply with the mandated dress code (regardless of whether it was professional dress or business casual). When students were told to dress business casual, business professional, etc. they had trouble grasping exactly what that meant. The first intervention made was to purchase large clocks/timers for the students. This allowed for them to see exactly how long their presentation had been and adjust their speed to either reach the minimum or not exceed the maximum amount of time. The second intervention was the coordination of the “Dress for Success” luncheon held by the College of Business. Faculty, staff, and 86 students attended the luncheon in business professional clothing to learn about office dress wear. The luncheon featured several speakers as well as a representative from Modern Woodman Financial, Matt Ward. It was used to demonstrate appropriate interview, business professional, and business casual dress. Using both the clock and Dress for Success interventions, the COB reassessed these objectives during the Fall 2019 semester and found that the students scored significantly better on the rubrics in both the timing and appropriate dress objectives.
The College of Business will use the following assessment schedule over the next five years. The five undergraduate and four graduate objectives will continue to be assessed using rubrics and the resulting data will be analyzed to ensure continuous improvement within the college. The following assessment schedules allow the COB to “close the loop” two and one-half times within each five year accreditation cycle.
Summer Bruch, Associate Professor, took over the leadership of the Art Department as Department Head in the fall of 2018. As a significant function in keeping the department on track, Ms. Bruch uses a combination of tools to complete the assessment process and to stay up to date from year to year. Summer uses Weave to break down student learning in each degree program throughout the Art department. Each project within Weave lists current learning outcomes, measures, expectations, and the findings/results. Summer believes staying current with assessment makes the accreditation process manageable. She also uses QuestionPro for informal evaluation in the form of senior exit surveys along with an alumni survey. Since Arkansas Tech has a university-wide QuestionPro account, the price was right.
“When we have programs with rapid growth, like Game and Interactive Media, current assessment materials make it easier to allocate resources.”
Since taking over the leadership of the Department of Art, Summer has implemented an effective assessment practice to stay current and continuously take action to improve every aspect of the student learning in the Department of Art.