Program Assessment

When designing a learning assessment activity, it is important to consider the following: (1) Assessment is an opportunity to address faculty concerns related to student learning and should start with genuine questions, (2) It is important to articulate the learning goals of the program and the ways in which progress on these goals can be meaningfully measured, and (3) Good assessment is manageable and sustainable. It is neither prudent nor productive to measure everything. It is also unlikely that you will be able to assess progress on all of a program’s learning goals in a single year. Remember that learning assessment is an ongoing process and create a long-term strategy for the assessment of your program’s learning goals.

“Assessment must not be treated as an end in itself, but rather as one (albeit important) means for educational improvement. Educational values should guide not only what instructors choose to assess but also how they do so.” (APA on Outcomes Assessment, 2008)

There are two fundamentally different types of program-level assessment activities: (1) Activities that explore ways in which the program is meeting its mission and goals, identifying both strengths and opportunities for improvement, and (2) Activities that determine the effects of an intervention that has been carried out in order to strengthen the program in some way, such as changes in curriculum or instructional methodology. Naturally, the second type can be evaluated meaningfully only if it is preceded by an activity of the first type.

Annual Activity Guidelines

1. Define the mission/goals of the department or program.

If a department has not done so recently, it should review and possibly update its mission statement. Does the mission identify the student population the program intends to serve? Does it say what students should achieve by taking part in the program? Mission statements should also be consistent with the Queens College mission statement and the University’s. If the department has not yet identified its program learning goals, define the learning goals students to achieve as a result of participating in the program and the corresponding desired and observable outcomes; how progress on these goals can be meaningfully measured, aka. student learning objectives or outcomes (SLOs)“.

[SLOs: Statements of observable or measurable behaviors that indicate the level of mastery students are expected to demonstrate as a result of a learning experience.]

2. Establish how well the curriculum accomplishes its mission/goals.

Develop a Curriculum Map (AKA Degree Map): A Curriculum Map identifies how the courses in a degree program contribute to the program’s learning goals and addresses questions such as:

• Is the curriculum well-sequenced for student learning?
• What does the curriculum say about our student learning priorities?
• How do the required learning experiences of our curriculum shape a graduate of our program?

For faculty, the process of curriculum mapping facilitates understanding of how courses are sequenced and “fit” together, and shifts the focus from “my course” to “our degree program.” Mapping also supports curriculum revision, as the activity often reveals curricular strengths, gaps, necessary and unnecessary overlaps, and needs (e.g., Do students have enough practice to achieve a goal?). For students, curriculum maps facilitate understanding of how (1) each course is intended to contribute to specific knowledge or skills of the program, and (2) elective courses could be strategically selected to further strengthen knowledge or skills or to explore new areas of interest.

A common method for developing a basic curriculum map is to:

• Identify the courses you will be mapping to your program learning goals
• List the courses in a row across the top of an Excel Sheet/chalkboard/sheet of paper
• Place program learning goals in a column down the right side
• Convene the faculty teaching within your degree program for 1 to 2 hours
• Have faculty identify the program learning goals they are covering in their courses
• Transfer the finished map into Excel once it is complete
• ALWAYS follow up the creation of a Map with an examination of the Map

Carnegie Mellon’s Eberly Center offers a great Blank Mapping Tool (xls).

Design an Assessment Activity: When designing an assessment activity, consider both the logistics of the activity (How many classes are you assessing? What will be necessary to ensure that the students complete the assessment?) as well as the validity of the assessment (How seriously will the students take the assessment? How will students and instructor perceive the assessment?). Remember that while the extent to which a desired outcome is easy to measure has little to do with the degree to which it is worthy of measurement, it is important to avoid easy measures that do not sufficiently mirror the complexity of student learning.

Explain the procedures used to determine whether the curriculum is effective in achieving the learning goals sought by the department. Depending on the mission of the department, these methods may include in-course evaluations of student achievement, capstone courses, and measurement of student attitudes and learning through surveys.
Assistance can be provided for web-based surveys. Copies of surveys or other instruments used should be included in appendices. Examples include:

• Curriculum coverage, but be aware of Barr and Tagg
• Retention, grades
• Attainment of learning goals
• Comparison to other: departments, national norms; institutions
• Placement success: jobs/salaries; graduate schools
• Alignment with industry/professional needs

See the section on Program-Level Assessment Methods below for more example measures of student learning.

3. Identify where the department or program can improve and make suggestions for the future (“Close the Loop)”

An essential part of assessment is to use the information obtained either to publicize the program’s strengths or to identify opportunities for improvement and to act on them. If the activity identified a noteworthy strength, how will this result be disseminated? If the activity reveals an area of concern, what concrete actions will the department take to address it? Use assessment results to spark meaningful conversation among faculty as to what steps should be taken to enhance student learning, and how implemented changes might be assessed in the future.

It is also important to remember that no individual faculty member has the sole responsibility for ensuring that students will acquire one or more of the program’s learning goals. Student attainment of learning goals should result from the collective learning experiences they engage in during their time at the college. Therefore, learning assessment must not be used to evaluate any individual faculty to measure the performance of any instructional program.

Learning Assessment Feedback

Often, assessing progress on program learning goals involves a series of decisions dependent on your sample size, resources, needs, and other project variables. The Office of Institutional Effectiveness can provide feedback to help faculty conduct meaningful and worthwhile assessment activities.

Feedback on annual activities:

• Is the activity clearly related to the mission and goals?
• Is the program prepared to carry out the activity?
• Is the scope of the activity suitable for completion in the timeframe planned?
• For assessments that span multiple years, how well does the activity integrate with the overall project?
• Will it provide meaningful, actionable results?
• Can the usefulness of the activity be improved?

Feedback on annual reports:

• In hindsight, was this a useful activity? 
• Does the report indicate clearly what actions the academic unit will be taking, based on the results of this activity?
• Are there suggestions for future annual activities based on the results of this one?