“The art and science of asking questions is the source of all knowledge.” ― Thomas Berger
Frequently Asked Questions
Does assessment include faculty or staff evaluation?
No – absolutely not. The purpose of learning assessment is to improve student learning and it is not to be used as a way of evaluating instructors. Similarly, the purpose of administrative assessment is to improve the student experience at QC. No individual faculty or staff member has the sole responsibility for ensuring that program/office goals are met. Therefore, any assessment of progress on academic and administrative goals must not be used to evaluate any individual faculty or staff member.
How is grading different from assessment?
Generally, the goal of grading is to evaluate student learning, whereas the goal of assessment is to improve it. Grading is the analysis and use of data by faculty to make judgments about an individual student’s performance. Assessment, on the other hand, is a way to measure student learning overall and improve student learning within a course or program.
Grading plays an important role in assessment, of course, as grades can provide useful evidence of student learning when they are based on direct measures (tests, projects, papers, etc.) that are clearly linked to course learning goals and consistent with standards. However, assessment typically goes beyond grading by systematically examining patterns of student learning and using this information to improve educational practices.
|2||Process-oriented (how students learn)||Product-oriented (what students learned)|
|3||Diagnostic: identify areas for improvement||Judgmental: assign a letter grade|
How is assessment different from research?
Assessment and research are similar in many ways: Both involve asking questions, collecting data, analyzing results, and using the results as reliable evidence. Like research, assessment activities may use quantitative or qualitative research methods, and often benefits from a mixed methods approach.
However, there are important differences. The goal of research is to confirm or challenge hypotheses to guide theory, whereas the goal of assessment is to produce reasonably accurate information about how well we are meeting our goals and to guide local practice. Many factors limit the scope and impact of assessment, including limitations on time, resources, design and implementation. As such, assessment findings typically have implications for a single course, program or institution, whereas research findings are intended to have broader implications.
Decisions about curriculum, teaching techniques, assignments, and syllabi are made regularly and such decisions are best informed by evidence – the findings of assessment activities. Good assessment practices, in the context of a particular course, program or institution, can and should guide local practice and decisions for the continual improvement of the student experience.
Assuming assessment findings will not be published or presented at a conference, assessment activities that are conducted for internal decision-making (and not as part of a “systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge”) do not require IRB review. IRB approval is required for generalizable research involving human subjects. To learn more, see CUNY’s exemption policy, or CUNY’s policies on Human Research.
What constitutes evidence of student learning?
This question is best rephrased as, ‘How do we know that our students are learning?’ and it might be useful to keep in mind that gathering evidence of students’ learning is something you have been doing for most of your academic career.
While the evidence we gather is not the result of randomized controlled trials, we can, however, triangulate our measures of student learning, keep up to date with the literature on teaching and learning, and continually reflect on how learning can be improved. For example, a grade is one measure of student learning, and grades can provide useful evidence of student learning if they are based on direct measures (exams, projects, papers, etc.) that are clearly linked to course learning goals.
Student learning is best measured using several assessment methods (direct and indirect measures ). To avoid easy measures that do not sufficiently mirror the complexity of student learning, we must also take into account the peculiarities of a discipline to which assessment methods are applied.
To learn more, see Linda Susky’s Examples of Evidence of Student Learning, or check out Susan A. Ambrose’ How Learning Works: Seven Research-Based Principles for Smart Teaching.
What is the difference between direct and indirect measures?
‘Direct measurement’ refers to measuring exactly the thing that you’re looking to measure, while ‘indirect measurement’ means that you’re measuring something by measuring something else.
Direct measures are those that measure student learning by assessing actual samples of student work. Examples include: exams/tests, papers, projects, presentations, portfolios, performances, etc. Because direct measures capture what students know and can do, and even how well, these are best for measuring achievement of student learning on specific objectives.
Indirect measures imply student learning by employing self-reported data and reports. Indirect measures help to substantiate instances of student learning. Indirect measures include surveys, interviews, course evaluations, and reports on retention, graduation, and placement, etc. These measures are commonly used in conjunction with direct measures of student learning, as best practices recommend the use of both direct and indirect measures when determining the degree of student learning that has taken place.
What is the difference between learning goals and measurable learning objectives (aka. SLOs)?
Learning goals are the broad statements we make about what we expect students to know and be able to do as a result of completing a learning experience (e.g. a reading assignment, a course module, a course or a program). Measurable learning objectives, aka. “Student learning outcomes (SLOs)” are statements that describe the observable or measurable behaviors that would indicate the level of mastery students have achieved as a result of a learning experience. Rephrasing our learning goals for a course or program in this way can help us determine whether the learning goals we set for our students have been met, and is especially helpful for standards-based grading, which focuses on mastery.
Why is it important for a syllabus to include course learning goals and objectives?
Having clear learning goals and measurable objectives (aka. SLOs) on syllabi is helpful for both students and instructors. Clearly stated goals and objectives helps students to know what the expectations are for a course ( the learning goals) and how they will be evaluated (the measurable objectives), which can positively impact student motivation and self-efficacy, and encourages students to self-assess. For instructors, having clear learning goals and objectives for a course helps to structure course content, which aids in the effective evaluation of student work. This is also known as mapping — the process of aligning learning goals to assignments and activities to ensure that learning goals are addressed in the course (or curriculum) in ways that support students’ opportunities to develop needed skills and knowledge.
Why is it important for a program to formulate a mission statement?
A mission statement is a public-facing document that identifies а program’s constituents and what the program intends to provide to them. There is a hierarchy: CUNY has a mission statement; Queens College has a mission statement (and a somewhat longer “mission summary!”); the four divisions of the the College have missions; departments have missions; and academic programs have missions. Each mission statement provides the context for the ones below it, and each mission statement must not be inconsistent with the ones above it.
For example, consider this excerpt from the University’s history and mission website:
The Legislature’s intent is that The City University be supported as an independent and integrated system of higher education on the assumption that the University will continue to maintain and expand its commitment to academic excellence and to the provision of equal access and opportunity for students, faculty and staff from all ethnic and racial groups and from both sexes. The City University is of vital importance as a vehicle for the upward mobility of the disadvantaged in the City of New York.
In this example, the encompassing constituency is the state legislature, which provides the funding context for the university; the served constituents are students, faculty, and staff. Socioeconomic mobility is an explicitly stated service to be provided.
In sum, the mission statement for an academic program should establish the context in which the program operates and an overview of its goals. Mission statements, which are broad and general, are closely tied to a program’s statements of goals, which provide a bridge between the mission of a program and the strategies to be used to assess and guide the ways in which the program achieves its mission.
Once the mission and goals for a program are in place, a curriculum map can be used to show how and when progress towards a program goal is developed in what courses. A particular course might be responsible for only part of an individual learning outcome and will probably span multiple learning outcomes for the program. Some courses may well include learning outcomes that are not part of the program’s mission and goals. The latter often appear in courses that are an elective part of the program. The main value of a curriculum map is that it provides a clear way to determine whether the courses that make up the program actually provide the coverage desired and thus whether the mission of program can actually be achieved. It can also make clear whether there is unintended overlap between courses, and verify that the prerequisite structure of the program makes sense.
What are missions and goals?
A mission statement is a general statement that succinctly outlines the overall purpose of an academic program or administrative office at the college. Mission statements describe what the unit does and for whom, and should be aligned with the college’s mission. Mission statements change only in the event of fundamental changes to the role or purpose of the administrative unit.
Goals are broad statements about the desired ends to which a program or unit aspires—a vision for how the program or unit will fulfill its mission. Hence, goals are more specific than mission statements, but still general enough that they apply for many years. Essentially, goals provide direction when setting measurable objectives.
How are goals different from objectives?
An objective is a specific, measurable step that can be taken to meet a goal. Objectives describe how a goal will be accomplished – the intended outcomes of activities or processes that have been implemented to make progress towards a goal. Where goals are broad and general, objectives are concrete and measurable, bridging the gap between goals and assessment methods.
In the administrative assessment process, measurable objectives align to operational goals and student goals.
In the learning assessment process, measurable objectives align to learning goals and are called “student learning outcomes (SLOs)” at other colleges.
|2||general intentions||specific actions|
|3||difficult to measure||measurable|
How is academic program assessment different from the Academic Program Review (APR) process?
The APR process is a comprehensive self-assessment to identify a program’s challenges and opportunities in areas of quality, service, efficiency and resources. Assessment of student learning is just one component of this process. Program assessment, on the other hand, is much more focused on student learning. Furthermore, while APR self-studies take place once every few years, program assessment takes place annually. The aim is to gather information that will enable a program to identify issues that may (or may not) require attention in a timely fashion.
What is the connection between assessment and accreditation?
The short answer is that institutions and state policymakers both make use of assessment findings. However, assessment and accreditation are very different processes with very different goals.
Assessment is an internally-managed activity, as opposed to accreditation, in which an outside body evaluates how well an organization is performing. And while accreditation self studies aim to demonstrate that fiscal and human resources are being invested responsibly, faculty and staff conduct assessment activities to improve their programs. Assessment findings are an important part of the accreditation review process since they are a key source of the evidence that we provide to our accreditation bodies, but the primary focus of assessment is the student experience – assessment activities are student-centered and are meant to inform local decisions pertaining to courses, majors, programs, and support offices, whereas accreditation activities are meant to inform external reviewers.
Here at Queens College, we believe student learning assessment is best guided by the intellectual curiosity of faculty about their programs and that assessment should be approached in the spirit of experimentation and innovation. We also abide by our Guiding Principles for Assessment.
Queens College is accredited by the Middle States Commission on Higher Education (MSCHE). To learn more about MSCHE expectations with regards to assessment, see their document on “Assessing Student Learning and Institutional Effectiveness: Middle States Expectations”. To learn more about the accreditation review process at QC, see our Provost’s website on MSCHE Accreditation.
What is administrative assessment?
Each administrative unit has a unique role at the college – a distinct and important mission. That mission is met by setting goals that are aligned to the mission and by making progress towards those goals. Because meeting these goals are crucial to the effectiveness of the college and the success of its students, Queens College engages all of its administrative offices in assessment. We assess progress on administrative office goals because decisions should be based on evidence rather than assumptions and because we want to improve strategic planning efforts, decision support, resource allocation, and operational excellence. The assessment process (1) provides feedback to determine how the administrative unit can be improved, (2) informs decision-makers of the contributions and impact of the administrative unit to the development and growth of students, and (3) provides support for strategic planning, as well as external accountability activities such as accreditation.
What are some tips for designing an administrative assessment plan?
- Involve all members of your staff in the process of establishing a mission statement, and office goals and objectives. Staff participation and ownership are key to effective assessment.
- Write clearly and concisely. Do not use jargon or professional terms that someone outside your area of expertise will not understand.
- Be sure that your goals and objectives are not tasks. Why is it important to you to complete a certain task? That is likely your objective.
- Be sure that your objectives are measurable. How will you know that the objective has been achieved?
- After the mission, goals and objectives have been established, use our logic modeling and assessment activity planning worksheets to describe an assessment activity in terms of concrete steps.
- Provide at least one direct measure for each objective being assessed.
- If you cannot specifically describe how the findings from your assessment activities can be used to improve your program, services or operations, reconsider your proposed means of assessment.
- Think about how assessment findings will be used to ‘close the loop’. Showing how the results of assessment are used for improvement is the most critical component of an assessment report.
Learn more here: QC Guidelines for Administrative Assessment.