Assessment and Accreditation
Annual Reporting Measures
- Completer Impact and Effectiveness
- Employer Satisfaction and Stakeholder Involvement
- Candidates Competency at the Time of Program Completion
- Ability of Completers to be Hired in Positions for Which They Were Prepared
The EPP’s Self-Study Report will provide analysis of trends, comparisons with benchmarks, identification of changes made in the preparation curricula and experiences, how/where/with whom results are shared, resources allocations affected by the EPP’s uses of the information, and indications of future directions.
Measure 1 (Initial): Completer Impact and Effectiveness
The EPP developed a data collection plan for this measure as follows:
New York State does not offer the EPP access to teacher effectiveness data at a state level. The EPP has used the NYC teacher effectiveness data for graduates and sampled program-level data to cobble together a narrative. We have since fleshed out a more comprehensive model that includes three data sources:
- NYCDOE HEDI growth scores,
- The Employer Survey
- The Alumni Impact Study
NYCDOE HEDI Growth Scores: The EPP was reviewing HEDI data on EPP completers who were employed by the NYCDOE. This annual rating that all teachers in the NYCDOE received is called a “HEDI score.” HEDI is an acronym for an overall teacher impact rating. The H stands for “Highly Effective” (overall rating of 4), the E for “Effective” (overall rating of 3), D for “Developing” (overall rating of 2), and I for “Ineffective” (overall rating of 1). The NYCDOE defines the HEDI rating system as a multiple-measure system that allows for a more holistic assessment of a teacher’s contribution to students’ progress. The HEDI score results from a 50/50 combination of summary data across an academic school year from two different sources. The first are Measures of Teacher Practice (MOTP), which are grounded in eight components of the Danielson 2013 Framework for Teaching and the second are Measures of Student Learning (MOSL).
The annual HEDI rating is plotted on a matrix with the MOSL ratings down the Y axis and the MOTP ratings across the X axis. Using this matrix, for a teacher to receive an overall annual rating of Highly Effective, they must receive ratings of Highly Effective on both the Y (MOSL) axis and the X (MOTP) axis, or a rating of Highly Effective on one axis (i.e., either x or y) and a rating of Effective on the other axis. An overall annual summary HEDI rating of Effective is given when a teacher is either rated Effective on both the y and x axis (i.e., MOSL and MOTP) or they are rated Effective on one axis and Developing on the other axis.
Fact Sheet 21-11 (updated): Changes to Annual Professional Performance Review (APPR) (nysut.org)
Note: On June 7, the Governor signed Chapter 112 of the laws of 2021. This bill, which passed unanimously in both the senate and assembly, eliminates the requirement for school districts and BOCES to complete an annual professional performance review (APPR) for any classroom teacher or building principal for the 2020-21 school year. The bill also assures that state funding will not be withheld from any school district for not completing the APPR in the 2020-21 school year. A chapter amendment requested by the Governor makes it clear that districts do not have to complete an APPR this year. The HEDI Ratings were also affected by this decision and will not be available to CUNY schools moving forward.
With this in mind, the EPP is working on other methods to obtain this information. One method that is currently being investigated is to obtain the ratings from alumni directly. The EPP will continue to pursue avenues to obtain this data.
Employer Survey: The EPP has developed a new EPP-wide Employer Survey, which aligns with the newly developed and implemented Completer Survey and the Alumni Follow-Up Survey to allow data triangulation. The Employer Survey will be piloted in Summer 2023. The EPP will send the Employer Survey pilot to employers collected through the Alumni Follow-Up Survey that is sent to all alumni that are one year out in the field. Based on stakeholder feedback, the EPP is confident that the new Employer Survey will provide the EPP with quality data on employers’ satisfaction.
The Alumni Impact Study: In the Fall of 2022, the EPP piloted the new Alumni Impact Study (AIS). Data from the pilot is currently being culled through for improvements and expansion in Spring and Summer of 2023. Data will be displayed once analysis has been done by the EPP. The general idea behind AIS is to engage graduates of initial programs in a “study” of their teaching practice. Participants will be asked to choose a Unit that they plan on teaching. They will be asked to create a matching pre and post-assessment to assess student learning on the stated objectives of their Unit. Once they have collected their data, they will analyze it to determine “impact,” e.g., student gain or loss. In addition to presenting their results, they will be asked interviewed about their case and share insights about their impact on student learning.
HEDI Ratings of EPP Completers of Initial Programs 2 Years after Program Completion, by Academic Year of Program Completion of the 2013-2017 Completers
Fact Sheet 21-11 (updated): Changes to Annual Professional Performance Review (APPR) (nysut.org)
Note: On June 7, the Governor signed Chapter 112 of the laws of 2021. This bill, which passed unanimously in both the senate and assembly, eliminates the requirement for school districts and BOCES to complete an annual professional performance review (APPR) for any classroom teacher or building principal for the 2020-21 school year. The bill also assures that state funding will not be withheld from any school district for not completing the APPR in the 2020-21 school year. A chapter amendment requested by the Governor makes it clear that districts do not have to complete an APPR this year. The HEDI Ratings were also affected by this decision and will not be available to CUNY schools moving forward.
Advance, which is New York City’s teacher development and evaluation system, includes multiple measures – Measures of Teacher Practice (MOTP) and Measures of Student Learning (MOSL) – to create a robust picture of teacher performance and provide teachers with various sources of feedback to help them develop as educators. Throughout the school year, New York City teachers are observed by trained evaluators. At the end of the school year, teachers receive an Advance Overall Rating. The teacher’s Advance Overall Rating is used for personnel decisions (e.g., tenure determinations and Teacher Improvement Plans). The Advance Overall Rating is also referred to as HEDI ratings, which is an abbreviation of the four rating categories – Highly Effective, Effective, Developing, and Ineffective. As part of the CUNY-DOE data sharing agreement, CUNY Central annually provides the EPP with data on Queens College completers that are working as pedagogues in DOE schools. A file is shared with the EPP that contains the overall HEDI ratings of the completers that have been evaluated. The file is de-identified, meaning that all personal identifying information are removed from the file. In addition to the HEDI rating, the program, and the semester of graduation for each candidate is also included in the file.
The data table below includes the HEDI ratings 2 years after program completion by academic year of program completion.
During the 2019 school year 435 EPP candidates were identified as completing an Initial program. Of the 435 completers, 154 (35.4%) were identified as employed by the NYC DOE. HEDI ratings were available for 127 of the 154 of the 2016-2017 completers. Ninety four percent of the 2016-2017 completers that were enrolled by the NYC DOE were rated as either effective or highly effective 2 years after program completion.
| Rating | 2016 Rating of 2013-2014 Completers | 2017 Rating of 2014-2015 Completers | 2018 Rating of 2015-2016 Completers | 2019 Rating of 2016-2017 Completers | 2 Year Rating All Completer Years |
| Highly Effective | 10% (15/150) | 19% (30/161) | 15% (27/180) | 14% (18/127) | 15% (90/618) |
| Effective | 83% (125/150) | 73% (118/161) | 84% (151/180) | 80% (101/127) | 80% (495/618) |
| Developing | 7% (10/150) | 5% (8/161) | 1% (2/180) | 6% (8/127) | 5% (28/168) |
| Ineffective | 0% (0/150) | 3% (5/161) | 0% (0/180) | 0% (0/127) | 1% (5/618) |
EMPLOYER SURVEY DATA & FINDINGS
After reviewing data from our initial implementation of the NExT Employer Survey, it was clear that the survey was cumbersome and did not address the initial issue related to a low response rate. In Spring of 2021, the EPP decided to move away from the suite of NExT Common Metrics Surveys and create an EPP Employer Survey that is more in line with the mission, vision, and goals of the EPP. The new EPP Employer Survey will be implemented in Summer 2023.
NExT Employer Survey Results 2020
The NExT Employer Survey is administered every year following the administration of the NExT Transitions to Teaching Survey to Alumni. The NExT Transitions to Teaching Survey is administered 1 year after program completion in which candidates are asked to provide information in regard to their current employment. The first administration of the NExT Transitions to Teaching Survey was during the spring of 2020 and the NExT Employer Survey was first administered during the summer of 2020.
In spring 2020, the NExT: Transition to Teaching Survey was sent via SurveyMonkey to alumni. The EPP had access to valid emails for 400 of the 2018-2019 completers. The EPP received 90 responses from the pool of 400, with 28 providing valid employer school names. During a major health crisis, the low response rate while not ideal was also not surprising. Nine respondents provided supervisor emails and of the nine employers, 2 completed the Employer survey. Only one employer responded to the additional questions added to the survey by the EPP. The employer shared that there is consistent evidence of positive impact on student learning across groups of students and throughout the lessons, sharing that the teacher was a “solid teacher” in her subject area. At first the teacher struggled with classroom management. However, she readily implemented feedback throughout the year. By the 3rd trimester, the students were showing higher proficiency levels on assessments. The teacher engaged students effectively in class, which is critical as she instructs students in the first year of a three-year program. Students indicated that the instruction was clear and that her materials were very well prepared. The EPP is currently taking steps to increase the response rate for the NExT Employer Survey.
Please see the links below for results from EPP Employer Surveys for prior years.
2019 EPP Employer Survey Results
2018 EPP Employer Survey Results
Measure 2 (Initial & Advanced) Employer Satisfaction and Stakeholder Involvement
NExT Employer Survey Results 2020
The NExT Employer Survey is administered annually following the administration of the NExT Transitions to Teaching Survey to Alumni. The NExT Transitions to Teaching Survey is administered 1 year after program completion in which candidates are asked to provide information in regard to their current employment. The first administration of the NExT Transitions to Teaching Survey was during the spring of 2020 and the NExT Employer Survey was first administered during the summer of 2020.
In spring 2020, the NExT: Transition to Teaching Survey was sent via SurveyMonkey to alumni. The EPP had access to valid emails for 400 of the 2018-2019 completers. The EPP received 90 responses from the pool of 400, with 28 providing valid employer school names. During a major health crisis, the low response rate, while not ideal, was also not surprising. Nine respondents provided supervisor emails, and of the nine employers, 2 completed the Employer survey. Only one employer responded to the additional questions added to the survey by the EPP. The employer shared that there is consistent evidence of a positive impact on student learning across groups of students and throughout the lessons, sharing that the teacher was a “solid teacher” in her subject area. At first the teacher struggled with classroom management. However, she readily implemented feedback throughout the year. By the 3rd trimester, the students showed higher proficiency levels on assessments. The teacher engaged students effectively in class, which is critical as she instructs students in the first year of a three-year program. Students indicated that the instruction was clear and that her materials were very well prepared. The EPP is currently taking steps to increase the response rate for the NExT Employer Survey.
Please view the results of the prior EPP Employer survey for prior years at the links below.
2019 EPP Employer Survey Results
2018 EPP Employer Survey Results
Measure 3 (Initial & Advanced) Candidates Competency at the Time of Program Completion
This report contains the pass rates on the EAS and the Certification exams required for NY State Teacher Certification of the Queens College EPP completers. The data files were obtained from edReports Results Analyzer. If a candidate took an exam more than once only the best attempt is included in the results. The data is disaggregated by Licensure Area and academic year of program completion.
EAS
In the past 3 academic years completers of an EPP Undergraduate Initial program have had an average 93% pass rate on the EAS exam. The completers of Graduate Initial programs had an overall pass rate of 97% for the past 3 academic years.
CST
The data is disaggregated for Initial and Advanced programs and by licensure area and academic year of completion. A completer may be required to take more than one exam, i.e., for Elementary Initial Certification, a candidate is required to take the MS 1-6 Part One Lit/ELA, the MS 1-6 Part Two Math and the MS Part Three Arts & Science exams. Overall, the completers of an EPP Undergraduate Initial program had an average 89% pass rate on the CST exam for the past 3 academic years. The completers of a Graduate Initial programs have had a 91% pass rate on the CST exams. The completers of Graduate Advanced programs had an overall pass rate of 91% for the past 3 academic years.
Overall, 89% of Undergraduate Initial Completers and 90% of Graduate Initial completers that took all required exams for certification, passed all required examinations for certification.
Measure 4 (Initial & Advanced) Ability of Completers to be Hired in Positions for Which They Were Prepared
Total Queens College EPP Completers Hired by NYC DOE by Completer Year
As part of the CUNY-DOE data sharing agreement, CUNY Central annually provides the EPP with data on Queens College completers that are working as pedagogues in DOE schools. A file is shared with the EPP containing information on Teacher Education candidates who have completed an EPP Program are received from CUNY annually. Variables such as the school where completers are teaching, dates of employment, among various other data points are shared with the EPP. The table below contains the counts and percentages of completers that have ever obtained employment with the NYC DOE.
As of as of 03/2022, 60% of the 2018-2019 Completers of an EPP program had been employed by the NYC DOE. Of the Initial completers that were employed by the DOE, approximately 42% received employment within 1 year of program completion during the past 3 years. (Dropped to 20% DURING 2019-2020 which is most likely due to COVID pandemic). Candidates of Graduate Advanced programs are more likely to already be employed with the NYC DOE prior to program completion.
Please note that the most recent years of data provided to the EPP are represented below. Data for 2021-2022 will be available next year.
8.23