[et_pb_section fb_built=”1″ fullwidth=”on” _builder_version=”4.27.0″ _module_preset=”default” global_colors_info=”{}”][et_pb_fullwidth_image src=”https:\/\/www.qc.cuny.edu\/ie\/wp-content\/uploads\/sites\/39\/2024\/10\/QC-4.jpeg” alt=”People at Queens College Campus sitting on the grass.” title_text=”People at Queens College Campus sitting on the grass.” _builder_version=”4.27.0″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_fullwidth_image][\/et_pb_section][et_pb_section fb_built=”1″ _builder_version=”4.27.0″ custom_padding=”40px||40px||false|false” global_colors_info=”{}”][et_pb_row _builder_version=”4.16″ global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ global_colors_info=”{}”][et_pb_text _builder_version=”4.27.4″ _module_preset=”default” text_font=”Open Sans||||||||” text_text_color=”#000000″ header_4_font=”Open Sans|600|||on||||” header_4_text_color=”#E71939″ hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″]<\/p>\n
When developing a survey instrument, be sure to adhere to the following best practices:<\/span><\/p>\n <\/p>\n <\/p>\n You can learn more about survey design here: Writing Survey Questions (Pew Research Center)<\/a>.<\/strong><\/p>\n Faculty engaging in program assessment can see examples of appropriate survey questions and answer options here:<\/p>\n The OIE can tailor the surveys above for your program and field such surveys on your behalf. Learn more about the kinds of assistance OIE can offer here: OIE Assessment Support<\/strong><\/a>.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ _builder_version=”4.27.0″ _module_preset=”default” background_color=”#efefef” use_background_color_gradient=”on” background_color_gradient_stops=”#efefef 0%|#ffffff 100%” background_color_gradient_start=”#efefef” background_color_gradient_end=”#ffffff” custom_padding=”20px||20px||false|false” global_colors_info=”{}”][et_pb_row _builder_version=”4.27.0″ _module_preset=”default” background_color=”RGBA(255,255,255,0)” custom_padding=”|20px||20px|false|true” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.21.0″ _module_preset=”default” text_font=”Open Sans||||||||” text_text_color=”#000000″ header_4_font=”Open Sans|600|||on||||” header_4_text_color=”#E71939″ global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_accordion open_toggle_text_color=”#E71939″ open_toggle_background_color=”#ffffff” closed_toggle_text_color=”#b7123b” closed_toggle_background_color=”#ffffff” icon_color=”#e71939″ _builder_version=”4.27.4″ _module_preset=”default” toggle_text_color=”#000000″ toggle_font=”Open Sans|600|||||||” body_font=”Open Sans||||||||” body_text_color=”#000000″ body_ul_position=”inside” hover_enabled=”0″ border_color_all=”#000000″ global_colors_info=”{}” sticky_enabled=”0″][et_pb_accordion_item title=”Institutional Review Board Approval” open=”on” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n If the data to be gathered and used are part of an academic research project, the project will need to be reviewed and approved by the IRB to ensure it meets their requirements. If the data are to be used for administrative purposes, it may be exempt from a formal review by the IRB, but the survey must be voluntary and the results kept confidential. You and the College may incur legal liability if the treatment of survey participants is unethical, if data resulting from your survey are misused, or if any part of the survey violates certain protected rights of individuals. Survey researchers should be aware of their responsibilities and make every effort to protect the rights of survey respondents.\u00a0<\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”The Rights of Respondents” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}” open=”off”]<\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”Confidentiality and Anonymity” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}” open=”off”]<\/p>\n An anonymous survey means that the researcher cannot determine who participated in the survey and who has not. Anonymous surveys typically do not collect identifying information, and cannot restrict responses to one response per person.<\/p>\n A confidential survey collects information that can identify participants either alone or in combination (e.g., race\/ethnicity, gender, major, class year) and allows the researcher to analyze responses along a variety of demographic factors, but also means researchers have a responsibility to protect the privacy of the participant. When data are collected confidentially, it is helpful to let participants know that results will be presented in aggregate only and that any results that can potentially identify participants will not be presented.<\/p>\n If your survey results will include academic or contact information for HMC students, you may be subject to Federal Family Educational Rights and Privacy Act (FERPA) regulations. Non-directory student data (e.g., GPA, Major(s), race\/ethnicity) can be used without a student\u2019s consent by college employees for \u201clegitimate educational\u201d purposes, provided the data are not reported in such a way that individual students can be identified. Otherwise, users must receive written consent from students.<\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”Sample vs Census” _builder_version=”4.27.4″ _module_preset=”default” hover_enabled=”0″ global_colors_info=”{}” open=”off” sticky_enabled=”0″]<\/p>\n It is typically not necessary to survey all QC students or all QC employees (census surveys). A sample needs to be large enough to protect the privacy of respondents and large enough for your results to be considered representative, but that does not mean you need to survey everyone. If you wish to target all students or all employees, OIE will typically employ a random sampling method to ensure that these populations are not over-surveyed. To calculate the minimum sample size to be considered representative within a given margin of error, visit\u00a0Sample Size Calculator<\/a>.<\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”Pilot Testing” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}” open=”off”]<\/p>\n It is always best to pilot your survey to a small group from the population you intend to survey. As an assessment of the survey’s face validity, this can help you understand how the questions you have written will be perceived and answered and whether the survey is too long or overly complicated.\u00a0 Be sure to ask your pilot group whether any survey question was difficult to understand or difficult to answer.<\/p>\n If your survey project requires more rigorous testing, please consult with OIE. We can estimate the construct validity of an instrument using confirmatory factor analysis, and the internal consistency and reliability of the instrument can be estimated with Cronbach Alphas. Factor analysis and Cronbach Alphas can determine whether items in a survey should be edited or removed.<\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”Race\/Ethnicity Items” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}” open=”off”]<\/p>\n If the OIE fields a survey on your behalf, there is no need for demographic items on the survey such as a race\/ethnicity question, as we can tie confidential survey responses to administrative data. Anonymous surveys that require demographic information should ask these types of questions near the end of the survey and these questions MUST be voluntary. Please also note that as per QC and CUNY policy, any survey that includes a race\/ethnicity item must include “Italian decent” or “Italian American” as a response option since Italian Americans are included among protected groups at CUNY.<\/span><\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”Survey Invitations” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}” open=”off”]<\/p>\n We’re more likely to participate in a survey when (A) the purpose of the survey is clear and (B) we know who is asking the questions. Survey invitations should be short and concise, but explain why participants’ responses are important and how the survey results will be used. Also, be sure to sign the survey invite as the PI. If you are sending a survey to faculty or staff at QC, it can be helpful to send out a longer email to your target population prior to the survey invitations so that people know what to expect.<\/span><\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”Incentives” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}” open=”off”]<\/p>\n Depending on the population and the survey, you may offer incentives to survey respondents. Incentives must be positive (i.e., no negative consequences for failing to participate). They should also be relevant to the topic of the survey, and small enough so as not to be coercive (e.g. gift cards to Amazon, Starbucks, or the bookstore). Keep in mind that while offering incentives can increase a survey’s response rate, it can also invite less meaningful responses. Please consult with OIE if you are considering offering an incentive.<\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”Timing” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}” open=”off”]<\/p>\n It is important to request assistance with a survey well in advance. Please understand that your preferred dates of administration may be delayed due to our availability to assist you with your project, or due to a pre-existing survey. Concurrent surveys that target the same population lower response rates, overburden respondents, and lead to poor quality data.<\/p>\n [\/et_pb_accordion_item][et_pb_accordion_item title=”Sharing results” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}” open=”off”]<\/p>\n We urge you to share your results in a way that is accessible to the community you surveyed, unless your results are for very small groups. By being transparent with results, the populations we survey can see the value in participating in surveys.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/article>\n<\/div>\n\n
<\/strong>A survey is most effective if its purpose can be clearly and succinctly stated. Surveys that are vague or overly-broad can become too long or difficult to analyze. Additionally, depending on the question(s) to be answered, a survey may not be the best method<\/span><\/li>\n<\/ul>\n\n
<\/strong>Open-ended questions allow the greatest variety of responses, but are time consuming to answer and require a lot of work to analyze. Fixed-response questions (questions that use pre-populated answer choices for the respondent to choose from) are easier for respondents to answer, easier to analyze, and when well designed, ensure that respondents interpret questions the same way.\u00a0\u00a0<\/span><\/li>\n<\/ul>\n\n
<\/strong>Question prompts must be asked in a balanced and neutral way to avoid bias. Leading questions influence responses in a way that does not reflect respondents\u2019 true opinions\/experiences. \u201cThe choice of words and phrases in a question is critical in expressing the meaning and intent of the question to the respondent\u2026 Even small wording differences can substantially affect the answers people provide\u201d – Pew Research Center.<\/span><\/li>\n<\/ul>\n\n
<\/strong>A common mistake is to ask questions that most people cannot easily remember. Limit the timeframe for recall to a reasonable period. Less than 2 weeks is ideal, but a single term can be feasible. Another common mistake is to ask people to answer for others. Make sure your respondents are answering only for themselves.<\/span><\/li>\n<\/ul>\n\n
<\/strong>For research-focused projects, such as impact evaluations and cohort studies, attitude\/rating scales should be at 4- or 5-points. For quick polls and needs assessments, 3-point scales are sufficient. Include \u201cI don\u2019t know\u201d and \u201cN\/A\u201d options where applicable.<\/span><\/li>\n<\/ul>\n\n
<\/strong>Questions should measure one thing. Double barreled questions try to measure two or more things.<\/span><\/li>\n<\/ul>\n\n
<\/strong>Respondents should only be asked questions that apply to them.<\/span><\/li>\n<\/ul>\n\n
<\/strong>Survey responses can be impacted by previous questions. Think about the context in which respondents are reading your questions.\u00a0\u00a0<\/span><\/li>\n<\/ul>\n\n
<\/strong>If the OIE fields a survey on your behalf, there is no need for demographic items since we can tie confidential survey responses to administrative data. Anonymous surveys that include demographic items MUST ensure these questions are voluntary. Additionally, any race\/ethnicity items must include “Italian decent” or “Italian American” as a response option since Italian Americans are included among protected groups at CUNY.<\/li>\n<\/ul>\n\n
<\/strong>The shorter and simpler the survey, the more likely respondents are to answer all the questions and answer them honestly. Complex or long surveys can annoy respondents and cause them to stop responding part-way through or to pick arbitrary answers to get to the end quickly.<\/span><\/li>\n<\/ul>\n\n
Important Considerations<\/span><\/h3>\n
\n
\n\n\n\n\n\n\n\n\n