Standards for the Conduct of Government of Canada Public Opinion Research—Telephone surveys

1.1. Introduction

1.2. Technical specifications of the research

2.1. General

Survey questionnaires must be designed:

1. To collect only the information relevant to the stated research objectives of the study.
2. To minimize respondent burden while maximizing data value and utility.
3. To be completed in an average duration of 15 minutes when administered by a live interviewer, or an average duration of five minutes for Interactive Voice Response (IVR). If supported by a strong rationale, exceptions to these duration limits could include projects with specialized audiences and those with pre-arranged interviews when the respondent is aware the survey will take longer than 15 minutes.

2.2. Questionnaire introductions

1. The following are required elements of all Government of Canada telephone survey questionnaire introductions:
   1. In accordance with the Official Languages Act and policies, introductions must include an active offer of both official languages (English and French). The language that is offered first will depend on the province in which the respondent resides (in Quebec “Bonjour/Hello”; in the rest of Canada “Hello/Bonjour”.) When relevant to a specific survey, non-official languages may also be offered.
   2. Identify the Government of Canada or the department/agency sponsoring the survey. When there is a strong rationale, the sponsor may be identified at the end of the survey instead of in the introduction (for example, for advertising post-tests).
   3. Inform respondents of the general subject and purpose of the study. The subject and purpose of the study may be expressed in general terms, as long as the wording used cannot be construed as an attempt to misinform respondents. When the researcher makes a convincing case that stating the subject and purpose of the survey at the outset is likely to bias the sample that chooses to participate and/or to bias participants’ responses, these statements may be deferred to the conclusion of the survey.
   4. Offer alternative modes of data collection and/or assistive technologies that are available to make the survey accessible to persons with disabilities.
   5. Inform respondents of the expected length of the interview.
   6. Identify the researcher (research firm) and interviewer (a pseudonym may be used for the interviewer as long as the individual remains identifiable to the research firm’s internal management, for quality control purposes).
   7. Inform respondents that their participation in the study is voluntary and confidential.
   8. Inform respondents that their responses will remain anonymous. In exceptional cases where the research objectives require that respondents’ identity be revealed, the informed consent of the respondent must be obtained.
   9. Confirm that respondents are in a location where it is safe and convenient to be interviewed. If the interviewer perceives that the respondents’ setting may endanger them (for example, driving a vehicle), hinder their participation (for example, background distractions), or hamper their ability to respond openly (for example, lack of privacy), the interviewer should take proactive steps, such as rescheduling the interview.
   10. Inform respondents how they can verify the legitimacy of the survey as a research initiative sponsored by the Government of Canada.

The following introduction provides a mandatory first paragraph that offers respondents both official languages, as well as additional sample wording that could be used for telephone questionnaires:

“Hello / Bonjour (pause), the Government of Canada is conducting a research survey on subject. Would you prefer that I continue in English or French? Préférez-vous continuer en français ou en anglais?”

“My name is first name of name of firm, the company hired to do the survey. The survey takes about ## minutes to complete. Should you have any questions about the survey, I can give you a contact person within the Government of Canada / Department name. Your participation is voluntary and confidential. Your answers will remain anonymous, and the information you provide will be administered according to the requirements of the Privacy Act, the Access to Information Act, and any other pertinent legislation. Is this a safe and convenient time for you? May I continue?”

2. Interviewers must be prepared to repeat all or parts of the survey introduction or instructions, if requested by the respondent. In addition, interviewers must have information readily available for answering respondents’ questions regarding how respondents were chosen, how privacy is protected, how they can verify that the survey is legitimately sponsored by the Government of Canada, and where the survey results can be obtained. (See section 6.1 for more information on project briefing of fieldworkers).
3. Interviewers must be prepared to address individual respondents’ accessibility issues by offering and facilitating their participation through alternative modes of data collection and/or assistive technologies, as feasible and available.

2.3. Demographic questions

1. For surveys of the general population, the data from the demographic questions (along with the recording of geographic location) should allow comparison with official Statistics Canada population data for the purpose of non-response analysis, and facilitate the comparison of results among Government of Canada public opinion research studies. (See section 14.2.5 for further detail on non-response bias.)
2. The standard set of demographic questions below must be included in all telephone surveys of the general population, unless a convincing argument is made that the research objectives are better served by excluding one or more of them. This does not preclude the use of additional demographic questions.
3. When surveying business or organizational populations, these standard demographic questions should be included whenever relevant, such as when the unit of analysis is the individual rather than the business or organization.
4. The wording used for each demographic question and its response categories must be as provided below, unless a convincing argument is made that particular research objectives require alternative wording. Note that for certain questions (for example, age, income), the response categories may be further broken down if desired, as long as they can be reconstituted to yield the mandatory categories.

   The exception is that for demographic questions that have relatively long lists of response options that proceed linearly (for example age, household income), researchers have the option of modifying the wording of the question to allow the interviewer to instruct a respondent to stop at the category that applies to them. For example, the required wording of a question could be modified by prefacing it with an introduction like: Please stop me at the category that best describes your…(Insert demographic dimension).

5. For screening purposes, one or more of the demographic questions may be administered near the start of the questionnaire, as required. Prior to administering demographic questions at the end of the questionnaire, respondents should be informed of the purpose of the questions, and be reassured that their answers will remain anonymous and confidential.
6. Although the ordering of questions and the preamble wording may be modified, the following demographic questions must be used for telephone questionnaires:

7. Additional demographic questions may be included when required to satisfy the survey objectives. When a demographic question from the list below is included in a survey, the researcher is strongly encouraged to use the exact wording provided for both the question and its response categories, in order to facilitate the cross-comparability of surveys commissioned by the Government of Canada.

2.4. Multi-mode questionnaires

When designing questionnaires for multi-mode surveys, researchers should:

1. Ensure as much comparability as possible in the wording, ordering and presentation of questions and response options across the different survey modes, to facilitate combining and comparing the collected data.
2. Include benchmarking questions for which there are high-quality population data available to facilitate the detection of any mode biases, for example, demographic, behavioural or attitudinal questions that may correlate with key measures in the survey.

4.1. General

1. Researchers must make every reasonable effort to gain, retain and increase public confidence in their own organizations, and in public opinion research conducted for the Government of Canada.
2. Researchers must act in a manner that respects survey respondents and ensures their rights are protected as they relate to participation in Government of Canada survey research. Interviews must be conducted free of embarrassment and in an environment of trust and goodwill.
3. The researcher must take all reasonable steps to ensure that respondents are not in any way harmed or embarrassed by any interview, and that they are not in any way adversely affected as a result of it. Researchers must address sensitive subject matter in a way that will minimize discomfort and apprehension for both respondents and interviewers.

4.2. Respondent rights and protections

4.3. Avoiding nuisance or harassment

1. Using the respondent’s local time, telephone interviewing can be conducted only from 9:00 to 21:00 Monday to Friday, 10:00 to 21:00 Saturday, and 12:00 to 21:00 Sunday and statutory holidays, except when a pre-arranged interview is requested by the respondent outside these hours.
2. There must be no or minimal (one-second) pause before the interviewer acknowledges that a potential respondent has answered the telephone.

4.4. Use of interactive voice response

The Government of Canada usually wishes to engage Canadians in a personal manner when conducting telephone interviewing, rather than using an automated approach.

However:

1. Interactive Voice Response (IVR) surveys can be a suitable alternative data collection method when the specific information to be collected is essential for making important decisions and cannot be obtained effectively through other means. For example,
   1. When the opinions of a hard-to-reach (low-incidence) group are critical to the research objectives, and the very high call volume made economical by IVR is likely to markedly increase participation from members of that group in the survey.
   2. For time-sensitive surveys where the field duration is very short.
   3. As part of a mixed-mode design, for example, when respondents identified through another survey mode are directed to an IVR, or a design in which there is switching between a live interviewer and IVR.
   4. When there is significant risk of social desirability bias, and for recruitment of participants for other phases of research (for example, to complete a longer online survey).
   5. When respondents have agreed beforehand to this method.
2. When IVR is used, the survey introduction must include the same information required for interviews conducted by live interviewers (active offer in English and French, sponsor, purpose, researcher, participation is voluntary, assurance of confidentiality, etc.)
3. As required by the Unsolicited Telecommunications Rules, Part IV, section 4(d), of the Canadian Radio-television and Telecommunications Commission (CRTC), the introduction must provide an email address or postal mailing address and a local or toll-free number at which a representative can be reached, in addition to identifying the survey sponsor and purpose. If the IVR message exceeds sixty seconds, the same information must be repeated at the end of the call.
4. Early in the introduction, respondents must be provided with an easy method to opt out of the survey (for example, by pressing a specific key) so that the call is terminated gracefully and no more calls are made to that number.
5. The same requirements for the time-of-day of calls and delay in acknowledging an answered call (see section 4.3.) apply to the use of IVR.
6. IVR surveys must follow the relevant rules in the Unsolicited Telecommunications Rules of the CRTC.

4.5. Photographs and recordings

Photographs, video and audio recordings collected from survey respondents must be collected, processed and stored as personal data. They can be shared with a client only after respondents give informed consent, with knowledge of the specific purpose for which it will be used and once the data has been stripped of all personally identifiable information (such as through pixelization or voice modification). The client must agree to make no attempt to identify individual identities.

If respondents are asked to make recordings as part of their survey input, they should be given specific limited tasks (for example taking images of objects) that do not have the potential to capture the personal data of others.

5.1. General

1. Researchers must take special care when interviewing children and young people and respondents who are otherwise vulnerable. (A "child" is defined as "under the age of 13" and a "young person" as "aged 13-15".) The welfare of children, young people and vulnerable persons must be the researcher’s primary consideration. These respondents must not be disturbed or harmed by the interview experience. Sensitive topics or issues that could upset children and young people (for example, relationships with other children or with parents, sexual activities, use of drugs or alcohol) or vulnerable persons (for example, social embarrassment related to the source of their vulnerability) must be dealt with using special care, to minimize any discomfort or apprehension.
2. All explanations related to data protection, privacy policy, adult consent and other notices must be capable of being understood by children.
3. In addition to these standards, researchers must observe all relevant laws specifically related to children and young people.

5.2. Consent of responsible adult

1. When interviewing a child or young person under 16, interviewers must obtain the consent of a responsible adult (parent or guardian) before the child/young person is approached for an interview. Sufficient information must be provided to this responsible adult to enable an informed decision about giving such consent. In particular, the researcher must specify the nature of any potentially sensitive questions when seeking parental/guardian consent for the child’s/young person’s participation in the research. The name of the responsible adult giving the consent must be recorded, as well as that person’s relationship to the child/young person.
2. The responsible adult’s consent allows the interviewer only to invite the child/young person to participate. The child/young person must be given an opportunity to agree or refuse to take part in the survey. To that end, the researcher must inform the child/young person of the purpose of the research in terms that the person is likely to readily understand.
3. When conducting telephone interviews, it might be difficult to determine with certainty the age of the respondent and whether or not the consent of a responsible adult is required. The researcher must make every reasonable effort to comply with the requirements set out in these standards.
4. When it is not possible to obtain parental/guardian consent (for example, street kids) and the client deems that the benefits of the research will outweigh the possible harm to respondents (for example, youth engaged in risky behaviour), the client may choose to waive the researcher’s responsibility to follow this standard.

5.3. Adult accompaniment

For studies with sensitive subject matter, it is recommended that interviewers suggest that a responsible adult (aside from the interviewer) remains close – not necessarily in the same room – while the interview is carried out, in the event that the child or young person reacts negatively during the interview and requires attention.

5.4. Collection of personal information

Personal information relating to other people (for example, parents) must not be collected from children. Consistent with the Privacy Act and the Personal Information Protection and Electronic Documents Act, “personal information” means information about an identifiable individual. The researcher may collect the contact information of a parent or guardian for purposes of obtaining parental consent.

6.1. Project briefing

1. Fieldworkers must be given a briefing and/or instructions for each project.
2. The briefing and/or instructions will be the responsibility of a staff member with a full understanding of the project requirements.
3. Briefings and/or instructions should be delivered face-to-face whenever possible; however, other means are acceptable (for example, phone or recordings). Records must be kept to show the content of the briefing/instructions and that all fieldworkers allocated to the project were briefed/instructed. Briefings and/or instructions in writing only should be avoided.
4. The briefing must be of sufficient detail as to ensure that each interviewer is familiar with the questionnaire, the sample and respondent selection procedures, and potential problem areas in the administration of the survey. Interviewers must be familiar with question wording before commencing the interviews. Normal standards of practice include rehearsal interviews and pre-testing.
5. The briefing and/or instructions should also include, as appropriate, the following information:
   1. Purpose of the project.
   2. Sponsor of the research.
   3. How the information will be used.
   4. Means for respondents to verify survey legitimacy (for example, client contact information).
   5. Fieldwork dates.
   6. Sampling procedure.
   7. Quotas to be attained.
   8. Methodology.
   9. Alternative modes of data collection and/or assistive technologies that are available to address respondents’ potential accessibility issues.
   10. Other special requirements of the project.
   11. Instructions/conditions for administration of the questionnaire.

6.2. Call attempts

1. For landline telephone numbers and cellphone numbers, researchers must make a minimum of five call attempts before retiring a number or substituting it with another number. The call attempts must be made at varying days and times over a minimum seven-day period, except when the field period is too short to allow this.
2. Call attempts must be recorded and retained as part of the disposition of the contact sample, so that outcomes rates may be calculated, as per section 7.
3. If an appointment is arranged, every effort must be made to call back the respondent on the agreed date and time.
4. Researchers must make no attempt to call back refusals, except when the prospective respondent indicates that a call back may be convenient at another time.

6.3. Fieldwork validation

6.4. Incentives / honoraria

7.1. General

1. Researchers should apply the definitions below when assigning cases to each case category and when calculating outcome rates.
2. Researchers must calculate the response rate/participation rate for telephone surveys in the manner described below, and include it in the final report that the Government of Canada’s client will submit to Library and Archives Canada, as per the Public Opinion Research Contract Regulations.
3. Monitoring of call dispositions/reasons for non-response must be carried out on an ongoing basis, throughout the entire field period, and this information must be provided to the client upon request.

7.2. Case category definitions

Telephone survey cases should be assigned to four broad categories:

Invalid numbers

These can include only clearly invalid telephone numbers when surveying households, such as numbers not in service, fax/modem, and business/non-residential.

Unresolved (U)

These include all the cases where it cannot be established whether the call was made to an eligible or an ineligible respondent or unit (for example, household). Examples include when a line is always busy, there is no response, or an answering machine or voice mail is reached but it cannot be established that this is not a household.

In-scope non-responding (IS)

These include all refusals, either implicit or explicit, whether at the household or respondent level; early break-offs of known eligible respondents; and other eligible non-respondents (for example, illness/incapacity, language barrier).

Responding units (R)

These include cases (for example, households) who would have participated but who were disqualified by the interviewer (for example, not of the language group targeted; no one 18 years old or older living at this address). It also includes all completed interviews or partially completed interviews that meet the criteria set by the researcher to be included in the analysis of the data.

Unresolved (U), in-scope (IS), and responding units (R) are all included in the broad category of “potentially eligible” cases. However, invalid numbers/cases are not included in the calculation of outcome rates.

7.3. Outcome rates definitions

Researchers must calculate and report the response rate/participation rate as defined below. However, the other rates (cooperation, refusal and contact) are optional; researchers are encouraged to include them in reports when the rates help to understand the causes of low response rates or provide context and insight regarding data quality.

Response Rate/Participation Rate

Total number of responding units divided by the total number of potentially eligible cases (including those whose eligibility could not be determined).

Response Rate/Participation Rate = R/(U + IS + R)

The term “response rate” pertains only to probability sample surveys and censuses, while the term “participation rate” pertains only to non-probability sample surveys.

Cooperation Rate

Total number of responding units divided by the total number of known eligible cases (excluding those whose eligibility could not be determined).

Refusal Rate

Total number of refusals (including break-offs) divided by the total number of potentially eligible cases (including those whose eligibility could not be determined).

Contact Rate

Total number of contacts that were made successfully (regardless of whether or not a contact was made with the targeted respondent, for example, when a family member who does not qualify intercepts the call) divided by the total number of potentially eligible cases (including those whose eligibility could not be determined).

7.4. Outcome rates for multi-mode surveys

When calculating outcome rates for mixed-mode surveys, researchers should calculate the outcome rate for each mode or stage of sampling, and if feasible and appropriate, calculate a single aggregated outcome rate.

9.1. Developing code frames

1. The initial code list/frame must be developed based on a systematic review of a minimum of 10% of open-ended responses and 50% of partial open-ended responses, where a frame does not already exist. The initial code list/frame must be provided to the client for approval.
2. The researcher must ensure that coders working on the project are fluent in the language of the collected data that they are coding, and provided with instructions and training, which must include, as a minimum:
   1. An overview of the project.
   2. Identification of questions or variables to be coded.
   3. The minimum proportion or number of completed questionnaires used to produce code frames.
   4. Where necessary or appropriate, specific segments of the sample to consider when developing code frames (for example, regions, user or non-user).
   5. Guidelines for the inclusion of codes in the code frame (for example, decisions or rules regarding what must be included or excluded from a given code).
   6. Any use to be made of code frames from a previous project or stage.
   7. Any other requirements or special instructions specific to the project.

9.2. Coding procedures

1. Where “don’t know” and “no response” options have been used during data collection, these must remain distinguishable from each other during coding.
2. “Other” or catch-all categories must not exceed 10% of responses to any question.
3. Researchers must ensure that there is a systematic method of verifying the accuracy of coding for a minimum of 10% of questionnaires per project, and that the verification must be undertaken by a second person.

11.1. Inferences and comparisons

Researchers must base statements of comparisons and other statistical conclusions derived from survey data on generally accepted statistical practice.

11.2. Analysis records

Researchers must keep accurate and descriptive records of the analysis process for a minimum of three years, to ensure that any analysis undertaken can be replicated at a later date.

11.3. Data analysis verification

Researchers must establish and follow procedures to verify the tabulations and other outputs to ensure the following:

1. Their completeness, that is, that all tables are present as specified, including the results of all reported significance tests.
2. That abbreviations for headings or open-ended responses accurately reflect the full content.
3. That the base for each table is correct against other tables or frequency counts.
4. That the standard breaks/banner points are checked against source questions.
5. That all derived data items are checked against their source.
6. That the figures for subgroups and nets are correct.
7. That weighting is applied appropriately (for example, by test tables).
8. That frequency counts are run prior to running tables, in order to ensure the accuracy of data and to determine base sizes for subgroups.
9. Spelling and legibility.
10. That any statistical analysis used is appropriate and correct, in both its descriptive and inferential aspects.

12.1. Producing standalone data tables

When data are provided to the client, the following must be included, as appropriate:

1. The source question to which the data pertain.
2. A description of any weighting method applied to the data.
3. Clear identification of any subgroups used.
4. The sample bases for each question (that is, the number of respondents who have actually answered the question).
5. The number or proportion of respondents who replied “don’t know” or gave no answer.
6. Both weighted and unweighted bases.
7. A definition and explanation of all variables used in the analysis of the data, such as any significance testing, indexing, scoring, scaling and calculations of means, medians, modes and standard deviations.
8. The types of statistical tests being used and their level of precision.
9. Information on cell suppression and other measures to assure confidentiality.
10. Warnings on results that may be unreliable because of very small sample sizes.

12.2. Electronic data delivery

Researchers must provide the client with a data file containing individual records in the original language of each interview, which has been checked for the following:

1. Completeness (that is, all records are in the file).
2. Inclusion of all appropriate documentation to allow for replication of the data analysis and additional analyses, including, where applicable:
   1. Labelling of the contents of the file, that is, fully labelled variables and value labels.
   2. Identification and description of any computed or recoded variables, and instructions on limitations of use.
   3. Labelled weighting variables and a description of how these were applied.
   4. All personal identifiers per the Personal Information Protection and Electronic Documents Act have been removed from the files (that is, any information that could identify specific individuals).

12.3. Respondent confidentiality

1. To ensure respondent confidentiality, tabulated data must not show demographic or respondent characteristic categories with a column or row total of fewer than 10 respondents.
2. Particular care must be taken when respondent verbatim statements are included in data files.

13.1. Retention of technical data

Researchers must, for a period of three years, maintain the technical data that are not already included in the final report and provide them to the client, if requested. Technical data to be maintained include:

1. The information pertaining to data processing and analysis, such as:
   1. Raw data files.
   2. Code frames.
   3. Project files, including project management information and survey programming files.
   4. Relevant correspondence.
2. Documentation of all changes must be kept when the format, content or layout of the raw data has been changed in any way, for example through editing, cleaning, or recoding the data.

13.2. Retention of personally identifiable information

After their contract ends, researchers should retain personally identifiable information or data (whether provided by the client for sampling or data collection, or collected during the research) for no longer than one year, without a strong rationale agreed to by the client.

13.3. Protection of data / servers

1. Researchers must use up-to-date technologies to protect survey data collected or stored on servers or on portable devices (for example, laptops, USB keys) against illegal or unsanctioned access by third parties (that is “hacking”). The researcher must also control access to all databases on which any data relating to the survey are stored, so that only individuals with the appropriate security clearance are able to access the database, either by using a password or other form of access control (such as biometric controls).
2. Because some jurisdictions allow their authorities, under certain circumstances, to access all data stored on servers located in that jurisdiction (for example, in the United States under provisions of the Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism Act, known as the USA Patriot Act: see the Treasury Board Secretariat’s overview), the researcher must protect data against legally sanctioned access by ensuring that all call centres and interviewers employed in survey data collection be located in Canada. The researcher must also ensure that all databases containing any information related to the survey be stored on servers and back-up servers located solely in Canada.
   1. If the client has first consented in writing, these call centres and/or servers may be located in another country where:
      1. Equivalent protections are given to personal information as in Canada under legislation such as the Privacy Act, R.S. 1985, c. P-21, and the Personal Information Protection and Electronic Documents Act, S.C. 2000, c. 5, and under any applicable policies of the Government of Canada.
      2. The laws do not allow the government of that country or any other entity or person to seek or obtain the right to view or copy any information relating to the survey without first obtaining the client’s written consent.
   2. In connection with giving consent to locating a call centre and/or server in another country, the client may, at the client’s option, require the researcher to provide a legal opinion (from a lawyer qualified in the foreign country) that the laws in that country meet the requirements of subsection 2) a) above, or may require the researcher to pay for the Government of Canada to obtain such a legal opinion. The Government of Canada has the right to reject any request to store survey data in a country other than Canada if there is any reason to be concerned about the security, privacy, or integrity of the data. The Government of Canada may also require that any data sent or processed outside of Canada be encrypted with Government of Canada-approved cryptography, and that the private key required to decrypt the data be kept in Canada, in accordance with key management and storage processes approved by the Government of Canada.
   3. The researcher must ensure that all servers, including back-up servers, on which any data relating to the survey are stored, be physically and logically independent (meaning there is no direct or indirect connection of any kind) from all other databases, unless those databases are located in Canada (or in another country approved by the client under subsection 2) a)) and otherwise meet the requirements of this section.
   4. The researcher must ensure that all data relating to the survey be processed only in Canada or in another country approved by the client under subsection 2) a).
   5. The researcher must ensure that all domestic network traffic (meaning traffic or transmissions initiated in one part of Canada to a destination or individual located in another part of Canada) be routed exclusively through Canada, unless the client has first consented in writing to an alternative route. The client will consider only requests to route domestic traffic through another country that meets the requirements of subsection 2) a).
3. The researcher must not subcontract (including to an affiliate) any function that involves providing a subcontractor with access to any data relating to the survey unless the client first consents in writing.
4. Protection of data from international surveys: When the target population for a survey comprises individuals residing in a jurisdiction outside Canada, researchers must ensure that data be stored in a manner consistent with the relevant laws of that jurisdiction.
5. Protection against physical damage to servers: Researchers must also put in place measures to ensure the physical security of data and servers.

13.4. Temporary storage of data on servers

If the temporary storage of data collected takes place on a server that is operated by another provider, the researcher must place the provider under the obligation to take the necessary steps to ensure that the requirements described in section 13.3 be met. Temporary storage of the collected data on the server must be terminated at the earliest possible time.

13.5. Transmission of data internationally

Before data are sent over the Internet to another country, researchers must check with competent authorities that the data transfer is permissible. The recipient may need to provide safeguards necessary for the protection of the data and ensure compliance with the requirements described in section 13.3.

13.6. In the event of any data breach

In the event of any data breach, the client must be informed immediately, and provided with details about both the nature and the extent of the data breach.

14.1. Regulatory and administrative mandatory requirements

It is a legal requirement for federal institutions to send written reports to Library and Archives Canada (LAC) within six months of the completion of the data collection for all contracted public opinion research.

Pursuant to the collective requirements of the Public Opinion Research Contract Regulations, the Directive on the Management of Communications (Appendix C: Mandatory Procedures for Public Opinion Research), LAC deposit instructions, and PSPC instructions, final reports must be submitted separately in English and French and include the following, at a minimum:

1. On the covering page:
   1. The title of the project.
   2. The name of the research firm that entered into the contract.
   3. The contract number, the contract value, and the award date.
   4. The POR Registration Number.
   5. The delivery date (the date the final version of the report was received by the project authority).
   6. The name of the client department or agency sponsoring the research.
   7. The departmental contact information in the form of a generic email address.
   8. The departmental signature and the Canada wordmark.
   9. For the English report, the statement « Ce rapport est aussi disponible en français. »
   10. For the French report, the statement “This report is also available in English.”
2. Copyright information, including the Government of Canada catalogue number, the International Standard Book Number (ISBN), copyright notice and year of publication, and departmental notice on the rights to reproduce the report.
3. A narrative executive summary, submitted separately in English and French and consisting of, at a minimum:
   1. A statement of the research purpose and objectives.
   2. A summary of key findings, except where the person who entered into the contract is not responsible for the design, development of the methodology and analysis of the research.
   3. A brief description of the methodology used.
   4. A statement as to the extent to which the findings can be extrapolated to a broader audience.
   5. An outline of how the results were used, if possible, and if not, how the information is expected to be used.
   6. A political neutrality certification signed by the research firm.
   7. The contract value of the POR study.
4. Appendices containing:
   1. A full set of tabulated data submitted separately in English and French, which, to ensure respondent confidentiality, must in no case show demographic or respondent characteristic categories with a column or row total of fewer than 10 respondents.
   2. Sample size, sampling procedures and dates of research fieldwork.
   3. A discussion of the potential for bias in the research findings due to sampling errors or non-sampling errors (for example, coverage, non-response).
   4. If applicable, weighting procedures, the confidence interval, and the margin of error.
   5. If applicable, the response rate/participation rate and method of calculation.
   6. The research instruments in all languages and, if applicable, the test material in all languages in which they were used and tested.
   7. All other information about the execution of the fieldwork that would be needed to replicate the research initiative.

14.2. Additional reporting requirements

In addition to the reporting requirements described in 14.1, the following minimum details must be documented in the project report. These allow the reader to understand how the research project was conducted and the implications of its results.