Standards and Guidelines for Data Collection

This section details the standards and guidelines related to the following aspects of in-field procedures:

Required Notification to Potential Survey Respondents

The Panel considered the following MRIA standards specific to conducting research using the Internet, as detailed in the MRIA's Code of Conduct and Good Practice:

  • Respondent cooperation must be a voluntary and informed choice
  • Researcher's identity and list sources must be disclosed
  • Respondent's anonymity must be protected
  • Privacy
  • Interviewing children and young people

There was a consensus to adopt the MRIA position as is with regard to the standards for the first three topics. These sections have been reproduced in the section titled, Responsibilities of Research Firms to the Public.

The following are comments on modifications made to the two remaining topics.

Privacy

The Advisory Panel agreed with adopting the MRIA privacy standard, supplemented with the sample privacy statement included in the Appendix of the MRIA's Code of Conduct.

In addition to commenting on the MRIA privacy standard, the Panel was asked whether or not the MRIA's privacy statements should be supplemented with the privacy-related guidance in the ESOMAR document Conducting Market and Opinion Research Using the Internet.

The MRIA standards do not list any required elements for privacy statements. However, ESOMAR does list both "standard elements for all privacy statements", and "three major variants" corresponding to three different sampling methods.

The Panel agreed that:

  • It was unnecessary to reproduce the "standard elements" for all privacy statements on the grounds that the MRIA sample privacy statement already conforms with these "standard elements."
  • The "three major variants" should be included to provide guidance on how to apply the MRIA standards to different sampling methodologies, since some specific aspects of the policy will vary by the survey method being used. For reference, these major variants are:

Variant #1

"Surveys where the respondent has, or is in the process of, voluntarily joining a panel for market research purposes"

Variant #2

"Surveys where the research agency has been given or has acquired a list of email addresses in order to send invitations to participate in a survey"

Variant #3

"Intercept surveys where the respondent is selected as a 1 in n sample of visitors to a website"

Interviewing Children and Young People

Overall, there was agreement to adopt the MRIA standards related to interviewing children and young people. Several panel members noted that it is more difficult in an online environment to identify who one is actually surveying (e.g., on a telephone survey, there may be auditory cues as to the age of the person). Additional language was added to the standards to acknowledge this difficulty.

Standards: Responsibilities of Research Firms to the Public

1) Respondent cooperation must be a voluntary and informed choice

Voluntary Participation

  • Survey Respondents' co-operation must at all times be voluntary. Personal information must not be sought from, or about, Respondents without their prior knowledge and agreement.

Misleading and Deceptive Statements

  • In obtaining the necessary agreement from Respondents, the Researcher must not mislead them about the nature of the research or the uses which will be made of the findings. In particular, the Researcher must avoid deceptive statements that would be harmful to or irritate the Respondent.

Use of Survey Information

  • Survey introductions or a survey description to which a link has been provided must assure Respondents that data will be collected only for research purposes. Any other purpose, such as rectifying a specific customer complaint, must have the proven express consent of the respondent. Researchers must not under any circumstances use personal information for direct marketing or other sales approaches to the respondent.
  • Commercial researchers must not use respondents contacted in the course of conducting GC online surveys to build their own access panels.

Duration of the Online Survey

  • For surveys completed on-line, respondents should be informed, at the beginning of the survey, about the length of time the questionnaire is likely to take to complete under normal circumstances.

Email Invitations to Respond

  • Researchers should reduce any inconvenience or irritation their email invitations might cause the recipient by clearly stating its purpose in the first sentence and keeping the total message as brief as possible.

Links to Privacy and Cookie Policies

  • Any links to data protection, privacy policy or cookie policy statements should be given at the start of the questionnaire.

2) Researchers's identity and list sources must be disclosed

Disclosure of the Identity of the Researcher

  • Respondents must be told the identity of the Researcher carrying out the project and given contact information so that they can, without difficulty, re-contact the Researcher should they wish to do so.

Providing Information about Research Agency/Sponsor

  • Respondents must be given the opportunity to find out more about the research agency or sponsor carrying out the study, by giving them the name of the organization together with contact information (postal address, telephone number, agency's website or email address) or a registration number and the MRIA's toll-free telephone number for any research registered in the MRIA's Research Registration System. A corresponding hyperlink is recommended for this purpose.

Disclosure of Client

  • For customer database surveys (i.e., surveys based on client-provided lists), the identity of the Client must be revealed.

Disclosure of List Sources

  • Where lists are used for sample selection, the source of the list must be disclosed. Researchers should ensure that lists are permission-based for research purposes and that the data are current.

3) Respondent's anonymity must be protected

Protection of Respondent Anonymity and Use of Information

  • The anonymity of Respondents in consumer research must always be preserved unless they have given their informed and express consent to the contrary. If these Respondents have given permission for data to be passed on in a form which allows them to be personally identified, the Researcher must ensure that the information will be used for research purposes only, OR, if requested, to rectify a customer complaint. Such personally identified information must not be used for subsequent non-research purposes such as direct marketing, list-building, credit rating, fund-raising or other marketing activities relating to those individuals.

4) Privacy

Disclosure of Privacy Policies

  • Canadian organizations that collect personal information are required by law to have a privacy policy. Marketing Research and Intelligence Association members carrying out research on the Internet should post their privacy policy on their website, with a Privacy hyperlink from every page of the website. The order and wording of the published privacy statement is a matter for each member to decide according to its specific circumstances.
  • The MRIA Privacy Protection Handbook includes a sample corporate privacy policy. An example of the MRIA privacy statement for Internet research and the variants, depending on the sampling methodology follows at the end of this section.

Respondent's Email Address is Personal Information

  • A Respondent's email address is personal information and must be protected in the same way as other identifiers.

Disclosure of the Use of Cookies, Log Files or Software

  • Researchers must have a readily accessible policy statement concerning the use of cookies, log files and, if applicable, software. This statement may be either included in their privacy policy or it may appear in a separate document. Software must not be installed on Respondents' computers without their knowledge or consent. In addition, Respondents must be able to remove the Researcher's software easily from their machines (e.g., for Windows users, the software must appear in the Add/Remove Programs folder in their Control Panel).

Deletion of Respondent's Record

  • Respondents are entitled to ask that part or all of the record of their interview be destroyed or deleted and the Researcher should conform to any such request where reasonable.

5) Interviewing children and young people

General

  • Children may be familiar with using the Internet but research has found them to be naive and trusting, happily disclosing information about themselves or their households without realizing the implications of doing so. Parent groups, consumer groups and legislators are particularly concerned about potential exploitation of children on the Internet and it is for this reason that guidelines place greater burdens on Researchers than would be the case in adult research. While validating respondent identity and age can be a challenge in online research, it is important that researchers make every effort to do so.
  • A "child" is to be defined as "under the age of 13" and a "young person" as "aged 13-17."

Observation of Laws and National Codes

  • Researchers must observe all relevant laws and national codes specifically relating to children and young people although it is recognized that the identification of children and young people is not possible with certainty on the Internet at this time.

Conformance to Industry Guidelines

  • Researchers must use their best endeavours to ensure that they conform to the requirements of this guideline, for example by introducing special contacting procedures to secure the permission of a parent, legal guardian, or other responsible adult before carrying out an interview with children under 13. Where necessary Researchers should consult the MRIA for advice.

Adult Consent

  • Permission of a responsible adult must be obtained before interviewing children under the age of 13 years.

Consent

  • Researchers must ensure that the principle of consent is met, so if Internet research is conducted, special measures must be taken to ensure verifiable and explicit consent.

Process for Obtaining Consent: Online Panels or Other Approved Lists

  • In cases where interviews with children of adult online panelists or children of other online list members are desired, the following measures must be implemented.

    The email invitation to the adult panelist or list member must contain the following:

    1. A notice stipulating that the online survey is intended for the child within the household
    2. Name and contact details of the agency/agencies
    3. The nature of the data to be collected from the child
    4. An explanation of how the data will be used

Process for Obtaining Consent: Recruiting Children from Websites

  • In cases where children are being recruited from websites, the following measures must be implemented:
    1. For websites aimed at children, a notice to children, informing them of the requirement for adult consent must be shown at the beginning of the survey. This notice should be clear and prominent and must include an explanation of the subject matter and nature of the research and details of the agency undertaking it, with contact information. To obtain consent, the notice must request the adult's contact information (e.g. email address). It must also refer to the fact that consent will be verified.
    2. Questionnaires on websites aimed at children must require a child to give their age before any other information is requested. If the age given is less than 13 years, the child must be excluded from giving further information until the appropriate consent has been obtained.
    3. For websites aimed at adults, a notice to parent or guardian, seeking their consent for their child to be asked to participate in the research, must be posted on the website. This notice must include:
      1. A heading explaining that this is a notice for parents
      2. Name and contact details of the agency/agencies and the name of the Client (if the Client agrees)
      3. The nature of the data to be collected from the child
      4. An explanation of how the data will be used
      5. A description of the procedure for giving and verifying consent
      6. A request for a parent's contact email address, address or phone number for verification of consent

Parent Contact Details

  • It is permissible to ask children to provide contact details for their parents in order for consent to be sought as long as this purpose is made clear in the request for information.

Acceptable Forms of Consent for Classic Research

  • Where personal information collected from children will only be used for classic research purposes and no personal data will be passed on for any other purpose, a return email from parent or guardian giving their consent is acceptable, as long as additional steps are taken to ensure that the consent actually came from a parent - for example, following up with an email, letter or phone call.

Situations When Parental Consent is NOT Required

  • Prior parental consent will not be required to:
    1. Collect a child's or parent's email address solely to provide notice of data collection and request consent.
    2. Collect a child's age for screening and exclusion purposes. If this screening leads to the decision that a child does qualify for interview, parental consent must then be sought to continue with the interview.

EMails to Children

  • Email communications must not be addressed to children without verifiable and explicit prior consent.

Types of Information Collected

  • Personal information relating to other people (for example, parents) must not be collected from children.

Sensitive Questions

  • Asking questions on topics generally regarded as sensitive should be avoided wherever possible and in any case handled with extreme care.

Policies Must be Understandable

  • All data protection, privacy policy, consent and other notices must be capable of being understood by children.

Sample privacy policy statement and three major variants

MRIA Sample Privacy Policy Statement

NameOfCompany would like to thank you for taking part in this Market Research survey about GeneralDescriptionOfTheSurvey. We are not trying to sell or promote anything. We are interested only in your opinions. The answers you give us will be treated as Confidential unless you have given your consent to the contrary. In the relatively few instances where we ask you for permission to pass data on in a form which allows you to be personally identified, we will ensure that the information will be used only for the purposes stated.

We will not send you unsolicited mail or pass on your email addresses to others for this purpose. As with all forms of marketing and opinion research, your co-operation is voluntary at all times. No personal information is sought from or about you, without your prior knowledge and agreement. You are entitled at any stage of the interview, or subsequently, to ask that part or all of the record of your interview be destroyed or deleted. Wherever reasonable and practical we will carry out such a request.

We use cookies and other similar devices sparingly and only for quality control, validation and to prevent bothersome repeat surveying. You can configure your Browser to notify you when cookies are being placed on your computer. You can also delete cookies by adjusting your browser settings.

We automatically capture information about your browser type for the sole purpose of delivering an interview best suited to your software.

Our web site has security measures in place to protect the loss, misuse, and alteration of the information under our control. Only certain employees have access to the information you provide us with. They have access only for data analysis and quality control purposes.

You can contact us at e-mailaddress@company.com to discuss any problems with this survey. You can find out more about us at www.ourwebsite.com. We are members of the Marketing Research and Intelligence Association and follow their code of conduct for market research.

Variant #1: Panels

Surveys where the respondent has, or is in the process of, voluntarily joining a panel for market research purposes

  • The sign up process - describe the registration process.
  • The panel database - describe information that is stored for panel management, control and sample selection.
  • Frequency of contact - Give some statement of how often or for how long.
  • Password identity system - if it is used describe how it works and the security it offers.
  • Opt in and opt out policies for communications other than surveys such as panel maintenance or reward schemes. State what communications will be sent, which are optional and clarify any potential communications for third parties.
  • Reward - explain any reward scheme and if this forms the basis for a contract.

Variant #2: List of EMail Addresses

Surveys where the research agency has been given or has acquired a list of email addresses in order to send invitations to participate in a survey

  • Source of information - clear statement of where the email address came from or that this will be included in the information given in the survey itself. Also, if a list has been provided, state that the list provider has verified to the research agency that the individuals listed have a reasonable expectation that they will receive email contact.
  • Spamming - will not knowingly send email to people who have not consented to helping in research. May include mechanism for removing your name from future surveys or notifying the provider of the email list.
  • Password identity system - if it is used describe how it works and the security it offers.
  • Stop and start interview process - if this is possible explain how, and any information stored to allow it.

Variant #3: Intercept Surveys

Intercept surveys where the respondent is selected as a 1 in n sample of visitors to a web site

  • Explain intercept technique - random selection.
  • Password identity system - if it is used describe how it works and the security it offers.
  • Stop and start interview process - if this is possible explain how, and any information stored to allow it.
  • Invisible processing - describe any invisible processing used to make the intercept or re-direct respondents to the survey.
Top of Page

Use of Email for Identifying or Contacting Potential Survey Respondents

The standards recommended by the Advisory Panel are closely modeled on the MRIA standards regarding the use of email for identifying or contacting potential survey respondents.

The MRIA standards use the terminology of "consumers" and "business-to-business". The following are examples of how these terms are to be interpreted in a public opinion research context:

  • "Business-to-business research": This includes surveys of businesses, but also could include other types of organizations - e.g., NGOs (Non-Governmental Organizations), other government organizations, and so forth. This research would also include surveys of professionals on subjects relevant to their profession. Examples of professionals who might be surveyed include self-employed entrepreneurs, small office/home office workers (SOHOs), scientists, doctors or nurses, and so forth.
  • "Consumers": This includes the general public, as well as employees in an employee survey.

There were several additions made to the standards as follows:

  • Permission requirements in the case of Government of Canada lists: Some Panelists recommended it be made clear that in the case of GC lists provided to third-party research providers, that the appropriate permission for such usage must exist. Their concerns were that (a) this may not always have been the case in the past, and (b) it is important moving forward for departments/agencies to consider seeking such permission as they gather client information, in order to enable them to use the information for research purposes at a later point in time.

    Standard 1.1.2 was added to address these concerns.

  • Under what circumstances "collecting email addresses from public domains" qualifies as "subterfuge", and when it does not: In the MRIA wording of Standard 3.1, the linkage between "subterfuge" and "collecting email addresses from public domains" triggered some concerns that this language could be overly restrictive. A sentence was added to Standard 3.1 to define the circumstances in which it would be acceptable to collect email addresses from public domains.
  • Meaning of the term "agent": In Standard 4.1 on Data Collection and Recruitment Techniques, the MRIA language referred to "agents" without any accompanying definition. Some Panelists suggested adding clarification of the term. Terminology from a similar standard suggested by IMRO (Interactive Marketing Research Organization) - namely, "The use of Spambots, Spiders, Sniffers or other 'agents' that collect personal information without the respondents' explicit awareness" - was added to address these concerns. Note: Spambots, etc. refer to automated methods or programs which collect email addresses or files from the Internet.

Standards for Use of Email

1.1 Unsolicited -mail

  • Researchers must not use unsolicited email to invite consumers to participate in research. Researchers must verify that consumers contacted for research by email have a reasonable expectation that they will receive email contact for research, irrespective of the source of the list (i.e. Client, list owner, etc.). Such agreement can be assumed when all the following conditions exist:
    1. A substantive pre-existing relationship exists between the individuals contacted and the research organization, the Client, or the list owners contracting the research (the latter being so identified).
    2. Individuals have a reasonable expectation, based on the pre-existing relationship, that they may be contacted for research. In the case of lists provided by the Government of Canada to third-party marketing research providers, there must be legislative authority to provide information for this purpose, or those on the list must have previously given permission for their information to be used for this purpose.
    3. Individuals are offered the choice to be removed from future email contact in each invitation.
    4. The invitation list excludes all individuals who have previously taken the appropriate and timely steps to request the list owner to remove them.

2.1 Business-to-Business Research

  • Unsolicited survey invitation emails may be sent to business-to-business research Respondents provided that Researchers comply with points 3 and 4 in clause 1.1 above, as well as the anti-spam policies of their Internet service providers and email service providers.

3.1 Collection of Email Addresses

  • Research organizations are prohibited from using any subterfuge in obtaining email addresses of potential respondents, such as collecting email addresses from public domains, using technologies or techniques to collect email addresses without individuals' awareness, and collecting email addresses under the guise of some other activity. An exception to the above is that it is acceptable to collect email addresses from public domains for business-to-business research relevant to their professional interests.

4.1 Data Collection & Recruitment Techniques

  • Researchers must not make use of surreptitious, misleading or unsolicited data collection or recruitment techniques - including using spambots, spiders, sniffers or other 'agents' that collect personal information without the Respondent's explicit awareness, spamming, scamming or baiting Respondents.

5.1 Misleading Email Return Addresses

  • Research organizations are prohibited from using false or misleading return email addresses, including spoofing the from label of email messages, when recruiting Respondents over the Internet.

6.1 Opt-out

  • A Respondent must be able to refuse participation in the survey via a suitable option, and to refuse further contact by email in connection with the survey.

Access Panels

This section details the recommendations of the Advisory Panel related to access panels, including:

  • Standards for access panel management
  • Standards and guidelines with respect to use of multiple panels in the execution of a survey
  • Guidelines pertaining to category exclusion when using access panel samples
Top of Page

Standards for Access Panel Management

The Advisory Panel recommended adopting the ESOMAR standards for access panel management with only relatively minor modifications.

The Panel also recommended these standards be applied both to commercial access panels and to any access panels established and operated by the Government of Canada.

Notably, some of the ESOMAR requirements are of the form "have a policy", but ESOMAR does not state what the policy should be. For reference, the following are areas where ESOMAR used this approach:

  • Definition of an active panel member (Section 1.2)
  • Frequency of selection for participation in surveys (3.3)
  • Maximum number of research projects in which a panel member can participate (3.4)
  • Validation checks that a panel member did indeed answer a survey (4.4)
  • Frequency of updating panel member background information (5.2)
  • How long active panelists are allowed to remain on a panel before being removed (5.3)

Panelists supported the ESOMAR "have a policy" approach because of the flexibility this gives in terms of permitting different approaches to dealing with the various issues related to panel management. This resulted in two issues that needed to be addressed:

  • Some Panel members were concerned the use of the word "policy" in a GC context could be misinterpreted. To address this, the word "rule" has been substituted for "policy."
  • In the case of commercial access panels, suppliers' "rules" should be subject to review (e.g., through the RFSO process), in order that the GC can apply quality assessments to the rules. The Panel agreed to the following recommendation:

    The Advisory Panel recommends these access panel management standards with the understanding that, for commercial access panels, the standards which specify a requirement to have a rule but do not specify the content of the rule will be subject to quality assessment in a competitive bidding process.

On the matter of applying category exclusion to samples based on access panels, the Advisory Panel recommended two guidelines: one that encourages consideration of applying category exclusion, and one that suggests at least a 6-month exclusionary period for most types of surveys (see Standard 4.1 for the wording of the guidelines).

Note that the possibility of applying category exclusion is supported in the recommended standards. Specifically, Standard 4.1 requires the following:

Panel owners should keep detailed records for each panel member of:

  • The research projects or surveys for which they have been sampled
  • The nature of the panelist's response to each project or survey

With records of this sort on each panelist, the ability exists to exclude panelists from a survey who have recently been sampled for or completed another survey on a similar topic.

ESOMAR provides the following commentary as to why category exclusion should be a consideration:

  • Response rates and data quality may suffer if panelists are offered repeated opportunities to complete interviews on the same topic. Panelists may show signs of conditioning (repeated interviewing influencing subsequent opinions). These concerns can be mitigated by ensuring the panelists are provided an appropriate number of invitations and including a mix of topics.
  • Providing a mix of different subject matter opportunities, and/or by creating topical sub-panels so that panelists are explicitly recruited as subject matter experts can help response rates and prevent either over use or serial disqualifications.

Standards for Internet Access Panel Management

1.0 Panel Size

1.1

  • The size of the panel should be stated honestly and be based on the number of individuals who have personally joined the panel. Even though the panel owner might have data on other household members in panelists' homes, the panel size should not be calculated to include additional household members who have not actively joined the panel.

    The claimed size of the panel should be based on active panel members.

1.2

  • Panel owners should have a clear and published definition of an active panel member. The size of the active panel will normally be lower than the total number of panelists. The following definition is recommended, but the final definition rests with the panel owner:

    An individual panel member whose set of background variables are complete and up to date (see point 3 below) and who in the preceding 12 months has either:

    1. Joined the panel following procedures set out in the Panel Recruitment section below.
    2. Co-operated by completing an on-line questionnaire (including replying to a screener)
    3. Indicated to the panel owner that they wish to remain a member of the panel

2.0 Panel Recruitment

2.1

  • Panel members must be told that they are a member of a panel and be asked to voluntarily and actively indicate that they wish to be on the panel. A double opt-in recruitment process is recommended particularly where respondents are recruited on-line. This procedure requires the respondent to initiate an approach to the panel owner, the panel owner replies confirming the panel details and double checks that the respondent is who they seem to be and that they do wish to join. The respondent then replies to complete the double opt-in and joins the panel.
  • There may be circumstances when the panel owner already has email addresses for potential panelists, where a simplified opt-in process is acceptable. This would start with an email from the panel owner followed by the panel member replying or visiting a web site to enrol.
  • The panel owner should retain documentary proof (either hard copy or electronic) of each panel member's agreement to join the panel.

2.2

  • The panel owner should retain documentary proof of how each panel members was recruited - from what type of source their name and email address was obtained including, where relevant, the web site from which they were invited to join the panel. In particular, respondents who have been actively recruited through a traditional sampling approach and invited to join the panel should be identified. An overall analysis of type of recruitment source for the active panel or for any sample drawn from it should be available to potential buyers. Panel owners may protect commercially sensitive information about the exact sources used.

2.3

  • The panel owner should have documented procedures for checking that new panel members are not already panel members and thereby avoid duplication in the panel.

2.4

  • On recruitment all panel members should provide a set of basic descriptive information about themselves in order that the representativeness of the panel can be assessed and that targeted or stratified sample can be drawn.

2.5

  • There is not a prescribed mandatory minimum set of background variables that should be recorded about each active panel member. And, the nature of appropriate background variables can be different for different types of panels - e.g., for a general public panel versus a business panel. However, depending on the panel target group, the following variables can have valuable roles in strategies to avoid duplication or clarify individual identity, stratification of samples for research projects, development of sample weights, and conducting analyses of potential non-response bias:
    • Sex
    • Language
    • For a business panel: occupation or position, type of business, business size
    • Level of education
    • Household size
    • Region
    • Location (preferably including the first 3 postal code digits)
    • Age (date of birth)
    • Presence of children in household
    • Working status
    • Weight of Internet usage (hours per week)
    • Type of Internet access (dial-up vs. high-speed, as this can make a difference in ability or willingness to do online surveys)
    • Use of adaptive technologies
  • Wherever possible, the coding of background variables should be compatible with coding systems used by Statistics Canada.
  • Panel owners should have a published list of background variables for which data are available from all panel members.

2.6

  • All panel members must be given a clear and unambiguous guarantee that the access panel is used solely for the purpose of conducting market research surveys (i.e. there will be no attempt to sell products or approach panel members for telemarketing or any other form of marketing activity).

3.0 Project Management

3.1

  • Panel owners should have a clearly defined list of data about panelists that can be used in the definition of a sample to be selected from the panel. This list should include both background variables provided by all panel members and items of panelist history such as recency of selection for a previous project and co-operation history.

3.2

  • Panel owners should provide to clients a clear and honest description of the nature of their panel - the population it covers - and be transparent about partnership arrangements with other panel owners.

3.3

  • Panel owners should have a published rule about how frequently they select individual panel members to participate in surveys.

3.4

  • Panel owners should have a clearly stated rule about the maximum number of research projects or the maximum time commitment for which a panel member will be selected to participate in any given period of time.

3.5

  • Panel owners should maintain a profile of each panelist that can be used to identify specific panelists from the entire panel who should or should not be asked to participate as part of a specific project sample.

3.6

  • Panel owners should have a clearly defined rule on how they reward panelists. The research buyer should be informed of the reward method to be used on their project.

3.7

  • Panel owners should provide a comprehensive response analysis at the end of each survey. This should also include a copy of the solicitation email sent to panel members and the full wording of any screening or introductory questions put to panelists before the main survey started.
  • The following content is recommended for inclusion in a project technical summary:
    • Original invite text(s)
    • Date(s) of invites, and date(s) of reminder(s)
    • Date of closing fieldwork (days in field)
    • Panel used (proprietary or third party and amounts)
  • Response based on the total amount of invites (% or full numbers) per sample drawn (country, questionnaire):
    • % questionnaires opened
    • % questionnaires completed (including screen-outs)
    • % in target group (based on quotas)
    • % validated (rest is cleaned out, if applicable).
  • A short description of how the response and the project relate to the standard criteria - is it less or more than usual, and any peculiarities with the survey.

3.8

  • Panel owners should have documented procedures to ensure that a panel member can answer a survey for which they have been selected, only once.
Top of Page

4.0 Panel Monitoring

4.1

  • Panel owners should keep detailed records for each panel member of:
    • The research projects or surveys for which they have been sampled
    • The nature of the panelist's response to each project or survey
  • The records should be stored in such a way that it is easy to determine:
    • When a panelist was last selected for a survey
    • When a panelist last co-operated with a survey
    • The number of surveys the panelist has completed in any given period of time.

    Guideline 1: Consider excluding panel members who have recently participated in a survey on the same subject (a practice called "category exclusion").

    Guideline 2: When category exclusion is applied, consider using an exclusionary period of at least 6 months. Note that 6 months may not be appropriate in all circumstances - e.g., in longitudinal panel surveys where clients may wish to obtain trend data on a particular topic on a more frequent basis.

4.2

  • Panel owners should calculate regularly and be able to make available to potential clients key data about their panel including:
    • Average number of projects selected for, per panelist per period
    • Maximum number of projects selected for, per panelist per period
    • Average number of complete questionnaires per panelist per period.

4.3

  • Where panel owners adopt an electronic storage system that allows all responses given by a respondent (across many surveys), all data collected exclusively on behalf of a client must be treated as confidential and may not be used in the future on behalf of a second client either in the selection of sample or analysis of data.

4.4

  • Panel owners should have a clear and published rule about validation checks. They should maintain records of any checks they carry out to validate that the panel member did indeed answer a survey.

5.0 Panel Maintenance

5.1

  • Panel owners should regularly remove from the panel non-active members. Each panel member's record of participation should be reviewed regularly and the panel owner should have clearly defined rules for when to remove panelists as non-active based on their cooperation history in the preceding period. Panel members who appear to be inactive because they have not been selected for a survey since the last review of their status should be contacted in order to confirm their willingness to continue as panel members.

5.2

  • Panel owners should have a clearly defined policy on how frequently panel members will be asked to update their background information. This policy should also define whether or not changes in circumstances discovered during survey projects will be recorded in the data record.

5.3

  • Panel owners should have a clearly defined policy on how long they will allow an active panelist to remain on the panel before they are removed and replaced by new panel members.

6.0 Privacy/Data Protection

6.1

  • The panel must be managed in accordance with local data protection laws and, if legally required, should be registered with the appropriate authority. To ensure coherent, cost-effective management of public opinion research through the Government of Canada, institutions must ensure that the principles of fair information practices embodied in Sections 4 to 8 of the Privacy Act, as well as in the Personal Information Protection and Electronic Documents Act, are respected in any public opinion research.

6.2

  • Panel members must, on their request, be informed which personal data relating to them are to be stored. Any personal data that are indicated by panel members as not correct or obsolete must be corrected or deleted.

6.3

  • Panel members must be given a simple and effective method for leaving the panel whenever they choose to. Panel members who have stated that they wish to leave the panel must not be selected for any further surveys and must be removed form the panel as soon as practicable. Further, their email addresses cannot be traded, sold or given away to another research supplier.

Document "The Advisory Panel on Online Public Opinion Survey Quality - Final Report June 4, 2008" Navigation