PARTNERSHIPS AND MULTIPLE PANEL MEMBERSHIP
Do you supplement your samples with samples from other providers? How do you select these partners? Is it your policy to notify a client in advance when using a third party provider? Do you de-duplicate the sample when using multiple sample providers?
As the size and capabilities of our panels increases, we are becoming less and less dependant upon other suppliers to fulfill requests.
However, when this situation arises, we rely on our global partner network. We aim to select partners who share our ethics and philosophy on quality and authenticity. We generally look for smaller companies that are situated in the country from which this information is required. On the whole, these companies not only have the local knowhow but are also quick to respond. We appreciate the fact that our clients prefer an open and honest communication. As a result, we notify them upfront if a third party provider would be necessary and would ask for their permission.
Do you have a policy regarding multi-panel membership? What efforts do you undertake to ensure that survey results are unbiased given that some individuals belong to multiple panels?
Panel-overlap is an extremely crucial issue, especially when it comes to multi-sourcing surveys (low incidence, large number of providers working on one project).
Also, when surveys are running badly (long questionnaires, poor quality screenings, boring questions, etc.). This can lead to fast click-through rates, fake answers, false screening information, all of which are caused by learning effects from different panels. There is of course no 100% watertight method of sniffing out multi-panel members but at ODC, we take every effort to keep it to a minimum: Through our on-invitation only method and carefully selected choice of partner.
DATA QUALITY AND VALIDATION
What are likely survey start-rates, drop-out and participation rates in connection with a provided sample? How are these computed?
The start rate (or response rate) is the number of people who start a survey, divided by the total number of invitations sent. The overall ODC response rates are generally at least 35% and can be as high as 85%.
This depends on the country but is of course also influenced by many other factors such as the field time, length of survey, topic (if it is mentioned in the invitation), age of panel, target group specification, etc. In terms of dropout rates: Our average is around 2%-5%. We go to great lengths to keep our response rates high and our dropout rates low. On the one hand we invest time and energy into providing high quality panels which consists of genuine and motivated members. Inactive members are removed so as not to falsify our possibilities. On the other hand, all surveys are tested by our project management prior to release. Our clients are consulted when seen as appropriate. These measures are carried out as an effort to ensure high quality surveys. A survey considered good, is one with a reasonable length and topic, an appealing general layout and one with no or few technical limitations (browser capabilities, software demands such as Java or Flash). These elements can of course, not always be influenced by us.
Do you maintain individual level data such as recent participation history, date of entry, source, etc. on your panelists? Are you able to supply your client with a per job analysis of such individual level data?
Yes, this information is stored and available upon request. Of course personal identifiable information is strictly protected.
Do you use data quality analysis and validation techniques to identify inattentive and fraudulent respondents? If yes, what techniques are used and at what point in the process are they applied?
The validity of respondent's data begins with the double opt-in method.
The "on invitation only" methods and the fact that every member requires a valid a bank account also reduces the risk of "professional" respondents. We use test surveys to establish whether or not the person's registration data is consistent with the answers in the survey. We can also determine how genuine a survey is being processed, based on the click-through rate.
Do you measure respondent satisfaction?
Yes. Measuring and maintaining the satisfaction of our respondents is an integral part of our quality process.
Only a happy and motivated respondent is one we can expect to continue taking part in surveys and providing genuine answers. After every complete, our respondents are asked for their honest opinion about the various aspects of the survey. Our panel support team ensures that all enquiries or complaints from our respondents are promptly answered and solved. Not content to rest on our laurels, in addition, ODC is taking an innovative approach by introducing a so-called Panelist Motivation Score©. Based on our experience and past behavioral patterns of our panelists, we have developed a tool which, by implementing particular mathematical formulas, assumes the theoretical motivation level of each respondent. For example, respondents with a (virtual) low level are favoured for future surveys with a high incidence, thus increasing the chances of completing and in turn, positively influencing the motivation score.
What information do you provide to debrief your client after the project has finished?
Once a project is finished, ODC Services provides its clients with a field report containing detailed information on the project's performance data, such as number of invitations, starters, screenouts, quotafulls, dropouts and completes.
Additionally, we provide information about the panelist's feedback on the survey.
