to: Index and Menu

Simple Interactive Statistical Analysis


Go to procedure

Response Rates

Input.

Program presumes that the input consist of positive (integer) numbers without decimals. Categorize the sampled cases in the boxes provided, one can choose to use the CASRO boxes or the AAPOR/ISER boxes, or to use them in combination, as required. The program will sort the cases between AAPOR/ISER and CASRO. Optionally a fraction eligible respondents can be given. If you do not give this fraction the fraction will be estimated for you.

Explanation.

Often response rates in survey research are calculated simply by dividing the number of completed interviews by the number of individuals who were selected to participate in the research, i.e. the sample. However, this method is too simplistic and does not do justice to the complexity of research design, sampling process and the practical difficulties of contacting and assessing potential survey participants. Therefore the Council of American Research Organization (CASRO) proposed a method to better consider the various situations encountered in survey research. This method formed the basis for the development of a standard for the calculation of response rates by the American Association for Public Opinion Research (AAPOR). This standard was further refined by the Institute for Social and Economic Research (ISER). Both the publication by the AAPOR and the publication by ISER are available for free on the internet. The user is advised to download these publications and study them before using this SISA procedure.

Both the AAPOR and the ISER system consist of a large number of detailed categories. Each sampled address, number or case should be coded according to one of these categories. The number of coded cases must sum to the total sample size (N). The detailed categories are summarized into 8 sub-categories, numbered in the SISA procedure on the basis of the ISER system as -1,2,3,4,5,6a,6b,7-. These 8 sub-categories can be summed in turn into the four main CASRO categories, numbered -A,B,C,D- in the SISA procedure. Table one can be used to convert the SISA, AAPOR and ISER coding, whereby it should be considered that the overlap between 6a into 3.2 and 6b into 3.1 is not perfect.

Conversion table for sampled cases final disposition codes:

Name

SISA

AAPOR

ISER

Completed (>80%)

1

1.1

1

Partial Completed (50-80%)

2

1.2

2

Completed (>50%)

A

1.0

 

No-Contact

3

2.1

3

Refusal/Break-off (<50%)

4

2.2

4

Other (language problem etc)

5

2.3

5

Refused eligible

B

2.0

 

Contacted/ Unknown Other

6a

3.2

641, 651, 661, part of 67

Not-Contacted/ Unknown if household/occupied HU

6b

3.1

61-63, 642, 652, 662, remainder of 67

Unknown

C

3.0

6

Ineligible

7D

4.0

7

The eight AAPOR/ISER categories will here be discussed. The user should note that in a research situation sampled cases are not coded according to these higher level codes, they will be coded according to the detailed category codes used in the AAPOR/ISER systems and the result will be summarized to the higher level code.

Output.

Confidence Intervals.
The only thing that is fixed in a response rate calculation is the N, the number of sampled cases. All other variables are empirical variates that will fluctuate between measurements. Therefore, the researcher can ask him or herself the question: "if I would repeat this design in a similar situation, in between what values can I expect my estimates to be?" The procedure gives confidence intervals for the four main CASRO response categories that make up the four main proportions of all sampled cases. For some other measures confidence intervals are also presented.

CASRO response rates.
Response rates according to the CASRO procedure. The third response rate, the actual CASRO response rate, is the most important. In calculating the CASRO response rate it is assumed that the proportion of eligible cases in the unknown cases is equivalent to the proportion of eligible cases in the sum of cases in the sample of which the eligibility or ineligibility could be determined. This proportion is given in the output as the e for CASRO.

Response rates.
The response rates are the proportion of completed interviews in the total number of eligible respondents. The various versions of the response rate take the respondents of whom eligibility could not be determined as all eligible, as all not eligible, or a fraction "e" of the unknown respondents is considered to be eligible. Another question is whether to consider partial interviews as completed.

Cooperation rates.
The co-operation rate is the number of completed interviews in the number of contacted eligible respondents. In the case of the ISER cooperation rate an estimate is considered of the number of contacted eligible persons in the unknown category, in the case of the AAPOR cooperation rate unknowns are not considered.

Contact rates.
The contact rate measures the number of eligible persons which were contacted. ISER considers the number of contacted possibly eligible unknowns; AAPOR does not consider unknowns.

Refusal rates.
The refusal rate gives the proportion of eligible respondents who refused to give an interview. This is the least important rate.

Comment.

Instead of one standard we now have three. Note that the CASRO standard continuous to be at the basis of things and the CASRO response rate should continue to have the focus in any judgement of a research project. The more detailed AAPOR and ISER methods will make it easier to calculate the CASRO rate in a standard way. Both AAPOR and ISER present other measures as well, the co-operation rate, the refusal rate, and the contact rate. Although these rates can provide valuable information they are and continue to be considerably less important than the CASRO response rate! Note that for these additional rates you have to fill in all the eight AAPOR/ISER boxes, for the CASRO response rate you have to fill in the four CASRO boxes only. SISA does not have a preference for either the AAPOR or the ISER system. ISER seems to be more internationally applicable. AAPOR is closer to the original 4-category CASRO system. The major important difference is in the definition of the unknowns, were the split in contacted and not contacted respondents of whom eligibility could not be determined is used in calculating the ISER co-operation and contact rate, but not in the AAPOR system. For the AAPOR system it would suffice to only fill in the total number of unknown cases in the C box, this would not be appropriate in the ISER system. However, the overlap between both systems is considerable and to use both systems side by side, or even in combination, probably does not cause too many discrepancies.

References.

The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. Lenexa, Kansas: The American Association for Public Opinion Research, 2000.

Lynn P, Beerten R, Laiho J, Martin J. Recommended Standard Final Outcome Categories and Standard Definitions of Response Rate for Social Surveys. Colchester, Essex: The Institute for Social and Economic Research, 2001.

TOP of page

Go to procedure


to: Index and Menu

All software and text copyright by SISA