top

This exemplar is about using data from the Scottish Household Survey (SHousS) to look at factors determining internet use by Scottish adults. It uses interviews carried out in 2001/2002 with data from the "Random Adult" data set from this survey. There were 28 685 respondents in these two years. To find out how data were prepared for analysis click here. To prevent identification of individuals we have modified the data and taken other precautions as described here.

The analyses examine factors that determine which adults in Scotland use the internet and, in particular,

• differences between geographic areas in internet use
• modelling internet use by age and sex of the adult and other factors

Fig 1.1 Internet use by age and gender 2001/02.

top

top

We look at the proportion of random adults who have access to the internet and then at the proportions who spend different numbers of hours per week on the internet.

The weighted percentage for internet using adults was 34%, compared with the unweighted percentage of 31%. This difference reflects the fact that people who live alone (and are thus relatively downweighted in the survey) are older and less likely to be internet users.

We can calculate the percentage internet use by men and women separately. All the packages agreed on the answers for proportions, their design factors and confidence intervals . The proportions also agreed with the results in Chapter 6 of the report of the SHousS. Because of the large numbers the confidence intervals are fairly narrow.

The results below were taken from Stata, but other programs gave very similiar results

We can readily test this out by analysing the data as if it were from another design. The estimate of internet use is the same for all cases as the weights do not change. To find out more about clustering click here.

We can see that there is a substantial design effect due to unequal sampling fractions (line 1). This is made worse by clustering (line 2), improved by stratification (line 3) and allowing for the full design (line 4) gives a DE of 1.48.

Note The design effect of 1.48 for the effect of all the design aspects on this measure is different from the design effect of 1.04 (the quoted design factor of 1.02 squared) quoted in the technical report for this variable. This is being discussed with the survey contractors.

The large increase in the DE due to clustering seems, at first sight, a bit surprising here. Only a part of the sample used cluster sampling (60%, weighted data). But the use of relatively small units (Enumeration Districts) may have had the effect of making them very homogeneous for something like internet use which may be heavily clustered by geography.

top

To investigate differences between groups, such as differences in internet use between men and women a chi-squared test would be the normal procedure.

But the ordinary formula for a chi-squared test needs to be modified to allow for the design of the survey more about chi squared tests. Here are the results obtained for a weighted table of internet use by gender.

Results here, from Stata, give the and the commonest adjusted test which is expressed as an F statistic. The uncorrected chi-squared value here is the value for the weighted table. It would not have a chi-squared distribution if there were no association in the table, because of the weighting and other features of the design. The adjusted test shown here is an F test. Since there is only 1 degree of freedom here the chi-square and F tests are directly comparable. Other packages produce similar results based on slightly different adjusted tests.

Obviously, we did not need either test to show that there was overwhelming evidence that men used the internet more than women. Things are not quite so clear when we investigate the proportion of adults who use the internet for grocery shopping (variable RC7E recoded to GROC so that non-internet users are coded as 'no')). Is this the one area of internet use where women are more frequent users than men? The chi-squared test shows that although women's percentage in the table is higher, this could just be a chance finding.

Chi-squared (X2) tests for larger tables can be used to screen variables for evidence of an association. In the table below there appears to be an association, for internet users, between employment status and the time spent on the internet per week. Presentation here follows recommendations for weighted tables. But some of the bases are rather small, so perhaps some of the associations are just chance.

top

Survey analyses of subgroups can be done in two ways:

1. Subdivide the data and then define the survey
2. Set up the survey and request the analysis of a subgroup

The first one is usually wrong for a stratified sample as it would assume the subgroup was stratified, which was not the case.

But there is an exception when a survey has all its design features (startification, clustering, post-stratification ) carried out within sub-groups. This was true for the SHousS for local authority areas and subsets of the data by local authority can be analysed as if they were independent surveys.

For subgroups that were not designed in this way it is essential to use method 2. Design effects for subgroups that select members from different clusters and strata tend to have design effects closer to 1 than the analyses for the whole survey. The theory section on subgroups explains this in more detail. An example for this survey is in the results by gender in Table 2.3 above although the effect is very modest here. The P|E|A|S code always uses method 2, and exemplar 1 illustrates how design effects for a subgroup can be very different to those for the whole survey. (NOTE ADD LINK HERE)

The SHS was designed to be large enough to give results with good precision at the local authority (LA) level. The design was stratified and post-stratification within each LA.

We can see that internet use varies sharply by local authority and that the aim of the survey, is to get estimates of similar precision by LA , has been met.

LA % use s.e. Design Effect Aberdeen Aberdeenshire Angus Argyll_&_Clyde Clackmannan Dumfries & G Dundee East Ayrshire East_Dunbart East_Loth East_Renf Edinburgh Eilean_Siar Falkirk Fife Glasgow Highland Inverclyde Midlothian Moray North_Ayr North_La Orkney Perth_&_K Renfrewshire Borders Shetland South_Ayr South_Lanark Stirling West_Dunbart West_Lothian 43 37 36 33 30 25 31 30 47 38 44 45 26 34 33 28 38 30 35 31 25 28 31 38 32 36 47 33 32 43 28 35 2 2 3 3 3 2 2 3 2 3 2 1 2 3 2 1 2 2 3 3 2 2 3 2 2 3 3 3 2 3 2 3 1.15 1.53 1.38 1.84 2.28 1.16 1.22 1.61 1.26 1.64 1.19 1.17 1.37 1.29 1.52 1.22 1.32 1.26 1.46 1.76 1.92 1.72 2.72 1.48 1.27 1.94 2.05 1.95 1.73 1.65 1.40 1.43

Figure 2.2 Internet use by LA

We can use the width of confidence intervals to indicate when there is evidence of a difference by LAs. The design effects presented above were calculated by Stata.

There are two different ways we can specify design effects for sub-populations.

1) The presented here is an option in Stata and the default method used in R gives the design effect for taking a random sample from the subpopulation that is the same size as those in the sample. So if we were to the data to interviews in Orkney we get exactly the same estimate and confidence interval but a Design Effect of 2.7. This shows the price we are paying for a clustered design in a rural area.

2) The alternative which is the default Stata option calculates the DE with respect to a random sample of all households for the whole of Scotland. This gives a very small design effect for Orkney and Shetland because these areas were heavily over-sampled. A random sample for all of Scotland would give many fewer interviews to the islands.

top

From the tables above it is clear that there are many factors influencing internet use. A multivariate analysis should shed some light on this. Since internet use is a binary variable, we need a logistic regression model (or something similar). Only R and Stata currently offer this for surveys.

The modelling process is typically long, and we illustrate only a small part of it for each package.

In Stata the analysis looks at the joint impact of household income and urban/rural classification on Internet use. Fitting grouped income first we get the table of odds ratios below for two methods of analysis.

top

Survey design analyses require the data to agree with the sampling design. Problems arose with the data for this exemplar because

1. Two PSUss had addresses that were in two different local authorities

2. Some of the strata included only one PSU.

This was mainly because the PSU identifiers were not supplied with the data file sent to the data archive and so had been little tested. They were obtained from the survey contractors directly. There were several interviews with missing or wrong Mosaic codes. Also, a few less common Mosaic codes had only one interview in a local authority (e.g. remote rural in a largely urban LA).

A lot of programming was needed to fix this.

If the data breaks any of these rules then the programs may do various things, as shown below.

To overcome these problems the data sets for this exemplar have been corrected by:

1. Re-assigning local authority codes in the two PSUs
2. Pooling strata with only one PSU

What happens when a variable has missing values so that a few strata may be reduced to only one PSU?

The same problems can arise, again Stata fails.

top

A comprehensive description of the design of the survey can be found in the Technical Report for the 2001/02 surveys, Scotland's People; Volume 8 on which the summary below is based.

The sampling frame is the post code address file a list from the post office of all addresses in the UK. We are using data from the two years 2001 and 2002. The data set includes 28,685 records of interviews with random adults.

A simple random sample of households was selected in local authorities (LA's.) with densities of 500 or more people per sq km. For the remaining LAs a cluster sample was selected with the enumeration district (ED's) as the PSU. These are fairly small areas that represent one census enumerator's work load. The survey aimed to achieve 11 interviews per PSU. The sampling fractions varied by LA, with larger sampling fractions in the smaller LAs in order to assure a sample size of 500 households in each LA over a two year period.

The SHS sample was first stratified by LA and then by 10 mosaic categories within each local authority. Mosaic is a socio-economic clasifiaction applied at the post code level. For areas with cluster sampling stratification was the commonest mosaic code in the ED was used. The stratification was explicit at the first stage and then implicit, by ordering the units, within each LA. The theory section describes implicit and explicit stratification. The strata in the file are labelled by a combination of Mosaic code and LA. Some strata can contain only a small number of units. If, perhaps due to the selection of subgroups, we are left with only one unit in a stratum problems can arise, as discussed in the section on data checking above. The main data set provided here has the strata merged to avoid these problems.

One adult was selected at random per household. This means that people from larger households have less chance of being selected than those in smaller households. To adjust for this a weight is applied to the random adult data that is proportional to the number of adults in the household. This for households with 1,2,3,4,.,. adults the weights are proportional to 1,2,3,4,.,.

The two largest factors contributing to the weights for the random adult data (IND_WT) are the unequal fractions by local authority and the selection of just one adult per household.

Additional weighting is carried out compensate for differential non-response by LA. These final weights make the weighted survey totals of 'random adults' match the 2001 census population in private households. This adjustment is minor compared to the others. No further weighting is carried out to make the sample match the populations by age and sex.

The justification for this, along with a discussion of the representativeness of the sample can be found in the Technical Report, Scotland's People; Volume 8 The weights span a fairly wide range, from 0.07 to 6.2 as is illustrated in the histogram on the right.

���� Fig. 2.3 Random adult weights

top

We have constructed a data set for this analysis with just those variables we will use in these analyses. The process by which the data set has been constructed and the programmes used to make it from data that is available from the Essex Data Archive are explained in detail for anyone who has an interest in this. The variable to identify the PSUs is not available from the archive, but was provided to us directly by the SHousS team.

Several procedures to anoymise the data have been carried out along the lines described here. This means that the answers obtained from analysing this data set may be very slightly diferent from what would be obtained from the archive data. The variables in the data files are:

NOTE ON VARIABLE NAMES:
Some of the programs we are using are case sensitive (R and Stata)
- In Stata the variables names are all lower case.
- In R they are the elements of a data frame called shs (lower case), but the elements of the data frame are upper case e.g shs$UNIQID.