Peter Chen and S.M. Hinton (1999) 'Realtime Interviewing Using the World Wide Web'

Online Interviewing Pilot Study Overview

Introduction

During development of the online interviewing method a small pilot study was undertaken using a group of undergraduate students to test the technology. The study consisted on a brief online interview (between 30 and 45 minutes) and self-assessment questionnaire, followed by a focus group that discussed the method used. The experiment was run over two days at the Australian National University in 1998.

Method

Four students were randomly selected and asked to engage in an interview and focus group. The aim of the interview was described as "Observations of observed political phenomena". For the purposes of evaluating the method, the interviewees were not informed about the nature of the study until the beginning of the focus group, held on the second day of the study. This deception was seen as necessary as not to bias the study or encourage the participants to focus their attention on the computer, rather, a series of questions on current political events was used as the basis of the interview.

The interview was conducted in this manner:

1. The interviewee was selected and asked to come to a office in which the interview would be conducted;
2. Upon arrival, the interviewee was introduced to the computer to be used in the experiment and instructed on its operation;
3. The interviewer left the room and began the interview from another room using a second computer;
4. At the conclusion of the interview process, the interviewer returned, thanked the interviewee and terminated the process.

At the conclusion of the interview a self-assessment questionnaire was given to all participants to gauge their demographic characteristics and assess their familiarity and attitudes to computers. This survey also contained a realistic set of attitudinal questions regarding current political events, to maintain the initial fiction of the interview and limit the chance of participants from discussing the online interview process in detail, should they meet between the interview and the focus group process.

The final part of the experiment was a focus group consisting of all four interviewees and the researcher. In this interview, the purpose of the research was revealed and the online interview was discussed. This session was video taped with the consent of the group, for later analysis and transcription of key points. The focus group was conduced as a free discussion period facilitated by the interviewer, who served to guide the discussion and follow-up on points raised by the interviewees.

The Sample

The sample for the research consisted of one women and three men. Details are presented in Table 1. This information was generated from a questionnaire presented after the initial interview using the online interviewing process.

#	Gender	Age	Edu*	Own**	Use Comp.	Use WWW	Use Email	Inter***
1	M	41	13yrs	yes	daily	every few days	every few days	yes
2	M	13yrs	71	yes	daily	every few days	rarely	no
3	M	51	14yrs	yes	every few days	rarely	rarely	yes
4	F	40+	14yrs	no	rarely	rarely	never	no

Table 1: Summary of interviewee self-reported questionnaire
 * Education Level (number of years in education)
 ** Own a Computer
 *** Is the interviewee interested in computers and computing

The group can be summarised as mature, with some degree of familiarity with computers, who are engaged in study after (or during) a career in public and private sector employment. The group had mixed reactions to computers, some interested in learning more about them and some unconcerned. All, however, used computers on a relatively regular basis (due to the requirement of university students). One reported having a medical impairment that prevented prolonged use of computers.

Output

At the completion of the interviews, the recorded logs of the sessions were analysed and the amount of output was matched to the interviewees self-reported typing speed and eyesight, as summarised in Table 2, below. Overall, the output of the sessions was modest with an average of twenty-two words per minute during the exchange.

interviewee*	typing speed**	eyesight**	words per minute***	words per hour***
A	poor	above average	20	1200
B	poor	average	23	1380
C	excellent	above average	26	1560
D	poor	above average	19	1140
average:	na	na	22	1320

Table 2: WWW Interviewing Output Speed - Raw Output
 * This does not correspond to the ordering listed in table 1.
 ** From self-assessment questionnaire
 *** averaged over the course of the interview

Notes and Limitations

The main limitations of this study are the size of the sample and the limited variation of conditions in which it was conducted. Each interviewee, within the variation of discussion topics, had a near identical experience during the study, and no additional factors were introduced that may have been valuable (such as the inclusion of a non-physical contact/presence with the interviewees, use of other machines, differential explanations of the process, etc.). As this study was used to explore the method, these limitations were seen as acceptable.