Overview and workshop pack

An introduction to online research methods
Thursday 28th June 2007
University of Leicester

Workshop Report

The aim of the workshop was to introduce participants to a selection of online research methods. The workshop covered theoretical, methodological and ethical issues and involved a lot of hands-on-activities. The structure of the day is presented below:

Observations of the day and participants feedback

The introductory session worked very well. The group gelled well together, this was helped by the opportunity for them to introduce themselves and describe their interests.

The morning session focused on online questionnaires, covering theory and opportunities for hands on activities. The participants felt this session gave them a clear picture of the issues surrounding online questionnaires:

 “Designing an online questionnaire is not as simple as it seems there are a lot of issues to be considered”

The opportunity for hands on experience working with the Bristol Online Survey was identified by many participants as a highlight of the morning session.

The afternoon session focused on online interviews covering the theory and presenting an illustrative case study. A scenario was presented and participants were given the opportunity for hands on experience of an online interview. Working in groups one member of the group was given the role of online interviewer and others adopted the roles of interviewees using the scenario that had been presented. Hands on experience really gave participants a taste of online interviews, they all enjoyed the role play and also realised;

“… that online interviews can be very difficult and require great skill”.

“The difficulty of dealing with the numbers of people online and establishing rapport. The necessity of up front info about the research. The speed of questions and answers”.

The “Speed of response when interviewing more than one person at a time – it is essential to pilot and practice interviews”.

The final session of the day focused on the important issue of ethics. Participants worked in groups using scenarios to consider the key issues. They found the scenarios a very useful method for discussing the issues, “Case studies useful to link theory to practice” . The session highlighted:

“The complexities involved and deep thinking involved before undertaking any online methods”.

The feedback from participants on the workshop was very positive; some of their comments are presented below:

“nice mix of information giving and practical work, quite a lot of info in a short space of time”

“Hands on practice on designing the web questionnaire. Having experienced convenors and some experienced participants at the workshop does help in my learning”.

“Really enjoyed practicing online interviews, it was fun but also great to see how it would work and potential complications”.

“A very good intro”.

“Perfect day”.

 

Julia Meek
External Evaluator

 

An introduction to online research methods
Friday 12th October 2007
University of Manchester

Workshop Report

The aim of the workshop was to introduce participants to a selection of online research methods. The workshop covered theoretical, methodological and ethical issues and involved a lot of hands-on-activities.

Participants attending the workshop came from a range of subject disciplines, for example education, social policy, consumer research and health psychology, to name just a few. Most participants were involved in research either as a postgraduate student or research associate/fellow. The majority of participants described their experience of online research methods as basic, only a few had used online methods as a researcher.

The workshop presenters were Dr Clare Madge and Dr Tristram Hooley. The day was spilt between presentations, hands on sessions and discussions. All participants received copies of the ‘Workshop pack’ which contained handouts of the presentation slides, guidance notes on creating an online survey, information about the ‘Exploring Online Research Methods’ website and the new ‘Online Module’. The structure of the day is presented below:

Observations of the day and participants feedback

An initial problem experienced by several participants was actually finding the workshop! Roadwork’s made finding the building difficult as the expected entrance road was blocked, which resulted in some participants arriving a little late. However, once they had arrived they experienced a very interesting day.

During the introductory session the presenters outlined what would be covered during the course of the day. There was also an opportunity for participants to describe their roles and interests in online methods.

The introductory session on online questionnaires was conducted in the seminar room before moving to the computer room (which unfortunately was a rather long walk from the seminar room) for participants to get some hands on experience of online questionnaires. This led into a discussion on the advantages and disadvantages of online questionnaires. Participants then experimented with the Bristol Online Survey (BOS) to design their own online questionnaire. Some participants commented that they would have liked more time to complete their questionnaire. Unfortunately, this was not possible in the time available. However, participant’s access to BOS was extended for a few extra days to enable them to work on their questionnaire in their own time. In their feedback forms participants were asked what was the ‘best thing’ about the morning session, a selection of their responses are :

“Discussion of advantages and disadvantages of different types of online questionnaire – it helped me to think more about this more critically.”

“Hands on activity – it was really useful to consider how questions can be set online.”

“Being signposted to useful resources and introduced to Bristol Online survey. It was really useful to have practical experience of using this.”

The afternoon session focused on online interviews covering the theory and presenting an illustrative case study. A scenario was presented and participants were given the opportunity for hands on experience of an online interview, using Adobe Connect. Working in groups one member of each group was given the role of online interviewer and others adopted the roles of interviewees, using the scenario that had been presented. The hands on experience really gave participants a taste of online interviews, they all enjoyed the role play and also realised;

“That you need to experience an online interview as a participant before trying it out as an interviewer – to understand the problems that participants may have.”

In their feedback forms participants were asked what was the ‘best thing’ about afternoon session on online interviews; participants commented:

“Well presented session with practical experience. Lots of very useful information.”

“The fun we had !”

“Experiencing an online interview.”

The final session of the day focused on the important issue of ethics. Participants worked in groups and discussed the scenarios they had been given to consider key issues. In their feedback forms participants were asked what was the ‘best thing’ about the session on ethics their responses were:

“Discussion was useful to get different ideas and identity subtlety of issues.”

“The lecture – introducing these issues.”

“Opportunity to discuss scenarios covering difficult ethical issues.”

“The complexity and necessity of it.”

The workshop was very successful. The only negative comments from participants focused on the venue, that it was initially difficult to find and they had to move a long distance between rooms.

“Different venue – computer room too far away from seminar room . Morning break earlier.”

“A break in the morning. Computers in the same room.”

“Venue not ideal – too far between computer and lecture room – wasted time.”

Apart from these comments the feedback from participants on the workshop was all very positive.

 

Julia Meek
External Evaluator

 

An introduction to online research methods
Friday 13th February 2008
University of Cardiff

Workshop Report

The aim of the workshop was to introduce participants to a selection of online research methods. The workshop covered theoretical, methodological and ethical issues and involved a lot of hands on activities.

Due to demand for places at this workshop the groups was larger than the usual twenty maximum, with thirty people registering and twenty seven finally attending. Participants attending the workshop came from a range of subject disciplines, for example Anthropology, Health Services Research, Psychiatry, Sociology of Sport, Children's Social Policy and Research. Participants ranged form postgraduate students to senior researchers. The majority of participants had some experience of using online methods as a researcher, most described this knowledge as basic with a few describing their knowledge as intermediate and one described their knowledge as advanced.

The workshop presenters were Dr Jane Wellens and Dr Tristram Hooley. The day was spilt between presentations, hands on sessions and discussions. All participants received copies of the ‘Workshop pack’ which contained handouts of the presentation slides, guidance notes on creating an online survey, information about the ‘Exploring Online Research Methods’ website and the new ‘Online Module’. The structure of the day is presented below:

Observations of the day and participants feedback

The venue was impressive, registration, presentations and lunch in an ornate boardroom with a computer room a short walk along the corridor. The help and support the workshop presenters received form the support staff at Cardiff was also impressive. Ranging from a representative being available to register workshop participants; to technical support opening a port in the firewall to enable the afternoon’s online synchronous discussion to take place.

The day began with an introduction. Participants were then asked to work in small groups (3 or 4) to discuss the advantages and disadvantages of online research methods. Each group fed back one key point which was discussed. This led into a discussion on the advantages and disadvantages of online questionnaires.

Then participants took a short walk to the computer room for the technical introduction to online questionnaires and presentation of usability issues. Working in groups of two’s or three’s using a scenario participants discussed potential questions that they might include in an online questionnaire. Participants remained in these groups to experiment with the Bristol Online Survey (BOS) and design their own online questionnaire.

In their feedback forms participants were asked what was the ‘best thing’ about the morning session, a selection of their responses were:

“Bringing all the points/ information together in one place. Providing links to tools/sites, particularly the Leicester site where I can go back and look at the online support”

“Detailed information on structure of online questionnaires. Some insightful points were highlighted”

“Chance to have a go at designing an online questionnaire”

“Opportunities to explore using and designing an online questionnaire. Good that tutoring staff were clearly talking from personal experience which enabled them to personalise the methodology”

The afternoon session focused on online interviews covering the theory and presenting an illustrative case study. Working in groups one member of each group was given the role of online interviewer and others adopted the roles of interviewees. The interviewers were briefed on their role before the afternoon session began. This enabled them to think about the role of the interviewer as they listened to the introduction to online interviews. A scenario was presented and participants were given the opportunity for hands on experience of an online interview, using Adobe Connect. The hands on experience really gave participants a taste of online interviews, they all enjoyed the role play and also realised the;

“Difficulties around managing an online (synchronous focus group)”

When asked for feedback on the ‘best thing’ about the afternoon session, a selection of their responses were:

“Great experience of participating in an online focus group highlighted some of the practical issues and reinforced value of looking at online in the same way as doing real time research”

“Trying out online focus groups to get real understanding of how it might feel from a participant’s perspective”

“To learn more about this method, hands on activity was great”

The final session of the day focused on the important issue of ethics, the group were introduced to key issues and pointed to websites where they could find out more information. The participants then were given one of three case studies presenting fictional research project seeking ethics approval. The participants working in groups of three’s or four discussed the case study they had been given and then fed back to the wider groups with their thoughts/comments. This activity worked well all groups made good observations, which the presenters used to reinforce key issues. In their feedback forms participants were asked what was the ‘best thing’ about the session on ethics their responses were:

“Useful food for thought”

“Raised questions which I hadn’t previously considered”

“Thought provoking”

“Space for discussion and debate”

“Thinking about ethics of collecting sensitive data”

This was a very successful workshop, the presenters dealt effectively with a large group. In their feedback forms the majority of participants commented that they would be using online research methods in their own research or to support students.

 

Dr. Julia Meek
External Evaluator
25/02/08

 

An introduction to online research methods
Friday 1st July 2008
University of Leicester

Workshop Report

This report is based on the feedback provided by the participants on their paper based feedback forms. Twenty four people were due to attend the workshop. Seventeen did attend and sixteen evaluation questionnaires were completed and returned.

Participants feedback

The participants were asked to note the main issues that stood out in the morning session; the importance of reflecting on the use of online methods and questionnaire design were highlighted.

"The design of the questionnaire is incredibly important – I think this could be a course in itself."

"That using online methods should be thought about carefully, not just as a quick, easy and cheap way of doing research."

The opportunity for hands on experience working with the Bristol Online Survey was identified by many participants as a best thing about the morning session.

"Having a go at creating a questionnaire using BOS. Also I like the way that both Jane and Clare were very responsive to questions and discussion."

"Trying BOS. Highlighting the issues about Qs."

"Trying out and experimenting on how as online question is devised."

When asked to highlight the worst thing about the morning session, limited time was mentioned and the fact BOS was not very user friendly. One respondent would have liked feedback on the questions they wrote in BOS, another would have liked to send the questionnaire to others and get some feedback to see the reporting tool. Although both these suggestions might be desirable from a participant’s perspective they would not be feasible within the already constrained time limits of the workshop.

"Limited time – always a difficulty!"

"Not trying out the questionnaire, that is sending it to others and receiving responses to have a feel of how responses looks like."

"Working with the software (not very user friendly)."

"We did not get any feedback on the questionnaires that we wrote in BOS."

The main issues that stood out for participants in the afternoon session were the importance of interview technique and building rapport in online interviews.

"Creating an online rapport and the challenges around this."

"How online interviews could produce a very different response and questioning style etc than that used in face to face interviews."

"The need for clarity of communication and to be careful about building rapport."

"Possibilities created by online interviews."

"It is important to perfect your interview technique before launching into this if using it for rigorous research. It may be easier to use for more casual research."

Hands on experience really gave participants a taste of online interviews, many highlighted the practical session as the best thing about the afternoon session.

"Getting the opportunity to conduct an online interview / and the intro to adobe connect."

"Highlighting the importance preparing for the interviews, setting ground rules, importance of pace, dealing with sciences."

"The practice of online interviewing – very thought provoking."

"Actually trying it out ourselves."

"Having a go on online interview."

"It was easier that I thought to grapple with the software! I’d use it again."

"Practical session – especially briefing and feedback."

When asked to note the worst thing about the afternoon session the majority said “Nothing”. Others mentioned limited time. Two mentioned the content of the interview, one said they felt uncomfortable and the second noted that the topic should be one relevant to everyone.

"I found conducting the interview uncomfortable (because of the content of the interview) but that’s not a criticism just part of the process of research."

"Having to finish so quickly when interview hardly under way."

"Perhaps not quite enough time for the practical session – not enough time to develop a real sense of the issues."

"Might have been easier to have chosen a topic relevant to everyone – might have made the situation more realistic e.g. public transport."

The final session of the day focused on ethics, when asked what were the main issue that stood out participants highlighted; privacy, dilemmas, the importance to considering the issues.

"Dilemmas. No easy answers to ethical questions."

"Ethical pluralism."

"Confidentiality/anonymity."

"Privacy, confidentiality and issues related to online research."

" Not to make assumptions. To consider the issues in relation to online research as opposed to the historical route – something to reflect on further."

When asked to highlight the best thing about the final session of the day the opportunity for discussion, references, examples and tour of the website were highlighted.

"Examples put forward for discussion."

"The information on resources will be very useful."

"Tour through the website, especially the response tool."

"Detailed walkthrough and the website."

"Opportunity for some discussion."

"Discussing the issue."

When asked to highlight the worst thing about the final session of the day lack of time was the issue highlighted.

Participants were asked to make suggestions for improving the workshop; four suggested making it longer and three of those mentioned a two day format. On participant suggested sending the ethical scenarios in advance to enable participants to become familiar with them prior to the workshop.

The participants were asked to rate the workshop, on a 1 to 5 scale, where 1 was not satisfactorily and 5 was very satisfactory. Nine of the participants rated the workshop as 4 and seven rated it as 5 very satisfactory.

The participants were asked to rate the administrative arrangements for the workshop, on a 1 to 5 scale, where 1 was not satisfactorily and 5 was very satisfactory. Eight of the participants rated the workshop as 4 and eight rated it as 5 very satisfactory. Two made comments on the administrative/room arrangements.
"Moving between rooms would have been better if room was locked for people to leave bag in if travelled up t he night before."
"You might include some suggestions for those staying over night – e.g. Ibis Hotel round corner from railway station is handy/ reasonable but there may be other accommodation to know about."

Participants were asked how they thought they might use the information from the day in the future. The majority mentioned that they would either use what they had learnt in their research or consider using it in their research.

All agreed for me to get in touch in six months time to find out how they had used what they learnt and fifteen or the sixteen said they would like to receive further information about the online module.

Julia Meek
16/09/08