We measured the stability and speed of the programs using a 2GHz Pentium 4 computer with 512MB RAM running the Windows 2000 operating system with several, typical Office programs running in the background: A mail program (Qualcomm Eudora), an inter browser (Mozilla Firefox), a Word processor (Microsoft Word), a statistical package (SPSS), and Adobe Acrobat. Each CAQDAS was supplied with 2,626 plain text (in the case of MAXqda, which does not allow for plain text: rich text) documents, which ranged from 30 Bytes to 10 Kilobytes in size. All features of the programs were then tried out over a period of several days and the number of crashes were noted down. Since the differnt programs have different ranges of functions, the crash figures are not fully comparable: A program might have a bug in a function some of the other programs might not even offer. Likewise, different versions of Windows might yield very different results. Yet, we feel that a program should be bug free to ensure a smooth working process. Using the standard Windows Task Manager, the CPU time was measured. Finally, we ran simple, Boolean and proximity text searches and measured the time it took to perform these searches.

Except for MAXqda, all CAQDAS frooze at least once, i.e. did not respond to user input after a waiting period of 3 hours. It therefore seems advisable to always create backup copies of the project one is working with. Kwalitan was the least stable program. Most of its bugs did not crash the program entirely, but often just displayed error messages — some of them in Dutch — which require a subsequent restart of the program. While there is little loss of information, this can become quite a nuisance. NVivo, which to be sure is not intended for the amount of data we used in the test, tended to use up so much CPU time, that it quasi frooze, when some functions (such as the display of coding stripes during coding or editing) were performed. Much of the search and coding functions moreover became unusable, once a project surpassed a few hundred files. Below that limit, the program worked fine. ATLAS.ti was considerably more stable, but it too ran into time problems, when searches and codings were performed over several hundred files. Qualrus improved yet again over ATLAS.ti, both in terms of stabilty and in terms of speed for those functions it offered an equivalent to ATLAS.ti. Finally, QDA Miner and MAXqda both were the most stable and the fastest CAQDAS.

1. Additionally, Kwalitan frequently produced error messages, which did not correspond to actual errors.
2. When displaying coding stripes, NVivo frequently freezes.
3. As seaches could not be completed, autocoding time could not be measure.
4. QDA Miner does not show, when coding is finished.
5. Autocoding in Qualrus requires prior definition of segments.