Zetoc Evaluation Report



Ken Eason and Martin Ashby
Loughborough University
November, 2002

Summary

An electronic questionnaire yielded 659 responses from Zetoc users in a wide
variety of institutions. The findings demonstrate that users have a very positive
view of Zetoc. The majority (75%) feel that Zetoc offers them a service they have
not had before which gives them the ability to keep on top of current developments
in their field. The majority of users used the TOC Alerting service to keep up-to-date
with new issues of relevant journals and about half of the sample had searched
the database. Very few ordered full text articles from British Library via Zetoc
preferring to seek full text electronic copies or to find local sources. Most users
made use of other services as well as Zetoc and would value a seamless and
integrated service from search to full text.

About 20% of the sample were knowledgeable and active in using the array of
services Zetoc offers and in finding ways of relating its services to the many other
services available to them. The majority of users, however, had a more restricted
understanding of Zetoc and primarily treated it as an Alert service. This restricted
usage is not based upon a shortage of training and support opportunities or
difficulties in using the service. The service is generally regarded as relatively
simple to use and good support materials are available. The reasons for restricted
usage appear to relate to the array of services that confront users and the limited
time and effort they can devote to understanding and using specific services such as
Zetoc. The questionnaire yielded many suggestions for enhancements to Zetoc
which would enable active users to achieve an even more integrated service. The
problem that remains is how to encourage the passive majority to utilise the
facilities already on offer as well as to exploit any new enhancements that are
introduced.

Acknowledgements

We are grateful to the staff of MIMAS and The British Library for helping make
the questionnaire available to Zetoc users and, of course, to the large number of
users who contributed their responses.

Contents

skip contents menu to main text 1. INTRODUCTION: OBJECTIVES AND STRUCTURE
2. THE SAMPLE
2.1 User Types
3. RESULTS
3.1 The Use of Zetoc
3.1.1 The benefits of Zetoc and the services used
3.2 The Use of Parts of the Service
3.2.1 The Alerting Service
3.2.2 Search
3.2.3. Search History
3.2.4 Document Ordering
3.2.5 Access via bibliographic software
4. THE CONTEXT OF USE
4.1 Awareness, Training and Support
4.2 Usability and Explanation in the Service
4.2.1 Usability of Alert Services
4.2.2. Usability of Search Facilities
4.3 Other Services: Competition or Integration?
4.3.1 Competition
4.3.2 Integration
5. ENHANCEMENTS TO ZETOC
5.1 Enhancements to the set-up process
5.2 Enhancements to searching the database
5.3 Enhancements to alerting services
5.4 Enhancements to access via bibliographic software or Z39.50
5.5 Enhancements to document ordering
5.6 Enhancements to integration with other services
5.7 Enhancements to support available
5.8 Final comments
6. DISCUSSION AND CONCLUSIONS
APPENDIX 1: THE QUESTIONNAIRE
APPENDIX 2: Q002 WHAT IS THE NAME OF YOUR INSTITUTION?

1. Introduction: Objectives and Structure

In August 2002 an electronic questionnaire was mounted to ask Zetoc users about their use of the service, the benefits they were obtaining, the problems they might be experiencing and the enhancements they would like to see in the service in the future.
A total of 659 usable responses were obtained from a wide range of institutions and a good mix of user types. This report documents the responses of these users and is constructed as follows:-

Sections ii to iv provide a detailed account of the results. Appendix 1 gives details of how to obtain a copy of the original questionnaire. The questions were both quantitative and qualitative. Qualitative responses have been categorised and are presented in tabular form. The results are presented in descriptive form with the minimum of interpretation. The final discussion offers an explanation of the usage patterns found by examining the place of the Zetoc service in the information using work practices of the users.

This is an interim report of the Zetoc evaluation. A fuller analysis of the questionnaire data will be undertaken to examine, for example, the patterns of use of different types of user. A follow up study will also be undertaken of a limited number of users to explore in more detail how Zetoc contributes to user work practice and how in the future it might contribute to a more integrated service.

2. The Sample

The questionnaire attracted a very good response from the user population and can be considered a good representation of the range and types of users of the Zetoc service. The sample can be categorised in a number of ways as shown below.

2.1 User Types

The 659 respondents described their primary activity as follows:

primary activity of zetoc users

Table 1. The Primary Activity of Users

The significant groups of users were researchers, librarians, university teachers and postgraduate users. Users had to nominate one primary activity and there is a possibility that some users could have also been in other categories, e.g. health
workers who are researchers and vice versa.

Table 2 summarises the responses by broad sector. Not surprisingly the vast majority of responses were from Higher Education Institutions although the NHS was quite well represented.

response to questionnaire by sector
Table 2. Responses by Sector

Appendix 2 provides a breakdown of the responses by institution. The largest number of responses were received from King’s College London (23), the Open University (22), the University of Cambridge (18), the University of Oxford (15), the University of Nottingham (15) and De Montfort University, Leicester (15).

3. Results

3.1 The Use of Zetoc

3.1.1 The benefits of Zetoc and the services used

The general response to Zetoc was very positive and 75% of users (Table 3 below)
felt that it offered them a service that they had not had before.


response to whether zetoc offers new service

Table 3. A new service?

Two questions were asked to explore what it was that the service provided that was of value. Table 4 below lists all of the benefits respondents found of value. Respondents could chose more than one benefit and the total is therefore greater than 100%. Respondents were also asked to nominate the greatest benefit and this is shown in Table 6.

valued benefits of zetoc

Table 4. Valued Benefits

The ‘other’ category contained 44 responses, categorised in Table 5.

other help zetoc offers

Table 5. Other Ways Zetoc Helps

The dominant feature of the ‘other’ category is ways in which Zetoc helps the respondent provide a service to others, e.g. assisting students (perhaps as a teacher) or providing extra services to library users (perhaps as a librarian).

most valued benefit of zetoc

Table 6. The Most Valued Benefit

Keeping on top of current developments in the respondents own field is the most valuable benefit that Zetoc provides although keeping up to date with related fields is also a significant benefit.

One of the key questions is which parts of the service are providing the major benefits. Table 7 summarises the services in use. Respondents could select any number of services.

key parts of zetoc used

Table 7. Parts of the Service Used

The mechanism by which most users use Zetoc to keep up to date is Table of Contents Alerting. The various search mechanisms are also popular but some parts of the service e.g. document ordering and search history are used by very few users.

3.2 The Use of Parts of the Service

The questionnaire explored how users made use of the different parts of the service and these results are presented below progressing from services in frequent use to those that are used by relatively few users. Since not all respondents made use of the parts of the service the percentages in the results relate to the sub-sample that made use of the part of the service under discussion.

3.2.1 The Alerting Service

Most users set journal alerts and the number of journals they selected are summarised in Table 8

number of journals in alerts

Table 8. Number of Journals Alerts Are Set For

The largest number of users set between 6 and 10 journals but the spread is quite large. It is notable that over 24% of users setting alerts created them for more than 20 journals.


One issue of interest is who actually sets the alerts. Is Zetoc used by individuals working alone or do people co-operate in the use of the service? Table 9 shows the extent to which users help others to set up alerts.

alerts set up for colleagues

Table 9. Setting Alert Lists

This result shows that not many of these respondents set alerts for others which suggests Zetoc is primarily a service set up and used by individuals. This theme is explored through other questions later in this analysis.

The general attitude of users to the alerting service was explored through two questions summarised in Table 10.

attitudes to alerting services

Table 10. General Attitudes to the Alerting Service

The responses show that most users felt in control of the Zetoc Alerting service and understood what it could do for them (or were neutral to the question). Very few of them found the interface confusing. There were further questions about usability and specific aspects users found confusing will be reported in the context of use section below.

To get the best service it is likely that users will have to refine their initial alert set up in order, for example, to eliminate messages that prove of no interest to them or to add new alerts to cover new interests. The results of three related questions covering the refinement of alerts are summarised in Table 11.

refining zetoc alerts

Table 11. The Refinement of Alerts

This data suggests that not many users have refined their alerts. Not many respondents have been triggered, either by the amount or quality of information they have been receiving, to change the alerts they set initially. They could have been discouraged from so doing because they felt it was hard to make such changes but most respondents disagreed with this proposition. For some reason therefore most respondents set alerts and leave them at their initial setting.

3.2.2 Search

Over 50% of the users had made use of the Zetoc search facilities. The next three tables report the search techniques employed by users.

using zetoc general search

Table 12. General Search

41% of users said they made general searches and they used all fields, title or author. ISBN/ISSN was not a popular route to making a search.

using zetoc conference search

Table 13. Conference Search

Conference search was less popular, used by 20% of respondents. All fields, paper
title and author were again the way in but keywords and the title of the conference
were also used.

using zetoc journal search

Table 14. Journal Search

Journal search was the most popular domain to search and was used by 52% of the respondents. The favourites were all fields, titles and authors once again. ISSN and start page were not popular routes.

Several questions were asked about what the users did with search results.

using search results

Table 15. Using Search Results

Most respondents used the neutral category when responding to these questions which may imply that they did not fully understand the functions listed. There is a slight tendency towards disagreement with the statement ‘view all Zetoc brief record results as full records’, a tendency to agree that ‘the tagging process is straightforward’ and to disagree that they used ‘resort’ as a means to change the order in which records are presented.

Another question asked whether the search results were emailed to anyone (table 16).

email zetoc search results

Table 16. Emailing Search Results

The most likely target of the search results is the respondent themselves. There is little sign that they are emailing colleagues or groups which again suggests using Zetoc is for individual purposes rather than to supportive a team effort.

3.2.3. Search History

The history function which enables previous searches to be repeated, was used by only 7% of the respondents.

use of zetoc history function

Table 17. Use of the History Function

Table 17 suggests that one of the main reasons the History function is not in great use is that most respondents were unaware of it. The question of the awareness of users about the features of Zetoc will be considered in the Context of Use section below. It is instructive to examine which classes of users actually use the history function (Table 18).

users of zetoc history function

Table 18. Users of the History Function

Librarians are the most frequent users followed by researchers. These are two of the biggest groups of respondents and may also be two groups who are most aware of the extent of Zetoc facilities.

3.2.4 Document Ordering

Only 2% of the respondents used the facility to order a full text article from British Library. Table 19 shows what the users are doing when they want full text articles.

obtaining full text article

Table 19. Ordering Full Text Articles

All of the other alternatives (Inter Library Loans, local sources, e.g. the Library), and looking for electronic full text on the web, were popular. Respondents were also asked how they hoped to be locating full text articles in the future.

obtaining full text in future

Table 20. Ordering Articles in the Future

The profile of responses remains broadly the same with a slight movement towards locating full text articles on the web. Overall, the responses suggest three factors are at work; obtaining an integrated electronic service – from search to full text, locating full text locally and not paying yourself.

3.2.5 Access via Bibliographic Software

Another service offered by Zetoc is the opportunity to access it via bibliographic software such as Endnote or Reference Manager or using Z39.50.

alternative access to zetoc

Table 21. Access via Bibliographic Software/Z39.50

The data suggests that, at most, there are only 30 of these respondents using these facilities. As with the History function one of the main reasons for the relative lack of use of these facilities is that most respondents are unaware of them. However, even when they are aware of them the majority of respondents do not make use of the facilities.

4. The Context of Use

The pattern of use described in section three is of a great many respondents using the alert service to keep up-to-date with current development, a considerable number using the search facilities and relative few using the other facilities that Zetoc offers. In order to understand the reasons for this pattern of usage the questionnaire explored three aspects of the context of use which are known to influence the form and extent of services such as Zetoc. These are the awareness of the service (including training and support mechanisms), the usability of the service and the way its use relates to other services available (which may be regarded as competitors or as services to be integrated with Zetoc). An overview of the relative importance of these issues was sought by asking respondents why they were not using unused facilities.

unused zetoc facilities

Table 22. Reasons for Not Using Zetoc Facilities

These results suggest that the main issue is that respondents have not fully explored what Zetoc has to offer. There is an 18% response which suggests the other services are of no value to the users and the 5% ‘tried but difficult’ response suggests there are relatively few usability issues. The comments in the 14% ‘other’ category were made up of the following:

reasons zetoc facilities not used

Table 23. Specific Reasons for not using Parts of Zetoc

Most of these comments can be re-categorised into the awareness (didn't know it existed), usability (usage problems) and no value (doesn't apply to my needs) categories. The most frequent statement 'I use other services' is a version of the 'no value' judgement because of use of competitive services. The additional, and minor, category in the table is cost. Each of the three major categories, awareness, usability and value in relation to other services will now be explored in more detail.

4.1 Awareness, Training and Support

An initial question in this section asked how respondents heard of Zetoc.

how hear about zetoc

Table 24. Hearing of Zetoc

This table reinforces the conclusion that local factors are the major mechanisms stimulating awareness and usage of services like Zetoc, in this case the local library through mailings and courses, and colleagues are the major vehicle for dissemination of information. The degree to which respondents had received training in Zetoc at their local institution was explored further.

aware of zetoc training

Table 25. Zetoc Training

Only 22% of the respondents were aware if their local institution had any provision for training in Zetoc and, of these respondents, 48% had taken advantage of the training. Thus, approximately 10% of the respondents can be considered to have received some form of training in Zetoc use.

Other routes to learning about Zetoc include looking at and working through the training materials on the Zetoc website.

training materials available

Table 26. Awareness and Use of Support Materials on the Website

The responses to these rating scales suggest that the majority of respondents are unaware that there are training materials on the website. 75% of the respondents used the neutral category when judging the examples in the workbook, probably because they had never looked at them, and most people reported that they had not used the step-by-step guide.

Respondents also have the opportunity of getting support through the email hotline.


emails to zetoc helpline

Table 27. Use of the Zetoc Helpline

Very few of the respondents had in fact ever used the hotline. The few that did offered the following views on the service.

helpline support offered to users

Table 28. Support offered by the Hotline

The respondents who had used the hotline were mostly positive about the speed and usefulness of the response. It will be noted that these questions also include one about whether there was a group of users of Zetoc in their Department which would raise the possibility of local support from colleagues or from the staff of the library. This was explored further as below.

local support for zetoc

Table 29. Local sources of Support

It appears that very few people get much help from colleagues and are more likely to turn to their library for support although, even here it is only about one third of respondents. This section shows there is limited awareness and training amongst the users of Zetoc. They have a wide variety of forms of support available to them but are making rather limited use of these opportunities. It appears therefore that most Zetoc users are lone operators going straight to the service and trying to make what use they can of it without further assistance. Such practices put a premium on the usability of Zetoc facilities and the explanation included in the service itself. These are the issues addressed in the next section.

4.2 Usability and Explanation in the Service

A number of questions were asked to assess the overall usability of the Zetoc service. The results are summarised below.

general usability of zetoc

Table 30. Control of the Zetoc Service

Very few Zetoc users felt that they were not in control of it or that they did not understand what it could do for them. Similarly very few felt confused by the interface to the system. The list below categorises the specific instances where users felt confused (table 31).

Examples of when the User Interface was confusing (number of responses given below)
Basic navigation
Sometimes just get lost 5
Title/author alert instructions not easy to find 2
Hard to find start page 1
Slow, not intuitive 1
Expectations
Can't access texts of references recommended 8
Confusing for non-IS professionals to understand what they can get 2
Alert gives much unasked info to wade through 2
Don't know how exhaustive my search is 1
Process (set-up, usage)
Have I finished setting up? No feedback. 7
Difficult to navigate initial lists of journals (categorise by subject?) 3
Unclear how to cancel an alert 3
Confused about how to go back and refine alerts 3
Saving marked lists 1
No logout button 1
Switching between lists of journals for searches and list in alert 1
Instructions
Title/author alert instructions not easy to use 2
Don't understand the info resulting from clicking an email alert link 2
Directions for navigating about the site are confusing 2
Searching
Lack of flexibility/advanced searching 2
Understanding how to use search 1
Returning to general searches 1
Issues of design
Too much information on home page 4
Discovering '?' was a help button 1
Students don't understand magnifying glass symbol 1
Zetoc wouldn't work with my browser 1
System won't allow some characters/diacritics 1

Table 31. Examples of when the User Interface was confusing

It should be noted that each item in this list is a comment from a very limited number of respondents. In some cases it is not a user interface issue but, as in the section on expectations, a matter of understanding what Zetoc can and cannot do. Other questions explored the strategy users were employing with Zetoc.

general strategy using zetoc

Table 32. Setting Up and Using the Service

The majority of the users had set up alerts and were just letting them get on with the job. They considered themselves to be regular users and likely to make more use of the service in the future. These answers reinforce the conclusion that these respondents are getting good value from whatever use they are making of the service and are finding it relatively trouble free to use the facilities they know and understand.

Respondents were asked about the initial stage of locating the Zetoc service.

locating the zetoc service

Table 33. Locating Zetoc

The majority of the respondents report that is was easy to find Zetoc and they did not have to refer to colleagues or other ‘non-electronic’ sources to find it. This again suggests that users work independently to access services and had no trouble finding Zetoc this way.

Respondents were asked to comment separately on the usability of Alert Services and Searching the database and this data is reported below.

4.2.1 Usability of Alert Services

Respondents were asked to comment on setting up an alert and the need for Athens authentication.

accessing zetoc alert

Table 34. Access to Zetoc Alert

90% of the respondents felt that setting up an Alert was straightforward and 79% did not consider Athens authentication a barrier. Several more questions explored the process of establishing an alert.

setting up zetoc alerts

Table 35. Setting up Zetoc Alerts

74% of users had little trouble deciding which journals they wanted although fewer knew which title keyword or alerts to set. There is confirmation that this process is seen as relatively trouble free from the final question: 90% said they did not have a problem setting a Zetoc alert.

The next question asked how easy it was to find the journals they wanted alerts for.

finding journals for alerts

Table 36. Finding the Journals for Alerts

81% agreed with the statement that it was easy to find the journals (Table 36). As previously cited (Table 8 above) respondents set a wide number of alerts and they were asked about their strategy in determining the number of journals.

specifying journals in alert

Table 37. Specifying which journals

Respondents were almost equally divided between those who set alerts for all the journals of interest to them and those who only set some. The reasons given for only setting some alerts were as follows.

why use alert for select journals

Table 38. Reasons for setting only ‘some’ alerts

There are three major reasons given. First, there is a perception that not all the necessary journals are covered by Zetoc. Second, there is a fear of overload which can, for example, lead to usability problems with the email system. Third, respondents have access to journals and their TOCs by other means and don’t duplicate the service, especially if the existing service is perceived to be faster, cheaper of give direct access to abstracts or full text. This latter reason raises the issue of Zetoc in relation to other services which is examined in section 4.3.

Respondents were also asked about the way alerts arrived and were presented on their screens:

usability of zetoc alerts

Table 39. Alert Screen Usability

Very few of the respondents found any problem navigating the alert screens and they were either happy or neutral on other issues about the layout of information on the screen.

table of contents results

Table 40. Table of Contents Results

Very few people received email from other people’s alert lists and it must be presumed they are therefore only dealing with their own alerts. 70% of the respondents felt the table of contents arrived in a timely manner and they were either happy or neutral on the issue of whether the format makes it easy to use them elsewhere. It seems likely that the 38% ‘neutral’ response to the latter question means that many respondents have not tried to use them elsewhere.

4.2.2. Usability of Search Facilities

Respondents commented on searching the database as follows (Table 41).

usability of search facilities in zetoc

Table 41. Searching the Database

57% of the respondents agreed that Zetoc gave sufficient guidance to search the database and most of the rest were neutral suggesting there are no specific problems. This is reinforced by the fact that only 14% said they noticed the interface. (In general when users notice the interface of the system they are using it is because it is not behaving as they expect).

Another question asked about understanding how to search and the use of examples on the website.

using zetoc search facilities

Table 42. Support for Searching

Only 9% of the users felt they had any difficulty understanding how to search using Zetoc and only 20% felt the need to read ‘frequently asked questions’. 12% agreed that they had looked at the workbook example searches. Respondents again seem to be adopting the practice of trying to do it themselves based on what they already know and how the interface explains itself and, by and large, they are successful with this strategy.

We have noted earlier (Table 15) that users are ‘neutral’ on subjects such as the way they view search results, the use of tagging and the use of ‘re-sort’ which suggest that most users do little processing of the results they obtain. They were also asked what they used stored searches for.

use of stored zetoc searches

Table 43. Use of Stored Searches

76% of users are saying they do not use stored searches that again implies that limited use is made of search results.

Zetoc provides the opportunity to search the database via bibliographic software and Z39.50. We have noted that relatively few respondents used this service but those that did were asked about its usability.

searching with z39.50

Table 44. Searching with Z39.50

Respondents either felt this was a straightforward process or were neutral on the topic. 42% said they preferred to use this software rather than a web browser and 38% felt it gave greater control over the results Zetoc generates.

The results in this section demonstrate that there are no major usability problems with Zetoc. There remains the question of Zetoc and its relation to other services which is the final ‘context of use’ issue to address.

4.3 Other Services: Competition or Integration?

Zetoc users could have access to many other services that relate in a variety of ways to Zetoc services. Some of these systems provide services that overlap with those offered by Zetoc and could be regarded by respondents as alternatives or competitors to Zetoc. Others may be used in conjunction with Zetoc to provide an integrated service that stretches from search to full text.

To explore the context within which Zetoc was used with respect to other services respondents were first asked what other services they used.

other services used

Table 45. Other Services Used

Respondents made widespread use of services in every category of Zetoc service. They were most likely to be using other web based search engines such as Google, online reference services such as COPAC and their own Library catalogues and Abstract and Indexing databases e.g. ISI Web of Science and Compendex. Most had access to some full text electronic journal services, e.g. Science Direct and JSTOR and other alerting services, e.g. IDEAL Alert and BioMed Net TOC Alert. As discussed above (Table 19) they also have access to other ways of ordering full text documents e.g. through their Library, via Inter Library Loans etc.

In addition to these services respondents also cited the following under the ‘other’ categories.

Specific sites/services

Emerald alert [2 responses]
Medline [2 responses]
Ovid [2 responses]
BioMed
Bopcas
British Pharmacopoeia
British Standards Online
EEBO
Ebsco
Food Navigator
JustFood.com
Freepint
jiscmail
Ingenta
Jake
JSTOR
JVIR Contents
MathSciNet
midirs
Net Doctor health news
ProQuest
Pubmed
Refscout
Scifinder Scholar
Portals such as SOSIG
SDI search
Subscription Agent's TOCs service

Non-specific services

Publishers alerts [2 responses]
Full text databases
Journal Notices
librarian mailings
newspapers
Website change monitoring services
Good old references in books...
browse library shelves
R and D...

Table 46. Other electronic service used

Another kind of service users could be employing are bibliographic reference sources:

bibliographic software used

Table 47. Bibliographic Software in Use

Exactly 100 respondents reported they made use of this kind of service and the majority (74) made use of Endnote (Table 47).

4.3.1 Competition

We have noted above some of the circumstances in which the use of other services are seen as competing with Zetoc. For example, the use of the alert service is restricted by some respondents because they can use specific alerting services for some journals which they consider to be faster, cheaper or leading directly to abstracts or electronic full text articles (Table 36). The process of ordering full text articles direct from the British Library is favoured by very few of the respondents: one reason is that respondents prefer to seek electronic full text articles and another reason may be cost. Other methods of obtaining full text articles may, of course, also incur a cost to the respondent, for example, Inter-Library Loans.

charges for inter-library loans

Table 48. Charging for Inter-Library Loans

Policy for charging back the costs of Inter-Library Loans to the person placing the order appears to be evenly split across institutions. A number of questions were asked to obtain respondent views on their strategies and preferences when ordering full text articles.

obtaining full text articles

Table 49. Obtaining full Text Articles

At least 72% of the respondents restrict their use of Zetoc to finding what is available and switch to other methods of obtaining the full text versions of articles. There is little sign that this is because the ordering options are unclear or unhelpful although the 65% ‘neutral’ response probably means that most respondents have not even tried them. 51% of respondents would like a better link from Zetoc to their Institution’s library for this purpose and, from other responses, it is likely that most of the rest would like to go straight to full text electronic articles.

There is some evidence here that users are adopting a comparative approach to the services on offer. They are not necessarily deciding to use one service or another but are deciding which parts of a service to use for what purpose. The effect on Zetoc use is, for example, that users are selective in setting alerts and in choosing how to obtain full text articles.

4.3.2 Integration

The selective use of a number of services can also result in a different strategy. It can be an attempt by users not just to use beneficial parts of services independently of one another but to use them to obtain an integrated service, i.e. to move seamlessly in the electronic medium from alerts and searches to full text articles. Several questions addressed whether users were seeking this kind of outcome and whether they were making progress towards it.

zetoc and integrated services

Table 50. Towards Integrated Services

These three questions (Table 50) produced bipolar responses. 32% feel that Zetoc is helping them achieve an integrated service but 34% conclude it is not. 42% say they use Zetoc in conjunction with other Alerting services and 48% do not. 45% say they use Zetoc on an ad hoc basis and the 41% who disagree with this statement presumably feel they are more planful in their use of the service. This bipolar response is obviously not a product of what Zetoc can or cannot do but a statement of what users want to achieve and what they are able to achieve. A number of questions explored what people were doing in particular to achieve an integrated service. A limited number of respondents, for example, responded to the questions ‘do you ever combine Zetoc with other ‘targets‘ in your searches?

combining zetoc with other targets

Table 51. Combining Searches

A small percentage of users were attempting this form of integration reporting for example, that they combined Zetoc and ‘web of science’ searches on the same topic. Respondents were also asked if the had established a current awareness/searching routine.

established searching routine

Table 52. Establishing a Routine

Most respondents did not claim to have developed a routine but the 20% who did gave many examples of the way they had integrated Zetoc and other services into their working practice in order to search and select new material of relevance to their professional activities. It is important to note the classes of user who have achieved this type of usage.

070.b Have you established a current awareness/searching ‘routine’?

Librarian:

Defined routine: providing a service
Create regular current awareness service for department 3
I forward alerts to interested groups 3
Defined routine – personal
Store all alerts/records in a folder, work on it once a week/fortnight 3
Receive toc alerts, try to find full text electronically 1
I check Zetoc alerts on a daily basis 1
Undefined routine
I use Zetoc alerts and other services 5
I scan a list of sources and alerts including Zetoc 4
Just use contents alerting 2
I use Zetoc searches, other alerts and auto searches 1
I instruct others how to set up Zetoc alerts 1
Zetoc questionnaire report 41 of 54
Specifically no routine
No routine – depends on subject matter I am working on 1
No routine – I don’t understand Zetoc 1

Other users:

Undefined routine – personal
I use a combination of services 3
I go through all alert emails on a regular basis 1
Keyword searching, ToC alerting from publishers for abstracts 1

Postgraduate:

Defined routine – personal
Browse Zetoc alerts as they arrive, then get/order papers 4
On daily basis, paste articles into Word – regular trip to library 2
Read alert, check library catalogue on web, search net – if not
available (free) then ILL 2
Read alert, check library catalogue on web, if available then print
out, if not wait for it to arrive on shelving 1
Read alert, download via direct online access if available, else go to
library 1
Read alert, print title, use PubMed/other service for abstract, then
find paper electronically/in library 1
Read alert, save interesting ones, print out each quarter, check
library, use ILL if not available 1
Categorise alert content for future acquisition 1
Read alert, use EndNote to keep track 1
Periodic database searches as work requires 1
Undefined routine – personal
Zetoc alerts and other search services 3
Combination of Zetoc alerts and browsing in library 3
Browse Zetoc alerts as they arrive 3
Weekly keyword search and alerts 1
Use journal’s own toc alert (faster than Zetoc), Zetoc for other
journals 1
Use Zetoc for journals not in ScienceDirect 1

Researcher

Defined routine – personal
Read Zetoc alert and then follow it up 5
Download Zetoc alert papers from library e-journals or via ILL 1
Transfer relevant alert items to Access database, use that to get full
text 1
Search WoS monthly, download output to Papyrus 1
Check Zetoc alert author details on WoS, then request reprint direct 1
Use Zetoc alert – use EndNote to access abstracts 1
Use Zetoc alert and other toc alerts, but use other toc alerts for downloads 1
I save Zetoc alerts for future use 1

Undefined routine – personal
Check Zetoc alerts regularly 8
Use alerting services including Zetoc and ad hoc searches 4
Use Zetoc alerts with specific keyword searches 4
Use Zetoc alert 4
Use Zetoc alert and scan other journals 3
Use Zetoc alert and other alerts 2
ToC alert from journals, other searches, Zetoc search when required 1
Zetoc search quarterly 1
Build up a reading list 1
Use Zetoc alert but can’t get electronic access and library too far away 1

University teacher

Defined routine – personal
Use Zetoc alert, then WoS/citation index for abstract, then library for copy 2
Use Zetoc alert and others, then publisher site for full text 2
Check/search every morning and go to library fro copies 1

Undefined routine – personal
Use Zetoc alert 2
Use Zetoc alert for all journals I don’t subscribe to 2
Use Zetoc and other services 2
Use Zetoc search by subject 2
Use keywords for Zetoc alert 2

Table 53. Respondents Specifying a Routine

The three main user types are researchers, librarians and post-graduate students who could all be considered to be users for whom information is of central importance in their work and who have time to devote to setting up services to help them. To end this section a few quotes are offered from these three groups of users to illustrate how Zetoc is becoming an integral part of regular working practice and being used as an integrating tool in information management.

Researcher 1 “I get alerts from a range of web resources (ScienceDirect, INGENTA, Zetoc etc). I have moved a number of searches from other sources to Zetoc as I find Zetoc alerts allow me to combine most of the things I need to know about into a single search/alert service”.

Researcher 2 “Read Zetoc and ScienceDirect new issue alerts via email to keep abreast of current publications and development. Download full text from Science Direct or other sources for particular articles of interest”.

Researcher 3 “I check Zetoc alerts daily and send out reprint request to authors of articles that interest me. Since Zetoc does not provide author’s addresses I must look these up on ISI Web of Science. It would be better I Zetoc Alert provided author’s addresses with the Table of Contents”.

Librarian 1 “Receive Zetoc toc alerts and try to find full text electronically if possible. Problem when get Zetoc alert before article is available electronically (especially with Ingenta). Even when requesting Intersite loan of print journal kept in college often get alert before print issue received”.

Postgraduate Student 1 “I receive alert, check to see if own library holds journals and then search net for on-line journals. If journal not available on–line (for fee) then order article through inter-library loan”.

Postgraduate Student 2 “Once an alert is issued, I check it, delete if no use an print off the title of the paper if it is relevant. I then use PubMed to look at the abstract and get the paper from the library or electronically if possible if interesting”.

5. Enhancements to Zetoc

Respondents were asked to specify what enhancements they would like to see in the Zetoc service. They were asked to consider each part of the service. The final comments section was well-supported, given it was a free-text field at the end of a long questionnaire. Many respondents felt compelled to give a brief summary of their experiences with Zetoc. The ‘responses’ count for each section is the number of non-blank responses – the actual number of useful responses will be marginally lower.

5.1 Enhancements to the set-up process

Reponses: 42

Most frequent suggestions:
· No enhancements needed;
· Make it more obvious/straightforward (how to navigate and set-up);
· More frequent prompts to review the service – unaware of many services;
· More transparency to add/delete journal lists function;
· Haven’t changed set-up – can’t remember how.

Interesting other suggestions:
· Offer pre-sorted bundles of subject-specific journals;
· Link to main website to amend list from email alerts;
· Institutions need to know about resetting every 6 months (leads to non-use).

5.2 Enhancements to searching the database

Responses: 63

Most frequent suggestions:
· Include (links to) abstracts;
· Provide advanced searching options (Boolean, volume number etc);
· Provide by-subject searching;
· Provide searchable abstracts;
· Better ease of use.

Interesting other suggestions:
· Wider selection of articles/journals;
· Make output more usable and less in a ‘librarian’ format;
· Ability to create and combine sets.

5.3 Enhancements to alerting services

Responses: 175

Most frequent suggestions:
· Provide link to abstracts and/or full text (78);
· Link to library OPAC/e-journals list if available locally, ILL if not;
· Alerts need to be more prompt/timely/in order;
· Full journal reference for each title;
· Want ability to download abstract into EndNote;
· More advanced search (Boolean, wildcards).

Interesting other suggestions:
· Improve formatting of email alert (no CAPS, include bold title, remove urls for Zetoc, generally ‘improve’);
· Inform me of new journals in my field;
· Keyword alerting results in full ToC for journal – just want the article with the appropriate keyword;
· Would like author/institution email/snailmail address;
· Give date and volume info in email subject;
· Give ability to exclude non-English language articles in current awareness profile.

5.4 Enhancements to access via bibliographic software or Z39.50

Responses: 39

Most frequent suggestions:
· I need to read the manual/learn more about this (increase awareness/provide training/on-screen support);
· Support BibTex, MS Access, Biblioscape, Reference Manager prior to version 10;
· Speedier access.

Interesting other suggestions:
· Provide consistency between EndNote and website search results;
· Include abstracts;
· Offer fully detailed, high spec connection and filter for EndNote.

5.5 Enhancements to document ordering

Responses: 52

Most frequent suggestions:
· Direct transfer of data to our ILL system/automatically fill in an ILL form;
· Make it cheaper/link to free versions of the documents so don’t have to go through BL;
· Link to full text, not ILL;
· More awareness on Zetoc.

Interesting other suggestions:
· Non-electronic delivery is too slow;
· Improve server stability;
· Link to abstract so can read before deciding to order;
· More accessible info on cost/where to get doc from for free (locally).

5.6 Enhancements to integration with other services

Responses: 60

Most frequent suggestions:
· Link to local electronic holdings;
· Link to electronic journals on the web/full text;
· Link to other services (KA24, NeLH, WoS mentioned);
· Increase awareness;
· ToCs for book chapters.

Interesting other suggestions:
· No local holding – keep link to BL;
· Need to allow downloading to bibliographic software.

5.7 Enhancements to support available

Responses: 28

Most frequent suggestions:
· Make it clear who to contact for support (at a given point in the service);
· Send out more awareness updates/generally increase awareness;
· Person to person training would be nice;
· “You are wonderful!”

5.8 Final comments

Responses: 290

Some interesting comments:
“Include abstracts and you will rule the world of current awareness.”

“I am aware that I am not using Zetoc to its full potential but never seem to have the time. They don't appear to be services that I would miss if I didn't learn how to use them.”

“A brilliant idea, even though it has given the library more work. Definitely improved patient care.”

Most frequent comments:
· Very useful (86)
· Very good service (44)
· I’m not making full use of the service (38)
· I love it/great/thanks (37)
· I’ve learned more about it today, will specifically go and use it more (20)
· Want direct links to online text/abstracts (16)
· It’s essential to me/part of my daily routine (11)
· A time saver
· Good coverage/ would have missed articles without Zetoc
· Needs more coverage
· Could be more timely
· Could be better
· Will now look at training materials.

Other interesting comments:
· Excellent addition to research tools
· Lack of awareness of Endnote/Z39.50)
· Difficulty in cancelling/renewing
· Good ease of use
· Increase general awareness of Zetoc and its services
· I use Web of Science to obtain abstracts/text which Zetoc has alerted me to
· More coverage of non-English speaking journals
· Just set it up and left it running – most questions don’t apply to me
· Needs better search facilities
· Questionnaire too long (ho hum).

6. Discussion and Conclusions

Zetoc is a successful service. The majority of the respondents praise the way if offers them a service they have not had before which enables them to keep on top of current developments in their fields. It is a service that they find relatively straightforward to use with very few usability problems. The alerting service in particular is obviously becoming a regular part of the working life of professionals in many fields and many institutions.

There are, however, considerable differences in the way respondents use the Zetoc service. It may be an oversimplification but it is useful to consider two types of user who use Zetoc in different ways and achieve different benefits as a result.

The Early Integrators. There is a group of users, probably about 20% of the sample, who are active users of Zetoc. They appear to have explored its capabilities and to have worked out how best to use it in relation to the range of other information management services that are available to them. They are working towards a personal, integrated and manageable routine which keeps them up-to-date and gives them a seamless (and hopefully cost free) route from search to full text article. Where possible they want this integrated process to be electronic. It looks as though the majority of the users in this class are researchers, librarians and postgraduate students who are prepared to put in the time to understand the different services available to them and find the best way of personalising them for their usage. Some of these users, especially the librarians, may have a formal responsibility to establish appropriate combinations of service for their colleagues.

The Passive Majority The majority of the users, perhaps 80% of this sample, are more passive towards Zetoc. They have heard of it, set up some Alerts and perhaps make occasional searches of the database. They seem unaware of most of the refinements in the service and do not attempt to make connections with other services. They let their initial alerts stand whether they are giving the best information or not. They are probably self-taught about Zetoc, do not attend training courses or visit website support facilities and use Zetoc independently of others. Their usage appears characterised by a short period of interest when they set the service up and then it is ‘left to get on with its job’ whilst the user attends to other matters. This is a classic pattern of usage found in many other applications in circumstances where busy people have many other priorities and seek services which require least effort and minimum cognitive resources from the user. The pattern of interest is short bursts triggered by particular requirements with long periods of inflexible and limited usage. These users also seem interested in an integrated, electronic service but are faced with an array of potential services which they do not have the time to evaluate. Zetoc becomes one such service and it is likely that they pigeonhole it as a more limited service than it is, for example, as a TOC Alerting Service, and are not aware of any greater potential it may have. The majority of University Teachers and many researchers and postgraduates are no doubt in this category. Since it takes effort to complete a questionnaire it is likely that a greater percentage of the population of Zetoc users are in the passive majority; active integrators are more likely to take the opportunity of the questionnaire to express their views and suggest enhancements.

This division poses some interesting questions for the development of Zetoc. There is obviously a desire by users for a more integrated and seamless electronic service and enhancements in this direction will help overcome many of the gaps in service that currently exist. It is likely that most of the suggestions for enhancements come from the active integrator group and that they will be the first to make use of advancements that are introduced. They are already a large group and providing a service to them makes such developments worthwhile in itself.

However, what can be done to help the passive majority develop a more integrated and better targeted service for themselves? There are two issues. How to get this group to make fuller use of the Zetoc services that are currently available and how to offer enhancements in a way which they will take up? The classic way of enhancing a service is to launch the new features with a programme of announcements, training courses and new support facilities. Given that these users are relatively unresponsive to training, awareness and support opportunities it could be that, even so, enhancements will go unnoticed by them. Are there any indications in the results of this survey which can help with this dilemma? There are two routes that look promising. The passive users work by relying on the main screens and interfaces to guide them in setting up and using services. To become visible to them any enhancements have to made apparent in these screens. To get a positive response they need two further properties; they have to explain how they provide a valuable service and they have to be seamless and effort-free to implement. These are often difficult goals to achieve but if they are not, the users will experience barriers and obstructions which will stop them trying. A related route is to use local institutional mechanisms to foster awareness and implement local integrated services which require minimum effort from end users. Most institutions make available to end users a complex array of electronic and non-electronic information resources and are working continually to render these coherent, understandable, integrated and tailored to the needs of different kinds of users. They too struggle because many users do not respond to training courses and support services. Most users appear to operate as individuals in determining how they will use information resources but there are some indications that they respond to colleagues and to librarians who focus on their specific needs. The existence of any ‘active integrators’ in a user department can also be a focus for the spread of awareness and the growth of new working practice. The more local and specific the source of help, the less effort is required from the end user and the greater the progress that can be made.

There can be little doubt from this survey that Zetoc is offering to its users a service they want and value. It is also true that the plans to develop a more integrated service match the stated needs of the user population. Part of that user population seems ready to exploit what Zetoc can offer. The majority of the user population constitute a more difficult target, not because they do not want the services, but because they are locked into limited and habitual practices. Finding seamless and effort free ways of helping them to exploit new facilities is the major task that this survey reveals.

Appendix 1: the questionnaire

This is available either as an accompanying Acrobat PDF file (Zetoc questionnaire.pdf) or can be viewed at:
http://www.lboro.ac.uk/departments/hu/groups/hfsd/Zetoc-q/Zetoc.html

Appendix 2: q002 What is the name of your institution?

instutions stats using zetoc

HE Institutions with two responses:

Arts Institute at Bournemouth,
Aston University,
Bradford University,
Canterbury Christ Church University College ,
Chester College,
Imperial College of Science Technology and Medicine ,
Kingston University,
Lancaster University,
Leeds Metropolitan University,
London Guildhall University,
London School of economics,
Manchester Metropolitan University,
Queen Margaret University College,
Queen Mary, University of London,
Southampton Institute,
University of Kent at Canterbury,
University of St Andrews,
University of Stirling,
University of the West of England,
University of Wales Bangor,
University of Wales College, Newport ,
University of Wales Swansea.

HE Institutions with one response:

Institute of Education,
John Rylands University Library of Manchester,
Universidad Central de Venezuela,
B.C.F.T.C.S,
Bodleian Law Library,
Bournemouth University,
Central School of Speech and Drama,
City & Islington College,
College of St Mark & St John,
DeMontfort University, Bedford,
Goldsmiths' College, University of London ,
Health Services Management Centre,
Highland Health Sciences Library,
Keele University,
King AbdulAziz University (Saudi Arabia),
LBS,
Liverpool Hope,
Liverpool John Moores University,
Middlesex University,
Napier,
Nuffield Institute for Health,
Salford University,
Salomons,
SOAS,
South Bank University,
St George's Hospital Medical School,
Staffordshire university,
Strathclyde University,
Swansea Institute,
The Rural History Centre,
Totton college,
Trinity College Carmarthen,
University of Central Lancashire,
University of Derby,
University of East Anglia,
University of Essex,
University of Glamorgan,
University of Greenwich,
University of Maryland (USA),
University of Nottingham (Malaysia campus),
University of Teesside,
University of Tlemcen (Algeria),
University of Wales,
University of Wales College of Medicine,
University of Wales Institiute Cardiff (UWIC),
University of Warwick.