EBONI: Project Plan

Project Plan

Project Acronym: EBONI
Project Title: Electronic Books ON-screen Interface
Start Date: 01 August 2000
End Date: 31 January 2002
Lead Institution: Strathclyde University (Centre for Digital Library Research)
Project Manager: Monica Landoni

Contact: Ruth Wilson (Research Assistant)
Dept. of Information Science
University of Strathclyde
Livingstone Tower
26 Richmond Street
Glasgow G1 1XH

Tel: 0141 548 3705
Email: ruth.m.wilson@strath.ac.uk
URL: http://ebooks.strath.ac.uk/eboni/

Contents

1. Introduction
1.1. Project Description
1.2. How the project builds on earlier and parallel work
1.3. Relationship to other projects

2. Aims and Objectives

2.1. Aims
2.2. Objectives
2.3. Project Outputs

3. Overall Approach

3.1. Structure
3.2. Methodology
3.3. Generic issues
3.4. User communities
3.5. Accessibility

4. Project Consortium

4.1. Partners and Associates
4.2. End User Representation

5. Project Management

5.1. Project Management Structure
5.2. Project Team
5.3. Management Group
5.4. Steering Committee

6. Overall Project Structure

6.1. Workpackages and their Interrelationships
6.2. Milestones
6.3. Gantt Chart

7. Detailed Project Plan
8. Dissemination Strategy
9. Steering Group Terms of Reference

9.1. Composition
9.2. Conduct of Meetings
9.3. Main Functions of the Committee

10. Quality Assurance and Evaluation

10.1. Quality Assurance
10.2. Evaluation Strategy

10.2.1. Purposes of evaluation
10.2.2. Stakeholder groups
10.2.3. Evaluation questions
10.2.4. Priorities for Formative Evaluation
10.2.5. Methods and Techniques for Answering these Questions
10.2.6. Timetable
10.2.7. Summative evaluation

11. Risk Assessment
12. Budget

Appendix A: Detailed Project Plan

1. Introduction

1.1. Project Description
EBONI will identify and compare the variety of methods which have emerged in the publication of learning and teaching material on the Web in order to determine the most effective way of representing information in electronic books, aiming to maximise usability and information intake by users. An evaluation of texts by an appropriate mix of key stakeholders including education professionals from Higher Education and the National Grid for Learning and students from a range of disciplines and backgrounds will be utilised to develop guidelines for best practice in the publication of (non-journal) educational material on the Internet. It will also attempt to obtain access to stand alone e-books to test the applicability of these Web-based guidelines to other media. This will enable the needs of an emerging Higher Education community of readers and creators of digital content to be met more satisfactorily. The work will be done in conjunction with the University's Digital Information Office which aims to develop and promote standards in this area, the inter-institutional Clyde Virtual University project which jointly creates and manages electronic material across Glasgow, and the National Grid for Learning in Scotland.

1.2. How the project builds on earlier and parallel work
Recent movements towards student-centred resource-based learning in UK Higher Education have seen increasing use of Communications and Information Technology for curriculum delivery, with initiatives such as HERON , Project Phoenix and Eurotext improving the flow of course material and text to students.

The Internet, in particular, is proving a popular platform for the publication of learning and teaching resources, its interactive environment seeing the arrival of new guides, tutorials and textbooks every week, and students are increasingly turning to these digital resources as a first port of call when seeking material to support their studies.

Morkes and Nielsen have demonstrated that users' ability to retrieve information from Web publications can be improved by up to 159% by altering the on-screen design of the text , and it is probable that the usability of existing and future electronic books could be much improved by applying guidelines for best practice in respect of on-screen design based on a critical examination, and subsequent extension, of their work. With the People's Network , the National Grid for Learning and the University for Industry aiming for seamless access to online material, it is timely to pay attention to the internal design of the resources themselves so that, once accessed, the required data can be retrieved as quickly and easily as possible.

1.3. Relationship to other projects
EBONI is not directly related to other projects funded under the JISC Circular 5/99, and so does not fall within a "project cluster". However, all projects involved in the selection or provision of learning and teaching resources online, or Web interface design, will benefit from EBONI's experience and project outcomes, and vice versa. Projects resulting from other JISC calls will also be monitored to identify areas of overlap.

EBONI works closely with the Glasgow Digital Library, the Digital Information Office and Clyde Virtual University, all of which focus on the provision of content to the UK HE community. This relationship is mutually beneficial: access to material to evaluate is increased for EBONI, and the related services and projects will benefit directly from EBONI's findings.

EBONI is aware of the CETIS IMS Project which aims to tackle obstacles to the development of the use of learning technologies, and the growth of a global market in online learning materials.

Top

2. Aims and Objectives

2.1. Aims
The project aims to develop a set of recommendations for publishing educational works on the Web which reflect the needs of academics and a diversifying population of students throughout the UK. This will be achieved through an evaluation of texts which are found to be representative of approaches to the design of learning and teaching material on the Internet. Styles and techniques which will be evaluated include use of hypertext, navigation icons and indexes.

Evaluation of the quality of the content of resources, and of the use of audio and video in Web books will not be within the scope of the project.

It is important to note that the set of guidelines which emerge from the evaluation process are not intended to establish a strict uniformity of interface for all learning and teaching resources on the Web, but rather to encourage use of those styles and techniques which are found to be most successful in terms of usability.

2.2. Objectives
The precise objectives of EBONI are:

to evaluate the different approaches to the design of learning and teaching resources on the Web, and to identify which techniques/styles are most successful in enabling users to retrieve, quickly and easily, the information they require;

to identify and report on the individual requirements of academics, students and NGfL professionals in learning and teaching on the Web;

to compile a set of best practice guidelines for the publication of guides, tutorials and textbooks on the Internet for the UK Higher Education community; and

to promote with publishers of educational material the best practice guidelines.

The above objectives will be achieved by:

conducting a survey of the range of learning and teaching material available on the Internet, to identify the different designs and techniques used for displaying content;

selecting particular guides, tutorials and textbooks which are representative of the variety of approaches to the electronic publication of such texts. These will be used as the material for evaluation;

developing a methodology for evaluating selected guides, tutorials and textbooks. This will measure both the ease with which users can retrieve the information they need from the text, and their subjective satisfaction with the experience of reading the material on the Internet. The two measures will combine to produce an overall usability score for each text;

working with representative user groups from each discipline and background to carry through this methodology;

developing a means of analysing the results of the experiment;

using these results to examine the extent to which students from different disciplines and backgrounds have separate requirements for the electronic delivery of learning and teaching resources, and identifying mechanisms for improving the usability of such material;

contacting companies such as Gemstar and Franklin to obtain copies of portable e-books;

disseminating project results within and beyond the UK Higher Education community;

developing a Web site to support the project, describing its background and purpose, methodology and project results and containing relevant publications; and

investigating ways in which best practice guidelines can be implemented on new and existing texts.

Top

2.3. Project Outputs
Outputs of the project will include:

A Web site containing information about EBONI. This will alter in appearance throughout the course of the project to reflect our findings and, at the end of the project, will be presented as an electronic text to which the guidelines have been applied.

A survey of the range of learning and teaching material available on the Internet. Findings will be presented as a small catalogue on the project Web site;

A report on the individual requirements of academics, students and NGfL professionals in learning and teaching on the Web;

The development and implementation of a user-centred methodology for evaluating the various techniques employed in producing educational material online;

A report on the methodology, including a detailed description and an outline of any difficulties encountered in devising and implementing it;

A set of best practice guidelines for the publication of guides, tutorials and textbooks on the Internet for the UK Higher Education community;

Two project reports will be submitted to the Programme Office, in January and July 2001, and a final project report in January 2002;

Throughout the 18 months of the project, at least 3 papers will be submitted to journals and conferences; and

Consideration will be given to holding a series of workshops to teach resource creators how to apply the guidelines.

Top

3. Overall Approach

3.1. Structure

EBONI will conduct a survey of the range of teaching material available on the Internet, select key texts which are representative of various subject areas and design techniques, develop a methodology for evaluating these texts, implement this methodology using HE students and education professionals as subjects, and analyse the results, producing a set of guidelines for best practice in publishing educational material online.

3.2. Methodology
The methodology employed to achieve the project's objectives will involve four main phases:

Phase 1. Selection of material
- by design
Conduction of a survey of the range of teaching material available on the Internet, identifying, classifying and finally selecting resources according to use of techniques such as:

hypertext;

tables of contents;

navigation icons;

search mechanisms (for searching within the text);

indexes;

graphics; and

specific features of HTML such as frames, tables, coloured text and lists.

- by subject
In addition, the Web books to be evaluated will cover a number of subjects and students from each area will be sought to participate in their evaluation. The Project Team will look at a range of schemes for classifying resources, including the DNER collection policy and divisions of subjects within the RDN, to draw out a list of disciplines to be represented.

Phase 2. Selection of participants
It is anticipated that 80-100 paid subjects will be used to evaluate these texts, mainly drawn from the HE population at the three universities in Glasgow. The user communities addressed are outlined below.

Phase 3. Procedure
All users will be involved in both quantitative and qualitative aspects of the experiment:

quantitative feedback will be sought by asking users to search selected material for specific information and to participate in memory tasks. Success in answering questions correctly, time taken to complete tasks, ability to recall information and (depending on facilities available) covert observation of users' behaviour will be taken into account in interpreting this feedback; and

qualitative feedback will be sought via questionnaires and interviews immediately following the quantitative phase, and will aim to record users' subjective satisfaction with the experience of reading the material. Consideration will also be given to employing "think-aloud" techniques to record users' thoughts as they read and use the texts.

Phase 4. Measurement of results
Both quantitative and qualitative feedback from tasks, questionnaires and interviews will be analysed to determine the overall usability of each text, and this analysis will form the basis of the guidelines for the design of learning and teaching material on the Internet.

Data will also be analysed on a comparative basis, to identify differences between the needs of the representative user groups participating in the experiment.

Top

3.3. Generic issues
While generic issues such as interoperability, scalability and sustainability are not directly relevant to this project, their concepts can certainly be applied to the e-book environment and, as such, are important to the success of EBONI.

Interoperability
In order to ensure the guidelines are applicable across a variety of formats, EBONI will keep abreast of standards (such as the Open eBook Publication Structure ) and emerging standards. Attempts are being made to acquire portable electronic books such as Gemstar's REBs and the eBookman to test the applicability of the guidelines on emerging dedicated devices.

Scalability
The material selected for evaluation will include texts of varying sizes, ensuring that any differences in user requirements when reading short and long documents are registered and are reflected in the guidelines.

Sustainability
To maximise actual uptake and implementation of the guidelines among the target audience, the draft set will be tried out by an experienced digital content creator to test for ease of application and usability of the guidelines themselves. His comments and suggestions will be fed into the final guidelines, increasing the likelihood that they will be used widely, rather than becoming a redundant document.

3.4. User communities
The user communities addressed by EBONI will be directly involved in the experimental phase of the project, ensuring not only that their needs are assessed but also that they form the basis of the set of recommendations produced at the end of the project. They will be students in each of the subjects selected in phase 1 of the methodology as well as:

education professionals including lecturers and tutors;

professionals from NGfL;

undergraduates;

postgraduates; and

mature students and distance and part-time learners.

A literature review of the requirements of students and academics on the Web will form the basis of this methodology, which will be refined once it is piloted on a small group of subjects.

3.5. Accessibility
EBONI will follow eLib Standards Guidelines Version 2 (http://www.ukoln.ac.uk/services/elib/papers/other/standards/version2/) where relevant and with particular respect to the accessibility of services over the Web.

In addition, accessibility guidelines developed by the World Wide Web Consortium (W3C) for HTML 4.0 for content accessibility will be consulted. EBONI's recommendations for publishing learning and teaching material on the Web will be in accordance with the W3C Web Content Accessibility Guidelines 1.0 (http://www.w3.org/TR/1999/WAI-WEBCONTENT-19990505/) which explain how to make Web content accessible to people with disabilities; recommendations from the W3C Mobile Access Activity Statement http://www.w3.org/Mobile/Activity/ which aim to encourage seamless access from mobile devices will be monitored; and the W3C Web Accessibility Initiative's User Agent Guidelines 1.0 (http://www.w3.org/TR/UAAG10/) will be reviewed and applied.

Top

4. Project Consortium

4.1. Partners and Associates
This is a single-institution project, based at the University of Strathclyde. Other participants include:

Centre for Digital Library Research (CDLR): Brings together long-standing University research interests in the digital information area previously spread across two University departments. Managed jointly by the University's Directorate of Information Strategy and Department of Information Science, the CDLR seeks to combine theory with practice in innovative ways with the aim of being a centre of excellence on digital library issues ranging from information policy and information retrieval to document storage technologies and standards.

Digital Information Office (DIO): The Office aims to create a metadata repository and associated Web-based service interface for the University's electronic resources and will develop standards associated with the creation, description, storage, organisation, maintenance, security and copyright of digital information.

Clyde Virtual University (CVU): Joint venture to develop and deliver Internet-based teaching materials to students at five institutions in the West of Scotland. CVU brings together custom-built tools and materials such as online assessment and automated marking and a state of the art virtual library in a unique infrastructure for the provision of education over the Internet.

National Grid for Learning (NGfL): The NGfL is the national focal point for learning on the Internet, collecting resources brought together by the UK Government to help raise standards in education and to support lifelong learning. It aims to accommodate the needs of learners in all sectors of education, both formal and informal.

4.2. End User Representation
End user interests are represented throughout the evaluation procedure and by the inclusion of student representatives on the Steering Committee. Further, an experienced creator of digital content will be consulted about his views on the goals of the project; towards the end of the project he will apply the draft set of guidelines and be asked to comment on the practicalities involved in their implementation. In this way, the interests of both users and producers of learning and teaching material will be represented.

Top

5. Project Management

5.1. Project Management Structure
The following diagram represents, in general terms, the project management structure of EBONI:

Top

5.2. Project Team
The Project Team, comprising the Project Manager and the Research Assistant, will be responsible for the day-to-day management and will normally meet weekly. The project requires two members of staff for the 18 months of the project, one an experienced researcher with extensive subject knowledge to act as Project Manager, the other a suitably qualified LIS professional to act as Research Assistant.

The Project Manager (.2) will supervise project definition and progress, advise on developing a methodology and conducting relevant research, supervise the Research Assistant, advise on data analysis and dissemination, and edit reports, papers and guidelines.

This position has been filled by Monica Landoni, a lecturer in the Department of Information Science, Strathclyde University. She has completed a PhD on the role of the paper book metaphor in electronic book design, has several years' research experience in projects related to IR and electronic publishing including STAMP and MultiBrowser and her special interests include digital books and libraries and interface design and evaluation.

The Research Assistant (1.0 FTE) will research literature, devise, document and refine procedures, recruit subjects, conduct surveys and user evaluation, analyse data, write project reports, articles, guidelines and Web pages, and carry out dissemination activities.

This position is filled by Ruth Wilson who has previously worked for BUBL Information Service and as a Research Assistant on the SCONE and SEED projects, both within the CDLR. Prior to this she completed an MSc in Information and Library Studies from Strathclyde University, which included a dissertation on the importance of appearance in the design of Web books.

Top

5.3. Management Group
The Project Team will report to a small Project Management Group, consisting of the team together with the Project Director and a member of non-Strathclyde staff from the Clyde Virtual University. The Group will oversee the work of the project staff and meet every two months.

Dennis Nicholson is Director of Research in the Directorate of Information Strategy at Strathclyde University. Since 1991, he has managed a range of funded projects, including the BUBL Information Service (1991-), CATRIONA (1994-95), CATRIONA II (1996-98), CAIRNS (1998-), SCONE (1999-), and Glasgow Digital Library (1999-).

Dave Whittington works at the Robert Clark Centre for Technological Education at the University of Glasgow and is Technical Director of Clyde Virtual University. He is experienced in providing courseware on the Web and in use of the Internet to support flexible and distance learning.

5.4. Steering Committee
In turn, the Project Management Group will report to a Project Steering Committee, which will meet two or three times a year. To ensure the needs of the whole HE community are met, the Committee represents key stakeholder groups and includes a student, a representative from the book industry, researchers in digital information retrieval and HCI and a psychologist. The composition and terms of reference of the Steering Committee are detailed below (section 9).

Top

6. Overall Project Structure

6.1. Workpackages and their Interrelationships

Workpackage No. Title

1 Project Setup

2 Web Site Development

3 Survey and Key Text Selection

4 Methodology

5 Key Text Evaluation

6 Data Analysis

7 Draft Guidelines

8 Implementation and Testing of Guidelines

9 Best Practice Guidelines

10 Dissemination of Guidelines

11 Closedown

Top

The following PERT diagram illustrates the interrelationships and dependencies between each workpackage:

Top

6.2. Milestones

By Month Milestone

1 Project set up: staff and committees in place; initial Web site complete

3 Initial survey of the range of HE material available on the Internet complete; completed detailed project plan

4 Key texts selected for use as material for the experiment

6 Methodology for evaluating selected texts developed, piloted and refined; selection of user groups

11 Large scale evaluation of key texts by each user group complete

13 Analysis of data complete and mechanisms identified for improving usability

14 Draft guidelines for publishing learning and teaching material on the Web collated

15 Recommendations tested; incorporation of feedback

16 Final set of best practice guidelines and recommendations for their implementation complete

17 Guidelines disseminated within and beyond the UK HE community

18 Project closedown activities complete and exit strategy implemented. Continued maintenance of Web site and availability of project outputs

Top

6.3. Gantt Chart

EBONI schedule Month: A S O N D J F M A M J J A S O N D J Post Project

Activity:

Project set-up, staff and committees in place

Dissemination strategy in place

Web site development

Completion of Project Plan

Survey of material and selection of key texts

Development of methodology; selection of user groups

Evaluation of texts by users

Data analysis

Formation of draft recommendations

Testing of recommendations and feedback of results

Collation of best practice guidelines

Dissemination of final guidelines and recommendations

Project report, closedown activities, implementation of exit strategy and continued maintenance of Web site.

Top

7. Detailed Project Plan

See Appendix A

8. Dissemination Strategy

The process of dissemination will aim to maximise awareness among creators of learning and teaching material and producers of digital resources and will include the following media:

a Web site has been set up at http://eboni.cdlr.strath.ac.uk/ to describe the project and its methodology, explain key developments as they occur and give details of relevant publications. Best practice guidelines will be applied to the site once completed, to illustrate how they might be implemented. The site encourages users to get in touch with enquiries or suggestions;

a mailing list (eboni@mailbase.ac.uk) has been set up to disseminate information and encourage discussion between EBONI staff and Steering and Management Group members;

a second, open mailing list (open-eboni@mailbase.ac.uk) has been established to disseminate information and encourage discussion among all stakeholder communities, and other interested parties;

appropriate mailing lists (such as lis-link, lis-elib and e-collections) will be emailed on a regular basis with key developments;

progress reports will be sent to relevant organisations and stakeholders;

papers will be periodically produced for submission to the appropriate professional literature;

papers will be submitted to relevant UK and international conferences;

news items will be submitted periodically to journals used by publishers of learning and teaching material and producers of digital information;

user groups participating in the evaluation process will be informed of project results via email;

a final report listing recommendations and guidelines will be sent to relevant organisations, targeting publishers of electronic material, similar or related programmes, libraries and museums involved in digitising collections and interested parties in the HE community in general;

EBONI will consult with the Programme Office and will be happy to set up mechanisms to enable joint approaches to dissemination and assisted take-up to be pursued;

the Project Manager is involved in the broader DNER strategy through participation in the Electronic Book Working Group;

a term of reference for the Steering Committee will be to help the project to establish and maintain a very high level of visibility in the UK HE community;

the Steering Committee will also provide information and feedback to stakeholder groups as appropriate;

the Project Team will talk to the LTSN about promoting EBONI's guidelines throughout the HE community, particularly through the subject centres.

during the second year of the project, when the guidelines are near completion, consideration will be given to holding a series of workshops to teach resource creators how to apply the guidelines. The workshops would need to have sufficient reach to achieve their objectives in a community-wide context;

EBONI will adhere to JISC PR Guidelines (http://www.jisc.ac.uk/pr/pr_strategy/) and any additional guidelines established by the DNER Communications Manager.

The project will work closely with the DNER in integrating its dissemination strategy with the appropriate DNER mechanisms.

Top

9. Steering Group Terms of Reference

9.1. Composition
The Committee will comprise the Project Manager and the Research Assistant along with the following independent members:

Chair: Hazel Woodward
University Librarian and Director of Cranfield University Press
Cranfield University
Cranfield
Bedfordshire MK43 0AL

Tel: 01234 754446
Fax: 012324 752391
Email: H.Woodward@cranfield.ac.uk

Cliff McKnight

Professor of Information Studies
Department of Information Science
Loughborough University
Loughborough
Leics LE11 3TU

Tel: 01509 223061
Fax: 01509 223053
Email: c.mcknight@lboro.ac.uk

Cliff's teaching interests include hypermedia and digital libraries and he has published widely on the usability of electronic journals and user centred design of hypertext for education. He is Senior Editor of the Journal of Digital Information.

Paul Mayes

Head of Academic Information Services
Library & information Services
University of Teesside
Middlesbrough TS1 3BA

Tel: 01642 342111
Fax: 01642 342190
Email: paul.mayes@tees.ac.uk

Paul is heading a project to monitor developments in both on-demand publishing and e-books with the aim of establishing a regional centre for good practice.

Tony Anderson

Lecturer
Centre for Research into Interactive Learning
Department of Psychology
University of Strathclyde
Room 585
Graham Hills Building
40 George Street
Glasgow G1 1QE

Tel: 0141 548 2583
Fax: 0141 548 4001
Email: Tony.Anderson@strath.ac.uk

Tony's research interests include computer-based learning, collaborative learning and hypertext, and the effect of software style on interaction around the computer.

Willie Anderson

Managing Director
John Smith & Son Bookshops
26 Colquhoun Avenue
Hillington
Glasgow G52 4PY

Tel: 0141 880 2500
Fax: 0141 880 2527
Email: wtca@johnsmith.co.uk

Willie is interested in the early setting of standards for electronic books from a commercial perspective.

Lesley Keen

Director
Total Immersion Ltd
325 Kilmarnock Road
Glasgow G43 2DS

Tel 0141 569 4209
Fax 0141 636 6286
Email: lesley@totaltrance.com

Lesley founded Scotland's first multimedia company in 1991, which developed a range of interactive children's books on CD-ROM for publishers including Oxford University Press and Dorling Kindersley. Her current work involves developing content for the Internet using dynamic server technologies. Lesley is an Honorary Research Fellow of the Computing Science Department of Glasgow University.

Norshuhada Shiratuddin

PhD Research Student
Department of Information Science
26 Richmond Street
University of Strathclyde
G1 1XH

Tel: 0141 548 4344
Fax: 0141 553 1393
Email: shuhada@dis.strath.ac.uk

Shuhada is studying for a PhD in e-book models and prototypes for smart school environments at the Department of Information Science, University of Strathclyde, prior to which she lectured at the Department of Information Technology, Universiti Utara Malaysia. Her research interests focus on the development of multimedia applications specifically for training and learning.

Paul Burton

Course Coordinator
Department of Information Science
University of Strathclyde

Tel: 0141 548 3706
Fax: 0141 553 1393
Email: paul@dis.strath.ac.uk

Paul has assured his assistance in finding student representatives from the Postgraduate ILS course for both years of the project.

Top

9.2. Conduct of Meetings

A Deputy Chair will be appointed from among the independent members.

The Steering Committee will meet (physically) twice in the first year and at least once in the second.

The Committee will reach decisions on the basis of consensus.

The quorum at meetings will be one half plus one member of the Project Team.

Members can nominate someone to attend in their place if unable to do so themselves.

The Project Director will be invited to each of the meetings of the Committee. Other individuals may be invited by the group to attend meetings.

Top

9.3. Main Functions of the Committee
The following are the key functions which the EBONI Steering Committee should undertake:

To oversee the project on behalf of the Higher Education Funding Council for England, the Scottish Higher Education Funding Council and the Joint Information Systems Committee (JISC), providing reports back to those agencies (via the Programme Office) from time to time;

To assist EBONI in aligning the work being undertaken with the development of the DNER, paying particular attention to interoperability within the overall managed framework and to current strategic issues and directions;

To represent the best interests of the broader Higher Education community in the United Kingdom in advising EBONI how best to develop a set of guidelines and recommendations commensurate with the information needs of learning and teaching, research and scholarship in UK universities and colleges;

To receive regular reports from the project on progress including the achievement of milestones and on future plans and associated milestones and deliverables; to comment on such plans in the light of available resources and the need for economy and efficiency; and to ensure that there are in place appropriate management structures and project management methodologies to secure the realisation of the plans;

To support EBONI and to help the project and its staff to further its aims, with particular reference to the need to maintain for the project a very high level of visibility in the UK HE community;

To monitor the quality of project work and outputs on an ongoing basis;

To oversee, review, comment on, and provide expertise during the preparation of draft guidelines;

To identify sources of information and expertise relevant to the project;

To act as stakeholder representatives to analyse issues, generate options and make recommendations concerning potential solutions;

To provide feedback to stakeholder groups as appropriate; and

To determine any areas of "overlap" with other UK projects and services and propose appropriate methods for coordinating in those areas.

The work of the Committee will be serviced from within the projects. The chair will be supported by the Programme Office.

Top

10. Quality Assurance and Evaluation

10.1. Quality Assurance
A number of mechanisms have been built into the project to monitor quality on a continual basis:

Pilot Phase
The methodology used to evaluate the usability of selected texts will be piloted first on a small group of around 10 subjects. This will enable the Project Team to identify problems with the experiment and ensure that they are fixed before the full-scale evaluation begins.

Testing Phase
A testing phase has been built into the latter stages of the project. Draft guidelines will be applied by experienced producers of digital content to an electronic text, and this text will be evaluated by a small selection of users. The re-evaluation results and the feedback from the content producer (specifically with regard to ease of implementation and usefulness of the guidelines) will be incorporated into the final guidelines.

Monitoring User Satisfaction with the Project
The EBONI Web site will encourage user comments, particularly during the latter stages of the project when the site is redesigned to reflect the results of the evaluation.

Further, subjects who participate in the experiment will be asked to complete a structured questionnaire designed to monitor their satisfaction with the procedure. Their feedback will be used to refine the experiment, if necessary.

Role of Management and Steering Groups
It is a main function of both the Management Group and the Steering Committee to monitor the quality of project work and outputs on an ongoing basis, particularly in the selection of texts to be evaluated, the development of a methodology, interpretation of data from the experiment, compilation of best practice guidelines, and dissemination activities.

In addition, the quality assurance issues of interoperability, scalability and sustainability are addressed in section 3.3, and EBONI's approach to accessibility is outlined in section 3.4.

Top

10.2. Evaluation Strategy

10.2.1. Purposes of evaluation
This section sets out a plan for evaluation activities in line with project aims and objectives for the duration of the project, with an emphasis on self-critical awareness of our performance, progress and impact. EBONI will be subject to a continuous process of review, evaluation and dissemination. Evaluation will be divided into formative and summative components, although the valuable iterative process means that the strategy will be focused on formative elements. The evaluation will be carried out by project staff on a continual basis, and monitored regularly by the Management Group, through meetings (every two months) and monthly progress reports.

The EBONI evaluation strategy follows the Guidelines for Project Evaluation developed by the Tavistock Institute, available at http://www.ukoln.ac.uk/elib/services/papers/tavistock/.

The purposes of evaluation are closely tied to the objectives of the project:

To provide evidence for the value of the best practice guidelines to users, and of the impact of the project at both individual and institutional levels;

To contribute to the overall learning in the programme as a whole that will be useful for future projects and programmes;

To improve performance by helping the project manage the process of developing, piloting and implementing best practice guidelines; and

To provide evidence as to the usability and cost-effectiveness of implementing the guidelines.

Top

10.2.2. Stakeholder groups
In order to achieve these goals, it is important to involve the key players whose cooperation and support are necessary for the project to succeed. The following groups have been identified as EBONI project stakeholders:

JISC, the DNER programme management, and related JISC projects and services;
EBONI project staff;
Students in Higher Education;
Information professionals;
Lecturers in Higher Education;
Students and lecturers involved in distance or off-campus learning;
Writers of scholarly digital information;
Publishers of scholarly digital information;
Researchers in digital information retrieval and HCI;
Psychologists;
The National Grid for Learning; and
Funding and other agencies which invest in the creation of scholarly digital resources and/or relevant research and development initiatives.

For each stakeholding community the project will:

assess how to involve them in the project;

identify what information or data needs to be provided at different stages in the project life-cycle to inform their decision making; and

evaluate where and into what EBONI activities, input may be most appropriately solicited from the community's members.

Top

10.2.3. Evaluation questions
In consultation with the Steering Committee and Management Group, the following questions that the evaluation will endeavour to answer (as understood at the present time) have been identified:

Are JISC-funded resources being used appropriately, and do they provide value for money?

Are the objectives being met?

Are the milestones being achieved on time?

Are the outcomes being achieved?

Are the needs of a HE community of readers of digital content being met?

Are the needs of a HE community of creators of digital content being met?

Are the various stakeholders participating as expected? Is there sufficient involvement from "professionals from NGfL" for example?

Should another group of people, not previously identified, be involved?

Is everyone working to a common agenda?

Do the various forces pull in the same direction?

Is everyone happy with the way the project is going?

Are the individuals involved in the project happy, and do they have successful working relationships?

Are the various approaches to the design of learning and teaching resources represented in the material selected for evaluation?

Does the methodology for evaluating different versions of electronic texts adequately measure usability?

Are the best practice guidelines appropriately documented for ease of use?

Are the best practice guidelines reaching the UK HE and stakeholder communities?

Are all the dissemination activities being fulfilled?

Is time being taken to reflect on the process?

Top

10.2.4. Priorities for Formative Evaluation
Evaluation and quality assurance are integral components of EBONI and, as such, some of the elements identified here as part of the formative evaluation are built into the project plan and are key elements of the methodology of the project. For example, the experimental stage of the project involves five months of intensive user evaluation, the outcome of which will form the basis of the final product. Therefore, the project aims to place particular emphasis on gathering ongoing feedback from participants in EBONI activities, and users of the project's resources.

Evaluation of the project must be based firmly within the project's overall objectives. However evaluation cannot be unlimited within a specific project budget. Although it would be desirable to address all of the questions listed above, within the project budget and timetable it is more appropriate to identify the key areas and focus our evaluation activities on these:

Project Management
The progress and quality of the project as it develops will be continually monitored; the Management Group will meet twice a month and, in addition, will receive monthly progress reports. We will evaluate whether JISC-funded resources are being used appropriately and are providing value for money and whether objectives are being met and milestones being achieved on time, and monitor whether the Project Team are happy with the work and are maintaining successful working relationships.

Users
The meeting of user requirements is central to the success of EBONI. Therefore, the project will monitor the needs of a HE community of readers and creators of digital content (and other stakeholders) and ask if their needs are being met.

Methodology Evaluation
A sound methodology will be key to the integrity of the guidelines. Evaluation activities will therefore focus on the questions: are the various approaches to the design of learning and teaching resources represented in the material selected for evaluation?; and does the methodology for evaluating different versions of electronic texts adequately measure usability?

Dissemination Evaluation
The success of the guidelines is largely dependent on their wide dissemination throughout the stakeholder communities. EBONI will be monitoring whether the best practice guidelines are appropriately documented for ease of use, they are reaching the UK HE and stakeholder communities, and all the dissemination activities (outlined in the dissemination strategy) are being fulfilled.

The timing of evaluation activities will relate to the life-cycle of each strand of the project's activity, as outlined below (10.2.6).

Top

10.2.5. Methods and Techniques for Answering these Questions

Feedback from Stakeholders
A broad-based Steering Committee, including representatives of the most important users of project outputs (information professionals, lecturers, students, and researchers in related fields) has already been set up and will provide formative feedback and assist with disseminating learning throughout the project. Its composition and terms of reference are outlined in section 9 of this document. The Steering Committee's views on evaluation requirements have been sought and form the basis of this strategy.

With specific regard to the priority areas outlined above, the Committee will:

be chaired by a representative of the DNER, ensuring JISC-funded resources are being used appropriately and provide value for money;

advise on whether the various approaches to the design of learning and teaching material are represented in the material selected for evaluation;

include student representatives to ensure the needs of a HE community of readers of digital content are being met;

report back to stakeholder groups to ensure the guidelines are reaching stakeholder communities;

include a psychologist who will ensure a sound methodology, or as sound as possible, and point out potential pitfalls; and

receive regular progress reports from the Project Team.

Feedback from Peers
The eBook Group, based at Strathclyde and comprising the EBONI Project Team and researchers in electronic books for children, on-demand publishing and use of the Internet for selling books, will meet weekly. The group will aim to benefit all members through the exchange of knowledge and experience in the e-book field, particularly by:

reporting on progress with individual research;

discussing areas of common ground or overlap;

advising on sources of relevant information; and

inviting guest speakers in the field to impart knowledge and advice.

In addition, the Project Manager is a member of the DNER Electronic Book Working Group, which ensures involvement in the broader DNER strategy and aims to encourage collaboration between projects in this field.

Top

Feedback from Creators of Content
Experienced creators of digital content will be consulted at different times throughout the development process to provide feedback about the guidelines from the point of view of the potential implementers. They will particularly be involved in the testing of draft guidelines. During this phase they will implement the guidelines on an electronic text and provide feedback on their usefulness and ease of application. In addition, they will:

provide feedback on the development of the project, enhancing the usability of the guidelines; and

help EBONI to model conditions under which the guidelines will be applied, and guide project direction appropriately.

This will complement other data gathering exercises as part of an ongoing process of adjustment and project refinement and will be useful for exploring likely reactions to the guidelines when they are launched and how they can be tailored to the changing requirements of users when they are introduced into their working practices and wider working environment.

End-user Involvement
End users will be involved as active participants in the development process, increasing the likelihood that the final set of guidelines will meet the needs of users and achieve project goals. The user communities being addressed are set out in section 3.4 of this document.

During months 7-11, all users will be involved in both quantitative and qualitative aspects of the experiment (the full methodology is set out at 3.2):

quantitative feedback will be sought by asking users to search selected material for specific information and to participate in memory tasks. Success in answering questions correctly, time taken to complete tasks, ability to recall information and (depending on facilities available) covert observation of users' behaviour will be taken into account in interpreting this feedback; and

qualitative feedback will be sought via questionnaires and interviews immediately following the quantitative phase, and will aim to record users' subjective satisfaction with the experience of reading the material. Consideration will be given to using "Think Aloud" techniques for recording users' thoughts as they read and use the texts.

The results of this experiment will feed directly into the draft set of guidelines. Users will be involved once again, towards the end of the project, in testing the effectiveness of these guidelines, their responses being used to identify any major design problems, suggest improvements and guide revisions.

Top

Self-monitoring
EBONI will produce two reports in its first year, in January and July 2001, to be submitted to the Programme Office, in addition to more regular progress reports submitted to the Steering Group. These will aim to capture and disseminate evaluative information, in particular:

reporting on progress against the project plan;

reporting on dissemination activities carried out;

outlining reasons for any changes to the plan, including changes to aims and objectives in the light of experience;

providing interim evaluation results;

indicating how expectations about the project have evolved; and

reporting on cultural change effects, learning, usefulness and future scenarios.

As a result, project staff will constantly be critically aware of and, indeed, focused on the priority areas for formative evaluation. Common elements in project reports will include:

material selection requirements;

user selection requirements;

progress with methodology design and implementation; and

progress with production of best practice guidelines.

Collection of Data
EBONI will collect and assess formative user feedback in a structured and systematic fashion using techniques which will vary according to the product/activity being examined. The following methods will be included:

survey of the range of learning and teaching material available on the Internet

questionnaires

retrieval tasks

memory tasks

covert observation of users' behaviour while undertaking the tasks (depending on facilities available)

structured interviews

Utilisation
Evaluation will be integrated into the project in the form of reports to the Steering Committee (representing the major stakeholders) and the Programme Office, the exchange of knowledge and ideas through the Electronic Books Group, and attendance at conferences. All of these activities will aim to capture the learning from the project and make it available to other and future research, ensuring that the process of evaluation is useful and increases the prospects for the uptake of the final set of guidelines.

Top

10.2.6. Timetable

Months 1-3
· Develop evaluation strategy
· Establish Management Group
· Establish Steering Committee
· First Management and Electronic Book Group meetings
· Ask Steering Committee about their evaluation requirements

Months 4-6
· Survey of available material
· Identification and selection of material
· Identification and selection of user groups
· Conduction of pilot experiment
· Project Report
· Management and Electronic Book Group meetings
· First Steering Committee meeting

Months 7-11
· Large scale user evaluation
· Management and Electronic Book Group meetings

Months 12-15
· Summative evaluation begins
· Testing of draft guidelines and incorporation of feedback
· Management and Electronic Book Group meetings
· Steering Committee meeting
· Project Report

Months 16-18
· Summative evaluation
· Management and Electronic Book Group meetings
· Steering Committee meeting
· Final Project Report

Top

10.2.7. Summative evaluation
Summative evaluation will be undertaken by the Project Team who will report to the Management Group and the Steering Committee and will provide evidence of achievements and effects.

The results of formative evaluation reports will be synthesised and measured against overall project aims, ensuring the outputs from each stage of the project make an appropriate contribution to the achievement of overall project goals and that suitable levels of quality are maintained.

Summative evaluation will begin in the second year of the project as the draft set of guidelines is being produced and will be completed in the closedown phase, culminating in the final project report.

Top

11. Risk Assessment

A risk assessment exercise has highlighted a number of risks to the successful outcome of the project and are summarised in the following sections:

Staff
EBONI has only one full-time member of staff and a part-time Project Manager. Therefore, one of the more likely risks to the project is that the absence of the Research Assistant could cause work to be set back and deadlines and opportunities to be missed.

This risk can be reduced through the strong communication lines between the Research Assistant, the Project Manager and the Project Director, all of whom have regular input to the project and meet frequently to discuss progress. Therefore, in the event of the absence of the RA, the rest of the team will know the status of the project and be aware of tasks needing completion.

Staff risks such as this would have short term implications and could delay overall progress, but the risks can be minimised and effects limited to ensure they are not detrimental to the success of the project in the long term.

Institutional Support
EBONI is a project within the Centre for Digital Library Research and, as such, will be affected by changes undergone by the Centre as a whole. For example, it is likely that the CDLR will move to new offices during the lifespan of the project; staff time taken up with moving could result in minor delays to EBONI.

Such institutional risks can be minimised by maintaining strong links between the project and the CDLR, ensuring that EBONI staff are kept informed of likely future changes, and that the CDLR is aware of EBONI's needs.

Lack of Uptake
A lack of uptake of the guidelines among stakeholder communities would be detrimental to the project, as feedback is necessary to "fine-tune" the guidelines.

This risk can be diminished by the adoption of an aggressive dissemination strategy to increase awareness and encourage wide uptake. Moreover, the involvement of a producer of digital content at various stages throughout the project will enable EBONI to identify from an early stage any potential problems in the uptake of the guidelines, and to act on this knowledge.

The likelihood of this risk is low but the effects would be detrimental to the impact of the project as a whole.

Rate of Change of Technology
The ebook arena is currently undergoing vast changes at a fast pace and, while the production of new handheld hardware devices may not directly impact on the project, developments in browser technologies or markup languages could have an effect on the design of new material online, and the ways in which users interact with this material. This could result in the project's guidelines becoming outdated.

The EBONI Project Team will diminish this risk by keeping up to date with such developments through monitoring the popular and professional press for announcements of new technologies and considering their implications.

Top

12. Budget

Year 1

Staff Costs
Research Assistant (RA1B)
Project Manager (.2 ASO4)
Staff Sub-total
£
21,978
7,348
29,326

Equipment Costs
PC, HTML editor and suitable software for RA
1,500

Travel and Subsistence
To cover attendance at appropriate conferences and meetings 1,000

Evaluation Costs
Payment to subjects participating in the user study. Estimated at £30 each for first 50 students 1,500

Year 1 Total Costs 33,326

Year 2 (6 months)

Staff Costs
Research Assistant (RA1B)
Project Manager (.2 ASO4)
Staff Sub-total
£
11,374
3,803
15,177

Travel and Subsistence
To cover attendance at appropriate conferences and meetings, and cost of workshops 1,000

Dissemination Costs
Costs incurred in publication of final set of guidelines 500

Evaluation Costs
Payment to further 50 subjects participating in user study 1,500

Year 2 Total Costs 18,177

Staffing costs include NI and USS contributions and have been incremented by 3.5% a year to account for national salary increases/increments.

The total budget over the life of the project will be £51,503.

Top

Appendix A: Detailed Project Plan

Workpackage Number: 1
Workpackage Title: Project Setup

Start Date: 01 August 2000
End Date: 31 August 2000

Objectives:
· To put in place staff and set up Management and Steering Committees;
· To purchase and set up necessary equipment; and
· To design an initial Web site.

Brief Description of Work: Establish mechanisms for beginning EBONI in earnest: hire staff, select Committees, buy equipment and create a basic Web presence.

Quality Assurance & Review: The Project Team will enable mechanisms for quality assurance and review, namely by creating an evaluation strategy and setting up the Management and Steering Committees.

Outputs including reports: Production of initial Web site.

Task Descriptions

Task Title Description

1.1 Staff in place EBONI Research Assistant and Project Manager begin work.

1.2 Management Group Select and contact Management Group. Arrange initial meeting.

1.3 Steering Committee Select and contact Steering Committee.

1.4 Equipment Purchase and set up PC for Research Assistant.

1.5 Mailing lists Set up two mailing lists: one for use by EBONI staff and the Management and Steering Group, the other for the wider stakeholder community.

1.6 Initial Web site Establish an initial Web presence, containing an overview of EBONI, contact details and the project proposal.

Top

Workpackage Number: 2
Workpackage Title: Web Site Development

Start Date: 01 August 2000
End Date: 31 January 2002

Objectives:
· To make available all information relating to the project, including reports, meeting minutes, articles and other project outputs on the Internet;
· To keep this information up-to-date; and
· To alter the design of the site to reflect findings throughout the project.

Brief Description of Work: This workpackage involves the ongoing evolution of the EBONI Web site, keeping it up-to-date, adding project outputs, and revising the design to reflect the guidelines.

Quality Assurance & Review: The Web site will be subject to ongoing revision, based on findings throughout the project. A feedback mechanism is available at the site.

Outputs including reports: The EBONI Web site

Task Descriptions

Task Title Description

2.1 Update Keep contact details of staff, Management and Steering Group members up to date; maintain any other dynamic information.

2.2 Add Make regular additions to the content in the form of progress reports, and include any other project outputs that are produced, such as articles, minutes and presentations.

2.3 Revise Make any necessary changes to the design of the site, in accordance with the guidelines that are produced.

Top

Workpackage Number: 3
Workpackage Title: Survey and Key Text Selection

Start Date: 01 August 2000
End Date: 30 November 2000

Objectives:
· To identify the range (in terms of design and subject) of teaching material available on the Internet; and
· To select resources to be used as the material for evaluation.

Brief Description of Work: A survey of the range of teaching material available on the Internet, identifying, classifying and finally selecting resources according to techniques used (hypertext, tables on contents, etc.) and subject (looking at schemes such as the DNER collection policy and divisions of subjects within the RDN).

Quality Assurance & Review: The Steering Committee will be asked to advise on identifying material. The Steering Committee and the Electronic Book Group will review the collection of resources and the Management Group will oversee the selection of key texts. EBONI will collaborate with the Digital Information Office, which is carrying out similar work.

Outputs including reports: A section of the Web site will be devoted to this workpackage, and will probably take the form of an organised bibliography or a small catalogue of learning and teaching material.

Task Descriptions

Task Title Description

3.1 Survey Conduct extensive Internet search of UK HE teaching material available online.

3.2 Classification Classify this material by subject area and by technique.

3.3 Collation Display the collated information on the Web site, in the form of a bibliography or small catalogue.

3.4 Selection Select key texts from the collection which are representative of various subject areas and methods.

Top

Workpackage Number: 4
Workpackage Title: Methodology

Start Date: 01 November 2000
End Date: 31 January 2001

Objectives:
· To develop a methodology for evaluating selected guides, tutorials and textbooks; and
· To measure overall usability by calculating:
o The ease with which users can retrieve the information they need from the text; and
o Users' subjective satisfaction with the experience of reading the material on the Internet.

Brief Description of Work: The development and implementation of a user-centred methodology for evaluating the various techniques employed in producing educational material online.

Quality Assurance & Review: The Steering Committee and Management Group will advise on methodology creation. In particular, the psychologist on the Steering Committee will advise on developing a sound methodology and avoiding pitfalls. A pilot stage has been incorporated into the methodology development to ensure that problems are identified and fixed before the full-scale evaluation.

Outputs including reports: A report on the creation of a methodology, including a detailed description and an outline of any difficulties encountered in devising and implementing it.

Task Descriptions

Task Title Description

4.1 Literature Review Review Conduct a literature review of the requirements of students and academics on the Web and use this as the basis for developing a concept of "usability".

4.2 Consultation Consult the Steering Committee as to what to measure, how to measure it, etc.

4.3 Development Develop an initial methodology, based on the concept of usability derived from the literature review and the consultation with the Steering Committee.

4.4 Pilot Pilot the methodology on a small group of around 10 users, identifying problems with the methodology.

4.5 Methodology Incorporate feedback from the pilot session to produce a methodology to be used for the full-scale evaluation. [N.B. Ongoing feedback on the methodology will also be sought and the experiment will be refined if necessary.]

4.6 Experiment Conduct the experiment (next workpackage).

Top

Workpackage Number: 5
Workpackage Title: Key Text Evaluation

Start Date: 01 February 2001
End Date: 30 June 2001

Objectives:
· To engage users in evaluating the usability of a variety of texts on the Web by collecting:

Quantitative data from the performance of search and memory tasks; and

Qualitative data from the completion of questionnaires and participating in interviews.

Brief Description of Work: Participants selected for the experiment will be asked to search the key texts for specific information, participate in memory tasks, fill out questionnaires and be interviewed by the EBONI Research Assistant.

Quality Assurance & Review: Each subject will be asked, at the end of their participation, to indicate their satisfaction with the experiment in the form of a questionnaire. The procedure will be overseen by the Management Group.

Outputs including reports: Raw data from questionnaires, the completion of tasks and interviews.

Task Descriptions

Task Title Description

5.1 Selection Select 80-100 paid subjects to evaluate the key texts, mainly from the HE population at Glasgow's three universities.

5.2 Quantitative data collection Instruct each subject to carry out specific search and memory tasks using key texts

5.3 Qualitative data collection Instruct each participant to complete a questionnaire, and ask them to participate in a structured interview.

5.4 Record the results Record the results for each subject accurately and systematically.

Top

Workpackage Number: 6
Workpackage Title: Data Analysis

Start Date: 01 February 2001
End Date: 31 August 2001

Objectives:
· To identify which techniques/styles are most successful in enabling users to retrieve, quickly and easily, the information they require; and
· To identify the different requirements of subjects from different disciplines, under- and postgraduates, and mature, distance and part-time learners.

Brief Description of Work: Quantitative and qualitative feedback from tasks, questionnaires and interviews will be analysed to determine the overall usability of each text. Data will also be analysed on a comparative basis, to identify differences between the needs of the representative user groups participating in the experiment.

Quality Assurance & Review: The Steering Committee will oversee the interpretation of the data. Problems with the procedure will be addressed to Tony Anderson who represents Strathclyde's Psychology Department on the Steering Committee.

Outputs including reports: This analysis will form the basis of the guidelines for the design of learning and teaching material on the Internet.

Task Descriptions

Task Title Description

6.1 Collation Collate the results from each individual participant

6.2 Analysis Analyse the results, using graphs and charts to illustrate differences in usability between texts

6.3 Comparison Analyse the results on a comparative basis, identifying differences between different groups of users.

Top

Workpackage Number: 7
Workpackage Title: Draft Guidelines

Start Date: 01 September 2001
End Date: 30 September 2001

Objectives:
· To interpret the outcome of the data analysis, identifying which techniques are most successful for each group; and
· Using this information about the success of various methods, construct a set of guidelines advising which techniques are most effective for producing electronic teaching material.

Brief Description of Work: Compilation of a draft set of best practice guidelines for producing educational material on the Web, based on the outcome of the data analysis.

Quality Assurance & Review: The Management Group and the Steering Committee will oversee the interpretation of the data and compilation of guidelines.

Outputs including reports: Draft set of best practice guidelines

Task Descriptions

Task Title Description

7.1 Interpretation Interpret the results, identifying which techniques are most successful for each group.

7.2 Produce Guidelines Use this interpretation to construct recommendations for how best to produce guides, textbooks and teaching material online.

Top

Workpackage Number: 8
Workpackage Title: Implementation and Testing of Guidelines

Start Date: 01 October 2001
End Date: 31 October 2001

Objectives:
· To test the ease of implementation and applicability of the guidelines; and
· To test the overall difference in usability the guidelines can make.

Brief Description of Work: The draft guidelines are applied (by a creator of digital material) to an electronic text and evaluated once again by a selection of students.

Quality Assurance & Review: Feedback from re-evaluation, the creator of digital material and Steering Committee.

Outputs including reports: Feedback from discussion with the content creators and data from the re-evaluation

Task Descriptions

Task Title Description

8.1 Application Draft guidelines are implemented by an experienced digital content creator on an electronic text.

8.2 Discussion with Digital Content Creators The digital content creators comment on how useful the guidelines are and their ease of application.

8.3 Re-evaluation A smaller selection of users engage in the tasks outlined in Workpackage 5, using the text on which the guidelines have been implemented.

8.4 Recording The outcome of the discussion with the content creator and the re-evaluation results are recorded.

Top

Workpackage Number: 9
Workpackage Title: Best Practice Guidelines

Start Date: 01 September 2001
End Date: 30 November 2001

Objectives:
· To incorporate the comments of the digital content creator and the results of re-evaluation to produce a final set of best practice guidelines.

Brief Description of Work: Using the feedback from Workpackage 8, compile a revised, final set of best practice guidelines for the publication of guides, tutorials and textbooks on the Internet for the UK Higher Education community.

Quality Assurance & Review: The Management Group and the Steering Committee will oversee the revision of the guidelines.

Outputs including reports: Final set of best practice guidelines

Task Descriptions

Task Title Description

9.1 Analysis Analyse the digital content creator's comments on the draft guidelines and the results of the evaluation of the text to which they were applied.

9.2 Interpretation Interpret the implications of this feedback and identify necessary changes to be made to the guidelines in light of the re-evaluation and fresh feedback.

9.3 Incorporation Incorporate these findings into the draft guidelines to produce a revised, final version.

9.4 Production Produce the guidelines in an easy-to-use format for distribution among the stakeholder communities.

Top

Workpackage Number: 10
Workpackage Title: Dissemination of Guidelines

Start Date: 01 December 2001
End Date: ongoing

Objectives:
· To ensure the final guidelines and recommendations reach the UK HE community and other stakeholders.
· To maximise take-up of the guidelines.

Brief Description of Work: Disseminate the final set of recommendations as widely as possible in the UK HE and other stakeholding communities to maximise take-up of the guidelines.

Quality Assurance & Review: A term of reference for the Steering Committee will be to provide information and feedback to stakeholder groups.

Outputs including reports:
· The final recommendations and guidelines will be sent to relevant organisations, targeting publishers of electronic material, similar or related programmes, libraries and museums involved in digitising collections and interested parties in the HE community in general;
· News items will be submitted to journals used by publishers of learning and teaching material and producers of digital information, announcing the outcome of the project and advertising the availability of the information;
· The guidelines will be available from the project Web site;
· Consideration will be given to holding a series of workshops to teach resource creators how to apply the guidelines. The workshops would need to have sufficient reach to achieve their objectives in a community-wide context.
· EBONI will consult with the Programme Office and will be happy to set up mechanisms to enable approaches to dissemination and assisted take-up to be pursued.

Task Descriptions

Task Title Description

10.1 Web site publication Publish the guidelines on the project Web site and apply them to the design of the site.

10.2 Prepare news items Prepare news items and send to relevant journals.

10.3 Distribution Distribute the guidelines as widely as possible (given financial constrictions) in the stakeholder communities.

10.4 Workshops Depending on time and resources available at this stage of the project, design and implement a series of workshops teaching how to implement the guidelines.

Top

Workpackage Number: 11
Workpackage Title: Closedown

Start Date: 01 January 2002
End Date: 31 January 2002

Objectives:
· To complete successfully all project work; and
· To set up mechanisms to ensure the maintenance of the Web site and the continued availability of project outputs.

Brief Description of Work: Project report, closedown activities, implementation of exit strategy, and continued maintenance of Web site.

Quality Assurance & Review: The Steering Committee and the Management Group will oversee the production of the final report.

Outputs including reports: Final Project Report

Task Descriptions

Task Title Description

10.1 Project Report Compile a Final Report detailing activity throughout the 18 months of the project and containing evaluative information and a breakdown of expenditure.

10.2 Closedown Complete closedown activities and implement exit strategy.

10.3 Maintenance Arrange for the Web site to be maintained, and for project outputs to be available after funding has expired.

Ruth Wilson
31 October 2000

Top