Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
The QA Focus Perspective Marieke Guy UKOLN [email protected] http://www.ukoln.ac.uk/qa-focus/ QA Focus – Supporting JISC's 5/99 Programme The QA Perspective If you want to sort out the problems on your Web site you can do so by developing a Quality Assurance Framework This involves: • Testing • Problem tracking and corrective action • Policies • Procedures • Documentation QA Focus – Supporting JISC's 5/99 Programme What Can You Test? • • • • • • • • • • • Functional testing Compatibility testing Load/performance testing Stress testing Usability testing Security testing Integration of unit testing Link testing HTML and CSS validation Reliability testing Regression testing • … QA Focus – Supporting JISC's 5/99 Programme Automated Testing • Should be scalable • Only suitable approach for large sites • Can push (email) information about problems • Many free testing tools available but: • Will not spot all usability/accessibility problems • Can provide too much information • Comprehensive and configurable testing tools can be expensive QA Focus – Supporting JISC's 5/99 Programme Manual Testing • Can spot usability/accessibility problems • Can make use of one’s community/user base/peers • Can use of key pages which will identify problems which occur elsewhere • Useful for testing new Web sites • “5 users can spot most of your problems” but: • Not suitable for testing large numbers of pages • Quality of feedback may be variable • Often not useful for testing established Web sites QA Focus – Supporting JISC's 5/99 Programme Approaches to Automated Testing Automated testing can make use of: • Web-based tools • Desktop tools UKOLN has made use of Web-based testing tools: • Monitor accessibility, HTML, CSS, compliance, page size, links quality, nos. of links, etc. • Benchmarking approach provides comparison with one’s peers http://www.ukoln.ac.uk/qa-focus/surveys/ QA Focus – Supporting JISC's 5/99 Programme Peer Review • “Give me comments on my Web site and I’ll also test yours” • Can help in community-building • Particularly useful for programmes in which project Web sites being developed at same time But: • Can be difficult in processing unstructured responses • Need to manage process QA Focus – Supporting JISC's 5/99 Programme Problem Tracking There are three stages in a corrective action system: • The short term fix • The cure • Prevention of errors – identify potential causes of error by analysing data to setect trends which if allowed to continue would result in errors QA Focus – Supporting JISC's 5/99 Programme Documentation 3 Levels of documentation: • Policy e.g. Quality Manual • System e.g. Processes and procedures • Work practices e.g. instructions More detailed, more specific, applies to fewer people QA Focus – Supporting JISC's 5/99 Programme Definitions Policy A plan or course of action intended to influence and determine decisions Procedure A way of performing or effecting something Process A series of actions, changes, or functions bringing about a result QA Focus – Supporting JISC's 5/99 Programme Documentation • Good documentation • • • • • • • • • • Policies, procedures and audits Define procedures for all processes Requirements Specifications Mission statements or statement of direction Define your audience Document the testing suite and tools Document authoring tools etc. Document problems tracking A Quality Management System (QMS)?? QA Focus – Supporting JISC's 5/99 Programme Issues I • Goals for testing will be related to the kind of site you have • Testing time is limited – use automated tools • Documentation is critical – to make sure practices are repeatable • Implement your QA procedures slowly over time to avoid stifling productivity • QA planning at the start of development means less time fixing things at the end QA Focus – Supporting JISC's 5/99 Programme Issues II • Motivating/persuading people to implement a QA Culture can be difficult – run workshops to show people what you are aiming for • Make sure you have a feedback mechanism for the procedures you are implementing • Having a great team/management is crucial QA Focus – Supporting JISC's 5/99 Programme The QA Focus Perspective QA Focus: • Funded by JISC • Currently provided by UKOLN and AHDS • Currently supports JISC's 5/99 programme • Staff: • • • • Brian Kelly, Project manager Marieke Guy, QA Focus officer, UKOLN Hamish James, QA Focus officer, AHDS Gareth Knight, QA Focus officer, AHDS QA Focus – Supporting JISC's 5/99 Programme Approach Taken Two possible approaches to ensuring compliance with standards and best practices: Enforce • Inspect all project’s work • Strict auditing, with penalties for nocompliance The latter approach is preferable, Encourage especially in a HE context • Training for project staff • Developmental, explaining reasons for compliance, documenting examples of best practices and providing advice on implementation and monitoring QA Focus – Supporting JISC's 5/99 Programme QA Focus Work • Documentation and Advice • • • • Briefing Papers Case Studies FAQs Descriptions of tools and architectures for projects to implement best practices • In-house QA • Surveys • Providing Motivation • Workshops • Liasing • The QA Focus Toolkit QA Focus – Supporting JISC's 5/99 Programme Documentation: Briefing Papers Advisory briefing documents are being produced These are: • Brief, focussed documents • Informed by findings of the surveys QA Focus – Supporting JISC's 5/99 Programme Documentation: Case Studies Case Studies are being commissioned These are: • Written by projects themselves • Describe the solution adopted to a particular problem • Include details of lessons learnt – not just a press release! QA Focus – Supporting JISC's 5/99 Programme Web Site Testing Tools • Full list of Web site testing tools available from database • Surveys run with a number of the tools • In the future there will be reviews of tools http://www.ukoln.ac.uk/qa-focus/qa-projects/ QA Focus – Supporting JISC's 5/99 Programme Describing Tools and Approaches As an example of approach which provides easier access to testing tools, see the ,tools approach deployed on UKOLN Web site. This approach: • Applies to all resources on Web site • Covers HTML and CSS validation and various other tests • Some recursive apps (,rvalidate) • Easily implemented withJISC's single line redirect QA Focus – Supporting 5/99 Programme In-house QA • Policies • Procedures • Audits To date: • Web Standards - XHTML, CSS • Linking - links to and from • Accessibility • Usage Statistic • Metadata QA Focus – Supporting JISC's 5/99 Programme Documentation and Advice I We have: • Surveyed project Web sites and identified areas of lack of compliance with standards and best practices • Demonstrated examples of the potential importance of compliance for repurposing resources QA Focus – Supporting JISC's 5/99 Programme Documentation and Advice II In addition we need to provide: • Brief focussed advice on the standards • Information on how to monitor compliance • More Case studies on solutions deployed by projects themselves • Guidance on dealing with implementation difficulties and what to do when strict compliance is difficult to achieve • Reviews of testing tools QA Focus – Supporting JISC's 5/99 Programme Surveying The Community Surveys of project Web sites have been carried out in order to: • Obtain a profile for the community • Identify examples of best practices • Identify areas in which further advice is needed Surveys included: • Accessibility • 404 error pages • HTML & CSS compliance QA Focus – Supporting JISC's 5/99 Programme Survey Philosophy I The surveys made use of freely-available Webbased tools: • Methodology is open • No software needs to be installed locally (apart from Web browser) • Findings can be reproduced • Latest results can be obtained by clicking on link to testing service QA Focus – Supporting JISC's 5/99 Programme Survey Philosophy II The surveys typically examined project entry points and not entire Web site as: • This page has the highest profile • The aim is to validate a methodology which can be deployed by projects themselves, not to test every page on behalf of the projects QA Focus – Supporting JISC's 5/99 Programme Survey Findings Findings available from http://www.ukoln.ac.uk/qa-focus/surveys/ QA Focus – Supporting JISC's 5/99 Programme Providing Motivation There is a need to explain why compliance with standards is important and to provide motivation for projects to update their tools, authoring procedures, etc.This may involve • More workshops • Evaluating project's QA plans • Liaising with projects over the requirements of the key QA procedures applicable to their work • Liaising with the Advisory Services over their provision of advice and address maintenance and feedback mechanisms • Overseeing QA work QA Focus – Supporting JISC's 5/99 Programme Self Assessment Toolkit We will be developing a self-assessment toolkit for projects to use, by individual projects or across project clusters The toolkit will consist of: • Examples of QA procedures • Documented examples of use of testing tools • Self-assessment questionnaires • Advice on standards and best practices • Case studies, FAQs etc. http://www.ukoln.ac.uk/qa-focus/toolkit/ QA Focus – Supporting JISC's 5/99 Programme QA for Digitisation Do it once…..do it right: • Project is fundamentally dependent upon the quality of original product • Quality is the pre-requisite to preservation • Quality expectations will only grow • Delivery problems can be fixed, but capture problems normally can’t QA Focus – Supporting JISC's 5/99 Programme QA For Web Sites • The Web is the main delivery mechanism for projects and services • An increasing awareness of the importance of accessibility, use of new devices (PDAs, WAP, e-books, …), repurposing of Web content (e.g. archiving) • But invalid HTML is still the norm • Look to standards – XHTML, XML, CSS, WAI • If proprietary formats need to be used, flag them and use in most open way • QA Focus is trying to explain why and how QA Focus – Supporting JISC's 5/99 Programme Next Steps Next Steps Extended Coverage We will be moving on from Web and digitisation to include other areas including: • Metadata • Multimedia • Software development • Deployment into service •… QA Focus – Supporting JISC's 5/99 Programme Next Steps – Metadata Thoughts on QA for metadata: Syntax Check syntax on embedded DC Cataloguing Rules Ensure projects have appropriate cataloguing rules Interoperability Ensure metadata can interoperate with third parties Fitness for purpose Ensure metadata is appropriate for its purpose Change control Ensure architecture for managing metadata can cope with change control (cf Exploit Case Study) QA Focus – Supporting JISC's 5/99 Programme Next Steps – Software Thoughts on QA for software: Specification Methodology Testing Deployment Process for developing systems spec, agreeing spec with stakeholders, etc. Documentation of systems development methodology (cf UML case study) Documentation of testing methodology Awareness of deployment challenges QA Focus – Supporting JISC's 5/99 Programme Limitations There are a number of limitations to the work we have carried out so far: • Project Web sites have different purposes (information about the project; communications with project partners; project deliverables themselves; etc.) • Projects have different levels of funding, resources, expertise, etc. • Projects are at different stages of development (and some have finished) QA Focus – Supporting JISC's 5/99 Programme QA Focus And You! What relevance has QA Focus for you? • You can deploy QA Focus practices for your projects, etc • You can write case studies and advisory documents: • Seen to be working with JISC • We can promote your approaches and work • This will help instigate best practices within your group • You can develop QA procedures in innovative areas (e.g. OAI, thesaurii, ePrints, e-Books, etc.) which QA Focus can use QA Focus – Supporting JISC's 5/99 Programme Questions Any questions? QA Focus – Supporting JISC's 5/99 Programme