Executive Summary
The 'Evaluation Challenges' model - issues in IT investment and usage
Cost and risk issues
Benefits and value concerns
Addressing the IT productivity paradox
Organizational variations in IS/IT performance
IT eras - a further complication to the measurement problem
The systems-centric era - 1964-1981
The PC -centric era - 1981-1994
The network-centric era - 1994-2005
A content-centric era 2005 -?
Implications for IT assessment practices
Answering the challenges - life-cycle evaluation
Alignment
Prioritization
Feasibility
Development and implementation
Post-implementation
Ongoing operations
Evaluating IT sourcing options
Evaluation - additional perspectives and enhancements
Cost/contribution model and IT value
The value proposition - external and internal customers
Developing IT life cycle evaluation as a process

2 A SURVEY OF CURRENT EVALUATION TRENDS AND PRACTICES

Executive summary
IT evaluation issues - overview
New evaluation methods
Assessing IT evaluation practice and IT performance
'Inhibitors' to IT evaluation
Perceived IT quality (rated by respondents)
Customer expectations - measurement and perceived IT performance
The evaluation of IT investment proposals
Evaluation criteria for IT investment proposals
Problems in evaluating proposals for IT investment
Benefits from evaluating investment proposals
Evaluation of IT development projects
Organizations evaluating at the development/project stage
Reasons for project abandonment
Problems experienced in IT project evaluation
Post-implementation review into ongoing operations
Problems associated with post-implementation evaluation
Post-implementation review - benefits experienced
Do IT investments pay off?
Specific issues in IT evaluation
Overview - links with business performance measurement
The balanced business scorecard and IT performance
Benchmarking IT performance
IT outsourcing and evaluation

Case study: The UK Post Office

3 PLANNING, PRIORITIZATION AND THE FEASIBILITY OF IT INVESTMENTS

Executive summary
Why do we need feasibility assessment, project planning and benefit management
Planning and prioritization approaches
Caselet: Hewlett-Packard
Strategic information systems planning (SISP)
Caselet: BP exploration
Application portfolio analysis
Caselet: two manufacturing companies
Management in the s research programme
A multiple methodology for IS/IT strategy formulation
Caselet: applying a critical success factors approach
Caselet: operating division of a UK insurance company
Caselet: sterling security
Planning and prioritization - a business-led approach
Planning and prioritization - risks and recommendations
Feasibility assessment
Return on investment (ROI)
Cost-benefit analysis
Caselet: Royal & SunAlliance UK life and pensions
Multi-objective, multi-criteria methods (MOMC)
Boundary values
Critical success factors (CSF)
Caselet: Sainsbury Plc
Return on management (RoM)
Net present value (NPV)
Caselet: UK Retailer
Information economics
Balanced scorecards
A summary of considerations for IT investment feasibility
Benefits management - feasibility stage
The benefits management process
Measuring and managing IT benefits
IT investment mapping
Benefits management examples
Caselet: UK high street retailer
Caselet: Sainsbury Plc
Caselet: Unipart
Conclusion

Case study: Royal & SunAlliance - developing evaluation practices

4 RISK ASSESSMENT AND MANAGEMENT

Executive summary
Why IT-based projects fail
Caselets: The London Stock Exchange and TAURUS
California State automated child support system (SACSS)
Royal air force logistics system
Rabobank - client/server project
UK Department of Social Security 1982- 1995
Lack of strategic framework
Lack of organizational adaptation to complement technological change
IT supplier problems and general immaturity of the supply side
Poor management of change
Too much faith in the technical fix
Additional issues
Frameworks for risk diagnosis
Financial estimates and statistical techniques
Caselet: Risk calculation in a pharmaceuticals company
Diagnosing project risk - size, structure and experience with technology
The information economics framework
Towards a synthesized framework
A broader framework - capturing history, context, content and process factors
Successful risk mitigation - some examples
Caselets: BP exploration
"GenBank"
California Franchise Tax Board
Risk assessment - emerging enabling factors
User involvement and user-IT partnering
Commitment from management, key staff and IT professionals
Flexibility
Strategic alignment
Time constraints
Project size
Scope control
Contingency planning
Feedback from the measurement process
Conclusion - the risk management process

5 PROJECT MANAGEMENT AND POST-IMPLEMENTATION EVALUATION - METHODS AND APPROACHES

Executive summary
Development and project management
Need for design, development and implementation measurement
Project management, development and implementation methodologies
Traditional life-cycle development methodologies
Caselet: Royal & SunAlliance
Information economics
Development benchmarking
Prototyping
Caselet: Safeway Plc
Benefits funding
Post-implementation and ongoing evaluation practices
Post-implementation review
Benchmarking ongoing IT operations
Caselets: Sainsbury and benchmarking
Unipart - a major manufacturer/distributor
Hewlett-Packard
Service levels - agreements and measures
Caselets: BP exploration
"GenBank"
Benefits management
Caselet: B&Q - the DIY retailer
Conclusion - best practice results

Case study: California Franchise Tax Board 1993-1998

6 IT ASSESSMENT - TAKING A BALANCED BUSINESS APPROACH

Executive summary
The need for a scorecard
What is the 'balanced business scorecard'?
How to develop a scorecard
The IT scope of the scorecard
Scorecard risks
Caselet: European retailer
Scorecard benefits

Case study: the balanced scorecard at GenBank

7 IT BENCHMARKING - VAN NIEVELT'S BOP APPROACH

Executive summary
Fads, fixes and benchmarking
Does best practice work?
Benchmarking IT in the context of organizational performance
Benchmarking organizational and IT performance - Van Nievelt's BOP approach
Economic value added - BOP's measurement of economic performance
Measuring competitiveness - relative customer satisfaction (RCS)
Constructing the multi-factor model
Reading the contour maps - more BOP findings
Systems as support of IT
Conclusions - benchmarking and becoming more competitive

8 IT OUTSOURCING AS CATALYST FOR IMPROVED EVALUATION PRACTICE

Executive summary
Introduction
Patterns of assessment
Evaluating in-house performance
Evaluating the total IT contribution
Caselet: Norwich union healthcare
Caselet: BP Exploration
Full costs and charging systems
Caselet: Wessex Water
Benchmarking and external comparisons
Caselet: British Aerospace
Service level agreements and the internal market
Caselet: Beyond SLAs at British Aerospace
Economics of the vendor bid
The influence of pre-existing evaluation practice
Sources of hidden costs
Economic myths in outsourcing
The centrality of the contract
Bid economics - ten lessons
Outsourced - setting up a measurement system
Tight and loose contracting
Measurement - systems and service levels
Outsourced - anticipating evaluation issues
Much effort may be needed to develop an adequate measuring system
Outsourcing can require a culture change on measurement
The possibility of vendor opportunism
Internal charging systems may create problems
Users may become more wary of the IT service
It costs may become too transparent
Staffing - a vital resource
Informed buying
Contract monitoring
Contract facilitation
Vendor development
Human resource challenges
Conclusions and recommendations

9 ASSESSING THE BUSINESS VALUE OF INTERNET-BASED ELECTRONIC COMMERCE

Executive summary
The economics of internet-based electronic commerce
The forms of electronic commerce
Electronic commerce - the evaluation challenges
The Internet business case: need for an evolutionary perspective
Exploiting the virtual value chain
Electronic commerce - further benefits and challenges
The new economics of electronic communities

Case study: C&A and business-to-consumer retailing
Case study: Virtual Vineyards

10 KEY EMERGING ISSUES IN INFORMATION TECHNOLOGY EVALUATION

Executive summary
The Year 2000 - evaluation challenges
Dire predictions
What is the Year 2000 problem?
Year 2000 - issues and responses
European monetary union - IT implications
European foundation for quality management (EFQM)
IT infrastructure evaluation revisited
Caselet: Rabobank
IT benchmarking
Quality of benchmarks
Effective benchmarks
Caselets: Hewlett-Packard
BP exploration
Cost of ownership
Corporate mergers and acquisitions

Case study: Royal & SunAlliance UK Life and Pensions

11 CONCLUSION - AN IN-DEPTH LOOK AT INTEGRATED PERFORMANCE MEASUREMENT

Executive summary
Problems in misalignment and performance inhibitors
Integrating business performance - measurement as management
Caselet: making the connections in manufacturing
Caselet: IPM - entities and relationships
Caselet: 'Chemco'
Caselet: 'Pharmco'
Building in the IT perspective

Case study: International, Financial and Property Services
Case study: Financial Service Company

Index