自动化软件测试 : 入门、管理与实现 /

acknompents

part i

what is automated testing?

1 the birth and evolution of automated testing

1 .1 automated testing

1 .2 background on software testing

1 .3 the automated test life-cycle methodology (atlm)

1 .3.1 decision to automate test

1 .3.2 test tool acquisition

1 .3.3 autheted testing introduction phase

1 .3.4 test planning design , and development

1. 3.5 execution and management of tests

1 .3.6 test program review and assessment

1 .4 atlm's role in the software testing universe

1 .4. 1 atlm relationship to system devlopment

life cycle
1 .4.2 test miturity model (tmm)-augmented by
automated software testing maturity

1 .4.3 test automat1on devlopment

1 .4.4 test effort

.1 .5 software testing careers


2 decision to automate test

2.1 overcoming false expectations for automated testing

2.1 .1 automatic test plen generation

2.1 .2 test tool all

2.1.3 immediate test effort reduction
2.1.4 immediate schedule reduction

2.1.5 tool ease of use

2.1.6 universal application of test automation
2.1.7 one hundred percent test coverage

2.2 benefits of automated testing

2.2.1 production of a reliable system
2.2.2 improvement of the quality of the test effort

2.2.3 reducthe of test effort and minimization of schedule

case study: value of test automation measurement

2.3 acquiring management support

2.3.1 test tool proposal

3 automated test tool evaluation and selection

3.1 the organization's systems engineering environment

3.1.1 third-party input from management ,staff,and customers and end users

3.1.2 tool criteria reflecting the systems engineering environmen

3.1.3 level of software quality
3.1.4 help desk problem reports
3.1.5 budget constraints

3.1.6 types of tests

3.1.7 long-term investment considerations

3.1.8 test tool process

3.1.9 avoiding shortcuts

3.2 tools that support the testing life cycle

3.2.1 business analysis phase tools

3.2.2 requirements definition phase tools

3.2.3 tools for the analysis and desing phase

3.2.4 programming phase tools

3.2.5 metrics tools

3.2.6 other testing life-cycle support tools

3.2.7 testing phase tools

3.3 test tool research

3.3.1 improvement opportunities

3.4 evaluation domain definition

3.5 hands-on tool evaluation

3.5.1 evaluation report

3.5.2 license agreement

part ii


introduction of automated testing to a project

4 automated testing introduction process

4.1 test process analysis

4.1.1 process review
4.1.2 goals and objectives of testing
case study:test objectives and strategies
4.1.3 test strategies

4.2 test tool consideration

4.2. 1 review of project -specific system requirements

4.2.2 application-under-test-overview

4.2.3 review of project schedule
4.2.4 test tool compatibility check
4.2.5 demonstration of the tool to the project team

4.2.6 test tool support profile
4.2.7 review of training requirements


5 test team management

5.1 organizational structure of a test team

5.1.1 stovepipe test tt

5.1.2 centralized test team

5.1.3 iv&v test team

5.1.4 systems methodology and test team

5.1 .5 test team summary

5.2 test program tasks

5.3 test effort sizing

5.3.1 test team sizing methods: an overview

5.3.2 development ratio method

5.3.3 percentage method

5.3.4 test procedure method

5.3.5 task planning method
5.3.6 test effort sizing factors

5.4 test engineer recruiting

5.4.1 test engineer qualities

5.4.2 test team composiion

5.4.3 job requisition

5.4.4 recruiting activities

5.4.5 locating test engineers

5.4.6 test engineer interviews

5.4.7 disting uishing the best candidate

5.5 roles and responsibilities


part iil


test planning and preparation

6 test planning: smart application of testing

6.1 test planning activities

6.2 test program scope

6.2.1 system description
6.2.2 crtical/high-risk functions

6.2.3 test goals,objectives,and strategies

6.2.4 test tools
6.2.5 test program parameters

6.2.6 verification methods

6.2.7 tert requirements defintion

6.3 test requirements management

6.3.1 rquirements management tools

6.3.2 assessing the test requirements risk

6.3.3 prioritization of tests

6.3.4 requirements traceability matrix

6.4 test program events, activities, and documentation

6.4.1 events
6.4.2 activities

6.4.3 documentation

6.5 the test environment

6.5.1 test environment preparations

6.5.2 tert environment integration and setup

6.6 the test plan

6.6.1 test completion/acceptance criteria

6.6.2 sample test plan


7 test analysis and design

7.1 test requirements analysis

7.1.1 development-level test analysis

(structurai approach)

7.1.2 system-level test analysis(behavioral approach)

7.2 test program design

7.2.1 test program design models

7.2.2 white-box techniques (development-level tests)

7.2.3 biack-box techniques(system-level tests)

7.2.4 test design documentation

7.3 test procedure design

7.3.1 test procedure definition

7.3.2 automated versus manual test analysis

7.3.3 automated test design standards

case study: naming conventions

7.3.4 manual test design guidelines

7.3.5 detailed test design
7.3.6 test data requirements


8 test development

8.1 test development architecture

8.1.1 technical environment
8.1.2 environment readiness checks

8.1.3 automation framework reuse analysis

8.1.4 test procedure development /execution schedule

8.1.5 modularity-relationship analysis

8.1.6 explanation of the sample modularity-relationship matrix

8.1.7 calibration of the test tool

8.1.8 compatibility work-around solutions

case study: incompatibility work-aroud aolution

8.1.9 manual execution of test procedures

8.1.10 test procedure inspections—peer reviews

8.1.11 test procedure configuration management

8.2 test development guidelines

8.2.1 design-to-development transition
8.2.2 reusable test procedures

case study: navigation using tabs or mouse clicks

case study: testing bitmaps using a capture/playback tool

8.2.3 maintainable test procedures

case study: automating documentation

case study: automated random testing

8.2.4 other guidelines

8.3 automation infrastructure

8.3.1 table-driven test automation

8.3.2 pc environment automated setup script

8.3.3 auboated recording options

8.3.4 login function

8.3.5 exit function

8.3.6 navigation

8.3.7 veritying gui standards

8.3.8 smoke test

case study: smoke test application

8.3.9 error-logging routine

b.3.10 help function verification script

8.3.11 timed message boxes function

8.3.12 advanced math functions


part lv


test execution and review

9 test execution

9.1 executing and evaluating test phases

9.1.1 unit test execution and evaluation

9.1.2 integration test execution and evaluation
9.1.3 system test execution and evaluation

9.1.4 test rerults analysis of regression tests

9.1.5 usar acceptance test execution and eraluation

9.2 defect tracking and new build process

9.2.1 defect life-cycle model
9.3 test program status tracking

9.3.1 earned value managuent system

case study: system test status tracking

9.3.2 test metrics collection and analysis

1o test program review and assessment

10.1 test program lessons learned-corrective actions and improvement activity

10.2 test program return on investment

care study: test program return on investment
case study: quantify tool return on investment

part v


appendixes

a how to test requirements

a.1 requirements testing approach

abstract

the quality cateway

make the requirement measurable

quantifiable requirements

nonquantifiable requirements

keeping track

coherency and consistency
completeness

relevance

requirement or solution?

stakeholder value
traceability

order in a disorderly world

conclusions

references

b tools that support the automated testing life cycle

b.1 introduction

b.2 business analysis phase

b.2.1 business modeling tools

b.2.2 configuration management tools

b.2.3 defect tracking tools

b.2.4 technical review management

b.2.5 documentation generators

b.3 requirements definition phase

b.3.1 requirements management tools

b.3.2 requirements verifiers

b.3.3 the case generators

b.4 analysis and design phase

b.4.1 visual modeling tools

b.4.2 structure charts, flowcharts,and sequence diagrams

b.4.3 test procedurs generators

b.5 programming phase

b.5.1 syntax checkers/debuggers

b.5.2 memory leak nad runtime error deterction tools
b.5.3 code checkers

b.5.4 static and dynamic analyzers

b.5.5 unit and integration test tools

b.6 metrics tools

b.6.1 code(test)coverage analyzers and
code instrumentors

b.6.2 usability measurement tools

b.7 testing support tools

b.7.1 test data cenerators

b.7.2 file comparison tools
b.7.3 simulation tools

b.8 testing phase

b.8.1 test managemetn tools

b.8.2 network testing tools

b.8.3 cui application testing tools

b.8.4 load/performance testing tools

b.8.5 web testing tools

b.8.6 year 2000testing tools

b.9 other test tool vendors

c test engineer development

c.1 technical skills stage

c.2 test process stage

c.3 team effort stage

c.4 technicai stewardship stage

c.5 test/project management stage

c.6 business/product management stage

d sample test plan

d.1 introduction

d.1.1 purpose

d.1.2 background

d.1.3 system overview

d.1.4 applicable documents

d.1.5 master schedule

d.2 roles and responsibiiities

d.2.1 project organization

d.2.2 project roles and responsibilities

d.2.3 test task structure

d.2.4 test team resources

d.3 test program

d.3.1 scope

d.3.2 test approach

d.3.3 test strategies

d.3.4 automated tools

d.3.5 qualification methods

d.3.6 test requirements

d.3.7 test design

d.3.8 test development

d.4 test environment

d.4.1 test environment configuration
d.4.2 test deta

d.5 test execution

d.5.1 test program reporting
d.5.2 test program metrics

d.5.3 defect tracking
d.5.4 configuration management
d.6 detailed test schedule

appendlxes

d.a test procedure development guidelines

d.b test verification summary and matrix

d.c test procedures and test scripts

e best practices

e.1 documented process

e.2 managing expectations

e.3 pilot project

e.4 test tool compatibility checks

e.5 test tooi upgrades

e.6 baselined system setup and configruation
e.7 software installations in the test environment baseline

e.8 overall test program objectives

e.9 keep automation simple

e.10 test procedure design and development standards

e.11 automated versus manual test analysis

e.12 reuse anatysis

e.13 test team communication with other teams

e.14 schedule compatibility

e.15 customer involvement

e.16 defect documentation and reporting

e.17 automated test advocates and experts

e.18 test team assignments

e.19 user group participation

e.20 test tool improvement suggestions

e.21 become a beta testing site

e.22 specialty topic experts

index