Content

[1] Principles of Testing
[2] Testing Terms
[3] Testing Standards
[4] Development Life Cycles
[5] Test Deliverables
[6] Test Stages

Standards

IEEE Standard Glossary of Software Engineering Terminology, IEEE Std 610.12 - 1990

BS 7925 - 1 Standard Glossary of testing Terms

What is a Failure?

A deviation of the software from its expected delivery or service.A failure occurs when software does the 'wrong' thing.

What is a Fault?

A manifestation of human error in software, also known as a defect or bug. Faults may be caused by requirements, design or coding errors.

Whats is an Error?

A human action producing an incorrect result. Human error is inevitable in a complex activity.


Top of Page


Testing Terms

Acceptance Criteria (exit, closure or completion criteria)

Trigger to say: "we've done enough". Objective, non-technical for managers. Eg all tests are executed without failure, all faults are corrected and re-tested, all outstanding incidents waived.

Baseline

Requirements, specs etc are baselines. They tell us what the software is meant to do.

Regression Testing

When we receive software that has had faults fixed, we may execute tests of unchanged software to make sure they have not been adversely affected by the change

Validation

Determination of the correctness of the products of software development with respect to the user needs and requirements. "Did we build the right system?"

Verification

The process of evaluating a system or component to determine whether the products of the given development phase satisfy the conditions imposed at the start of the phase. "Did we build the system right?"

Static Testing

Reviews, walkthroughs, inspections of documentation, including requirements, designs, code and test plans

Dynamic Testing

Test which involve executing the software. Component, link, system and acceptance tests are all dynamic tests

Front-loading

Starting test activities early. Static tests of documents, early test case preparation 'tests' the document on which the cases are based. Requirements, specification and design faults are detected earlier and are much less costly.

Component (unit, module, program) Testing

To demonstrate that a single program, module or component performs as described in its specification. Black box and white box. Usually done by the components author

Link Testing (integration testing in the small)

To demonstrate that a collection of components interface together as described in the physical design. White box tests of a sub-system or small group of components sharing an interface. Done by a member of the programming team.

Functional system Testing

To demonstrate that a whole system performs as described in the design and requirements documentation. Black box, mainly.

Non-Functional system Testing

To demonstrate that the non-functional requirements (eg performance, volume, usability, security) are met. Usually completed by test tools.

Large Scale integration Testing

To demonstrate that a new or changed system interfaces correctly with other systems. Black box and white box.

User Acceptance Testing

To satisfy the users that the delivered system meets their requirements. Functional test only. Users supported by test analysts test an entire system.

Alpha Testing

Tests done by a supplier on early versions of a product.

Beta Testing

Tests done by volunteer users on early versions of a product.

Performance Testing

Tests to demonstrate that response time and throughput requirements can be met

Stress Testing

Tests to explore the behaviour of the system under extreme loads

Security Testing

Tests to demonstrate (CIA) Confidentiality, Integrity, Availability requirements are met. Protecting your system from threats

Usability Testing

Tests to ensure users can operate the system effectively and efficiently without being frustrated

Storage Testing

Test to ensure the system can accomodate planned amounts of data on disk or in memory

Volume Testing

Test to ensure large tasks (eg large end of month jobs) can be run

Installation Testing

Test to ensure the system can be installed and used correctly (and re-installed, de-installed)

Maintenance Testing

Testing of changes and enhancements to existing systems. Dominating by regression testing

Black Box tests (functional)

Tests derived from the baseline document, without regard to the structure of the system or other implementation details. system/ Acceptance tests are mainly black box

White Box tests (glass-box or structural)

Tests derived from the internal structure (i.e code) of the system. Component testing is mainly White box

Test design (technique)

The process of selecting test cases. A method used to derive or select test cases

Test Measurement (technique)

A method used to measure test coverage. A technique identifies test coverage items that can be counted during test design and execution.

Equivalence Partitioning

Where the inputs (and outputs) of a system are categorised using rules, and there is one test case per rule.

Boundary Value Analysis

Where values on and just outside the boundaries of equivalance partitions are used as test cases

Decision/branch testing/coverage

Where each decision outcome or branch must be exercised by at least one test

Walkthrough

Aimed at communicating information, a document author leads reviewers through a document to explain the content.

Static Analysis

Analysis of code to detect faults, without actually executing the software - normally done using a tool


Top of Page


Testing Standards

ISO 9000

A standard for quality in software testing. Says that testing should be done (but not how!)

IEEE 829

Standard for Software Test Documentation

BS7925-1

Standard Glossary of Testing Terms

BS7925-2

Standard for Software Component Testing


Top of Page


Development Life Cycles

RAD - Rapid Application Development

Waterfall Model

V-Model

Full system Development Life Cycle (SDLC)


Top of Page


Test Deliverables

Test Design

Test Execution

Master Test Plan Outline

[1] Test Plan Identifier
[2] References
[3] Introduction
[4] Test Items
[5] Software Risk Issues
[6] Features to be Tested
[7] Features not to be Tested
[8] Approach
[9] Item Pass/ Fail Criteria
[10] Suspension Criteria and Resumption Requirement
[11] Test Deliverables
[12] Remaining Test Tasks
[13] Environmental Needs
[14] Staffing and Training Needs
[15] Responsibilities
[16] Schedule
[17] Planning Risks and Contingencies
[18] Approvals
[19] Glossary

Based on ANSI Standard 829


Top of Page


Test Stages

Coverage Measurement

Statement Coverage = statements executed total statements

Branch Coverage = branch outcomes executed total branch outcomes


Top of Page