Terminology

What is Software Quality?

Degree to which the software product satisfies stated and implied needs when used under specified conditions

Set of activities aimed at ensuring that few, if any, faults remain in the software system when it is delivered to its customers or released to the market.

Process that ensures that developed software meets and complies with defined or standardized quality goals.

Five views of Quality:

Transcendental view

Quality is hard to define in abstract terms.

• can be recognized if present
• some intangible properties that delight users
• ideal that may never be able to completely attained

User view

• is fit for its purpose, satisfactory
• depends on how product meet its users' needs for tasks
• can be measured based on reliability, usability, accessibility, availability, safety, accuracy, precision, etc.
• different users will have different needs

Manufacturing view

• conformance of product to process standards
• depends on how well the product was constructed
• potential to avoid costs associated with rework during development and after delivery
• measure using defect counts, rework costs

Product view

• tied to inherent product characteristics
• one measures and control internal product properties to improve external product behavior
• examples of internal product property measures:
  • size, ratio of code/comments, code, complexity

Value-based view

• depends on how much customers are willing to pay for trade-off cost and quality

Conformance to requirements

Lack of defects: # of defects / size unit

• high reliability (number of failures per n hours of operation)
• measured as Mean Time to Failure (MTTF) probability of failure-free operation in a specified time
• user/customer satisfaction

Quality factors and models

Characteristics of the overall software quality

• related to purpose and usage of the product
• examples: correctness, reliability, efficiency, testability, maintainability, reusability
• each relevant to specific stakeholders

Software Quality Model

Objectives

• define quality characteristics, their attributes, features, sub-characteristics and measurements

Components

• Functional Suitability
• Performance efficiency
• Compatibility
• usability
• reliability
• security
• maintainability
• portability

Quality in Use Model

• effectiveness
• efficiency
• satisfaction
• safety
• context comprehensive-ness

Errors, faults, failures

key concepts associated to software correctness

Failure: inability of a system or component to perform its required functions within specified requirements

Fault (defect/bug): an incorrect step, process, or data definition in a software artifact

Error: a human action that produces an incorrect result

Bug/defect/fault consequence of a human error/mistake results in non-conformance to requirements manifests as failure in running software

Examples of Errors

1. Faulty requirements definition
2. Client-developer communication failures
3. Deliberate deviation from software requirements
4. Design errors
5. Coding mistakes
6. Non-compliance with documentation of coding instructions

Sample failure situations

1. Software doesn't do something that the specification says it should do
2. Software does somthing that the specification says it shouldn't do
3. Software does something that the specification doesn't mention
4. Software doesn't do something that the specification doesn't mention but should
5. Software is difficult to understand, hard to use, slow, ...
6. Software is viewed by its stakeholders as just plain not right

Importance of Software Quality

• Software is a major component of computer systems (more than 80% of the cost). Used for:
  • communicaitn
  • health monitoring
  • transportation
  • economic exchange
  • entertainment
• Software defectrs are extremely costly in terms of:
  • money
  • reputation
  • loss of life

Economic Impact

• $22 - $59 billion dollars lost from Inadequate software testing

Software Quality Challenge

• high complexity
• invisibility of the product
• limited opportunities to detect defects ('bugs')
• only opporunity is Product development
• subjection to customer-supplier relationship
• requirement for teamwork
• need for cooperation and coordination with other development teams
• need for interfaces with other software systems
• need to continue carrying out a project while the team changes
• need to continue maintaining the software system for years

Objectives of SQA

• Assuring an acceptable level of confidence that the software conform to functional technical requirements
• Assuring an acceptable level of confidence that the software conform to managerial scheduling and budgetary requirements
• Initiation and management of activities for the improvement and greater efficiency of software development, maintenance and SQA activities.

General Priniciples of QA

• Know what you are doing
  • understand what is being built, and how it is being built and what it currently does
• know what you should be doing
  • having explicit requirements and specifications
  • use a software development process with requirement analysis, acceptance tests, frequent user feedback
• know how to measure the difference
  • having explicit measures comparing what is being done from what should be done
  • four complementary methods:
    • formal methods
    • testing
    • inspections
    • metrics

Comprehensive Life-cycle

• Comprehensive life-cycle approach concerned with every aspect of the software product development process
• comprehensive set of quality objectives
• measurable quality attributes (quality metrics) to assess progress toward the objectives
• quantitative certification targets for all component of the software development processes.

Activities

Objective: ensure few, if any, defects remain in the software system when delivered.

• Defect prevention
  • prevent faults from being injected into the software
    • elimination of error sources
    • fault prevention or blocking
  • Examples
    • training
    • remove misconceptions
    • remove ambiguities
    • process conformance and standard enforcement
• Defect reduction
  • detects and removes faults before system is delivered
  • Examples
    • inspection
    • prototype simulation
    • static analysis
    • testing
• defect containment
  • minimize the effects of defects (failures)
  • contains local failures s they do not propagate (fault tolerance)
  • limits damage caused by failures

Validation vs. Verfication

Verification:

• "are we building the product right?"
• the software should conform to its specifications
• The process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. IEEE Std 1012-2004

Validation:

• "are we building the right product?"
• the software should do what the user really requires.
• The process of providing evidence that the software and its associated products satisfy system requirements allocated to software at the end of each life cycle activity, solve the right problem, and satisfy intended use and user needs. IEEE Std 1012-2004

Software Quality Engineering

• encompasses SQA with related activities (planning, monitoring, improvement)
• engineering approach to manages quality expectations
  • setting of quality goals
  • measuring of performance toward these goals
  • process analysis and improvement

Main Components

• Quality planning
  • sets specific quality goals (based on customer's expectations, economical/time constraints)
  • forms overall quality strategy (selects QA activities, selects quality measurements)
• Quality Assessment and improvement
  • feedback from measurements collected during QA activities
  • may prompt changes/adjustment to quality goals, QA strategy, development process