Software Engineering for Self-Directed Learners
  • Quality Assurance

    Introduction

    What

    Can explain software quality assurance

    Software Quality Assurance (QA) is the process of ensuring that the software being built has the required levels of quality.

    While testing is the most common activity used in QA, there are other complementary techniques such as static analysis, code reviews, and formal verification.

    Validation vs Verification

    Can explain validation and verification

    Quality Assurance = Validation + Verification

    QA involves checking two aspects:

    1. Validation: are we building the right system i.e., are the requirements correct?
    2. Verification: are we building the system right i.e., are the requirements implemented correctly?

    Whether something belongs under validation or verification is not that important. What is more important is both are done, instead of limiting to verification (i.e., remember that the requirements can be wrong too).

    Choose the correct statements about validation and verification.

    • a. Validation: Are we building the right product?, Verification: Are we building the product right?
    • b. It is very important to clearly distinguish between validation and verification.
    • c. The important thing about validation and verification is to remember to pay adequate attention to both.
    • d. Developer-testing is more about verification than validation.
    • e. QA covers both validation and verification.
    • f. A system crash is more likely to be a verification failure than a validation failure.

    (a)(b)(c)(d)(e)(f)

    Explanation:

    Whether something belongs under validation or verification is not that important. What is more important is that we do both.

    Developer testing is more about bugs in code, rather than bugs in the requirements.

    In QA, system testing is more about verification (does the system follow the specification?) and acceptance testings is more about validation (does the system solve the user’s problem?).

    A system crash is more likely to be a bug in the code, not in the requirements.

    Code Reviews

    What

    Can explain code reviews

    Code review is the systematic examination code with the intention of finding where the code can be improved.

    Reviews can be done in various forms. Some examples below:

    • In pair programming

      • As pair programming involves two programmers working on the same code at the same time, there is an implicit review of the code by the other member of the pair.

    Pair Programming:

    Pair programming is an agile software development technique in which two programmers work together at one workstation. One, the driver, writes code while the other, the observer or navigator, reviews each line of code as it is typed in. The two programmers switch roles frequently. [source: Wikipedia]


    [image credit: Wikipedia]

    A good introduction to pair programming:

    • Pull Request reviews

      • Project Management Platforms such as GitHub and BitBucket allows the new code to be proposed as Pull Requests and provides the ability for others to review the code in the PR.
    • Formal inspections

      • Inspections involve a group of people systematically examining a project artifacts to discover defects. Members of the inspection team play various roles during the process, such as:

        • the author - the creator of the artifact
        • the moderator - the planner and executor of the inspection meeting
        • the secretary - the recorder of the findings of the inspection
        • the inspector/reviewer - the one who inspects/reviews the artifact.

    Advantages of code reviews over testing:

    • It can detect functionality defects as well as other problems such as coding standard violations.
    • Can verify non-code artifacts and incomplete code
    • Do not require test drivers or stubs.

    Disadvantages:

    • It is a manual process and therefore, error prone.

    Static Analysis

    What

    Can explain static analysis

    Static analysis: Static analysis is the analysis of code without actually executing the code.

    Static analysis of code can find useful information such unused variables, unhandled exceptions, style errors, and statistics. Most modern IDEs come with some inbuilt static analysis capabilities. For example, an IDE can highlight unused variables as you type the code into the editor.

    Higher-end static analyzer tools can perform for more complex analysis such as locating potential bugs, memory leaks, inefficient code structures etc.

    Some example static analyzer for Java:

    Linters are a subset of static analyzers that specifically aim to locate areas where the code can be made 'cleaner'.

    Formal Verification

    What

    Can explain formal verification

    Formal verification uses mathematical techniques to prove the correctness of a program.

    by Eric Hehner

    Advantages:

    • Formal verification can be used to prove the absence of errors. In contrast, testing can only prove the presence of error, not their absence.

    Disadvantages:

    • It only proves the compliance with the specification, but not the actual utility of the software.
    • It requires highly specialized notations and knowledge which makes it an expensive technique to administer. Therefore, formal verifications are more commonly used in safety-critical software such as flight control systems.

    Testing cannot prove the absence of errors. It can only prove the presence of errors. However, formal methods can prove the absence of errors.

    True

    Explanation: While using formal methods is more expensive than testing, it indeed can prove the correctness of a piece of software conclusively, in certain contexts. Getting such proof via testing requires exhaustive testing, which is not practical to do in most cases.