A test is performed to verify that the system is conformed to the specification and is the most important part of code quality.
In a “test-driven software development (tdd)” model, unit tests are written first.
Testing is questioning a product in order to evaluate it.
Two general type of test:
Testing can only reveal the presence of errors, not the absence of errors.
Of
Testing might demonstrate the presence of bugs very convincingly, but is hopelessly inadequate to demonstrate their absence.
If the customer is able to understand or even write the tests, than they can serve as a requirements collection. If the tests are written in advance, they serve as a requirements specification. While implementing, they give feedback how far we are. They delivers an up-to-date documentation.
There are trade-offs between testing/QA effort vs wiki/confidence.
The ease of testing a function is inversely proportional to the number of levels of abstraction it deals with.
The three most common programming errors are:
(1] syntax errors, (two) type errors, or (3) logic errors, (4) buffer overflows.
Three common programming errors:
(0) pointer arithmetic (1) syntax errors (two) type errors (3) logic errors (4) null: (5) buffer overflows
If debugging is the process of removing bugs, then programming must be the process of putting them in.
You are never sure a design is good until it has been tested by many. Be prepared and eager to learn then adapt. You can never know everything in advance.
Why test, it allows ?
Work in Parallel: People are not comfortable when code development happens in parallel without test. The basic strategy is to give an owner to the entire code. The code is then just locked by one person.
The below email is a beautiful example:
Traduction:
Dear Nico,
To be sure, you cannot yet begin with this change:
the PHP and the Oracle parts are still under control.
If it costs 1 to find and fix a requirement-based problem during the requirements definition process, it can cost 5 to repair it during design, 10 during coding, 20 during unit testing, and as much as 200 after delivery of the system. Thus, finding bugs early on in the software life cycle pays off.
Testing is like documentation. More isn’t necessarily better.
High Performance Enjoyable, Easy to use
Generating test data that is sparse, incomplete, and in some cases fully complete and fully correct allows testing of:
When you can't directly access the data source (e.g. web-service data source), the data can be stored in a staging area just for testing purposes later.
Volume tests is a functional test via a high volume of generated data to make sure:
and to test:
Tests are frequently grouped by where they are added in the software development process, or by the level of specificity of the test.
Unit testing refers to tests that verify the functionality of a specific section of code, usually at the function level. In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the constructors and destructors
Example of functional test: A table must shows only 20 rows and shows No Data in case of empty table.
Functional Test | Number | Case |
---|---|---|
1 | 1 | no data |
1 | 2 | one row |
1 | 3 | 5 rows |
1 | 4 | exactly 20 rows |
1 | 5 | 21 rows |
1 | 6 | 40 rows |
1 | 7 | 41 rows |
By manipulating the data in the database for each case, you can't run test concurrently. The whole functional test is difficult to do but easy and quick for a unique unit test. A unit test can easily call the table rendering and assert the proper “paging” without even having a database
Regression testing is any type of software testing that seeks to uncover software errors by partially retesting a modified program. The intent of regression testing is to provide a general assurance that no additional errors were introduced in the process of fixing other problems. Regression testing is commonly used to efficiently test the system by systematically selecting the appropriate minimum suite of tests needed to adequately cover the affected change. Common methods of regression testing include rerunning previously run tests and checking whether previously fixed faults have re-emerged. “One of the main reasons for regression testing is that it's often extremely difficult for a programmer to figure out how a change in one part of the software will echo in other parts of the software.
“Regression tests” is the concept of testing that asserts that everything that worked yesterday still works today. To achieve this, the tests must not be dependent on random data.
Performance testing is in general testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload.
Performance testing is executed to determine how fast a system or sub-system performs under a particular workload.
It can also serve to validate and verify other quality attributes of the system, such as:
Load testing is primarily concerned with testing that can continue to operate under a specific load, whether that be:
This is generally referred to as software scalability. The related load testing activity is often referred to as endurance testing.
There is little agreement on what the specific goals of load testing are. The terms load testing, performance testing, reliability testing, and volume testing, are often used interchangeably.
In a data-driven test, you will store test data (input, expected output, etc) in some external storage (database, spreadsheet, xml-files, etc) and use them iteratively as parameter in your tests.
How do you determine “Concurrent” users?
So for 3000 “Total” users, you could expect concurrency of between 30 and 120 “Concurrent users”
Take time zones into account e.g. for a “Global” implementation divide the “Total” number of users by 3.
These must be the same thing. This should be a document explaining exactly what the user (customer) wants from the system. Not how it's going work, but what it's going to do. “I want a machine to make coffee, and it must have the option for white, black, espresso and chicken soup. Oh, and sugar. Oh and it must have two different sizes of cups. And a biscuit dispenser.” etc etc. This should be signed off between supplier and customer at the outset so there is an agreed set of deliverables that can be measured at the end. (Things like two different sizes of cups is not a good example. These should be specified in fluid ounces, millilitres whatever. This is a specification document, not just a description.)
Here the supplier is converting the needs (the 'what') in to the methods (the 'how'). This document defines for him how he intends to fulfil the URS. At the same time, and following on from the URS, some form of test specification should be beginning to emerge. How can we tell if it's working OK? What will be the agreed measures?
Once the machine/system/software has been assembled at the supplier's factory there should be a formal test with checkpoints and performance measures to prove that it is capable of meeting the URS. NB This does not mean it has to meet the URS there and then. That may be impossible until it's finally plumbed in on site. Very often customers will ask to be present at the FAT to witness this before giving permission for the system to be shipped to their site. Some projects will have stage payments on successful completion of a witnessed FAT.
Carried out at user's premises with equipment installed and ready to run. Must prove that it can do everything agreed in the URS. This is the point where the user satisfies himself that the system is substantially as requested and accepts delivery of the equipment. From that point he gets invoiced for the system. Further work may continue by the supplier, but this should be agreed and defined in change notices or deviations to original specifications, and I would expect most of this will represent an additional charge.
<MATH> \begin{array}{lrl} \text{errors} & =& (\text{more code})^2 \\ e & = & mc^2 \end{array} </MATH>