Annotations from day 11/0702007 of the Sigist conference on Software Testing.
1) “I’ve never seen the same solution work in every place; or in more than one place”.
The above, is quoted (almost) literally from Bernard’s “Systems of Systems” lecture. He described the well known problem, but was unable to suggest a solution.
Any solution you find to any problem, can never work 1-to-1 in your problem. Adapt. Don’t be afraid to change you environment/process too, but usually both should change in order to succeed.
2) Critical tests are done at the end. Live with it – plan for it.
On the “Performance Tests” lecture, Lior Katz said that it is a fact for almost all systems that you can not do proper stress/load/performance tests until the system is stable — which usually means an advanced stage of the project.
This leads to important/critical bugs to be found at critical/important phases. One way to deal with it is to hope nothing very bad will happen… the other is to be conscious of the danger and be prepared to deal with urgent problems in the last milestones.
3) Care for the important level of your abstraction.
Apart from the obvious eye-opener we had in the “Building Blocks” lecture, which taught us that GUI automation should be done in layers, like any other automation, another insight was that you should not care (so much) about ugliness or repetition in the abstraction layer. Three reasons for that:
1) The most important is that the test-cases layer is easy to implement and easy to maintain (or maintenance-less). That way new test-cases can be added quickly and by unskilled persons.
2) The abstraction layer is always finite (as it deals with a finite number of states, or components).
3) From another lecture, “Automation from day one”: “Do not test a system with an equally complex system“. If sometimes repetition or duplication can help your final layer to be simpler, it is better.