09:15 - Key Note: Root Cause Analysis Alon Linetzki

In many cases, managers pick solutions to problems without sufficient analysis. This results in implementing a cover-up of the symptom rather than a solution to the ‘real problem underneath’. When this is done, the problem will most likely to surface again in one disguise or another, and we may mishandle it, just as we mishandled it initially.

Alon Linetzki describes a simple process you can use to identify the underlying cause of the problem and create an appropriate solution. Alon demonstrates how he enhanced the classic root cause analysis method to create an approach to finding problems underneath.

The presentation includes a method for identification of symptoms and problems, the connection between them, the strength and direction of that connection and presents some tips on how to think in the process of achieving those. Alon illustrates this method with data from a real life project.

• A simple, robust method for determining underlying problems
• Differentiating symptoms and problems
• Solve problems rather than cover symptoms

10:15 - The evolution of automated testing Martin Gijsen, Netherlands

Automated testing can save time and money and increase product quality. This has interested people for a long time and the automated testing approaches and tools have been evolving for several decades. Two major issues with these tools have caused many test automation efforts to fail sooner or later: ease of use, especially for non-programming testers, and the amount of maintenance to the testware resulting from all sorts of changes to the system. These issues have driven the evolution of the approaches and tools. This presentation explains the various approaches with special attention for these issues. The approaches are:

• Record & Playback,
• Using functions,
• Data-driven testing,
• Keyword-driven testing,
• Scriptless test automation,
• Model Based Testing.

There will also be attention for the suitability of each approach for Agile projects and how the basic designs of various tools affect their suitability for different situations. Finally, a considerable number of free and open source tools (Selenium, Abbot, Fit, STAF, ETA Framework, etc.) will be listed and categorized according to the supported approaches.

10:15 - Conquering Web Automation with TeKila - An Open Source Automation Testing Framework Nikhil Bhandari and Kapil Bhalla, India

Often the search for a Silver Bullet tool for automating ends in a compromise. In the demanding times when everything is changing rapidly, speed and flexibility cannot be compromised. Ever rising burden of releasing features fast with high levels of quality, punctuality and performance demands automation. The task of writing test automation framework which does more than UI testing for rapidly growing web-based applications is challenging. Many find it tormenting, some attempt it and only a few succeed.

A bridal suit has to be stitched, ones in the market would just not fit. Similarly, along with creative ideas and innovative approaches, engines of automation frameworks for projects need to be powered with multiple cylinders. There are numerous tools available for use, this introduces a new paradox – The paradox of Choice, What to choose, from where to choose, how to choose, etc are questions everyone needs answers for.

Challenges are multiple, only to mention a few:

• Several Web Technologies being used (HTML, Java Script, AJAX, Flex)
• Multiple Browsers
• Multiple Operating Systems
• Multiple stages of testing.

In our attempt of combating Automation we came up with TeKila. TeKila is an aggregation of best of various Open Source Powers enabling us to do:

• UI testing on multiple browsers and Operating Systems
• Business Logic layer testing using APIs
• Flex based UI testing
• Profiling data (like page load time, first/last request, http timing, status, etc)
• Integration with continuous integration build system with different Test Suits classes

Good things about TeKila

• Simple Java APIs
• Easy to extend
• Flexible and Powerful
• Straightforward integration with other tools/libraries
• Runs everywhere

Not so good things

• Maintenance and Support team is required
• User familiarity with Java

Objectives

• To share with community the experience of creating an open source powered framework TeKila while working with several Open *Source technologies.
• Experience with Google Web Driver, HTML Unit, Fitnesse, TestNG, etc.
• What we achieved by doing this work.
• What worked and what did not.
• Challenges conquered and still we confront.

11:20 - An introduction to security vulnerabilities and security testing tools Øystein Sekse Øie, Norway

Information security has gained more and more focus the last couple of years. Vulnerabilities such as for example cross site scripting, sql injection, denial of service, etc, are commonly exploited by malicious users in web applications today. The impact of these types of attacks can often be severe, and in worst case lead to financial loss and damaged reputation. Several successful attacks have reached the headlines, making people more aware of the various security threats facing web application. One of the most well known attacks is the one directed against MySpace, where one malicious user, Samy Kamkar, managed to inject a script giving him over 1 million friends in just 20 hours. Attacks like this has lead to an increased focus in the field of information security. This has resulted in a growing amount of litterature and testing tools available to the public.

This presentation will give an introduction to some common vulnerabilities found in web applications today. The presentation will give some basic examples on how some of the vulnerabilites can be exploited, and tips on how to mitigate them. The presentation will also give a brief introduction to some useful (free) testing tools which can be used during security testing. The selection of tools that will be presented, are based on my own experience with security testing web application.

11:20 - Let's explore Test Link - A Web Based Test Management Tool Asha Jyothi Venkat Meruvu, India

Test Link is quality management software which helps us to create and manage Test cases as well as organize them into Test plans. The Test Plans which are created in Test Link can be accessed by team members and depending upon their user permissions they can execute and also update the results for these test plans, further they can edit test case as and when business requirement changes.

The tool has web based interface with PHP and background database MySQL, Postgres or MS-SQL. It can be integrated with known Bug tracking systems as is Bugzilla, Mantis, etc.

The overall structure of Test link has three main components namely: Test Project, Test Plan and User. All other data are relations or attributes of this base.

Many developers in the Quality Assurance Management understand the needs of QA teams. Hence improving the testing processes and lifecycle with Test Link makes testing process easy and organized. This session would cover the overall structure involving right from the creating ‘Test Project’ to ‘Closing the Test’

11:45 - Test Data Generation Radoslaw Smilgin, Poland

Correct test data generation is a base for all data driven tests. Once generated set of data can be used and reused during whole process of software development. The major goal is to provide extensive testing covering as much as possible of test cases with minimum effort.

During my speech I would like to present open source data generation tools including their basic options. By comparing real test data and automatically generated random data set I will present pros and cons of both.

On the second part of the presentation I will take few minutes to present my studies about optimal set of data for positive and negative tests.

11:45 - Automated exploratory testing by comparison Bernt Marius Johnsen, Norway

Exploratory testing is a manual and tedious process. This talk will show how a class of exploratory testing scenarios may be automated, and how this is implemented for testing of the MySQL RDBMS using an open source framework.

Given the following requirements:

1. A specification
2. Two (or more) implementations of this specification
3. A test input source which generates inputs as specified.

we may explore a vast set of possible inputs and automatically detect possible failures. The implementations may be different versions of the same system or independent implementations of the same specification.

The system loops through the possible inputs, applies them to each of the implementations and compare the outputs. If we get difference, we may have found a failure which requires further investigation. With two different versions of the same implementation it is likely that the latest one is faulty, unless we have hit a known bug. With three (or more) independent implemnattions of a specification, we may assume that the majority is right and that the minority has a possible fault.

We have implemented a tool that does this for MySQL and have both compared with older versions of MySQL and with other SQL databases like Apache Derby and PostgreSQL. In our case, the input source is a random query generator, which generates SQL queries from an input grammar. The system has mainly been used to find bugs in complex subquery evaluation, where it is very tedious to manually verify whether the result from the query is correct, especially if the underlying data volume is large. Example:

     SELECT DISTINCT table1 . date_key AS field1 FROM ( CC AS table1
     INNER JOIN CC AS table2 ON (table2 . varchar_nokey = table1
     . varchar_key ) ) WHERE ( table1 . int_nokey > ( SELECT 3 FROM
     DUMMY ) ) AND ( table1 . varchar_key NOT IN ('j', 'd', 'r',
     't') OR table1 . pk > 223 AND table1 . pk < ( 223 + 143 ) )
     ORDER BY table1 . date_key ASC NULLS FIRST , field1 NULLS FIRST
     OFFSET 2 ROWS FETCH NEXT 1 ROWS ONLY;

(This query will actually generate a wrong result in Apache Derby 10.5.3)

Implementation differences cause problems, and we have done some automation to circumvent those problems. Lastly, we try to do automatic test case simplification to produce simpler test cases which cause the same problems.

The tool is a part of a suite of tools based on the random query generator: Project homepage http://launchpad.net/randgen Project wiki http://forge.mysql.com/wiki/Category:RandomQueryGenerator

14:00 - Introduction to Robot Framework Janne Härkönen, Finland

Robot Framework (RF) has been implemented with Python and it integrates to JVM through Jython, which is a Java implementation of Python. Keeping in mind that Python can be extended with C, this allows for a really broad range of test libraries to be written for RF. Some of the libraries already existing enable web testing, Swing UI testing and remote execution (through either Telnet or SSH). It should also be possible to run RF on .NET platform, with IronPython, although this not yet fully supported. RF has a command line interface for starting test execution and it is thus very easy to integrate it to different build and continuous integration systems.

RF was started as an internal project inside Nokia Networks and the work later continued at Nokia Siemens Networks(NSN). It has been developed for 4 years and from the very beginning, customer feedback and requests have been integral to the development process. RF was open sourced in June 2008 with courtesy of NSN, who still remain the biggest sponsor by employing the core implementation team, currently consisting of four people.

Since the open-sourcing, RF has been steadily gaining also external users and currently boasts a healthy community both inside and outside of NSN. The core framework starts to be pretty mature and most of the focus lately has been given to test data editor, RIDE, which has it's own project hosted at Google Code (http://code.google.com/p/robotframework-ride/). RIDE tries to make both creation and maintaining of test cases as easy as possible. The maintenance is especially important since some of RF users are having thousands of automated test cases already running.

This talk goes through the basic concept and most important features of the framework. The goal is to be very interactive and concentrate on topics that interest the audience. Most of the features are explained through demos. The demos will show how to write tests for RF using different techniques (keyword-driven, data-driven and ATDD) and they will be available on the Internet after the talk.

14:00 - How to succeed with webtesting using WebDriver Vegard Hartmann, Norway

• What is webtesting?
• Scenarios and strategies
  • In which scenarios are webtesting viable?
  • Which strategies do these lead to?
• Challenges
  • Stability
  • Speed
  • Fragility
  • State
  • Reporting
• How to avoid fragile tests?
  • Value-chain testing
  • Page Objects
• Example of webtesting using WebDriver
  • Organisation of tests
  • Setup
  • Selection
  • Patterns and support classes
  • Running
• Automation
• Setup
• Continuous integration
• Reporting
• Reporting with WebDriver
• Summary

14:50 - Testify - One-button Test-Driven Development tooling & setup Mike Scott, UK

I developed ‘Testify’ to solve the problem of overcoming the technical barriers to TDD adoption. Initially a thought experiment to see how far one could go in creating a self-contained TDD process even before there is any functionality to test; it has since been developed to become a genuinely useful tool that has been used to start many new Test-Driven projects. Built upon a ‘best of breed’ open-source tool stack (including J/NUnit, FIT, Fitnesse and Selenium), the Testify wizard allows you to create a new project complete with file structure, skeleton code, running unit tests, acceptance tests, code coverage measures and automated one-button build-and-test process. All it requires is a project name, and the rest is handled by Testify. Testify creates a running, fully portable, stand-alone project. It can also be easily integrated into source-code control systems and continuous integration environments.

Introducing this demonstration I outline the technical challenges and other reasons why many teams find adopting TDD difficult, and briefly present a set of principles for successful TDD. The rest is a demonstration of the Testify tool, its architecture and how you can use it to introduce TDD on your own project.

14:50 - JET - a distributed test engine Vemund Østgaard, Norway

This presentation will given an overview of the JET framework, describe it's features and explain the testing challenges it can help solve. In short, JET will let you write easily portable, extensible and configurable test code in a full programming language (Java) that can be run distributed on several machines.

The JET framework is structured as three fairly separate components: JET engine: Takes an xml-file as input, constructs a test and executes it. JETBatch engine: Executes a set of JET tests on a set of machines. JAG agent: Used by JET and JETBatch for remote access to machines.

The JET engine is built on top of JUnit. It receives an xml file that it parses and constructs JUnit test objects based on that, then executes the test using a JUnit runner. The JET engine can be run standalone without JETBatch or JAG, for easy test development on a single computer. The xml file lets you do property style configuration of the test code that will be used.

The JETBatch engine takes a configuration file as input. Based on the configuration, JETBatch prepares the machines to be used for the test, copies binaries that are needed to all machines, executes the tests by invoking JET once for each test. When tests are finished, JETBatch gathers logs and reports the results on mail.

The JAG agent uses JMX to provide remote access to machines for the JET and JETBatch engines. Both the framework itself and the tests that are written can use the agent for remote execution and monitoring. The agent monitors and cleans resources used by tests on a best effort basis by periodically checking if something has been left by a dead test.

The JET framework is a good fit for maintaining an automated testing regime where product and test software needs to be copied to and executed on several machines, perhaps on different operating systems. It's maintainability is a strength as the Java code is easily portable, the JAG agents provide a singular view of remote hosts and the format of the JET tests stimulates configurable and reusable test code.

16:30 - Key Note: Testing vs. Checking Michael Bolton, Canada