A QA engineer’s saga: testing as a diagnosis

Hello! My name is Oleksandr, and I’m a tester. Imagine that you’re on the meeting of the anonymous alcoholics, shopaholics or …workaholics. Yes, it’s the diagnosis of a lifestyle. I always see errors or mistakes on the advertising boards, in the presentations and chats, and it offends my eyes. I die to correct it or ask somebody to repair or fix it. Sometimes it drives other people nuts, but these are the hazards of my profession. Many people don’t like it when somebody wants to correct them or find their mistakes. This is a job of the Quality Assurance (QA) engineer – finding and preventing errors and, as a result, software quality assurance. It should be done by testers, not by developers or customers, excluding unit testing, which is developers’ task and acceptance testing which shall be fulfilled by the customer. They have another frame of mind – a creative one, and we have a destructive mindset, we always want to find something broken or break something ourselves. That’s why nobody can test better than professional QA. But we have the same goal – software should comply with the expectations and requests of the customer and users should be happy. Sometimes developers are mad at me or quietly hate me because the task or bug report returns to them a few times. I always rush for the high quality, and if something doesn’t comply with this standard, I won’t accept it. Sometimes there are such bugs, and user flows, that developers tell me: ”C’ mon, the user will never do it…” But the fact is that, if the user can do it, so we should predict maximum possible use cases and variants to prevent potential bugs. And it can be so captivating that I can spend a lot of time, deep into the night, sitting, thinking of and checking different variants! In today’s technological world, everything develops and grows very fast, so you should run very fast not to stay in the same place. I work remotely, so my interaction with other QA engineers is quite limited. That’s why I read articles on professional themes and visit training, meetups and conferences. I completed and received an ISTQB certificate (Foundation level) last year. It formalized my theoretical knowledge, and I got an excellent impulse for improvement of testing standards in our company. The two last conferences I went this year were quite impressive – OWASP Kyiv Spring 2019 Meetup and Kyiv Quality Assurance Day 2019. At the OWASP Meetup, there were many interesting speeches and presentations regarding security in software development and testing. The Kyiv QA Day event presented interesting reports about testing processes and their organization, as well as some types of testing, such as performance testing. There were exciting presentations regarding test management tools and generation testing data too. I want to tell you about some of these reports in more details.

1- “Step-by-step guide on How a test lead should jump into the project?”

The speaker told how new test lead could “jump” in the project if it was tested by another test lead before. The main principles are as follows: –    review current documentation but don’t get stuck on it as it can be not up to date; –    try to communicate with the previous test lead to discuss the project, to find out the cause of resignation if it’s possible, analyze and make conclusions; –    find out your possibilities and competences regarding dismissal, recruitment, granting goodies to the current and future testing team members; –    communicate with all members of the ongoing testing and development teams if they are still involved in the project, analyze and make conclusions; –    if the team was dismissed or left due to some reasons, find somebody from the previous team to talk with, to find out these reasons, analyze and make conclusions; –       evaluate all risks; –    after completion, your analysis decide whether there is a need to recruit somebody else to become a part of the testing team if you have a budget for this –    use rational methods and approaches to find common ground with the current team to prevent the appearance of the conflicts on the stage of joining the project

2- “How to start the process: the first tester on the project.”

It was a beneficial presentation for junior testers. And I remembered my first job and the very first project. It was a mobile application for renting apartments in the Netherlands on IOS and then on Android. Testing on iOs was a new task for me as I had only Android devices before. Mobile testing was gaining popularity at that time. According to the report and based on my personal experience, I can give the following advice: –    find out and examine all details of the project getting information from developers, project manager, client if it’s acceptable; –    find out all timelines, deadlines in the project, – maybe you should start testing and give information regarding status or readiness of the project to release even today; –    examine all project documentation (design, prototypes, specifications, SOW, emails) if it is available, of course –    create testing documentation, for a start simple checklist for testing will be enough; –    conduct “exploratory testing”, if there is any available build –    compare documentation with a build and analyze if documentation is up to date; –    get along with the development team; –    introduce (create) templates of the testing documentation and bug reports within the project.

3- “Start performance testing from scratch.”

This report was about the way to start performance testing from a clean sheet of paper or scratch. There are a few applications for executing this type of testing. One of them is Jmeter. This is an open source application that allows run testing for beginners and experts in testing as there is a possibility to record and upload scripts or create them manually. The main criteria that you should know and be able to handle are as follows: –    models and profiles of the loading –    metrics for analysis –    NFRs Models of loading consist of several parameters: –    number of flows –    period of time, within which all flows become active (ramp-up) –    duration of the test There can be the following models of loading: capacity, load, stress, SOAK (endurance). The main app-side metrics, with which we can analyze the work of the system, include response time that can be as follows: –       average (avg-mean) –       median –       90, 95, 99 percentile –    min and max Besides it, there are server metrics such as CPU, RAM, DISK I/O, Network, that allow monitoring of how servers “feel” during the test and help to see bottlenecks related to configuration and hardware. To fulfil performance testing properly, we should evaluate the system, starting with simple scenarios. Moreover, have the scheme of relationships between the base, balancers and servers. Then we look at the trends of the metrics, compare obtained results, find dependencies and conduct an analysis of the results. At the close, we submit a report where all results will be present.

4- “Models of the testing processes maturity, their differences, and when we can use them.”

This report gives us an idea of the models of testing processes maturity. It tells us how we can evaluate which model we have and how we can move to a higher level of the testing model. There are five levels of maturity models: 1.Initial – unpredictable and reactive. Work gets completed but is often delayed and over budget. 2. Managed – managed on the project level. Projects are planned, performed, measured and controlled. 3.Defined – proactive, rather than reactive. Organization-wide standards guide projects, programs and portfolios 4. Quantitatively Managed – measured and controlled. The organization is data-driven with quantitative performance improvement objectives that are predictable and align to meet the needs of internal and external stakeholders. 5. Optimizing – stable and flexible. The organization is focused on continuous improvement and is built to pivot and respond to opportunity and change. The organization’s stability provides a platform for agility and innovation. Maturity models can be of the following three types: –    levels deep, where achievement of the goals within predetermined areas defines a certain level, which is the basis for the next levels –    contiguous, where the set of capabilities proposes a particular way of development and improvement of the processes in each specific process area –    leafed, where each characteristic that identifies the maturity of the company in the field of project management organization is evaluated according to appropriate grade that allows seeing the underperformance of the company according to each of the specified characteristics. There are the following maturity testing models: –    Testing Maturity Model integration (TMMi) –    Test Process Improvement (TPI) Next –    Critical Testing Processes (CTP) –    Systematic Test and Evaluation Process (STEP) –    IDEAL Model The maturity testing models are used under the following conditions: –    longtime projects –    multiple-complex systems –    a large number of competitors –    the cycle of operation and frequency of the processes –    need for documentation –    need for evaluation and control of expenses –    demand for high forecastability of the terms To implement the use of testing models, we should: –    think well over the question “why?” –    think well over the question “in which way?” –    ponder over the terms and establish goals –    conduсt retrospective and revaluation regularly –    forge ahead –    no to be afraid of the changes and opposition

5- “Automatic test data generation.”

This report reveals how to make your work simpler and make test design excellent again by using automatic test data generation. There are the following types of automatic test data generation: –        Random Test Data Generation –       Search-Based Test Data Generation –       Genetic Algorithms and Programming –       Test Sequence Generation –       Tools Random test data generation has the following characteristics: –       Arbitrary inputs –       Fast and easy –       Enormous test suites –       Small programs only –       Fuzzing Following qualities can characterize Search-Based Test Data Generation: –       Optimization problem –       Huge input domain –       Coverage threshold –       No single correct answer Genetic Algorithms and Programming: –       Fitness function –       Populations and generations –       Multiple targets in one run –       Reproduction, crossover –       and mutation Test Sequence Generation –       Combinatorial testing –       Formal models –       Generation rules –       Dependency rules –       Prioritization and weights We can choose the necessary type of data generation, depending on our project and examples of the testing data. All of these will simplify your life and set free from performing monotonous tests. I want to express my gratitude to 247Labs for support in visiting testing conferences and meetups. And my gratefulness is always reflected in my good work as we have one main overall goal – to make users happy!