Beyond Continuous Testing with Artificial Intelligence. By Wolfgang Platz, Founder and CPO. tricentis.com

Transcription

1 Beyond Continuous Testing with Artificial Intelligence By Wolfgang Platz, Founder and CPO tricentis.com

2 From the golden robots of Hephaestus, to Dr. Frankenstein s monster, to Hal 9000, we ve been fascinated by the idea of artificial intelligence (AI) for centuries. Creating artificial selfdirecting intelligence is the #2 dream of mankind second only to being able to fly like a bird. However, while the Wright brothers were off the ground in 1903, AI hasn t been ready to take off until very recently. AI requires data + computing power + algorithms. We ve had the algorithms for a long time; now, big data and colossal computing power have brought us past the tipping point. The feasibility of AI couldn t come at a better time. Today, Digital Disruption is forcing enterprises to innovate at lightning speed. We need to dedicate resources to creating new sources of customer value while continuously increasing operational agility. Otherwise, we risk waking up one day to find out like Nokia that even though we did nothing wrong, we somehow lost. The speed of digital disruption is already staggering, and it s only going to increase. To put this into some very concrete terms, consider that: There are 7.4 billion people in the world 4.5 billion have regular access to a toilet 5.5 billion own a mobile phone All of a sudden, a huge number of people jumped from a very provincial lifestyle straight into digital times creating a tremendous demand for more, and more innovative, software. Anyone responsible for producing software knows that the traditional ways of developing and delivering software aren t adequate for meeting this new demand. Not long ago, most companies were releasing software annually, bi-annually, or quarterly. Now, iterations commonly last 2 weeks or less. While delivery cycle time is decreasing, the technical complexity required to deliver a positive user experience and maintain a competitive edge is increasing as is the rate at which we need to introduce compelling innovations. 2

3 In terms of software testing, these competing forces create a gap. We ve turned to Continuous Testing to bridge the gap that we re facing today. But how do we test when, over time, these trends continue and the gap widens? We will inevitably need to go beyond Continuous Testing. We will need additional assistance to be able to deliver a positive user experience given such high delivery speeds and technical complexity. We ve already undergone quite a journey to arrive at Continuous Testing. Classical testing was designed for software delivery cycles that span months (or sometimes even a year). Agile has made 2-week development iterations the norm but now even more is needed to meet today s voracious demand for software. Attempts to further accelerate the process exposed a chasm between Development, Test, and Operations. That chasm needed to be bridged with DevOps and Continuous Testing in order to move beyond that acceleration plateau. Today, the vast majority of organizations are talking about Continuous Testing and trying to implement it. Nevertheless, when we look into the future, it s clear that even Continuous Testing will not be sufficient. We need help. We need Digital Testing to achieve further acceleration and meet the quality needs of a future driven by IoT, robotics, and quantum computing. AI, imitating intelligent human behavior for machine learning and predictive analytics, can help us get there. What is AI? Before we look more closely at how AI can take software testing to the next level, let s take a step back and review what AI really means. Forrester defines AI as 3

4 A system, built through coding, business rules, and increasingly self-learning capabilities, that is able to supplement human cognition and activities and interacts with humans natural, but also understands the environment, solves human problems, and performs human tasks. I think the key points in that definition are self-learning (it needs to have learning capabilities) and human cognition (which is achieved by understanding the environment and context). Another interesting definition is that AI is A field of study that gives computers the ability to learn without being explicitly programmed. One of the key points of AI is that you do not need to explicitly program algorithms. Algorithms are certainly used, but they re not designed for an explicit solution of a distinct problem. The machines can learn, and they use data to do so. Any discussion of deep machine learning must involve the term neural networks. The idea is that you have a large set of input data, and this input data leads to a narrower set of outputs. To process that data, you use neurons, which are in between the input and the output. These neurons are assessment points; parameters that help classifiy the input. In a neural network, you don t have just one layer; you have multiple ones. They need to parameterize themselves, which means they learn in order to provide the right output results from the inputs. To make this concept more tangible, let s consider the example of optical facial recognition. The input layer photos are sent to a machine, assesses these different pixel areas in the photo to determine if it s a face: 4

5 All the layers in the neural networks are parameterizing themselves learning how to make the correct assessment. In summary, let s conclude that: 1. If it isn t learning, it s not AI 2. If you need to explicitly program the transition from input to output, it s not learning and therefore not AI How Can AI be Applied? As a rule of thumb, any complex task that a human can solve without consciously making a decision is a candidate for AI. The most prominent example of AI is image recognition capabilities. If you look out the window, you instantly recognize what s there without having to think about every little detail and consciously analyzing all the patterns. It s really an incredibly complex task, but you can do it instantly. It s a huge challenge for a machine, but it is feasible with AI. The results of what AI can deliver in this respect are already astonishing. Since the machine is applying intelligence to a very specific problem, this application falls under the category of narrow AI. Voice recognition is another example of this. You hear what people say and you immediately understand what they re talking about. You don t need to decompose the pattern of speech; you just know it. This is another area where AI is already achieving amazing results. Siri, Alexa, Google, and other voice recognition capabilities are fantastic. The advancements over the past couple years are breathtaking. More sophisticated (general) applications of AI build on these elementary capabilities. For example, autonomous driving builds on AI-based visual recognition. 5

6 Here s a more expansive look at the various areas where AI is being used and advanced: Human emotions: Analyzing facial features, reacting to human emotions, identifying criminals Natural language processing: Contextual understanding, generation, translation, transcription Visual and image processing: Visual emotions, activity recognitions, object recognition Audio: Voice recognition, sound recognition, intonation Negotiation: Negotiate and understand negotiation processes, consider cultural factors Robotics: Robot learning, multi-robot systems, multi-legged walking, embodied cognition Probabilistic reasoning: Common sense knowledge, reasoning about actions, nonmonotonic reasoning Applying AI to Software Testing To meet the challenges presented by accelerating delivery speed with increasing technical complexity, we need to follow a very simple imperative: Test smarter, not harder As I mentioned earlier, we re fast approaching a time when Continuous Testing will be unable to keep pace with shrinking delivery cycles times, increasing technical complexity, and accelerating rates of change. We re already starting to use basic forms of AI, but we need to continue the testing evolution in order to achieve the efficiency needed for Digital Testing of robotics, IoT, and so forth. For example, consider how image recognition can be used to bring UI testing to the next level so that dynamic UI controls (e.g., for responsive sites) can be automatically recognized in all their various shapes and forms. We can already recognize UI controls from the human perspective, beyond plain template matching. The UI s pixel structures can be interpreted with the goal of recognizing image patterns and identification information such as text. For example, edge detection can be used to identify the following control as a button and OCR/ GDI layers can be used to identify its text. 6

7 Using AI, the software testing tool can learn to identify controls during the scanning and test execution process independent of the control size, color, text alignment, etc.. A learning approach is applied:, images are generalized by adding new image patterns and adapting existing image patterns during the scan and test execution process. Each control knows its context on the graphical interface via anchor identification. Control properties are extracted from the image pattern, and can in turn be used to constrain automation of certain controls (for example, a scrollbar) during test execution. This eliminates the need to recognize them from their technical implementation (ID, etc.). The end result of this learning? It enables repeatable and stable test execution, even on responsive UIs. This will be the next chapter of UI test automation. Not AI But Does That Matter? Given AI s buzzword status, it s all-too-commonly applied to innovations that are incredibly exciting and valuable but are not AI because they don t involve self-learning. 7

8 One of the most prominent examples of this is when IBM Deep Blue beat Gary Kasparov in It passes the Turing test 1, but it s not AI. Why? Because it s explicit programming without learning. The next logical move is determined by algorithms, but it s not a selflearning system. Another achievement commonly mislabeled as AI is General Electric s project where jet engine logs are sent to a data hub every second and continuously analyzed to determine whether the engines need a sanity check. This innovation has been able to reduce the need for engine checks by over 60%. Since engine checks are incredibly costly, this will have a significant impact on the bottom line. Again, this is an incredibly innovative and valuable computing feat, but it s not AI. The trend of labelling any breakthrough algorithmic innovation AI applies to the software testing industry as well. Self-healing technologies fall into this category. One of the most interesting self-healing technologies in software testing is technology that updates broken (out-of-sync) test cases at runtime. There are typically multiple ways to identify a control (such as a button) that you re trying to automate. Each control is usually identified by a technical property, such as an ID, a name, etc.). In many cases, multiple sets of properties are redundantly unique to a control. For example, assume 2 of the 3 identifiers being used to identify a particular control can no longer locate it. If just 1 of the 3 identifiers is still current, self-healing technology can be used to identify the control, as well as update the other 2 identifiers. I don t dispute the value of this technology at all. In fact, it s something we ve developed at Tricentis, and we have seen first-hand how it can save testers from having to constantly update test cases as applications evolve. Yet, despite the self in its title, self-healing technology is driven by advanced algorithms, not self-learning. It s not AI. It s what I call cognitive testing. Nevertheless, it is a prime example of the type of innovation we need to satisfy the demands of digital testing beyond Continuous Testing. 1 The Turing Test was designed to be a rudimentary way of determining whether or not a computer counts as "intelligent." The test is carried out as a sort of imitation game. On one side of a computer screen sits a human judge, whose job is to chat to some mysterious conversation partners on the other side. Most of those conversation partners will be humans; one will be a chatbot, created for the sole purpose of tricking the judge into thinking that it is the real human. 8

9 Wrap-Up Unquestionably, we need to learn how to work smarter, not harder. To ensure quality in an era where software will be processing an unimaginable number of data points in real time for example, both figuratively driving the internet of things and literally driving self-driving cars we need all the help we can get! Innovations labelled AI might get all the attention, but we ll need to extend beyond AI in order to excel. In terms of software testing, some of the non-ai / cognitive testing inventions that I expect to be critical include automated test portfolio optimization, self-adjusting risk assessment, automated defect diagnosis, and smart environment provisioning. The bottom line: AI is perfectly poised to advance software testing by automating tasks that involve self-learning and traditionally required human cognition. We can and must continue to dream about AI and make it a reality but we can t afford to dismiss the role of smart testing technologies that lack the trendy AI label. Learn more about AI and the future of testing Watch "The Future of Testing" on-demand. About Tricentis Tricentis, the Continuous Testing Company, specializes in market leading agile software testing tools for enterprises. We help Global 2000 companies adopt DevOps and gain success by achieving automation rates of over 90%. Top analysts recognize Tricentis as a leader in Software Test Automation, with Model-based Test Automation and Test Case Design as standout features. Our 400+ customers include global names from the Top 500 brands such as A&E, Allianz, Deutsche Bank, HBO, JetBlue, Orange, Swiss Re, Telstra, Toyota, UBS, Vantiv, Virgin Airlines, and Zurich Insurance. Tricentis has offices in Austria, United States, Germany, Switzerland, UK, Netherlands, Poland, and Australia. 9