What is Behave?
Behave is a behavioral driven testing language that uses natural wording as its syntax. It is backed by python code, and encourages collaboration between developers, testers, and non-technical product participants. Behave tests are written in plain english so they are easily read and easy to understand. It allows everyone involved in the software project to have a clear picture of the business requirements and expected outcomes just by reading the tests. Here at Narrative Science, we expect all of our engineers to ‘behave’ when writing code.
Why do we behave?
We write our tests with Behave for many reasons. One of the biggest is for test readability and organization. As a tester or developer, you may have been in a situation where there are tests that are unorganized or do not make sense. You may have asked yourself or others, “what does this test do?” or “why is this test here?”. In Behave, each “feature” is classified as a test. Each feature has many test cases or “scenarios”. Each scenario is organized and written for a specific purpose. Each scenario is written in a way that can be understood from an outside perspective using given, when, then syntax. If someone new is joining the team, they are able to read a feature and understand what the test is trying to achieve and validate from just a few sentences written in plain english.
Second, using Behave allows for code reusability. It is very easy to do generic REST API testing with the regular expression step matching of Behave. Take this example below for instance: The first picture shows a few scenarios for user operations, and the second picture shows an overview of how the step matching python code would be written.
With little python code, you can write a framework that will send and validate API requests and responses. After the generic python code is written, you can test basic APIs just by writing new feature files. This makes writing new status code so easy almost anyone can write them, even if they have little to no technical experience or no testing background.
The third reason to behave is one that frustrates developers and testers the most, debugging. Everyone hates debugging failed tests, especially when they have no idea what the tests are doing. Remember we talked about how Behave is easy to read? Well guess what, it is easy to debug as well. When a test fails you can see exactly what step the test failed on. Depending on how descriptive you are in your test steps and logging, Behave will highlight what assertion the test failed on and give you a clear path to look into the failure.
The final reason to use Behave is for test reporting. Behave has many different types of reporters and test data outputters built in. Whether you want an XML, HTML, or JSON report of your test data, Behave has the capability out of the box. At Narrative Science we output our test data in JSON. We then pipe that test data through our ETL pipelines where the data eventually ends up in Athena. We can then view this data in Lexio! This is the kind of stuff that gets most of us here at Narrative Science excited. We are so data centric here that most of us eat, breathe, and dream about data. The fact that our test data can be turned into a story that gives us insights on how our tests are running in our PRs or deploy pipelines is unbelievable.
Where do we behave?
Narrative Science likes to ‘behave’ wherever possible. End-to-end, integration, and functional tests are areas we have found that Behave works really well. We have found that our selenium and UI tests were taken to the next level when integrating them with Behave. It really makes things clear when you can directly state the actions you want to perform in the browser in plain English. Imagine wanting a user to enter a username and password into a textbox and click login. Instead of having to write a few functions and a bunch of python code, you can just write a Scenario that will read exactly how the actions should be performed. To achieve this, we use the selenium page object model mixed in with Behave. We also have our own custom libraries of functions that are callable from each Behave step definition. With this combination it makes writing new UI tests seamless with very little new python code. Every “action” that can be performed in selenium is wrapped into a clean and easily understood Behave step that can be reused in any scenario. All of our test code is very modularized. Behave features talk to the step libraries. The step libraries contain small functions that call our test functions library. Each of the test functions do all of our heavy lifting. This makes our Behave code clean and easy to maintain. When an engineer wants to write a new test, 9 out of 10 times they have everything they need already built into our test frameworks. There is no need to write any more python code. There are exceptions to this, but they are few and far between.
How quickly do we behave?
We believe that all tests should run quickly but still provide value. Our goal for our PR tests is they all must run in under 10 minutes and our test frameworks are what makes this possible. The test framework starts up and sets up all necessary users and infrastructure in less than 10 seconds. For the most part, each of our Behave features runs in under 1 second. There are some exceptions to this with our selenium tests, but they do not take too long to execute. We can run API tests in milliseconds, therefore allowing us to run our entire suite of status code tests in under a minute. If you are curious about the number of API calls we make during that time, it’s roughly 1000. These API calls test our 200, 400, and 500 level status codes. Effective API testing is essential to any platform, but creating something that runs quickly and is still effective is the trick.
How do you start?
It’s easy! Behave has a quick start guide and even example projects that you can clone and play with. There are many examples of how to write your Features, Scenarios, and Steps as well as the step definitions in python.