Android Testing Strategy
Testing an Android application was always hard. In the past couple of years, we got access to better tools, but we still rely a lot on architecture and design decisions in order to properly test an Android application. This article will discuss the layers of automated testing an Android application should have, from End-to-end, Component to Unit tests.
Arrange, Act and Assert!
Automated testing can be summarized into three steps. Arrange: the step where we elaborate on the test. Act: when the code under test is exercised. And finally Assert: where the result or behavior of the code is verified.
Dependency Injection is a crucial part of the Arrange step since it allows different configurations for a specific class. In other words, we can inject different dependencies for a specific class so when we Act, we can Assert different outcomes.
In the Android world, there are limitations for Dependency Injection. Developers don’t have access to Android components’ constructors since the Android OS controls the instantiation of Activity, BroadcastReceiver, Service, ContentProvider and Application classes.
There are some libraries like Dagger or some strategies one can use to be able to inject dependencies in Android components, but there are other limitations in order to Arrange tests. For example, if a test depends on a specific Android OS event (e.g. Batter Level, Sensors, etc…), there is currently no easy way to simulate them.
Another problem is the speed of the Android emulator. There were huge improvements in Android emulator speed, yet it is still cumbersome to do TDD in case your tests depend on the emulator.
Clean Architecture
In order to mitigate the aforementioned problems, a well-defined architecture is necessary. By separating the code into different layers and clearly defining borders with abstractions between them, it is possible to test parts of an Android application independently. It is even possible to completely abstract Android SDK from the application’s use cases, so they could be reused for a different platform.
If one attempts to follow Clean Architecture by the book in an Android application it can end-up with accidental complexity, due to the multitude of classes that need to be created. But one can use the concepts behind Clean Architecture to isolate the layers of an Android application so they can be easily tested.
One of the key concepts of Clean Architecture — if not the most important one — is the central role of the Domain. If we divide our application into three parts: Presentation, Domain, and Data. Where Presentation is composed of Views, Fragments, Activities, Presenters, ViewModels, and Controllers. Domain is composed of all the classes that implement application use cases. Data is a thin layer that adapts to the needs of the Domain, by providing data, for example from a Database via SQL, from an external API via HTTP or from a Bluetooth connection via Android SDK’s BluetoothAdapter.
Domain has a central role because it is unaware of any implementation details of the Presentation and Data layers. Usually, Presentation will have a reference to the Domain classes, requesting the execution of use cases according to GUI events fired by user interaction. The Domain will then request data from the Data layer through an abstraction (e.g. an interface), so it will be indifferent to where the data comes from. When the Domain is done processing the use case it will notify Presentation via another abstraction (e.g. a Callback or Observable).
Note that throughout the flow the Domain always used abstractions to talk to the other layers, and most importantly abstractions which are defined within the Domain itself. If we were to create a library from the Domain classes, the interfaces that represent the abstractions would be part of the library.
Unit Tests
Domain First
Once the application architecture establishes that the Domain layer is independent, it is easy to write Unit Tests for it. Also since there are no dependencies on the Android SDK, the tests can run in the local JVM.
While the tests are being written for the Domain the developer can create the contract (abstraction) that the Data layer will later fulfill. Using Test Doubles(Stubs, Fakes or Mocks) the Domain can be exercised and understood without the burden of thinking about Data implementation details.
The developer doesn’t need to focus on Presentation at first, as well. These are the reasons why the Domain usually is the best starting point for any new feature and after it is implemented, the developer can focus its attention on Data and Presentation layers.
Data
Data layer should be a thin layer that will adapt a specific data source to the Domain contract.
Always be suspicious of a complex Data layer, it may be implementing business logic that should be in the Domain.
In order to help test the Data layer, typically I would go for a tool related to the technology I’ve chosen to implement it. For example, OkHttp is quite a famous library in Android universe and it has a tool called MockWebServer, which basically makes it easy to mock responses and verify requests. If we went for SQLite to implement the Data layer, SQLiteOpenHelper has an option to create an in-memory database that is perfect for testing purposes.
The key factor is that the test has to be hermetic and should not perform any I/O operations. In that way we avoid flakiness (e.g. a test failing because of network issues) and it will make the tests runway faster. If the chosen technology has no tool to facilitate testing, we can create new abstractions to isolate the logic related to data fetching/serialization and then use Test Doubles for technology-specific classes.
Presentation
Presentation is a tricky part of Android because it is difficult not to have a hard dependency on the Android SDK. After all, we use Fragments, Views, and Activities to create the GUI. We could create Presenters, ViewModels, and Controllers to help abstract the Android SDK, and to be sincere that was my strategy till recent times.
Since the Google I/O of 2017 there is a push to integrate Robolectric into Espresso testing framework. Robolectric helps in the implementation of Unit Tests by creating a Stub version of the Android SDK classes, which allows the tests to run on the local JVM.
Today it is possible to add Espresso tests to the /test folder of your Android project and they will run on the local JVM using Robolectric, instead of on an Android emulator.
It is needless to say the tests run way faster without the emulator and since that breakthrough, from a testing perspective, the importance of Presenters, ViewModels and Controllers have diminished. Of course, if a specific screen of an Android application has complicated presentation logic, those classes are helpful, but there is no need to create them for every screen anymore.
Component Tests
After all layers of the architecture are tested in isolation, there is still a need for a test that integrates all of them to verify a use case is properly implemented. A Component test can be created with Espresso to exercise Presentation, Domain, and Data through the GUI.
Component tests are going to be the ones that will allow you to sleep at night after a major refactor of the application. Because they are inherently Black Box tests, they don’t need to be changed when implementation details change. Unit Tests tend to be White Box, therefore more tighten to implementation details. If we were to elect the most important type of tests, they would be the Component ones.
A Component test has to be hermetic, therefore we should fake any external services (e.g. Database or Web Service). In order to do that, we can use the same tools and abstractions that we’ve used to test the Data layer. Also, I favor executing Component tests on an Android device or emulator, instead of using Robolectric, so we can have maximum possible classes integrated.
Fake Data
Due to the lack of access to Android components’ constructors, the trick part in Component tests is to replace parts of the Data layer with Test Doubles. The first thing is to separate the classes that provide dependencies from the classes that actually use them. Dagger helps to accomplish that, but there are other ways, like using the Application as a dependency provider. Actually, for Dagger to work properly, the Application has to provide a Dagger Component anyways.
Once we have a centralized place that provides the dependencies for the application, there are several ways to replace some of them for testing purposes. One is to use a custom AndroidJunitRunner to replace the entire Application only when running the tests (it would allow us, for example, to replace a Dagger Component). Another way is to use reflection to replace specific dependencies with Test Doubles. The former is easier but it replaces the entire Application removing a part of the code that should be under test. The latter is more complicated to implement, but it changes less code, therefore resulting in a more realistic test suite.
End-To-End Tests
Finally, End-To-End tests are the ones that exercise the full stack of an application, including external services. End-To-End tests are not hermetic and are flaky by default, thus they can provide false negatives. When such tests fail, their results should be double-checked by a human being to verify that the reason is not, for example, due to intermittent internet connection.
Also, End-To-End tests are hard to maintain because their Arrange step is complicated. Sometimes it means setting up an isolated environment (e.g. Staging or QA environment) or calling APIs to prepare the state of the system before the tests run.
Because of the aforementioned issues, an application should have few End-To-End tests. Usually, they are focused on the critical paths of the user in the application, therefore error states and edge cases should not be part of an End-To-End test suite.
Conclusion
In order to thoroughly test an Android application different types of automated tests are necessary. By missing one of those types the application becomes vulnerable and the confidence the developers have on changing it diminishes. Unit Tests serve to drive the implementation (especially when TDD is being used), Component Tests verify the use cases are properly implemented and finally End-to-End Tests verify the interaction of the application with the rest of the system.
