Testing Nexus With Selenium: A Lesson in Complex UI Testing (Part 3)

All web applications have some concept of state. Usually that state is what is contained in the database and possibly a caching layer. But in the case of Nexus, there are additional application states that are especially hard to reproduce. For example, recreating the state necessary to produce a dialog box warning that a file download attempt from the central repository failed is not trivial to do. Doing so would require the central repository to be temporarily taken offline - something that wouldn't make the millions of Maven users very happy!

So clearly there are some things we need to mock out. But the central repository example is just one thing. How many more behaviors must we also mock out? At this point, it is worth looking at the Nexus UI architecture to see if there is perhaps a single entry point that we can mock out.

Nexus uses a "single page" architecture. What I mean by that is there is conceptually only one page for the entire UI. The rest of the interaction and UI complexity is created dynamically using AJAX. Think about how Google Maps works: the location bar in your browser never really changes despite very big changes within the page. Nexus works exactly the same way.

The UI components themselves are generated using ExtJS and a lot of JavaScript. But the data that controls how and when those components get rendered comes from the Nexus web server via  REST API. So if we could mock out that API, we could essentially create only one mock rather than trying to mock all the edge cases where the application state is hard to reliably reproduce.

You might be asking yourself, "If you mock out the REST interface, are you even testing Nexus at this point?" While it's true that we would no longer be exercising any of the server-side application, since everything would be intercepted with mocks, it's important to note that that is not the goal. Nexus already has a very large suite of integration tests that are designed to test exactly that. These Selenium tests, on the other hand, were specifically designed for UI testing only. Because our goal isn't to do an end-to-end test, we can mock out the REST API without any problem.

Recall the ChangePasswordTest we saw earlier - you may have noticed there was a "..." comment in the middle where we cut out part of the test. Let's now look at the test in its entirety:

public class ChangePasswordTest extends SeleniumTest {
    @Test
    public void changePasswordSuccess() {
        main.clickLogin()

                .populate(User.ADMIN)
                .loginExpectingSuccess();

        ChangePasswordWindow window = main.securityPanel().clickChangePassword();

        PasswordChangedWindow passwordChangedWindow = window
                .populate("password", "newPassword", "newPassword")
                .changePasswordExpectingSuccess();

        passwordChangedWindow.clickOk();
    }
}

What we've done is made a call to MockHelper to tell it that the next call to /users_changepw (one of the many REST APIs) should return a mock response in which there is no data returned, but that we examine the data submitted to the REST API and confirm it matches what was entered in to the change password window.

We can use this technique to effectively stub out unique logic that is hard to reproduce. Even better, because Nexus uses a Plexus-based REST framework, we can work with these stubs using Java and they will be automatically marshaled to a format that the Nexus UI can understand. 

The key thing that enables this simple test design is the fact that the mock web server (it is not a fully working Nexus instance) and the test that controls Selenium are both running within the same JVM. Recall that ChangePasswordTest extends SeleniumTest. It turns out, SeleniumTest extends NexusTestCase, which is responsible for spinning up the mock web server, which will host the Nexus UI (HTML, CSS, JS, etc) and the mock REST API framework.

Once both the test and the mock server are running inside the same runtime environment, it's easy for the test to quickly set expectations that relate to the Selenium code that is before and after the MockHelper callout. 

One thing we didn't tackle with this approach was concurrent or parallel test executions. Modern unit test frameworks, such as JUnit4 and TestNG, now allow for the running of parallel test cases. This is used to significantly speed up the time it takes to complete a build. But given the way that MockHelper is currently being used, it would be impossible to know or guarantee that a call to /users_changepw from test X or test Y should be mapped to browser session X or browser session Y.

Fortunately, this isn't a terribly difficult problem to overcome. With a little work in SeleniumTest, one could easily assign a random string to a ThreadLocal and and a cookie in the browser session. You could then use that string to uniquely associate mock calls with browser sessions and a map of expected response to REST API calls on the server side. The test itself would look the same, but now they could be run in parallel on a Selenium Grid.