Skip to content

Death to sleeps! Raise of Conditions!

by on June 3, 2013

This is a response to a blog post by Matt Wynne: Death to sleeps! (2013-06-03). It is actually a great summary of problems we observed in our automated Selenium-WebDriver Tests and I do not get tired repeating the great BDD-step which Matt also observed: “And I wait for 3 seconds” – yes, I found that one many times!

The solution we have found is Java-based and derived from the ExpectedCondition and its factory ExpectedConditions available in Selenium WebDriver. But we were in need of a much broader approach as we not only wait for WebElement’s to appear but also for documents indexed in the search-engine. So let’s extend Matt’s example:

Given I am on the google homepage
And Wikipedia Page for Matt Damon is indexed // inserted
When I type "matt" into the search box
Then I should see a list of results
And the wikipedia page for Matt Damon should be the top result

with And Wikipedia Page for Matt Damon is indexed implemented like:

googleWrapper.indexed(matt).waitUntilTrue();

assuming that we have direct access to the search engine and the step Then I should see a list of results implemented like:

resultPage.getList().visible().waitUntilTrue();
resultPage.getList().length().assertThat(greaterThan(0));

where resultPage and the returned list are wrapper objects for UI-elements and length() and visible() return what we call a condition.

The general concept can be found in this outline:

WaitPattern_Responsibilities_Joala_160dpi

Actually we have three responsibilities in this pattern:

  • Expression: knows how to query the System Under Test (SUT) for a certain value
  • Matcher: (see Hamcrest) will decide if a returned value matches expectations or not
  • Condition: is responsible for repeating the whole process and eventually end with a timeout if expectations were not met in time

Having the conditions we were also able to identify and fix another problem: If we poll the SUT too often it might not even been able to process its update loops. We observed these for search-machines which we polled to often. Olaf Kummer described this problem and the solution in Haste makes waste.

Important about the whole concept of Expression and Condition was a clear contract how the Expression could send a signal to the condition if it assumes that a re-evaluation will lead to a better result or not. So we identified three termination paths illustrated here:

WaitPattern_Termination_Joala_160dpi

So the Expression might end in three ways:

  • returning a value: the Condition will ask the Matcher if this is what we waited for
  • ExpressionEvaluationException: the contract says that this signals that the Expression expects that a second query might be successful; this for example wraps ElementNotFound-Exceptions.
  • any other Exception: will tell the Condition to not ask again; a severe error got observed

And the Condition also might end in three ways:

  • Timeout: the value did not match in a given time or we always only get the ExpressionEvaluationException
  • Exception: any other exception raised by the Expression
  • Matched: signals that the assertion (in JUnit-context) passed

Experience shows that this approach got well accepted by our developers and Thread.sleeps() are gone nowadays – especially since we also solved the problem how to wait for “something not to happen” like “the error dialog should not pop up”. If you are interested in this approach read on here: The Calm after the Storm – also by Olaf Kummer.

I presented our solution on the XP-Days 2012 in a session “UI-Tests which are fun”. Find the slides (in German though) in my blogpost from November last year.

If you are interested in the implementation you can find it in Joala on GitHub, more specifically in the conditions-module.

From → Dev, Testing

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s