Death to sleeps! Raise of Conditions!
This is a response to a blog post by Matt Wynne: Death to sleeps! (2013-06-03). It is actually a great summary of problems we observed in our automated Selenium-WebDriver Tests and I do not get tired repeating the great BDD-step which Matt also observed: “And I wait for 3 seconds” – yes, I found that one many times!
The solution we have found is Java-based and derived from the ExpectedCondition and its factory ExpectedConditions available in Selenium WebDriver. But we were in need of a much broader approach as we not only wait for WebElement’s to appear but also for documents indexed in the search-engine. So let’s extend Matt’s example:
Given I am on the google homepage And Wikipedia Page for Matt Damon is indexed // inserted When I type "matt" into the search box Then I should see a list of results And the wikipedia page for Matt Damon should be the top result
with And Wikipedia Page for Matt Damon is indexed implemented like:
assuming that we have direct access to the search engine and the step Then I should see a list of results implemented like:
resultPage and the returned list are wrapper objects for UI-elements and
visible() return what we call a condition.
The general concept can be found in this outline:
Actually we have three responsibilities in this pattern:
- Expression: knows how to query the System Under Test (SUT) for a certain value
- Matcher: (see Hamcrest) will decide if a returned value matches expectations or not
- Condition: is responsible for repeating the whole process and eventually end with a timeout if expectations were not met in time
Having the conditions we were also able to identify and fix another problem: If we poll the SUT too often it might not even been able to process its update loops. We observed these for search-machines which we polled to often. Olaf Kummer described this problem and the solution in Haste makes waste.
Important about the whole concept of Expression and Condition was a clear contract how the Expression could send a signal to the condition if it assumes that a re-evaluation will lead to a better result or not. So we identified three termination paths illustrated here:
So the Expression might end in three ways:
- returning a value: the Condition will ask the Matcher if this is what we waited for
- ExpressionEvaluationException: the contract says that this signals that the Expression expects that a second query might be successful; this for example wraps ElementNotFound-Exceptions.
- any other Exception: will tell the Condition to not ask again; a severe error got observed
And the Condition also might end in three ways:
- Timeout: the value did not match in a given time or we always only get the ExpressionEvaluationException
- Exception: any other exception raised by the Expression
- Matched: signals that the assertion (in JUnit-context) passed
Experience shows that this approach got well accepted by our developers and
Thread.sleeps() are gone nowadays – especially since we also solved the problem how to wait for “something not to happen” like “the error dialog should not pop up”. If you are interested in this approach read on here: The Calm after the Storm – also by Olaf Kummer.
I presented our solution on the XP-Days 2012 in a session “UI-Tests which are fun”. Find the slides (in German though) in my blogpost from November last year.