5.12.15 QA Lab: тестирование ПО. Upcoming events: goo.gl/I2gJ4H Доклад о популярных и/или лучших практиках в веб-автоматизации с точки зрения принципа KISS. Мы быстро пробежимся и оставим след в следующих темах и технологиях: Selenium, обертки вокруг Selenium, xUnit, BDD, длинные и не очень - End to End сценарии, маленькие независимые тесты Unit стиля, простые или красивые репорты, Allure репортинг, PageObject, виджеты-элементы, парадигмы программирования для автоматизации - ООП, процедурное и модульное.

1. KISS Automation
@yashakaitlabs.net.ua

2. Afterwords
There are good practices in context,
but there are no best practices.
(c) Cem Kaner, James Bach

3. Afterwords Preface
There are good practices in context,
but there are no best practices.
(c) Cem Kaner, James Bach

4. KISS?

5. Keep It Simple Stupid!

6. Web UI Automation…

7. Selenium vs Wrappers

8. Wait for text

9. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(textToBePresentInElement(page.element, ":-*"));
Wait for text

10. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(textToBePresentInElement(page.element, ":-*"));
Wait for text

11. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(
driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(
textToBePresentInElement(page.element, ":-*"));

12. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(
driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(
textToBePresentInElement(page.element, ":-*"));

13. Wait for text

14. Wait for text
SelenideElement element = $(“#element");
...
element.shouldHave(text(":-*"));

15. Not mandatory, still possible…

16. Not mandatory, still possible…
public class ExamplePage{
WebElement element = $(“#element");
}
...
ExamplePage page = new ExamplePage();
page.element.shouldHave(text(":-*"));

17. WrappersSelenium

18. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(
driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(
textToBePresentInElement(page.element, ":-*"));
SelenideElement element = $(“#element");
...
element.shouldHave(text(":-*"));vs
WrappersSelenium

19. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(
driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(
textToBePresentInElement(page.element, ":-*"));
SelenideElement element = $(“#element");
...
element.shouldHave(text(":-*"));vs
WrappersSelenium
Test logic
Test logic messed up
with tech details

20. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(
driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(
textToBePresentInElement(page.element, ":-*"));
SelenideElement element = $(“#element");
...
element.shouldHave(text(":-*"));vs
WrappersSelenium
Test logic
Test logic messed up
with tech details

21. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(
driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(
textToBePresentInElement(page.element, ":-*"));
SelenideElement element = $(“#element");
...
element.shouldHave(text(":-*"));vs
WrappersSelenium
may be SlowerFast

22. public class ExamplePage{
@FindBy(css = "#element")
WebElement element;
}
...
WebDriver driver = new FirefoxDriver();
ExamplePage page = PageFactory.initElements(
driver, ExamplePage.class);
(new WebDriverWait(driver, 6)).until(
textToBePresentInElement(page.element, ":-*"));
SelenideElement element = $(“#element");
...
element.shouldHave(text(":-*"));vs
WrappersSelenium
may be SlowerFast

23. End to End vs “Unit”

25. def test_task_life_cycle():
given_at_todomvc()
add("a")
edit("a", "a edited")
toggle("a edited")
filter_active()
assert_no_tasks()
filter_completed()
delete("a edited")
assert_no_tasks()
#...
End to End Scenario

26. def test_task_life_cycle():
given_at_todomvc()
add("a")
edit("a", "a edited")
toggle("a edited")
filter_active()
assert_no_tasks()
filter_completed()
delete("a edited")
assert_no_tasks()
#...
End to End Scenario
implicit
checks

27. “Unit” / “1-feature-per-test” style

28. def test_add():
given_at_todomvc()
add("a", "b")
assert_tasks("a", "b")
assert_items_left(2)
“Unit” / “1-feature-per-test” style

29. def test_add():
given_at_todomvc()
add("a", "b")
assert_tasks("a", "b")
assert_items_left(2)
“Unit” / “1-feature-per-test” style
def test_delete():
given_at_todomvc("a", "b", "c")
delete("b")
assert_tasks("a", "c")
assert_items_left(2)

30. def test_add():
given_at_todomvc()
add("a", "b")
assert_tasks("a", "b")
assert_items_left(2)
“Unit” / “1-feature-per-test” style
def test_delete():
given_at_todomvc("a", "b", "c")
delete("b")
assert_tasks("a", "c")
assert_items_left(2)
def test_edit():
given_at_todomvc("a", "b", "c")
edit("c", "c edited")
assert_tasks("a", "b", "c edited")
assert_items_left(3)

31. def test_add():
given_at_todomvc()
add("a", "b")
assert_tasks("a", "b")
assert_items_left(2)
“Unit” / “1-feature-per-test” style
def test_delete():
given_at_todomvc("a", "b", "c")
delete("b")
assert_tasks("a", "c")
assert_items_left(2)
def test_edit():
given_at_todomvc("a", "b", "c")
edit("c", "c edited")
assert_tasks("a", "b", "c edited")
assert_items_left(3)
def test_toggle():
#...

32. def test_add():
given_at_todomvc()
add("a", "b")
assert_tasks("a", "b")
assert_items_left(2)
“Unit” / “1-feature-per-test” style
def test_delete():
given_at_todomvc("a", "b", "c")
delete("b")
assert_tasks("a", "c")
assert_items_left(2)
def test_edit():
given_at_todomvc("a", "b", "c")
edit("c", "c edited")
assert_tasks("a", "b", "c edited")
assert_items_left(3)
def test_toggle():
#...
#...

33. def test_add():
given_at_todomvc()
add("a", "b")
assert_tasks("a", "b")
assert_items_left(2)
“Unit” / “1-feature-per-test” style
def test_delete():
given_at_todomvc("a", "b", "c")
delete("b")
assert_tasks("a", "c")
assert_items_left(2)
def test_edit():
given_at_todomvc("a", "b", "c")
edit("c", "c edited")
assert_tasks("a", "b", "c edited")
assert_items_left(3)
def test_toggle():
#...
#...

34. “Unit”/“Feature”E2E

35. vs
“Unit”/“Feature”E2E
def test_task_life_cycle():
given_at_todomvc()
add("a")
edit("a", "a edited")
toggle("a edited")
filter_active()
assert_no_tasks()
filter_completed()
delete("a edited")
assert_no_tasks()
#...
def test_add():
given_at_todomvc()
add("a", "b")
assert_tasks("a", "b")
assert_items_left(2)
def test_delete():
#...
#...

36. vs
“Unit”/“Feature”E2E
SimpleEasy
def test_task_life_cycle():
given_at_todomvc()
add("a")
edit("a", "a edited")
toggle("a edited")
filter_active()
assert_no_tasks()
filter_completed()
delete("a edited")
assert_no_tasks()
#...
def test_add():
given_at_todomvc()
add("a", "b")
assert_tasks("a", "b")
assert_items_left(2)
def test_delete():
#...
#...

37. Simple is not Easy

38. –Rich Hickey
“Simple Made Easy”

39. “Unit”/“Feature”E2E

40. vs
“Unit”/“Feature”E2E
+ more coverage
+ in less time
+ in case of bugs, gives
more complete report
+ easier to identify reason
from the report
=>
+ less time and efforts in
support
+ with less efforts during
POC implementation
+ integration coverage

41. vs
“Unit”/“Feature”E2E
SimpleEasy
+ more coverage
+ in less time
+ in case of bugs, gives
more complete report
+ easier to identify reason
from the report
=>
+ less time and efforts in
support
+ with less efforts during
POC implementation
+ integration coverage

42. vs
“Unit”/“Feature”E2E
SimpleEasy
+ more coverage
+ in less time
+ in case of bugs, gives
more complete report
+ easier to identify reason
from the report
=>
+ less time and efforts in
support
+ with less efforts during
POC implementation
+ integration coverage

43. vs
“Unit”/“Feature”E2E
SimpleEasy
+ more coverage
+ in less time
+ in case of bugs, gives
more complete report
+ easier to identify reason
from the report
=>
+ less time and efforts in
support
+ with less efforts during
POC implementation
+ integration coverage

44. vs Simple“Pretty”
Reports

45. public class TodoMVCTest {
@Test
public void testTaskLifeCycle(){
givenAtTodoMVC();
add("a");
toggle("a");
filterActive();
assertNoTasks();
filterCompleted();
edit("a", "a edited");
toggle("a edited");
assertNoTasks();
//...
}
@Test
public void testAddTasks(){
givenAtTodoMVC();
add("a", "b", "c");
assertTasks("a", "b", "c");
assertItemsLeft(3);
}
@Test
public void testDeleteTask(){
givenAtTodoMVC("a", "b", "c");
delete("b");
assertTasks("a", "c");
assertItemsLeft(2);
}
//...
}

46. Simple

47. <plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<argLine>
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-$
{aspectj.version}.jar"
</argLine>
<properties>
<property>
<name>listener</name>
<value>ru.yandex.qatools.allure.junit.AllureRunListener</value>
</property>
</properties>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
Allure reporting
<reporting>
<excludeDefaults>true</excludeDefaults>
<plugins>
<plugin>
<groupId>ru.yandex.qatools.allure</groupId>
<artifactId>allure-maven-plugin</artifactId>
<version>2.0</version>
</plugin>
</plugins>
</reporting>

48. @Step
public static void add(String... taskTexts) {
for(String text: taskTexts){
newTodo.setValue(text).pressEnter();
}
}
@Step
public static void filterActive(){
$(By.linkText("Active")).click();
}
//...
Allure reporting

49. “Pretty” with Allure

50. vs
SimplePretty
SimpleMore configs & annotations
Reporting

51. vs
SimplePretty
Enough for “Unit”Good for E2E
Reporting

52. BDD vs xUnit

53. BDD

54. BDD

55. xUnitBDD

56. vs
xUnitBDD
1. Write scenarios with steps
2. Define steps
2.1 Introduce state
3. Map steps
4. Configure Runner
5. Run
6. Get pretty reports
1. Write tests with steps
2. Define steps
3. Run
4. Get reports

57. vs
xUnitBDD
1. Write scenarios with steps
2. Define steps
2.1 Introduce state
3. Map steps
4. Configure Runner
5. Run
6. Get pretty reports
1. Write tests with steps
2. Define steps
3. Run
4. Get reports
still pretty with Allure

58. vs
xUnitBDD
Simple & Easy? :)
1. Write scenarios with steps
2. Define steps
2.1 Introduce state
3. Map steps
4. Configure Runner
5. Run
6. Get pretty reports
1. Write tests with steps
2. Define steps
3. Run
4. Get reports
still pretty with Allure

59. vs
xUnitBDD
Simple & Easy? :)
1. Write scenarios with steps
2. Define steps
2.1 Introduce state
3. Map steps
4. Configure Runner
5. Run
6. Get pretty reports
1. Write tests with steps
2. Define steps
3. Run
4. Get reports
still pretty with Allure

60. vs
xUnitBDD
Less coding,
Full power of PL
No vars & return from steps
1. Write tests with steps
2. Define steps
3. Run
4. Get reports
still pretty with Allure

61. vs
xUnitBDD
Less coding,
Full power of PL
No vars & return from steps
1. Write tests with steps
2. Define steps
3. Run
4. Get reports
still pretty with Allure

62. vs
xUnitBDD
Less coding,
Full power of PL
Monkeys-friendly
1. Write tests with steps
2. Define steps
3. Run
4. Get reports
still pretty with Allure

63. vs
xUnitBDD
Less coding,
Full power of PL
Monkeys-friendly
1. Write tests with steps
2. Define steps
3. Run
4. Get reports
still pretty with Allure

64. PageModules
vs
PageObjects

65. PageModules
vs
ModularOOP
PageObjects

66. from pages import tasks
def test_filter_tasks():
tasks.visit()
tasks.add("a", "b", "c")
tasks.should_be("a", "b", "c")
...
from pages.tasks import TasksPage
def test_filter_tasks():
tasks = TasksPage()
tasks.visit()
tasks.add("a", "b", "c")
tasks.should_be("a", "b", "c")
...
PageModulesPageObjects
ModularOOP

67. class TasksPage:
def __init__(self):
self.tasks = ss("#todo-list>li")
def visit(self):
tools.visit('https://todomvc4tasj.herokuapp.com/')
...
def should_be(self, *task_texts):
self.tasks.filterBy(visible).should_have(exact_texts(*task_texts))
...
PageObjects
OOP

68. #tasks.py
tasks = ss("#todo-list>li")
def visit():
tools.visit('https://todomvc4tasj.herokuapp.com/')
...
def should_be(*task_texts):
tasks.filterBy(visible).should_have(exact_texts(*task_texts))
...
PageModules
Modular

69. PageModules
vs
ModularOOP
PageObjects

70. PageModules
vs
ModularOOP
PageObjects
+ simple
+ easy
+ “newbies” friendly

71. PageModules
vs
ModularOOP
PageObjects
+ ? + simple
+ easy
+ “newbies” friendly

72. PageModules
vs
ModularOOP
PageObjects
+ ? + simple
+ easy
+ “newbies” friendly
+ parallelised tests

73. PageModules
vs
ModularOOP
PageObjects
+ ? + simple
+ easy
+ “newbies” friendly
+ parallelised tests
(where there is no automatic
driver management per
thread)

74. Pages
of Widgets
vs
Plain
Pages

75. Pages
of Widgets
vs
OOP
Modular/
Procedural
Plain
Pages

76. OOP
Pages of “plural” Widgets: Usage
def test_nested_custom_selements():
given_active("a", "b")
main = TodoMVC("#todoapp")
main.tasks.find(text("b")).toggle()
main.clear_completed()
main.tasks.assure(texts("a"))

77. OOP
Pages of “plural” Widgets: Usage
def test_nested_custom_selements():
given_active("a", "b")
main = TodoMVC("#todoapp")
main.tasks.find(text("b")).toggle()
main.clear_completed()
main.tasks.assure(texts("a"))

78. OOP
Pages of “plural” Widgets: Usage
def test_nested_custom_selements():
given_active("a", "b")
main = TodoMVC("#todoapp")
main.tasks.find(text("b")).toggle()
main.clear_completed()
main.tasks.assure(texts("a"))

79. OOP
Pages as Widgets: Usage
def test_nested_custom_selements():
given_active("a", "b")
main = TodoMVC("#todoapp")
main.tasks.find(text("b")).toggle()
main.clear_completed()
main.tasks.assure(texts("a"))

80. Pages (as widgets) of “plural” Widgets: Implementation
OOP
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
def clear_completed(self):
s(“#clear-completed").click()
return self
class Task(SElement):
def toggle(self):
self.s(".toggle").click()
return self

81. Pages (as widgets) of “plural” Widgets: Implementation
OOP
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
def clear_completed(self):
s(“#clear-completed").click()
return self
class Task(SElement):
def toggle(self):
self.s(".toggle").click()
return self

82. Pages (as widgets) of “plural” Widgets: Implementation
OOP
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
def clear_completed(self):
s(“#clear-completed").click()
return self
class Task(SElement):
def toggle(self):
self.s(".toggle").click()
return self

83. Pages (as widgets) of “plural” Widgets: Implementation
OOP
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
def clear_completed(self):
s(“#clear-completed").click()
return self
class Task(SElement):
def toggle(self):
self.s(".toggle").click()
return self

84. Pages (as widgets) of “plural” Widgets: Implementation
OOP
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
def clear_completed(self):
s(“#clear-completed").click()
return self
class Task(SElement):
def toggle(self):
self.s(".toggle").click()
return self

85. Pages (as widgets) of “plural” Widgets: Implementation
OOP
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
def clear_completed(self):
s(“#clear-completed").click()
return self
class Task(SElement):
def toggle(self):
self.s(".toggle").click()
return self

86. Pages (as widgets) of “plural” Widgets: Implementation
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
def clear_completed(self):
s(“#clear-completed").click()
return self
class Task(SElement):
def toggle(self):
self.s(".toggle").click()
return self OOP

87. Plain Pages: Usage
import tasks
def test_nested_custom_selements():
given_active("a", "b")
tasks.toggle("b")
tasks.clear_completed()
tasks.shouldBe("a")
Procedural

88. Plain Pages: Usage
import tasks
def test_nested_custom_selements():
given_active("a", "b")
tasks.toggle("b")
tasks.clear_completed()
tasks.shouldBe("a")
Procedural

89. Plain Pages: Implementation
#tasks.py
tasks = ss("#todo-list>li")
def toggle(task_text):
tasks.findBy(exact_text(task_text).s(".toggle").click()
...
def should_be(*task_texts):
tasks.filterBy(visible).should_have(exact_texts(*task_texts))
...
Procedural

90. OOP
Pages of “singular” Widgets: Usage
def test_nested_custom_selements():
given_active("a", "b")
main = TodoMVC("#todoapp")
main.tasks.find(text("b")).toggle()
main.clear_completed()
main.tasks.assure(texts("a"))
main.footer.assure_items_left(1)

91. OOP
Pages of “singular” Widgets: Usage
def test_nested_custom_selements():
given_active("a", "b")
main = TodoMVC("#todoapp")
main.tasks.find(text("b")).toggle()
main.clear_completed()
main.tasks.assure(texts("a"))
main.footer.assure_items_left(1)

92. OOP
Pages of “singular” Widgets: Usage
def test_nested_custom_selements():
given_active("a", "b")
main = TodoMVC("#todoapp")
main.tasks.find(text("b")).toggle()
main.clear_completed()
main.tasks.assure(texts("a"))
main.footer.assure_items_left(1)

93. OOP
Pages of “singular” Widgets: Implementation
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
self.footer = self.Footer("#footer")
...
class Footer(SElement):
def init(self):
self.clear_completed = self.s("#clear-completed")
def assure_items_left(self, number_of_active_tasks):
self.s("#todo-
count>strong").assure(exact_text(str(number_of_active_tasks)))

94. OOP
Pages of “singular” Widgets: Implementation
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
self.footer = self.Footer("#footer")
...
class Footer(SElement):
def init(self):
self.clear_completed = self.s("#clear-completed")
def assure_items_left(self, number_of_active_tasks):
self.s("#todo-
count>strong").assure(exact_text(str(number_of_active_tasks)))

95. Pages of “singular” Widgets: Implementation
class TodoMVC(SElement):
def init(self):
self.tasks = ss("#todo-list>li").of(self.Task)
self.footer = self.Footer("#footer")
...
class Footer(SElement):
def init(self):
self.clear_completed = self.s("#clear-completed")
def assure_items_left(self, number_of_active_tasks):
self.s("#todo-
count>strong").assure(exact_text(str(number_of_active_tasks)))

96. Plain Pages: Usage
def test_nested_custom_selements():
given_active("a", "b")
tasks.toggle("b")
tasks.clear_completed()
tasks.shouldBe(“a")
footer.assert_items_left(1)
Modular

97. Plain Pages: Implementation
#tasks.py
tasks = ss("#todo-list>li")
def toggle(task_text):
tasks.findBy(exact_text(task_text).s(".toggle").click()
...
Modular
#footer.py
def assure_items_left(self, number_of_active_tasks):
s("#todo-count>strong").assure(exact_text(str(number_of_active_tasks)))
...

98. Plain Pages
vs
Procedural/ModularOOP
Pages with Widgets

99. Plain Pages
vs
Procedural/ModularOOP
Pages with Widgets
+ simple
+ easy
+ “newbies” friendly
+ less code
+ visible

100. Plain Pages
vs
Procedural/ModularOOP
Pages with Widgets
+ ?
+ simple
+ easy
+ “newbies” friendly
+ less code
+ visible

101. Plain Pages
vs
Procedural/ModularOOP
Pages with Widgets
+ ?
+ simple
+ easy
+ “newbies” friendly
+ less code
+ visible
+ for complex UI with many
“plural” widgets

102. Afterwords
There are good practices in context,
but there are no best practices.
(c) Cem Kaner, James Bach

103. Q&A

104. Thank you!
github.com/yashaka
slideshare.net/yashaka
gitter.im/yashaka/better-selenium
youtube.com/c/ItlabsNetUa
@yashakaitlabs.net.ua