The document discusses Flinkspector, a tool for testing Apache Flink applications, highlighting its need for transparency, traceability, and reproducibility in multichannel retail and financial services. It outlines the methodology for testing, including input specification, data stream acquisition, expectation definition, and execution verification. Finally, it notes the tool's extensibility, potential future integrations, and invites hiring inquiries.

1. Flinkspector
Taming the Squirrel

2. Alexander Kolb
otto group BI
@lofifnc
Alexander Kolb
Otto Group BI
@lofifnc

3. Alexander Kolb - 2016 - otto group
Agenda
• Introduction
• Testing Apache Flink Applications
• Flinkspector
• Concept
• Input
• Expectations
• Execution
• Conclusion
• Outlook

4. Alexander Kolb - 2016 - otto group
Introduction

5. Alexander Kolb - 2016 - otto group
• Multichannel Retail
• Financial Services
• Services
• 30 countries
• 123 companies

6. Alexander Kolb - 2016 - otto group
Problem
• Transparency
• Traceability
• Reproducibility

7. Alexander Kolb - 2016 - otto group
Solutions
• Visualisation
• Datalineage graph / Message tracing
• Unit Testing

8. Alexander Kolb - 2016 - otto group
Testing Apache Flink
Applications

9. Alexander Kolb - 2016 - otto group
Apache Flink Application
final StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<String> text = env.socketTextStream("localhost", 9999);
DataStream<Tuple2<String, Integer>> words =
text.flatMap(new WordCount.LineSplitter());
DataStream<Tuple2<String, Integer>> aWords =
words.filter(new StartsWithAFilter());
DataStream<Tuple2<String, Integer>> counts =
aWords.keyBy(0).timeWindow(Time.of(2, TimeUnit.MINUTES)).sum(1);
counts.print();
env.execute("Wordcount Example");

10. Alexander Kolb - 2016 - otto group
User Defined Functions
public final class StartsWithAFilter
implements FilterFunction<Tuple2<String,Integer>> {
@Override
public boolean filter(Tuple2<String, Integer> t)
throws Exception {
return t.f0.startsWith("a");
}
}

11. Alexander Kolb - 2016 - otto group
User Defined Functions
@Test
public void testAFilter() throws Exception {
StartsWithAFilter filter = new StartsWithAFilter();
Assert.assertEquals(true,
filter.filter(Tuple2.of("aTest",1)));
Assert.assertEquals(false,
filter.filter(Tuple2.of("bTest",1)));
}

12. Alexander Kolb - 2016 - otto group
Stream Transformations
public static DataStream<Tuple2<String,Integer>>
countAWords(DataStream<Tuple2<String,Integer>> aWords) {
return aWords
.keyBy(0)
.timeWindow(Time.of(2, TimeUnit.MINUTES))
.sum(1);
}

13. Alexander Kolb - 2016 - otto group
Stream Transformations
@Test
public void aWordCountTest() {
StreamExecutionEnvironment env =
StreamExecutionEnvironment.createLocalEnvironment();
env.setParallelism(1);
DataStream<Tuple2<String, Integer>> words =
env.fromCollection(Arrays.asList(
Tuple2.of("a1", 1),
Tuple2.of("a2", 1),
Tuple2.of("a1", 1)
));
DataStream<Tuple2<String, Integer>> results =
SocketTextStreamWordCount.countAWords(words);
Iterator<Tuple2<String, Integer>> output =
DataStreamUtils.collect(results);
Assert.assertEquals(output.next(), Tuple2.of("a1", 1));
Assert.assertEquals(output.next(), Tuple2.of("a2", 2));
Assert.assertEquals(output.hasNext(), false);
}

14. Alexander Kolb - 2016 - otto group
Flinkspector

15. Alexander Kolb - 2016 - otto group
Concept
1. Specify input.
2. Aquire data stream from the input specification.
3. Define expectations for the resulting data stream.
4. Apply the expectations to the produced data stream.

16. Alexander Kolb - 2016 - otto group
TestStreamEnvironment
Runtime
Subscriber
Test Source Input
Filter
KeyedWindowReducer
Concept
Verifier
Test Sink Publisher
Trigger
(default)
paired

17. Alexander Kolb - 2016 - otto group
Expectations

18. Alexander Kolb - 2016 - otto group
Input
EventTimeInput<Tuple2<String,Integer>> input =
EventTimeInputBuilder.startWith(Tuple2.of("a1", 1))
.emit(Tuple2.of("a2", 1), after(1, minutes))
.emit(Tuple2.of("a3", 1), before(1, seconds), times(2))
.emit(Tuple3.of(“a4”, 1), intoWindow(4, seconds))
.repeatAll(after(1, minutes), times(3));

19. Alexander Kolb - 2016 - otto group
List Based
ExpectedRecords.create(asList(1,2,3,4));
ExpectedRecords.create(asList(1,2,3,4))
.refine().only();
ExpectedRecords.create(asList(1,2,3,4))
.refine().sameFrequency();
(4,1,2,3,3,5)
(4,1,2,3,3,5)
(4,1,2,3,3,5)

20. Alexander Kolb - 2016 - otto group
List Based
(4,1,2,3,3,5)
(4,1,2,3,3,5)
(4,1,2,3,3,5)
ExpectedRecords.create(asList(1,2,3,4));
ExpectedRecords.create(asList(1,2,3,4))
.refine().only();
ExpectedRecords.create(asList(1,2,3,4))
.refine().sameFrequency();

21. Alexander Kolb - 2016 - otto group
List Based
(4,1,2,3,3,5)
(4,1,2,3,3,5)
(4,1,2,3,3,5)
ExpectedRecords.create(asList(1,2,3,4));
ExpectedRecords.create(asList(1,2,3,4))
.refine().only();
ExpectedRecords.create(asList(1,2,3,4))
.refine().sameFrequency();

22. Alexander Kolb - 2016 - otto group
List Based
(4,1,2,3,3,5)
(4,1,2,3,3)
(4,1,2,3,3,5)
ExpectedRecords.create(asList(1,2,3,4));
ExpectedRecords.create(asList(1,2,3,4))
.refine().only();
ExpectedRecords.create(asList(1,2,3,4))
.refine().sameFrequency();

23. Alexander Kolb - 2016 - otto group
List Based
(4,1,2,3,3,5)
(4,1,2,3,3)
(4,1,2,3,3,5)
ExpectedRecords.create(asList(1,2,3,4));
ExpectedRecords.create(asList(1,2,3,4))
.refine().only();
ExpectedRecords.create(asList(1,2,3,4))
.refine().sameFrequency();

24. Alexander Kolb - 2016 - otto group
List Based
(4,1,2,3,3,5)
(4,1,2,3,3)
(4,1,2,3,5)
ExpectedRecords.create(asList(1,2,3,4));
ExpectedRecords.create(asList(1,2,3,4))
.refine().only();
ExpectedRecords.create(asList(1,2,3,4))
.refine().sameFrequency();

25. Alexander Kolb - 2016 - otto group
List Based
ExpectedRecords.create(asList(1,2,3,4))
.refine().inOrder(notStrict).all();
ExpectedRecords.create(asList(1,2,3,4))
.refine().inOrder(strict).from(1);
List(4,1,2,3,3,5)
List(5,2,3,4,1)

26. Alexander Kolb - 2016 - otto group
List Based
List(4,1,2,3,3,5)
List(5,2,3,4,1)
ExpectedRecords.create(asList(1,2,3,4))
.refine().inOrder(notStrict).all();
ExpectedRecords.create(asList(1,2,3,4))
.refine().inOrder(strict).from(1);

27. Alexander Kolb - 2016 - otto group
List Based
List(1,2,3,3,5,4)
List(5,2,3,4,1)
ExpectedRecords.create(asList(1,2,3,4))
.refine().inOrder(notStrict).all();
ExpectedRecords.create(asList(1,2,3,4))
.refine().inOrder(strict).from(1);

28. Alexander Kolb - 2016 - otto group
List Based
List(1,2,3,3,5,4)
List(5,2,3,4,1)
ExpectedRecords.create(asList(1,2,3,4))
.refine().inOrder(notStrict).all();
ExpectedRecords.create(asList(1,2,3,4))
.refine().inOrder(strict).from(1);

29. Alexander Kolb - 2016 - otto group
List Based
List(1,2,3,4,1)
MatcherBuilder
only
sameFrequency
order
ExpectedRecords.create(asList(1,2,3,4))
.refine()
.only()
.sameFrequency()
.inOrder(strict).from(1);

30. Alexander Kolb - 2016 - otto group
List Based
ExpectedRecords.create(asList(1,2,3,4))
.refine()
.only()
.sameFrequency()
.inOrder(strict).from(1);
List(1,2,3,4,1)
MatcherBuilder
only
sameFrequency
order

31. Alexander Kolb - 2016 - otto group
List Based
List(1,2,3,4,1)
MatcherBuilder
only
sameFrequency
order
ExpectedRecords.create(asList(1,2,3,4))
.refine()
.only()
.sameFrequency()
.inOrder(strict).from(1);

32. Alexander Kolb - 2016 - otto group
List Based
List(1,2,3,4,1)
MatcherBuilder
only
sameFrequency
order
ExpectedRecords.create(asList(1,2,3,4))
.refine()
.only()
.sameFrequency()
.inOrder(strict).from(1);

33. Alexander Kolb - 2016 - otto group
List Based
List(2,3,4,1)
MatcherBuilder
only
sameFrequency
order
ExpectedRecords.create(asList(1,2,3,4))
.refine()
.only()
.sameFrequency()
.inOrder(strict).from(1);

34. Alexander Kolb - 2016 - otto group
List Based
List(2,3,4,1)
MatcherBuilder
only
sameFrequency
order
ExpectedRecords.create(asList(1,2,3,4))
.refine()
.only()
.sameFrequency()
.inOrder(strict).from(1);

35. Alexander Kolb - 2016 - otto group
Assertion Based
new MatchTuples<Tuple2<String,Integer>>("word","count")
.assertThat("word", startsWith("a"))
.assertThat("count", greaterThan(3))
.onEachRecord();

36. Alexander Kolb - 2016 - otto group
Assertion Based
new MatchTuples<Tuple2<String,Integer>>("word","count")
.assertThat("word", startsWith("a"))
.assertThat("count", greaterThan(3))
.oneOfThem()
.onEachRecord();

37. Alexander Kolb - 2016 - otto group
Assertion Based
.assertThat(A,…)
.assertThat(B,…)
.assertThat(C,…)
.eachOfThem()
.onAnyRecord()
A B C

38. Alexander Kolb - 2016 - otto group
Assertion Based
A B C
.assertThat(A,…)
.assertThat(B,…)
.assertThat(C,…)
.eachOfThem()
.onAnyRecord()

39. Alexander Kolb - 2016 - otto group
Assertion Based
.assertThat(A,…)
.assertThat(B,…)
.eachOfThem()
.onAnyRecord()
A B C

40. Alexander Kolb - 2016 - otto group
Assertion Based
.assertThat(A,…)
.assertThat(B,…)
.assertThat(C,…)
.eachOfThem()
.onAnyRecord()
A B C

41. Alexander Kolb - 2016 - otto group
Assertion Based
.assertThat(A,…)
.assertThat(A,…)
.assertThat(A,…)
.atLeastNOfThem(2)
.onEach()
A B C

42. Alexander Kolb - 2016 - otto group
Assertion Based
.assertThat(A,…)
.assertThat(B,…)
.assertThat(C,…)
.atExactlyNOfThem(2)
.onEach()
A B C

43. Alexander Kolb - 2016 - otto group
Assertion Based
public class Each<T> extends UntilCombineMatcher<T> {
public Each(Iterable<Matcher<? super T>> matchers) {
super(matchers);
}
@Override
public String prefix() {
return "each of";
}
@Override
public boolean validWhen(int matches, int possibleMatches) {
return matches == possibleMatches;
}
@Factory
public static <T> Each<T> each(Iterable<Matcher<? super T>> matchers) {
return new Each<T>(matchers);
}
}

44. Alexander Kolb - 2016 - otto group
TestStreamEnvironment
Runtime
Subscriber
Test Source Input
Filter
KeyedWindowReducer
Execution
Verifier
Test Sink Publisher
Trigger
(default)
paired

45. Alexander Kolb - 2016 - otto group
Execution
Test SourceInput
Filter
KeyedWindowReducer
Test Sink Publisher
Test Source Input
Filter
KeyedWindowReducer
Test Sink Publisher
1
1
1
1
2
2
2
2
LocalCluster
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
(default)

46. Alexander Kolb - 2016 - otto group
Execution
LocalCluster
TS I1
F
KWR
TSi P1
1
1
TS I2
2
F
2
KWR
1
1
2
TSi P2
2
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
(default)
Timeout

47. Alexander Kolb - 2016 - otto group
Execution
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
(default)
LocalCluster
TS I1
F
KWR
TSi P1
1
1
TS I2
2
F
2
KWR
1
1
2
TSi P2
2
Timeout

48. Alexander Kolb - 2016 - otto group
Execution
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
(default)
LocalCluster
TS I1
F
KWR
TSi P1
1
1
TS I2
2
F
2
KWR
1
1
2
TSi P2
2
Timeout

49. Alexander Kolb - 2016 - otto group
LocalCluster
TSI1
F
KWR
TSiP1
1
1
TSI2
2
F
2
KWR
1
1
2
TSiP2
2
Execution
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
(default)
Timeout
CLOSE 1
CLOSE 2

50. Alexander Kolb - 2016 - otto group
TSI1
F
KWR
TSiP1
1
1
TSI2
2
F
2
KWR
1
1
2
TSiP2
2
Execution
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
(default)
LocalCluster

51. Alexander Kolb - 2016 - otto group
Execution
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
LocalCluster
TS I1
F
KWR
TSi P1
1
1
TS I2
2
F
2
KWR
1
1
2
TSi P2
2
TimeoutTimeout

52. Alexander Kolb - 2016 - otto group
Execution
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
LocalCluster
TS I1
F
KWR
TSi P1
1
1
TS I2
2
F
2
KWR
1
1
2
TSi P2
2

53. Alexander Kolb - 2016 - otto group
Execution
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
(default)
LocalCluster
TS I1
F
KWR
TSi P1
1
1
TS I2
2
F
2
KWR
1
1
2
TSi P2
2
TimeoutTimeout

54. Alexander Kolb - 2016 - otto group
Execution
TestStreamEnvironment
Runtime
Subscriber
Verifier Trigger
(default)
LocalCluster
TS I1
F
KWR
TSi P1
1
1
TS I2
2
F
2
KWR
1
1
2
TSi P2
2
TimeoutTimeout

55. Alexander Kolb - 2016 - otto group
Conclusion

56. Alexander Kolb - 2016 - otto group
Conclusion
• Small and concise test cases.
• Extensible dedicated runtime.
• Integration into existing test
platforms.
• ProcessingTimeWindows.
• Input specification too
expansive.

57. Alexander Kolb - 2016 - otto group
Outlook

58. Alexander Kolb - 2016 - otto group
Outlook
• Scala (Test) support.
• Symbolic testing integration.
• System tests on the actual cluster.

59. Alexander Kolb - 2016 - otto group
https://github.com/ottogroup/flink-spector

60. Alexander Kolb - 2016 - otto group
ottogroup.com
WE ARE
HIRING!