Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Integration tests: testcontainers to the rescue

673 views

Published on

Integration testing is hard, and often teams are tempted to do it in production. Testcontainers allows writing meaningful integration tests spawning Docker containers for databases, queue systems, kv-store, other services. Don't mock your database with fake data anymore, work with real data!

Published in: Software
  • Be the first to comment

Integration tests: testcontainers to the rescue

  1. 1. Integration tests: Testcontainers to the rescue Roberto “Frank” Franchini, CTO @robfrankie
  2. 2. whoami(1) https://www.linkedin.com/in/robfrank/
  3. 3. What we do
  4. 4. Same output from different data-sources
  5. 5. One new connector per month
  6. 6. New features every week for every connector
  7. 7. Tests: features x connectors
  8. 8. Deploy without
  9. 9. What we need?
  10. 10. Integration tests
  11. 11. Test Feature B Test Feature A Test Feature C Test Feature D
  12. 12. Nobody likes integration tests
  13. 13. Unit tests Integration tests UI Tests Slower Faster More integration More isolation
  14. 14. Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
  15. 15. Prerequisites
  16. 16. Main features
  17. 17. Temporary database containers: specialized MsSQL Server, MariaDB, MySQL, PostgreSQL, Oracle XE, Virtuoso
  18. 18. PostgreSQL container @ClassRule public static PostgreSQLContainer container = new PostgreSQLContainer(); @Test public void shouldTestSimpleQuery() throws SQLException { Connection conn = DriverManager.getConnection(container.getJdbcUrl(), container.getUsername(), container.getPassword()); Statement stmt = conn.createStatement(); stmt.execute("SELECT 1"); ResultSet resultSet = stmt.getResultSet(); resultSet.next(); assertThat(resultSet.getInt(1)).isEqualTo(1); }
  19. 19. DB’s containers? Isn’t enough?
  20. 20. H2 is fast, BUT doesn’t emulate specific features
  21. 21. Testcontainers is slower, BUT gives 100% db compatibility
  22. 22. Webdriver containers: run a Dockerized Chrome or Firefox browser ready for Selenium/Webdriver operations - complete with automatic video recording
  23. 23. Selenium private val chrome: BrowserWebDriverContainer<Nothing> = BrowserWebDriverContainer<Nothing>().apply { withDesiredCapabilities(DesiredCapabilities.chrome()) withRecordingMode(RECORD_ALL, File("target")) start() }
  24. 24. Generic containers: run any Docker container as a test dependency
  25. 25. Generic container, image based @ClassRule public static GenericContainer container = new GenericContainer("orientdb:2.2.36") .withExposedPorts(2424, 2480) .withEnv("ORIENTDB_ROOT_PASSWORD", "rootpassword") .waitingFor(Wait.forListeningPort());
  26. 26. ROM debian:stretch-slim LABEL maintainer="NGINX Docker Maintainers <docker-maint@nginx.com>" ENV NGINX_VERSION 1.15.5-1~stretch ENV NJS_VERSION 1.15.5.0.2.4-1~stretch RUN set -x && apt-get update && apt-get install --no-install-recommends --no-install-suggests -y gnupg1 apt-transport-https ca-certificates && NGINX_GPGKEY=573BFD6B3D8FBC641079A6ABABF5BD827BD9B F62; found=''; for server in ha.pool.sks-keyservers.net hkp://keyserver.ubuntu.com:80 hkp://p80.pool.sks-keyservers.net:80 version: '2' services: elasticsearch: build: context: elasticsearch/ args: ELK_VERSION: $ELK_VERSION volumes: - ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch .yml:ro ports: - "9200:9200" - "9300:9300" environment: ES_JAVA_OPTS: "-Xmx256m -Xms256m" networks: - elk logstash: build: context: logstash/ args: ELK_VERSION: $ELK_VERSION volumes: - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro - ./logstash/pipeline:/usr/share/logstash/pipeline:ro ports: - "5000:5000" - "9600:9600" environment: LS_JAVA_OPTS: "-Xmx256m -Xms256m" networks: - elk depends_on: - elasticsearch kibana: build: Use a compose Start from a Dockerfile
  27. 27. Generic container from Dockerfile private val container: GenericContainer<Nothing> = GenericContainer<Nothing>( ImageFromDockerfile("robfrank/ngnix") .withFileFromPath("Dockerfile", Paths.get("./src/main/docker/nginx/Dockerfile")) ).apply { withExposedPorts(80) waitingFor(Wait.forListeningPort()) start() followOutput(Slf4jLogConsumer(log)) } @Test fun `should get 200 OK`() { Assertions.assertThat(container.isRunning).isTrue() val url = URL("http://${container.containerIpAddress}:${container.firstMappedPort}") val conn = url.openConnection() as HttpURLConnection Assertions.assertThat(conn.responseCode).isEqualTo(200) }
  28. 28. Use in your CI env
  29. 29. DOOD: Docker outside of Docker
  30. 30. version: '2' services: jenkins: image: arcade/jenkins:latest ports: - 8080:8080 - 50000:50000 privileged: false volumes: - /lib/x86_64-linux-gnu/libc.so.6:/lib/x86_64-linux-gnu/libc.so.6 - /lib/x86_64-linux-gnu/libdl.so.2:/lib/x86_64-linux-gnu/libdl.so.2 - /lib/x86_64-linux-gnu/libpthread.so.0:/lib/x86_64-linux-gnu/libpthread.so.0 - /lib64/ld-linux-x86-64.so.2:/lib64/ld-linux-x86-64.so.2 - /usr/lib/x86_64-linux-gnu/libltdl.so.7:/usr/lib/x86_64-linux-gnu/libltdl.so.7 - /usr/bin/docker:/usr/bin/docker - /var/run/docker.sock:/var/run/docker.sock - /usr/local/bin/docker-compose:/usr/local/bin/docker-compose
  31. 31. /usr/bin/docker /usr/bin/docker
  32. 32. Benefits
  33. 33. NoSQL databases: Redis, Neo4j, Cassandra, Mongo, OrientDB
  34. 34. Kafka, Log aggregations, search engines
  35. 35. Whatever is containerized by your team(s)
  36. 36. Scenarios
  37. 37. Test over different versions of a single database with parametric test
  38. 38. Test a feature over multiple databases
  39. 39. Test your app against a complex env with compose: queue, kv-store, log aggregator, search engine
  40. 40. RDBMS, let’s go
  41. 41. Add to your project <dependency> <groupId>org.testcontainers</groupId> <artifactId>testcontainers</artifactId> <version>[1.9,)</version> <scope>test</scope> </dependency> <dependency> <groupId>org.testcontainers</groupId> <artifactId>postgresql</artifactId> <version>[1.9,)</version> <scope>test</scope> </dependency>
  42. 42. Use in test val container: PostgreSQLContainer<Nothing> init { container = PostgreSQLContainer<Nothing>().apply { start() } } @Test fun `should perform simple query`() { val conn = DriverManager.getConnection(container.jdbcUrl, container.username, container.password) val stmt = conn.createStatement() stmt.execute("SELECT 1") val resultSet = stmt.resultSet resultSet.next() Assertions.assertThat(resultSet.getInt(1)).isEqualTo(1) }
  43. 43. Customization
  44. 44. Jdbc url and init method @Test public void shouldSelectFromBar() throws SQLException { String jdbcUrl = "jdbc:tc:postgresql:9.6.8://hostname/databasename?&TC_INITFUNCTION=io.github.robfrank.testc ontainers.JavaJdbcUrlTest::sampleInitFunction"; Connection conn = DriverManager.getConnection(jdbcUrl); Statement stmt = conn.createStatement(); stmt.execute("SELECT * FROM bar"); ResultSet resultSet = stmt.getResultSet(); resultSet.next(); assertThat(resultSet.getString("foo")).isEqualTo("hello world"); }
  45. 45. Jdbc url and init method public static void sampleInitFunction(Connection connection) throws SQLException { connection.createStatement().execute("CREATE TABLE bar (n" + " foo VARCHAR(255)n" + ");"); connection.createStatement().execute("INSERT INTO bar (foo) VALUES ('hello world');"); connection.createStatement().execute("CREATE TABLE my_counter (n" + " n INTn" + ");"); }
  46. 46. Jdbc url script @Test public void shouldSelectFromBar() throws SQLException { String jdbcUrl = "jdbc:tc:postgresql:9.6.8://hostname/databasename?&TC_INITSCRIPT=initdb.sql"; Connection conn = DriverManager.getConnection(jdbcUrl); Statement stmt = conn.createStatement(); stmt.execute("SELECT * FROM bar"); ResultSet resultSet = stmt.getResultSet(); resultSet.next(); assertThat(resultSet.getString("foo")).isEqualTo("hello world"); }
  47. 47. Custom Dockerfile FROM postgres:9.6.5 COPY ./dump/dvdrental.tar /tmp/dvdrental.tar COPY 1-init.sql /docker-entrypoint-initdb.d/ COPY 2-restore-dump.sh /docker-entrypoint-initdb.d/
  48. 48. Code
  49. 49. https://github.com/robfrank/testcontainers-examples
  50. 50. Thank you!

×