Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

End to end testing a web application with Clojure


Published on

Using a event sourcing bank emulator it will be shown how to do end to end testing performance testing a web application.

Published in: Software
  • Be the first to comment

  • Be the first to like this

End to end testing a web application with Clojure

  1. 1. End to end performance testing with Clojure
  2. 2. About me Gerard Klijs ● @GKlijs ● ● Java developer with ● Used Kafka several times ● Author of schema_registry_converter ● Living in Papendrecht
  3. 3. Contents ● Demo ● Test setup ● Used libraries ● Results comparing 0 and 100 ● Questions
  4. 4. Bank simulation with end-to-end performance test
  5. 5. Overview of the whole system Every yellow component will be explained in details, and are all but the command handler only written in Clojure. We use the schema registry for schema management, also making the messages smaller. All messages will have string as key and avro as value type. Meaning of the colors: ● Orange are the Confluent platform parts ● Yellow are the parts of open bank mark ● Green is an NginX instance ● Light blue are PostgreSQL databases
  6. 6. Code example start consuming
  7. 7. Code example producer
  8. 8. Topology Common dependency of the other parts. Several functions: ● One way to set the logging for all components. ● All components have knowledge over the topics and data types without needing to connect. ● Will generate Avro object for (de)serialization. ● Functions wrapping the (Java) Kafka Consumer and Producer. ● Functions for dealing with IBAN and UUID.
  9. 9. Synchronizer Makes sure both the correct topics and schema’s are set. Checks if it’s possible to set the replication factor to what’s in the config, takes the minimum of the available nodes and the config. Note that this has been used only on a clean Kafka cluster, and there is currently no check for topic properties being correct.
  10. 10. Heartbeat Just a simple producer for easy debugging. Used a simple message with just a long value. Exposes and nrepl. A nrepl is a network repl, which can be used to execute code remotely and get the result back. This is a powerful concept, making it possible to apply fixes as the code runs, or interactively solve bugs. With the nrepl the pace of the send messages can be changed.
  11. 11. Command generator Consumes the heartbeats and generates a command for each received heartbeat. This will be ConfirmAccountCreation first, as it runs there will be less of these. It rondomly ceates different kinds of ConfirmMoneyTransfer which might fail because it would cause the balance to become below the limit.
  12. 12. Using seancorfield/next.jdbc for PostgreSQL
  13. 13. Command handler Handles the different kinds of command. ● AccountCreationCommand: generates a new iban, if it not already exists creates a balance using the default values, if it does exists gives back an AccountCreationFailed. ● ConfirmMoneyTransfer: if the supplied token is correct, and there is enough money, makes the transfer. Updates both to and form if they are ‘open-bank’ ibans. And creates a BalanceChanged event for each changed balance.
  14. 14. GraphQL-endpoint
  15. 15. GraphQL-endpoint Exposes a GraphQL endpoint to make it easy to issue commands and get the results back in the frontend. All services have there own consumer, and share the producer and the database. ● Transaction service: makes it possible to query or subscribe to balance changed events. ● Account creation service: used to create an account. Will link the username used to log in with the uuid send for the account creation, in order to get the same iban back should the user log in at another time. ● Money transfer service: tries to transfer money, and provides feedback.
  16. 16. GraphQL subscriptions using Lacinia-pedestal
  17. 17. Frontend
  18. 18. Frontend The frontend is build on several parts that all end up in a NginX container to be served. ● The javascript part is build using clojurescript, an important part is the re- graph library. For clojurescript re-frame is often used, which uses react to update the dom depending on a global state. Clojurescript is using the Google Closure compiler to reduce the size of the resulting javascript. ● Bulma is used for the css with just the colors set differently and some additional animations. ● The output from the tests are added to NginX to make them easily accessible.
  19. 19. Running a test When the test is run it will do several kind of transaction that either increase or lower the money on the balance in such a way as much goes in as goes out after 10 runs. It measures the time till the new balance comes in. During the test the load of the system is increased by using the nrepl of the heartbeat. Increasing the number of heartbeats which in turn will trigger additional commands to be processed. Also during the test using lispyclouds/clj-docker-client both the cpu and memory of parts of the system are measured. Al the data is written into a file so it can be analyzed later on.
  20. 20. Test frontend with etaoin using chrome
  21. 21. Using lispyclouds/clj-docker-client for docker
  22. 22. Output a test The generated files can be compared to other files to generate graphs. All the data is combined, and for each point with the same load some statistics are calculated. Most often the mean and the standard error. For different values graphs are generated in the public folder for the frontend so they can be easily viewed. They are available at the background tab at open-bank.
  23. 23. Using kixi/stats to prepare the data
  24. 24. Using metasoarous/oz to display using vega
  25. 25. Example test, difference 0 vs 100 is an important setting of the kafka producer. Default is 0 for the client, but 100 for kafka streams. Instead of 1 message at a time, multiple are send at a time allowing higher efficiency. Run on 6-core Macbook Pro with for Docker: ● 4 Cpu ● 12 GiB Memory ● 3 GiB Swap
  26. 26. Questions? Code available at open-bank-mark