Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

0

Share

Download to read offline

PAC 2019 virtual Antoine Toulme

Download to read offline

Blockchain testing

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

PAC 2019 virtual Antoine Toulme

  1. 1. Testing distributed decentralized systems by Antoine Toulme, CTO @ Whiteblock
  2. 2. New requirements of distributed, decentralized systems • Pure peer-to-peer networks • Consensus among all nodes is critical • Eventual, asynchronous finality • The chain can reorganize! • Proof of work • Proof of stake • Applications deployed on those systems expose their own performance bottlenecks • Oracles • Bridges • New attack vectors…
  3. 3. New metrics to capture • Transaction size • Sender address • Receiving address • Sent and received timestamps • Block information: number, hash, size • Transaction mempool size • Block time • Block size • Block mempool size • Block transmission time (Difference of send and receive time) • Block header validation time • Block processing time • Total delay • Uncle Rate
  4. 4. Collecting metrics • Logs • Block data • Prometheus • Use data science tools like Jupyter, Grafana, the ELK stack to process. • Each run and client implementation is different
  5. 5. Test suites • Control group • Network testing • Data testing • Compatibility testing • Security testing • Load testing • Consensus testing • More!
  6. 6. Network testing
  7. 7. Control group • The control test will involve a test case where latency, packet loss, and bandwidth will be configured to 0ms, 0%, and 1000 MB*, respectively. The test will run 50 transactions per second to the network in order to determine the transactional throughput of the network. Upon completion of these tests, Whiteblock will export the raw block data of all nodes. The results will be used as a reference point for future tests. • Optimal, peak performance • “Works on my machine feel”
  8. 8. Latency testing • The latency test will involve 3 test cases where latency will be configured and varied from 0ms, 50ms, and 100ms. The test will run 50 transactions per second to the network in order to determine the transactional throughput of the network. Upon completion of these tests, Whiteblock will export the raw block data of all nodes. • We usually see some performance degradation. • Good benchmark as a warm up. • Note nodes peer randomly throughout the network - this means a node in New York may peer with a node in Tokyo.
  9. 9. Bandwidth testing • The bandwidth test will involve 3 test cases where the bandwidth between nodes will be configured and varied from 10 MB, 50 MB, and 100 MB. The test will run 50 transactions per second to the network in order to determine the transactional throughput of the network. Upon completion of these tests, Whiteblock will export the raw block data. • We notice some blockchains are bandwidth heavy and are quite affected by this testing.
  10. 10. Packet loss testing • The packet loss test will involve 3 test cases where the packet loss between nodes will be configured and varied from 0.01%, 0.1%, and 1%. The test will run 50 transactions per second to the network in order to determine the transactional throughput of the network. Upon completion of these tests, Whiteblock will export the raw block data of all nodes. • This exercises the capability of the blockchain to sustain and recover incomplete messages. This causes important issues, having to replay and request data again. • In some scenarios, this may create situations where nodes drop peers.
  11. 11. Network stress testing • The stress test will involve 3 test cases where latency, packet loss, and bandwidth between nodes will be configured to 150ms, 0.01%, and 10MB respectively. The test will run 50 transactions per second to the network in order to determine the transactional throughput of the network. Upon completion of these tests, Whiteblock will export the raw block data of all nodes of all nodes. • Goal is to create something that looks like the real Internet.
  12. 12. Network topology testing • The network topology test will involve 3 test cases where the network topologies for all nodes will be configured in a custom manner. The cases will be a linear peering topology, static peering to X number of nodes, and a full mesh peering topology. The test will run 50 transactions per second to the network in order to determine the transactional throughput of the network. Upon completion of these tests, Whiteblock will export the raw block data. • The block propagation time is the metric to watch here.
  13. 13. Discovery topology testing • The network topology test will involve 3 test cases where the network topologies for all nodes will be configured using a discovery mechanism, against a boot node. The cases will involve 10, 20 and 30 nodes. In each case, the network should be configured to allow a maximum number of peers of 5, 10 and 20. The test will run 50 transactions per second to the network in order to determine the transactional throughput of the network. Upon completion of these tests, Whiteblock will export the raw block data. • Can the network self-organize in an optimized manner?
  14. 14. Data testing
  15. 15. Sync testing • Whiteblock will set up a regular testnet, and identify a node which it will cut from the rest of the network for a given duration. Whiteblock will monitor usage and functional resources of the node while it is cut from the network. Upon reestablishing connections, Whiteblock will check the node is able to reconnect and synchronize with the rest of the network. • How fast is sync?
  16. 16. Title
  17. 17. Smart contract testing • Whiteblock will set up a regular testnet, and deploy the smart contract to it. • An external dedicated helper will then exercise the contract by calling its endpoints and ensuring the results of the interactions matches the expected behavior. • Whiteblock will output the result of the testing as the state of the application after the script has executed. • Exercises the smart contract. • Possibly exercises the infrastructure using the smart contract.
  18. 18. Compatibility testing • Whiteblock will set up nodes of different implementations implementing the same protocol. • Whiteblock will deploy smart contracts and inject the same transactions across all nodes. • It will then compare the output of the result and ensure they all reach the same output. It will also benchmark the time to process the transactions to check the performance of the implementations.
  19. 19. Compatibility testing
  20. 20. Security testing
  21. 21. Eclipse attack testing • The eclipse attack test will be the overtaking of a node’s peerlist with bad acting nodes. This will be done using a static peers list to simulate the connecting to dishonest nodes. The bad acting nodes will be on a fork of the network and will propagate blocks that are false (different than those of the canonical chain). Upon completion of these tests, Whiteblock will export the raw block data. • How do you counter eclipse attacks?
  22. 22. Double spend testing • Whiteblock will set up a regular testnet, and create two competing transactions so that only one transaction may be successful at a time. Whiteblock will then inject both transactions at the same time to two different nodes of the network, and check the target account is only being credited by one transaction. One of the two transactions should be rejected. • The double spend attack is a classic. PoW is meant to ensure these attacks are defeated. • … but not all blockchains use proof of work :)
  23. 23. 51% attack testing • Whiteblock will set up a testnet made of normal nodes and modified nodes that have been instructed to always follow the chain built on top of a block with a specific transaction hash. • The network is exercised with 50 transactions per second, until the attacking transaction is injected. Ideally, the consensus algorithm is able to withstand this attack and reject the transaction. • Ethereum and Bitcoin are vulnerable to 51% attacks.
  24. 24. Load testing
  25. 25. Local load testing • Whiteblock will set up a regular testnet, and select a node to be the recipient of the load. • Whiteblock will inject continuously over a thousand transactions per second to the node for a configurable period of times, in minutes. During the test, Whiteblock will monitor resources usage on the node and check the transaction success rate over the network. • What will make a node fall over?
  26. 26. Distributed load testing • Whiteblock will set up a regular testnet. • Whiteblock will inject continuously over a thousand transactions per second to all nodes for a configurable period of time, in minutes. During the test, Whiteblock will monitor resources usage on nodes and check the transaction success rate over the network. • Good measure of stress of the network under heavy load
  27. 27. Consensus testing
  28. 28. Block time testing • In blockchains, block times may vary depending on implementation. The block time can affect the way a network behaves and a shorter block time will usually lead to a higher orphan (uncle) rate in a proof-of-work network. This is important as a higher orphan (uncle) rate equates to a lower level of security in the network until those orphans (uncles) are pruned and the nodes are synced up to the latest canonical block. This test will focus on measuring block times in the network and upon completion of this test, Whiteblock will export the raw block data.
  29. 29. State transition testing • Whiteblock will initiate one node with an existing, static state. • It will create pre-determined conditions by which the state of the application will mutate, such as a state transition or a new block. • Whiteblock will output the state in a machine-readable format and compare it to an expected output. • Great to debug and check compatibility between clients of a new chain.
  30. 30. Finality testing • Some blockchains support eventual finality. New blockchains based on proof of stake support strong, deterministic finality. Whiteblock will set up a regular testnet and observe block finalization. It will compare the times at which blocks are finalized between clients. When applicable, Whiteblock will create situations where finalization is difficult based on the network topology or the transaction load, and check finalization still operates uniformly across clients.
  31. 31. MOAR testing • Custom testing cases, such as: • Comparisons between blockchains • Network bad actors • Rendez-vous nodes • TCP hole punching tests • UDP-based traffic • Tor and VPN traffic • …
  32. 32. Thank you!

Blockchain testing

Views

Total views

29

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

1

Shares

0

Comments

0

Likes

0

×