Building applications that leverage blockchain data can be cumbersome, but it’s not impossible. Because blockchains don’t typically have RESTful APIs, delivering data to your web app can be complicated. However, we found a way around this challenge by deploying a blockchain node to an EC2 instance. With this approach, we were able to request data and deliver it to our web app using Lambda.
In this presentation, you will learn:
-How we used AWS Lambda and blockchain to build a robust web app
-Best practices for delivering blockchain data to your web app
-Cost-effective ways of serving blockchain data to your application
Learn more in this blog post: https://www.verypossible.com/blog/how-to-serve-blockchain-data-on-the-web
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
How to Serve Blockchain Data with AWS Lambda
1. How to Serve Blockchain Data With AWS Lambda
August 17, 2018
2. 2
Assumptions For Time
•Familiar with Blockchains such as
Bitcoin
•Familiar with AWS
•Familiar with Relational and NoSQL
Databases
3. 3
Don’t worry
There is a blog post
https://verypossible.com/blog/how-to-serve-blockchain-data-on-the-web
4. 4
Block Chains are Different
• Data Retrieval via Remote Procedure Calls (RPC)
• Setup and maintain server
• Different from RESTful APIs and managed databases
5. 5
• Unspent Transaction Output (UTXO) Model
• Bitcoin
• No accounts, senders, receivers, etc. Only transactions
• Transactions are formed of inputs and outputs
• Inputs for a transaction are the output of a previous
transaction
• Account Model
• Ethereum
• Accounts can have their own signature verification algorithm
• Allows for Decentralized Apps (dapps)
Transaction Data Structure
6. 6
UTXO Explained By Example
• Alice needs to pay Bob for renting his car
• Both have public/private encryption key pair for signing transactions
• Alice has unspent Bitcoin from Dan
• Alice has 0.25 Bitcoin
• Alice owes Bob 0.1 Bitcoin
7. 7
• Alice signs her UTXO from Dan to prove ownership
• This is the input to the new transaction
• Bob receives 0.1 Bitcoin for the car rental
• Bob can now use this to pay someone else
• Alice receives 0.15 Bitcoin
• This output gives Alice change for paying Bob
• The whole transaction had to be signed
Transferring the Funds
9. 9
Remote Procedure Calls (RPC)
• Execute code that is not a part of your program
• Requires the application to be running
• Need to maintain a server to retrieve data
10. 10
• Need to download entire chain on first boot
• For our project and chain this took about 30 minutes
• Need to configure indexes
• Each node has to sync
• Need to protect against catastrophic failure
Standing Up the Blockchain
11. 11
What Documentation?
• Bitcoin and Ethereum have good documentation
• Forked chains, like QTUM, have poor documentation
• Will have to search more for correct documentation
12. Investing in Mining Operation with Automated Distributions
Recently we were tasked with building an investment management platform for a
cryptocurrency mining service. Clients would send funds to the investment address
and would receive a portion of mining rewards. The system needed to accept
investments from multiple addresses for one client, show them a balance of their
entire investment, allocate a portion of mining rewards to their account, and graph
the balance growth over time. We needed to initially support a web interface and
later support a native mobile application with wallet integration for easier
investment. As always, the service needed the ability to scale to demand and be
highly available.
TheUseCase
12
13. 13
The Architecture
• Serverless API on Lambda written in Python
• PostgreSQL database managed by RDS
• React SPA Deployed to S3 with CloudFront Distribution
14. 14
• Two EC2 instances
• Load Balancer with Auto Scaling Group
• EBS-backed AMI
Adding the Blockchain
15. 15
Automate Everything!
• Packer to create AMI for Blockchain Nodes
• CloudFormation
• Stacker to setup entire infrastructure
• EC2 instances
• Load Balancer with Auto Scaling Groups
• Database with RDS
• Virtual Private Cloud (VPC) for networking
• Serverless to deploy Lambda Functions
16. 16
• Automated testing using CircleCI
• Deployment executes only for Master
• Deploy Lambda Functions
• Execute Database Migrations
Ship It
18. 18
Lessons Learned
• Lambda is powerful but can be painful in development
• No hot reloading
• Cannot test entire stack locally because of dependency
on API Gateway
• Debugging is difficult
• CloudWatch logs leave much to be desired
• Lambda Cold Starts
19. 19
• Each Engineer deployed their own Lambda functions
• Only functions that were updated were deployed
• Still had issues with only using one database
How We Reacted
20. 20
Show me the Data
• Connect to the node using python-bitcoinrpc
21. 21
Requesting Data
from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException
import json
# rpc_user and rpc_password are set in the bitcoin.conf file
rpc = AuthServiceProxy("http://%s:%s@127.0.0.1:8332"%(rpc_user, rpc_password))
payload = {
'addresses': ['INSERT ADDRESS HERE'],
'start': STARTING BLOCK,
'end': ENDING BLOCK,
}
deltas = rpc.getaddressdeltas(json.dumps(payload))
print(deltas)