This document describes Medicost Guider, a tool that allows users to compare medical procedure prices across locations. It stores over 2 billion synthetic medical cost records in Amazon Redshift for fast query processing. The author's goals are to help users make better financial healthcare decisions by providing location-specific price ranges. The document discusses Redshift configuration details like columnar storage, distribution keys, sort keys, and compression codes which were optimized to improve query performance and reduce latency. Sample queries are also presented.

1. Medicost Guider
Parimala Killada
Insight Data Engineering Fellowship, New York

2. My motivation
• Helping users to make better medical financial
decisions
• Users can compare their price ranges within their
geographic area.
• This same business case can be used to see price
ranges/ discount sales for several retail markets.

3. Data Creation Process

4. Queries On Data
• Location wise prices for user selected procedures
• Procedure wise prices across locations
• Year wise price comparison
• How much does a certain procedure cost in a particular
city?

5. Pipeline

6. Challenges
• Creating synthetic data as near to real data ~2 billion
Records
• Processing queries in low latency
MedicostGuider

7. Redshift
• Data stored in columnar format
• Distribution keys: Used to distribute data among nodes
• Sort keys: Data sorted within a distribution key
• Compression codes: Reduces storage space
• Shared-nothing Architecture: Queries are run on each node
using own set of data

8. Sample Query

9. Challenges
Slow Redshift Queries :
Inefficient Zone maps due to absence of sort keys

10. Challenges

11. Challenges
After Changing the Sort Keys

12. About me
• BS and MS in Computer Science.
• Worked as Software Engineering analyst for 2.5
years optimizing Queries
• Loves to Cook, Paint and Travel.

13. Configuration of Redshift Cluster
Node Type dc1.large
Nodes 4
CPU 7 EC2 Compute Units (2 virtual cores) per node
Memory 15 GB Per Node
Storage 160GB SSD