An Introduction To Palomino

An Introduction To Palomino






Total Views
Slideshare-icon Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    An Introduction To Palomino An Introduction To Palomino Presentation Transcript

    • Laine Campbell,CEOJay Edwards, CTOCharlie Killian, VP of Professional ServicesKevin Bowman, VP of OperationsAn Introduction to Palomino
    • Who we arePalomino staff have been managing large production systems since before the dot com era began.Technorati, Twitter, Mozilla Firefox, Digg, Travelocity,Friendster, Chegg, Obama for America, Activision and StumbleUpon are only part of our pedigree. Wevescaled companies through years of explosive growth. *
    • Our Partners Palomino builds partnerships that make sense for our clients and staff, allowing us to augment our expertise.● Amazon Web Services● SkySQL● MariaDB● Basho● Couchbase● 10Gen We also build key relationships with products we believe are useful and appropriate to our clients, such as Scalearc, Continuent and TokuTek. Palomino never resells, or takes commission on any products they recommend. *
    • Professional Services● Choosing your DBMS● High availability and disaster recovery● Search, caching, queuing and security● BI, ETL and real-time analytics● Large distributed database systems (aka Big Data)● Configuration management via Chef, Puppet and Ansible● Scripting and automation● Cloud Solutions, AWS and Joyent● Operational tools development and implementation● Backup, recovery and data management *
    • Operational Support● Tier 1 Oncall, integration with customer monitoring● Tier 2 Escalation and problem solving.● Team integration, including email, chat room and IM collaboration, as well as regular stand-ups and project mgmt meetings.● Proactive health checks, recovery tests, capacity and security reviews.● Project level work as needed.● Ownership of release management and configuration management. ○ Query Reviews. ○ DDL and DML Reviews ○ Safe production change management● Ongoing improvements of incident and problem management. *
    • Remote Engineering Services● Pre-deployment● Data modeling● Data migration● Cluster sizing● Load testing and QA● Application profiling● Performance tuning *
    • How we can help● 24x7 operational support -● MySQL revamping and support - Echosign● Operations team needed - SoHalo● Verifying systems will scale under load - Contest Factory● Lousy application performance - Zeitbyte● Tools development - Zendesk and Slideshare● Proactive services - SendGrid● Release management and tool development - Zendesk● DBMS tuning and performance - Slideshare and Houzz● Search solutions - Zenlok and Discover Books *
    • 24x7 operational support needed -● 48 MySQL Nodes in 10 separate clusters● Tier 1 support for web, search, application, middleware and RDBMS layers● Tier 1 and 2 support for MySQL, Cassandra and MongoDB● 24x7 operational support with 30 minutes SLA● Amazon EC2 tooling and architectural management● Manage backup and recovery, database release, incident and configuration management processes.● Onsite support during Amazon EC2 region migrations● Advanced DBMS tuning and management● Upgrades of tooling and diagnostics for alerting and trending *
    • MySQL revamping and support - Echosign● Monolithic MySQL with MMM to start.● New version upgrades.● New hardware and capacity improvements.● Functional partitioning and data management.● Migration to Tungsten for HA and DR.● BI/ETL to reduce DB load, support business needs and reduce load times for key processes.● Aggregation and denormalization for performance improvements. *
    • Operations team needed - SoHalo● Startup with no operations team● Provided DBA and techops resources● Architected environment● Using Amazon MySQL RDS as datastore● Fulfilled the entire operations function● Running both their US and EU sites *
    • Verifying systems will scale under load - Contest Factory ● Configuration review ● Hardware and O/S review ● Metrics and logs review ● AWS architecture review ● Implementation of recommended configuration changes *
    • Tools development - SlideShare and Zendesk● Replication aware rolling DDL implementations● Shard migration and monitoring utilities● Chef recipes for configuration management of configurations● MHA and HAProxy for availability● Cacti for MySQL Trending *
    • Lousy application performance - Zeitbyte● Customer blamed the DBMS (MongoDB)● Configuration review● Application profiling (Django)● Identify problem as code, not DBMS● Fix code● Integrate with ongoing monthly support post - tuning ○ New backups ○ New HA *
    • Proactive services - SendGrid● System health checks● Backup and recovery● Upgrades to monitoring and trending.● Chef recipes and support.● Query reviews, indexing, benchmarking● Upgrades● Support of Sharding and NoSQL evaluation ○ Sharding for Short URLs. ○ HBase for Counters *
    • Release management and tool development - Zendesk● Own the weekly migration process● Identify and manage risk, push changes throughout sharded infrastructure● Publish views to production● Built shard rebalancing utilities● Manage regular shard rebalancing and capacity planning● Slow data deleter tool *
    • DBMS tuning - Slideshare and Houzz● Improved slideshare site load times by factor of 3● Benchmarked InnoDB/XtraDB conversions for both clients● Tuned InnoDB and XtraDB to manage background IO, concurrency issues and optimal memory management● Eliminated locking and replication lag for *
    • Search solutions - Zenlok and Discover Books● Installed, configured and maintain Solr and Sphinx clusters● Gathered search requirements● Created and optimized index and delta queries● Created custom ranking formulas● Improved morphology and relevance● Reviewed and improve indexing speed by over 10x● Reduced query response over 4x by splitting and distributing indexes across the cluster● Configure HA and failover *
    • Engagement process● Palomino is a boutique offering bespoke solutions for our customers. How we interact is thus hard to predict.● We make ourselves available via email, IM, phone and chat.● Your primary DBA can spend their working hours available in your chat rooms.● Daily standups for busy times, and weekly project mgmt meetings are all part of the service.● How else can we help you? *
    • Who we are working with *
    • Zendesk & Palomino“Palomino got up to speed very quickly. Palomino works successfully on complex [data migration] issues that have stumped other DBAs.” *
    • SoHalo & Palomino “Palomino has been helping us build our Development andProduction systems from the beginning. Early on, they built the dev servers, set up our release management system, migrated us to MySQL and Couchbase, and architected our Amazon WebServices cluster. Now, day to day, they help us run both our US and EU sites, and continue to support us in projects, such as a recent German/EU Data Privacy compliance effort. We couldnt have done it all without Palomino.” *
    • Slideshare & Palomino “Palomino was instrumental in helping SlideSharedevelop an architecture that supported arbitrary schema updates without downtime.” *
    • Rent the Runway & Palomino“Palomino has shown persistence, talent and a greatbreadth of experience as they continue to bring us to new levels of stability and performance.” *
    • Technorati & Palomino “Palomino helped us to rethink our database architecture and to automate processes that wereformally manual, if done it all. Their deep experience indesigning large database systems for scale and stability and their ability to work side-by-side with operations staff and technology management, made them a huge asset.” *
    • Chegg & Palomino “Palomino are hands-down the best DBAs I haveworked with. Their knowledge, experience, and tenacity make a huge difference. I would highly recommend them.” *
    • WorldReader & Palomino Palomino’s partnership with WorldReader has allowedus to focus on our core efforts. Finding talented, reliable partners is crucial for a non-profit, and we are quite grateful for Palomino. *
    • Palomino LeadershipLaine Campbell Jay Edwards Charlie KillianChief Executive Chief Technical VP, Professional Officer Officer Services *
    • Palomino Leadership Kevin Bowman Kaichi Sung Tim EllisVP of Operations Account and Project Principal Big Data Management Lead Architect *
    • Laine is the Owner/CEO and Principal Consultant at Palomino. She is a proven database expert with a history of scaling high-performance and high-availability web applications.Laine Campbell Laine leverages extensiveOwner/CEO operational experience and best practices with extensive technical knowledge, troubleshooting skills and a passion for results to ease you through the life-cycles of your company. *
    • Charlie has created the back-bone technology for multiple million dollar companies. An expert in web applications, Charlie has built web application frameworks, setup and integrated numerous disparate systems, built long term client relationships, and contributed to open-Charlie Killian source projects.VP of Professional Services Key to Charlies success has been a love of cyberspace, hardware, software, an unwavering patience and an unstoppable urge to figure out how things work. *
    • Jay was the first dedicated DBA at Twitter. He later served as the lead database engineer at Obama for America during the 2012 election. In his role as CTO for Palomino, heJay Edwards focuses on technology strategy,Chief Technology Officer evangelism and mentoring/training staff and customers. *
    • Kevin began his career at Garmin offering desktop and network/server support and was soon serving as IT Manager, supervising areas including service desk, Internet application development, systems administration, network engineering, DBA, architecture and devops. He also worked on delivering internet-enabledKevin BowmanVP of Operations products. He contributed significantly on application and system architecture and devops for the backend systems that provide data services to products as well as for Garmin Connect. *
    • Kaichi occupies the dual role of Account Manager and Scrum Master at Palomino. During her time in Biotech, she focused on business analysis and project mgmt, supporting process and product improvements.Kaichi Sung She rolled out Agile Scrum and isAccount Manager and certified as both a Scrum Master andScrum Master Product Owner. A multi tasker and planner, she loves coordinating projects and helping teams deliver on customer requirements. *
    • Tim has been working with database clusters that serve billions of database operations daily since the year 2000 and has been involved in the NoSQL community from the earliest stages. He has a broad background in relational database systems. He was aTim Ellis core operations component to the DiggPrincipal Big Data Architect NoSQL rollout, and an active, participating operations member of the NoSQL community at StumbleUpon, Mozilla, and Riot Games. *
    • Contact Info Laine Campbell, Charlie Killian, Jay Edwards, jedwards@palominodb.comKevin Bowman, Kaichi Sung, Tim Ellis, @Palominodb on Twitter *