I was hired by a client to bring their new SaaS offering to production.
At this time they had no operations guy on staff to run it.
While working on the project, I asked the CTO and myself if it is possible to deploy methodologies and infrastructure that would enable this company to support their production without a designated ops guy.
This is a story of our journey.
Presented in the Continuous Delivery track at DevOps Con Israel 2013
Taking a SaaS Shop from No Ops to NoOps, Pavel Suchman
No ops to NoOpsA story of a journey...
About megithub.com/flypunk● In IT since 99, Unix SysAdmin, OPS@XIV, cloud/devops since 07 (@SeekingAlpha)● Cloud migration and deployment automation in AmadesaI am a freelancer specializing in WebApplications Delivery
WTF is NoOps?!Not an easy thing to describe - seeNoOps Debate Grows Heated@AppFog Carlson - Use 3rd party PaaS@Netflix Cockroft - Outsource messing withhardware to AWS and build your own PaaS@Etsy Allspaw - We do it the same way asNetflix, but I HATE the term, especially theword NO in it ;)
What do I mean by NoOpsCulture, software architecture,methodologies and tools that enable a(SaaS) company run their business without adedicated Operations team.
NoOps - Eliminating WasteWaste - Any activity not contributing tocustomer value (Agile)While you have No Nops you have...No DelaysNo Communications overheadNo Dev vs. Ops fingers pointing - Dev isalways to blame ;)
My goal as a (NoOps) consultantStudy what is neededBuild what is missingGet the hell out!If I did my job well, the company shouldfigure out production/operations on its own.
The client - MassiveImpactToday - www.massiveimpact.comEnd to end solution for ads on cellularConversions, not clicks
The company pivoted from adifferent productHired new peopleWritten new codeHad to start a new production site
MassiveImpact technology stack:.Net on the front endBig data (hive) on backend
Who is responsible for ops in thenew environment?The head of QA with some Ops experienceI need to ensure that he and his bossescould sleep well at night...Is it possible for MassiveImpact to continuegrowing without hiring a dedicated Opsemployee?I think - yes!
Requirements for the newenvironmentAmazon VPC: Harder to set up, moresecure, closer to collocated network setupLets make everything organized: Chef,Cloud FormationBtw, we have 2 weeks to launch!Start manually, add automation later
MonitoringServerDensity for both machine and servicemonitoring● Has built in alerts and graphs● Machines have to be able to connect to the internet to use it● Used the NAT-PAT instance inside VPC + static IPs to achieve it.NoOps advantage: A cloud solution.
AWS IAM credentialsWritten a naming/access policy docCreated IAM users, groups, policies andresources in accordance with the doc.Pro tip: dont use MixedCaseBucketNames -hive wont work with them.Another tip: EMR with IAM keys and policiesis not trivial
DeploymentSet up a server that will work as binariesrepositoryPut artifacts into sane default placesWill be used later for CI and deploymentscripts
Chef on WindowsIt (kinda;) works!You can bootstrap and manage nodes withWinRM!NoOps advantages:Configuration as codeNo need to manage AMIs (Its a nightmare)
Chef on Windows - continueNot everything worksFirst time in my career able to manage M$servers and install packages from CLI.== Awesome!!!Easy way to manage users in centralizedmanner without AD
Data crunching - Gurobi CloudThe system is fully automatic and is run byCronNotifications using SNS and emailNoOps advantage: It is fully data driven(Just put the new files to a specified S3bucket)
Current statusMonitoring - ImplementedChef - in developmentCloud Formation - plannedCI/CD - plannedOps/business console (GeckoBoard) - planned
TakeawaysPeople, then methodologies, last toolsArchitect for no ops!Change is hard, but engineers will acceptnew tech when shown that it worksMake the changes backwards compatible(Libraries for AWS calls)Use external services as a default choice