Coding Secure Infrastructure in the Cloud using the PIE framework

2,863 views

Published on

At National Instruments, we have developed an automation and provisioning framework called PIE (Programmable Infrastructure Environment) that we use daily on our devops team. Similar tools are available such as chef or puppet, but what makes PIE unique is its ability to work in multi-cloud deployments (Azure and AWS) along with multiple node OS types (linux and windows). It uses zookeeper to keep state and track dependencies across nodes and services.

When building PIE we actively considered how to implement it in a Rugged way for a DevOps team. As noted in the deck on slide 68, we are Rugged by Design and Devops by Culture. We see these as intersecting domains that have the ability to impact each other. For more info see ruggeddevops.org

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,863
On SlideShare
0
From Embeds
0
Number of Embeds
1,224
Actions
Shares
0
Downloads
19
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Coding Secure Infrastructure in the Cloud using the PIE framework

  1. 1. Coding SecureInfrastructure in the Cloud using the PIE framework Peco Karayanev James Wickett
  2. 2. @wickett• Operations and Security for software delivered on the cloud• National Instruments, R&D• Certs: CISSP, GSEC, GCFW, CCSK• Tags: OWASP, Cloud, DevOps, Ruby• Blogger at theagileadmin.com• I do stuff for LASCON (http://lascon.org)• Twitter: @wickett
  3. 3. @bproverb• Peco Karayanev• OpNet, Senior Applications Engineer• Tags: APM, Java, DevOps, Big Data• Blogger at theagileadmin.com• Twitter: @bproverb
  4. 4. About OPNET Technologies, Inc.Corporate Overview • Founded in 1986 • Publicly traded (NASDAQ: OPNT) • HQ in Bethesda, MD • 600 employees • Worldwide presence through direct offices and channel partnersBest-in-Class Solutions and Services • Application Performance Management • Network Performance ManagementStrong Financial Track Record • Long history of profitability • Annual revenue: $140M • 25% of revenue re-invested in R&D testBroad Customer Base • Corporate Enterprises • Government Agencies • Network Service Providers OPNET Confidential – Not for release to third parties4 © 2011 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc. Rev. 04-01-2011
  5. 5. National Instruments• 30 years old; 5000+ employees around the world, half in Austin, mostly engineers; $873M in 2010• Hardware and software for data acquisition, embedded design, instrument control, and test• LabVIEW is our graphical dataflow programming language used by scientists and engineers in many fields
  6. 6. From toys to black holes
  7. 7. Cloud @ NIWe built a DevOps team to rapidly delivernew SaaS products and product functionalityusing cloud hosting and services (IaaS, PaaS,SaaS) as the platform and operations, usingmodel driven automation, as a keydifferentiating element.With this approach we have deliveredmultiple major products to market quicklywith a very small staffing and financial outlay.
  8. 8. NI’s Cloud Products• LabVIEW Web UI Builder• FPGA Compile Cloud• more to come...
  9. 9. ni.com/uibuilder
  10. 10. FPGA Compile Cloud• LabVIEW FPGA compiles take hours and consume extensive system resources; compilers are getting larger and more complex• Implemented on Amazon - EC2, Java/Linux,C#/.NET/Windows, and LabVIEW FPGA• Also an on premise product, the “Compile Farm”
  11. 11. Our design tools are primitive
  12. 12. Our operation tools are primitive
  13. 13. Our system engineering challenges are greater< year 2000 the modern era
  14. 14. If you want to build a ship, dontdrum up people together to collectwood and dont assign them tasksand work, but rather teach them tolong for the endless immensity ofthe sea- Antoine Jean-Baptiste Marie Roger de Saint Exupéry
  15. 15. The vision
  16. 16. What is Rugged?
  17. 17. Adversity fueled innovation• NASA in Space• Military hard drives• ATMs in Europe
  18. 18. The Internets is Mean• Latency• Distribution• Anonymity• Varied protocols• People
  19. 19. Systems are complex• “How Complex Systems Fail”• Failure at multiple layers• Synonyms in other industries• Defense in Depth
  20. 20. Software needs to meet adversity
  21. 21. Intro to Rugged by analogy
  22. 22. Current Software
  23. 23. Rugged Software
  24. 24. Current Software
  25. 25. Rugged Software
  26. 26. Current Software
  27. 27. Rugged Software
  28. 28. Current Software
  29. 29. Rugged Software
  30. 30. Current Software
  31. 31. Rugged Software
  32. 32. Current Software
  33. 33. Rugged Software
  34. 34. Rugged Software Manifesto
  35. 35. I am rugged... and more importantly,my code is rugged.
  36. 36. I recognize that software has becomea foundation of our modern world.
  37. 37. I recognize the awesomeresponsibility that comes with thisfoundational role.
  38. 38. I recognize that my code will be usedin ways I cannot anticipate, in ways itwas not designed, and for longerthan it was ever intended.
  39. 39. I recognize that my code will beattacked by talented and persistentadversaries who threaten ourphysical, economic, and nationalsecurity.
  40. 40. I recognize these things - and Ichoose to be rugged.
  41. 41. I am rugged because I refuse to be asource of vulnerability or weakness.
  42. 42. I am rugged because I assure mycode will support its mission.
  43. 43. I am rugged because my code canface these challenges and persist inspite of them.
  44. 44. I am rugged, not because it is easy,but because it is necessary... and Iam up for the challenge.
  45. 45. Qualities of Rugged Software• Availability - Speed and performance• Longevity, Long-standing, persistent - Time• Scalable, Portable• Maintainable and Defensible - Topology Map• Resilient in the face of failures• Reliable - Time, Load
  46. 46. Security vs. Rugged• Absence of • Verification of Events quality• Cost • Benefit• Negative • Positive• FUD • Known values• Toxic • Affirming
  47. 47. DevOps
  48. 48. It’s not our problem anymore
  49. 49. DevOps is not justmaking the wall shorterbefore after
  50. 50. -Alex Honor, dev2ops
  51. 51. ProgrammableInfrastructureEnvironment
  52. 52. The PIE vision
  53. 53. People that built PIEPeco Karayanev crazy to dream up PIE and foolish to try to build it Ernest Mueller godfather and proponent of DevOps in PIE James Wickett chief evangelist of PIEMichael Truchard ensuring PIE is as much for dev as it is for ops William Hackett evolving PIE from hackery to legit software.Karthik Gaekwad reminding PIE to KISSKar Meng Chow ensuring PIE is a tool for daily opsMohd Hafiz Ramly boldly taking PIE to new heights John Hill herding the PIE cats
  54. 54. What is PIE?• a a framework to define, provision, monitor, and control cloud-based systems• written in Java, uses SSH as transport, currently supports Amazon AWS (Linux and Windows) and MS Azure• takes an XML-based model from source control and creates a full running system• to define, provision, monitor, and control cloud-based systems
  55. 55. What do we like about PIE? • Collaborative system design and development • Automation for building, provisioning and controlling cloud systems • From source to running multi- tier cloud system in minutes • Infrastructure as code
  56. 56. What do we use PIE for• Provisioning cloud instances• Creating new environments• Deploying & configuring software• Deploying & configuring applications• Backups• Log aggregation• Auto-scaling roles based on demand• Running tests• Continuous integration• Release workflows• Auditing cloud resource usage• Integrating with revision control & software build
  57. 57. can we do better?
  58. 58. Security features and use cases• Increase the visibility and auditability of the system• Track dependencies and what code is deployed where• Diff versions of the system and see changes• Role based auth for operators• No manual steps• Passwords able to be changed with a few PIE commands• Keys out of the model• Ports and user/pass changes• App Config• Audit / Running state vs. model diff
  59. 59. Architecture o Document based Architecture Definition Language o Command orchestration,dispatch and execution engine o Runtime Registry o Command Line Interface
  60. 60. ADL (Architecture Definition Language)• Structural model of the system described in XML documents.• 4 Level Hierarchy - System,SubSystem,Role,Service
  61. 61. ADL (continued)Each entity is defined in XML and holds meta-data aboutresources and commands.
  62. 62. Command Execution & Orchestration• The execution engine can dispatch and execute commands to remote machines.• More complex workflows can be created from simple commands.• Commands can be overloaded in different model components.
  63. 63. Runtime Registry & Name Service • The Runtime registry tracks the state and dependencies • The Name service keeps a namespace for instances.
  64. 64. Command Line InterfaceExample full command line invocation: Target Query string explained:
  65. 65. Demo• On Azure o Provision an environment. o Deploy an app and test. o Update the model and turn off port 80. o Redeploy and test the app. o De-provision the environment.
  66. 66. Rugged DevOps Results• repeatable – no manual errors• reviewable – model in source control• rapid – bring up, install, configure, and test dozens of systems in a morning• resilient – automated reconfiguration to swap servers (throw away infrastructure)• rugged by design devops by culture
  67. 67. Roadmap• Open Source• More security workflows• Azure data management workflows• Add support for external keystores• ADL and model usability features• Port AWS functionality to 2.0 version• New shiny distributed orchestration engine• Add user auth through LDAP or other repos• IDE to simplify design• More powerful script engine
  68. 68. PIE now and future (now) (28 years later)
  69. 69. Recommended Reading
  70. 70. Want the slides?send me a tweet @wickett

×