Adding to your Python Armory - OpenWest 2013

•

4 likes•879 views

The document discusses adding capabilities to Python programs through various techniques like list comprehensions, generators, decorators, and more. It provides an overview and examples of each technique. List comprehensions allow mapping and filtering operations to be condensed to a single line. Generators enable memory-efficient iteration and cooperative multithreading. Decorators can modify functions by wrapping them, for example to count calls, time code, or add type checking. The document encourages downloading additional materials and ends with inviting questions.

Adding to your Python
Armory
Colton Myers
@basepi
basepi.net

• Conﬁguration management
• Remote execution
• Orchestration
• Much, much more
• Easy to set up, easy to use
saltstack.com

Where we’re going
• List Comprehensions
• Generators
• Decorators

Most of the code is stolen.
ﬁles.basepi.net/talks/pyarmory.zip

List Comprehensions
• Shortcut to map/ﬁlter operations
• Condense a whole loop to one line
• Very readable (to a point)

Generators
• Inﬁnite iterators
• Memory-intensive iteration
• Cooperative Multi-threading

# Build and return a list
def firstn(n):
num = 0
nums = []
while num < n:
nums.append(num)
num += 1
return nums
sumofn = sum(firstn(1000000))

Continuations
• “The rest”
• Saves the execution stack, ready to resume
• Continuations deserve their own talk

Decorators
• Count function calls
• Add additional output (debugging!)
• Timing code
• Thread synchronization (locks)
• Type Checking

Questions?
Colton Myers
colton.myers@gmail.com
@basepi
http://basepi.net
http://ﬁles.basepi.net/talks/pyarmory.zip

The JVM memory model describes how threads in the Java eco-system interact through memory. While the memory model impact on developing for the JVM may not be obvious, it is the cause for certain number of "anomalies" that are, well, by design. In this presentation we will explore the aspects of the memory model, including things like reordering of instructions, volatile members, monitors, atomics and JIT.

Data oriented design and c++

Mike Acton

Monitoring your Python with Prometheus (Python Ireland April 2015)

Brian Brazil

Rails development environment talk

Reuven Lerner

An Introduction to the Laravel Framework (AFUP Forum PHP 2014)

daylerees

Systems Monitoring with Prometheus (Devops Ireland April 2015)

Brian Brazil

Monitoring means many things to many people. This talk looks at Systems Monitoring, that is how to keep an eye on a given system and use this as part of overall management of a system. This talk will cover Why one monitors, What to monitor, How to monitor, the general design of a monitoring system and how Prometheus is a good fit for this in terms of instrumentation, consoles, alerts, general system health and sanity. Prometheus is a next-generation monitoring system publicly announced earlier this year, developed by companies including SoundCloud, locals Boxever and Docker. Since launch there has been wide-spread interest, and many community contributions. For more information see http://prometheus.io or http://www.boxever.com/tag/monitoring

Python redis talk

Josiah Carlson

Storm

nathanmarz

Redis Everywhere - Sunshine PHPRicard Clau

Messaging, interoperability and log aggregation - a new framework

Tomas Doran

In this talk, I will talk about why log files are horrible, logging log lines, and more structured performance metrics from large scale production applications as well as building reliable, scaleable and flexible large scale software systems in multiple languages. Why (almost) all log formats are horrible will be explained, and why JSON is a good solution for logging will be discussed, along with a number of message queuing, middleware and network transport technologies, including STOMP, AMQP and ZeroMQ. The Message::Passing framework will be introduced, along with the logstash.net project which the perl code is interoperable with. These are pluggable frameworks in ruby/java/jruby and perl with pre-written sets of inputs, filters and outputs for many many different systems, message formats and transports. They were initially designed to be aggregators and filters of data for logging. However they are flexible enough to be used as part of your messaging middleware, or even as a replacement for centralised message queuing systems. You can have your cake and eat it too - an architecture which is flexible, extensible, scaleable and distributed. Build discrete, loosely coupled components which just pass messages to each other easily. Integrate and interoperate with your existing code and code bases easily, consume from or publish to any existing message queue, logging or performance metrics system you have installed. Simple examples using common input and output classes will be demonstrated using the framework, as will easily adding your own custom filters. A number of common messaging middleware patterns will be shown to be trivial to implement. Some higher level use-cases will also be explored, demonstrating log indexing in ElasticSearch and how to build a responsive platform API using webhooks. Interoperability is also an important goal for messaging middleware. The logstash.net project will be highlighted and we'll discuss crossing the single language barrier, allowing us to have full integration between java, ruby and perl components, and to easily write bindings into libraries we want to reuse in any of those languages.

Python高级编程（二）

Qiangning Hong

Performance and Abstractions

Metosin Oy

Speed geeking-lotusscriptBill Buchan

Operationnal challenges behind Serverless architectures by Laurent Bernaille

The Incredible Automation Day

Integrate LLM in your applications 101

Marco De Nittis

Go from a PHP Perspective

Barry Jones

Why ruby and rails

Reuven Lerner

Move Message Passing Interface Applications to the Next Level

Intel® Software

Integration testing for salt states using aws ec2 container service

SaltStack

A SaltConf16 use case talk by Steven Braverman of Dun & Bradstreet. Testing configuration changes for multiple server roles can be time consuming when real instances or legacy container systems are used. Applying configuration changes to each role in parallel can be difficult. So what's the best way to test configuration changes efficiently, quickly, and securely prior to applying them? See how an integrated test setup using AWS EC2 Container Service (ECS), AWS AutoScaling Group, and SaltStack simplifies the application of configuration changes and allows you to test configuration changes in parallel to reduce the time spent testing.

Spot Trading - A case study in continuous delivery for mission critical finan...

SaltStack

Similar to Adding to your Python Armory - OpenWest 2013

Travis Oliphant "Python for Speed, Scale, and Science"

Fwdays

Php johannesburg meetup - talk 2014 - scaling php in the enterprise

Sarel van der Walt

Jvm memory model

Yoav Avrahami

Data oriented design and c++

Mike Acton

Monitoring your Python with Prometheus (Python Ireland April 2015)

Brian Brazil

Rails development environment talk

Reuven Lerner

An Introduction to the Laravel Framework (AFUP Forum PHP 2014)

daylerees

Systems Monitoring with Prometheus (Devops Ireland April 2015)

Brian Brazil

Python redis talk

Josiah Carlson

Storm

nathanmarz

Redis Everywhere - Sunshine PHPRicard Clau

Messaging, interoperability and log aggregation - a new framework

Tomas Doran

Python高级编程（二）

Qiangning Hong

Performance and Abstractions

Metosin Oy

Speed geeking-lotusscriptBill Buchan

Operationnal challenges behind Serverless architectures by Laurent Bernaille

The Incredible Automation Day

Integrate LLM in your applications 101

Marco De Nittis

Go from a PHP Perspective

Barry Jones

Why ruby and rails

Reuven Lerner

Move Message Passing Interface Applications to the Next Level

Intel® Software

Similar to Adding to your Python Armory - OpenWest 2013 (20)

Travis Oliphant "Python for Speed, Scale, and Science"

Php johannesburg meetup - talk 2014 - scaling php in the enterprise

Jvm memory model

Data oriented design and c++

Monitoring your Python with Prometheus (Python Ireland April 2015)

Rails development environment talk

An Introduction to the Laravel Framework (AFUP Forum PHP 2014)

Systems Monitoring with Prometheus (Devops Ireland April 2015)

Python redis talk

Storm

Redis Everywhere - Sunshine PHP

Messaging, interoperability and log aggregation - a new framework

Python高级编程（二）

Performance and Abstractions

Speed geeking-lotusscript

Operationnal challenges behind Serverless architectures by Laurent Bernaille

Integrate LLM in your applications 101

Go from a PHP Perspective

Why ruby and rails

Move Message Passing Interface Applications to the Next Level

More from SaltStack

Integration testing for salt states using aws ec2 container service

SaltStack

Spot Trading - A case study in continuous delivery for mission critical finan...

SaltStack

SaltConf14 - Craig Sebenik, LinkedIn - SaltStack at Web Scale

SaltStack

This talk will focus on the unique challenges of managing Web scale and an application stack that lives on tens of thousands of servers spread across multiple data centers. Learn more about LinkedIn's unique topology, about the development of an efficient build environment, and hear more about LinkedIn plans for a deployment system based on Salt. Also, all of the software that runs LinkedIn sends a LOT of data. In order to stay ahead of this tidal wave of data, the team must address scale challenges seen in very few environments through efficient use of monitoring and metrics systems. This talk will highlight best practices and user training necessary for the use of SaltStack in large environments.

Salt Air 19 - Intro to SaltStack RAET (reliable asyncronous event transport)

SaltStack

Thomas Hatch, SaltStack CTO, and Sam Smith, SaltStack director of product development, introduce SaltStack RAET as a new alternative transport medium developed specifically with SaltStack infrastructure automation and configuration management in mind. SaltStack built RAET for customers needing substantial speed and scale to automate management of massive data center infrastructure environments. SaltStack RAET is primarily an async communication layer over truly async connections, defaulting to UDP. The SaltStack RAET system uses CurveCP encryption by default. SaltStack users can now leverage substantial flexibility via either Salt SSH, ØMQ, or RAET to best address numerous use cases.

Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...

SaltStack

SaltConf14 - Matthew Williams, Flowroute - Salt Virt for Linux contatiners an...

SaltStack

Configuration Management - Finding the tool to fit your needs

SaltStack

This presentation was originally given by Joseph Hall, SaltStack senior engineer, at the combined Montreal Python and DevOps Montreal meet up on April 14, 2014. Here is the talk abstract: In ye olde days of web, a company might manage a handful of servers, each manually and frequently tuned and re-tuned to the company's needs. Those days are gone. Server farms now dominate, and it is no longer reasonable to manage individual servers by hand. Various configuration management tools have stepped in to help the modern engineer, but which to choose? It is not an easy question, and canned pitches from sales people are unlikely to take into account all of your variables. This talk will attempt to discuss The Big Four objectively, and from what angles they approach the task at hand.

SaltConf14 - Ben Cane - Using SaltStack in High Availability Environments

SaltStack

SaltConf14 - Oz Akan, Rackspace - Deploying OpenStack Marconi with SaltStack

SaltStack

SaltConf14 - Yazz Atlas, HP Cloud - Installing OpenStack using SaltStack

SaltStack

OpenStack is an open source implementation of cloud computing, potentially at very large scale. However, it has many moving parts and is complex to operate. SaltStack appears to provide scalable and secure orchestration for OpenStack. But like all powerful solutions to complex problems, a great deal of the useful know-how has to be discovered by actual practice and hard-won experience. This session will share the inside knowledge gained through practical experience.

SaltConf14 - Anita Kuno, HP & OpenStack - Using SaltStack for event-driven or...

SaltStack

This talk will highlight how the OpenStack Infrastructure team uses SaltStack for event-driven orchestration of its various cloud infrastructure components. The speakers will review the flexibility of Salt in a complex automation environment. Salt plays very well with other tools, including Puppet, which is especially critical in the OpenStack Infrastructure environment which requires the event-driven orchestration functions of Salt to synchronize workflow timing of OpenStack Infrastructure components and events. To learn when and where the next SaltConf will be, subscribe to our newsletter here: http://www.saltstack.com/salt-ink-newsletter or follow us on Twitter: http://www.twitter.com/saltstackinc

SaltConf14 - Saurabh Surana, HP Cloud - Automating operations and support wit...

SaltStack

Using SaltStack to automate enterprise IT operations and support capabilities is not as well documented as the more traditional SaltStack use cases. This session will show how the HP Cloud team runs a secure and reliable SaltStack automation environment by writing Salt states and modules to simplify day-to-day operations and support while extending SaltStack capabilities through dynamic states and modules. The talk will also show how to protect sensitive information and safe guard against user errors.

SaltConf14 - Ryan Lane, Wikimedia - Immediate consistency with Trebuchet Depl...

SaltStack

Deployment is often omitted from configuration management and orchestration, which typically focuses on the infrastructure and not the core application that the infrastructure supports. This presentation will give an overview of a deployment system I wrote using SaltStack to provide immediately consistent deployment for software. The deployment system has the ability for manual deploys or for continuous deployments using a Heroku-like workflow. It has support for multiple deployment schemes, with a default of Git deployment. The deployment system uses a combination of Salt peer runner calls, runners, modules, returners, pillars and grains as well as a deployment host running Apache, Sartoris (a git-deploy frontend), and Redis (for reporting).

SaltConf14 - Forrest Alvarez, Choice Hotels - Salt Formulas and States

SaltStack

This session will expand your knowledge of cutting-edge techniques for creating Salt states and formulas. Users will obtain a thorough understanding of how states interact with pillars, as well as map.jinja files. We'll discuss how to make formulas OS agnostic and show how the usage of external pillars combined with a map file can result in formulas that are easy to explain, easy to learn, and easy to update.

SaltConf14 - Brendan Burns, Google - Management at Google Scale

SaltStack

As a leading developer of highly scalable, large-scale Web services, Google was forced early on to develop systems to support the deployment and management of diverse workloads at an immense scale. As the broader developer community embraces cloud technologies we see significant parallels between the internal management infrastructure which Google has built over the last decade, and open source management technologies of today. This talk will describe Google's experience in managing large-scale compute services, draw parallels to open source efforts underway today, and sketch out how our past experience shapes our future development of the Google Cloud Platform.

SaltConf14 - Justin Carmony, Deseret Digital Media - Teaching Devs About DevOps

SaltStack

Let's set aside the buzzwords for a moment and have an honest discussion about DevOps. There is the idea of putting more Dev into Ops, but just as crucial (if not more crucial) is getting your Devs to think more like Ops. Most developers have little to no experience dealing with production environments, and helping them add value to DevOps efforts can be difficult. This talk will cover practical ways of mentoring Devs into more DevOps skills and responsibilities. Ultimately, the goal is to help your Devs gain the skills leading to better production health, application performance and uptime. Of course, we'll also consider how SaltStack can help.

SaltConf14 - Thomas Jackson, LinkedIn - Safety with Power Tools

SaltStack

As infrastructure scales, simple tasks become increasingly difficult. For large infrastructures to be manageable, we use automation. But automation, like any power tool, comes with its own set of risks and challenges. Automation should be handled like production code, and great care should be exercised with power tools. This talk will cover how SaltStack is used at LinkedIn and offer tips and tricks for automating management with SaltStack at massive scale including a look at LinkedIn-inspired Salt features such as blacklist and prereq states. It will also cover Salt master and minion instrumentation and a compilation of how not to use Salt.

SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...

SaltStack

Google is making the power of its datacenter, network, and technology innovations available to the world through its Cloud services. This presentation will provide an overview of the Google Cloud Platform and a deeper dive on Google Compute Engine. Google recently made an open source contribution to SaltStack and now you can now use Salt Cloud to manage your Compute Engine resources (IaaS virtual machine services). Come find out more about Google's Cloud Platform and how you can leverage Google scale with SaltStack.

SaltStack - An open source software story

SaltStack

Real-time Cloud Management with SaltStack

SaltStack

More from SaltStack (20)