The talk "How to stop worrying and start a project with Python 3 " based on my production experience of using the technology. Typical fears of the engineers which use Python 2 are addressed.
WebCamp Ukraine 2016: Instant messenger with Python. Back-end developmentViach Kakovskyi
This document discusses building instant messaging platforms with Python. It covers common messaging protocols like XMPP and WebSocket, how they establish and send messages. It also discusses the life of a messaging platform, including authentication, delivery, parsing, and more. Lessons learned include handling bursty traffic, reconnect storms, and preventing incidents. Python is well-suited for messaging backends but other languages may be better for some tasks.
PyCon Poland 2016: Maintaining a high load Python project: typical mistakesViach Kakovskyi
The talk is about typical mistakes which a Python developer without much experience in high load systems can make. Possible issues and preventive actions will be discussed. Expected audience: developers who are new to an existing highly loaded service or folks who develop a system from scratch. All the stuff based on own production experience.
Austin Python Meetup 2017: What's New in Pythons 3.5 and 3.6?Viach Kakovskyi
In the talk, the changes introduced in Python 3.5 and Python 3.6 are covered. Primary based on standard documentation, but only my favourite ones are included:
* https://docs.python.org/3/whatsnew/3.3.html
* https://docs.python.org/3/whatsnew/3.6.html
This document discusses strategies for maintaining legacy Python codebases. It begins with background on the presenter and an introduction to the topic. The bulk of the document outlines a three-part "maintenance framework": 1) Understand the system through documentation, debugging, profiling and testing; 2) Establish a safety net with automated tests, continuous integration, monitoring and automation; and 3) Improve the code and development practices through code quality tools, test-driven development, and code reviews. Emphasis is placed on understanding context, architecture and behavior before refactoring code and establishing collaborative processes to raise quality and knowledge sharing.
This document discusses Uber's transition from a monolithic architecture to a microservices architecture and the adoption of Go as a primary programming language. It provides examples of some key Go services at Uber including Geofences, an early service, and Geobase, a more recent service. It also discusses Uber's development of open source Go libraries and tools like Ringpop, TChannel, go-torch, and others to help establish Go as a first-class language at Uber.
kikstart journey of Golang with Hello world - Gopherlabs sangam biradar
This document summarizes key concepts in Go programming including packages, functions, parameters vs arguments, and more. It discusses how every Go file begins with a package name, and the "main" package is the entry point for a program. Functions need to be capitalized to be accessible outside a package. It also provides review questions and references for further reading on Go.
The document provides an overview of functions in Go including function definitions, parameters and arguments, returning values, func expressions, closures, callbacks, recursion, and defer statements. It begins with basic concepts like defining functions and calling them, then covers more advanced topics like func expressions where a function is assigned to a variable, closures which allow functions to access variables from the enclosing scope, callbacks where functions are passed as arguments to other functions, and recursion where functions call themselves. Examples are provided for each concept using Go playground links. The document aims to explain how functions work and behave as first-class citizens in Go, providing a hands-on tutorial of key function-related ideas.
WebCamp Ukraine 2016: Instant messenger with Python. Back-end developmentViach Kakovskyi
This document discusses building instant messaging platforms with Python. It covers common messaging protocols like XMPP and WebSocket, how they establish and send messages. It also discusses the life of a messaging platform, including authentication, delivery, parsing, and more. Lessons learned include handling bursty traffic, reconnect storms, and preventing incidents. Python is well-suited for messaging backends but other languages may be better for some tasks.
PyCon Poland 2016: Maintaining a high load Python project: typical mistakesViach Kakovskyi
The talk is about typical mistakes which a Python developer without much experience in high load systems can make. Possible issues and preventive actions will be discussed. Expected audience: developers who are new to an existing highly loaded service or folks who develop a system from scratch. All the stuff based on own production experience.
Austin Python Meetup 2017: What's New in Pythons 3.5 and 3.6?Viach Kakovskyi
In the talk, the changes introduced in Python 3.5 and Python 3.6 are covered. Primary based on standard documentation, but only my favourite ones are included:
* https://docs.python.org/3/whatsnew/3.3.html
* https://docs.python.org/3/whatsnew/3.6.html
This document discusses strategies for maintaining legacy Python codebases. It begins with background on the presenter and an introduction to the topic. The bulk of the document outlines a three-part "maintenance framework": 1) Understand the system through documentation, debugging, profiling and testing; 2) Establish a safety net with automated tests, continuous integration, monitoring and automation; and 3) Improve the code and development practices through code quality tools, test-driven development, and code reviews. Emphasis is placed on understanding context, architecture and behavior before refactoring code and establishing collaborative processes to raise quality and knowledge sharing.
This document discusses Uber's transition from a monolithic architecture to a microservices architecture and the adoption of Go as a primary programming language. It provides examples of some key Go services at Uber including Geofences, an early service, and Geobase, a more recent service. It also discusses Uber's development of open source Go libraries and tools like Ringpop, TChannel, go-torch, and others to help establish Go as a first-class language at Uber.
kikstart journey of Golang with Hello world - Gopherlabs sangam biradar
This document summarizes key concepts in Go programming including packages, functions, parameters vs arguments, and more. It discusses how every Go file begins with a package name, and the "main" package is the entry point for a program. Functions need to be capitalized to be accessible outside a package. It also provides review questions and references for further reading on Go.
The document provides an overview of functions in Go including function definitions, parameters and arguments, returning values, func expressions, closures, callbacks, recursion, and defer statements. It begins with basic concepts like defining functions and calling them, then covers more advanced topics like func expressions where a function is assigned to a variable, closures which allow functions to access variables from the enclosing scope, callbacks where functions are passed as arguments to other functions, and recursion where functions call themselves. Examples are provided for each concept using Go playground links. The document aims to explain how functions work and behave as first-class citizens in Go, providing a hands-on tutorial of key function-related ideas.
Boosting command line experience with python and awkKirill Pavlov
It is often required to manipulate the data as fast as possible, be it either column average calculation or simple join and filtering.
Servers often do not have convenient tools, such as Python (NumPy/Pandas) of R, moreover, the data might not fit into memory.
This talk shows how to make fast but inconvenient command line tools great again.
GoLang is an open source programming language created by Google in 2009. It has a large community and was designed for scalability and concurrency. Some key features include being statically typed, compiled, and having built-in support for concurrency through goroutines and channels. Google uses GoLang extensively to build systems that scale to thousands of machines.
gRPC can help minimize the barrier of cross-system communication by providing language-agnostic API definitions, backward and forward compatible versioning with protocol buffers, and pluggable load balancing and tracing. You will see how to quickly get up and running with the gRPC framework using Node.js from creating a protocol definition, creating meaningful health checks, and securing the endpoint. Additionally, this session will go over best practices and how to take full advantage of what gRPC has to offer.
Presentation at NetPonto community: "We’re going to discuss gRPC, Google’s open-source RPC framework. I’ll dive a bit into the history of RPC as a protocol, and what its historical use has been. I’ll also highlight some benefits to adopt gRPC and how its possible to swap out parts of gRPC and still take advantage of gRPC’s benefits. Finally I’ll answer the question that has been on many lips since gRPC was announced — what does this mean for REST?"
Pig is a useful tool for exploring data sets and describing data flows easily through scripts. However, it has some weaknesses like inconsistent syntax, lack of testing support, and an aging code base. While Pig works well for many companies, its suitability may decline as companies' data needs grow more complex with distributed teams of data scientists and engineers. For Pig to remain useful, it will need improvements in areas like types, performance, UDF support, and testing to better support "big data" teams in the future.
The document discusses gRPC, an open-source framework for building distributed applications and services. It was motivated by Google's experience with internal RPC systems handling over 1010 RPCs per second. gRPC uses HTTP/2 for transport, allowing it to take advantage of existing network infrastructure while enabling features like bidirectional streaming. It generates client and server code from IDL definitions, supporting multiple languages.
Robert Kubis - gRPC - boilerplate to high-performance scalable APIs - code.t...AboutYouGmbH
The document discusses gRPC, an open source framework for building microservices. It was created by Google to be a high performance, open source universal RPC framework. gRPC uses HTTP/2 for transport, Protocol Buffers as the interface definition language, and generated client/server code for many languages to make cross-platform communications simple and efficient. The document provides an overview of gRPC's goals and architecture, how to define a service using .proto files, and examples of common RPC patterns like unary, streaming, and bidirectional calls.
Bringing Learnings from Googley Microservices with gRPC - Varun Talwar, GoogleAmbassador Labs
Varun Talwar, product manager on Google's gRPC project discusses the fundamentals and specs of gRPC inside of a Google-scale microservices architecture.
Design and Implementation of the Security Graph LanguageAsankhaya Sharma
Today software is built in fundamentally different
ways from how it was a decade ago. It is increasingly common
for applications to be assembled out of open-source components,
resulting in the use of large amounts of third-party code. This
third-party code is a means for vulnerabilities to make their
way downstream into applications. Recent vulnerabilities such
as Heartbleed, FREAK SSL/TLS, GHOST, and the Equifax data
breach (due to a flaw in Apache Struts) were ultimately caused
by third-party components. We argue that an automated way to
audit the open-source ecosystem, catalog existing vulnerabilities,
and discover new flaws is essential to using open-source safely.
To this end, we describe the Security Graph Language (SGL), a
domain-specific language for analysing graph-structured datasets
of open-source code and cataloguing vulnerabilities. SGL allows
users to express complex queries on relations between libraries
and vulnerabilities in the style of a program analysis language.
SGL queries double as an executable representation for vulnerabilities, allowing vulnerabilities to be automatically checked
against a database and deduplicated using a canonical representation. We outline a novel optimisation for SGL queries based on
regular path query containment, improving query performance up to 3 orders of magnitude. We also demonstrate the
effectiveness of SGL in practice to find zero-day vulnerabilities
by identifying sever
Per aspera ad grid To the grid computing through difficultiesFerenc Szalai
The document discusses the history and evolution of grid computing from the perspective of the author. It covers early work using clusters for computationally intensive tasks in the late 1990s, the development of grid middleware like Globus to schedule tasks on heterogeneous resources, and issues with standardization and interoperability between different grid projects. It then outlines the shift towards more modular, service-oriented architectures exemplified by the ARC middleware, as well as changes brought by volunteer desktop grids, MapReduce, and the rise of cloud computing where resources are virtualized and provided as a service.
The document summarizes recent updates in PHP, frameworks, CMSes, and upcoming conferences. Key points include the release of PHP versions 7.0.12 and 5.6.27 with security fixes. For frameworks, CakePHP released versions 3.3.6 and 3.3.7, and Laravel released versions 5.3.16 through 5.3.21. Drupal released version 8.2.0 with new features. Upcoming conferences highlighted include PHPWorld in November and ZendCon in November.
The summary provides an overview of the key topics from the November 2016 PHP newsletter:
1. PHP released security patches for versions 7.0.13, 5.6.28 and the last release candidate of 7.1 which will be finalized in December.
2. Upcoming RFC proposals discussed security classification of vulnerabilities, adding debugging functions to PDO prepared statements, and deprecating image conversion functions.
3. Popular frameworks like Drupal, WordPress, CakePHP and Laravel released updates with bug fixes and new features.
4. Information was provided on upcoming PHP conferences in early 2017 and a reminder for speakers for events later in the year. The next local user group meeting will
"Building, deploying and running production code at Dropbox" Васильев Леонид,...it-people
The document discusses Dropbox's approach to building, deploying, and running production code. For building, they migrated to Bazel for its reproducibility, hermetic builds, and ability to track dependencies across languages. For deploying, they use YAPS to package dependencies and services into read-only squashfs images. For running services, they use Dbxinit for process management and enforce resource limits, while also integrating containers via RunC. Overall their goal is robust and automated systems from building to deployment to operations.
The document discusses the status and plans for XMLHttpRequest. It notes that the specification reached TR publication in December 2012. There are test submissions from Microsoft, Ms2ger, and Opera. The WHATWG has made 14 commits to the spec. There are 13 open bugs. The plan is to finalize a baseline version as REC by 2013 focusing on implemented features, and a "bleeding edge" version addressing controversial issues with a goal of REC by 2014. Major issues include the behavior of overrideMimeType, HTTP authentication, and streaming responses.
gRPC is a modern open source RPC framework that enables client and server applications to communicate transparently. It is based on HTTP/2 for its transport mechanism and Protocol Buffers as its interface definition language. Some benefits of gRPC include being fast due to its use of HTTP/2, supporting multiple programming languages, and enabling server push capabilities. However, it also has some downsides such as potential issues with load balancing of persistent connections and requiring external services for service discovery.
Open Chemistry, JupyterLab and data: Reproducible quantum chemistryMarcus Hanwell
The Open Chemistry project is developing an ambitious platform to facilitate reproducible quantum chemistry workflows by integrating the best of breed open source projects currently available in a cohesive platform with extensions specific to the needs of quantum chemistry. The core of the project is a Python-based data server capable of storing metadata, executing quantum chemistry calculations, and processing the output. The platform exposes RESTful endpoints using programming language agnostic web endpoints, and uses Linux container technology to package quantum codes that are often difficult to build.
The Jupyter project has been leveraged as a web-based frontend offering reproducibility as a core principle. This has been coupled with the data server to initiate quantum chemistry calculations, cache results, make them searchable, and even visualize the results within a modern browser environment. The Avogadro libraries have been reused for visualization workflows, coupled with Open Babel for file translation, and examples of the use of NWChem and Psi4 will be demonstrated.
The core of the platform is developed upon JSON data standards, and encouraging the wider adoption of JSON/HDF5 as the principle storage mediums. A single page web application using React at its core will be shown for sharing simple views of data output, and linking to the Jupyter notebooks that documents how they were made. Command line tools and links to the Avogadro graphical interface will be shown demonstrating capabilities from web through to desktop.
OSMC 2019 | Grafana Loki: Like Prometheus, but for Logs by Ganesh VernekarNETWAYS
Loki is a horizontally-scalable, highly-available log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate, as it does not index the contents of the logs, but rather labels for each log stream. Loki initially targets Kubernetes logging, using Prometheus service discovery to gather labels and metadata about log streams. By using the same index and labels as Prometheus, Loki enables you to easily switch between metrics and logs, enhancing observability and streamlining the incident response process – a workflow we have built into the latest version of Grafana. In this talk we will discuss the motivation behind Loki, its design and architecture, and what the future holds. Its early days, but so far the response to the project has been overwhelming, with more the 6.5k GitHub stars and over 12hrs at the top spot on Hacker News. Loki is an opensource project, Apache licensed.
The document summarizes a student's final project presentation which had two parts:
1) A latency study of different network topologies that found no direct relationship between latency and topology. Further analysis is needed.
2) A demonstration of dynamically deploying a second controller to reduce the workload of the original controller once the number of hosts exceeded a threshold of 3. The new controller was given a copy of the network topology view.
Erik Bernhardsson presented Luigi, a Python framework for defining and running data pipelines and workflows. Luigi provides tools for batch data processing, scheduling dependencies between tasks, monitoring progress, and running tasks across multiple workers. It integrates with Hadoop, HDFS, Hive, Pig, and databases to run jobs and move data. Luigi aims to simplify workflow management compared to tools like Oozie by using Python and avoiding XML definitions.
This document summarizes Instagram's experience with Python and Django. It discusses why Instagram chose Python and Django, how they have scaled their Python/Django stack to support billions of users, and their process for migrating to Python 3 while minimizing disruptions. Key points include Instagram leveraging tools to profile performance bottlenecks, moving critical components to C/C++, and taking a gradual approach to migrating services to Python 3 one endpoint at a time over several months.
Semi-motivational talk about why today is a great time to learn Python. Slides include a brief overview of the current state of the language, its application areas, and Python's future.
Boosting command line experience with python and awkKirill Pavlov
It is often required to manipulate the data as fast as possible, be it either column average calculation or simple join and filtering.
Servers often do not have convenient tools, such as Python (NumPy/Pandas) of R, moreover, the data might not fit into memory.
This talk shows how to make fast but inconvenient command line tools great again.
GoLang is an open source programming language created by Google in 2009. It has a large community and was designed for scalability and concurrency. Some key features include being statically typed, compiled, and having built-in support for concurrency through goroutines and channels. Google uses GoLang extensively to build systems that scale to thousands of machines.
gRPC can help minimize the barrier of cross-system communication by providing language-agnostic API definitions, backward and forward compatible versioning with protocol buffers, and pluggable load balancing and tracing. You will see how to quickly get up and running with the gRPC framework using Node.js from creating a protocol definition, creating meaningful health checks, and securing the endpoint. Additionally, this session will go over best practices and how to take full advantage of what gRPC has to offer.
Presentation at NetPonto community: "We’re going to discuss gRPC, Google’s open-source RPC framework. I’ll dive a bit into the history of RPC as a protocol, and what its historical use has been. I’ll also highlight some benefits to adopt gRPC and how its possible to swap out parts of gRPC and still take advantage of gRPC’s benefits. Finally I’ll answer the question that has been on many lips since gRPC was announced — what does this mean for REST?"
Pig is a useful tool for exploring data sets and describing data flows easily through scripts. However, it has some weaknesses like inconsistent syntax, lack of testing support, and an aging code base. While Pig works well for many companies, its suitability may decline as companies' data needs grow more complex with distributed teams of data scientists and engineers. For Pig to remain useful, it will need improvements in areas like types, performance, UDF support, and testing to better support "big data" teams in the future.
The document discusses gRPC, an open-source framework for building distributed applications and services. It was motivated by Google's experience with internal RPC systems handling over 1010 RPCs per second. gRPC uses HTTP/2 for transport, allowing it to take advantage of existing network infrastructure while enabling features like bidirectional streaming. It generates client and server code from IDL definitions, supporting multiple languages.
Robert Kubis - gRPC - boilerplate to high-performance scalable APIs - code.t...AboutYouGmbH
The document discusses gRPC, an open source framework for building microservices. It was created by Google to be a high performance, open source universal RPC framework. gRPC uses HTTP/2 for transport, Protocol Buffers as the interface definition language, and generated client/server code for many languages to make cross-platform communications simple and efficient. The document provides an overview of gRPC's goals and architecture, how to define a service using .proto files, and examples of common RPC patterns like unary, streaming, and bidirectional calls.
Bringing Learnings from Googley Microservices with gRPC - Varun Talwar, GoogleAmbassador Labs
Varun Talwar, product manager on Google's gRPC project discusses the fundamentals and specs of gRPC inside of a Google-scale microservices architecture.
Design and Implementation of the Security Graph LanguageAsankhaya Sharma
Today software is built in fundamentally different
ways from how it was a decade ago. It is increasingly common
for applications to be assembled out of open-source components,
resulting in the use of large amounts of third-party code. This
third-party code is a means for vulnerabilities to make their
way downstream into applications. Recent vulnerabilities such
as Heartbleed, FREAK SSL/TLS, GHOST, and the Equifax data
breach (due to a flaw in Apache Struts) were ultimately caused
by third-party components. We argue that an automated way to
audit the open-source ecosystem, catalog existing vulnerabilities,
and discover new flaws is essential to using open-source safely.
To this end, we describe the Security Graph Language (SGL), a
domain-specific language for analysing graph-structured datasets
of open-source code and cataloguing vulnerabilities. SGL allows
users to express complex queries on relations between libraries
and vulnerabilities in the style of a program analysis language.
SGL queries double as an executable representation for vulnerabilities, allowing vulnerabilities to be automatically checked
against a database and deduplicated using a canonical representation. We outline a novel optimisation for SGL queries based on
regular path query containment, improving query performance up to 3 orders of magnitude. We also demonstrate the
effectiveness of SGL in practice to find zero-day vulnerabilities
by identifying sever
Per aspera ad grid To the grid computing through difficultiesFerenc Szalai
The document discusses the history and evolution of grid computing from the perspective of the author. It covers early work using clusters for computationally intensive tasks in the late 1990s, the development of grid middleware like Globus to schedule tasks on heterogeneous resources, and issues with standardization and interoperability between different grid projects. It then outlines the shift towards more modular, service-oriented architectures exemplified by the ARC middleware, as well as changes brought by volunteer desktop grids, MapReduce, and the rise of cloud computing where resources are virtualized and provided as a service.
The document summarizes recent updates in PHP, frameworks, CMSes, and upcoming conferences. Key points include the release of PHP versions 7.0.12 and 5.6.27 with security fixes. For frameworks, CakePHP released versions 3.3.6 and 3.3.7, and Laravel released versions 5.3.16 through 5.3.21. Drupal released version 8.2.0 with new features. Upcoming conferences highlighted include PHPWorld in November and ZendCon in November.
The summary provides an overview of the key topics from the November 2016 PHP newsletter:
1. PHP released security patches for versions 7.0.13, 5.6.28 and the last release candidate of 7.1 which will be finalized in December.
2. Upcoming RFC proposals discussed security classification of vulnerabilities, adding debugging functions to PDO prepared statements, and deprecating image conversion functions.
3. Popular frameworks like Drupal, WordPress, CakePHP and Laravel released updates with bug fixes and new features.
4. Information was provided on upcoming PHP conferences in early 2017 and a reminder for speakers for events later in the year. The next local user group meeting will
"Building, deploying and running production code at Dropbox" Васильев Леонид,...it-people
The document discusses Dropbox's approach to building, deploying, and running production code. For building, they migrated to Bazel for its reproducibility, hermetic builds, and ability to track dependencies across languages. For deploying, they use YAPS to package dependencies and services into read-only squashfs images. For running services, they use Dbxinit for process management and enforce resource limits, while also integrating containers via RunC. Overall their goal is robust and automated systems from building to deployment to operations.
The document discusses the status and plans for XMLHttpRequest. It notes that the specification reached TR publication in December 2012. There are test submissions from Microsoft, Ms2ger, and Opera. The WHATWG has made 14 commits to the spec. There are 13 open bugs. The plan is to finalize a baseline version as REC by 2013 focusing on implemented features, and a "bleeding edge" version addressing controversial issues with a goal of REC by 2014. Major issues include the behavior of overrideMimeType, HTTP authentication, and streaming responses.
gRPC is a modern open source RPC framework that enables client and server applications to communicate transparently. It is based on HTTP/2 for its transport mechanism and Protocol Buffers as its interface definition language. Some benefits of gRPC include being fast due to its use of HTTP/2, supporting multiple programming languages, and enabling server push capabilities. However, it also has some downsides such as potential issues with load balancing of persistent connections and requiring external services for service discovery.
Open Chemistry, JupyterLab and data: Reproducible quantum chemistryMarcus Hanwell
The Open Chemistry project is developing an ambitious platform to facilitate reproducible quantum chemistry workflows by integrating the best of breed open source projects currently available in a cohesive platform with extensions specific to the needs of quantum chemistry. The core of the project is a Python-based data server capable of storing metadata, executing quantum chemistry calculations, and processing the output. The platform exposes RESTful endpoints using programming language agnostic web endpoints, and uses Linux container technology to package quantum codes that are often difficult to build.
The Jupyter project has been leveraged as a web-based frontend offering reproducibility as a core principle. This has been coupled with the data server to initiate quantum chemistry calculations, cache results, make them searchable, and even visualize the results within a modern browser environment. The Avogadro libraries have been reused for visualization workflows, coupled with Open Babel for file translation, and examples of the use of NWChem and Psi4 will be demonstrated.
The core of the platform is developed upon JSON data standards, and encouraging the wider adoption of JSON/HDF5 as the principle storage mediums. A single page web application using React at its core will be shown for sharing simple views of data output, and linking to the Jupyter notebooks that documents how they were made. Command line tools and links to the Avogadro graphical interface will be shown demonstrating capabilities from web through to desktop.
OSMC 2019 | Grafana Loki: Like Prometheus, but for Logs by Ganesh VernekarNETWAYS
Loki is a horizontally-scalable, highly-available log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate, as it does not index the contents of the logs, but rather labels for each log stream. Loki initially targets Kubernetes logging, using Prometheus service discovery to gather labels and metadata about log streams. By using the same index and labels as Prometheus, Loki enables you to easily switch between metrics and logs, enhancing observability and streamlining the incident response process – a workflow we have built into the latest version of Grafana. In this talk we will discuss the motivation behind Loki, its design and architecture, and what the future holds. Its early days, but so far the response to the project has been overwhelming, with more the 6.5k GitHub stars and over 12hrs at the top spot on Hacker News. Loki is an opensource project, Apache licensed.
The document summarizes a student's final project presentation which had two parts:
1) A latency study of different network topologies that found no direct relationship between latency and topology. Further analysis is needed.
2) A demonstration of dynamically deploying a second controller to reduce the workload of the original controller once the number of hosts exceeded a threshold of 3. The new controller was given a copy of the network topology view.
Erik Bernhardsson presented Luigi, a Python framework for defining and running data pipelines and workflows. Luigi provides tools for batch data processing, scheduling dependencies between tasks, monitoring progress, and running tasks across multiple workers. It integrates with Hadoop, HDFS, Hive, Pig, and databases to run jobs and move data. Luigi aims to simplify workflow management compared to tools like Oozie by using Python and avoiding XML definitions.
This document summarizes Instagram's experience with Python and Django. It discusses why Instagram chose Python and Django, how they have scaled their Python/Django stack to support billions of users, and their process for migrating to Python 3 while minimizing disruptions. Key points include Instagram leveraging tools to profile performance bottlenecks, moving critical components to C/C++, and taking a gradual approach to migrating services to Python 3 one endpoint at a time over several months.
Semi-motivational talk about why today is a great time to learn Python. Slides include a brief overview of the current state of the language, its application areas, and Python's future.
This document provides an overview of Python and related topics in 3 sentences:
Python is a popular programming language and the document discusses Python projects, libraries, and APIs for tasks like processing PDFs and images. It also briefly mentions using NumPy and Google Cloud Vision API for image processing and deep learning concepts like convolutional neural networks.
This talk describes the design decisions, implementation steps and collected experiences related to migrating data oriented financial applications from Python 2.6 to Python 3.5 in the Raiffeisen Rechenzentrum.
This document provides an overview of the Python programming language, including its history, implementations, key features, examples of usage, and popularity. Some of the key points covered include:
- Python was created in the late 1980s and continues to be actively developed and improved. It has a simple yet powerful design philosophy.
- It has multiple implementations like CPython, PyPy, Jython, and is widely used across industries like Google, NASA, and YouTube.
- Python code is highly readable and emphasizes readability. It has a large standard library and ecosystem of over 100,000 packages.
- Popular applications include web development, science/data analysis, machine learning, desktop GUIs, and more.
The document discusses starting a Python Discussion Group at the University of Leeds School of Computing. It outlines reasons for using Python like it being free, easy to learn, and having many available packages. It also discusses myths about Python and potential activities for the group like mailing lists, wikis, data hacking sessions, tips/talks, and collaborating with other Python groups. The goal is to create a platform for Python enthusiasts to learn and discover its full capabilities.
Python 2 is dead! Drag your old code into the modern ageBecky Smith
The clock is ticking on Python 2.7, with support to be dropped in January 2020. With major dependencies such as Django, NumPy and pandas moving to Python 3 only, the time has come for even big established codebases to consider upgrading. Many organisations are still postponing for various reasons; we will attempt to demonstrate that with a bit of planning and perseverance, and the assistance of some handy tools, we can embrace the future!
This session will provide a first-hand perspective on how we upgraded a large (~65,000 lines of python code) 8-year-old Django project with multiple external dependencies from Python 2.7 to Python 3.6.
We will briefly discuss the benefits of upgrading to Python 3, and architectural considerations. The session will primarily focus on the practicalities of upgrading the code itself. We will not try to provide a single “best” solution for upgrading to Python 3, but rather will introduce some of the available tools, provide an insight into how we used them, and their advantages and disadvantages from our experience. We will discuss preparatory steps and approaches, strategies for dealing with external dependencies, and “gotchas” that we encountered during the process.
The aim of this session is to provide an example of how a Python 3 upgrade on an established commercial product can be successfully completed, and to furnish audience members with a set of tools and strategies to help them with their own projects.
The document discusses a presentation on the benefits of type hints in Python. It provides an outline of the presentation which includes an introduction to type hints, how to use type hints, and the benefits of type hints. Some key benefits mentioned are improved code completion, the ability to catch type errors without running code, and using type hint tools for static type analysis.
This document discusses the Python programming language. It provides an overview of Python's history and creator Guido van Rossum. It also outlines some of Python's key features like being cross-platform, popular uses like web and desktop apps, and standard libraries. The document also briefly describes Python concepts like PEPs, documentation, data types, functions, classes and provides some learning resources.
This document provides a tutorial on data science in Python. It discusses Python's history and the Jupyter notebook interface. It also demonstrates how to import Python packages, load data, inspect data, and munge data for analysis. Specific techniques shown include importing datasets, checking data types and dimensions, selecting rows and columns, and obtaining summary information about the data.
Open Source Community Metrics for FOSDEMDawn Foster
Presented in the Community DevRoom at FOSDEM 2013. A longer version of this presentation is available at http://fastwonderblog.com/2012/11/05/open-source-community-metrics-linuxcon-barcelona/
[GSoC 2017] gopy: Updating gopy to support Python3 and PyPyDong-hee Na
gopy is an excellent tool which generates (and compiles) a CPython extension module from a go package. And I hope more developers could make full use of gopy to migrate their go code into python code. To make gopy more advanced, It is necessary to provide APIs for various Python compiler versions, such as CPython 2/3 and PyPy. This can be improved with CFFI or ctypes. Moreover, many go’s implementations/features are not yet implemented in gopy. So we need to implement implementations such as slices, interfaces, and maps in the go.
My goal is to update gopy by using CFFI to support Python3 and PyPy and write detailed documents
Shinichi Nakagawa gave a presentation on Python, PyData tools like Jupyter and pandas, and analyzing baseball player Shohei Ohtani's stats from the 2016 season. He introduced himself and his background in Python and PyCon JP. He demonstrated how to create a virtualenv for a Jupyter lab environment to analyze Ohtani's stats from web pages, and shared the GitHub repository containing the Jupyter notebook analyzing Ohtani's batting and pitching stats from that season.
The Onward Journey: Porting Twisted to Python 3Craig Rodrigues
This document discusses porting the Twisted Python library to Python 3. It describes Twisted as a networking library that uses asynchronous programming. The author ported over 325 pull requests to help move Twisted to Python 3. Some challenges included changes to Python 3 like print becoming a function and str handling, as well as porting old C extensions. Running extensive unit tests was important. While difficult, porting benefits code quality and keeps Twisted compatible with Python's direction.
This document summarizes data from the Safari Books Online ebook archive about Python programming books and readership. Some key findings include: the number of pages and hours spent reading Python books has greatly increased over time; books have become more advanced and focused on specific topics like web development; and there is high demand and engagement for books on advanced Python topics like algorithms, scientific computing, and analytics. The document concludes with recommendations such as writing more Python books and videos, especially for advanced topics, and targeting Python books towards readers in different domains.
This document provides an overview of the history and evolution of the Python programming language from its origins in 1980 through modern Python 3. It details major releases including Python 2.0, 2.5, and 3.0. It also discusses the rise of web frameworks like Django and Flask that expanded Python's use in web development. Key aspects that contributed to Python's popularity like its use for science, education, automation and more are covered at a high level.
Evan Lin introduces Golang Taipei, a Golang user group he co-organizes in Taiwan. He discusses his background in Windows and web development and how he got involved with Golang. He shares about Project 52, a challenge he took on to create 52 Golang projects in a year. This helped him learn new tools and algorithms. Evan explains how blogging about his projects and sharing his knowledge led him to become a Golang Developer Experience (GDE). He encourages sharing what you learn to both teach others and further your own understanding of Golang.
What is Python? An overview of Python for science.Nicholas Pringle
Python is a general purpose, high-level, free and open-source programming language that is readable and intuitive. It has strong scientific computing packages like NumPy, SciPy, and Matplotlib that allow it to be used for tasks like MATLAB. Python emphasizes code readability and reusability through standards like PEP8 and version control, making it well-suited for collaboration between individual, institutional, and developer users in its large, diverse community.
Similar to Austin Python Meetup 2017: How to Stop Worrying and Start a Project with Python 3 (20)
The Key to Digital Success_ A Comprehensive Guide to Continuous Testing Integ...kalichargn70th171
In today's business landscape, digital integration is ubiquitous, demanding swift innovation as a necessity rather than a luxury. In a fiercely competitive market with heightened customer expectations, the timely launch of flawless digital products is crucial for both acquisition and retention—any delay risks ceding market share to competitors.
Measures in SQL (SIGMOD 2024, Santiago, Chile)Julian Hyde
SQL has attained widespread adoption, but Business Intelligence tools still use their own higher level languages based upon a multidimensional paradigm. Composable calculations are what is missing from SQL, and we propose a new kind of column, called a measure, that attaches a calculation to a table. Like regular tables, tables with measures are composable and closed when used in queries.
SQL-with-measures has the power, conciseness and reusability of multidimensional languages but retains SQL semantics. Measure invocations can be expanded in place to simple, clear SQL.
To define the evaluation semantics for measures, we introduce context-sensitive expressions (a way to evaluate multidimensional expressions that is consistent with existing SQL semantics), a concept called evaluation context, and several operations for setting and modifying the evaluation context.
A talk at SIGMOD, June 9–15, 2024, Santiago, Chile
Authors: Julian Hyde (Google) and John Fremlin (Google)
https://doi.org/10.1145/3626246.3653374
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesQuickdice ERP
Explore the seamless transition to e-invoicing with this comprehensive guide tailored for Saudi Arabian businesses. Navigate the process effortlessly with step-by-step instructions designed to streamline implementation and enhance efficiency.
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsPeter Muessig
The UI5 tooling is the development and build tooling of UI5. It is built in a modular and extensible way so that it can be easily extended by your needs. This session will showcase various tooling extensions which can boost your development experience by far so that you can really work offline, transpile your code in your project to use even newer versions of EcmaScript (than 2022 which is supported right now by the UI5 tooling), consume any npm package of your choice in your project, using different kind of proxies, and even stitching UI5 projects during development together to mimic your target environment.
Preparing Non - Technical Founders for Engaging a Tech AgencyISH Technologies
Preparing non-technical founders before engaging a tech agency is crucial for the success of their projects. It starts with clearly defining their vision and goals, conducting thorough market research, and gaining a basic understanding of relevant technologies. Setting realistic expectations and preparing a detailed project brief are essential steps. Founders should select a tech agency with a proven track record and establish clear communication channels. Additionally, addressing legal and contractual considerations and planning for post-launch support are vital to ensure a smooth and successful collaboration. This preparation empowers non-technical founders to effectively communicate their needs and work seamlessly with their chosen tech agency.Visit our site to get more details about this. Contact us today www.ishtechnologies.com.au
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...kalichargn70th171
In today's fiercely competitive mobile app market, the role of the QA team is pivotal for continuous improvement and sustained success. Effective testing strategies are essential to navigate the challenges confidently and precisely. Ensuring the perfection of mobile apps before they reach end-users requires thoughtful decisions in the testing plan.
INTRODUCTION TO AI CLASSICAL THEORY TARGETED EXAMPLESanfaltahir1010
Image: Include an image that represents the concept of precision, such as a AI helix or a futuristic healthcare
setting.
Objective: Provide a foundational understanding of precision medicine and its departure from traditional
approaches
Role of theory: Discuss how genomics, the study of an organism's complete set of AI ,
plays a crucial role in precision medicine.
Customizing treatment plans: Highlight how genetic information is used to customize
treatment plans based on an individual's genetic makeup.
Examples: Provide real-world examples of successful application of AI such as genetic
therapies or targeted treatments.
Importance of molecular diagnostics: Explain the role of molecular diagnostics in identifying
molecular and genetic markers associated with diseases.
Biomarker testing: Showcase how biomarker testing aids in creating personalized treatment plans.
Content:
• Ethical issues: Examine ethical concerns related to precision medicine, such as privacy, consent, and
potential misuse of genetic information.
• Regulations and guidelines: Present examples of ethical guidelines and regulations in place to safeguard
patient rights.
• Visuals: Include images or icons representing ethical considerations.
Content:
• Ethical issues: Examine ethical concerns related to precision medicine, such as privacy, consent, and
potential misuse of genetic information.
• Regulations and guidelines: Present examples of ethical guidelines and regulations in place to safeguard
patient rights.
• Visuals: Include images or icons representing ethical considerations.
Content:
• Ethical issues: Examine ethical concerns related to precision medicine, such as privacy, consent, and
potential misuse of genetic information.
• Regulations and guidelines: Present examples of ethical guidelines and regulations in place to safeguard
patient rights.
• Visuals: Include images or icons representing ethical considerations.
Real-world case study: Present a detailed case study showcasing the success of precision
medicine in a specific medical scenario.
Patient's journey: Discuss the patient's journey, treatment plan, and outcomes.
Impact: Emphasize the transformative effect of precision medicine on the individual's
health.
Objective: Ground the presentation in a real-world example, highlighting the practical
application and success of precision medicine.
Data challenges: Address the challenges associated with managing large sets of patient data in precision
medicine.
Technological solutions: Discuss technological innovations and solutions for handling and analyzing vast
datasets.
Visuals: Include graphics representing data management challenges and technological solutions.
Objective: Acknowledge the data-related challenges in precision medicine and highlight innovative solutions.
Data challenges: Address the challenges associated with managing large sets of patient data in precision
medicine.
Technological solutions: Discuss technological innovations and solutions
Enhanced Screen Flows UI/UX using SLDS with Tom KittPeter Caitens
Join us for an engaging session led by Flow Champion, Tom Kitt. This session will dive into a technique of enhancing the user interfaces and user experiences within Screen Flows using the Salesforce Lightning Design System (SLDS). This technique uses Native functionality, with No Apex Code, No Custom Components and No Managed Packages required.
WWDC 2024 Keynote Review: For CocoaCoders AustinPatrick Weigel
Overview of WWDC 2024 Keynote Address.
Covers: Apple Intelligence, iOS18, macOS Sequoia, iPadOS, watchOS, visionOS, and Apple TV+.
Understandable dialogue on Apple TV+
On-device app controlling AI.
Access to ChatGPT with a guest appearance by Chief Data Thief Sam Altman!
App Locking! iPhone Mirroring! And a Calculator!!
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...XfilesPro
Wondering how X-Sign gained popularity in a quick time span? This eSign functionality of XfilesPro DocuPrime has many advancements to offer for Salesforce users. Explore them now!
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemPeter Muessig
Learn about the latest innovations in and around OpenUI5/SAPUI5: UI5 Tooling, UI5 linter, UI5 Web Components, Web Components Integration, UI5 2.x, UI5 GenAI.
Recording:
https://www.youtube.com/live/MSdGLG2zLy8?si=INxBHTqkwHhxV5Ta&t=0
How Can Hiring A Mobile App Development Company Help Your Business Grow?ToXSL Technologies
ToXSL Technologies is an award-winning Mobile App Development Company in Dubai that helps businesses reshape their digital possibilities with custom app services. As a top app development company in Dubai, we offer highly engaging iOS & Android app solutions. https://rb.gy/necdnt
3. 3
3 months back from today...
https://www.slideshare.net/mariczhuck/austin-python-meetup-2017-whats-new-in-pythons-35-and-36
4. Agenda
1. The story about my journey from 2 to 3
2. Should I use Python 3, not 2?
3. Discover the benefits with your team
4. How to discuss switching to Python 3 with your
manager
5. Summary
4
6. The story about my journey
6
# python2
@defer.inlineCallbacks
# python2, small projects
@app.route('')
7. The story about my journey
7
# python2, big project
d = defer.Deferred()
d.addCallback(foo)
d.addErrback(bar)
# python3, non-commercial project
async / await
11. Reasons to use Python 3 for a developer
11
New syntax features make our life easier
● PEP 448, additional unpacking generalizations
● PEP 465, a new matrix multiplication operator `a @ b`
● PEP 492, coroutines with async and await syntax
● PEP 498, formatted string literals
● PEP 515, underscores in numeric literals
● PEP 525, asynchronous generators
● PEP 526, syntax for variable annotations
● PEP 530, asynchronous comprehensions
● and others
30. Experience of my team:
Dev speed is pretty good
after the first month
30
31. Summary
● The future is already here
● Learn new Python 3 features - they are great
● Make side projects and prototypes to gain experience
● New learners - invest your time only in Python 3
● Start new projects only with Python 3
31