Migrating our monolith to Python 3
•Download as PPTX, PDF•
0 likes•245 views
The document summarizes a company's plan to migrate their monolithic Python application from Python 2.7 to Python 3. It discusses setting up the code to be Python 3 compatible, making it run on test environments, passing all tests, upgrading database migrations, manually testing, releasing the new version, and rolling back when issues were discovered after release. It ends by thanking the audience.
Report
Share
Report
Share
Recommended
Introduction into GIT
Git is a version control system initially created by Linus Torvalds for Linux kernel development. It is a distributed system, meaning the full history of the codebase is hosted on each user's machine rather than a centralized server. This makes it more robust than centralized systems like SVN - even if the remote server is unavailable, users can still work on code independently and sync up later. Git is also generally faster than SVN, with smaller file sizes and quicker upload/download times for projects of the same size due to its distributed nature.
Kubernetes
Kubernetes is open-source container-centric platform for automating deployment, scaling, and operations of application containers across clusters of hosts.
Atomic Developer Bundle
Atomic Developer Bundle packages some of the most widely used container orchestration tools into a single Vagrant box to provide developers with a configuration as close to production as possible!
Select tools ~ why we choose the way ~
Hiroaki Suginaka discusses tools used at wovn.io including:
1. Using EC2, S3, Elastic Cache, and CloudFront on the backend.
2. Support tools include Rollbar, CircleCI, Logentries, Jenkins, and Zeplin.
3. The request flow involves translating text, caching translations, and purging caches when new text is detected.
4. Wovn.io migrated from AngularJS to Vue.js for a lighter framework and from CloudFront to Fastly as a CDN for immediate cache updates.
5. Current problems include scaling, logs, and other unspecified issues.
Extracting source code of apk file
The document provides instructions for extracting Java code and XML code from an Android APK file in 4 steps:
1) Convert the APK file to a ZIP and extract the classes.dex file
2) Use dex2jar to convert the classes.dex to a JAR file
3) Open the JAR file in a Java decompiler to view the Java code
4) Use apktool to decode the APK and view the XML code in the generated res folder
Python workshop
This document provides an overview of a Python programming crash course workshop. It discusses what Python is, its history and goals, available versions, why it is popular, and key features like its standard library, modules, and popular third-party libraries like NumPy, Pandas, and scikit-learn that extend its functionality for scientific computing, data analysis, and machine learning. The workshop also covers Python basics and more advanced topics.
Cutting edge android stack. One year later
This document discusses the core ideas and technologies used in the Android stack created by Juno, including immutability, concise code, modularity, testability, and use of Kotlin and RxJava. The stack emphasizes safety through immutable and final classes, easy concurrency, and null safety. It aims for concise code through properties, type inference, and stream-like APIs. Modularity is achieved through clean architecture principles. Tests are enabled by separation of concerns and isolating business logic. Kotlin and RxJava integration works well overall, though some challenges remain around testing and null safety.
Introduction to Git
This document introduces version control systems (VCS) like Git, which allow tracking changes to documents and enabling multiple people to work on the same files. Git is a free, open source, distributed VCS developed by Linus Torvalds and Junio Hamano that is very fast and efficient. It is used by many large projects like Linux Kernel, Android, and Ruby on Rails. The document then outlines basics of Git like snapshots and being distributed, before concluding with references for further reading on Git.
Recommended
Introduction into GIT
Git is a version control system initially created by Linus Torvalds for Linux kernel development. It is a distributed system, meaning the full history of the codebase is hosted on each user's machine rather than a centralized server. This makes it more robust than centralized systems like SVN - even if the remote server is unavailable, users can still work on code independently and sync up later. Git is also generally faster than SVN, with smaller file sizes and quicker upload/download times for projects of the same size due to its distributed nature.
Kubernetes
Kubernetes is open-source container-centric platform for automating deployment, scaling, and operations of application containers across clusters of hosts.
Atomic Developer Bundle
Atomic Developer Bundle packages some of the most widely used container orchestration tools into a single Vagrant box to provide developers with a configuration as close to production as possible!
Select tools ~ why we choose the way ~
Hiroaki Suginaka discusses tools used at wovn.io including:
1. Using EC2, S3, Elastic Cache, and CloudFront on the backend.
2. Support tools include Rollbar, CircleCI, Logentries, Jenkins, and Zeplin.
3. The request flow involves translating text, caching translations, and purging caches when new text is detected.
4. Wovn.io migrated from AngularJS to Vue.js for a lighter framework and from CloudFront to Fastly as a CDN for immediate cache updates.
5. Current problems include scaling, logs, and other unspecified issues.
Extracting source code of apk file
The document provides instructions for extracting Java code and XML code from an Android APK file in 4 steps:
1) Convert the APK file to a ZIP and extract the classes.dex file
2) Use dex2jar to convert the classes.dex to a JAR file
3) Open the JAR file in a Java decompiler to view the Java code
4) Use apktool to decode the APK and view the XML code in the generated res folder
Python workshop
This document provides an overview of a Python programming crash course workshop. It discusses what Python is, its history and goals, available versions, why it is popular, and key features like its standard library, modules, and popular third-party libraries like NumPy, Pandas, and scikit-learn that extend its functionality for scientific computing, data analysis, and machine learning. The workshop also covers Python basics and more advanced topics.
Cutting edge android stack. One year later
This document discusses the core ideas and technologies used in the Android stack created by Juno, including immutability, concise code, modularity, testability, and use of Kotlin and RxJava. The stack emphasizes safety through immutable and final classes, easy concurrency, and null safety. It aims for concise code through properties, type inference, and stream-like APIs. Modularity is achieved through clean architecture principles. Tests are enabled by separation of concerns and isolating business logic. Kotlin and RxJava integration works well overall, though some challenges remain around testing and null safety.
Introduction to Git
This document introduces version control systems (VCS) like Git, which allow tracking changes to documents and enabling multiple people to work on the same files. Git is a free, open source, distributed VCS developed by Linus Torvalds and Junio Hamano that is very fast and efficient. It is used by many large projects like Linux Kernel, Android, and Ruby on Rails. The document then outlines basics of Git like snapshots and being distributed, before concluding with references for further reading on Git.
Python fundamental course curriculum
EngiGyan train you in Python like a professional. Start from the basic & grow to develop own application & games.
I believe in rust
The document is a presentation about the Rust programming language. It introduces Rust as a compiled, concurrent language from Mozilla that uses message passing and immutable variables by default. It discusses Rust's task-based concurrency model and provides examples of Rust concepts like pattern matching, ports and channels, modules and crates, and unit testing. It also briefly mentions the Servo browser project in Rust and compares Rust to other languages like Go in achieving goals from Yegge's laundry list.
effective java
Geralmente aplicativos buscam resolver uma única tarefa. Mas conforme o passar do tempo, a aplicação tende a ganhar novas features e otimizações, consequentemente ela tende a crescer. Mais features e mais usuários significam mais código. E quanto mais código você escreve, mais código você precisa manter e reescrever. Nos livros "Effective Java" e "Android programming - pushing limits", Joshua Bloch e Erik Hellman, respectivamente, demonstram boas práticas para o desenvolvimento em Java e na plataforma Android. A idéia é fazer um apanhado das melhores práticas trazidas por estes livros. Detalharei o que deve o que não deve ser feito ao aplicar tais práticas.
Why Rust? - Matthias Endler - Codemotion Amsterdam 2016
Rust is the new kid on the block. It's a system programming language that is blazingly fast and offers powerful high-level abstractions better known from dynamic languages like Ruby or Python. Rust offers memory safety without a Garbage Collector and makes low-level programming easier and more secure. I will give an introductory talk about Rust and show some of its nicest features.
Deep drive into rust programming language
This document introduces Rust and provides an overview of its key concepts. It begins with an introduction to the presenter and agenda. It then covers basic terminology, common system programming errors, why Rust was created, how to install Rust, data types including primitive types, complex data structures, ownership and borrowing rules, lifetimes, and how to get involved in the Rust community. Key concepts discussed include Rust's type system, memory safety features, and package manager.
Python training-course-content
Python training in Chennai at Credo Systemz helps you to get an extensive knowledge of Python programming language. Python classes in Chennai by Credo Systemz is an instructor-led training conducted in Chennai premises.
TDC2016SP - Groovy como você nunca viu
Groovy is an object-oriented scripting language for the Java Virtual Machine (JVM) that can be optionally statically typed. It aims to provide features like dynamic typing, closures, builders and other conveniences while retaining compatibility with Java. Groovy code compiles to Java bytecode and passes through several phases including parsing, semantic analysis, instruction selection and class generation. It has a large ecosystem and is designed to integrate well with Java while allowing programming in a more concise, less boilerplate-oriented style.
Rust Programming Language
Rust is a systems programming language developed by Mozilla that provides memory safety without garbage collection. It uses concepts of ownership and borrowing to ensure memory safety issues like use-after-free do not occur. Rust offers zero-cost abstractions meaning abstraction mechanisms like generics have little to no performance overhead. It allows high levels of concurrency but requires memory references be valid during their entire lifetime.
Python Linters at Scale.pdf
Black, Flake8, isort, and Mypy are useful Python linters but it’s challenging to use them effectively at scale in the case of multiple codebases, in a large codebase, or with many developers. Manually managing consistent linter versions and configurations across codebases requires endless effort. Linter analysis on large codebases is slow. Linters may slow down developers by asking them to fix trivial issues. Running linters in distributed CI jobs makes it hard to understand the overall developer experience.
To handle these scale challenges, we developed a reusable linter framework that releases new linter updates automatically, reuses consistent configurations, runs linters on only updated code to speedup runtime, collects logs and metrics to provide observability, and builds auto fixes for common linter issues. Our linter runs are fast and scalable. Every week, they run 10k times on multiple millions of lines of code in over 25 codebases, generating 25k suggestions for more than 200 developers. Its autofixes also save 20 hours of developer time every week.
In this talk, we’ll walk you through popular Python linters and configuration recommendations, and we will discuss common issues and solutions when scaling them out. Using linters more effectively will make it much easier for you to apply best practices and more quickly write better code.
Testing Django APIs
This document summarizes the testing approach and strategies used for an application built with Django and Django Rest Framework. It discusses the use of both unit and integration tests, along with different testing frameworks like Django's built-in test tools, pytest, and libraries like pytest-django and pytest-xdist. It also covers strategies like database fixtures, parameterized testing, and mocking HTTP requests. The document concludes by discussing plans to improve testing in the future such as enabling parallel test runs and adding performance regression testing.
Koji and pulp
The document discusses Koji, Pulp, and errata management in CentOS. Koji is an rpm building automation tool that allows centralized building and tracking of packages. Pulp is a Python application for repository management that allows mirroring, errata management, and pushing updates to consumer systems. The document notes that CentOS has an unusual errata management process that relies on mailing lists rather than traditional metadata files, but includes the centos-errata-builder tool.
Continuous delivery of Windows micro services in the cloud
Chef london meetup talk about windows micro services and continuous delivery at justgiving, and the journey we took
Launchpad: Lessons Learnt
Launchpad is a Python-based web application with over 350k lines of code and 25k tests. It was developed primarily by Canonical using Zope 3, Twisted, and other libraries. The presentation discusses best practices from Launchpad's development including extensive testing, code reviews, use of branches, and interfaces. It also covers challenges like slow tests and database evolution.
Migrating python.org to buildbot 9 and python 3
Craig Rodrigues presented on migrating Python.org's build system from Buildbot 8 to Buildbot 9 and from Python 2 to Python 3. Buildbot is open source build automation software used by Python.org to build and test Python (CPython). The migration involved porting Buildbot and its dependencies like Twisted to be compatible with Python 3, porting Python.org's Buildbot configuration, and working with an ops team to complete the migration. This updated Python.org's infrastructure to a newer, actively supported version of Buildbot that enables new features and works with both Python 2 and Python 3.
How we realized SOA by Python at PyCon JP 2015
The document discusses how to realize a service-oriented architecture (SOA) using Python. It describes a case study of migrating a newspaper company's monolithic web application to microservices. Key practices covered include using Django and Django REST Framework to easily create REST APIs, tox for testing across environments, automated deployments to ElasticBeanstalk, and tools like Sentry and NewRelic for monitoring the distributed system of microservices. While SOA provides benefits, it also requires effort to manage the increased number of independent services.
Go at Skroutz
Presentation from the 4th Athens Gophers Meetup.
At a glance we present:
- why we introduced a new language in the organization and why that
was Go
- how we approached the transition
- some of the projects we built in Go
- the challenges we faced and the lessons we learned in the process
Introduction to GitHub Actions
Github Actions enables you to create custom software development lifecycle workflows directly in your Github repository. These workflows are made out of different tasks so-called actions that can be run automatically on certain events.
The Five Stages of Enterprise Jupyter Deployment
Meetup talk from May 30, 2018.
Jupyter notebooks are an important tool for data science. For a single user on a laptop, these notebooks are a simple, straightforward tool. But Jupyter in the enterprise is a much more complex affair. Enterprises have large teams of data scientists who need to run their notebooks atop scalable compute infrastructure with secure, audited access to massive, proprietary data sets; all while keeping hardware costs down.
Here at IBM’s Center for Open-Source Data and AI Technologies, we’ve seen multiple enterprise rollouts of Jupyter notebooks, both first-hand, in IBM products and services; and second-hand, in our discussions with other members of the Jupyter community.
In this talk, we merge together the stories of these projects and walk through the process of deploying high-performance, secure, mulitentant Jupyter notebooks in an enterprise setting. Our goal is here is inform others who may be at the beginning of this journey of what is coming and how to navigate the challenges ahead.
Along the way, we answer five important questions: What are Jupyter notebooks? What makes Jupyter so attractive to data scientists? Why is deploying Jupyter in the enterprise difficult? What are your deployment options today? And, what are the tradeoffs of those approaches?
We’ll finish with a description of how how IBM and other members of the Jupyter community are working towards reducing those tradeoffs with the Jupyter Enterprise Gateway project. Finally, we’ll give a demonstration of multitenant Jupyter notebooks in action.
This talk is aimed at enterprise architects who need to support growing data science teams with multi-user deployments of Jupyter. No knowledge of data science is required.
Python: Thanks for the memories
Typically, Python software engineers don’t necessarily care about how the language handles memory. However, sometimes it’s very useful to understand what’s going on under the hood. In this talk, I’ll give you a brief overview of how Python manages memory and some useful tips and tricks that you may not already know.
Programming with Python - Basic
s[:] makes a copy.
– s[:]
– slice(0, 3)
– s[slice(0, 3)]
– s[0:3]
– s[:3]
– s[3:]
– s[:-3]
So slice is more flexible.
– s[start:stop:step]
– s[::-1] # reverse
– help(slice)
– s = 'Hello World'
– s[5:11:2]
– 'oWl'
– s[::2]
– 'Hlo rd'
– s[-
Git 1
This document provides an overview and introduction to using the version control system Git. It begins with explaining what version control is and why it is useful for collaboration, storing versions, restoring previous versions, understanding changes, and backup. It then discusses using Git from the command line versus a GUI. The remainder of the document outlines Git's basic workflow, including configuring Git, committing related changes with good commit messages, using branches extensively, never committing half-done work by using stash, keeping the remote in sync, and merging changes. It concludes by inviting the reader to join them in learning more about Git.
Dmytro Dziubenko "Developer's toolchain"
There is not enough knowledge of programming languages to be good developers nowadays. It is important to understand the processes of continually improving the quality of the code and its performance. The ability to configure the development processes in a team more valuable than just writing code. To do this, you need to start using new tools or improve existing ones. We will go through all stages of development and find out what can be done better.
More Related Content
What's hot
Python fundamental course curriculum
EngiGyan train you in Python like a professional. Start from the basic & grow to develop own application & games.
I believe in rust
The document is a presentation about the Rust programming language. It introduces Rust as a compiled, concurrent language from Mozilla that uses message passing and immutable variables by default. It discusses Rust's task-based concurrency model and provides examples of Rust concepts like pattern matching, ports and channels, modules and crates, and unit testing. It also briefly mentions the Servo browser project in Rust and compares Rust to other languages like Go in achieving goals from Yegge's laundry list.
effective java
Geralmente aplicativos buscam resolver uma única tarefa. Mas conforme o passar do tempo, a aplicação tende a ganhar novas features e otimizações, consequentemente ela tende a crescer. Mais features e mais usuários significam mais código. E quanto mais código você escreve, mais código você precisa manter e reescrever. Nos livros "Effective Java" e "Android programming - pushing limits", Joshua Bloch e Erik Hellman, respectivamente, demonstram boas práticas para o desenvolvimento em Java e na plataforma Android. A idéia é fazer um apanhado das melhores práticas trazidas por estes livros. Detalharei o que deve o que não deve ser feito ao aplicar tais práticas.
Why Rust? - Matthias Endler - Codemotion Amsterdam 2016
Rust is the new kid on the block. It's a system programming language that is blazingly fast and offers powerful high-level abstractions better known from dynamic languages like Ruby or Python. Rust offers memory safety without a Garbage Collector and makes low-level programming easier and more secure. I will give an introductory talk about Rust and show some of its nicest features.
Deep drive into rust programming language
This document introduces Rust and provides an overview of its key concepts. It begins with an introduction to the presenter and agenda. It then covers basic terminology, common system programming errors, why Rust was created, how to install Rust, data types including primitive types, complex data structures, ownership and borrowing rules, lifetimes, and how to get involved in the Rust community. Key concepts discussed include Rust's type system, memory safety features, and package manager.
Python training-course-content
Python training in Chennai at Credo Systemz helps you to get an extensive knowledge of Python programming language. Python classes in Chennai by Credo Systemz is an instructor-led training conducted in Chennai premises.
TDC2016SP - Groovy como você nunca viu
Groovy is an object-oriented scripting language for the Java Virtual Machine (JVM) that can be optionally statically typed. It aims to provide features like dynamic typing, closures, builders and other conveniences while retaining compatibility with Java. Groovy code compiles to Java bytecode and passes through several phases including parsing, semantic analysis, instruction selection and class generation. It has a large ecosystem and is designed to integrate well with Java while allowing programming in a more concise, less boilerplate-oriented style.
Rust Programming Language
Rust is a systems programming language developed by Mozilla that provides memory safety without garbage collection. It uses concepts of ownership and borrowing to ensure memory safety issues like use-after-free do not occur. Rust offers zero-cost abstractions meaning abstraction mechanisms like generics have little to no performance overhead. It allows high levels of concurrency but requires memory references be valid during their entire lifetime.
What's hot (8)
Why Rust? - Matthias Endler - Codemotion Amsterdam 2016
Why Rust? - Matthias Endler - Codemotion Amsterdam 2016
Similar to Migrating our monolith to Python 3
Python Linters at Scale.pdf
Black, Flake8, isort, and Mypy are useful Python linters but it’s challenging to use them effectively at scale in the case of multiple codebases, in a large codebase, or with many developers. Manually managing consistent linter versions and configurations across codebases requires endless effort. Linter analysis on large codebases is slow. Linters may slow down developers by asking them to fix trivial issues. Running linters in distributed CI jobs makes it hard to understand the overall developer experience.
To handle these scale challenges, we developed a reusable linter framework that releases new linter updates automatically, reuses consistent configurations, runs linters on only updated code to speedup runtime, collects logs and metrics to provide observability, and builds auto fixes for common linter issues. Our linter runs are fast and scalable. Every week, they run 10k times on multiple millions of lines of code in over 25 codebases, generating 25k suggestions for more than 200 developers. Its autofixes also save 20 hours of developer time every week.
In this talk, we’ll walk you through popular Python linters and configuration recommendations, and we will discuss common issues and solutions when scaling them out. Using linters more effectively will make it much easier for you to apply best practices and more quickly write better code.
Testing Django APIs
This document summarizes the testing approach and strategies used for an application built with Django and Django Rest Framework. It discusses the use of both unit and integration tests, along with different testing frameworks like Django's built-in test tools, pytest, and libraries like pytest-django and pytest-xdist. It also covers strategies like database fixtures, parameterized testing, and mocking HTTP requests. The document concludes by discussing plans to improve testing in the future such as enabling parallel test runs and adding performance regression testing.
Koji and pulp
The document discusses Koji, Pulp, and errata management in CentOS. Koji is an rpm building automation tool that allows centralized building and tracking of packages. Pulp is a Python application for repository management that allows mirroring, errata management, and pushing updates to consumer systems. The document notes that CentOS has an unusual errata management process that relies on mailing lists rather than traditional metadata files, but includes the centos-errata-builder tool.
Continuous delivery of Windows micro services in the cloud
Chef london meetup talk about windows micro services and continuous delivery at justgiving, and the journey we took
Launchpad: Lessons Learnt
Launchpad is a Python-based web application with over 350k lines of code and 25k tests. It was developed primarily by Canonical using Zope 3, Twisted, and other libraries. The presentation discusses best practices from Launchpad's development including extensive testing, code reviews, use of branches, and interfaces. It also covers challenges like slow tests and database evolution.
Migrating python.org to buildbot 9 and python 3
Craig Rodrigues presented on migrating Python.org's build system from Buildbot 8 to Buildbot 9 and from Python 2 to Python 3. Buildbot is open source build automation software used by Python.org to build and test Python (CPython). The migration involved porting Buildbot and its dependencies like Twisted to be compatible with Python 3, porting Python.org's Buildbot configuration, and working with an ops team to complete the migration. This updated Python.org's infrastructure to a newer, actively supported version of Buildbot that enables new features and works with both Python 2 and Python 3.
How we realized SOA by Python at PyCon JP 2015
The document discusses how to realize a service-oriented architecture (SOA) using Python. It describes a case study of migrating a newspaper company's monolithic web application to microservices. Key practices covered include using Django and Django REST Framework to easily create REST APIs, tox for testing across environments, automated deployments to ElasticBeanstalk, and tools like Sentry and NewRelic for monitoring the distributed system of microservices. While SOA provides benefits, it also requires effort to manage the increased number of independent services.
Go at Skroutz
Presentation from the 4th Athens Gophers Meetup.
At a glance we present:
- why we introduced a new language in the organization and why that
was Go
- how we approached the transition
- some of the projects we built in Go
- the challenges we faced and the lessons we learned in the process
Introduction to GitHub Actions
Github Actions enables you to create custom software development lifecycle workflows directly in your Github repository. These workflows are made out of different tasks so-called actions that can be run automatically on certain events.
The Five Stages of Enterprise Jupyter Deployment
Meetup talk from May 30, 2018.
Jupyter notebooks are an important tool for data science. For a single user on a laptop, these notebooks are a simple, straightforward tool. But Jupyter in the enterprise is a much more complex affair. Enterprises have large teams of data scientists who need to run their notebooks atop scalable compute infrastructure with secure, audited access to massive, proprietary data sets; all while keeping hardware costs down.
Here at IBM’s Center for Open-Source Data and AI Technologies, we’ve seen multiple enterprise rollouts of Jupyter notebooks, both first-hand, in IBM products and services; and second-hand, in our discussions with other members of the Jupyter community.
In this talk, we merge together the stories of these projects and walk through the process of deploying high-performance, secure, mulitentant Jupyter notebooks in an enterprise setting. Our goal is here is inform others who may be at the beginning of this journey of what is coming and how to navigate the challenges ahead.
Along the way, we answer five important questions: What are Jupyter notebooks? What makes Jupyter so attractive to data scientists? Why is deploying Jupyter in the enterprise difficult? What are your deployment options today? And, what are the tradeoffs of those approaches?
We’ll finish with a description of how how IBM and other members of the Jupyter community are working towards reducing those tradeoffs with the Jupyter Enterprise Gateway project. Finally, we’ll give a demonstration of multitenant Jupyter notebooks in action.
This talk is aimed at enterprise architects who need to support growing data science teams with multi-user deployments of Jupyter. No knowledge of data science is required.
Python: Thanks for the memories
Typically, Python software engineers don’t necessarily care about how the language handles memory. However, sometimes it’s very useful to understand what’s going on under the hood. In this talk, I’ll give you a brief overview of how Python manages memory and some useful tips and tricks that you may not already know.
Programming with Python - Basic
s[:] makes a copy.
– s[:]
– slice(0, 3)
– s[slice(0, 3)]
– s[0:3]
– s[:3]
– s[3:]
– s[:-3]
So slice is more flexible.
– s[start:stop:step]
– s[::-1] # reverse
– help(slice)
– s = 'Hello World'
– s[5:11:2]
– 'oWl'
– s[::2]
– 'Hlo rd'
– s[-
Git 1
This document provides an overview and introduction to using the version control system Git. It begins with explaining what version control is and why it is useful for collaboration, storing versions, restoring previous versions, understanding changes, and backup. It then discusses using Git from the command line versus a GUI. The remainder of the document outlines Git's basic workflow, including configuring Git, committing related changes with good commit messages, using branches extensively, never committing half-done work by using stash, keeping the remote in sync, and merging changes. It concludes by inviting the reader to join them in learning more about Git.
Dmytro Dziubenko "Developer's toolchain"
There is not enough knowledge of programming languages to be good developers nowadays. It is important to understand the processes of continually improving the quality of the code and its performance. The ability to configure the development processes in a team more valuable than just writing code. To do this, you need to start using new tools or improve existing ones. We will go through all stages of development and find out what can be done better.
A Python Petting Zoo
This document discusses Knewton's use of ZooKeeper and PettingZoo to implement distributed machine learning on a Python cluster. It begins by explaining what ZooKeeper is and how it provides services for distributed synchronization. It then discusses the state of ZooKeeper libraries for Python, including incomplete bindings and lack of high-level recipes. PettingZoo is introduced as Knewton's library that implements common ZooKeeper recipes for Python, allowing their machine learning models to be sharded and scaled across multiple machines. Distributed discovery, distributed bags, leader queues, and role matching are highlighted as key recipes that enable dynamic reconfiguration and load balancing of their distributed system.
OpenStack Cinder On-Boarding Education - Boston Summit - 2017
These slides were presented at the Boston Summit for people interested in learning how to start contributing to OpenStack's Block Storage project, Cinder. Includes an overview of Cinder's architecture, an introduction to our development processes and a description of our code tree.
Spark and S3 with Ryan Blue
Netflix’s Big Data Platform team manages data warehouse in Amazon S3 with over 60 petabytes of data and writes hundreds of terabytes of data every day. At this scale, output committers that create extra copies or can’t handle task failures are no longer practical. This talk will explain the problems that are caused by the available committers when writing to S3, and show how Netflix solved the committer problem.
In this session, you’ll learn:
– Some background about Spark at Netflix
– About output committers, and how both Spark and Hadoop handle failures
– How HDFS and S3 differ, and why HDFS committers don’t work well
– A new output committer that uses the S3 multi-part upload API
– How you can use this new committer in your Spark applications to avoid duplicating data
GeoServer Developers Workshop
GeoServer is an amazing project, and an amazing project to work on!
Please attend this workshop to:
* Get Started with the GeoServer codebase
* Orientation with a Tour of the GeoServer architecture
* Introduction the service dispatch framework, includin creating your own service
* Built chain and test facilities
* Create a custom function for use with map styling
* Create a custom process for use with style transformations and web processing service
* Anatomy of a successful pull request
Attendees will build their own GeoServer, learn a bit about how our community operates, and enjoy extending the base application.
If you are a developer looking to support GeoServer, or join us for a sprint or bug-stomp, this workshop is great introduction.
This course features hands-on development. We encourage and expect you to bring your favourite Java development environment.
For a good time with open source join GeoServer today!
Hacktoberfest 2022
Hacktoberfest is a month long event that encourages participation in open source software projects through contributions like code, documentation, translation, and outreach. Participants who make four approved pull requests to public GitHub repositories between October 1-31 will earn a limited edition T-shirt. The document provides instructions on how to register and participate, defines key open source concepts like Git and GitHub, and outlines useful Git commands for initializing a repository, cloning a remote repository, staging and committing changes, and pushing commits to remote repositories.
Similar to Migrating our monolith to Python 3 (20)
Continuous delivery of Windows micro services in the cloud
Continuous delivery of Windows micro services in the cloud
OpenStack Cinder On-Boarding Education - Boston Summit - 2017
OpenStack Cinder On-Boarding Education - Boston Summit - 2017
Recently uploaded
UI5con 2024 - Bring Your Own Design System
How do you combine the OpenUI5/SAPUI5 programming model with a design system that makes its controls available as Web Components? Since OpenUI5/SAPUI5 1.120, the framework supports the integration of any Web Components. This makes it possible, for example, to natively embed own Web Components of your design system which are created with Stencil. The integration embeds the Web Components in a way that they can be used naturally in XMLViews, like with standard UI5 controls, and can be bound with data binding. Learn how you can also make use of the Web Components base class in OpenUI5/SAPUI5 to also integrate your Web Components and get inspired by the solution to generate a custom UI5 library providing the Web Components control wrappers for the native ones.
一比一原版(USF毕业证)旧金山大学毕业证如何办理
USF硕士毕业证成绩单【微信95270640】一比一伪造旧金山大学文凭@假冒USF毕业证成绩单+Q微信95270640办理USF学位证书@仿造USF毕业文凭证书@购买旧金山大学毕业证成绩单USF真实使馆认证/真实留信认证回国人员证明
#一整套旧金山大学文凭证件办理#—包含旧金山大学旧金山大学本科毕业证成绩单学历认证|使馆认证|归国人员证明|教育部认证|留信网认证永远存档教育部学历学位认证查询办理国外文凭国外学历学位认证#我们提供全套办理服务。
一整套留学文凭证件服务：
一：旧金山大学旧金山大学本科毕业证成绩单毕业证 #成绩单等全套材料从防伪到印刷水印底纹到钢印烫金
二：真实使馆认证（留学人员回国证明）使馆存档
三：真实教育部认证教育部存档教育部留服网站永久可查
四：留信认证留学生信息网站永久可查
国外毕业证学位证成绩单办理方法：
1客户提供办理旧金山大学旧金山大学本科毕业证成绩单信息：姓名生日专业学位毕业时间等（如信息不确定可以咨询顾问：我们有专业老师帮你查询）；
2开始安排制作毕业证成绩单电子图；
3毕业证成绩单电子版做好以后发送给您确认；
4毕业证成绩单电子版您确认信息无误之后安排制作成品；
5成品做好拍照或者视频给您确认；
6快递给客户（国内顺丰国外DHLUPS等快读邮寄）。
教育部文凭学历认证认证的用途：
如果您计划在国内发展那么办理国内教育部认证是必不可少的。事业性用人单位如银行国企公务员在您应聘时都会需要您提供这个认证。其他私营 #外企企业无需提供！办理教育部认证所需资料众多且烦琐所有材料您都必须提供原件我们凭借丰富的经验帮您快速整合材料让您少走弯路。
实体公司专业为您服务如有需要请联系我: 微信95270640奈一次次令他失望山娃今年岁上五年级识得很多字从走出小屋开始山娃就知道父亲的家和工地共有一个很动听的名字——天河工地的底层空空荡荡很宽阔很凉爽在地上铺上报纸和水泥袋父亲和工人们中午全睡在地上地面坑坑洼洼山娃曾多次绊倒过也曾有长铁钉穿透凉鞋刺在脚板上但山娃不怕工地上也常有五六个从乡下来的小学生他们的父母亲也是高楼上的建筑工人小伙伴来自不同省份都操着带有浓重口音的普通话可不知为啥山娃不仅很快与他们熟识了
Mobile app Development Services | Drona Infotech
Drona Infotech is one of the Best Mobile App Development Company In Noida Maintenance and ongoing support. mobile app development Services can help you maintain and support your app after it has been launched. This includes fixing bugs, adding new features, and keeping your app up-to-date with the latest
Visit Us For :
Artificia Intellicence and XPath Extension Functions
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
Enums On Steroids - let's look at sealed classes !
These are slides for my session"Enums On Steroids - let's look at sealed classes !" - delivered among others, on Devoxx UK 2024 conference
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
Regarding mobile operating systems, two major players dominate our thoughts: Android and iPhone. With Android leading the market, software development companies are focused on delivering apps compatible with this OS. Ensuring an app's functionality across various Android devices, OS versions, and hardware specifications is critical, making Android app testing essential.
Using Query Store in Azure PostgreSQL to Understand Query Performance
Microsoft has added an excellent new extension in PostgreSQL on their Azure Platform. This session, presented at Posette 2024, covers what Query Store is and the types of information you can get out of it.
Requirement Traceability in Xen Functional Safety
Using OpenFastTrace to trace requirements in Xen functional safety
在线购买加拿大英属哥伦比亚大学毕业证本科学位证书原版一模一样
原版一模一样【微信：741003700 】【加拿大英属哥伦比亚大学毕业证本科学位证书】【微信：741003700 】学位证，留信认证（真实可查，永久存档）offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原海外各大学 Bachelor Diploma degree, Master Degree Diploma
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
No one wants their application to drag like a car stuck in the slow lane! Yet it’s all too common to encounter bumpy, pothole-filled solutions that slow the speed of any application. Symfony apps are not an exception.
In this talk, I will take you for a spin around the performance racetrack. We’ll explore common pitfalls - those hidden potholes on your application that can cause unexpected slowdowns. Learn how to spot these performance bumps early, and more importantly, how to navigate around them to keep your application running at top speed.
We will focus in particular on tuning your engine at the application level, making the right adjustments to ensure that your system responds like a well-oiled, high-performance race car.
Energy consumption of Database Management - Florina Jonuzi
Presentation from Florina Jonuzi at the GSD Community Stage Meetup on June 06, 2024
Malibou Pitch Deck For Its €3M Seed Round
French start-up Malibou raised a €3 million Seed Round to develop its payroll and human resources
management platform for VSEs and SMEs. The financing round was led by investors Breega, Y Combinator, and FCVC.
Transform Your Communication with Cloud-Based IVR Solutions
Discover the power of Cloud-Based IVR Solutions to streamline communication processes. Embrace scalability and cost-efficiency while enhancing customer experiences with features like automated call routing and voice recognition. Accessible from anywhere, these solutions integrate seamlessly with existing systems, providing real-time analytics for continuous improvement. Revolutionize your communication strategy today with Cloud-Based IVR Solutions. Learn more at: https://thesmspoint.com/channel/cloud-telephony
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Presentation from Ghazal Aakel and Eric Jochum at the GSD Community Stage Meetup on June 6th, 2024
Measures in SQL (SIGMOD 2024, Santiago, Chile)
SQL has attained widespread adoption, but Business Intelligence tools still use their own higher level languages based upon a multidimensional paradigm. Composable calculations are what is missing from SQL, and we propose a new kind of column, called a measure, that attaches a calculation to a table. Like regular tables, tables with measures are composable and closed when used in queries.
SQL-with-measures has the power, conciseness and reusability of multidimensional languages but retains SQL semantics. Measure invocations can be expanded in place to simple, clear SQL.
To define the evaluation semantics for measures, we introduce context-sensitive expressions (a way to evaluate multidimensional expressions that is consistent with existing SQL semantics), a concept called evaluation context, and several operations for setting and modifying the evaluation context.
A talk at SIGMOD, June 9–15, 2024, Santiago, Chile
Authors: Julian Hyde (Google) and John Fremlin (Google)
https://doi.org/10.1145/3626246.3653374
Using Xen Hypervisor for Functional Safety
An update on making Xen hypervisor functionally safe and enhancing its usage in automotive and industrial use cases
Top 9 Trends in Cybersecurity for 2024.pptx
Security and risk management (SRM) leaders face disruptions on technological, organizational, and human fronts. Preparation and pragmatic execution are key for dealing with these disruptions and providing the right cybersecurity program.
How Can Hiring A Mobile App Development Company Help Your Business Grow?
ToXSL Technologies is an award-winning Mobile App Development Company in Dubai that helps businesses reshape their digital possibilities with custom app services. As a top app development company in Dubai, we offer highly engaging iOS & Android app solutions. https://rb.gy/necdnt
GreenCode-A-VSCode-Plugin--Dario-Jurisic
Presentation about a VSCode plugin from Dario Jurisic at the GSD Community Stage meetup
Recently uploaded (20)
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
Migrating our monolith to Python 3
- 1. Migrating our monolith to Python 3 Django Bulgaria Meetup Group 18th of November, 2019
- 2. About Me Ilian Iliev Platform Engineer at Lifesum ilian@ilian.io www.ilian.io
- 3. The Status Quo ● One monolith running Python 2.7 ● 7 years old codebase ● MySQL, Django, Celery, Redis ● 38M users all around the world
- 4. The Plan ● Make the code Python 3 compatible ● Have it run on our test environments for a while ● Do a round of manual testing ● Ship it
- 5. I. Set it up ● Not all of our dependencies were Python 3 compatible ● Suds ● Add tests to mock the calls ● Replace it with zeep ● Verify
- 6. II. Make it run ● Failing imports ● Local imports ● Six ○ Moved libs ○ six.iteritems ○ Six.text_type ● Setup CircleCI to test both 2.7 and 3.6
- 7. III. Make tests pass ● six.iteritems ● six.text_type ● hashlib requires bytes ● exception.message ● xrange ● dict_keys ● filter ● Comparing with None ● round behaving differently
- 8. IV. Upgrade migrations & Manual test ● Custom column names existed as bytes in migrations ● Same for choices and help_text ● Some third party apps generated new migrations ● Manual tests
- 9. V. Release
- 12. VI. Rollback and aftermath ● Sentry could not handle the error load ● Redis is an issue ○ Via Django ○ Direct usage ● Python 3 incompatible code not covered by tests ● filter ● Lesson learned - Monday morning == Friday evening
- 13. VI. Second Try
- 15. Questions?
- 16. Thank you
Editor's Notes
- Code was PEP8 compatible, which helped a lot
- We started in February
- Six - fixes moved modules compatibility Local import - In Python 3 you need to explicitly specify if you want module or package level import. We also had print statements
- Remember filter, we will get back to it later
- Remember filter, we will get back to it later
- Remember filter, we will get back to it later
- Remember filter, we will get back to it later
- Remember filter, we will get back to it later
- Remember filter, we will get back to it later
- Remember filter, we will get back to it later
- Remember filter, we will get back to it later