This document summarizes Instagram's experience with Python and Django. It discusses why Instagram chose Python and Django, how they have scaled their Python/Django stack to support billions of users, and their process for migrating to Python 3 while minimizing disruptions. Key points include Instagram leveraging tools to profile performance bottlenecks, moving critical components to C/C++, and taking a gradual approach to migrating services to Python 3 one endpoint at a time over several months.
Python offers several tool and public services that simplify starting and maintaining an open source project. This presentation show cases some of the most helpful one and explains the process, beginning with an empty folder and finishing with a published PyPI package.
Easy contributable internationalization process with Sphinx @ PyCon APAC 2016Takayuki Shimizukawa
Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favorite editors or services to translate your sphinx docs.
In this slide, I'll explain 3 things; (1) entire process to translate sphinx docs. (2) automation mechanism for the process. (3) tips for writing docs and translating.
Python offers several tool and public services that simplify starting and maintaining an open source project. This presentation show cases some of the most helpful one and explains the process, beginning with an empty folder and finishing with a published PyPI package.
Easy contributable internationalization process with Sphinx @ PyCon APAC 2016Takayuki Shimizukawa
Sphinx can extract paragraphs from sphinx document and store them into gettext format translation catalog files.
Gettext format translation catalog is easy to translate from one language to other languages.
Also Sphinx support internationalization by using such catalog files.
You can use your favorite editors or services to translate your sphinx docs.
In this slide, I'll explain 3 things; (1) entire process to translate sphinx docs. (2) automation mechanism for the process. (3) tips for writing docs and translating.
Streaming Trend Discovery: Real-Time Discovery in a Sea of Events with Scott ...Databricks
Time is the one thing we can never get in front of. It is rooted in everything, and “timeliness” is now more important than ever especially as we see businesses automate more and more of their processes. This presentation will scratch the surface of streaming discovery with a deeper dive into the telecommunications space where it is normal to receive billions of events a day from globally distributed sub-systems and where key decisions “must” be automated.
We’ll start out with a quick primer on telecommunications, an overview of the key components of our architecture, and make a case for the importance of “ringing”. We will then walk through a simplified solution for doing windowed histogram analysis and labeling of data in flight using Spark Structured Streaming and mapGroupsWithState. I will walk through some suggestions for scaling up to billions of events, managing memory when using the spark StateStore as well as how to avoid pitfalls with the serialized data stored there.
What you’ll learn:
1. How to use the new features of Spark 2.2.0 (mapGroupsWithState / StateStore)
2. How to bucket and analyze data in the streaming world
3. How to avoid common Serialization mistakes (eg. how to upgrade application code and retain stored state)
4. More about the telecommunications space than you’ll probably want to know!
5. Learn a new approach to building applications for enterprise and production.
Assumptions:
1. You know Scala – or want to know more about it.
2. You have deployed spark to production at your company or want to
3. You want to learn some neat tricks that may save you tons of time!
Take Aways:
1. Fully functioning spark app – with unit tests!
JS Fest 2018. Никита Галкин. Микросервисная архитектура с переиспользуемыми к...JSFestUA
Нарушение DRY принципа особенно часто возникает в микросервисах. Чтобы избежать этой проблемы, вы можете использовать повторно используемые компоненты, например, приватные пакеты npm. Лучшие практики, которые помогут вам достичь этого включают в себя паттерн ECB для организации кода, манифест 12-ти факторного приложения, использование генерации кода. В нашем проекте мы используем технический стек на основе Node.js, Docker, RabbitMQ, но идеи из этого доклада могут быть использованы для любого технического стека микросервисов
Python is a great programming language. It is a complete tutorial of using this programming language.
This slides is split into two parts, and it is the first part. Another part is at: http://www.slideshare.net/moskytw/programming-with-python-adv.
The PyConTW (http://tw.pycon.org) organizer wishes to improve the quality and quantity of the programming cummunities in Taiwan. Though Python is their core tool and methodology, they know it's worth to learn and communicate with wide-ranging communities. Understanding cultures and ecosystem of a language takes me about three to six months. This six-hour course wraps up what I - an experienced Java developer - have learned from Python ecosystem and the agenda of the past PyConTW.
你可以在以下鏈結找到中文內容：
http://www.codedata.com.tw/python/python-tutorial-the-1st-class-1-preface
DevOps Practice in Nonprofit - Abdurrachman MappujiDevOpsDaysJKT
DevOps practice in nonprofit and how we develop and deploy software to resource-constrained device
Web and commercial space is overrated, there are many other computation that also make a good cause become possible. In this talk I will talk about how we work at nonprofit organization with distributed team consist of developer, volunteer, intern, leaders. This is taken from my background as volunteer in the team. Then we will have a biggest part of the talk discussing about how we employ docker to package our application to resource-constrained and low power device to be deploy in our field (remote area with limited electricity and internet). We create our own linux distro which has some tools embedded for Raspberry Pi. Infrastructure-as-code is somewhat unique in this arena. In this talk we will close our talk with some lesson learned that commercial space can learn from nonprofit like us. How we appreciate contribution accross project. How to attract people to automagically contribute. And many other stuffs. Hope you enjoy.
Fast data mining flow prototyping using IPython NotebookJimmy Lai
Big data analysis requires fast prototyping on data mining process to gain insight into data. In this slides, the author introduces how to use IPython Notebook to sketch code pieces for data mining stages and make fast observations easily.
Austin Python Meetup 2017: How to Stop Worrying and Start a Project with Pyth...Viach Kakovskyi
The talk "How to stop worrying and start a project with Python 3 " based on my production experience of using the technology. Typical fears of the engineers which use Python 2 are addressed.
Christian Strappazzon - Presentazione Python Milano - Codemotion Milano 2017Codemotion
PyMI: siamo un gruppo di Sviluppatrici, Sviluppatori, Appassionati e Appassionate di Python a Milano. Ci incontriamo una volta al mese in Mikamai/LinkMe. Abbiamo degli eventi ricorrenti e molto apprezzati: "Pillole di Python" e "PyBirra". * Presentazione del gruppo * Python Blueprint: the language, the tools, the packages and the ecosystem.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2nwSwEh.
Marco Bonzanini discusses the process of building data pipelines, e.g. extraction, cleaning, integration, pre-processing of data; in general, all the steps necessary to prepare data for a data-driven product. In particular, he focuses on data plumbing and on the practice of going from prototype to production. Filmed at qconlondon.com.
Marco Bonzanini is Data Scientist and co-organizer of PyData London Meetup.
How to Turn a Pile of Python Files Into an Open Source Project.
You have written some Python code, you think it would be useful to the world, and you would like to give back to the open source world. But where do you start? This talk will give you some guidance on how you can release your project
در این جلسه از کلاس در خصوص تاریخچه پایتون و زبان پایتون صحبت شد
PySec101 Fall 2013 J1E1 By Mohammad Reza Kamalifard
Talk about : Python History and Python language Essentials.
CrashCourse: Python with DataCamp and Jupyter for BeginnersOlga Scrivner
Crash course for beginners is based on Python Introduction by Philip Schowenaars from DataCamp and Jupyter Introduction adapted from Adapted from Pryke, B. (2018). Jupyter Notebook for Beginners: A Tutorial. DataQuest. https://www.dataquest.io/blog/jupyter-notebook-tutorial/
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
NO1 Uk best vashikaran specialist in delhi vashikaran baba near me online vas...Amil Baba Dawood bangali
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
Final project report on grocery store management system..pdfKamal Acharya
In today’s fast-changing business environment, it’s extremely important to be able to respond to client needs in the most effective and timely manner. If your customers wish to see your business online and have instant access to your products or services.
Online Grocery Store is an e-commerce website, which retails various grocery products. This project allows viewing various products available enables registered users to purchase desired products instantly using Paytm, UPI payment processor (Instant Pay) and also can place order by using Cash on Delivery (Pay Later) option. This project provides an easy access to Administrators and Managers to view orders placed using Pay Later and Instant Pay options.
In order to develop an e-commerce website, a number of Technologies must be studied and understood. These include multi-tiered architecture, server and client-side scripting techniques, implementation technologies, programming language (such as PHP, HTML, CSS, JavaScript) and MySQL relational databases. This is a project with the objective to develop a basic website where a consumer is provided with a shopping cart website and also to know about the technologies used to develop such a website.
This document will discuss each of the underlying technologies to create and implement an e- commerce website.
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
10. AND HERE’S WHAT THEY FOUND OUT:
“Friends don’t let friends use RoR”
11. THINGS INSTAGRAM LOVES ABOUT PYTHON
Easy to become productive
Practicality
Easy to grow
engineering team
Popular language
12. THINGS INSTAGRAM LOVES ABOUT DJANGO
Maturity of the
language and
Django framework
Use Django user
model for
supporting 3B+
registered users
13. WE HAVE TAKEN OUR PYTHON+DJANGO STACK QUITE FAR
1 Sharded database support
2
Run our stack across multiple geographically distributed data centers3
Disable garbage-collection to improve memory utilization
16. PYTHON IS SIMPLE AND CLEAN, AND FAVORS PRAGMATISM.
1 Scope the problem, AKA, do simple things first
2 Use proven technology
3 User first: focus on adding value to user facing features
17. Subtitle
AT INSTAGRAM, OUR BOTTLENECK IS
DEVELOPMENT VELOCITY,
NOT PURE CODE EXECUTION
BUT PYTHON IS STILL SLOW, RIGHT?
18.
19. SCALING PYTHON TO SUPPORT USER AND FEATURE GROWTH
20
40
60
80
00
0 2 4 6 8 10 12 14 16 18 20 22 24Server growthUser growth
20. PYTHON EFFICIENCY STRATEGY
1 Build extensive tools to profile and understand perf bottleneck
2 Moving stable, critical components to C/C++, e.g., memcached access
3
Async? New python runtime?4
Cythonization
35. Create separate branch?
MIGRATION OPTIONS
• Branch sync overhead, error prone
• Merging back will be a risk
• Lose the opportunity to educate
36. MIGRATION OPTIONS
One endpoint at a time?Create separate branch?
• Common modules across end points
• Context switch for developers
• Overhead of managing separate pools
37. MIGRATION OPTIONS
One endpoint at a time?Create separate branch? Micro services?
• Massive code restructuring
• Higher latency
• Deployment complexity
57. from __future__ import absolute_import
from __future__ import print_function
from __future__ import division
from __future__ import unicode_literals
CHALLENGE: UNICODE/STR/BYTES
58. from __future__ import absolute_import
from __future__ import print_function
from __future__ import division
from __future__ import unicode_literals
CHALLENGE: UNICODE/STR/BYTES
from __future__ import absolute_import
from __future__ import print_function
from __future__ import division
from __future__ import unicode_literals
59. CHALLENGE: UNICODE/STR/BYTES
mymac = hmac.new(‘abc’)
TypeError: key: expected bytes or bytearray, but got 'str'
value = ‘abc’
if isinstance(value, six.text_type):
value = value.encode(encoding=‘utf-8’)
mymac = hmac.new(value)
Error
Fix
68. 1 CYTHON_SOURCES = [a.pyx, b.pyx, c.pyx]
2 builds = map(BuildProcess, CYTHON_SOURCES)
3 while any(not build.done() for build in builds):
4 pending = [build for build in builds if not build.started()]
<do some work>
CHALLENGE: ITERATOR
69. 1 CYTHON_SOURCES = [a.pyx, b.pyx, c.pyx]
2 builds = map(BuildProcess, CYTHON_SOURCES)
3 while any(not build.done() for build in builds):
4 pending = [build for build in builds if not build.started()]
<do some work>
CHALLENGE: ITERATOR
70. 1 CYTHON_SOURCES = [a.pyx, b.pyx, c.pyx]
2 builds = map(BuildProcess, CYTHON_SOURCES)
3 while any(not build.done() for build in builds):
4 pending = [build for build in builds if not build.started()]
<do some work>
CHALLENGE: ITERATOR
71. 1 CYTHON_SOURCES = [a.pyx, b.pyx, c.pyx]
2 builds = list(map(BuildProcess, CYTHON_SOURCES))
3 while any(not build.done() for build in builds):
4 pending = [build for build in builds if not build.started()]
<do some work>
CHALLENGE: ITERATOR
80. Video View Notification
Save Draft
Comment Filtering
Story Viewer Ranking
First Story Notification
Self-harm Prevention
Live
81. MOTIVATION: TYPE HINTS
Type hints - 2% done
def compose_from_max_id(max_id: Optional[str]) -> Optional[str]:
MyPy and typeshed contribution
Tooling - collect data and suggest type hints
