Automate cleaning code in few easy steps!

•

0 likes•6 views

How annoying is it to find out that everything went to hell on the pipeline because you forgot to run the formatters? Don’t waste precious time and learn how it is possible to automate these little things, but most importantly understand why it is important to have them in your code! Have you ever been in a situation where you check your pipelines to see if he’s finished running the tests and you find that he hasn’t even started because you forgot to run the linters locally? I contribute to the development of PyCon Italia, Strawberry GraphQL and Poetry Package Manager and coding with these organizations I discovered fantastic tools such as pre-commit, Poetry and many others. In this talk I would like to share with you my findings, think about the pros and cons and understand a little bit better why you should start using them. The Author: Ester Beltrami I'm a Python Developer living in London who is passionate about clean code and architecture. I try to give my contribution to the development of PyCon Italia, Strawberry GraphQL and Poetry Package Manager.

Technology

Automate cleaning
code in few easy
steps!
Ester Beltrami
Twitter: @esterbeltrami
GitHub: github.com/estyxx

Ester
•PyCon Italia
organizer
•Strawberry-graphql
core-dev
•Poetry Package
manager contributor

CLEARANCE JUST
GOT BETTER
New lines added
Up to 50% off

Some nice hooks to include!
• pyupgrade to automatically upgrade syntax for newer
versions
• flake8-eradicate to find commented out (or so called "dead")
code
• autoflake to removes unused imports and unused variables

Django hooks!
• pre-commit-hooks-django to not forget to commit the
migrations files!!
• django-upgrade to upgrade Django syntax

Identify issues
*before* the
code review

We still have
to remember
to pre-
commit
install
but*

Run pre-commit on pipeline
or use pre-commit.ci

Can we
remove some
Makefile
commands
maybe?

What on Earth is a pyproject.toml?
It’s the specified file format of PEP 518 which
contains the build system requirements of Python
projects.
We can specify configuration data within the [tool]
section!

Poetry
Python packaging and dependency
management made easy the new,
standardized pyproject.toml.
In other words, it replace setup.py,
requirements.txt, setup.cfg,
MANIFEST.in and Pipfile.
Documentation: https://python-
poetry.org/

Poetry 1.2 is coming!
Introduce the ability do
group dependencies in
groups! Not only the
default `dev` one.
And plugins to add your
own functionality to
Poetry!

Python
Developers
Survey 2021
Results:
Poetry is
gaining
popularity!!!
https://bit.ly/3nXjcQm

sync
Poetry &
pre-
commit,
how to?
Our last hook:
sync_with_poetry

Thank you!
Twitter: @esterbeltrami
GitHub: github.com/estyxx

Have you ever been in the situation where you check the pipeline to see if it has finished and you discover it didn’t even start because you forgot to run formatters and linters locally? or when the code you’re reading isn’t formatted properly and you’re dying to stop doing everything you’re doing and fix it once for all this crap? There are wonderful tools out there to avoid this problem, that runs every commit or push so you don’t forget anymore! Learn how to configure them for your needs and more importantly learn why you need them now!!

PyCourse - Self driving python course

Eran Shlomo

Github for Serious Business Professionalzwheller

A Ci Experience

Umut IŞIK

Porting a legacy app to python 3

Mark Rees

How to Upgrade to the Newest Shiniest Django Version

Susan Tan

If you’re coding Django, you will inevitably need to upgrade your project’s version of Django to a newer version. Is this process really tedious, you ask? YES. But before you jump off a cliff in despair, come to my talk and learn time-saving strategies on this upgrade process. I'll go over my work in upgrading a web application from Django 1.3 to 1.7 and the lessons that I've learned in that arduous process.

Detecting secrets in code committed to gitlab (in real time)

Chandrapal Badshah

The Information Technology have led us into an era where the production, sharing and use of information are now part of everyday life and of which we are often unaware actors almost: it is now almost inevitable not leave a digital trail of many of the actions we do every day; for example, by digital content such as photos, videos, blog posts and everything that revolves around the social networks (Facebook and Twitter in particular). Added to this is that with the "internet of things", we see an increase in devices such as watches, bracelets, thermostats and many other items that are able to connect to the network and therefore generate large data streams. This explosion of data justifies the birth, in the world of the term Big Data: it indicates the data produced in large quantities, with remarkable speed and in different formats, which requires processing technologies and resources that go far beyond the conventional systems management and storage of data. It is immediately clear that, 1) models of data storage based on the relational model, and 2) processing systems based on stored procedures and computations on grids are not applicable in these contexts. As regards the point 1, the RDBMS, widely used for a great variety of applications, have some problems when the amount of data grows beyond certain limits. The scalability and cost of implementation are only a part of the disadvantages: very often, in fact, when there is opposite to the management of big data, also the variability, or the lack of a fixed structure, represents a significant problem. This has given a boost to the development of the NoSQL database. The website NoSQL Databases defines NoSQL databases such as "Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open source and horizontally scalable." These databases are: distributed, open source, scalable horizontally, without a predetermined pattern (key-value, column-oriented, document-based and graph-based), easily replicable, devoid of the ACID and can handle large amounts of data. These databases are integrated or integrated with processing tools based on the MapReduce paradigm proposed by Google in 2009. MapReduce with the open source Hadoop framework represent the new model for distributed processing of large amounts of data that goes to supplant techniques based on stored procedures and computational grids (step 2). The relational model taught courses in basic database design, has many limitations compared to the demands posed by new applications based on Big Data and NoSQL databases that use to store data and MapReduce to process large amounts of data. Course Website http://pbdmng.datatoknowledge.it/ Contact me to download the slides

Big feature - small sprint

Igor Goldshmidt

My talk presentation from SQA Days 21 conference. During last two years I've been working in Agile teams and quite often have come across the situation when the feature is pretty big, but they still want to release it at the end of the sprint. That is in spite of the fact, that there is a great risk of violation of the timing and poor product quality. So what should we, quality guards, do when eventually everything depends on us? My report presents the analysis of the development stages from the testing point of view in such a difficult situation and provides tips for success.

Short Introduction of software engineering for bioinformatics

丈宮本

Configuring python linting to be part of CI/CD using GitHub actions

iamtodor

Build software like a bag of marbles, not a castle of LEGO®

Hannes Lowette

If you have ever played with LEGO®, you will know that adding, removing or changing features of a completed castle isn’t as easy as it seems. You will have to deconstruct large parts to get to where you want to be, to build it all up again afterwards. Unfortunately, our software is often built the same way. Wouldn’t it be better if our software behaved like a bag of marbles? So you can just add, remove or replace them at will? Most of us have taken different approaches to building software: a big monolith, a collection of services, a bus architecture, etc. But whatever your large scale architecture is, at the granular level (a single service or host), you will probably still end up with tightly couple code. Adding functionality means making changes to every layer, service or component involved. It gets even harder if you want to enable or disable features for certain deployments: you’ll need to wrap code in feature flags, write custom DB migration scripts, etc. There has to be a better way! So what if you think of functionality as loose feature assemblies? We can construct our code in such a way that adding a feature is as simple as adding the assembly to your deployment, and removing it is done by just deleting the file. We would open the door for so many scenarios! In this talk, I will explain how to tackle the following parts of your application to achieve this goal: WebAPI, Entity Framework, Onion Architecture, IoC and database migrations. And most of all, when you would want to do this. Because… ‘it depends’.

Be a Happier Developer with Git / Productive Team #gettinggitright

Shunsuke (Sean) Osawa

Writing Your First Plugin

George Ornbo

Packaging perl (LPW2010)

p3castro

ppt notes for python language variable data types

SukhpreetSingh519414

Creating a reasonable project boilerplate

Stanislav Petrov

Steelcon 2014 - Process Injection with Python

infodox

This is the slides to accompany the talk given by Darren Martyn at the Steelcon security conference in July 2014 about process injection using python. Covers using Python to manipulate processes by injecting code on x86, x86_64, and ARMv7l platforms, and writing a stager that automatically detects what platform it is running on and intelligently decides which shellcode to inject, and via which method. The Proof of Concept code is available at https://github.com/infodox/steelcon-python-injection

Hadoop Demystified + Automation Smackdown! Austin JUG June 24 2014

datafundamentals

Django: Beyond Basicsarunvr

Salesforce CI (Continuous Integration) - SFDX + Bitbucket Pipelines

Abhinav Gupta

Exploring Salesforce Continous Integration options, with external and internal CI Tools like CircleCI, TravisCI, Bitbucket, Jenkins, and Gitlab. What are the best practices when planning to go for CI in Salesforce, like - Active Scratch Org Limit - Which branches to pick for running CI scripts - Cost and security of external vs internal tools. - Branch Permissions - Other hidden features like integrations, env variable, etc.

Quantum Computing: Current Landscape and the Future Role of APIs

Vlad Stirbu

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

Similar to Automate cleaning code in few easy steps!

enjoy sublime pluginLim Hyung-Joo

Python to goWeng Wei

The New Frontend Toolchain

Bruno Abrantes

Ci of js and apex using jasmine, phantom js and drone io df14

Kevin Poorman

Python testing like a pro by Keith Yang

PYCON MY PLT

Introduction to Git

Colin Su

3 Git

Fabio Fumarola

Big feature - small sprint

Igor Goldshmidt

Short Introduction of software engineering for bioinformatics

丈宮本

Configuring python linting to be part of CI/CD using GitHub actions

iamtodor

Build software like a bag of marbles, not a castle of LEGO®

Hannes Lowette

Be a Happier Developer with Git / Productive Team #gettinggitright

Shunsuke (Sean) Osawa

Writing Your First Plugin

George Ornbo

Packaging perl (LPW2010)

p3castro

ppt notes for python language variable data types

SukhpreetSingh519414

Creating a reasonable project boilerplate

Stanislav Petrov

Steelcon 2014 - Process Injection with Python

infodox

Hadoop Demystified + Automation Smackdown! Austin JUG June 24 2014

datafundamentals

Django: Beyond Basicsarunvr

Salesforce CI (Continuous Integration) - SFDX + Bitbucket Pipelines

Abhinav Gupta

Similar to Automate cleaning code in few easy steps! (20)

enjoy sublime plugin

Python to go

The New Frontend Toolchain

Ci of js and apex using jasmine, phantom js and drone io df14

Python testing like a pro by Keith Yang

Introduction to Git

3 Git

Big feature - small sprint

Short Introduction of software engineering for bioinformatics

Configuring python linting to be part of CI/CD using GitHub actions

Build software like a bag of marbles, not a castle of LEGO®

Be a Happier Developer with Git / Productive Team #gettinggitright

Writing Your First Plugin

Packaging perl (LPW2010)

ppt notes for python language variable data types

Creating a reasonable project boilerplate

Steelcon 2014 - Process Injection with Python

Hadoop Demystified + Automation Smackdown! Austin JUG June 24 2014

Django: Beyond Basics

Salesforce CI (Continuous Integration) - SFDX + Bitbucket Pipelines

Recently uploaded

Quantum Computing: Current Landscape and the Future Role of APIs

Vlad Stirbu

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Enhancing Performance with Globus and the Science DMZ

Globus

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

UiPath Test Automation using UiPath Test Suite series, part 4

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap. The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies. Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques What will you get from this session? 1. Insights into SAP testing best practices 2. Heatmap utilization for testing 3. Optimization of testing processes 4. Demo Topics covered: Execution from the test manager Orchestrator execution result Defect reporting SAP heatmap example with demo Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

Assure Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

Essentials of Automations: The Art of Triggers and Actions in FME

Safe Software

In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation. We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios. Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

91mobiles

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

The Future of Platform Engineering

Jemma Hussein Allen

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

Pushing the limits of ePRTC: 100ns holdover for 100 days

Adtran

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

RESUME BUILDER APPLICATION Project for students

KAMESHS29

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

Peter Spielvogel

Building better applications for business users with SAP Fiori. • What is SAP Fiori and why it matters to you • How a better user experience drives measurable business benefits • How to get started with SAP Fiori today • How SAP Fiori elements accelerates application development • How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities • How SAP Fiori paves the way for using AI in SAP apps

The Metaverse and AI: how can decision-makers harness the Metaverse for their...

Jen Stirrup

The Metaverse is popularized in science fiction, and now it is becoming closer to being a part of our daily lives through the use of social media and shopping companies. How can businesses survive in a world where Artificial Intelligence is becoming the present as well as the future of technology, and how does the Metaverse fit into business strategy when futurist ideas are developing into reality at accelerated rates? How do we do this when our data isn't up to scratch? How can we move towards success with our data so we are set up for the Metaverse when it arrives? How can you help your company evolve, adapt, and succeed using Artificial Intelligence and the Metaverse to stay ahead of the competition? What are the potential issues, complications, and benefits that these technologies could bring to us and our organizations? In this session, Jen Stirrup will explain how to start thinking about these technologies as an organisation.

Recently uploaded (20)

Quantum Computing: Current Landscape and the Future Role of APIs

Epistemic Interaction - tuning interfaces to provide information for AI support

Enhancing Performance with Globus and the Science DMZ

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

UiPath Test Automation using UiPath Test Suite series, part 4

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

Assure Contact Center Experiences for Your Customers With ThousandEyes

Essentials of Automations: The Art of Triggers and Actions in FME

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

The Future of Platform Engineering

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

FIDO Alliance Osaka Seminar: Overview.pdf

Pushing the limits of ePRTC: 100ns holdover for 100 days

The Art of the Pitch: WordPress Relationships and Sales

RESUME BUILDER APPLICATION Project for students

Securing your Kubernetes cluster_ a step-by-step guide to success !

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

The Metaverse and AI: how can decision-makers harness the Metaverse for their...

Automate cleaning code in few easy steps!

1. Automate cleaning code in few easy steps! Ester Beltrami Twitter: @esterbeltrami GitHub: github.com/estyxx

2. Ester •PyCon Italia organizer •Strawberry-graphql core-dev •Poetry Package manager contributor

3. CLEARANCE JUST GOT BETTER New lines added Up to 50% off

4. We write some code

5. We push the code

6. Time to relax and wait

7. We came back to the pipeline and…

8. pre-commit

9. .yaml file config

10. Some nice hooks to include! • pyupgrade to automatically upgrade syntax for newer versions • flake8-eradicate to find commented out (or so called "dead") code • autoflake to removes unused imports and unused variables

11. Django hooks! • pre-commit-hooks-django to not forget to commit the migrations files!! • django-upgrade to upgrade Django syntax

12. not only code quality, also security!

13. What are the benefits?

14. Identify issues *before* the code review

15. It’s not all about python…

16. Why developers don’t like it?

17. but they are slow L

18. From 1.8.0 ‘manual’ stage!

19. We still have to remember to pre- commit install but*

20.

21. Run pre-commit on pipeline or use pre-commit.ci

22. Can we remove some Makefile commands maybe?

23. one ring tool to rule them all

24. What on Earth is a pyproject.toml? It’s the specified file format of PEP 518 which contains the build system requirements of Python projects. We can specify configuration data within the [tool] section!

25. Poetry Python packaging and dependency management made easy the new, standardized pyproject.toml. In other words, it replace setup.py, requirements.txt, setup.cfg, MANIFEST.in and Pipfile. Documentation: https://python- poetry.org/

26. How the pyproject.toml looks like

27. Poetry 1.2 is coming! Introduce the ability do group dependencies in groups! Not only the default `dev` one. And plugins to add your own functionality to Poetry!

28.

29. Python Developers Survey 2021 Results: Poetry is gaining popularity!!! https://bit.ly/3nXjcQm

30. sync Poetry & pre- commit, how to? Our last hook: sync_with_poetry

31. Thank you! Twitter: @esterbeltrami GitHub: github.com/estyxx

Automate cleaning code in few easy steps!

Recommended

Recommended

More Related Content

Similar to Automate cleaning code in few easy steps!

Similar to Automate cleaning code in few easy steps! (20)

Recently uploaded

Recently uploaded (20)

Automate cleaning code in few easy steps!