Flake8 is a Python linter that is fast, simple, and extensible. It can be configured through setup.cfg or .flake8 files to ignore certain checks or select others. The summary recommends using the flake8-bugbear plugin and avoiding all print statements with flake8-print. Linters like Flake8 help find errors, improve code quality, and avoid historical baggage, but one does not need every check and it is okay to build a long ignore list.
Digital RSE: automated code quality checks - RSE group meetingHenry Schreiner
Given at a local RSE group meeting. Covers code quality practices, focusing on Python but over multiple languages, with useful tools highlighted throughout.
This document discusses software quality assurance tooling, focusing on pre-commit. It introduces pre-commit as a tool for running code quality checks before code is committed. Pre-commit allows configuring hooks that run checks and fixers on files matching certain patterns. Hooks can be installed from repositories and support many languages including Python. The document provides examples of pre-commit checks such as disallowing improper capitalization in code comments and files. It also discusses how to configure, run, update and install pre-commit hooks.
This document discusses getting started with a first Python project. It covers installing Python and choosing an IDE, following coding best practices like PEP8 style guidelines, using built-in data structures, testing tools, virtual environments, project structure, and deployment tools like Supervisor. The goal is to help new Python programmers understand the basics of starting their first project.
Package a PyApp as a Flatpak Package: An HTTP Server for Example @ PyCon APAC...Jian-Hong Pan
Flatpak is a framework for distributing desktop applications and supported by most of Linux distributions. This talk shares how to package a HTTP server written in Python as a Flatpak app. And, runs it like a desktop application by launching a browser connecting to the server automatically.
https://hackmd.io/@starnight/Have_an_HTTP_Server_in_Flatpak
Openstack Third-Party CI and the review of a few Openstack Infrastructure pro...Evgeny Antyshev
Presentation for QA:Conference held in Moscow, Russia on April 23rd.
Author: Evgeny Antyshev, Virtuozzo
These slide discover some Openstack Infrastructure tools to ease the task of creating generic CI systems. As an illustration, I setup "model" CI stand to test Libvirt project. Important ideas originated from Openstack testing also mentioned: pre-review integration testing, testing infrastructure in a cloud, project gating, etc.
Take your CI to the next level! Learn how to optimize your pipelines for faster and more efficient builds through parallelization, caching, failing early, and more.
2012 coscup - Build your PHP application on Herokuronnywang_tw
The document discusses deploying PHP applications on Heroku. It provides an overview of Heroku, including that it is a Platform-as-a-Service, was launched in 2007, uses Amazon Web Services, offers many add-ons, allows easy scaling, supports PostgreSQL, and offers some free usage. It then walks through deploying a basic "Hello World" PHP app on Heroku, including creating an app, adding code, committing and pushing to Heroku, and viewing the deployed app.
Flake8 is a Python linter that is fast, simple, and extensible. It can be configured through setup.cfg or .flake8 files to ignore certain checks or select others. The summary recommends using the flake8-bugbear plugin and avoiding all print statements with flake8-print. Linters like Flake8 help find errors, improve code quality, and avoid historical baggage, but one does not need every check and it is okay to build a long ignore list.
Digital RSE: automated code quality checks - RSE group meetingHenry Schreiner
Given at a local RSE group meeting. Covers code quality practices, focusing on Python but over multiple languages, with useful tools highlighted throughout.
This document discusses software quality assurance tooling, focusing on pre-commit. It introduces pre-commit as a tool for running code quality checks before code is committed. Pre-commit allows configuring hooks that run checks and fixers on files matching certain patterns. Hooks can be installed from repositories and support many languages including Python. The document provides examples of pre-commit checks such as disallowing improper capitalization in code comments and files. It also discusses how to configure, run, update and install pre-commit hooks.
This document discusses getting started with a first Python project. It covers installing Python and choosing an IDE, following coding best practices like PEP8 style guidelines, using built-in data structures, testing tools, virtual environments, project structure, and deployment tools like Supervisor. The goal is to help new Python programmers understand the basics of starting their first project.
Package a PyApp as a Flatpak Package: An HTTP Server for Example @ PyCon APAC...Jian-Hong Pan
Flatpak is a framework for distributing desktop applications and supported by most of Linux distributions. This talk shares how to package a HTTP server written in Python as a Flatpak app. And, runs it like a desktop application by launching a browser connecting to the server automatically.
https://hackmd.io/@starnight/Have_an_HTTP_Server_in_Flatpak
Openstack Third-Party CI and the review of a few Openstack Infrastructure pro...Evgeny Antyshev
Presentation for QA:Conference held in Moscow, Russia on April 23rd.
Author: Evgeny Antyshev, Virtuozzo
These slide discover some Openstack Infrastructure tools to ease the task of creating generic CI systems. As an illustration, I setup "model" CI stand to test Libvirt project. Important ideas originated from Openstack testing also mentioned: pre-review integration testing, testing infrastructure in a cloud, project gating, etc.
Take your CI to the next level! Learn how to optimize your pipelines for faster and more efficient builds through parallelization, caching, failing early, and more.
2012 coscup - Build your PHP application on Herokuronnywang_tw
The document discusses deploying PHP applications on Heroku. It provides an overview of Heroku, including that it is a Platform-as-a-Service, was launched in 2007, uses Amazon Web Services, offers many add-ons, allows easy scaling, supports PostgreSQL, and offers some free usage. It then walks through deploying a basic "Hello World" PHP app on Heroku, including creating an app, adding code, committing and pushing to Heroku, and viewing the deployed app.
Helpful pre commit hooks for Python and Djangoroskakori
Pre-commit hooks can help to keep your source code consistent and discover broken code before it makes it into the repository. This lightning talk describes pre-commit hooks that can be helpful when developing with Python, especially when using the Django framework. It also provides consistent example configurations for hooks that have conflicting defaults.
This document discusses packaging Ruby and Rails applications for production. It covers using system packages versus gems, configuration management tools like Chef and Puppet, creating Debian packages, packaging gems, build servers, pain points like outdated Rubygems packages, and ideas for deeper Bundler integration and packaging gems by default. Overall it presents strategies for deploying Ruby applications as system packages for production servers.
Python from zero to hero (Twitter Explorer)Yuriy Senko
This document outlines steps to build a Twitter explorer application using Python and Flask. It begins with setting up the virtual environment and cloning the GitHub repository. It then walks through steps to add basic functionality like configuration, templates, a database with SQLAlchemy ORM, user authentication with Flask plugins, and finally integrating the Twitter API. Each step includes changes to files, dependencies in requirements.txt, and commands to test and view progress. The goal is to create a full-stack web application to explore tweets from the Twitter API.
Explains how Docker and Nix work as deployment solutions, in what ways they are similar and different, and how they can be combined to achieve interesting results.
In this talk, we'll discover how Docker comes to the rescue of the Ops Team, while rebuilding from scratch our monitoring infrastructure. We'll start by quickly describing the challenges, to focus on why and how using docker saved the project. From fixing dependencies and isolation issues, implementing rolling upgrades and new features hot addition, to building a completely modular, scalable and resilient infrastructure, we'll talk about why CI/CD workflows, docker tooling and Docker Swarm were the key to success.
Slides from my DockerCon EU 2017 Talk.
Find the abstract below:
"In this talk, we'll discover how Docker comes to the rescue of the Ops Team, while rebuilding from scratch our monitoring infrastructure. We'll start by quickly describing the challenges, to focus on why and how using docker saved the project. From fixing dependencies and isolation issues, implementing rolling upgrades and new features hot addition, to building a completely modular, scalable and resilient infrastructure, we'll talk about why CI/CD workflows, docker tooling and Docker Swarm were the key to success."
This blog is to provide steps for installation and configuration of Odoo for production environment using Git source and Python environment on an Ubuntu system. To install Odoo 17 on Ubuntu you just follow here steps. Call us!
Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...Shift Conference
What's the first thing you should do when starting a new project...? Setup a good CI system! With Github Actions you can do it in a couple of seconds. You can easily setup a workflow to build your project, test it on different machines, and deploy the results. In this talk we're going to see how you can setup a simple Github Action for your repository and start enjoying it right after.
My talk from DevOpsCon Berlin 2016.
Ansible is a radically simple and lightweight provisioning framework which makes your servers and applications easier to provision and deploy. By orchestrating your application deployments you gain benefits such as documentation as code, testability, continuous integration, version control, refactoring, automation and autonomy of your deployment routines, server and application configuration. Ansible uses a language that approaches plain English, uses SSH and has no agents to install on remote systems. It is the simplest way to automate and orchestrate application deployment, configuration management and continuous delivery.
In this tutorial you will be given an introduction to Ansible and learn how to provision Linux servers with a web-proxy, a database and some other packages. Furthermore we will automate zero downtime deployment of a Java application to a load balanced environment.
Lean Drupal Repositories with Composer and DrushPantheon
Composer is the industry-standard PHP dependency manager that is now in use in Drupal 8 core. This session will show the current best practices for using Composer, drupal-composer, drupal-scaffold, Drush, Drupal Console and Drush site-local aliases to streamline your Drupal 7 and Drupal 8 site repositories for optimal use on teams.
RustPython is a project that implements the Python programming language in Rust. It started in May 2018 and released version 0.3 in September 2023. The standard library has been written in Rust and pip package management works. RustPython can compile to WebAssembly to run Python code in web browsers and has an experimental just-in-time compiler. Several popular Python packages were tested with varying results, with some packages like Django and others not working due to missing Python modules or compiler issues. Packaging Python modules that use Rust is becoming increasingly common using tools like PyO3, with over 50,000 downloads per day of related projects.
PyCon 2013 : Scripting to PyPi to GitHub and MoreMatt Harrison
This document discusses various aspects of developing and distributing Python projects, including versioning, configuration, logging, file input, shell invocation, environment layout, project layout, documentation, automation with Makefiles, packaging, testing, GitHub, Travis CI, and PyPI. It recommends using semantic versioning, the logging module, parsing files with the file object interface, invoking shell commands with subprocess, using virtualenv for sandboxed environments, Sphinx for documentation, Makefiles to automate tasks, setuptools for packaging, and GitHub, Travis CI and PyPI for distribution.
The document summarizes the steps taken to set up a Django project called "he" on Ubuntu. It shows commands used to install Python, virtualenv, Django and other dependencies. Database setup with PostgreSQL is also demonstrated. An app called "board" is created, with a Post model defined and admin configured. Templates are added and the development server is run. Authentication and registration are implemented along with forms to add new posts. The project is developed iteratively through multiple versions.
Modern binary build systems have made shipping binary packages for Python much easier than ever before. This talk discusses three of the most popular build systems for Python packages using the new standards developed for packaging.
This document discusses container technologies including App Container (appc) and rkt. It provides an overview of appc components like the image format, discovery, and executor. It then discusses rkt, an implementation of appc, describing its modular architecture with stages 0-2 and use of systemd and cgroups for isolation. It also touches on rkt security, networking, and integration with systemd and user namespaces.
The document summarizes Henry Schreiner's work on several Python and C++ scientific computing projects. It describes a scientific Python development guide built from the Scikit-HEP summit. It also outlines Henry's work on pybind11 for C++ bindings, scikit-build for building extensions, cibuildwheel for building wheels on CI, and several other related projects.
Helpful pre commit hooks for Python and Djangoroskakori
Pre-commit hooks can help to keep your source code consistent and discover broken code before it makes it into the repository. This lightning talk describes pre-commit hooks that can be helpful when developing with Python, especially when using the Django framework. It also provides consistent example configurations for hooks that have conflicting defaults.
This document discusses packaging Ruby and Rails applications for production. It covers using system packages versus gems, configuration management tools like Chef and Puppet, creating Debian packages, packaging gems, build servers, pain points like outdated Rubygems packages, and ideas for deeper Bundler integration and packaging gems by default. Overall it presents strategies for deploying Ruby applications as system packages for production servers.
Python from zero to hero (Twitter Explorer)Yuriy Senko
This document outlines steps to build a Twitter explorer application using Python and Flask. It begins with setting up the virtual environment and cloning the GitHub repository. It then walks through steps to add basic functionality like configuration, templates, a database with SQLAlchemy ORM, user authentication with Flask plugins, and finally integrating the Twitter API. Each step includes changes to files, dependencies in requirements.txt, and commands to test and view progress. The goal is to create a full-stack web application to explore tweets from the Twitter API.
Explains how Docker and Nix work as deployment solutions, in what ways they are similar and different, and how they can be combined to achieve interesting results.
In this talk, we'll discover how Docker comes to the rescue of the Ops Team, while rebuilding from scratch our monitoring infrastructure. We'll start by quickly describing the challenges, to focus on why and how using docker saved the project. From fixing dependencies and isolation issues, implementing rolling upgrades and new features hot addition, to building a completely modular, scalable and resilient infrastructure, we'll talk about why CI/CD workflows, docker tooling and Docker Swarm were the key to success.
Slides from my DockerCon EU 2017 Talk.
Find the abstract below:
"In this talk, we'll discover how Docker comes to the rescue of the Ops Team, while rebuilding from scratch our monitoring infrastructure. We'll start by quickly describing the challenges, to focus on why and how using docker saved the project. From fixing dependencies and isolation issues, implementing rolling upgrades and new features hot addition, to building a completely modular, scalable and resilient infrastructure, we'll talk about why CI/CD workflows, docker tooling and Docker Swarm were the key to success."
This blog is to provide steps for installation and configuration of Odoo for production environment using Git source and Python environment on an Ubuntu system. To install Odoo 17 on Ubuntu you just follow here steps. Call us!
Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...Shift Conference
What's the first thing you should do when starting a new project...? Setup a good CI system! With Github Actions you can do it in a couple of seconds. You can easily setup a workflow to build your project, test it on different machines, and deploy the results. In this talk we're going to see how you can setup a simple Github Action for your repository and start enjoying it right after.
My talk from DevOpsCon Berlin 2016.
Ansible is a radically simple and lightweight provisioning framework which makes your servers and applications easier to provision and deploy. By orchestrating your application deployments you gain benefits such as documentation as code, testability, continuous integration, version control, refactoring, automation and autonomy of your deployment routines, server and application configuration. Ansible uses a language that approaches plain English, uses SSH and has no agents to install on remote systems. It is the simplest way to automate and orchestrate application deployment, configuration management and continuous delivery.
In this tutorial you will be given an introduction to Ansible and learn how to provision Linux servers with a web-proxy, a database and some other packages. Furthermore we will automate zero downtime deployment of a Java application to a load balanced environment.
Lean Drupal Repositories with Composer and DrushPantheon
Composer is the industry-standard PHP dependency manager that is now in use in Drupal 8 core. This session will show the current best practices for using Composer, drupal-composer, drupal-scaffold, Drush, Drupal Console and Drush site-local aliases to streamline your Drupal 7 and Drupal 8 site repositories for optimal use on teams.
RustPython is a project that implements the Python programming language in Rust. It started in May 2018 and released version 0.3 in September 2023. The standard library has been written in Rust and pip package management works. RustPython can compile to WebAssembly to run Python code in web browsers and has an experimental just-in-time compiler. Several popular Python packages were tested with varying results, with some packages like Django and others not working due to missing Python modules or compiler issues. Packaging Python modules that use Rust is becoming increasingly common using tools like PyO3, with over 50,000 downloads per day of related projects.
PyCon 2013 : Scripting to PyPi to GitHub and MoreMatt Harrison
This document discusses various aspects of developing and distributing Python projects, including versioning, configuration, logging, file input, shell invocation, environment layout, project layout, documentation, automation with Makefiles, packaging, testing, GitHub, Travis CI, and PyPI. It recommends using semantic versioning, the logging module, parsing files with the file object interface, invoking shell commands with subprocess, using virtualenv for sandboxed environments, Sphinx for documentation, Makefiles to automate tasks, setuptools for packaging, and GitHub, Travis CI and PyPI for distribution.
The document summarizes the steps taken to set up a Django project called "he" on Ubuntu. It shows commands used to install Python, virtualenv, Django and other dependencies. Database setup with PostgreSQL is also demonstrated. An app called "board" is created, with a Post model defined and admin configured. Templates are added and the development server is run. Authentication and registration are implemented along with forms to add new posts. The project is developed iteratively through multiple versions.
Modern binary build systems have made shipping binary packages for Python much easier than ever before. This talk discusses three of the most popular build systems for Python packages using the new standards developed for packaging.
This document discusses container technologies including App Container (appc) and rkt. It provides an overview of appc components like the image format, discovery, and executor. It then discusses rkt, an implementation of appc, describing its modular architecture with stages 0-2 and use of systemd and cgroups for isolation. It also touches on rkt security, networking, and integration with systemd and user namespaces.
Similar to Princeton Wintersession: Software Quality Assurance Tooling (20)
The document summarizes Henry Schreiner's work on several Python and C++ scientific computing projects. It describes a scientific Python development guide built from the Scikit-HEP summit. It also outlines Henry's work on pybind11 for C++ bindings, scikit-build for building extensions, cibuildwheel for building wheels on CI, and several other related projects.
The document describes various productivity tools for Python development, including:
- Pre-commit hooks to run checks before committing code
- Hot code reloading in Jupyter notebooks using the %load_ext and %autoreload magic commands
- Cookiecutter for generating project templates
- SSH configuration files and escape sequences for easier remote access
- Autojump to quickly navigate frequently visited directories
- Terminal tips like command history search and referencing the last argument
- Options for tracking Jupyter notebooks with git like stripping outputs or synchronizing notebooks and Python files.
SciPy22 - Building binary extensions with pybind11, scikit build, and cibuild...Henry Schreiner
Building binary extensions is easier than ever thanks to several key libraries. Pybind11 provides a natural C++ language for extensions without requiring pre-processing or special dependencies. Scikit-build ties the premier C++ build system, CMake, into the Python extension build process. And cibuildwheel makes it easy to build highly compatible wheels for over 80 different platforms using CI or on your local machine. We will look at advancements to all three libraries over the last year, as well as future plans.
This document discusses the history and development of Python packages for high energy physics (HEP) analysis. It describes how experiments initially used ROOT and C++, but Python gained popularity for configuration and analysis. This led to the creation of packages like Scikit-HEP, Uproot, and Awkward Array to bridge the gap between ROOT files and the Python data science stack. Scikit-HEP grew to include many related packages and provides best practices through its developer pages. The future may include adopting Scikit-build for building Python packages with C/C++ extensions and running packages in the browser via WebAssembly.
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packagingHenry Schreiner
This was a PyCon 2022 lightning talk over the Scikit-HEP developer pages. It highlights best practices and guides shown there, and the quick package creation cookiecutter. And finally it demos the Pyodide WebAssembly app embedded into the Scikit-HEP developer pages!
Talk at PyCon2022 over building binary packages for Python. Covers an overview and an in-depth look into pybind11 for binding, scikit-build for creating the build, and build & cibuildwheel for making the binaries that can be distributed on PyPI.
This document provides best practices for using CMake, including:
- Set the cmake_minimum_required version to ensure modern features while maintaining backward compatibility.
- Use targets to define executables and libraries, their properties, and dependencies.
- Fetch remote dependencies at configure time using FetchContent or integrate with package managers like Conan.
- Import library targets rather than reimplementing Find modules when possible.
- Treat CUDA as a first-class language in CMake projects.
HOW 2019: Machine Learning for the Primary Vertex ReconstructionHenry Schreiner
The document describes a machine learning approach for primary vertex reconstruction in high-energy physics experiments. A hybrid method is proposed that uses a 1D convolutional neural network to analyze histograms produced from tracking data. The network is able to find primary vertices with high efficiency and tunable false positive rates, demonstrating the potential of machine learning for this task. Future work involves adding more tracking information and iterating between track association and vertex finding to improve performance.
HOW 2019: A complete reproducible ROOT environment in under 5 minutesHenry Schreiner
The document discusses setting up a ROOT environment using Conda in under 5 minutes. It describes downloading and installing Miniconda and then using Conda commands to create a new environment and install ROOT and its dependencies from the conda-forge channel. The ROOT package provides full ROOT functionality, including compilation and graphics, and supports Linux, macOS, and multiple Python versions.
ACAT 2019: A hybrid deep learning approach to vertexingHenry Schreiner
This document presents a hybrid deep learning approach for vertex finding in high-energy physics experiments. It uses a 1D convolutional neural network to analyze kernel density estimates of track information in order to identify primary vertex positions. The approach achieves primary vertex finding efficiencies of 88-94% with low false positive rates comparable to traditional algorithms. The authors demonstrate tuning of the efficiency-false positive rate tradeoff and discuss plans to improve performance by incorporating additional track information and iterative refinement.
2019 CtD: A hybrid deep learning approach to vertexingHenry Schreiner
This document presents a hybrid deep learning approach for vertex finding using 1D convolutional neural networks. It describes generating 1D kernel densities from tracking information, building target distributions, and using a CNN architecture with an adjustable cost function to optimize the false positive rate versus efficiency. The approach achieves 93.87% efficiency with a 0.251 false positive rate on test data. Future work includes incorporating additional xy information and exploring full 2D kernel densities.
2019 IRIS-HEP AS workshop: Boost-histogram and histHenry Schreiner
The document discusses the current state of histograms in Python and the need for a new histogramming library. It introduces boost-histogram, a C++ histogramming library, and its new Python bindings. The bindings aim to provide a fast, flexible and easily distributable histogram object for Python. Key features discussed include histogram design that treats it as a first-class object, fast filling via multi-threading, a variety of axis and storage types, and performance benchmarks showing it can be over 10x faster than NumPy for filling histograms. Distribution is focused on providing binary wheels for many platforms via continuous integration.
The document discusses the current state of histograms in Python and the need for a new library. It introduces boost-histogram, a C++ histogram library, and its new Python bindings. The bindings aim to provide a fast, flexible, and easily distributable histogram object for Python with support for multiple axis types and storage options. It also discusses plans for an additional wrapper library called hist for easy plotting and interfacing with other tools.
2019 IRIS-HEP AS workshop: Particles and decaysHenry Schreiner
The Scikit-HEP project aims to create an ecosystem for particle physics data analysis in Python. It includes packages like Particle and DecayLanguage that provide tools for working with particle data and decay descriptions. Particle allows users to easily access and search particle property data from sources like the PDG. DecayLanguage allows parsing decay file formats, representing and manipulating decay chains, and converting between decay model representations. Future work includes expanding particle ID support and improving visualization of decay trees.
The document discusses plans for the boost-histogram and hist Python libraries. Boost-histogram is a multidimensional histogram library inspired by ROOT that provides flexibility through many axis and storage types. Hist will provide plotting and analysis functionality by interfacing with libraries like mpl-hep. Future plans include improved indexing, slicing, and NumPy conversions for boost-histogram as well as statistical functions, serialization, and integration with fitters for hist.
The document discusses the new features of Python 3.8, which was recently released. Some key updates include positional-only arguments, the walrus operator for variable assignment, improved static typing support, and performance enhancements. The document also notes additional developer changes and provides resources for obtaining Python 3.8.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceIndexBug
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
1. Software Quality Assurance Tooling
After today you will know SQAT
Henry Schreiner
January 23, 2023
ISciNumPy
https://iscinumpy.dev
Scikit-HEP Developer
https://scikit-hep.org/developer
https://github.com/henryiii/sqat-example
2. Language
We’ll focus on Python (especially at
fi
rst)
But the general concepts are around in most other languages
You just need to
fi
nd the matching tool(s)
3. Packaging aside: pipx
3
$ pip install <application>
$ <application>
I’m sure you’ve seen this: Examples of applications:
build: make SDists and wheels
twine: upload SDists and wheels
cibuildwheel: make redistributable wheels
nox/tox: Python task runners
jupylite: WebAssembly Python site builder
black: Python code formatter
pypi-command-line: query PyPI
uproot-browser: ROOT file browser (HEP)
tiptop: fancy top-style monitor
rich-cli: pretty print files
cookiecutter: template packages
clang-format: format C/C++/CUDA code
pre-commit: general CQA tool
cmake: build system generator
meson: another build system generator
ninja: build system
Packages can con
fl
ict
Updates get slower over time
Lose track of why things are installed
Manual updates are painful
Hates Python being replaced
$ pipx install <application>
$ <application>
Better!
Automatic venv for each package
No con
fl
icts ever
Everything updatable / replaceable
Doesn’t like Python being replaced
$ pipx run <application>
Best!
Automatic venv caching
Never more than a week old
No pre-install or setup
No maintenance
Replace Python at will
pipx run --spec git+https://github.com/henryiii/rich-cli@patch-1 rich
pipx has
fi
rst
class support
on GHA & Azure!
4. Task runner aside: Nox
4
Make
fi
les
Custom language
Painful to write
Painful to maintain
Looks like garbage
OS dependent
No Python environments
Everywhere
Tox
Custom language
Concise to write
Tricky to read
Ties you to tox
OS independent
Python environments
Python package
Nox
Python, mimics pytest
Simple but verbose
Easy to read
Teaches commands
OS independent
Python environments
Python package
Other task runners available for other purposes, like Rake (Ruby)
5. Writing a nox
fi
le.py
5
import nox
@nox.session(python=["3.7", "3.8", "3.9", "3.10", "3.11"])
def tests(session: nox.Session) -> None:
"""
Run the unit and regular tests.
"""
session.install(".[test]")
session.run("pytest", *session.posargs)
7. Features of nox
7
Full control over environments
Easy
fl
y-by contributions
Transparent, simple .nox directory
Conda support
Trade speed for reproducibility
Some ideas for sessions
lint
tests
docs
build
bump
pylint
regenerate
update_pins
check_manifest
make_changelog
update_python_dependencies
See
pypa/cibuildwheel
pypa/manylinux
scikit-hep/hist
scikit-hep/boost-histogram
pybind/pybind11
scikit-hep/cookie
scikit-hep/scikit-hep.github.io
Optional environment reuse
8. Python launcher for Unix
8
Rust implementation of “py” for UNIX
But also automatically picks up .venv folder!
Meant for lazy experts
Launcher
$ py -m pytest
Classic
$ . .venv/bin/activate
(.venv) $ python -m pytest
(.venv) $ deactivate
Classic, take 2
$ .venv/bin/python -m pytest
9. Cookiecutter
9
Quickly set up a project
Takes options
Scikit-hep/cookie is a great cookiecutter for Python!
How to run
pipx run cookiecutter gh:scikit-hep/cookie
11. Code Quality
11
Why does code quality matter?
Improve readability
Find errors before they happen
Avoid historical baggage
Reduce merge con
fl
icts
Warm fuzzy feelings
How to run
Discussion of checks
(Opinionated)
Mostly focusing on Python today
12. pre-commit
12
Poorly named?
Has a pre-commit hook mode
You don’t have to use it that way!
Generic check runner
conda
coursier
dart
docker
docker_image
dotnet
fail
golang
lua
node
perl
python
python_venv
r
ruby
rust
swift
pygrep
script
system
Written in Python
pipx, nox, homebrew, etc.
Designed for speed & reproducibility
Ultra fast environment caching
Locked environments
Easy autoupdate command
pre-commit.ci
Automatic updates
Automatic
fi
xes for PRs
Large library of hooks
https://pre-commit.com/hooks.html
Custom hooks are simple
13. Con
fi
guring pre-commit
13
Design
A hook is just a YAML dict
Fields can be overridden
Environments globally cached by git tag
Supports checks and
fi
xers
# .pre-commit-config.yaml
hooks:
- repo: https://github.com/psf/black
rev: "22.12.0"
hooks:
- id: black
# Black’s .pre-commit-hooks.yaml
- id: black
name: black
description: "Black: The uncompromising code formatter"
entry: black
language: python
minimum_pre_commit_version: 2.9.2
require_serial: true
types_or: [python, pyi]
- id: black-jupyter
name: black-jupyter
description: "Black (with Jupyter Notebook support)"
entry: black
language: python
minimum_pre_commit_version: 2.9.2
require_serial: true
types_or: [python, pyi, jupyter]
additional_dependencies: [".[jupyter]"]
You write this
Formatter author writes this
14. Options for pre-commit
14
Selected options
fi
les: explicit include regex
exclude: explicit exclude regex
types_or/types/exclude_types:
fi
le types
args: control arguments
additional_dependencies: extra things to install
stages: select the git stage (like manual)
15. Running pre-commit
15
Run all checks
pre-commit run -a
Update all hooks
pre-commit autoupdate
Install as a pre-commit hook
pre-commit install
(Skip with git commit -n)
Skip checks
SKIP=… <run>
Run one check
pre-commit run -a <id>
Run manual stage
pre-commit run --hook-stage manual
16. Examples of pre-commit checks
16
Almost everything following in this talk
- repo: local
hooks:
- id: disallow-caps
name: Disallow improper capitalization
language: pygrep
entry: PyBind|Numpy|Cmake|CCache|Github|PyTest
exclude: .pre-commit-config.yaml
Don’t grep the
fi
le this is in!
“Entry” is the grep, in this case
Using pygrep “language”
Custom hook
18. pre-commit/pygrep-hooks
18
Small common pygreps
- repo: https://github.com/pre-commit/pygrep-hooks
rev: "v1.10.0"
hooks:
- id: python-check-blanket-noqa
- id: python-check-blanket-type-ignore
- id: python-no-eval
- id: python-use-type-annotations
- id: rst-backticks
- id: rst-directive-colons
- id: rst-inline-touching-normal
Opinion: blanket CQ ignores are bad. Ignores should be speci
fi
c
Better readability/searchability - avoid hiding unrelated issue
Optional, some
fi
les might need to be excluded
19. CI (GitHub Actions)
19
on:
pull_request:
push:
branches:
- main
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v3
- uses: pre-commit/action@v3.0.0
Great, fast caching, but maintenance only - replaced by pre-commit.ci
on:
pull_request:
push:
branches:
- main
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: pipx run nox -s lint
@nox.session
def lint(session: nox.Session) -> None:
session.install("pre-commit")
session.run("pre-commit", "run", "--all-files", "--show-diff-on-failure", *session.posargs)
22. Using code formatters
22
Existing projects
Apply all-at-once, not spread out over time
Add the format commit to .git-blame-ignore-revs
23. Black
23
hooks:
- repo: https://github.com/psf/black
rev: "22.12.0"
hooks:
- id: black-jupyter
Python code formatter
Close to the one true format for Python
Almost not con
fi
gurable (this is a feature)
A good standard is better than perfection
Designed to reduce merge con
fl
icts
Reading blacked code is fast
Write your code to produce nice formatting
You can disable line/lines if you have to
Workaround for single quotes (use double)
Magic trailing comma
Online version
https://black.vercel.app
24. Write for good format
24
raise RuntimeError(
"This was not a valid value for some_value: {}".format(repr(some_value))
)
Bad:
msg = f"This was not a valid value for some_value: {some_value!r}"
raise RuntimeError(msg)
Good:
Better stacktrace
More readable
Two lines instead of three
Faster (f-string)
25. Notebook cleaner
25
hooks:
- repo: https://github.com/kynan/nbstripout
rev: "0.6.0"
hooks:
- id: nbstripout
Remove outputs from notebooks
Best if not stored in VCS
You can render outputs in JupyterBook, etc.
Use Binder or JupyterLite
26. NBQA: Notebook adaptor
26
- repo: https://github.com/nbQA-dev/nbQA
rev: 1.6.1
hooks:
- id: nbqa-pyupgrade
args: ["--py37-plus"]
- id: nbqa-isort
args: ["--float-to-top"]
Run Python formatters on notebooks
Lots of formatters supported
Black natively supports notebooks these days, FYI
27. isort
27
hooks:
- repo: https://github.com/PyCQA/isort
rev: "5.11.4"
hooks:
- id: isort
Sort your Python imports
Very con
fi
gurable
Reduces merge con
fl
icts
Grouping imports helps readers
Can inject future imports
# pyproject.toml
[tool.isort]
profile = "black"
args: ["-a", "from __future__ import annotations"]
Default groupings
Future imports
Stdlib imports
Third party packages
Local imports
28. pyupgrade
28
hooks:
- repo: https://github.com/asottile/pyupgrade
rev: "v3.3.1"
hooks:
- id: pyupgrade
args: [--py37-plus]
Update Python syntax
Avoid deprecated or obsolete code
Fairly cautious
Can target a speci
fi
c Python 3 min
(Mostly) not con
fi
gurable
Remove static if sys.version_info blocks
Python 2.7
Set literals
Dictionary comprehensions
Generators in functions
Format speci
fi
er & .format ⚙
Comparison for const literals (3.8 warn)
Invalid escapes
Python 3
Unicode literals
Long literals, octal literals
Modern super()
New style classes
Future import removal
yield from
Remove six compatibility code
io.open -> open
Remove error aliases
Python 3.x
f-strings (partial) (3.6) ⚙
NamedTuple/TypedDict (3.6)
subprocess.run updates (3.7)
lru_cache parens (3.8)
lru_cache(None) -> cache (3.9)
Typing & annotation rewrites (various)
abspath(__file__) removal (3.9)
29. pyupgrade examples
29
Before After
for a, b in c:
yield (a, b)
yield from c
"{foo} {bar}".format(foo=foo, bar=bar) f"{foo} {bar}"
dict([(a, b) for a, b in y]) {a: b for a, b in y}
30. pyupgrade limits
30
PyUpgrade does not over modernize
isinstance(x, (int, str)) -> isinstance(x, int | str) (3.10)
No match statement conversions (3.10)
Nothing converts to using walrus := (3.8) (probably a good thing!)
Except for a bit of typing
Optional[int] -> int | None (I like this one now, though)
❌
31. setup-cfg-fmt
31
hooks:
- repo: https://github.com/asottile/setup-cfg-fmt
rev: "v2.2.0"
hooks:
- id: setup-cfg-fmt
args: [--max-py-version=3.11, --include-version-classifiers]
Maintain setup.cfg
fi
le
Can add and
fi
x trove classi
fi
ers
Sorts and cleans
A bit opinionated, will not
fi
x some bugs
What about pyproject.toml?
Three projects (at least) popping up
Very young
Best process unclear
ini2toml useful for conversion
Or hatch new --init
33. Using code linters
33
Existing projects
Feel free to build a long ignore list
Work on one or a few at a time
You don’t have to have every check
34. hooks:
- repo: https://gitlab.com/pycqa/flake8
rev: "6.0.0"
hooks:
- id: flake8
additional_dependencies: [flake8-bugbear]
Flake8
34
Fast simple extendable linter
Very con
fi
gurable: setup.cfg or .
fl
ake8
Many plugins, local plugins easy
No auto-
fi
xers like rubocop (Ruby)
Opinion:
fl
ake8-bugbear is great
Example:
fl
ake8-print, avoid all prints
# .flake8
[flake8]
max-complexity = 12
extend-ignore = E203, E501, E722, B950
extend-select = B902, B903, B904
35. Flake8 example checks
35
Bugbear
Do not use bare except
No mutable argument defaults
getattr(x, "const") should be x.const
No assert False, use raise AssertionError
Pointless comparison ❤ pytest
PyFlakes (default)
Unused modules & variables
String formatting mistakes
No placeholders in f-string
Dictionary key repetition
Assert a tuple (it’s always true)
Various syntax errors
Unde
fi
ned names
Rede
fi
nition of unused var ❤ pytest
McCabe (default)
Complexity checks
PyCodeStyle (default)
Style checks
Flake8-print
Avoid leaking debugging print statements
36. Custom local
fl
ake8 plugin
36
import ast
import sys
from typing import NamedTuple, Iterator
class Flake8ASTErrorInfo(NamedTuple):
line_number: int
offset: int
msg: str
cls: type # unused
37. Custom local
fl
ake8 plugin
37
class Visitor(ast.NodeVisitor):
msg = "AK101 exception must be wrapped in ak._v2._util.*error"
def __init__(self) -> None:
self.errors: list[Flake8ASTErrorInfo] = []
def visit_Raise(self, node: ast.Node) -> None:
if isinstance(node.exc, ast.Call):
if isinstance(node.exc.func, ast.Attribute):
if node.exc.func.attr in {"error", "indexerror"}:
return
if node.exc.func.id in {"ImportError"}:
return
self.errors.append(
Flake8ASTErrorInfo(node.lineno, node.col_offset, self.msg, type(self))
)
38. Custom local
fl
ake8 plugin
38
class AwkwardASTPlugin:
name = "flake8_awkward"
version = "0.0.0"
def __init__(self, tree: ast.AST) -> None:
self._tree = tree
def run(self) -> Iterator[Flake8ASTErrorInfo]:
visitor = Visitor()
visitor.visit(self._tree)
yield from visitor.errors
39. Custom local
fl
ake8 plugin
39
[flake8:local-plugins]
extension =
AK1 = flake8_awkward:AwkwardASTPlugin
paths =
./dev/
def main(path: str) -> None:
with open(path) as f:
code = f.read()
node = ast.parse(code)
plugin = AwkwardASTPlugin(node)
for err in plugin.run():
print(f"{path}:{err.line_number}:{err.offset} {err.msg}")
if __name__ == "__main__":
for item in sys.argv[1:]:
main(item)
41. PyLint
41
PyLint recommends having your project installed, so it is not a good pre-commit hook (though you can do it)
It’s also a bit slow, so a good candidate for nox
@nox.session
def pylint(session: nox.Session) -> None:
session.install("-e", ".")
session.install("pylint")
session.run("pylint", "src", *session.posargs)
# pyproject.toml
[tool.pylint]
master.py-version = "3.7"
master.jobs = "0"
reports.output-format = "colorized"
similarities.ignore-imports = "yes"
messages_control.enable = ["useless-suppression"]
messages_control.disable = [
"design",
"fixme",
"line-too-long",
"wrong-import-position",
]
Code linter
Can be very opinionated
Signal to noise ratio poor
You will need to disable checks - that’s okay!
A bit more advanced / less static than
fl
ake8
But can catch hard to
fi
nd bugs!
For an example of lots of suppressions:
https://github.com/scikit-hep/awkward-1.0/blob/1.8.0/pyproject.toml
42. Example PyLint rules
42
Duplicate code
Finds large repeated code patterns
Attribute de
fi
ned outside init
Only __init__ should de
fi
ne attributes
No self use
Can be @classmethod or @staticmethod
Unnecessary code
Lambdas, comprehensions, etc.
Unreachable code
Finds things that can’t be reached
Consider using in
x in {stuff} vs chaining or’s
Arguments di
ff
er
Subclass should have matching arguments
Consider iterating dictionary
Better use of dictionary iteration
Consider merging isinstance
You can use a tuple in isinstance
Useless else on loop
They are bad enough when useful :)
Consider using enumerate
Avoid temp variables, idiomatic
Global variable not assigned
You should only declare global to assign
43. Controversial PyLint rules
43
No else after control-
fl
ow
Guard-style only
Can simply complex control
fl
ow
Removes useless indentation
if x:
return x
else:
return None
# Should be:
if x:
return x
return None
# Or:
return x if x else None
# Or:
return x or None
Design
Too many various things
Too few methods
Can just silence “design”
(I’m on the in-favor side)
44. Ruff: one linter to rule them all?
44
A very new entry in the Python linting space, but gaining traction like wild
fi
re
100x faster than existing Python linters
Has support for
fi
xers!
Implements all of (modern)
fl
ake8’s checks
Implements dozens of
fl
ake8 plugins
Implements most of pyupgrade (as rules!)
Can replace simple usage of isort
Can remove unused noqa’s
Can remove unused imports
Some of PyLint
0 dependencies
Con
fi
gured with pyproject.toml
Only binary platforms (Rust compiled)
Doesn’t support user plugins
Examples
https://github.com/scikit-build/scikit-build-core
https://github.com/pypa/build
46. Static type checking: MyPy
46
hooks:
- repo: https://gitlab.com/pre-commit/mirrors-mypy
rev: "v0.991"
hooks:
- id: mypy
files: src
args: [--show-error-codes]
Like a linter on steroids
Uses Python typing
Enforces correct type annotations
Designed to be iteratively enabled
Should be in a controlled environment (pre-commit or nox)
Always specify args (bad hook defaults)
Almost always need additional_dependencies
Con
fi
gure in pyproject.toml
Pros
Can catch many things tests normally catch, without writing tests
Therefore it can catch things not covered by tests (yet, hopefully)
Code is more readable with types
Sort of works without types initially
Cons
Lots of work to add all types
Typing can be tricky in Python
Active development area for Python
47. Con
fi
guring MyPy
47
[tool.mypy]
files = "src"
python_version = "3.7"
warn_unused_configs = true
strict = true
[[tool.mypy.overrides]]
module = [ "numpy.*" ]
ignore_missing_imports = true
Start small
Start without strictness
Add a check at a time
Extra libraries
Try adding them to your environment
You can ignore untyped or slow libraries
You can provide stubs for untyped libraries if you want
Tests?
Adding pytest is rather slow
I prefer to avoid tests, or keep them mostly untyped
48. Typing tricks
48
Protocols
Better than ABCs, great for duck typing
@typing.runtime_checkable
class Duck(Protocol):
def quack() -> str:
...
def f(x: Duck) -> str:
return x.quack()
class MyDuck:
def quack() -> str:
return "quack"
if typing.TYPE_CHECKING:
_: Duck = typing.cast(MyDuck, None)
Type Narrowing
Integral to how mypy works
x: Union[A, B]
if isinstance(x, A):
reveal_type(x) # A
else:
reveal_type(x) # B
Make a typed package
Must include py.typed marker
fi
le
Always use sys.version_info
Better for readers than try/except, and static
Also sys.platform instead of os.name
49. Future annotations
49
Classic code (3.5+)
from typing import Union, List
def f(x: int) -> List[int]:
return list(range(x))
def g(x: Union[str, int]) -> None:
if isinstance(x, str):
print("string", x.lower())
else:
print("int", x)
Modern code (3.7+)
from __future__ import annotations
def f(x: int) -> list[int]:
return list(range(x))
def g(x: str | int) -> None:
if isinstance(x, str):
print("string", x.lower())
else:
print("int", x)
Ultramodern code (3.10+)
def f(x: int) -> list[int]:
return list(range(x))
def g(x: str | int) -> None:
if isinstance(x, str):
print("string", x.lower())
else:
print("int", x)
With the future import, you get all the bene
fi
ts of future code in 3.7+ annotations
Typing is already extra code, simpler is better
58. pytest tips
58
Spend time learning pytest
Full of amazing things that really make testing fun!
Tests are code too
Or for C++: Catch2 or doctest, etc.
Also maybe learn Hypothesis for pytest
[tool.pytest.ini_options]
minversion = "6.0"
addopts = [
"-ra",
"--showlocals",
"--strict-markers",
"--strict-config",
]
xfail_strict = true
filterwarnings = [
"error",
]
log_cli_level = "info"
testpaths = [
"tests",
]
Don’t let warnings slip by!
Makes logging more useful
Strictness is good
Useful summary
Print out locals on errors
Use pytest.approx
Even works on numpy arrays
Remember to test for failures
If you expect a failure, test it!
Test your installed package
That’s how users will get it, not from a directory
59. pytest Tricks
59
Mock and Monkeypatch
This is how you make tricky tests “unit” tests
Fixtures
This keeps tests simple and scalable
@pytest.fixture(params=["Linux", "Darwin", "Windows"], autouse=True)
def platform_system(request, monkeypatch):
monkeypatch.setattr(platform, "system", lambda _: request.param)
Parametrize
Directly or in a
fi
xture for reuse
Use conftest.py
Fixtures available in same and nested directories
60. Running pytest
60
Show locals on failure
--showlocals/-l
Jump into a debugger on failure
--pdb
Start with last failing test
--lf
Jump into a debugger immediately
--trace or use breakpoint()
Run matching tests
-k <expression>
Run speci
fi
c test
filename.py::testname
Run speci
fi
c marker
-m <marker>
Control traceback style
--tb=<style>
61. In conclusion
61
Code quality tools can help a lot with
Readability
Reducing bugs
Boosting developer productivity
Consistency
Refactoring
Teaching others good practice too
Hopefully we have had some helpful discussions!
It’s okay to disable a check
Try to understand why it’s there
Remember there are multiple concerns involved in decisions