Simple Data Engineering in Python 3.5+ using Bonobo ETL, with real world example using Django2 and DBPedia.
https://www.bonobo-project.org/
Presentation from Pycon.DE 2017 in Karlsruhe
Simple ETL in Python 3.5+ - PolyConf Paris 2017 - Lightning Talk (10 minutes)Romain Dorgueil
Quick introduction (lightning talk) to Bonobo ETL at PolyConf 2017, in La Géode (Paris). Bonobo is the Extract Transform Load framework for python 3.5+. Write line by line data processors in minutes, using the best practices of software engineering.
Given at GopherFest 2015. This is an updated version of the talk I gave in NYC Nov 14 at GothamGo.
“We need to think about failure differently. Most people think mistakes are a necessary evil. Mistakes aren't a necessary evil, they aren't evil at all. They are an inevitable consequence of doing something new and as such should be seen as valuable. “ - Ed Catmull
As Go is a "new" programming language we are all experimenting and learning how to write better Go. While most presentations focus on the destination, this presentation focuses on the journey of learning Go and the mistakes I personally made while developing Hugo, Cobra, Viper, Afero & Docker.
Simple ETL in Python 3.5+ - PolyConf Paris 2017 - Lightning Talk (10 minutes)Romain Dorgueil
Quick introduction (lightning talk) to Bonobo ETL at PolyConf 2017, in La Géode (Paris). Bonobo is the Extract Transform Load framework for python 3.5+. Write line by line data processors in minutes, using the best practices of software engineering.
Given at GopherFest 2015. This is an updated version of the talk I gave in NYC Nov 14 at GothamGo.
“We need to think about failure differently. Most people think mistakes are a necessary evil. Mistakes aren't a necessary evil, they aren't evil at all. They are an inevitable consequence of doing something new and as such should be seen as valuable. “ - Ed Catmull
As Go is a "new" programming language we are all experimenting and learning how to write better Go. While most presentations focus on the destination, this presentation focuses on the journey of learning Go and the mistakes I personally made while developing Hugo, Cobra, Viper, Afero & Docker.
Go for Object Oriented Programmers or Object Oriented Programming without Obj...Steven Francia
Object Oriented (OO) programming has dominated software engineering for the last two decades. The paradigm built on powerful concepts such as Encapsulation, Inheritance, and Polymoprhism has been internalized by the majority of software engineers. Although Go is not OO in the strict sense, we can continue to leverage the skills we’ve honed as OO engineers to come up with simple and solid designs.
Gopher Steve Francia, Author of
[Hugo](http://hugo.spf13.com), [Cobra](http://github.com/spf13/cobra), and many
other popular Go packages makes these difficult concepts accessible for everyone.
If you’re a OO programmer, especially one with a background with dynamic languages and are curious about go then this talk is for you. We will cover everything you need to know to leverage your existing skills and quickly start coding in go including:
How to use our Object Oriented programming fundamentals in go
Static and pseudo dynamic typing in go
Building fluent interfaces in go
Using go interfaces and duck typing to simplify architecture
Common mistakes made by those coming to go from other OO languages (Ruby, Python, Javascript, etc.),
Principles of good design in go.
7 Common mistakes in Go and when to avoid themSteven Francia
I've spent the past two years developing some of the most popular libraries and applications written in Go. I've also made a lot of mistakes along the way. Recognizing that "The only real mistake is the one from which we learn nothing. -John Powell", I would like to share with you the mistakes that I have made over my journey with Go and how you can avoid them.
Presented on 27th September 2017 to a joint meeting of 'Cork Functional Programmers' and the 'Cork Java Users Group'
Based on the Kotlin Language programming course from Instil. For more details see https://instil.co/courses/kotlin-development/
Slides of presentation given at EuroGeographics KEN workshop on INSPIRE Data Harmonization, Paris oct 8-9, 2013: http://www.eurogeographics.org/event/inspire-ken-schema-transformation-workshop. Describes the Stetl ETL framework and cases of INSPIRE transformation. There is a video recording of this presentation: https://www.youtube.com/watch?v=vjdpYBm4AaM (first about XSLT and about halfway on Stetl for INSPIRE)
Explains basics of Stetl, an Open Source spatial ETL framework. More on http://stetl.org. Talk given on GeoPython conference, June 24, 2016, Basel, Switserland.
5-minute intro to Stetl an Open Source lightweight framework for handling the conversion and transformation (ETL) of in particular complex/rich GML. Stetl uses and combines existing tools like GDAL/OGR and XSLT. Python is used as glue. Users configure an ETL-process chain through a configuration.
More at http://stetl.org
It is the slides for COSCUP[1] 2013 Hands-on[2], "Learning Python from Data".
It aims for using examples to show the world of Python. Hope it will help you with learning Python.
[1] COSCUP: http://coscup.org/
[2] COSCUP Hands-on: http://registrano.com/events/coscup-2013-hands-on-mosky
Ange Albertini and Gynvael Coldwind: Schizophrenic Files – A file that thinks...Area41
As file format specs leave room for interpretation and sometimes are misunderstood or ignored by the programmers, some well-formed files may be interpreted inconsistently by different tools and libraries. As a result, this can be (ab)used for simple jokes, anti-forensics or to bypass sanitizers which might lead to data exfiltration.
Ange Albertini: Reverse Engineer, author of Corkami
Gynvael Coldwind: His main areas of interest are low-level security (kernel, OS, client), web security and reverse-engineering. Captain of Dragon Sector CTF team :) Currently working as an Information Security Engineer at Google.
Go for Object Oriented Programmers or Object Oriented Programming without Obj...Steven Francia
Object Oriented (OO) programming has dominated software engineering for the last two decades. The paradigm built on powerful concepts such as Encapsulation, Inheritance, and Polymoprhism has been internalized by the majority of software engineers. Although Go is not OO in the strict sense, we can continue to leverage the skills we’ve honed as OO engineers to come up with simple and solid designs.
Gopher Steve Francia, Author of
[Hugo](http://hugo.spf13.com), [Cobra](http://github.com/spf13/cobra), and many
other popular Go packages makes these difficult concepts accessible for everyone.
If you’re a OO programmer, especially one with a background with dynamic languages and are curious about go then this talk is for you. We will cover everything you need to know to leverage your existing skills and quickly start coding in go including:
How to use our Object Oriented programming fundamentals in go
Static and pseudo dynamic typing in go
Building fluent interfaces in go
Using go interfaces and duck typing to simplify architecture
Common mistakes made by those coming to go from other OO languages (Ruby, Python, Javascript, etc.),
Principles of good design in go.
7 Common mistakes in Go and when to avoid themSteven Francia
I've spent the past two years developing some of the most popular libraries and applications written in Go. I've also made a lot of mistakes along the way. Recognizing that "The only real mistake is the one from which we learn nothing. -John Powell", I would like to share with you the mistakes that I have made over my journey with Go and how you can avoid them.
Presented on 27th September 2017 to a joint meeting of 'Cork Functional Programmers' and the 'Cork Java Users Group'
Based on the Kotlin Language programming course from Instil. For more details see https://instil.co/courses/kotlin-development/
Slides of presentation given at EuroGeographics KEN workshop on INSPIRE Data Harmonization, Paris oct 8-9, 2013: http://www.eurogeographics.org/event/inspire-ken-schema-transformation-workshop. Describes the Stetl ETL framework and cases of INSPIRE transformation. There is a video recording of this presentation: https://www.youtube.com/watch?v=vjdpYBm4AaM (first about XSLT and about halfway on Stetl for INSPIRE)
Explains basics of Stetl, an Open Source spatial ETL framework. More on http://stetl.org. Talk given on GeoPython conference, June 24, 2016, Basel, Switserland.
5-minute intro to Stetl an Open Source lightweight framework for handling the conversion and transformation (ETL) of in particular complex/rich GML. Stetl uses and combines existing tools like GDAL/OGR and XSLT. Python is used as glue. Users configure an ETL-process chain through a configuration.
More at http://stetl.org
It is the slides for COSCUP[1] 2013 Hands-on[2], "Learning Python from Data".
It aims for using examples to show the world of Python. Hope it will help you with learning Python.
[1] COSCUP: http://coscup.org/
[2] COSCUP Hands-on: http://registrano.com/events/coscup-2013-hands-on-mosky
Ange Albertini and Gynvael Coldwind: Schizophrenic Files – A file that thinks...Area41
As file format specs leave room for interpretation and sometimes are misunderstood or ignored by the programmers, some well-formed files may be interpreted inconsistently by different tools and libraries. As a result, this can be (ab)used for simple jokes, anti-forensics or to bypass sanitizers which might lead to data exfiltration.
Ange Albertini: Reverse Engineer, author of Corkami
Gynvael Coldwind: His main areas of interest are low-level security (kernel, OS, client), web security and reverse-engineering. Captain of Dragon Sector CTF team :) Currently working as an Information Security Engineer at Google.
Clouds are Not Free: Guide to Observability-Driven Efficiency OptimizationsScyllaDB
Over the recent decade, cloud computing and cloud-native platforms emerged and promised lower costs, less effort, and more flexibility. Unfortunately, in practice, it is not so easy. Knowledge short gaps, pitfalls, the complexity of using cloud APIs, and misunderstanding of services and systems cause companies to lose millions of dollars every year. On top of that, the big data world demands more and more software and computing power every day.
Fortunately, there is a method to that madness! No matter if you build your own software, or use open-source or paid systems, there are many efficiency gains that will save tons of money. Sometimes it’s a code optimization, sometimes algorithm adjustment, sometimes system-level operation!
In this talk Bartek, an open-source maintainer of observability projects like Prometheus and Thanos, author of the “Efficient Go” book and the CNCF TAG Observability Tech Lead, will explain how to notice and uncover efficiency problems effectively thanks to the power of modern cloud-native observability and tooling. The audience will learn pragmatic practices they can do to ensure effective and sustainable optimizations and how to avoid regressions in the future. All to increase the business operational margin and enable more opportunities with more efficient software and systems.
Presentation on using the Arrow library for enhanced Functional Programming in the Kotlin Language. Delivered at the Northern Ireland Developer Conference 2018.
The quality of the python ecosystem - and how we can protect it!Bruno Rocha
The Python ecosystem is supported by some pillars that are
- community,
- theoretical material,
- tools,
- libraries,
- PSF
- and language itself.
In this talk I would like to reflect on each of these pillars of the ecosystem
What are the priorities and in terms of quality what are the vulnerabilities of each of them.
I will mention the importance of all but focus on the quality of the ecosystem of libraries, tools and theoretical material.
The reflection will be around answering some questions:
- How to maintain the quality of libraries published in PyPI?
- What are the biggest vulnerabilities and how can we help avoid the risks?
- The importance of quality theoretical material (generated by the community)
- Can we trust everything that is available in PyPI?
- Are ecosystem teaching and documentation approaches safe, inclusive and easy to assimilate?
- What can we do to help solve the problems identified?
I will present some real cases and examples of problems encountered and security issues involving mainly PyPI
Scala in-practice-3-years by Patric Fornasier, Springr, presented at Pune Sca...Thoughtworks
3 years ago, Springer decided to use Scala on a large, strategic project. This talk is about the journey the development teams made. Why did they choose Scala in the first place? Did they get what they hoped for? What challenges and surprises did they encounter along the way? And, most importantly, are they still happy with their choice?
Slides from a talk at a meetup organized by SF Scala at Spotify's San Francisco office. The slides present details of playlist recommendations at Spotify and how Spotify uses Scalding to develop robust and reliable pipelines to generate these recommendations.
Meetup details: http://www.meetup.com/SF-Scala/events/224430674/
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
7. Why don’t we have…
• Extract Transform Load using code as configuration.
• Preferably Python code.
• Something that can be tested (I mean, by a machine).
• Something that can use inheritance.
• Quick install on a laptop, great on servers too.
35. State of Bonobo ETL
• First commit : December 2016
• 25 releases, ~565 commits, 12 contributors
• Current « stable » 0.5.1
• Target : 1.0 early 2018 when it’s ready
36. Small scale
… < Tb
One minute to install. Easy to deploy
It is a Lean Manufacturing Toolkit for Data.