The document discusses improvements made to the tDiary blogging engine to improve performance. Profiling with StackProf identified areas for optimization, including replacing the REXML XML parser with Oga for faster parsing, and improving the calendar generation method. Benchmarks show the changes led to a 4x speedup of the Amazon plugin and over 1.2x speedup of calendar generation. Loading plugins was also optimized by evaluating the code in a single pass rather than per file.
Video presentation: https://www.youtube.com/watch?v=jLAFXQ1Av50
Most applications written in Ruby are great, but also exists evil code applying WOP techniques. There are many workarounds in several programming languages, but in Ruby, when it happens, the proportion is bigger. It's very easy to write Ruby code with collateral damage.
You will see a collection of bad Ruby codes, with a description of how these codes affected negatively their applications and the solutions to fix and avoid them. Long classes, coupling, misapplication of OO, illegible code, tangled flows, naming issues and other things you can ever imagine are examples what you'll get.
Video presentation: https://www.youtube.com/watch?v=jLAFXQ1Av50
Most applications written in Ruby are great, but also exists evil code applying WOP techniques. There are many workarounds in several programming languages, but in Ruby, when it happens, the proportion is bigger. It's very easy to write Ruby code with collateral damage.
You will see a collection of bad Ruby codes, with a description of how these codes affected negatively their applications and the solutions to fix and avoid them. Long classes, coupling, misapplication of OO, illegible code, tangled flows, naming issues and other things you can ever imagine are examples what you'll get.
Slides for my talk at SkyCon'12 in Limerick.
Here I've squeezed four talks into one, covering a lot of ground quickly, so I've included links to more detailed presentations and other resources.
An overview of the main questions/design issues when starting to work with databases in Perl
- choosing a database
- matching DB datatypes to Perl datatypes
- DBI architecture (handles, drivers, etc.)
- steps of DBI interaction : prepare/execute/fetch
- ORM principles and difficulties, ORMs on CPAN
- a few examples with DBIx::DataModel
- performance issues
First given at YAPC::EU::2009 in Lisbon. Updated version given at FPW2011 in Paris and YAPC::EU::2011 in Riga
Gisting is an implementation of Google\'s MapReduce framework for processing and extracting useful information from very large data sets. At the time of this writing, the code is available for PREVIEW at http://github.com/mchung/gisting. I am currently working to release this framework for general usage.
Roll Your Own API Management Platform with nginx and LuaJon Moore
We recently replaced a proprietary API management solution with an in-house implementation built with nginx and Lua that let us get to a continuous delivery practice in a handful of months. Learn about our development process and the overall architecture that allowed us to write minimal amounts of code, enjoying native code performance while permitting interactive codeing, and how we leveraged other open source tools like Vagrant, Ansible, and OpenStack to build an automation-rich delivery pipeline. We will also take an in-depth look at our capacity management approach that differs from the rate limiting concept prevalent in the API community.
Application Logging in the 21st century - 2014.keyTim Bunce
Slides for my talk at the Austrian Perl Workshop in Salzburg on October 10th.
A video of the talk can be found at https://www.youtube.com/watch?v=4Qj-_eimGuE
Slides for my talk at the London Perl Workshop in Nov 2013, featuring the Devel::SizeMe perl module.
See also the screencast at https://archive.org/details/Perl-Memory-Profiling-LPW2013
Keep hearing about Plack and PSGI, and not really sure what they're for, and why they're popular? Maybe you're using Plack at work, and you're still copying-and-pasting `builder` lines in to your code without really knowing what's going on? What's the relationship between Plack, PSGI, and CGI? Plack from first principles works up from how CGI works, the evolution that PSGI represents, and how Plack provides a user-friendly layer on top of that.
Slides for my talk at SkyCon'12 in Limerick.
Here I've squeezed four talks into one, covering a lot of ground quickly, so I've included links to more detailed presentations and other resources.
An overview of the main questions/design issues when starting to work with databases in Perl
- choosing a database
- matching DB datatypes to Perl datatypes
- DBI architecture (handles, drivers, etc.)
- steps of DBI interaction : prepare/execute/fetch
- ORM principles and difficulties, ORMs on CPAN
- a few examples with DBIx::DataModel
- performance issues
First given at YAPC::EU::2009 in Lisbon. Updated version given at FPW2011 in Paris and YAPC::EU::2011 in Riga
Gisting is an implementation of Google\'s MapReduce framework for processing and extracting useful information from very large data sets. At the time of this writing, the code is available for PREVIEW at http://github.com/mchung/gisting. I am currently working to release this framework for general usage.
Roll Your Own API Management Platform with nginx and LuaJon Moore
We recently replaced a proprietary API management solution with an in-house implementation built with nginx and Lua that let us get to a continuous delivery practice in a handful of months. Learn about our development process and the overall architecture that allowed us to write minimal amounts of code, enjoying native code performance while permitting interactive codeing, and how we leveraged other open source tools like Vagrant, Ansible, and OpenStack to build an automation-rich delivery pipeline. We will also take an in-depth look at our capacity management approach that differs from the rate limiting concept prevalent in the API community.
Application Logging in the 21st century - 2014.keyTim Bunce
Slides for my talk at the Austrian Perl Workshop in Salzburg on October 10th.
A video of the talk can be found at https://www.youtube.com/watch?v=4Qj-_eimGuE
Slides for my talk at the London Perl Workshop in Nov 2013, featuring the Devel::SizeMe perl module.
See also the screencast at https://archive.org/details/Perl-Memory-Profiling-LPW2013
Keep hearing about Plack and PSGI, and not really sure what they're for, and why they're popular? Maybe you're using Plack at work, and you're still copying-and-pasting `builder` lines in to your code without really knowing what's going on? What's the relationship between Plack, PSGI, and CGI? Plack from first principles works up from how CGI works, the evolution that PSGI represents, and how Plack provides a user-friendly layer on top of that.
SparkR - Play Spark Using R (20160909 HadoopCon)wqchen
1. Introduction to SparkR
2. Demo
Starting to use SparkR
DataFrames: dplyr style, SQL style
RDD v.s. DataFrames
SparkR on MLlib: GLM, K-means
3. User Case
Median: approxQuantile()
ID Match: dplyr style, SQL style, SparkR function
SparkR + Shiny
4. The Future of SparkR
go-git is a 100% Go libray used to interact with git repositories. Even if it already supports most of the functionality it still lags a bit in performance when compared with the git CLI or some other libraries. I'll explain some of the problems that we face when dealing with git repos and some examples of performance improvements done to the library.
ManageIQ currently runs on Ruby on Rails 3. Aaron "tenderlove" Patterson presents his effort to migrate to RoR 4, which entails some changes in the code to take advantage of the latest advances in RoR.
For more on ManageIQ, see http://manageiq.org/
These are the slides of my talk at iOSCon 2017: https://skillsmatter.com/skillscasts/9549-architecting-alive-apps
Our apps are ever more alive. They interact with the rest of the world talking to backends and receiving notifications from them. They get their input from us and from other sensors. They are even aware of the location of the device they run in, or its position. But, in our IoT world, they may also detect presence in a room, get the temperature of it, or change the color of its lights.
Sadly enough, many of the apps available today with those capabilities have some architectural limitations:
Many of them are written in a way that is really dependent on a specific hardware.
Some restrict their use cases to whatever is provided by the hardware devices.
And almost all of them expect having a connection with the real device as the only way to test if they work properly.
However, we can also use an advanced architecture, like the Clean Architecture, to create a beautiful, scalable, testable, and robust application. Join Jorge and he will share with you how you can do it!
This is a "Code or it didn't happen" (TM) talk.
Go beyond the documentation and explore some of what's possible if you stretch symfony to its limits. We will look at a number of aspects of symfony 1.4 and Doctrine 1.2 and tease out some powerful functionality you may not have expected to find, but will doubtless be able to use. Topics covered will include routing, forms, the config cache and record listeners. If you're comfortable in symfony and wondering what's next, this session is for you.
A talk on debugging of Python applications given at a local KharkivPy event.
A brief introduction into a set of tools that allow Python developers to debug common issues in their applications.
The Perforce Web Content Management System development team, lacking a pre-existing solution in PHP, designed and implemented their own object model and record layer to ease the interaction of the system with the Perforce Server. This session will focus on how users can access files in Perforce via a simple CRUD API, the subsystems exposed, and their usage.
Keeping Spark on Track: Productionizing Spark for ETLDatabricks
ETL is the first phase when building a big data processing platform. Data is available from various sources and formats, and transforming the data into a compact binary format (Parquet, ORC, etc.) allows Apache Spark to process it in the most efficient manner. This talk will discuss common issues and best practices for speeding up your ETL workflows, handling dirty data, and debugging tips for identifying errors.
Speakers: Kyle Pistor & Miklos Christine
This talk was originally presented at Spark Summit East 2017.
Building Lithium Apps (Like a Boss) was a workshop presented on the structure and philosophy of the Lithium framework and its applications, and how best to take advantage of them.
Long journey of Ruby Standard library at RubyKaigi 2024Hiroshi SHIBATA
Ruby has a lot of standard libraries from Ruby 1.8. I promote them democratically with GitHub today via default and bundled gems. So, I'm working to extract them for Ruby 3.4 continuously and future versions. It's long journey for me.
After that, some versions may suddenly happen LoadError at require when running bundle exec or bin/rails, for example matrix or net-smtp. We need to learn what's difference default/bundled gems with standard libraries.
In this presentation, I will introduce what's the difficult to extract bundled gems from default gems and the details of the functionality that Ruby's require and bundle exec with default/bundled gems. You can learn how handle your issue about standard libraries.
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
I will introduce what's the difficult to extract bundled gems from default gems and the details of the functionality that Ruby's require and bundle exec with default/bundled gems. You can learn how handle your issue about standard libraries.
Deep dive into Ruby's require - RubyConf Taiwan 2023Hiroshi SHIBATA
Since Ruby's bundled and default gems change every year with each release, some versions may suddenly happen LoadError at require when running bundle exec or bin/rails, for example matrix or net-smtp.
In this presentation, I will introduce the details of the functionality that extends Ruby's require to provide guidance to users on what they can do to load them. And I will also show how $LOAD_PATH is build behind Ruby and Rails by Bundler.
How to develop the Standard Libraries of Ruby?Hiroshi SHIBATA
I maintain the RubyGems, Bundler and the standard libraries of the Ruby language. So, I've been extract many of the standard libraries to default gems and GitHub at Ruby 3.0. But the some of libraries still remains in only Ruby repository. I will describe these situation.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
10. benchmark-ips
require 'benchmark/ips'
Benchmark.ips do |x|
xml = File.read('../spec/fixtures/jpB00H91KK26.xml')
require_relative '../misc/plugin/amazon'
x.report('rexml') do
item = AmazonItem.new(xml)
amazon_detail_html( item )
end
x.report('oga') do
require 'oga'
item = AmazonItem.new(xml, :oga)
amazon_detail_html(item)
end
end
11. Results of Improvement
% ruby benchmark_amazon_plugin.rb
Warming up --------------------------------------
rexml 2.000 i/100ms
oga 14.000 i/100ms
Calculating -------------------------------------
rexml 37.678 (±15.9%) i/s - 182.000 in 5.013022s
oga 159.203 (±13.2%) i/s - 784.000 in 5.034065s
4.2 times faster!!1
12. Migration strategy for REXML and Oga
class AmazonItem
def initialize(xml, parser = :rexml)
@parser = parser
if parser == :oga
doc = Oga.parse_xml(xml)
@item = doc.xpath('*/*/Item')[0]
else
doc = REXML::Document::new( REXML::Source::new( xml ) ).root
@item = doc.elements.to_a( '*/Item' )[0]
end
end
def nodes(path)
if @parser == :oga
@item.xpath(path)
else
@item.elements.to_a(path)
end
end
end
14. tDiary のテストのしにくさよ…
ひたすら stub/mock module を用意する
class DummyTDiary
attr_accessor :conf
def initialize
@conf = DummyConf.new
@conf.data_path = TDiary.root + "/tmp/"
end
def ignore_parser_cache
false
end
end
class DummyConf
attr_accessor :data_path
def cache_path
TDiary.root + "/tmp/cache"
end
def options
{}
end
def style
"wiki"
end
end
module TDiary
PATH = File::dirname( __FILE__ ).untaint
class << self
def root
File.expand_path(File.join(library_root, '..'))
end
def library_root
File.expand_path('..', __FILE__)
end
def server_root
Dir.pwd.untaint
end
end
end
15. benchmark-ips(2)
conf = DummyConf.new
conf.data_path = TDiary.root + '/tmp/data/'
diary = DummyTDiary.new
diary.conf = conf
io = TDiary::IO::Default.new(diary)
require 'benchmark/ips'
Benchmark.ips do |x|
x.report('calendar') do
io.calendar
end
x.report('calendar2') do
io.calendar2
end
end
16. Improvement for calendar
def calendar
calendar = {}
Dir["#{@data_path}????"].sort.each do |dir|
next unless %r[/d{4}$] =~ dir
Dir["#{dir.untaint}/??????.td2"].sort.each do |file|
year, month = file.scan( %r[/(d{4})(dd).td2$] )[0]
next unless year
calendar[year] = [] unless calendar[year]
calendar[year] << month
end
end
calendar
end def calendar2
calendar = {}
Dir["#{@data_path}????/??????.td2"].sort.each do |file|
if file =~ /(d{4})(d{2}).td2$/
calendar[$1] = [] unless calendar[$1]
calendar[$1] << $2
end
end
calendar
end
17. benchmark-ips
% ruby benchmark_io_default.rb
Warming up --------------------------------------
calendar 16.000 i/100ms
calendar2 22.000 i/100ms
Calculating -------------------------------------
calendar 195.073 (±14.9%) i/s - 960.000 in 5.070197s
calendar2 237.695 (±13.9%) i/s - 1.166k in 5.039136s
1.21 times faster!!1