This document discusses finding and fixing production issues in a LAMP stack. It emphasizes having a clear aim or goal for the system and defining performance norms. It suggests monitoring tools, testing suites, and utilities to help audit and find problems outside of norms. The document also discusses optimizing through techniques like sharding, caching, and taking advantage of Moore's law.
[DevDay2019] Paradigm shift towards effective Scrum - By Tam Doan, Agile Coac...DevDay.org
Scrum has become one of the most popular Agile frameworks in IT, as its lightweight and easy to understand. But why is it so difficult to apply? One of the challenges of effectively applying Scrum comes from the basic understanding of why Scrum was initially created in the first place. Having this paradigm shift will significantly enhance becoming an effective Scrum Team member.
5 practical tips to make a successful big data projectOri Netzer
5 Practical Tips for Implementing A Successful Big Data Project in Healthcare
Big Data is a big word that might cause big challenges, or perhaps the opposite… Join me for 30 minutes session of practical tips for making it work like magic.
"Developing a complicated ensemble model with hundreds of features fetched from a bunch of different sources? Give me two! Showing great metrics to the stakeholders and already discussing how it will hit a home run in production? Why not! And then getting stuck for months trying to deploy the model and fighting with data inconsistency and bugs? Sounds familiar?
This talk will focus on providing guidelines on how to build your model development process keeping in mind the deployment phase to come later on."
I presented this talk at PyCon & PyData DE 2019 in Berlin and Data Science fwdays 2019 in Kyiv.
While working on Agile transformations, through multiple organisations, I have realized one common roadblock in achieving this vision; Managers. Read on to know why managers fear agile implementation
[DevDay2019] Paradigm shift towards effective Scrum - By Tam Doan, Agile Coac...DevDay.org
Scrum has become one of the most popular Agile frameworks in IT, as its lightweight and easy to understand. But why is it so difficult to apply? One of the challenges of effectively applying Scrum comes from the basic understanding of why Scrum was initially created in the first place. Having this paradigm shift will significantly enhance becoming an effective Scrum Team member.
5 practical tips to make a successful big data projectOri Netzer
5 Practical Tips for Implementing A Successful Big Data Project in Healthcare
Big Data is a big word that might cause big challenges, or perhaps the opposite… Join me for 30 minutes session of practical tips for making it work like magic.
"Developing a complicated ensemble model with hundreds of features fetched from a bunch of different sources? Give me two! Showing great metrics to the stakeholders and already discussing how it will hit a home run in production? Why not! And then getting stuck for months trying to deploy the model and fighting with data inconsistency and bugs? Sounds familiar?
This talk will focus on providing guidelines on how to build your model development process keeping in mind the deployment phase to come later on."
I presented this talk at PyCon & PyData DE 2019 in Berlin and Data Science fwdays 2019 in Kyiv.
While working on Agile transformations, through multiple organisations, I have realized one common roadblock in achieving this vision; Managers. Read on to know why managers fear agile implementation
So you KNOW you need to put something in place for Governance, but you have no idea where to start?In this session I'll share some practical knowledge on achieving this - as we have more than enough theory out there on this subject.
- What should Governance cover?- How to document this - and keep it updated and informative- Understanding why you need to do what (you need to understand the value it will add)
Equipped with these tips and tricks - you'll go back to work - excited to put your Governance strategy in place AND you'll understand the WHAT and the WHY.
Once a problem has been identified then what do we do? Do we just go with the best intentions and trust our gut instincts or are there different techniques to apply to problems? In this presentation, we will explore various problem solving techniques.
A Test Management Christmas Carol - Agile Testing Days 2014Tom Roden
A Test Management Christmas Carol - the ghosts of test management past, present and future.
A look into the evolution and revolution of the function of testing management in the maturing world of agile software development.
Scrum is simple. It has 11 elements. 5 Events, 3 Roles and 3 Artifacts. But even though Scrum is simple, it is hard. Hard to do right. Daily scrum’s only on Tuesdays? A Sprint Demo? The team asking for approval from their Product Owner at Sprint Review… These are only a few of the ways scrum can get corrupted. Let’s scan out collective Scrum hard dive and run chkdks /f on it to get rid of some of the bad sectors, rebuild some of the indexes and resolve the lost files.
So you KNOW you need to put something in place for Governance, but you have no idea where to start?In this session I'll share some practical knowledge on achieving this - as we have more than enough theory out there on this subject.
- What should Governance cover?- How to document this - and keep it updated and informative- Understanding why you need to do what (you need to understand the value it will add)
Equipped with these tips and tricks - you'll go back to work - excited to put your Governance strategy in place AND you'll understand the WHAT and the WHY.
Once a problem has been identified then what do we do? Do we just go with the best intentions and trust our gut instincts or are there different techniques to apply to problems? In this presentation, we will explore various problem solving techniques.
A Test Management Christmas Carol - Agile Testing Days 2014Tom Roden
A Test Management Christmas Carol - the ghosts of test management past, present and future.
A look into the evolution and revolution of the function of testing management in the maturing world of agile software development.
Scrum is simple. It has 11 elements. 5 Events, 3 Roles and 3 Artifacts. But even though Scrum is simple, it is hard. Hard to do right. Daily scrum’s only on Tuesdays? A Sprint Demo? The team asking for approval from their Product Owner at Sprint Review… These are only a few of the ways scrum can get corrupted. Let’s scan out collective Scrum hard dive and run chkdks /f on it to get rid of some of the bad sectors, rebuild some of the indexes and resolve the lost files.
OSMC 2015: Testing in Production by Devdas BhagatNETWAYS
For most ecommerce companies, software is not the final deliverable product. It is a research tool, to determine what customers will pay for. To be able to get good data from software, monitoring and analytics must be built into the system. Alerting must come from business requirements and be based on application generated data.
In the traditional operations world, we monitor what is easy, and avoid monitoring that which is difficult. This talk is an attempt to show people that monitoring must be driven by metrics from the CxO office, and then potentially involve technical metrics if needed.
This talk explains why functional and business level monitoring is crucial. We also cover the tradeoffs from a DTAP model to continuous deployment. There will be a brief introduction to a couple of useful monitoring tools for functional monitoring. No special technical skills are expected of the audience, but having a general overview of the monitoring world is a good thing. This talk is not limited to ecommerce companies, but is most applicable to that environment.
Practical DevSecOps: Fundamentals of Successful ProgramsMatt Tesauro
From ONUG Fall 2022:
"Shift Left'' and automation have turned from ideals to meaningless buzzwords. Instead of riding the hype train, let's get real and cover practical and real-world examples taken from actual product security successes. Not every business is the same, neither will their DevSecOps program.
In this talk, I'll cover the fundamentals of common to successful DevSecOps programs as well as a grab bag of useful techniques to consider. These are lessons learned doing AppSec at a wide variety of companies including Rackspace, Pearson, a fortune 500 financial, Duo Security and Cognizant Healthcare. Bruce Lee said "Research your own experience. Absorb what is useful, reject what is useless, add what is essentially your own". The goal of this talk is to provide you with enough examples to build your own pragmatic and practical DevSecOps program or maybe absorb a new technique or two into your existing program.
This talk will focus on Techniques, metrics and different tests (code, models, infra and features/data) that help the developers of machine learning systems to achieve CD.
안녕하세요 딥러닝 논문읽기 모임입니다.
오늘 소개 드릴 논문은 2021 ICLR에 억셉이된
'The Deep Bootstrap Framework:Good Online Learners are good Offline Generalizers ' 라는 논문입니다.
오늘 발표를 위해 펀디 멘탈팀 이재윤님이 리뷰 도와주셨습니다.
문의 tfkeras@kakao.com
How do you maintain millions of lines of code for a project in the Julia programming language? In this talk we will showcase how the Julia SciML organization grew from being research solvers into an industrial-scale platform which is used throughout many highly regulated industries such as pharmacology and aerospace. How do you develop Julia code that is safe for highly regulated environments? Chris Rackauckas will walk through many of the major tenants of the SciML style of contributions: the pieces which has allowed the list of contributors to grow far beyond what's seen in many of the Python and R developer ecosystems.
After having run SDLC projects for more than 8 years, I made the transition to agile projects. This deck captures some of the insights from that journey. Read more here - http://restlesstempest.blogspot.in/2013/06/article-transition-from-sdlc-to-agile.html
Test Improvement - Any place, anytime, any whereRuud Teunissen
Test Improvement is all about giving an organization or a team the “means they can use” to help achieve their goals. Means that are in line with their skills and they can use in their context. That’s why successful Test Improvement requires leadership and management. In this presentation I share experiences in Test Improvement in a wide variety of environments, using different models and approaches.
A presentation outlining our experiences a year after adopting Scrum at Future Platforms, a software company based in Brighton, UK.
Presented by Tom Hume and Joh Hunt at The Werks, Brighton, on 15th October 2008
When designing, building, and maintaining a computer system, one can ask the following questions: "Is my data safe from being accidentally deleted or corrupted? How do I ensure data integrity in the long term?"
The main goal of the presentation is to analyze several data integrity pitfalls and review recommended solutions, so that you can construct a data integrity strategy appropriate for your service and your DevOps team.
Database basics for new-ish developers -- All Things Open October 18th 2021Dave Stokes
Do you wonder why it takes your database to find the top five of your fifty six million customers? Do you really have a good idea of what NULL is and how to use it? And why are some database queries so quick and others frustratingly slow? Relational databases have been around for over fifty years and frustrating developers for at least forty nine of those years. This session is an attempt to explain why sometimes the database seems very fast and other times not. You will learn how to set up data (normalization) to avoid redundancies into tables by their function, how to join two tables to combine data, and why Structured Query Language is so very different than most other languages. And you will see how thinking in sets over records can greatly improve your life with a database.
Longhorn PHP - MySQL Indexes, Histograms, Locking Options, and Other Ways to ...Dave Stokes
Slow query? Add an index or two! But things are suddenly even slower! Indexes are great tools to speed data lookup but have overhead issues. Histograms don’t have that overhead but may not be suited. And how you lock rows also effects performance. So what do you do to speed up queries smartly? This is a clear, detailed look at what you can do to really speed up your queries in a logical, orthogonal process. Updated Presentation for Longhorn PHP Conference October 14th, 2021
MySQL 8.0 New Features -- September 27th presentation for Open Source SummitDave Stokes
MySQL 8.0 has many new features that you probably need to know about but don't. Like default security, window functions, CTEs, CATS (not what you think), JSON_TABLE(), and UTF8MB4 support.
JavaScript and Friends August 20th, 20201 -- MySQL Shell and JavaScriptDave Stokes
The MySQL Shell has a JavaScript mode where you can use JS libraries to access you data and you can also write (and save) your own custom reports (or programs) for future use.
Dutch PHP Conference 2021 - MySQL Indexes and HistogramsDave Stokes
Slides from the 2021 Dutch PHP Conference on MySQL Indexes, histograms, and other things to speed up your database queries. Speeding up your database queries is mainly learning how to efficiently give the query optimizer what is needs to provide the best query plan for your data.
Validating JSON -- Percona Live 2021 presentationDave Stokes
JSON is a free form data exchange format which can cause problems when combined with a strictly typed relational database. Thanks to the folks at https://json-schema.org and the MySQL engineers at Oracle we can no specify required fields, type checks, and range checks.
Data Love Conference - Window Functions for Database AnalyticsDave Stokes
16 April 2021 presentation for the Data Love Conference on Window Functions for Data Base Analytics. Examples are on MySQL but will work for other RDMS's with window functions. Assumes no user background on window functions or analytics
Open Source 1010 and Quest InSync presentations March 30th, 2021 on MySQL Ind...Dave Stokes
Speeding up queries on a MySQL server with indexes and histograms is not a mysterious art but simple engineering. This presentation is an indepth introduction that was presented on March 30th to the Quest Insynch and Open Source 101 conferences
Confoo.ca conference talk February 24th 2021 on MySQL new features found in version 8.0 including server and supporting utility updates for those who may have missed some really neat new features
Confoo 2021 - MySQL Indexes & HistogramsDave Stokes
Confoo 2021 presentation on MySQL Indexes, Histograms, and other ways to speed up your queries. This slide deck has slides that may not have been included in the presentation that were omitted due to time constraints
A Step by Step Introduction to the MySQL Document StoreDave Stokes
Looking for a fast, flexible NoSQL document store? And one that runs with the power and reliability of MySQL. This is an intro on how to use the MySQL Document Store
2. Gorilla in cancer slide
Recently an image of a gorilla
was 'added' to slides of cancer
cells and reviewed by a group of
radiologists.
3. Gorilla in cancer slide
Recently an image of a gorilla
was 'added' to slides of cancer
cells and reviewed by a group of
radiologists.
The gorilla was recognized only
by 17% of the radiologists!
5. Who? Me worry?
● If you relied on the work of the
radiologists, would you be
concerned?
● Lets assume form here on your
LAMP stack is tuned, cached,
tweaked optimally.
6. Who? Me worry?
● If you relied on the work of the
radiologists, would you be
concerned?
7. Do not forget
The output from one woman
pregnant for nines months is
much different than that of nine
women pregnant for one month
– Werner Von Braun
9. What are your trying
to maximize/minimize
● What is the goal?
● Work systemically
● Optimization needs to be a
continuous process
● Be able to back out changes!!!!
● Really think
10. Who said?
"What is a system? A system is a network of
interdependent components that work together to
try to accomplish the aim of the system. A system
must have an aim. Without an aim, there is no
system. The aim of the system must be clear to
everyone in the system. The aim must include
plans for the future. The aim is a value judgment.
(We are of course talking here about a man-
made system.)”
11. Must have an aim
W. Edwards ● What is the
Demming 'aim' of your
LAMP stack?
● Hopefully not
'ready, fire,
aim'
12. Yogi Berra
If you don't know where you are going, you might
wind up someplace else.
13. Define
● What are your 'norms'
– Avg/Min/Max times
– End to end times or per section
– Reasonable variances
– Know your service levels
14. H. L. Mencken
For every complex problem there is an answer
that is clear, simple, and wrong.
15. Costs
● What is the ● What is the
cost of a cost of
query? Time, storage/
resources, retrieval?
and/or $ ● Growth deltas
17. Finding problems
● Outside of ● Re-norm-ing
norms – New and/or
– Change improved
causes – New
– Regression benchmark
18. Pipeline Pig
● Pigging in the context of
pipelines refers to the practice
of using pipeline inspection
gauges or 'pigs' to perform
various maintenance
operations on a pipeline. This
is done without stopping the
flow of the product in the
pipeline.
– How could you adapt this
idea for your environment?
19. Audit
● Random
● Every N
– Start to finish
– Section
20. Tools
● Testing suites ● Monitoring
– $$ tools
– No P-N-P
● Utilities, general system
– Systat, iostat, ps, show user