Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Caching and tuning fun for high scalabilityWim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond PHP - It's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond PHP - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just writing PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Caching and tuning fun for high scalabilityWim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Beyond php it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Caching and tuning fun for high scalabilityWim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond PHP - It's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond PHP - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just writing PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Caching and tuning fun for high scalabilityWim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Beyond php it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Caching and tuning fun for high scalability @ LOAD2012Wim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Beyond PHP - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just writing PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Caching and tuning fun for high scalability @ PHPTourWim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you. In this tutorial, we'll start from a Zend Framework based site. We'll add caching, begin to add servers and replace the standard LAMP stack, all while performing live benchmarks.
Remove php calls and scale your site like crazy !Wim Godden
Although tools like Varnish can improve performance and scalability for static sites, when user-specific content is needed, a hit to the PHP/Ruby/Python/.Net backend is still required, causing scalability issues. We'll look at a brand-new Nginx module which implements an ultra-fast and scalable solution to this problem, changing the way developers think about designing sites with user-specific content.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
When dynamic becomes static: the next step in web caching techniquesWim Godden
Although tools like Varnish can improve performance and scalability for static sites, when user-specific content is needed, a hit to the PHP/Ruby/Python/.Net backend is still required, causing scalability issues. We'll look at a brand-new Nginx module which implements an ultra-fast and scalable solution to this problem, changing the way developers think about designing sites with user-specific content.
Caching and tuning fun for high scalability @ FOSDEM 2012Wim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Service discovery and configuration provisioningSource Ministry
Slides from our talk "Service discovery and configuration provisioning" presented by Mariusz Gil at PHP Benelux 2016
Apache Zookeeper or Consul are almost completely unknown in the PHP world, although its use solves a lot of typical problems. In a nutshell, they are a central services of provisioning configuration information, distributed synchronization and coordination of servers/processes. It simplifies the processes of application configuration management, so it is possible to change its settings and operation in real time (eg. feature flagging). During the presentation the typical cases of use of Zookeeper/Consul in PHP applications will be presented, both strictly web and workers running from the CLI.
When high performance on a web application is a hard requirement Varnish can be of rescue. But does it’s name, the high-performance HTTP accelerator, really bring what you expect? What are the caveats, pitfalls and problems you introduce when developing your application when the released version is only able to run when there is a Varnish in front? This session will give you some answers, tips and tricks to aid in application design, development with PHP and solutions when there is no Varnish in front of your application.
We all have tasks from time to time for bulk-loading external data into MySQL. What's the best way of doing this? That's the task I faced recently when I was asked to help benchmark a multi-terrabyte database. We had to find the most efficient method to reload test data repeatedly without taking days to do it each time. In my presentation, I'll show you several alternative methods for bulk data loading, and describe the practical steps to use them efficiently. I'll cover SQL scripts, the mysqlimport tool, MySQL Workbench import, the CSV storage engine, and the Memcached API. I'll also give MySQL tuning tips for data loading, and how to use multi-threaded clients.
37 slides about taking care of your SolrCluster - Collections API, Core API, dynamic schema modification, segment merging, hard vs. soft commit, caches, monitoring, performance, JMX, it's all in here.
A high level introduction to Apache Cassandra followed by an introduction to pycassa, the Python client library for Cassandra.
Presented at PyTexas 2011 by Tyler Hobbs.
MySQL exposes a collection of tunable parameters and indicators that is frankly intimidating. But a poorly tuned MySQL server is a bottleneck for your PHP application scalability. This session shows how to do InnoDB tuning and read the InnoDB status report in MySQL 5.5.
Gamification for participation STC NY MetroJLSagan
Pacesetter Presentation for the Leadership Program at the STC Summit 2015. Topic covered: Gamification for Participation. NOTE: I am not the author of this presentation; I am just posting it via my account.
Caching and tuning fun for high scalability @ LOAD2012Wim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Beyond PHP - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just writing PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Caching and tuning fun for high scalability @ PHPTourWim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you. In this tutorial, we'll start from a Zend Framework based site. We'll add caching, begin to add servers and replace the standard LAMP stack, all while performing live benchmarks.
Remove php calls and scale your site like crazy !Wim Godden
Although tools like Varnish can improve performance and scalability for static sites, when user-specific content is needed, a hit to the PHP/Ruby/Python/.Net backend is still required, causing scalability issues. We'll look at a brand-new Nginx module which implements an ultra-fast and scalable solution to this problem, changing the way developers think about designing sites with user-specific content.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
When dynamic becomes static: the next step in web caching techniquesWim Godden
Although tools like Varnish can improve performance and scalability for static sites, when user-specific content is needed, a hit to the PHP/Ruby/Python/.Net backend is still required, causing scalability issues. We'll look at a brand-new Nginx module which implements an ultra-fast and scalable solution to this problem, changing the way developers think about designing sites with user-specific content.
Caching and tuning fun for high scalability @ FOSDEM 2012Wim Godden
Caching has been a 'hot' topic for a few years. But caching takes more than merely taking data and putting it in a cache : the right caching techniques can improve performance and reduce load significantly. But we'll also look at some major pitfalls, showing that caching the wrong way can bring down your site. If you're looking for a clear explanation about various caching techniques and tools like Memcached, Nginx and Varnish, as well as ways to deploy them in an efficient way, this talk is for you.
Service discovery and configuration provisioningSource Ministry
Slides from our talk "Service discovery and configuration provisioning" presented by Mariusz Gil at PHP Benelux 2016
Apache Zookeeper or Consul are almost completely unknown in the PHP world, although its use solves a lot of typical problems. In a nutshell, they are a central services of provisioning configuration information, distributed synchronization and coordination of servers/processes. It simplifies the processes of application configuration management, so it is possible to change its settings and operation in real time (eg. feature flagging). During the presentation the typical cases of use of Zookeeper/Consul in PHP applications will be presented, both strictly web and workers running from the CLI.
When high performance on a web application is a hard requirement Varnish can be of rescue. But does it’s name, the high-performance HTTP accelerator, really bring what you expect? What are the caveats, pitfalls and problems you introduce when developing your application when the released version is only able to run when there is a Varnish in front? This session will give you some answers, tips and tricks to aid in application design, development with PHP and solutions when there is no Varnish in front of your application.
We all have tasks from time to time for bulk-loading external data into MySQL. What's the best way of doing this? That's the task I faced recently when I was asked to help benchmark a multi-terrabyte database. We had to find the most efficient method to reload test data repeatedly without taking days to do it each time. In my presentation, I'll show you several alternative methods for bulk data loading, and describe the practical steps to use them efficiently. I'll cover SQL scripts, the mysqlimport tool, MySQL Workbench import, the CSV storage engine, and the Memcached API. I'll also give MySQL tuning tips for data loading, and how to use multi-threaded clients.
37 slides about taking care of your SolrCluster - Collections API, Core API, dynamic schema modification, segment merging, hard vs. soft commit, caches, monitoring, performance, JMX, it's all in here.
A high level introduction to Apache Cassandra followed by an introduction to pycassa, the Python client library for Cassandra.
Presented at PyTexas 2011 by Tyler Hobbs.
MySQL exposes a collection of tunable parameters and indicators that is frankly intimidating. But a poorly tuned MySQL server is a bottleneck for your PHP application scalability. This session shows how to do InnoDB tuning and read the InnoDB status report in MySQL 5.5.
Gamification for participation STC NY MetroJLSagan
Pacesetter Presentation for the Leadership Program at the STC Summit 2015. Topic covered: Gamification for Participation. NOTE: I am not the author of this presentation; I am just posting it via my account.
OpenStreetMap as base layer in a linked open data distribution platform - Ber...OSMFstateofthemap
The CitySDK Mobility API is a layer-based data distribution and service kit, which makes European mobility data and services interoperable. The API is part of CitySDK, a project funded by the European Union in which eight cities and more than 20 organisations collaborate.
The CitySDK Mobility API makes it possible for developers and data owners to access and modify mobility data (e.g. public transport schedules, real-time traffic data, planned roadwork and parking space availability) through a uniform interface, in Amsterdam, Manchester, Helsinki and Rome alike. The API links different datasets from different data sources to addressable objects in a city, such as train stations, bus stops, roads and neighbourhoods.
Many of those objects exist in the OpenStreetMap database; the CitySDK Mobility API uses OSM as a geospatial base layer to which it links data from other datasets. For example: GTFS schedules, Open311 service requests and real-time weather information can all be linked to a OSM node with the ""highway=bus_stop"" tag, making it very easy for data owners, city officials and developers to find and access data about the same object across different data sets.
Developer's portal: http://dev.citysdk.waag.org/
API endpoint: http://api.citysdk.waag.org/
Métricas de vaidade podem interferir muito o progresso realista de novas funcionalidades, no post a seguir apresento minhas idéias sobre o mesmo.
http://www.helmed.net/blog/2012/12/18/sucesso-na-medida-certa-metricas-de-vaidade/
The challenge of being a team representative in a scaled agile environmentagile42
Babbel has experienced enormous growth in the past years. The number of teams and people requires new ways of handling information and decisions. One approach established at Babbel was to streamline the cross-team information flow by inviting team representatives to the overall planning and retrospective meetings. At agile42 Connect in Berlin, November 2015, Roy Hausmann has presented some of the findings with this approach and what they have started in order to address the issues.
A version of my Rapid Product Design in the Wild talk at Agile Iceland 2014. http://www.agileisland.is
How do you know you're developing the right product? This talk will help you think creatively about how to do customer development using Agile and Lean User Experience methods. I share what we learnt about using rapid, iterative prototyping techniques to develop a minimum viable product at a software conference.
In August 2012 we attended Kscope, a conference for Oracle developers. Instead of doing the usual product demonstrations, we turned our stand into a live lab and took Agile development processes out of the office and in front of our customers. Our stand included an area for customer research, a Kanban board and information radiators in the form of a whiteboard, blank wall and a large digital screen. Over 3 days we ran 9 sprints and conducted 25 customer interviews, using a paper prototype to get feedback. We collected invaluable information about our customers' development environments, how they work with their teams, their processes, tasks and pain points. By the end of the conference my colleague had developed an interactive HTML/CSS prototype which potential customers could evaluate. The team went through several rapid build-measure-learn cycles to improve our product concept and validate the market need.
Opening up our development process at a trade show provided visitors to the stand with an opportunity to experience Agile and Lean methods first-hand.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Beyond php - it's not (just) about the codeWim Godden
Most PHP developers focus on writing code. But creating Web applications is about much more than just wrting PHP. Take a step outside the PHP cocoon and into the big PHP ecosphere to find out how small code changes can make a world of difference on servers and network. This talk is an eye-opener for developers who spend over 80% of their time coding, debugging and testing.
Wszyscy zostaliśmy oszukani! Automatyczne zarządzanie pamięci rozwiąże wszystkie Wasze problemy, mówili. W zarządzanych środowiskach takich jak CLR JVM nie będzie wycieków pamięci, mówili! Właściwie pamięć jest tania i nie musisz się już nią nigdy więcej martwić. Wszyscy kłamali. Automatyczne zarządzanie pamięcią jest wygodną abstrakcją i bardzo często działa dobrze. Ale jak każda abstrakcja, wcześniej czy później "wycieka" ona. I to najczęściej w najmniej spodziewanym i przyjemnym momencie. W tej sesji spróbuję otworzyć oczy na fakt, że błoga nieświadomość nt. tej abstrakcji może być kosztowna. Pokażę jak może się objawić frywolne traktowanie pamięci i co możemy zyskać pisząc kod zdając sobie sprawę, że pamięć jednak nie jest nieskończona, tania i zawsze jednakowo szybka.
Helping travelers make better hotel choices - 500 million times a month
TrustYou analyzes online hotel reviews to create a summary for every hotel in the world. What do travelers think of the service? Is this hotel suitable for business travelers? TrustYou data is integrated on countless websites (Trivago, Wego, Kayak), helping travelers make better choices. Try it out yourself on http://www.trust-score.com/
TrustYou runs almost exclusively on Python. Every week, we find 3 million new hotel reviews on the web, process them, analyze the text using Natural Language Processing, and update our database of 600,000 hotels. In this talk, Steffen will give insights into how Python is used at TrustYou to collect, analyze and visualize these large amounts of data.
In an R&D company fast prototyping is vital to develop new projects or proofs of concept quickly and inexpensively. In this talk we will demonstrate how real fast and agile development can be achieved with MongoDB and dynamic languages, with examples and best practices. All the code shown is already uploaded to a public Git repository - https://github.com/pablito56/py-eshop
This presentation provides an introduction to Azure DocumentDB. Topics include elastic scale, global distribution and guaranteed low latencies (with SLAs) - all in a managed document store that you can query using SQL and Javascript. We also review common scenarios and advanced Data Sciences scenarios.
How to process data using events on top of PHP and MongoDB. Introducing Eventsourcing and CQRS, how to handle events and how to generate Read Models and Aggregates leveraging the MongoDB Aggregation Framework.
Penetration Testing involves a lot of repetitive manual processes. This includes the execution of a multitude of security tools. These are traditionally executed based upon the analysis of an analyst over the duration of a vulnerability assessment. Automating a heuristic process allows an attacker additional resources for more valuable tasks through the automation of the acquisition, execution and information collection process.
A tool framework was developed by over the last few months effectively gluing over 30 unique security tools together. Each of these tools are selectively executed based of your targets available networked services dynamically.
The tools include a collection of open source, custom and commercial software with varying licensing requirements.
Hacking With Glue ℠
This talk is focused on tuning analysing and optimizing MongoDB query and index with the use of Database Profiler and "explain()" function.
Also, performance of database can also be impacted by configuring the underline ( Linux ) OS with some recommended settings which do not come by default.
Similar to Beyond php - it's not (just) about the code (20)
Who would have thought putting 140 charachter messages about one's life online or having a virtual farm game could ever be popular ? Then again, many of us have those weird (but sometimes brilliant) ideas.
But no matter how incredible your ideas might be, getting them launched successfully takes more than writing lots of php code, smacking a sleek design on it and dropping it on a server.
So what does it take ? Where do most ideas crashland and how can you avoid making the same mistakes and transform your ideas into reality ? We'll look at what steps are needed to make a service successful and sustainable.
With PHP 8.0 recently released and PHP 5.x still accounting for over 40% of all production environments, it's time to paint a clear picture on not just why everyone should move to 8.x, but on how to get code ready for the latest version of PHP. In this talk, we'll look at some handy tools and techniques to ease the migration.
With PHP 7.2 recently released and PHP 5.3 and 5.4 still accounting for over 40% of all production environments, it's time to paint a clear picture on not just why everyone should move to 7.0 (or preferably 7.1), but on how to get code ready for the latest version of PHP.
Using the version compatibility checker for PHP_CodeSniffer and a few simple step-by-step instructions, upgrading old code to make it compatible with the latest PHP versions becomes actually really easy. In this talk, we'll migrate an old piece of code and get rid of the demons of the past and ready for the present and future.
With more and more sites falling victim to data theft, you've probably read the list of things (not) to do to write secure code. But what else should you do to make sure your code and the rest of your web stack is secure ? In this tutorial we'll go through the basic and more advanced techniques of securing your web and database servers, securing your backend PHP code and your frontend javascript code. We'll also look at how you can build code that detects and blocks intrusion attempts and a bunch of other tips and tricks to make sure your customer data stays secure.
With more and more sites falling victim to data theft, you've probably read the list of things (not) to do to write secure code. But what else should you do to make sure your code and the rest of your web stack is secure ? In this tutorial we'll go through the basic and more advanced techniques of securing your web and database servers, securing your backend PHP code and your frontend javascript code. We'll also look at how you can build code that detects and blocks intrusion attempts and a bunch of other tips and tricks to make sure your customer data stays secure.
The time of static or dynamically generated sites is long gone. Non-stop interaction with users is the new normal. However, polling with Ajax requests is processor intensive and cumbersome. Websockets allow you to interact with users in real-time without increasing system load. We'll go through the basics and see all the different options, illustrated with live examples of how and when to use it, as well as when not to use it.
Who would have thought putting 140 charachter messages about one's life online or having a virtual farm game could ever be popular ? Then again, many of us have those weird (but sometimes brilliant) ideas.
But no matter how incredible your ideas might be, getting them launched successfully takes more than writing lots of php code, smacking a sleek design on it and dropping it on a server.
So what does it take ? Where do most ideas crashland and how can you avoid making the same mistakes and transform your ideas into reality ? We'll look at what steps are needed to make a service successful and sustainable.
Your app lives on the network - networking for web developersWim Godden
Our job might be to build web applications, but we can't build apps that rely on networking if we don't know how these networks and the big network that connects them all (this thing called the Internet) actually work.
I'll walk through the basics of networking, then dive a lot deeper (from TCP/UDP to IPv4/6, source/destination ports, sockets, DNS and even BGP).
Prepare for an eye-opener when you realize how much a typical app relies on all of these (and many more) working flawlessly... and how you can prepare your app for failure in the chain.
With PHP 7.2 recently released and PHP 5.3 and 5.4 still accounting for over 40% of all production environments, it's time to paint a clear picture on not just why everyone should move to 7.0 (or preferably 7.1), but on how to get code ready for the latest version of PHP.
Using the version compatibility checker for PHP_CodeSniffer and a few simple step-by-step instructions, upgrading old code to make it compatible with the latest PHP versions becomes actually really easy. In this talk, we'll migrate an old piece of code and get rid of the demons of the past and ready for the present and future.
With PHP 7.2 recently released and PHP 5.3 and 5.4 still accounting for over 40% of all production environments, it's time to paint a clear picture on not just why everyone should move to 7.0 (or preferably 7.1), but on how to get code ready for the latest version of PHP.
Using the version compatibility checker for PHP_CodeSniffer and a few simple step-by-step instructions, upgrading old code to make it compatible with the latest PHP versions becomes actually really easy. In this talk, we'll migrate an old piece of code and get rid of the demons of the past and ready for the present and future.
With more and more sites falling victim to data theft, you've probably read the list of things (not) to do to write secure code. But what else should you do to make sure your code and the rest of your web stack is secure ? In this tutorial we'll go through the basic and more advanced techniques of securing your web and database servers, securing your backend PHP code and your frontend javascript code. We'll also look at how you can build code that detects and blocks intrusion attempts and a bunch of other tips and tricks to make sure your customer data stays secure.
The time of static or dynamically generated sites is long gone. Non-stop interaction with users is the new normal. However, polling with Ajax requests is processor intensive and cumbersome. Websockets allow you to interact with users in real-time without increasing system load. We'll go through the basics and see all the different options, illustrated with live examples of how and when to use it.
Your app lives on the network - networking for web developersWim Godden
Our job might be to build web applications, but we can't build apps that rely on networking if we don't know how these networks and the big network that connects them all (this thing called the Internet) actually work.
I'll walk through the basics of networking, then dive a lot deeper (from TCP/UDP to IPv4/6, source/destination ports, sockets, DNS and even BGP).
Prepare for an eye-opener when you realize how much a typical app relies on all of these (and many more) working flawlessly... and how you can prepare your app for failure in the chain.
With more and more sites falling victim to data theft, you've probably read the list of things (not) to do to write secure code. But what else should you do to make sure your code and the rest of your web stack is secure ? In this tutorial we'll go through the basic and more advanced techniques of securing your web and database servers, securing your backend PHP code and your frontend javascript code. We'll also look at how you can build code that detects and blocks intrusion attempts and a bunch of other tips and tricks to make sure your customer data stays secure.
With more and more sites falling victim to data theft, you've probably read the list of things (not) to do to write secure code. But what else should you do to make sure your code and the rest of your web stack is secure ? In this tutorial we'll go through the basic and more advanced techniques of securing your web and database servers, securing your backend PHP code and your frontend javascript code. We'll also look at how you can build code that detects and blocks intrusion attempts and a bunch of other tips and tricks to make sure your customer data stays secure.
From ReactPHP to Facebook Hack's Async implementation and many more, asynchronous programming has been a 'hot' topic lately. But how well does async programming support work in PHP and what can you actually use it for in your projects ? Let's look at some real-world use cases and how they leverage the power of async to do things you didn't know PHP could do.
With more and more sites falling victim to data theft, you've probably read the list of things (not) to do to write secure code. But what else should you do to make sure your code and the rest of your web stack is secure ? In this tutorial we'll go through the basic and more advanced techniques of securing your web and database servers, securing your backend PHP code and your frontend javascript code. We'll also look at how you can build code that detects and blocks intrusion attempts and a bunch of other tips and tricks to make sure your customer data stays secure.
With more and more sites falling victim to data theft, you've probably read the list of things (not) to do to write secure code. But what else should you do to make sure your code and the rest of your web stack is secure ? In this tutorial we'll go through the basic and more advanced techniques of securing your web and database servers, securing your backend PHP code and your frontend javascript code. We'll also look at how you can build code that detects and blocks intrusion attempts and a bunch of other tips and tricks to make sure your customer data stays secure.
A practical step-by-step guide to Git, taking you through each phase of a project and explaining the use of Git at each step of the development process. Expect lots of how-to, but also some how-not-to, to avoid going down the wrong path.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Beyond php - it's not (just) about the code
1. Beyond PHP :
It's not (just) about the code
Wim Godden
Cu.be Solutions
@wimgtr
2. Who am I ?
Wim Godden (@wimgtr)
Founder of Cu.be Solutions (http://cu.be)
Open Source developer since 1997
Developer of OpenX, PHPCompatibility, Nginx SCL, ...
Speaker at PHP and Open Source conferences
3. Cu.be Solutions ?
Open source consultancy
PHP-centered
High-speed redundant network (BGP, OSPF, VRRP)
High scalability development
Nginx + extensions
MySQL Cluster
Projects :
mostly IT & Telecom companies
lots of public-facing apps/sites
4. Who are you ?
Developers ?
Anyone setup a MySQL master-slave ?
Anyone setup a site/app on separate web and database server ?
→ How much traffic between them ?
5. The topic
Things we take for granted
Famous last words : "It should work just fine"
Works fine today
→ might fail tomorrow
Most common mistakes
PHP code ↔ PHP ecosystem
How-to & How-NOT-to
7. Database queries – complexity
SELECT DISTINCT n.nid, n.uid, n.title, n.type, e.event_start, e.event_start AS
event_start_orig, e.event_end, e.event_end AS event_end_orig, e.timezone,
e.has_time, e.has_end_date, tz.offset AS offset, tz.offset_dst AS offset_dst,
tz.dst_region, tz.is_dst, e.event_start - INTERVAL IF(tz.is_dst, tz.offset_dst,
tz.offset) HOUR_SECOND AS event_start_utc, e.event_end - INTERVAL
IF(tz.is_dst, tz.offset_dst, tz.offset) HOUR_SECOND AS event_end_utc,
e.event_start - INTERVAL IF(tz.is_dst, tz.offset_dst, tz.offset) HOUR_SECOND +
INTERVAL 0 SECOND AS event_start_user, e.event_end - INTERVAL IF(tz.is_dst,
tz.offset_dst, tz.offset) HOUR_SECOND + INTERVAL 0 SECOND AS
event_end_user, e.event_start - INTERVAL IF(tz.is_dst, tz.offset_dst, tz.offset)
HOUR_SECOND + INTERVAL 0 SECOND AS event_start_site, e.event_end INTERVAL IF(tz.is_dst, tz.offset_dst, tz.offset) HOUR_SECOND + INTERVAL 0
SECOND AS event_end_site, tz.name as timezone_name FROM node n INNER
JOIN event e ON n.nid = e.nid INNER JOIN event_timezones tz ON tz.timezone =
e.timezone INNER JOIN node_access na ON na.nid = n.nid LEFT JOIN
domain_access da ON n.nid = da.nid LEFT JOIN node i18n ON n.tnid > 0 AND
n.tnid = i18n.tnid AND i18n.language = 'en' WHERE (na.grant_view >= 1 AND
((na.gid = 0 AND na.realm = 'all'))) AND ((da.realm = "domain_id" AND da.gid = 4)
OR (da.realm = "domain_site" AND da.gid = 0)) AND (n.language ='en' OR
n.language ='' OR n.language IS NULL OR n.language = 'is' AND i18n.nid IS NULL)
AND ( n.status = 1 AND ((e.event_start >= '2010-01-31 00:00:00' AND
e.event_start <= '2010-03-01 23:59:59') OR (e.event_end >= '2010-01-31 00:00:00'
AND e.event_end <= '2010-03-01 23:59:59') OR (e.event_start <= '2010-01-31
00:00:00' AND e.event_end >= '2010-03-01 23:59:59')) ) GROUP BY n.nid HAVING
(event_start >= '2010-02-01 00:00:00' AND event_start <= '2010-02-28 23:59:59')
OR (event_end >= '2010-02-01 00:00:00' AND event_end <= '2010-02-28 23:59:59')
OR (event_start <= '2010-02-01 00:00:00' AND event_end >= '2010-02-28
23:59:59') ORDER BY event_start ASC;
8. Database - indexing
'select id from stock where status = 2 order by qty'
→ aggregate index on (status, qty)
But if we use memory table :
'select id from stock where status > 2 order by qty'
→ aggregate index on (status, qty) ?
→ No : range selection stops use of aggregate index
→ separate index on status and qty (since recent versions)
9. Database - indexing
Indexes make database faster
→ Let's index everything !
→ DON'T :
Insert/update/delete → Index modification
Each query → evaluation of all indexes
"Relational schema design is based on data
but index design is based on queries"
(Bill Karwin, Percona)
10. Databases – detecting problematic queries
Slow query log
→ SET GLOBAL slow_query_log = ON;
Queries not using indexes
→ In my.cnf/my.ini : 'log_queries_not_using_indexes'
General query log
→ SET GLOBAL general_log = ON;
→ Turn it off quickly !
Percona Toolkit (Maatkit)
pt-query-digest
15. Databases – next step : explain
Type of lookup
'system', 'const' and 'ref' = good
'ALL' = bad
Extra info
Using index = good
Using filesort = usually bad
16. For / foreach
$customers = CustomerQuery::create()
->filterByState('MN')
->find();
foreach ($customers as $customer) {
$contacts = ContactsQuery::create()
->filterByCustomerid($customer->getId())
->find();
foreach ($contacts as $contact) {
doSomestuffWith($contact);
}
}
18. Better...
10001 → 1 query
Sadly : people still produce code with query loops
Usually :
Growth not anticipated
Internal app → Public app
19. The origins of this talk
Customers :
Projects we built
Projects we didn't build, but got pulled into
Fixes
Changes
Infrastructure migration
15 years of 'how to cause mayhem with a few lines of code'
20. Client X
Jobs search site
Monitor job views :
Daily hits
Weekly hits
Monthly hits
Which user saw which job
21. Client X
Originally : when user viewed job details
Now : when job is in search result
Search for 'php' → 50 jobs = 50 jobs to be updated
→ 50 updates for shown_today
→ 50 updates for shown_week
→ 50 updates for shown_month
→ 50 inserts for shown_user
= 200 queries for 1 search !
22. Client X : the code
foreach ($jobs as $job) {
$db->query("
insert into shown_today(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_week(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_month(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_user(
jobId,
userId,
when
) values (
" . $job['id'] . ",
" . $user['id'] . ",
now()
)
");
}
27. Client X : possible cause ?
Code changes ?
→ According to developers : none
Action : turn on general log, analyze with pt-query-digest
→ 50+-fold increase in queries
→ Developers : 'Oops we did make a change'
After 3 days : 2,5 days behind
Every hour : 50 min extra lag
28. Client X : But why is the slave lagging ?
File :
master-bin-xxxx.log
um
g d ad
n lo e
Bi thr
Master
p
Slave I/O thread
File :
master-bin-xxxx.log
Sl
av
th e S
re Q
ad L
Slave
31. Client X : fix ?
foreach ($jobs as $job) {
$db->query("
insert into shown_today(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_week(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_month(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_user(
jobId,
userId,
when
) values (
" . $job['id'] . ",
" . $user['id'] . ",
now()
)
");
}
32. Client X : the code change
insert into shown_today values (5, 1), (8, 1), (12, 1), (18, 1), … on duplicate key … ;
insert into shown_week values (5, 1), (8, 1), (12, 1), (18, 1), … on duplicate key … ;
insert into shown_month values (5, 1), (8, 1), (12, 1), (18, 1), … on duplicate key … ;
insert into shown_user values (5, 23, "2013-11-12 12:01:00"), (8, 23, "2013-11-12
12:01:00"), … ;
33. Client X : the code change
$todayQuery = "
insert into shown_today(
jobId,
number
) values ";
foreach ($jobs as $job) {
$todayQuery .= "(" . $job['id'] . ", 1),";
}
$todayQuery = substr($todayQuery, 0, strlen($todayQuery) - 1);
$todayQuery .= "
)
on duplicate key
update
number = number + 1
";
$db->query($todayQuery);
Careful : max_allowed_packet !
34. Client X : the chosen solution
$db->autocommit(false);
foreach ($jobs as $job) {
$db->query("
insert into shown_today(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_week(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_month(
jobId,
number
) values(
" . $job['id'] . ",
1
)
on duplicate key
update
number = number + 1
");
$db->query("
insert into shown_user(
jobId,
userId,
when
) values (
" . $job['id'] . ",
" . $user['id'] . ",
now()
)
");
}
$db->commit();
35. Client X : conclusion
For loops are bad (we already knew that)
Add master/slave and it gets much worse
Use transactions : it will provide huge performance increase
Result : slave caught up 5 days later
36. Database → Network
Customer Y
Top 10 site in Belgium
Growing rapidly
At peak traffic :
Unexplicable latency on database
Load on webservers : minimal
Load on database servers : acceptable
39. Client Y : network overload
Cause : Drupal hooks → retrieving data that was not needed
Only load data you actually need
Don't know at the start ? → Use lazy loading
Caching :
Same story
Memcached/Redis are fast
But : data still needs to cross the network
40. Network trouble : more than just traffic
Customer Z
150.000 visits/day
News ticker :
XML feed from other site (owned by same customer)
Cached for 15 min
41. Customer Z – fetching the feed
if (filectime(APP_DIR . '/tmp/cacheFile.xml') < time() - 900) {
unlink(APP_DIR . '/tmp/cacheFile.xml');
file_put_contents(
APP_DIR . '/tmp/cacheFile.xml',
file_get_contents('http://www.scrambledsitename.be/xml/feed.xml')
);
}
$xmlfeed = ParseXmlFeed(APP_DIR . '/tmp/cacheFile.xml');
What's wrong with this code ?
42. Customer Z – no feed without the source
Feed source
43. Customer Z – no feed without the source
Feed source
44. Customer Z : timeout
default_socket_timeout : 60 sec by default
Each visitor : 60 sec wait time
People keep hitting refresh → more load
More active connections → more load
Apache hits maximum connections → entire site down
50. Network resources
Use timeouts for all :
fopen
curl
SOAP
…
Data source trusted ?
→ setup a webservice
→ let them push updates when their feed changes
→ less load on data source
→ no timeout issues
Add logging → early detection
51. Logging
Logging = good
Logging in PHP using fopen
→ bad idea : locking issues
→ Use file_put_contents($filename, $data, FILE_APPEND)
For Firefox : FirePHP (add-on for Firebug)
Debug logging = bad on production
Watch your logs !
Don't log on slow disks → I/O bottlenecks
52. File system : I/O bottlenecks
Causes :
Excessive writes (database updates, logfiles, swapping, …)
Excessive reads (non-indexed database queries, swapping, small file
system cache, …)
How to detect ?
top
Cpu(s):
0.2%us,
iostat
avg-cpu:
%user
0.10
Device:
sda
sdb
dm-0
dm-1
3.0%sy,
0.0%ni, 61.4%id, 35.5%wa,
%nice %system %iowait
0.00
0.96
53.70
tps
120.40
2.10
4.20
0.00
Blk_read/s
0.00
0.00
0.00
0.00
%steal
0.00
Blk_wrtn/s
123289.60
4378.10
36.80
0.00
0.0%hi,
0.0%si,
0.0%st
%idle
45.24
Blk_read
0
0
0
0
Blk_wrtn
616448
18215
184
0
See iowait ? Stop worrying about php, fix the I/O problem !
53. File system
Worst of all : NFS
PHP files → lstat calls
Templates → same
Sessions
→ locking issues
→ corrupt data
→ store sessions in database, Memcached, Redis, ...
54. Much more than code
XML feed
User
Network
Webserver
DB
server
59. Step-by-step : most common issues
iowait on NFS server (lstat calls)
iowait on database server
I/O reads (use iostat) ? → missing/wrong indexes
I/O writes ?
→ no transactions ?
→ too many queries ?
→ too many indexes ?
→ bad DB engine settings
iowait on webserver (logs ? static files ?)
CPU on database server (missing/wrong indexes)
CPU on webserver (PHP)
Editor's Notes
5kbit/sec or 100Mbit/sec ?
Let's talk about code
Without : we don't exist
What are most common mistakes in ecosystem
Let's start with the database
time spent per query pattern
how many queries of that query pattern
Get back to what I said
Lots of people use ORM
- easier
- don't need to write queries
- object-oriented
but people start doing this
Imagine 10000 customers → 10001 queries
Not best code
Uses deprecated mysql extension
no error handling
Master : 16 CPU cores
12 cores for SQL
1 core for binlog dump
rest for system
Slave : 16 CPU cores
1 core for slave I/O
1 core for slave SQL
Grouping
Works fine, but :
maximum size of string ?
PHP = no limit
MySQL = max_allowed_packet
Grouping
Works fine, but :
maximum size of string ?
PHP = no limit
MySQL = max_allowed_packet
All in a single commit
Note : transaction has max. size
Possible : combination with previous solution
took few moments to figure out
No network monitoring
→ iptraf
→ 100Mbit/sec limit
→ packets dropped
→ connections dropped
Customer : upgrade switch
Us : why 100Mbit/sec ?
Databases → network
What other network related issues ?
Server on which feed located : crashed
Fine for few minutes (cache)
15 minutes : file_get_contents uses default_socket_timeout
Better, not perfect.
What else is wrong ?
Multiple visitors hit expiring cache
→ file delete
→ xml feed hit a lot
Better, not perfect.
What else is wrong ?
Multiple visitors hit expiring cache
→ file delete
→ xml feed hit a lot
Better, not perfect.
What else is wrong ?
Multiple visitors hit expiring cache
→ file delete
→ xml feed hit a lot
Better, not perfect.
What else is wrong ?
Multiple visitors hit expiring cache
→ file delete
→ xml feed hit a lot
Better, not perfect.
What else is wrong ?
Multiple visitors hit expiring cache
→ file delete
→ xml feed hit a lot
How do you treat your data :
- where do you get it
- how long did you have to wait to get it
- how is it transported
- how is it processed
minimize the amount of data :
retrieved
transported
processed,
sent to db and users