How we used ruby to build locaweb's cloud (http://presentations.pothix.com/ru...Willian Molinari
**The slides are not correctly rendered. The HTML/Javascript version is here: http://presentations.pothix.com/rubyconf2013/**
This presentation shows what we have done with Ruby to create Locaweb's cloud computing product.
Based on original presentation by Dan Webb http://slidesha.re/dan_on_rack
Added some bits about Rails at the end, instead of other examples he had originally.
Horizontal scaling in the Cloud is the way to adapt resources to load of systems. The Cloud allows users to scale virtually indefinitely, or enough for their needs.
This way the number of servers follows trend of requests, and TCO (Total Cost of Owneship) of IT infrastructure can be reduced. Nonetheless companies can avoid dealing with capacity planning and pre-provisioning issues.
This talk will show how to use Python and Rackspace/OpenStack API and SDK to implement an event-based scaling solution (software released under the open-source Apache License: stay tuned).
How we used ruby to build locaweb's cloud (http://presentations.pothix.com/ru...Willian Molinari
**The slides are not correctly rendered. The HTML/Javascript version is here: http://presentations.pothix.com/rubyconf2013/**
This presentation shows what we have done with Ruby to create Locaweb's cloud computing product.
Based on original presentation by Dan Webb http://slidesha.re/dan_on_rack
Added some bits about Rails at the end, instead of other examples he had originally.
Horizontal scaling in the Cloud is the way to adapt resources to load of systems. The Cloud allows users to scale virtually indefinitely, or enough for their needs.
This way the number of servers follows trend of requests, and TCO (Total Cost of Owneship) of IT infrastructure can be reduced. Nonetheless companies can avoid dealing with capacity planning and pre-provisioning issues.
This talk will show how to use Python and Rackspace/OpenStack API and SDK to implement an event-based scaling solution (software released under the open-source Apache License: stay tuned).
Securing Prometheus exporters using HashiCorp VaultBram Vogelaar
Things like Infrastructure as Code, Service Discovery and Config Management can and have helped us to quickly build and rebuild infrastructure but we haven't nearly spend enough time to train our self to review, monitor and respond to outages. Does our platform degrade in a graceful way or what does a high cpu load really mean? What can we learn from level 1 outages to be able to run our platforms more reliably.
This talk will focus on on creating a secure prometheus exporter ecosystem using HashiCorp Vault where we can we be sure that we are not leaking any business metrics from our observability stack. After which we ll investigate how to automatically rotate the certificates we created to do so.
An updated talk about how to use Solr for logs and other time-series data, like metrics and social media. In 2016, Solr, its ecosystem, and the operating systems it runs on have evolved quite a lot, so we can now show new techniques to scale and new knobs to tune.
We'll start by looking at how to scale SolrCloud through a hybrid approach using a combination of time- and size-based indices, and also how to divide the cluster in tiers in order to handle the potentially spiky load in real-time. Then, we'll look at tuning individual nodes. We'll cover everything from commits, buffers, merge policies and doc values to OS settings like disk scheduler, SSD caching, and huge pages.
Finally, we'll take a look at the pipeline of getting the logs to Solr and how to make it fast and reliable: where should buffers live, which protocols to use, where should the heavy processing be done (like parsing unstructured data), and which tools from the ecosystem can help.
"Roles and Profiles" is now the ubiquitous design pattern to create your puppet code tree. In this talk we will discuss writing reusable and maintainable profiles. We ll start by introducing creating module structures and will move on to type hinting and setting appropriate defaults. Finally we ll discuss the importance and the enforcing of code style conventions that allows multiple teams or projects to inner-source profiless
'Ansible Roles done right' is a talk about "Applying TDD while writing roles. Automatic tests powered by Continuous Integration + containers. Quick demo of the new ansible-container." Funny title: "When your applications don't have tests, at least your infrastructure does..."
Ansible is the simplest way to automate. MoldCamp, 2015Alex S
Ansible is a radically simple IT automation engine. This is new and great configuration management system (like Chef, Puppet) that has been created in 2012 year. Also Ansible is pretty simple and flexible system, that helps you in managing your servers and execute Ad-hoc commands.
During this session I will explain how to start using Ansible in infrastructure orchestration and what are pros and cons of this system. Also I will explain you our experience in deployments, provisioning and other aspects.
Yesterday I gave presentation on Ansible and it was successful . It give basic understanding of playbook and an example implementation of jboss application from scratch
Securing Prometheus exporters using HashiCorp VaultBram Vogelaar
Things like Infrastructure as Code, Service Discovery and Config Management can and have helped us to quickly build and rebuild infrastructure but we haven't nearly spend enough time to train our self to review, monitor and respond to outages. Does our platform degrade in a graceful way or what does a high cpu load really mean? What can we learn from level 1 outages to be able to run our platforms more reliably.
This talk will focus on on creating a secure prometheus exporter ecosystem using HashiCorp Vault where we can we be sure that we are not leaking any business metrics from our observability stack. After which we ll investigate how to automatically rotate the certificates we created to do so.
An updated talk about how to use Solr for logs and other time-series data, like metrics and social media. In 2016, Solr, its ecosystem, and the operating systems it runs on have evolved quite a lot, so we can now show new techniques to scale and new knobs to tune.
We'll start by looking at how to scale SolrCloud through a hybrid approach using a combination of time- and size-based indices, and also how to divide the cluster in tiers in order to handle the potentially spiky load in real-time. Then, we'll look at tuning individual nodes. We'll cover everything from commits, buffers, merge policies and doc values to OS settings like disk scheduler, SSD caching, and huge pages.
Finally, we'll take a look at the pipeline of getting the logs to Solr and how to make it fast and reliable: where should buffers live, which protocols to use, where should the heavy processing be done (like parsing unstructured data), and which tools from the ecosystem can help.
"Roles and Profiles" is now the ubiquitous design pattern to create your puppet code tree. In this talk we will discuss writing reusable and maintainable profiles. We ll start by introducing creating module structures and will move on to type hinting and setting appropriate defaults. Finally we ll discuss the importance and the enforcing of code style conventions that allows multiple teams or projects to inner-source profiless
'Ansible Roles done right' is a talk about "Applying TDD while writing roles. Automatic tests powered by Continuous Integration + containers. Quick demo of the new ansible-container." Funny title: "When your applications don't have tests, at least your infrastructure does..."
Ansible is the simplest way to automate. MoldCamp, 2015Alex S
Ansible is a radically simple IT automation engine. This is new and great configuration management system (like Chef, Puppet) that has been created in 2012 year. Also Ansible is pretty simple and flexible system, that helps you in managing your servers and execute Ad-hoc commands.
During this session I will explain how to start using Ansible in infrastructure orchestration and what are pros and cons of this system. Also I will explain you our experience in deployments, provisioning and other aspects.
Yesterday I gave presentation on Ansible and it was successful . It give basic understanding of playbook and an example implementation of jboss application from scratch
How I Learned to Stop Worrying and Love the Cloud - Wesley Beary, Engine YardSV Ruby on Rails Meetup
Wesley Beary: Cloud computing scared the crap out of me - the quirks and nightmares
of provisioning computing and storage on AWS, Terremark, Rackspace,
etc - until I took the bull by the horns. Let me now show you how I
tamed that bull.
Learn how to easily get started cloud computing with fog. It gives you
the reins within any Ruby application or script. If you can control
your infrastructure choices, you can make better choices in
development and get what you need in production.
You'll get an overview of fog and concrete examples to give you a head
start on your own provisioning workflow.
fog or: How I Learned to Stop Worrying and Love the Cloud (OpenStack Edition)Wesley Beary
Cloud computing scared the crap out of me - the quirks and nightmares of provisioning cloud computing, dns, storage, ... on AWS, Terremark, Rackspace, ... - I mean, where do you even start?
Since I couldn't find a good answer, I undertook the (probably insane) task of creating one. fog gives you a place to start by creating abstractions that work across many different providers, greatly reducing the barrier to entry (and the cost of switching later). The abstractions are built on top of solid wrappers for each api. So if the high level stuff doesn't cut it you can dig in and get the job done. On top of that, mocks are available to simulate what clouds will do for development and testing (saving you time and money).
You'll get a whirlwind tour of basic through advanced as we create the building blocks of a highly distributed (multi-cloud) system with some simple Ruby scripts that work nearly verbatim from provider to provider. Get your feet wet working with cloud resources or just make it easier on yourself as your usage gets more complex, either way fog makes it easy to get what you need from the cloud.
The OpenStack Edition adds my concerns about OpenStack API development, including things that have already been fixed and things that we haven't yet encountered. Hopefully this consumer perspective can help shed light on some rough spots.
ps: unfortunately SlideShare significantly decreases the quality of the presentation.
Modern science often requires the processing and analysis of vast amounts of data in search of postulated phenomena, and the validation of core principles through the simulation of complex system behaviors and interactions. This is the case in fields such as astronomy, bioinformatics, physics, and climate and ocean modeling, and others. In order to support the computational and data needs of today’s science, new knowledge must be gained on how to deliver the growing high-performance and distributed computing resources to the scientist’s desktop in an accessible, reliable and scalable way. In over a decade of working with domain scientists, the Pegasus project has developed tools and techniques that automate the computational processes used in data- and compute-intensive research. Among them is the scientific workflow management system, Pegasus, which is being used by researchers to model seismic wave propagation, to discover new celestial objects, to study RNA critical to human brain development, and to investigate other important research questions. This talk will provide an overview of scientific workflows, and discuss the main Pegasus’ features, including scheduling, fault-tolerance, data management, and monitoring.
Getting Started With Spark Structured Streaming With Dustin Vannoy | Current ...HostedbyConfluent
Getting Started With Spark Structured Streaming With Dustin Vannoy | Current 2022
Many data pipelines still default to processing data nightly or hourly, but information is created all the time and should be available much sooner. While the move to stream processing adds complexity, Spark Structured Streaming makes it achievable for teams of any size to switch to streaming.
This session shares techniques for data engineers who are new to building streaming pipelines with Spark Structured Streaming. It covers how to implement real-time stream processes with Apache Spark and Apache Kafka. We will discuss general concepts for Spark Structured Streaming along with introductory code examples. We will also look at important streaming concepts like triggers, windows, and state. To connect it all we will walk through a complete pipeline, including a demo using PySpark, Apache Kafka, and Delta Lake tables
fog or: How I Learned to Stop Worrying and Love the CloudWesley Beary
Learn how to easily get started on cloud computing with fog. If you can control your infrastructure choices, you’ll make better choices in development and get what you need in production. You'll get an overview of fog and concrete examples to give you a head start on your provisioning workflow.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
16. It will also not have access to the variables of the page that created it.
17. All file paths must be absolute from the hard disk root – [include] does not work
18.
19. An async thread will execute according to its priority and the load on Lasso
20. Basic Example Async threads are surrounded by curly-braces and initiated using ->asasyn c: [{'Hello World'}->asasync] The above code doesn't show anything in the browser, but it does run asynchronously.
21. Sending Data to Async Thread ->asasync allows you to add params to pass into the async thread [{ var('p1') = params->find('p1')->get(1)->value } ->asasync(-params=array('p1'=1, 'p2'=2))] Can also use globals (or other shared resources), but take care to correctly send the right message to the right thread and use [thread_atomic] to keep access to the global thread-safe
22. Template For Copying Vars To Async Thread <?LassoScript // from: http://www.lassotech.com/async // stuff on your page not async here... // -------------- BEGIN OF ASYNC CODE -------------- { // -------------- BEGIN OF ASYNC CODE -------------- // -------------------- // process async! // -------------------- // // NOTE: Code below this point is processed in it's own thread! // Lasso_ExecutionTimeLimit(60 * 60); // in seconds, set this to something reasonable Thread_SetPriority(Thread_GetCurrentID, Thread_Priority_Low); // set priority // recreate vars - this makes all the vars on the page available to the async thread iterate: params->(find:'-vars')->(get:1)->value, local:'var'; (var: #var->name) = #var->value; /iterate; // do stuff here that you want in it's own thread... // -------------- END OF ASYNC CODE -------------- }->(asasync: -params=(array: '-vars' = vars)); // -------------- END OF ASYNC CODE -------------- // do more stuff on the current page... ?>
23. Creating Context To create context within async threads, use variables to store the context of the current page (such as client_headers, file path, action_params) and use the template to pass the context into the async thread.
24.
25.
26. Task does not need to return data for the enduser to view
27. Task does not have to be finished by the time the concurrent page is served
28. Example of Off-Loading a Task Page originally created related records and made the enduser wait: [ loop(500); // create related record inline(-sql=”INSERT INTO mytable (id,this,that) VALUES (”+integer($userid)+”,”+integer($this)+”,”+integer($that)+”)”); /loop; 'Thanks for waiting!'; ]
29. Example of Off-Loading a Task (cont) Page now off-loads the related records creation and response much quicker to the enduser: <?LassoScript // from: http://www.lassotech.com/async // -------------- BEGIN OF ASYNC CODE -------------- { // -------------- BEGIN OF ASYNC CODE -------------- Lasso_ExecutionTimeLimit(60 * 60); // in seconds, set this to something reasonable Thread_SetPriority(Thread_GetCurrentID, Thread_Priority_Low); // set priority // recreate vars - this makes all the vars on the page available to the async thread iterate: params->(find:'-vars')->(get:1)->value, local:'var'; (var: #var->name) = #var->value; /iterate; loop(500); // create related record inline(-sql=”INSERT INTO mytable (id,this,that) VALUES (”+integer($userid)+”,”+integer($this)+”,”+integer($that)+”)”); /loop; // -------------- END OF ASYNC CODE -------------- }->(asasync: -params=(array: '-vars' = vars)); // -------------- END OF ASYNC CODE -------------- 'Thanks for waiting!'; ?>
32. Can use a global to use as a switch to shut down the thread – [while(global('on'))] – just be sure to use [thread_atomic] to make it thread-safe
33.
34. Example of a Recurring Event Creating a thread to watch a folder for new files to process: [{ Lasso_ExecutionTimeLimit(0); // run forever Thread_SetPriority(Thread_GetCurrentID, Thread_Priority_Low); // low priority while(true); iterate(file_listdirectory('///path/from/disk/root/'),local('file')); // do something with #file /iterate; Sleep(1000*60*10); // sleep 10 minutes (in milliseconds) /while; }->asasync] This would be placed in your LassoStartup folder.
38. Test Outside of Async Easiest testing method, comment out the ->asasync container and try running your code, does it work? For recurring event threads, you'll have to limit the number of loop_counts and shorten the sleep: [//{ Lasso_ExecutionTimeLimit(0); // run forever Thread_SetPriority(Thread_GetCurrentID, Thread_Priority_Low); // low priority while(true); loop_count > 10 ? loop_abort; iterate(file_listdirectory('///path/from/disk/root/'),local('file')); // do something with #file /iterate; Sleep(1000*60*2); // sleep 2 minutes (in milliseconds) /while; //}->asasync ]
39. Trapping for Errors Place async code within [protect] to log errors: [{ Lasso_ExecutionTimeLimit(0); // run forever Thread_SetPriority(Thread_GetCurrentID, Thread_Priority_Low); // low priority protect; handle_error; log_critical('Folder watcher encounter an error: '+error_msg); /handle_error; while(true); iterate(file_listdirectory('///path/from/disk/root/'),local('file')); // do something with #file /iterate; Sleep(1000*60*10); // sleep 10 minutes (in milliseconds) /while; /protect; }->asasync]