The document provides instructions for installing and configuring PyWPS, an open source Python library for implementing Web Processing Services (WPS). It discusses setting up a PyWPS instance, creating a sample process, and invoking it through a wrapper script. Key steps include installing PyWPS files, editing the configuration file to specify process and output paths, registering a sample process, and testing the instance by invoking WPS operations in a web browser.
Mike Guthrie - Revamping Your 10 Year Old Nagios InstallationNagios
Mike Guthrie - Revamping Your 10 Year Old Nagios Installation - Mike Merideth from VictorOps talks about the challenges of
sharing responsibility for monitoring in the DevOps world. Learn several strategies for keeping your configuration correct,
consistent, and up-to-date when several people are working on it.
Janice Singh - Writing Custom Nagios Plugins - New to Nagios and wanting to expand its use with your own
custom plugins? This presentation will show you how to write your own plugins and integrate it into Nagios.
Developed for the Denver Art Museum by Ashley Blewer, this slide-deck covers some of the basics of diagnosing issues with Archivematica. Ashley covers everything from the software components involved with Archivematica, to monitoring logs, system monitoring, and upgrading your system. The presentation concludes with some useful links for tech-savvy preservationists, and Archivematica-unfamiliar system's administrators!
Mike Guthrie - Revamping Your 10 Year Old Nagios InstallationNagios
Mike Guthrie - Revamping Your 10 Year Old Nagios Installation - Mike Merideth from VictorOps talks about the challenges of
sharing responsibility for monitoring in the DevOps world. Learn several strategies for keeping your configuration correct,
consistent, and up-to-date when several people are working on it.
Janice Singh - Writing Custom Nagios Plugins - New to Nagios and wanting to expand its use with your own
custom plugins? This presentation will show you how to write your own plugins and integrate it into Nagios.
Developed for the Denver Art Museum by Ashley Blewer, this slide-deck covers some of the basics of diagnosing issues with Archivematica. Ashley covers everything from the software components involved with Archivematica, to monitoring logs, system monitoring, and upgrading your system. The presentation concludes with some useful links for tech-savvy preservationists, and Archivematica-unfamiliar system's administrators!
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...Chris Fregly
Empowering the Data Scientist with "1-Click" Production Deployment and Canary Testing of High-Performance and Highly-Scalable Spark ML and TensorFlow Models directly from Jupyter/iPython Notebooks using Docker, Kubernetes, Netflix OSS, Microservices, and Spinnaker.
With proper tooling and metrics, Data Scientists can directly deploy, analyze, A/B test, rollback, and scale out their Spark ML and TensorFlow model into live production serving with zero friction.
We will show you the open source tools that we've built based on Docker, Kubernetes, Netflix Open Source, Microservices, Spinnaker - and even Chaos Monkey!
Speaker: Chris Fregly @ PipelineIO, formerly Databricks and Netflix
Overview of the Ratpack web framework.
Source for the talk (including demo apps) here:
https://github.com/ratpack/ratpack-talks/tree/master/talks/cdjdn
OPNFV Update: The Danube Release and What Lies Around the BendJill Jensen Lovato
OPNFV facilitates the development and evolution of NFV components across various open source ecosystems. Through system level integration, deployment and testing, OPNFV creates a reference NFV platform to accelerate the transformation of enterprise and service provider networks. The recently announced fourth release, Danube, represents a growing maturity for both the project and upstream partners and brings together elements across the stack to more quickly introduce technologies that meet the needs of operators. Tapio Tallgren, OPNFV TSC Chair, and Heather Kirksey, OPNFV Director, presented a session during ONS 2017 on how OPNFV Danube, Euphrates, and the many rivers to come are helping to build the next-generation network for NFV.
Watch the webinar on demand: https://www.nginx.com/resources/webinars/maximize-php-performance-with-nginx
Is your PHP app slowing to a crawl? PHP is a powerful programming language that powers roughly 80% of the internet, but it’s unfortunately subject to performance problems – as we all know. Luckily, for thousands of PHP-based applications, some relatively simple changes can lead to noticeable improvements in performance.
NGINX has greatly improved application performance for more than 150 million sites in production today. Using NGINX greatly improves the performance of PHP apps with features such as caching, load balancing, HTTP/2 support, and more, included in open source NGINX software and in our commercial-grade application delivery platform NGINX Plus.
Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013Puppet
"Running at Scale: Practical Performance Tuning with Puppet" by Sam Kottler Engineer, Red Hat.
Presentation Overview: This session will talk about some production issues I've seen running Puppet in large environments. From how to manage a single master with hundreds of hosts to real-life patterns for building high availability clusters that scale to 10's of thousands of agents. Another important topic that will be covered is how to deploy networked filesystems that perform well under high load and streaming files to many hosts simultaneously.
Speaker Bio: Sam Kottler is a software engineer in the Virtualization R&D group at Red Hat. He's helped build infrastructure for leading startups, including Digg.com, Acquia, and Venmo and is a contributor to Puppet, the Fedora Project, Drupal, and the Rubygems.org. Sam speaks around the world on the topics of internet security, systems automation, and software architecture.
HTTP2 in action - Piet Van Dongen - Codemotion Amsterdam 2017Codemotion
We've all heard about HTTP/2, but what's in it for us? Is it really that much better? How can we start using it? During this talk, we will explore HTTP/2's new features while creating our own web server, demonstrating new features like server push, multiplexing and header compression. At then end, we can proof how HTTP/2 benefits not only the end user, but developers and operations as well!.
Leonid Vasilyev "Building, deploying and running production code at Dropbox"IT Event
Reproducible builds, fast and safe deployment process together with self-healing services form the basis of stable and maintainable infrastructure. In this talk I’d like to cover, from the Site Reliability Engineering (SRE) perspective, how Dropbox addresses above challenges, what technologies are used and what lessons were learnt during implementation process.
A talk I gave at the Python Ireland meetup in June 2015 about rq (python-rq.org), a queuing library for the Python programming language that is backed by Redis.
This is a quick guide on using WeldPulse Software, Spark Edition. WeldPulse is a welding software which provides help to welding professionals in making welding decisions.
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...Chris Fregly
Empowering the Data Scientist with "1-Click" Production Deployment and Canary Testing of High-Performance and Highly-Scalable Spark ML and TensorFlow Models directly from Jupyter/iPython Notebooks using Docker, Kubernetes, Netflix OSS, Microservices, and Spinnaker.
With proper tooling and metrics, Data Scientists can directly deploy, analyze, A/B test, rollback, and scale out their Spark ML and TensorFlow model into live production serving with zero friction.
We will show you the open source tools that we've built based on Docker, Kubernetes, Netflix Open Source, Microservices, Spinnaker - and even Chaos Monkey!
Speaker: Chris Fregly @ PipelineIO, formerly Databricks and Netflix
Overview of the Ratpack web framework.
Source for the talk (including demo apps) here:
https://github.com/ratpack/ratpack-talks/tree/master/talks/cdjdn
OPNFV Update: The Danube Release and What Lies Around the BendJill Jensen Lovato
OPNFV facilitates the development and evolution of NFV components across various open source ecosystems. Through system level integration, deployment and testing, OPNFV creates a reference NFV platform to accelerate the transformation of enterprise and service provider networks. The recently announced fourth release, Danube, represents a growing maturity for both the project and upstream partners and brings together elements across the stack to more quickly introduce technologies that meet the needs of operators. Tapio Tallgren, OPNFV TSC Chair, and Heather Kirksey, OPNFV Director, presented a session during ONS 2017 on how OPNFV Danube, Euphrates, and the many rivers to come are helping to build the next-generation network for NFV.
Watch the webinar on demand: https://www.nginx.com/resources/webinars/maximize-php-performance-with-nginx
Is your PHP app slowing to a crawl? PHP is a powerful programming language that powers roughly 80% of the internet, but it’s unfortunately subject to performance problems – as we all know. Luckily, for thousands of PHP-based applications, some relatively simple changes can lead to noticeable improvements in performance.
NGINX has greatly improved application performance for more than 150 million sites in production today. Using NGINX greatly improves the performance of PHP apps with features such as caching, load balancing, HTTP/2 support, and more, included in open source NGINX software and in our commercial-grade application delivery platform NGINX Plus.
Running at Scale: Practical Performance Tuning with Puppet - PuppetConf 2013Puppet
"Running at Scale: Practical Performance Tuning with Puppet" by Sam Kottler Engineer, Red Hat.
Presentation Overview: This session will talk about some production issues I've seen running Puppet in large environments. From how to manage a single master with hundreds of hosts to real-life patterns for building high availability clusters that scale to 10's of thousands of agents. Another important topic that will be covered is how to deploy networked filesystems that perform well under high load and streaming files to many hosts simultaneously.
Speaker Bio: Sam Kottler is a software engineer in the Virtualization R&D group at Red Hat. He's helped build infrastructure for leading startups, including Digg.com, Acquia, and Venmo and is a contributor to Puppet, the Fedora Project, Drupal, and the Rubygems.org. Sam speaks around the world on the topics of internet security, systems automation, and software architecture.
HTTP2 in action - Piet Van Dongen - Codemotion Amsterdam 2017Codemotion
We've all heard about HTTP/2, but what's in it for us? Is it really that much better? How can we start using it? During this talk, we will explore HTTP/2's new features while creating our own web server, demonstrating new features like server push, multiplexing and header compression. At then end, we can proof how HTTP/2 benefits not only the end user, but developers and operations as well!.
Leonid Vasilyev "Building, deploying and running production code at Dropbox"IT Event
Reproducible builds, fast and safe deployment process together with self-healing services form the basis of stable and maintainable infrastructure. In this talk I’d like to cover, from the Site Reliability Engineering (SRE) perspective, how Dropbox addresses above challenges, what technologies are used and what lessons were learnt during implementation process.
A talk I gave at the Python Ireland meetup in June 2015 about rq (python-rq.org), a queuing library for the Python programming language that is backed by Redis.
This is a quick guide on using WeldPulse Software, Spark Edition. WeldPulse is a welding software which provides help to welding professionals in making welding decisions.
These slide gives overall welding process and gives the tips about the robotic welding process and then adaptability with the industrial welding robotic coordinates.
Robot Welding is a process of joining different materials.
The large bulk of materials that are welded are metals and their alloys although welding is also applied to the joining of other materials such as thermoplastics.
Guide to understanding welding procedure specification (wpsTonni Bruce
this is my private data, i just want to translate some of documents that i just find on the internet into my language to easy to learn, so if anyone have the same native language with me they can use my book.
thanks for all
The WPS 2.0 standard (preliminary information)Benjamin Proß
In this talk, we will present the Web Processing Service (WPS) 2.0 standard that is planned to be released by the Open Geospatial Consortium (OGC) shortly after FOSS4G-Europe 2014 (Due to possible changes during the release process, this information is preliminary.). We will describe the changes to the previous version of the standard (1.0.0) including:
• Improvements to the process description,
• Profiling mechanism for WPS processes,
• Changes to the WPS protocol, including synchronous and asynchronous invocation of processes and status monitoring,
• “Pluggable” process descriptions, and
• Improved conformance testing.
As a result of the modularization, the standard will contain several conformance classes which can be combined to tailor WPS implementations to various application scenarios. Request/response examples will give the audience an exact picture of what a service implementing the Basic WPS 2.0 interface will look like. We will also describe the mechanisms that will allow domain experts to specify further extensions of the WPS core, e.g. to create a RESTful WPS, a WPS based on SensorML or a CoverageWPS. We will talk about testing and a (possible) roadmap for creating an OGC Compliance Interoperability & Testing Evaluation (CITE) test suite that will eventually lead to certified compliant WPS 2.0 servers and also to reference implementations of the standard.
Using Apache Brooklyn to manage your application stack. Brooklyn is a cloud agnostic orchestrator that can deploy an application to any cloud (including the creation of infrastructure) without changing the blueprint.
InterConnect2016: WebApp Architectures with Java and Node.jsChris Bailey
Java has been the historical leader for enterprise web application development. However, Node.js is rapidly gaining in popularity for developing mobile apps, APIs and web applications. Java and Node.js are complimentary tools for enterprise web application development and this session will highlight the strengths and complimentary nature of each.
Presented at IBM InterConnect 2016
Talk given at JavaOne 2009 discussing how to build web applications using OSGi. The source for the demo found at http://github.com/mrdon/jforum-plugins/tree/master
Session Abstract: Enterprise Web applications tend to grow like weeds in monolithic complexity. OSGi, although more often associated with Java™ technology-based clients and application servers, can bring a new level of modularity, uptime, and stability that is needed with today's always-on hosted Web applications. OSGi gets really interesting when the pretty architecture diagrams meet the real world, because it consists of various deployment platforms, development environments, and application architectures. This presentation, for Java 2 Platform, Enterprise Edition (J2EE™ platform)-savvy architects and senior developers, provides a practical guide to the Web on OSGi, from integration approach to bundle development, to real-world code you can use today.
The session discusses
• What benefits OSGi brings to the J2EE platform
• Three integration strategies
• How to use Spring DM and Maven to ease development
• Lessons learned from Atlassian's recent OSGi deployment
• A production-ready example to use immediately
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Pywps a tutorial for beginners and developers
1. PyWPS a tutorial for beginners and developers Jorge de Jesus (Plymouth Marine Laboratory) Luca Casagrande (Università degli Studi di Perugia) Jachym Čepicky (Help Service – Remote Sensing Company)
2. Before starting Please put your OSGEO livecd or usb-stick inside your laptop and start your machine.
3. Program Introduction part WPS Standard and PyWPS Newbie part Installation, setup and first process Developers part PyWPS in detail, mod_python,jython, GRASS Gallery Examples of applications using PyWPS
5. Install the tutorial Open Firefox and download the script from the pyWPS main page: http://pywps.wald.intevation.org/ Make it executable (password is user) : sudo chmod +x install_pywps_svn.sh Start the script: sudo ./install_pywps_svn.sh You can edit files using nano, but remember to always use sudo (password is user) .
6. Definitions Web Processing Service (WPS) is an OGC standard protocol to make GIS calculation available to the internet Open Geospatial Consortium (OGC) is a non-profit, international, voluntary consensus standards organization that is leading the development of standard for geospatial and location based service.
7. WPS standard ... provides rules for standardizing how inputs and outputs (requests and responses) for geospatial processing services , such as polygon overlay. The standard also defines how a client can request the execution of a process, and how the output from the process is handled..
10. SOAP For this tutorial we will use just the HTTP GET method
11. Key Value Pairs request http://localhost/cgi-bin/wps.py?service=WPS&request=GetCapabilities http://localhost/cgi-bin/wps.py Is the server address The ? sign indicates, that the request parameters will start service=WPS&request=GetCapabilities The KVP-encoded request. We send two request parameters to the server: service - which we set to WPS request - which is set to GetCapabilities
12. XML Request In this case, the request is encoded in XML form and send to the server directly via HTTP POST (the WPS server will read the file from standard input directly). <?xml version="1.0" encoding="UTF-8"?> <ows:GetCapabilities xmlns:ows="http://www.opengis.net/ows/1.1" > <ows:AcceptVersions> <ows:Version>1.0.0</ows:Version> </ows:AcceptVersions> </ows:GetCapabilities>
16. Despite the specification that requests should be case insensitive, it is recommended to use the upper camel case standard in all sorts of WPS operation requests
26. An equal sign (=) shall be used to separate an input name from its value and attributes, and an attribute name from its value
27. An at symbol (@) shall be used to separate an input value from its attributes and one attribute from another.
28. All field values and attribute values shall be encoded using the standard Internet practice for encoding URLs
29. PyWPS also supports the use of [ ] to group the datainputs as follows: datainputs=[int=1;float=3.2]
30. Description of Data Inputs and Outputs Three types of inputs and outputs are defined in the OGC standard. LiteralData, ComplexData and BoundingBox data.
31. LiteralData LiteralData can be any character string, float,date, etc normally described as Primitive datatype in the W3C XML WPS standard also allows the use of UOM (Unit of Measures), default values and AllowedValues.
32.
33. Or you send or obtain just reference to the data – URL to the file or service, where the data can be downloaded.
34.
35. other CRS supported &bboxInput=71.63,41.75,-70.78,42.90,urn:ogc:def:crs:EPSG:6.6:4326,2
57. PyWPS history 2010 pyWPS is recomended as THE WPS tool in GIGAS project ( GEOSS, INSPIRE and GMES an Action in Support). As explained in the "GIGAS Technology Watch Report WPS" "PyWPS Web Processing Service: is a Python program which implements the OGC WPS 1.0.0 standard (with a few omissions). PyWPS was chosen as it is up to date with the WPS standard and has a low footprint, making it easy to install on most Linux systems.”
62. Install files can be found in: SVN access to the latest code: Latest package: http://pywps.wald.intevation.org/download/ svn checkout https://svn.wald.intevation.org/svn/pywps/trunk http://wald.intevation.org/frs/download.php/589/pywps-3.1.0.tar.gz
63. Clean install: There's DEB and RPM packages. Badly maintain :( > tar -xvzf /tmp/pywps-VERSION.tar.gz > cd pywsp-VERSION > python setup.py install
65. Testing the script by running the wps.py (/usr/bin) script If everything is ok.... > /usr/bin/wps.py PyWPS NoApplicableCode: Locator: None; Value: No query string found. Content-type: text/xml <?xml version="1.0" encoding="utf-8"?> <ExceptionReport version="1.0.0" xmlns="http://www.opengis.net/ows"... > <Exception exceptionCode="NoApplicableCode"> <ExceptionText> No query string found. </ExceptionText> </Exception> </ExceptionReport>
66. Configuration file for PyWPS can be located on several places. There are global and local PyWPS configuration files. Local files overwrite the global one Global : Local: /etc/pywps.cfg /usr/local/pywps-VERSION/etc/pywps.cfg Any path defined in the PYWPS_CFG environment variable
67. pywps.cfg is a Key = Value text file 4 sections are present in the file: Remember: The file is case-sensitive [wps] [provide] [server] [grass]
68. Baisic meta information necessary to populate the WPS doc. Other variables aren't shown in this example [wps] title version abstract fees keywords lang
69. Server configuration options, path locations and URL translation and service limits Other variables aren't shown in this example Most important: outputURL, outputPath, processPath [server] maxoperations maxinputparamlength maxfilesize outputUrl outputPath processesPath
70. outputURL: URL that will be used to point to the WPS outputs outputPath: Folder where PyWPS will drop the outputs (server accessible) http://localhost/wpsoutput http://rsg.pml.ac.uk/wps/wpsoutput /var/www/html/wpsoutput /usr/local/apache/htdocs/wps/wpsoutput /var/www/html/wpsoutput /var/www/html/wpsoutput
71. processPath: Folder path with stored processes It's important that these 3 parameters are properly configured /usr/local/pywps/processes /usr/local/pywps/processes /usr/local/pywps/processes /usr/local/pywps/processes /usr/local/pywps/processes /home/user/processes
72. PyWPS can be installed once in a server, but it may be configured to run several WPS services (instances). WPS instance Process folder pywps.cfg file
73. 1) Setup a process folder 2) copy configuration file-template and edit it to desired configuration 3) We need to populate the process directory > mkdir -p /usr/local/wps/processes > cp pywps-VERSION/pywps/default.cfg /usr/local/wps/pywps.cfg > nano /usr/local/wps/pywps.cfg > cp pywps-VERSION/examples/ultimatequestionprocess.py /usr/local/wps/processes/
74. 4) Every process in the process folder needs to be “registered in a file called __init__.py We've done 50% of an instance :) > cd /usr/local/wps/processes/ > echo "__all__=['ultimatequestionprocess']" > __init__.py __all__ it's a python array will the processe list
75. A WPS instance is just a script that alters some parameters before calling wps.py #!/bin/sh # Author: Jachym Cepicky # Purpose: CGI script for wrapping PyWPS script # Licence: GNU/GPL # Usage: Put this script to your web server cgi-bin directory, e.g. # /usr/lib/cgi-bin/ and make it executable (chmod 755 pywps.cgi) # NOTE: tested on linux/apache export PYWPS_CFG=/usr/local/wps/pywps.cfg export PYWPS_PROCESSES=/usr/local/wps/processes/ /usr/local/pywps-VERSION/wps.py $1 wps.cgi file
76. We need to configure PYWPS_CFG and PYWPS_CFG to specify the instance We can copy the wrapper script to Apache's cgi-bin folder Assuming that Apache is configure to support script execution... > cp wps.cgi /usr/lib/cgi-bin http://localhost/cgi-bin/pywps.cgi?request=DescribeProcess & service=WPS&version=1.0.0& process=ultimatequestionprocess
77. <?xml version="1.0" encoding="utf-8"?> <wps:ProcessDescriptions xmlns:wps=.... service="WPS" version="1.0.0" xml:lang="eng"> <ProcessDescription wps:processVersion="2.0" storeSupported="true" statusSupported="true"> <ows:Identifier>ultimatequestionprocess</ows:Identifier> <ows:Title>Answer to Life, the Universe and Everything</ows:Title> <ows:Abstract>....</ows:Abstract> <ProcessOutputs> <Output> <ows:Identifier>answer</ows:Identifier> <ows:Title>T he numerical answer to Life, Universe and Everything </ows:Title> <LiteralOutput> <ows:DataType >integer</ows:DataType> </LiteralOutput> </Output> </ProcessOutputs> </ProcessDescription> </wps:ProcessDescriptions>
78. PyWPS's assemble factory approach User's process POST GET SOAP GetCapabilities DescribeProcess Execute 1 2 1 Load Process Check Properties WPS output 2 Load Process getInput run setOuput WPS output
79. Process as an extended class of WPSProcess with method run() that will execute the code WPSProcess class Process1 Process2 ProcessN
80. All processes have the following skeleton: WPSProcess class provides extra functionalities like: - Command line util: self.cmd() - Status setting: self.status.set(message,percentage) from pywps.Process.Process import WPSProcess class Process(WPSProcess): def __init__(self): # init process WPSProcess.__init__(self, < Process's information like: identifier, title, status >) < Inclusion of inputs and outputs to process class > def execute(self): < code >
81. Process's attributes: The only mandatory attribute is: identifier class Process(WPSProcess): def __init__(self): # init process WPSProcess.__init__(self, identifier=”firstprocess”, #same file name title=”foo”, abstract=”bacon and eggs”, version = "0.1", storeSupported = "true", statusSupported = "true", <more WPS attributes if necessary> )
82. 3 types of Input/Output defined in WPS: Each Input/Output is a method of WPSProcess class Each Input/Output is created when class in initiated LiteralData ComplexData BBOX
84. Each self.add*() defines/creates an input. The class constructor accepts the WPS parameters: Only identifier and title are mandatory self.Input1 = self.addLiteralInput( identifier = "input1", title = "Input1 number", abstract=”foo”, minOccurs=1, type=types.IntType default="100")
85. A more “Complex” example Only identifier and title are mandatory self.dataIn = self.addComplexInput( identifier="data", title="Input vector data", abstract=”foo” formats = [{'mimeType':'text/xml'}])
86. What about outputs Identical syntax and procedure :) self.dataOut = self.addComplexOutput( identifier="output", title="Output vector data", formats = [{'mimeType':'text/xml'}]) self.Output1 = self.addLiteralOutput( identifier="output1", title="foo")
87. The add*Input and add*Output are set in the beginning of the class (__init__ method): from pywps.Process.Process import WPSProcess class Process(WPSProcess): def __init__(self): # init process WPSProcess.__init__(self, < Process's information like: identifier, title, status >) self.Input1=self.addLiteralInput(identifier=”input1”) self.dataOut =self.addComplexInput(identifier=”outputs”) <more inputs/outputs as needed> def execute(self): < code >
88. OK we have WPS inputs and output, how can I get them ?!?!?! Using the JAVA get and set “philosophy” :) Each Input has a getValue() method Each Output has a setValue() method This is done inside the execute method()
89. Please check wiki !!!! from pywps.Process.Process import WPSProcess class Process(WPSProcess): def __init__(self): # init process WPSProcess.__init__(self, < Process's information like: identifier, title, status >) self.Input1=self.addLiteralInput(identifier=”input1”) self.dataOut =self.addComplexInput(identifier=”outputs”) <more inputs/outputs as needed> def execute(self): input1=self.Input1.getValue() XMLdata=”<xml>foo</xml>” self.dataOut.setValue(XMLData) #input or file object
90. Fist Process, a returner process Process class initiation, identifier, WPS status and storeExecuteResponse definition from pywps.Process import WPSProcess class Process(WPSProcess): def __init__(self): ## # Process initialization WPSProcess.__init__(self, identifier = "returner", title="Return process", abstract="""This is demonstration process of PyWPS, returns the same file, it gets on input, as the output.""", version = "1.0", storeSupported = "true", statusSupported = "true")
94. Hacking ( wps.py code for dummies) Logging (and wood) GRASS (after the wood) Mod_Python (Pythons and Horses) Tomcat server (Pythons and Cats) Mapserver support (More OGC stuff) OpenLayers (Let there be layers.....) SOAP/WSDL (Beatiful soap... so they say...) PyCallGraph (The all enchilada!!!)
95. PyWPS's wps.py has the following pseudo-code structure: Start wps.py: 1. Determine request_method (GET or POST) 2. if no input: raise Exception and exit 3. try: initiate PyWPS class according to request_method parse Request do Request get Response and make proper reply 4. exception: reply Error response
97. In PyWPS you can use the logging module anywhere in the code. pywps.cfg file contains the path to the log file Then the file log will contain a line, like this:
98. Eclipse IDE is the default debugging platform using PyDEV tools Code to be debugged should contain a path to the PyDEV tools: After this path append, it is possible to import pydev module Now the code with pydevd is enabled for debbuging
99. Next step is to activate the debug server that will listen to the script. Now everytime that the python interperter finds: It will stop and send the variable to debug server From eclipse it will be possible to continue or stop the script
100.
101. PyWPS doesn't come with out-of-the-box tools PyPWS is Python, so connect, connect, connect !!!!! To GRASS GIS You may work with a predefined grassLocation or a temporary one WPSProcess.__init__(self, identifier = "foo", ... grassLocation = True ) Temporary grassLocation XY coordinate system
102. gisdbase base path specified in configuration file (pywps.cfg) Absolute path in grassLocation OR
105. " Mod_python is an Apache module that embeds the Python interpreter within the server ”. In: http://www.modpython.org/ version 3.2 provides a wps.py script designed to be integrated into mod_python So what is the advantage ?! SPEED !!!!! 50x faster on request processing Integration with Apache's API Ability to handle request phases, filters and connections
106. Default httpd.conf for PyWPS: Inform Apache that wps is the default handler of any request Pass env variables PYWPS_PROCESSES and PYWPS_CFG
108. mod_python can apply filters on HTTP request/response The filter needs to be register to Apache and mod_python The filter is applied to any WPS output, encrypting the response
109. WPS client/server 'secured' interaction The server provides “getCapabilities” and “describeProcess” to anyone. The “execute” is permitted only to authorized users Note: the base authentication credentials are used to allow the geo web service to receive delegation (downloading a proxy certificate) from another web service . The authentication/authorization of the “execute” is managed through X.509 certificates which are handled through the GridSite module for Apache ( http://www.gridsite.org/ ). Who is using mod_python and PyWPS ?! GENESI-DR, (Ground European Network for Earth Science Interoperations - Digital Repositories), www.genesi-dr.eu & www.genesi-dec.eu INFRA-2007-1.2.1 : Scientific Digital Repositories
110. No Voodoo , Just computer science !!!! Python Code Jython Compiler Java ByteCode TomCat Instance
112. - All PyWPS code needs to be copied to the Tomcat folder running the instance: https://svn.wald.intevation.org/svn/pywps/trunk/webservices/tomcat/ WEB-INF PywpsServlet.py Configuration file used by TomCat The twin brother or wps.py
113. Now we just need the Jython Library :) And we have just the last piece of the puzzle missing... What about PYWPS_PROCESSES ?!
116. Still in the SVN tree, highly experimental !!!!! Who's using it ?! ...or will be using..... Its intended to be the default WPS service for Conceptual Schema Transformer
117. Yes, PyWPS even supports Mapserver :) Still experimental in the SVN..... ComplexData reference link outputed as a OGC service
118. According to the data type the link will point to a WMS, WFS or WCS service. So how is it set ?!
119. In the SVN tree we have a WPS client specific for Openlayers Just append the file to the HTML's script tags Now a OpenLayers.WPS class should be available
120. We have 2 major classes, WPS and process WPS API will make all the requests, parse the result and when finish will run a call back function WPS +describeProcess() +getCapabilities() +execute() +onDescrivedProcess: callback +onGotCapabilities: callback +onExecuted: callback
121. A simple example: Please check wiki for an extensive explanation !!!!
122. SOAP == Simple Object Access Protocol WSDL == Web Services Description Language OGC defines that WPS 1.0.0 should support these standards PyWPS has “some” support for SOAP PyWPS generates a simple WSDL file
123. SOAP is a messaging framework, meaning, a structured way to pass, explain and process a message.
124. Example: - Currently PyWPS will accept SOAP XML requests - BUT it will not process any header content or “special Execute tags”
125. - WSDL is a XML document describes a Web service. -Considering a WPS process, then a WSDL would some something like: WSDL Doc == GetCapabilites+ DescribeProcess+ Execute+ OGC WPS standard definition (schema)
126.
127. WSDL file is served as follows: No specific process WSDL file request or support :(
128. SVN branch pywps-3.2-SOAP for WSDL and SOAP development Next PyWPS release will have better SOAP/WSDL support WPS 2.0.0 to have better SOAP/WSDL support Million dolar question ?! Why do we need SOAP/WSDL
129. - Orchestration and interaction with other web services - Ability to use BPEL (Bussines Procedure Language) to orchestrate services
130. -PyCallGraph is used to generate a graphic representation of code being run - Useful to check bottlenecks and code problems
131. - Major time consumption in initProcess() method. - More processes == Slower output A detailed analysis using pyCallGraphic has shown: NO MAJOR BOTTLE NECKS Execute/DescribeProcess spent most of time in process handling If PyWPS is slow, blame the process code Minimum overhead when calling Update and Exception reports
137. Acknowledgments: Simone Gentilini (JRC). GENESI-DR Project funded by FP7 program und e r ( INFRA-2007-1.2.1) Scientific Digital Repositories www.genesi-dr.eu & www.genesi-dec.eu Plymouth Marine Laboratory – Remote Sensing Group www.pml.ac.uk & http://rsg.pml.ac.uk Netmar project. Project partially funded by FP7 program under (ICT-2009.6.4) Information & Communication Technologies. http://netmar.nersc.no/ HS-RS Help Service – Remote Sensing http://www.bnhelp.cz/