1Spatial Australia: FME workbench performance tips and tricks
•Download as PPTX, PDF•
0 likes•59 views
FME 2017 includes performance improvements that allow users to create and execute FME workspaces faster. New features like remodeled CSV reading, improved attribute management, and a faster FeatureReader help reduce processing times. An example workspace demonstrates how to add geographical information to a list of FME World Tour stops by reading and filtering data sources and matching event locations to host cities. FME 2017 enhances both the creation and execution of workspaces for increased performance and productivity.
Report
Share
Report
Share
Recommended
1Spatial: Cardiff FME World Tour: FME workbench performance tips & tricks
This document discusses performance tips and tricks for using FME 2017. FME 2017 includes improvements that allow users to create FME solutions and execute them faster than ever. Specific enhancements include remodeled CSV and MapInfo reading, faster feature readers, enhanced polygon dissolving, and improved attribute management and list handling. An example is provided where reading and filtering geographical data took over a minute in 2016 but only seconds in 2017, demonstrating the performance gains in FME 2017.
FME Workbench Performance Tips & Tricks
Double double, less toil and trouble… FME has always had tricks for optimizing your workflows, and FME 2017 has a few more up its sleeve. Learn about faster reading and filtering of tabular data, better ways to preserve system resources, pulling data directly from a web page, and more.
Turbocharging FME: How to Improve the Performance of Your FME Workspaces
Getting the best performance out of FME is as important to us as it is to you. In this webinar, you’ll get tips from three FME experts - Mark Ireland (FME Evangelist), David Eagle (FME Certified Trainer and Professional) and Dale Lutz (Safe Software Co-founder). They’ll share easy-to-apply advice on: querying databases efficiently, making the most of FME's new multiprocessing capabilities, and simple techniques to speed up your workflows.
Using the Hausdorff distance to identify significant changes in polygon shapes
Our client presented us with a challenge of keeping two datasets in sync. One dataset was maintained by the GIS department and they are constantly in the process of updating the parcel boundaries. Sometimes these updates are small changes such as correcting the borders to match the property lines. Other times, these changes are significant. Significant changes can include merging parcels, splitting parcels or large corrections to the parcel shape. The other dataset is maintained by the maintenance department and it denotes which parcels they must go out and do a maintenance task on. This maintenance dataset was originally derived from the GIS dataset, but all synchronizations were being done manually. The main challenge is that if a parcel was previously a maintenance parcel and it has been merged or split, then a decision must be made on whether the new parcels are still considered maintenance parcels. This presentation will show how you can use the HausdorffDistanceCalculator transformer that is on the FME Hub, and some of the properties of the polygons to determine which changes are large enough to need to be reviewed. Anything that does not meet the threshold is automatically passed into the maintenance dataset, and anything that meets the threshold is passed into a “needs review” dataset. This immensely reduces the amount of data that the reviewer must look at and makes our client successful by saving them a ton of time.
FME 2020 Platform Scenarios
Join us on a tour of the FME Platform, with live demos that will showcase the latest and greatest features. First, we’ll build an app that connects to an API and helps us make intelligent choices based on that data. Next, we’ll build an app that uses machine learning tools to analyze images and make decisions based on what the images contain. Get tips and tricks from our experts and see what’s new in FME 2020.
Deep Dive into FME Server 2014
This document summarizes a webinar about new features in FME Server 2014. It introduces the presenters and outlines the agenda which includes highlights and demos of new features as well as a question and answer session. Some of the new features that will be highlighted are improved usability, scheduling capabilities, workflow management improvements, parameters, web services, integration with Amazon web services, a new REST API, and real-time notifications.
Workspace Authoring 101: Feature Caching
Feature Caching is a mode in FME for workspace authoring. It’s basically a way to store data for future access. Feature Caching can save you time and make you more productive – if you know how to use it.
Though not complicated to learn, Feature Caching is often overlooked by FME users because they don't know how to get started. In this webinar, learn the basics of Feature Caching, including:
- Common uses
- Partial runs
- Collapsed bookmarks
- Templates
- And more!
Afterwards, you’ll feel confident using Feature Caching in your workflows and will improve the way you author workflows.
A Whirlwind Tour of Spatial Joins
Anyone who has worked with spatial databases has had the need for spatial joins. It’s a GIS operation that allows you to combine information from different tables using their relative spatial locations.
Sounds complex, but in FME it’s as easy as selecting a transformer for the job. In this webinar, learn the basics of merging and joining spatial data, including:
- Understanding what transformer is useful in different scenarios
- Incorporating GIS capabilities within your workflows
- Automating your workflows
By using FME for spatial joins, you’ll be able to apply spatial joins to data independent of format, and get it done as fast as possible.
Recommended
1Spatial: Cardiff FME World Tour: FME workbench performance tips & tricks
This document discusses performance tips and tricks for using FME 2017. FME 2017 includes improvements that allow users to create FME solutions and execute them faster than ever. Specific enhancements include remodeled CSV and MapInfo reading, faster feature readers, enhanced polygon dissolving, and improved attribute management and list handling. An example is provided where reading and filtering geographical data took over a minute in 2016 but only seconds in 2017, demonstrating the performance gains in FME 2017.
FME Workbench Performance Tips & Tricks
Double double, less toil and trouble… FME has always had tricks for optimizing your workflows, and FME 2017 has a few more up its sleeve. Learn about faster reading and filtering of tabular data, better ways to preserve system resources, pulling data directly from a web page, and more.
Turbocharging FME: How to Improve the Performance of Your FME Workspaces
Getting the best performance out of FME is as important to us as it is to you. In this webinar, you’ll get tips from three FME experts - Mark Ireland (FME Evangelist), David Eagle (FME Certified Trainer and Professional) and Dale Lutz (Safe Software Co-founder). They’ll share easy-to-apply advice on: querying databases efficiently, making the most of FME's new multiprocessing capabilities, and simple techniques to speed up your workflows.
Using the Hausdorff distance to identify significant changes in polygon shapes
Our client presented us with a challenge of keeping two datasets in sync. One dataset was maintained by the GIS department and they are constantly in the process of updating the parcel boundaries. Sometimes these updates are small changes such as correcting the borders to match the property lines. Other times, these changes are significant. Significant changes can include merging parcels, splitting parcels or large corrections to the parcel shape. The other dataset is maintained by the maintenance department and it denotes which parcels they must go out and do a maintenance task on. This maintenance dataset was originally derived from the GIS dataset, but all synchronizations were being done manually. The main challenge is that if a parcel was previously a maintenance parcel and it has been merged or split, then a decision must be made on whether the new parcels are still considered maintenance parcels. This presentation will show how you can use the HausdorffDistanceCalculator transformer that is on the FME Hub, and some of the properties of the polygons to determine which changes are large enough to need to be reviewed. Anything that does not meet the threshold is automatically passed into the maintenance dataset, and anything that meets the threshold is passed into a “needs review” dataset. This immensely reduces the amount of data that the reviewer must look at and makes our client successful by saving them a ton of time.
FME 2020 Platform Scenarios
Join us on a tour of the FME Platform, with live demos that will showcase the latest and greatest features. First, we’ll build an app that connects to an API and helps us make intelligent choices based on that data. Next, we’ll build an app that uses machine learning tools to analyze images and make decisions based on what the images contain. Get tips and tricks from our experts and see what’s new in FME 2020.
Deep Dive into FME Server 2014
This document summarizes a webinar about new features in FME Server 2014. It introduces the presenters and outlines the agenda which includes highlights and demos of new features as well as a question and answer session. Some of the new features that will be highlighted are improved usability, scheduling capabilities, workflow management improvements, parameters, web services, integration with Amazon web services, a new REST API, and real-time notifications.
Workspace Authoring 101: Feature Caching
Feature Caching is a mode in FME for workspace authoring. It’s basically a way to store data for future access. Feature Caching can save you time and make you more productive – if you know how to use it.
Though not complicated to learn, Feature Caching is often overlooked by FME users because they don't know how to get started. In this webinar, learn the basics of Feature Caching, including:
- Common uses
- Partial runs
- Collapsed bookmarks
- Templates
- And more!
Afterwards, you’ll feel confident using Feature Caching in your workflows and will improve the way you author workflows.
A Whirlwind Tour of Spatial Joins
Anyone who has worked with spatial databases has had the need for spatial joins. It’s a GIS operation that allows you to combine information from different tables using their relative spatial locations.
Sounds complex, but in FME it’s as easy as selecting a transformer for the job. In this webinar, learn the basics of merging and joining spatial data, including:
- Understanding what transformer is useful in different scenarios
- Incorporating GIS capabilities within your workflows
- Automating your workflows
By using FME for spatial joins, you’ll be able to apply spatial joins to data independent of format, and get it done as fast as possible.
fmewt19 - Around the world stories master deck
The document contains summaries from various organizations describing how they use FME to solve geospatial data challenges. The summaries are:
1) Suncor Energy automated geospatial data integration to reduce time spent on data tasks from hours to minutes and remove human error.
2) Buccleuch Estates automated farm inspection reports by compiling photos and notes into standardized PDF reports faster than manual methods.
3) Valeron Enviro Consulting visualized windfield data models to create a marketable product for architects by analyzing and calculating the data in FME.
1Spatial Australia: Finding FME
Leon Bowers is the GIS Manager at Lendlease who discovered the value of FME after struggling with inefficient manual processes for handling GIS data. He realized FME could automate workflows to import survey and weather data on the Kingsford Smith Drive upgrade project. This saved significant time over ESRI tools. For subsequent projects, FME workspaces helped compile deliverables on tight deadlines and allowed flexible updating of changing data. Leon aims to further utilize FME Server for additional automation and integration across the organization. He emphasizes the power, pricing and learning curve benefits of FME compared to previous tools.
Meet the FME Product Managers and Take Your Data to New Heights
This document summarizes a webinar hosted by various product managers from Safe Software to discuss their FME products. Attendees could ask questions to the panelists about product roadmaps and priorities. Panelists included managers for FME Desktop, Server, Mobile, XML standards, and more. Questions covered topics like mobile app development, server automation improvements, tighter integration between products, and standards support. Polls gathered feedback on priorities and interests around features, community engagement, and software development.
Streamlining Processes Towards High-Quality Cartography
Cartography is both the science and the art of visually representing geographical data. It promotes a better understanding of our data, and ultimately, data-driven business decisions.
GIS tools produce excellent maps. But taking your cartography to the next level requires a vector graphics editor - even if it’s just for a few finishing touches - and that can be a tedious, manual effort. Using FME and Avenza as part of the process, you can eliminate the struggles of high-quality map creation. You’ll save many hours of work, leaving you more time for the fun, creative aspects of your cartography.
Join us and special guest speakers Hans van der Maarel from Red Geographics and Nick Burchell from Avenza to learn how FME shines as part of a high-end map production workflow. During the hour, we’ll walk you through:
- An overview of MAPublisher’s extensive cartographic capabilities
- An overview of FME and its basic cartographic capabilities, including the Illustrator format writer
- How Avenza’s MAPublisher integrates FME with Adobe Illustrator directly
- The power of automation to achieve faster map production
Don’t miss out on the chance to learn how you can make your maps with more precision, speed, and style than ever. See you there!
Présentation du FME World Tour du 13 avril 2017 à Quebec
Présentation de l'événement FME World Tour 2017 qui a eu lieu le 13 avril 2017 à Québec. Découvrez les nouveautés de FME 2017 et FME Server 2017. Voyez les trucs et astuces pour optimiser la performance de vos workbench, une solution pour comparer des workspaces ensemble, un portail de chargement et téléchargement de données avec FME Server ainsi que des outils de validation et correction topologique.
Authoring with FME 2019
Get a closer look at FME 2019 and see how to master your data challenges with new formats, machine learning transformers, and new usability features including Visual Preview and more.
Safe on FME: Migrating the FME Community
The FME Community has undergone many 'glow ups' over the years, with the most recent being a new Community backing onto Salesforce. This new Community not only looks great, but having all our customer information in one place means we’re able to offer a higher level of support. Win, win!
The risk in moving platforms was whether or not we’d lose any valuable content along the way.
So how did we successfully push 1,642 articles, 3,045 ideas, and 18,450 questions over to Salesforce? With FME, of course!
Join this webinar to learn how Safe Software used FME to perform data migration tasks, including connecting to various APIs, data validation, and restructuring. And, as always, you’ll get plenty of tips and tricks along the way.
Back to FME School - Day 1: Your Data and FME
It’s that time of year. The season is changing and FME ‘school’ is now in session! Join us for a series of 9 mini-talks to learn the latest tips for data transformation, see live demos, and get your FME questions answered. Registration gives you access for all three days — sign up now to tune in to the talks you’re most interested in.
Course Schedule – Day 1: Your Data and FME
8:00am – FME Workbench Performance Tips & Tricks
8:40am – A Database for Every Occasion
9:20am – Working with Attributes in FME
A Safer's Guide to Best Practices for Optimizing Jobs on FME Server
As an FME Server Administrator, part of your role is to oversee the efficient processing of jobs. But as your FME Server usage grows, you may face instances when the demand exceeds your engine processing capacity and jobs begin to pile up in the queue. If you have ever wondered:
How can I make my jobs more performant?
Why is my job taking longer to run on FME Desktop than FME Server?
How can I prioritize short over long jobs or those submitted by certain users?
How can I prevent engines from sitting idle while I have jobs waiting in the queue?
Then this presentation is the one for you. We’ll highlight different tools you can use to monitor both the individual job performance and the overall job throughput. We’ll also show you what methods you can implement for improved resource and job optimization.
Back to FME School - Day 2: Your Data and FME
It’s that time of year. The season is changing and FME ‘school’ is now in session! Join us for a series of 9 mini-talks to learn the latest tips for data transformation, see live demos, and get your FME questions answered. Registration gives you access for all three days — sign up now to tune in to the talks you’re most interested in.
Course schedule - Day 2
Automating Everything – Wednesday, September 27, 8:00am – 10:00am PDT
8:00am – Bulk data processing
8:40am – Ultimate Real-Time: Monitor Anything, Update Anything
9:20am – FME in the Enterprise
Mastering Geospatial Metadata with FME
This document discusses FME and metadata. It provides an overview of FME capabilities for reading, writing, updating, harvesting, and validating metadata according to various standards like FGDC and ISO 19115. The document demonstrates these capabilities through examples and discusses integrating metadata with catalog services. It also promotes online FME training courses for working with metadata and other data management tasks.
From Raw to Cooked: In-Depth FME Desktop to FME Server Workflow
FME Server allows spatial data to be distributed, accessed, and transformed flexibly across teams. It runs FME workspaces on a server, enabling data to be streamed, downloaded, uploaded and validated by various users. The presentation provides an example scenario where a CAD contractor and GIS analyst collaborate using FME Server - the contractor uploads and validates CAD data, while the analyst maintains workspaces and validates networks. FME Server provides services for data distribution, transformation and loading to efficiently share and process spatial data.
Membase, ShareThis, AOL - Phillips, Mukerji, Jackson - Hadoop World 2010
Better Ad, Offer and Content Targeting with Membase and Hadoop
James Phillips, Membase
Manu Mukerji, ShareThis
Ben Jackson, AOL
Learn more about Hadoop @ http://www.cloudera.com/hadoop/
IMGeospatial: FME World Tour 2018 Nottingham: Supporting Major Land Referenci...
Fisher German is a GIS team that has used FME for 5 years to help manage major infrastructure projects. They cleanse and organize large datasets containing land ownership records. FME is used to automate processes like parcel creation, grantor numbering, layer creation, and plan generation. It integrates data from various sources and formats it consistently. FME also helps automate tasks like records of condition documentation and hazard reporting to save significant time on large projects managing land referencing data.
Data to the Masses: Automated Word Document Creation with FME
Join us for an exciting webinar featuring special Guest Speaker Mary Brown from the Swiss Agency for Development and Cooperation, who will share her experiences using FME and Microsoft Word to tackle data challenges. Mary will discuss how FME was used to leverage GIS at the core of Swiss Humanitarian Aid, fight fires in combination with ArcGIS Online, and support the Foreign Ministry during the Covid-19 pandemic. Hear her use cases to learn how FME was used to make a real difference in the world.
Our Technical Support Specialist Kailin will then show you how to use FME and Microsoft Word Writer to automate the creation of professional documents, making your work faster and more efficient. You'll learn how to design custom templates, merge data from multiple sources, and generate documents with multiple pages and sections. Plus, Kailin will give a special bonus demo on using FME to write PowerPoint documents.
Whether you're a beginner or an experienced user, this webinar is a must-see. Sign up now to reserve your free spot and learn how to use FME to create Microsoft Word documents like a pro!
Save Time and Boost Productivity with No-Code Web Apps
Join us to discover how Dublin City Council saves 21,000 tradesperson hours, 250,000 printed pages, and 90 administrative hours annually with the power of FME Flow apps. Learn how you too can reduce requests and save time through self-serve data access offered by FME Flow apps, without the hassle and expense of custom coding.
Join Safe and FME Gold Partner, IMGS, and Dublin City Council as we explore the power of FME for delivering web applications. During the hour, we’ll show you how FME users can build progressive web apps. We’ll walk through tangible customer use cases, illustrating how FME empowers field personnel to view allocated jobs, update job statuses, and allow managers to efficiently delegate tasks to their teams.
Don’t miss this opportunity to learn how FME can elevate your organization by enabling dynamic web apps. Register now!
A Safer's Guide to Best Practices for Optimizing Jobs on FME Server
As an FME Server Administrator, part of your role is to oversee the efficient processing of jobs. But as your FME Server usage grows, you may face instances when the demand exceeds your engine processing capacity and jobs begin to pile up in the queue. If you have ever wondered:
- How can I make my jobs more performant?
- Why is my job taking longer to run on FME Desktop than FME Server?
- How can I prioritize short over long jobs or those submitted by certain users?
- How can I prevent engines from sitting idle while I have jobs waiting in the queue?
Then this presentation is the one for you. We’ll highlight different tools you can use to monitor both the individual job performance and the overall job throughput. We’ll also show you what methods you can implement for improved resource and job optimization.
Open Day July 2019
We share our activities of the month with each other and the community. It's when we take a step back and see where we stand.
Web Mapping 101: What Is It and Making It Work For You
Web mapping is the process of using the internet to visualize, analyze, and share your geospatial data through a map. Web maps are an important tool for many organizations as they provide the ability to distribute critical information to anyone, anywhere, and at any time.
Web maps provide endless potential for visualizing valuable data that may otherwise go unused. But, not everyone knows how to get started with creating one. In this webinar, we’ll cover:
- An overview of web mapping and how it works
- How OpenLayers and Leaflet work with web mapping
- How to use web mapping tools, including Esri Leaflet and Mapbox with the HTMLReportGenerator
- How to create vector tilesets in FME to make web mapping easier than ever
Join our team of Support Specialists to learn how to get started using FME to create a web map of your own to visualize and share your data.
Stop wasting the value of your geospatial data by letting it sit unused. You’ll leave this webinar with the tools to get you started with creating a web map of your own so you can present your data in a way thats easy to understand and share with others.
Back to FME School - Day 3: Expanding Frontiers
It's that time of year. The season is changing and FME 'school' is now in session! Join us for a series of 9 mini-talks to learn the latest tips for data transformation, see live demos, and get your FME questions answered. Registration gives you access for all three days — sign up now to tune in to the talks you're most interested in.
Course Schedule - 40 Minute Talks:
Expanding Frontiers - Day 3
- Remote sensing providers
- Web and business data into gold
- New + Upcoming in FME Cloud
1Spatial Australia: Batch Data Processing
This document discusses four different methods ("potions") for batch processing large datasets in FME: 1) Wildcards, 2) Batch Deploy, 3) Parent/Child Workspaces, and 4) Parent/Child Server Workspaces. Each method has advantages and disadvantages. Wildcards allow simple bulk processing but recovery from errors is difficult. Batch Deploy is easy to set up but has limited logging. Parent/Child Workspaces separate transformations from workflows but individual children run slowly. Parent/Child Server Workspaces leverage parallelism for speed but require data accessibility to server engines. Overall, FME Server is recommended for the most robust automation, but the best method depends on the specific needs and data.
1Spatial Australia: Dynamic Mapping
This document summarizes Powerlink Queensland's development of a dynamic mapping solution powered by FME software. The solution automated Powerlink's mapping processes, reducing manual production time from over 6 hours to under 5 minutes per map. This represents an efficiency gain of over 6,000 times faster than manual processes. The automated, standardized maps can now be produced on-demand and ensure access to up-to-date information. The dynamic mapping solution has demonstrated a 125% return on investment and enables further innovation.
More Related Content
Similar to 1Spatial Australia: FME workbench performance tips and tricks
fmewt19 - Around the world stories master deck
The document contains summaries from various organizations describing how they use FME to solve geospatial data challenges. The summaries are:
1) Suncor Energy automated geospatial data integration to reduce time spent on data tasks from hours to minutes and remove human error.
2) Buccleuch Estates automated farm inspection reports by compiling photos and notes into standardized PDF reports faster than manual methods.
3) Valeron Enviro Consulting visualized windfield data models to create a marketable product for architects by analyzing and calculating the data in FME.
1Spatial Australia: Finding FME
Leon Bowers is the GIS Manager at Lendlease who discovered the value of FME after struggling with inefficient manual processes for handling GIS data. He realized FME could automate workflows to import survey and weather data on the Kingsford Smith Drive upgrade project. This saved significant time over ESRI tools. For subsequent projects, FME workspaces helped compile deliverables on tight deadlines and allowed flexible updating of changing data. Leon aims to further utilize FME Server for additional automation and integration across the organization. He emphasizes the power, pricing and learning curve benefits of FME compared to previous tools.
Meet the FME Product Managers and Take Your Data to New Heights
This document summarizes a webinar hosted by various product managers from Safe Software to discuss their FME products. Attendees could ask questions to the panelists about product roadmaps and priorities. Panelists included managers for FME Desktop, Server, Mobile, XML standards, and more. Questions covered topics like mobile app development, server automation improvements, tighter integration between products, and standards support. Polls gathered feedback on priorities and interests around features, community engagement, and software development.
Streamlining Processes Towards High-Quality Cartography
Cartography is both the science and the art of visually representing geographical data. It promotes a better understanding of our data, and ultimately, data-driven business decisions.
GIS tools produce excellent maps. But taking your cartography to the next level requires a vector graphics editor - even if it’s just for a few finishing touches - and that can be a tedious, manual effort. Using FME and Avenza as part of the process, you can eliminate the struggles of high-quality map creation. You’ll save many hours of work, leaving you more time for the fun, creative aspects of your cartography.
Join us and special guest speakers Hans van der Maarel from Red Geographics and Nick Burchell from Avenza to learn how FME shines as part of a high-end map production workflow. During the hour, we’ll walk you through:
- An overview of MAPublisher’s extensive cartographic capabilities
- An overview of FME and its basic cartographic capabilities, including the Illustrator format writer
- How Avenza’s MAPublisher integrates FME with Adobe Illustrator directly
- The power of automation to achieve faster map production
Don’t miss out on the chance to learn how you can make your maps with more precision, speed, and style than ever. See you there!
Présentation du FME World Tour du 13 avril 2017 à Quebec
Présentation de l'événement FME World Tour 2017 qui a eu lieu le 13 avril 2017 à Québec. Découvrez les nouveautés de FME 2017 et FME Server 2017. Voyez les trucs et astuces pour optimiser la performance de vos workbench, une solution pour comparer des workspaces ensemble, un portail de chargement et téléchargement de données avec FME Server ainsi que des outils de validation et correction topologique.
Authoring with FME 2019
Get a closer look at FME 2019 and see how to master your data challenges with new formats, machine learning transformers, and new usability features including Visual Preview and more.
Safe on FME: Migrating the FME Community
The FME Community has undergone many 'glow ups' over the years, with the most recent being a new Community backing onto Salesforce. This new Community not only looks great, but having all our customer information in one place means we’re able to offer a higher level of support. Win, win!
The risk in moving platforms was whether or not we’d lose any valuable content along the way.
So how did we successfully push 1,642 articles, 3,045 ideas, and 18,450 questions over to Salesforce? With FME, of course!
Join this webinar to learn how Safe Software used FME to perform data migration tasks, including connecting to various APIs, data validation, and restructuring. And, as always, you’ll get plenty of tips and tricks along the way.
Back to FME School - Day 1: Your Data and FME
It’s that time of year. The season is changing and FME ‘school’ is now in session! Join us for a series of 9 mini-talks to learn the latest tips for data transformation, see live demos, and get your FME questions answered. Registration gives you access for all three days — sign up now to tune in to the talks you’re most interested in.
Course Schedule – Day 1: Your Data and FME
8:00am – FME Workbench Performance Tips & Tricks
8:40am – A Database for Every Occasion
9:20am – Working with Attributes in FME
A Safer's Guide to Best Practices for Optimizing Jobs on FME Server
As an FME Server Administrator, part of your role is to oversee the efficient processing of jobs. But as your FME Server usage grows, you may face instances when the demand exceeds your engine processing capacity and jobs begin to pile up in the queue. If you have ever wondered:
How can I make my jobs more performant?
Why is my job taking longer to run on FME Desktop than FME Server?
How can I prioritize short over long jobs or those submitted by certain users?
How can I prevent engines from sitting idle while I have jobs waiting in the queue?
Then this presentation is the one for you. We’ll highlight different tools you can use to monitor both the individual job performance and the overall job throughput. We’ll also show you what methods you can implement for improved resource and job optimization.
Back to FME School - Day 2: Your Data and FME
It’s that time of year. The season is changing and FME ‘school’ is now in session! Join us for a series of 9 mini-talks to learn the latest tips for data transformation, see live demos, and get your FME questions answered. Registration gives you access for all three days — sign up now to tune in to the talks you’re most interested in.
Course schedule - Day 2
Automating Everything – Wednesday, September 27, 8:00am – 10:00am PDT
8:00am – Bulk data processing
8:40am – Ultimate Real-Time: Monitor Anything, Update Anything
9:20am – FME in the Enterprise
Mastering Geospatial Metadata with FME
This document discusses FME and metadata. It provides an overview of FME capabilities for reading, writing, updating, harvesting, and validating metadata according to various standards like FGDC and ISO 19115. The document demonstrates these capabilities through examples and discusses integrating metadata with catalog services. It also promotes online FME training courses for working with metadata and other data management tasks.
From Raw to Cooked: In-Depth FME Desktop to FME Server Workflow
FME Server allows spatial data to be distributed, accessed, and transformed flexibly across teams. It runs FME workspaces on a server, enabling data to be streamed, downloaded, uploaded and validated by various users. The presentation provides an example scenario where a CAD contractor and GIS analyst collaborate using FME Server - the contractor uploads and validates CAD data, while the analyst maintains workspaces and validates networks. FME Server provides services for data distribution, transformation and loading to efficiently share and process spatial data.
Membase, ShareThis, AOL - Phillips, Mukerji, Jackson - Hadoop World 2010
Better Ad, Offer and Content Targeting with Membase and Hadoop
James Phillips, Membase
Manu Mukerji, ShareThis
Ben Jackson, AOL
Learn more about Hadoop @ http://www.cloudera.com/hadoop/
IMGeospatial: FME World Tour 2018 Nottingham: Supporting Major Land Referenci...
Fisher German is a GIS team that has used FME for 5 years to help manage major infrastructure projects. They cleanse and organize large datasets containing land ownership records. FME is used to automate processes like parcel creation, grantor numbering, layer creation, and plan generation. It integrates data from various sources and formats it consistently. FME also helps automate tasks like records of condition documentation and hazard reporting to save significant time on large projects managing land referencing data.
Data to the Masses: Automated Word Document Creation with FME
Join us for an exciting webinar featuring special Guest Speaker Mary Brown from the Swiss Agency for Development and Cooperation, who will share her experiences using FME and Microsoft Word to tackle data challenges. Mary will discuss how FME was used to leverage GIS at the core of Swiss Humanitarian Aid, fight fires in combination with ArcGIS Online, and support the Foreign Ministry during the Covid-19 pandemic. Hear her use cases to learn how FME was used to make a real difference in the world.
Our Technical Support Specialist Kailin will then show you how to use FME and Microsoft Word Writer to automate the creation of professional documents, making your work faster and more efficient. You'll learn how to design custom templates, merge data from multiple sources, and generate documents with multiple pages and sections. Plus, Kailin will give a special bonus demo on using FME to write PowerPoint documents.
Whether you're a beginner or an experienced user, this webinar is a must-see. Sign up now to reserve your free spot and learn how to use FME to create Microsoft Word documents like a pro!
Save Time and Boost Productivity with No-Code Web Apps
Join us to discover how Dublin City Council saves 21,000 tradesperson hours, 250,000 printed pages, and 90 administrative hours annually with the power of FME Flow apps. Learn how you too can reduce requests and save time through self-serve data access offered by FME Flow apps, without the hassle and expense of custom coding.
Join Safe and FME Gold Partner, IMGS, and Dublin City Council as we explore the power of FME for delivering web applications. During the hour, we’ll show you how FME users can build progressive web apps. We’ll walk through tangible customer use cases, illustrating how FME empowers field personnel to view allocated jobs, update job statuses, and allow managers to efficiently delegate tasks to their teams.
Don’t miss this opportunity to learn how FME can elevate your organization by enabling dynamic web apps. Register now!
A Safer's Guide to Best Practices for Optimizing Jobs on FME Server
As an FME Server Administrator, part of your role is to oversee the efficient processing of jobs. But as your FME Server usage grows, you may face instances when the demand exceeds your engine processing capacity and jobs begin to pile up in the queue. If you have ever wondered:
- How can I make my jobs more performant?
- Why is my job taking longer to run on FME Desktop than FME Server?
- How can I prioritize short over long jobs or those submitted by certain users?
- How can I prevent engines from sitting idle while I have jobs waiting in the queue?
Then this presentation is the one for you. We’ll highlight different tools you can use to monitor both the individual job performance and the overall job throughput. We’ll also show you what methods you can implement for improved resource and job optimization.
Open Day July 2019
We share our activities of the month with each other and the community. It's when we take a step back and see where we stand.
Web Mapping 101: What Is It and Making It Work For You
Web mapping is the process of using the internet to visualize, analyze, and share your geospatial data through a map. Web maps are an important tool for many organizations as they provide the ability to distribute critical information to anyone, anywhere, and at any time.
Web maps provide endless potential for visualizing valuable data that may otherwise go unused. But, not everyone knows how to get started with creating one. In this webinar, we’ll cover:
- An overview of web mapping and how it works
- How OpenLayers and Leaflet work with web mapping
- How to use web mapping tools, including Esri Leaflet and Mapbox with the HTMLReportGenerator
- How to create vector tilesets in FME to make web mapping easier than ever
Join our team of Support Specialists to learn how to get started using FME to create a web map of your own to visualize and share your data.
Stop wasting the value of your geospatial data by letting it sit unused. You’ll leave this webinar with the tools to get you started with creating a web map of your own so you can present your data in a way thats easy to understand and share with others.
Back to FME School - Day 3: Expanding Frontiers
It's that time of year. The season is changing and FME 'school' is now in session! Join us for a series of 9 mini-talks to learn the latest tips for data transformation, see live demos, and get your FME questions answered. Registration gives you access for all three days — sign up now to tune in to the talks you're most interested in.
Course Schedule - 40 Minute Talks:
Expanding Frontiers - Day 3
- Remote sensing providers
- Web and business data into gold
- New + Upcoming in FME Cloud
Similar to 1Spatial Australia: FME workbench performance tips and tricks (20)
Meet the FME Product Managers and Take Your Data to New Heights
Meet the FME Product Managers and Take Your Data to New Heights
Streamlining Processes Towards High-Quality Cartography
Streamlining Processes Towards High-Quality Cartography
Présentation du FME World Tour du 13 avril 2017 à Quebec
Présentation du FME World Tour du 13 avril 2017 à Quebec
A Safer's Guide to Best Practices for Optimizing Jobs on FME Server
A Safer's Guide to Best Practices for Optimizing Jobs on FME Server
From Raw to Cooked: In-Depth FME Desktop to FME Server Workflow
From Raw to Cooked: In-Depth FME Desktop to FME Server Workflow
Membase, ShareThis, AOL - Phillips, Mukerji, Jackson - Hadoop World 2010
Membase, ShareThis, AOL - Phillips, Mukerji, Jackson - Hadoop World 2010
IMGeospatial: FME World Tour 2018 Nottingham: Supporting Major Land Referenci...
IMGeospatial: FME World Tour 2018 Nottingham: Supporting Major Land Referenci...
Data to the Masses: Automated Word Document Creation with FME
Data to the Masses: Automated Word Document Creation with FME
Save Time and Boost Productivity with No-Code Web Apps
Save Time and Boost Productivity with No-Code Web Apps
A Safer's Guide to Best Practices for Optimizing Jobs on FME Server
A Safer's Guide to Best Practices for Optimizing Jobs on FME Server
Web Mapping 101: What Is It and Making It Work For You
Web Mapping 101: What Is It and Making It Work For You
More from 1Spatial
1Spatial Australia: Batch Data Processing
This document discusses four different methods ("potions") for batch processing large datasets in FME: 1) Wildcards, 2) Batch Deploy, 3) Parent/Child Workspaces, and 4) Parent/Child Server Workspaces. Each method has advantages and disadvantages. Wildcards allow simple bulk processing but recovery from errors is difficult. Batch Deploy is easy to set up but has limited logging. Parent/Child Workspaces separate transformations from workflows but individual children run slowly. Parent/Child Server Workspaces leverage parallelism for speed but require data accessibility to server engines. Overall, FME Server is recommended for the most robust automation, but the best method depends on the specific needs and data.
1Spatial Australia: Dynamic Mapping
This document summarizes Powerlink Queensland's development of a dynamic mapping solution powered by FME software. The solution automated Powerlink's mapping processes, reducing manual production time from over 6 hours to under 5 minutes per map. This represents an efficiency gain of over 6,000 times faster than manual processes. The automated, standardized maps can now be produced on-demand and ensure access to up-to-date information. The dynamic mapping solution has demonstrated a 125% return on investment and enables further innovation.
1Spatial Australia: Introduction and getting started with fme 2017
This document introduces new features in FME 2017 including over 20 new data formats that can be read and written, more than 10 new transformers, updates to existing transformers, improved user interface features for workflows, expanded web services and file system capabilities, an updated data inspector, and new automation capabilities for running workflows on demand or on a schedule. The overall goal of FME is to allow data to flow freely between systems and applications while enabling users to spend more time making decisions rather than struggling with data integration tasks.
1Spatial Australia: FME Server: Charms in the enterprise
FME Server 2017 introduced several new "charms" to enhance security, deployment, and job control capabilities for enterprise users. The security charms include support for Active Directory, user sharing and access control, and connection-based integration with third-party applications and databases. The deployment charm allows portable "projects" to easily move server solutions between machines. A new job termination charm ensures FME engines do not get stuck on long-running jobs by defining a maximum elapsed time before cancelling jobs.
1Spatial Australia: Ultimate real time - monitor anything, update anything
FME Server Notification Service allows monitoring of external systems and triggering actions in response to events. Notifications can be used to briefly communicate an event but not transmit large amounts of data continuously. Workflows can poll data on a schedule, use database triggers or webhooks to push data in real-time, or update data using FME writers, SQL executer, or HTTP caller. Choosing between polling, pushing, or updating depends on factors like simplicity, ability of the external system to push data, and job processing times.
1Spatial Australia: Remote sensing data - instant home delivery
This document discusses options for accessing satellite imagery data from various providers including Landsat, Sentinel, Planet, and UrtheCast. It provides details on the resolution, frequency, cost, and capabilities of each data source. It also demonstrates how to access the imagery through various readers and process it in FME for applications like 3D modeling, analysis, and quality checks. Remote sensing provides valuable data that can be utilized in many ways through cloud access and data transformation tools.
1Spatial: FME World Tour London: BIM/GIS Integration
This document describes research into extracting building information modeling (BIM) data from MicroStation and importing it into a geographic information system (GIS). Several challenges are outlined around reliably extracting geometry and metadata. Solutions involve using MicroStation scripts to extract element IDs and properties into a format readable by the FME data transformation software. The resulting workflow is able to merge BIM geometry, attributes, and appearance from multiple sources into a consolidated format while maintaining traceability back to the original model elements.
1Spatial: FME World Tour London: Digital surveying with FME
Network Rail owns 16,000 route km of track with over 190,000 earthworks assets. To better understand these assets, a survey was conducted using aerial imagery and LiDAR data. FME software was used to extract over 890,000 cross sections and 91 million spot heights from the imagery and develop slope geometries. This provided a complete and consistent picture of all earthwork assets for the first time. The data was also used to identify slopes outside of Network Rail property that could impact track operations.
1Spatial: FME World Tour London: Postal address clean-up
This document describes how FME was used to clean and normalize postal address data from electoral lists for a political campaign. Addresses from four columns were normalized into a single schema using string processing tools like StringSearcher, AttributeSplitter, and AttributeManager. The normalized addresses can now be easily uploaded and used to update electoral lists as new information is received.
1Spatial: FME World Tour London: Modelling the world in Minecraft
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive functioning. Exercise causes chemical changes in the brain that may help protect against mental illness and improve symptoms.
1Spatial: Leeds FME World Tour: Extracting value from our FME investment
1. Barnsley MBC has used FME since 2013 to migrate legacy GIS datasets to a SQL Server environment and interface with various software packages. FME is used for operational tasks like data processing, validation, and reporting as well as major projects involving data from over 10 sources.
2. Barnsley implemented SharePoint in 2016 to replace an legacy document management system. FME was used to migrate over 5 million items of content from other systems by extracting data, creating metadata and folders, and loading content into SharePoint via its web services.
3. Barnsley has continued finding new uses of FME with SharePoint for tasks like reporting project status from a SharePoint list, maintaining maps and metadata
1Spatial: Leeds FME World Tour: How FME helps us create & maintain a national...
This document discusses how FME software helps create and maintain a national buildings database in the UK called UKBuildings. It allows non-SQL experts to easily load, integrate, and validate building data from various sources. FME is used to clean, link, and publish UKBuildings data in different formats. Lessons learned include defining standard processes and producing consistent, reliable data without SQL expertise. Future plans include more automation, additional quality checks, and simplified data delivery.
1Spatial: Leeds FME World Tour: Improving the flow of asset information with FME
The document describes how FME was used to consolidate a client's large national drainage asset database. Overlapping asset schemes caused duplication that needed resolving. FME processed the data through stages including duplicate removal, subject matter expert review, quality assurance checks, and submission to the client's system. It automated tasks like updating upstream/downstream references and support connections. FME also determined asset elevations from LiDAR point clouds. The resulting single consolidated drainage network provides the client an improved understanding of their extended asset inventory.
1Spatial: Edinburgh FME World Tour: Performance tips
This document discusses performance tips for processing large spatial datasets used by the Improvement Service. It recommends using a Clipper transformer to process features as early as possible, batch processing with WorkspaceRunner, and using a spatial database with SQLExecutor. An example shows clipping a polygon dataset to local authority boundaries throughout Scotland was reduced from 122 minutes to just 33 minutes using these techniques. It also discusses using FeatureReader to read files without knowing the table name in advance and creating "master" workbenches that can be run by altered "runner" workbenches to batch processes and run them in parallel.
1Spatial: Edinburgh FME World Tour: Community council information service
This document discusses a community council information service that uses spatial data and RSS feeds to provide information from Tell Me Scotland, a national web portal for published notices. The service extracts notice information like type, title, date and address from RSS feeds and filters it by location using community council and local authority boundaries. It then publishes the information on a website and social media. The service has been running for over a year and is the most popular page on its associated website.
1Spatial: Edinburgh FME World Tour: Maintain a national buildings database
This document discusses how FME software helps create and maintain a national buildings database in the UK called UKBuildings. It allows non-SQL experts to easily load, integrate, validate, and publish building data from various sources. FME is used to load data into a PostGIS database, create and integrate attributes, validate data quality, and link buildings to other datasets like addresses. This process provides a consistent and reliable way to update the database with new data and publish the buildings information in different formats.
1Spatial: Edinburgh FME World Tour: Property flood risk in Google Earth
Thames Water was facing penalties for property flooding due to sewer blockages and collapses. They needed a way to prioritize response based on property attributes. MWH implemented a solution in Google Earth using KML to display flood risk data for each postcode district. Key attributes like property age, basement, usage and access/traffic were shown. FME was used to transform data into the required KML format, setting properties, styles, and regions. The tool is now successfully used in Thames Water's response center to reduce flooded properties.
1Spatial: Cardiff FME World Tour: Saving memory, saving time
The document discusses techniques for making FME workspaces more efficient when performing geospatial analysis on large datasets. It describes Landmark Information Group's environmental risk models, which calculate proximity for 35 million addresses across multiple datasets. The models were inefficient, processing over 1.7 billion features every 6 weeks. The document outlines three key strategies to improve efficiency: 1) Choosing the best FME transformer for each dataset, 2) Only processing updated portions of datasets, and 3) Increasing overall workspace efficiency through techniques like removing unused attributes and letting the database handle more processing.
1Spatial: Cardiff FME World Tour: Time machines and attribute alchemy
1. Attribute transformation is a key part of ETL work like data migration and involves techniques for parsing, formatting, and extracting attribute values.
2. Date/time attributes require special handling using datetime transformers and functions to deal with inconsistent formats, time zones, and calculations. Regular expressions and string functions help match patterns and extract substrings from attributes.
3. The document provides examples of using regular expressions and string functions in FME to clean up attribute values by extracting parts of strings, validating addresses, and finding postal codes.
1Spatial: Cardiff FME World Tour: Keeping track in kenya
A survey was conducted in Kenya to assess renewable energy resources, collecting over 10,500 surveys across 44 of 47 counties over 9 weeks using mobile devices. The surveys included over 1.1 million data attributes that were validated and sent weekly to UK staff. The solution involved simple scheduling, downloading data from ArcOnline without raster basemaps, and cross-referencing unique survey IDs to validate conditional values using regular expressions.
More from 1Spatial (20)
1Spatial Australia: Introduction and getting started with fme 2017
1Spatial Australia: Introduction and getting started with fme 2017
1Spatial Australia: FME Server: Charms in the enterprise
1Spatial Australia: FME Server: Charms in the enterprise
1Spatial Australia: Ultimate real time - monitor anything, update anything
1Spatial Australia: Ultimate real time - monitor anything, update anything
1Spatial Australia: Remote sensing data - instant home delivery
1Spatial Australia: Remote sensing data - instant home delivery
1Spatial: FME World Tour London: BIM/GIS Integration
1Spatial: FME World Tour London: BIM/GIS Integration
1Spatial: FME World Tour London: Digital surveying with FME
1Spatial: FME World Tour London: Digital surveying with FME
1Spatial: FME World Tour London: Postal address clean-up
1Spatial: FME World Tour London: Postal address clean-up
1Spatial: FME World Tour London: Modelling the world in Minecraft
1Spatial: FME World Tour London: Modelling the world in Minecraft
1Spatial: Leeds FME World Tour: Extracting value from our FME investment
1Spatial: Leeds FME World Tour: Extracting value from our FME investment
1Spatial: Leeds FME World Tour: How FME helps us create & maintain a national...
1Spatial: Leeds FME World Tour: How FME helps us create & maintain a national...
1Spatial: Leeds FME World Tour: Improving the flow of asset information with FME
1Spatial: Leeds FME World Tour: Improving the flow of asset information with FME
1Spatial: Edinburgh FME World Tour: Performance tips
1Spatial: Edinburgh FME World Tour: Performance tips
1Spatial: Edinburgh FME World Tour: Community council information service
1Spatial: Edinburgh FME World Tour: Community council information service
1Spatial: Edinburgh FME World Tour: Maintain a national buildings database
1Spatial: Edinburgh FME World Tour: Maintain a national buildings database
1Spatial: Edinburgh FME World Tour: Property flood risk in Google Earth
1Spatial: Edinburgh FME World Tour: Property flood risk in Google Earth
1Spatial: Cardiff FME World Tour: Saving memory, saving time
1Spatial: Cardiff FME World Tour: Saving memory, saving time
1Spatial: Cardiff FME World Tour: Time machines and attribute alchemy
1Spatial: Cardiff FME World Tour: Time machines and attribute alchemy
1Spatial: Cardiff FME World Tour: Keeping track in kenya
1Spatial: Cardiff FME World Tour: Keeping track in kenya
Recently uploaded
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
UiPath Test Automation using UiPath Test Suite series, part 5
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Recently uploaded (20)
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
1Spatial Australia: FME workbench performance tips and tricks
- 1. FME Workbench Performance Tips & Tricks
- 2. Double, Double Less Toil, Less Trouble GIS professionals are finding themselves tasked with doing double the work in half the time! Luckily, FME is designed to magically ease that burden.
- 3. Problem: The Deadline Dragon Only FME stands between you and being roasted by your clients …
- 4. Solution: The Performance Pixies … FME 2017 boasts the best pixie powers yet
- 5. Performance Goals ✓Create an FME solution faster than ever. ✓Execute it faster than ever.
- 6. Performance Highlights for FME 2017 “Turn to Page 394”
- 7. A Quick List ● Remodeled CSV Reading ● Improved Attribute Management ● Better List Handling ● Enhanced Polygon Dissolving ● Faster FeatureReader Workflows ● Empowered Reading of Data From Web Sources
- 8. In Action “It’s still magic, even if you know how it’s done”
- 9. Goal: Add geographical information to the list of FME World Tour stops.
- 10. Steps 1. Read a Geographical Names CSV Dataset 2. Filter the Geographical Names Data for a List of Cities Only 3. Read the Current List of FME World Tour Events 4. Merge the FMEWT Event with its Country Code 5. Match the FMEWT Event to its Host City 6. Ensure the Correct Location is chosen for each Event
- 11. Here’s the workspace to solve that problem.
- 12. 2016 • Read Data: 1 minute 35 seconds • Read/Filter: 1 minute 36 seconds 2017 •Read Data: 8.3 seconds •Read/Filter: 14.1 seconds Processing Times
- 13. Summary Performance in Creation • Attributes to Read • List Element Selection • HTML Table Reader • Select Source from Web • Modeless Parameter Dialogs • Quick Add Readers/Writers • User Parameter Copy/Paste • Open Containing Folder Performance in Execution • Attributes to Read • List Element Selection • Feature Table Formats • Feature Table Transformers • Faster Polygon Dissolving • Faster FeatureReader