Attendees will learn how eBay Germany has implemented Solr, why Solr was selected, which Solr features are utilized. and how Solr is configured and used in production. Recommended best practices will be profiled alomng with eBay Kleinanzeigen plans for future deployment of Solr.
Efficiently Implementing INSPIRE & Creating INSPIRE Mashups with FMESafe Software
This webinar will provide an overview on getting started with FME to deliver INSPIRE-compliant data services, and demonstrate ideas for leveraging the data made available in this open data initiative.
Drawing from over a decade of supporting EU member states, we will demonstrate how you can use FME to efficiently and cost-effectively:
- Transform data schema & geometry to meet annex requirements
- Create INSPIRE GML & ATOM feeds for data delivery
- Leverage the cloud for implementing INSPIRE
- Connect services and APIs to simplify sharing INSPIRE data
- Mash up INSPIRE data with in-house data & satellite imagery such as Copernicus
The INSPIRE directive is one of the largest deployments of Open Geospatial Consortium (OGC) standards and spatial data infrastructure (SDI) projects worldwide. At Safe Software, we’re working to ensure that FME makes it easy to achieve compliance, and for INSPIRE data to be leveraged by data users as the directive intends.
Presentation at FOSS ASIA 2015 in Singapore, on March 15th 2015. This presentation explains benefit of "SQL and NoSQL" hybrid data management solution MySQL Cluster. MySQL Cluster is designed as high available and high scalable database cluster. In the same time MySQL Cluster can work as ACID compliant transactional Key-Value Datastore aka KVS #FOSSASIA
Efficiently Implementing INSPIRE & Creating INSPIRE Mashups with FMESafe Software
This webinar will provide an overview on getting started with FME to deliver INSPIRE-compliant data services, and demonstrate ideas for leveraging the data made available in this open data initiative.
Drawing from over a decade of supporting EU member states, we will demonstrate how you can use FME to efficiently and cost-effectively:
- Transform data schema & geometry to meet annex requirements
- Create INSPIRE GML & ATOM feeds for data delivery
- Leverage the cloud for implementing INSPIRE
- Connect services and APIs to simplify sharing INSPIRE data
- Mash up INSPIRE data with in-house data & satellite imagery such as Copernicus
The INSPIRE directive is one of the largest deployments of Open Geospatial Consortium (OGC) standards and spatial data infrastructure (SDI) projects worldwide. At Safe Software, we’re working to ensure that FME makes it easy to achieve compliance, and for INSPIRE data to be leveraged by data users as the directive intends.
Presentation at FOSS ASIA 2015 in Singapore, on March 15th 2015. This presentation explains benefit of "SQL and NoSQL" hybrid data management solution MySQL Cluster. MySQL Cluster is designed as high available and high scalable database cluster. In the same time MySQL Cluster can work as ACID compliant transactional Key-Value Datastore aka KVS #FOSSASIA
Listado incluido en la propuesta de noma de calidad de los derivados cárnicos en el que se incluyen más de 90 productos tradicionales y denominaciones consagradas por el uso, con sus respectivas descripciones.
The presentation challenges people to think about where their organisation sits on the digital marketing life stages path and then asks are the people in your organisation and your customers – all at the same maturity level?
From pre-school into primary (elementary), through to high school, university and sometimes through to the nursing home or retirement village - every organisation grows through these stages and come sometimes even revert to a different age based on their management, resources, market forces, plans or vision.
This slide deck supported a keynote talk made by Todd Wright from Threesides Marketing at the Collabit ACT November 2013 members breakfast held at the Microsoft Offices in Canberra.
Even an obscure cricketing reference to Ian Chappell made it in somehow – see if you can pick it.
Listado incluido en la propuesta de noma de calidad de los derivados cárnicos en el que se incluyen más de 90 productos tradicionales y denominaciones consagradas por el uso, con sus respectivas descripciones.
The presentation challenges people to think about where their organisation sits on the digital marketing life stages path and then asks are the people in your organisation and your customers – all at the same maturity level?
From pre-school into primary (elementary), through to high school, university and sometimes through to the nursing home or retirement village - every organisation grows through these stages and come sometimes even revert to a different age based on their management, resources, market forces, plans or vision.
This slide deck supported a keynote talk made by Todd Wright from Threesides Marketing at the Collabit ACT November 2013 members breakfast held at the Microsoft Offices in Canberra.
Even an obscure cricketing reference to Ian Chappell made it in somehow – see if you can pick it.
Big Data: Explore Hadoop and BigInsights self-study labCynthia Saracco
Want a quick tour of Apache Hadoop and InfoSphere BigInsights (IBM's Hadoop distribution)? Follow this self-study lab to get hands-on experience with HDFS, MapReduce jobs, BigSheets, Big SQL, and more. This lab was tested against the free BigInsights Quick Start Edition 3.0 VMware image.
Cassandra Day SV 2014: Spark, Shark, and Apache CassandraDataStax Academy
This session covers our experience with using the Spark and Shark frameworks for running real-time queries on top of Cassandra data.We will start by surveying the current Cassandra analytics landscape, including Hadoop and HIVE, and touch on the use of custom input formats to extract data from Cassandra. We will then dive into Spark and Shark, two memory-based cluster computing frameworks, and how they enable often dramatic improvements in query speed and productivity, over the standard solutions today.
Got data? Let's make it searchable! This interactive presentation will demonstrate getting documents into Solr quickly, will provide some tips in adjusting Solr's schema to match your needs better, and finally will discuss how showcase your data in a flexible search user interface. We'll see how to rapidly leverage faceting, highlighting, spell checking, and debugging. Even after all that, there will be enough time left to outline the next steps in developing your search application and taking it to production.
Building a Large Scale SEO/SEM Application with Apache SolrRahul Jain
Slides from my talk on "Building a Large Scale SEO/SEM Application with Apache Solr" in Lucene/Solr Revolution 2014 where I talk how we handle Indexing/Search of 40 billion records (documents)/month in Apache Solr with 4.6 TB compressed index data.
Abstract: We are working on building a SEO/SEM application where an end user search for a "keyword" or a "domain" and gets all the insights about these including Search engine ranking, CPC/CPM, search volume, No. of Ads, competitors details etc. in a couple of seconds. To have this intelligence, we get huge web data from various sources and after intensive processing it is 40 billion records/month in MySQL database with 4.6 TB compressed index data in Apache Solr.
Due to large volume, we faced several challenges while improving indexing performance, search latency and scaling the overall system. In this session, I will talk about our several design approaches to import data faster from MySQL, tricks & techniques to improve the indexing performance, Distributed Search, DocValues(life saver), Redis and the overall system architecture.
Web Performance tuning presentation given at http://www.chippewavalleycodecamp.com/
Covers basic http flow, measuring performance, common changes to improve performance now, and several tools and techniques you can use now.
Innovation dank DevOps (DevOpsCon Berlin 2015)Wooga
“You build it, you run it!” - Wenn Du als Entwickler weisst, dass Du Deine Software selbst betreiben musst, was bist bereit zu tun, um den späteren Betrieb zu vereinfach?
Bei Wooga haben Dutzende von Teams ihre eigene Antwort auf die Frage gesucht und dabei von den Erfahrungen der anderen Teams gelernt. Herausgekommen ist ein großes Experimentierfeld beim Betrieb von Web Services - und eine technologische Innovation, die uns innerhalb weniger Iterationen von einem simplen LAMP-Stack zu lastabhängig skalierenden stateful Servern auf Basis von Erlang oder Akka gebracht hat.
Scaling Up Machine Learning Experimentation at Tubi 5x and BeyondScyllaDB
Scylla enables rapid Machine Learning experimentation at Tubi. The current-generation personalization service, Ranking Service, ramps up experimentation by 5x, while Popper, the next-generation experimentation engine, will grow by 10x and beyond. We'll talk about what's so special about these services.
Couchbase Connect 2014: Lucidworks CEO Will Hayes takes you on a fantastic voyage through the hope and the hype of big data and why the future is search-centric.
LucidWorks SiLK is an open source stack that combines Lucene/Solr with best in class open source data ingestion and analytics tools such as Flume, LogStash and Kibana. This webinar will explore the features of SiLK, and provide attendees with valuable information on how they can benefit from the following:
- A powerful UI to analyze time series data stored in Lucene/Solr
- Creating and sharing visualizations, dashboards and reports
- Discovery and analysis of data coming from servers, applications, devices and more
- Exploration of click, geospatial and social data in ways previously unimaginable
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Essentials of Automations: Optimizing FME Workflows with Parameters
Solr @ eBay Kleinanzeigen
1. Solr @ eBay Kleinanzeigen
Olaf Zschiedrich, eBay Classifieds Group
ozschiedrich@ebay-kleinanzeigen.de, 5/25/2011
2. Who I am?
! Olaf Zschiedrich
! eBay Classifieds Group
! Head of Technology @ eBay Kleinanzeigen
! Area of expertise/interest:
• High traffic web-applications
• Agile development
• Java/JEE
• Search technologies
3
3. Agenda
! About eBay Classifieds Group/ebay Kleinanzeigen
! Metrics & Traffic Numbers
! Why Solr?
! Solr Features in Action
! Data Indexing
! Solr in Production
! Best Practices
! Problems
! Outlook
! Questions
4
6. About eBay Kleinanzeigen
! Typilcal classifieds ad platform (horizontal, local trading)
! Launched 2009 after 4 months of development
! Small agile team (using Scrum)
• 12-15 people total
• 5-7 developers
! Leverages open source (Spring, Solr, MySQL, ActiveMQ)
! Applications:
• Public website
• Customer support tool
• API (Rest supporting JSON and XML)
• Iphone App (~ 250.000 installations)
• Facebook App
7
7. Metrics & Traffic Numbers
! Site metrics:
• ~ 3.2 M active ads
• 16 – 24 M PVs per day
• Peak hours = 1.8 M PVs (~ 500 PVs per second)
! Solr request metrics:
• ~ 60 M requests per day
• Peak hours = ~ 1500 request per second
! Avg. response time
• 20 ms (search) and 3 ms for auto-suggest
Site is rapidly growing !!!
8
8. Why Solr
! Open Source
! Good documentation / big community
! Java-based (the language we know/use)
! Widely used (especially lucene)
! Based on lucene (de-facto standard for full text search in java)
! Feature-rich (including enterprise features)
! Extensible (e.g. easy implementation of own tokenizers)
! Easy to integrate (HTTP, SolrJ client)
! Easy to setup (java web application)
Most promising option we looked at. Due to very aggressive
timelines no time consuming research was possible!
9
9. Solr Features in Action
! Faceting
! Language specific stemming
! More Like This
! Auto-Suggest based on TermComponent
! Spellchecking
! Synonyms
! Stopwords
! Dynamic fields
10
10. Data Indexing
! Use of Delta Import Handler
! Delta import runs every 10 minutes
JDBC
MySQL Solr Master ! Full import only done in case schema
Slave Delta Import Handler change requires full index rebuild
! Index optimized once a day
HTTP / REST API
Replication Handler
Solr Slave Solr Slave Solr Slave
11
11. Solr In Production
! 2 datacenters
! 1 Master + 6 Slaves per datacenter
Slaves show very low resource consumption. Could go down to 4
slaves per datacenter while still having 50% overcapacity
! Master only used for indexing
! Load balancer in front of slaves
! Varnish in front of slaves (for dedicated use cases)
! Working closely with SITE-OPS Team
! DEV-OPS are part of development process
12
12. Solr 3.1 in Production
! Solr 3.1 productive since mid of May
! Not plug and play. Needs migration path as:
• Index format has changed
• Java-bin format has changed
! Two major problems:
• Bug in spellchecker (SOLR-2462)
Leads to infinite GC loops
• Bug in replication handler (SOLR-2469)
Leads to growing disk usage as old index files are not removed is
case “replicateAfter=startup” is used.
13
13. Best Practises
! Use solr cores right from the beginning
Allows you to run mutiple indexes on one box in dev and distribute indexes to mutiple boxes in production
! Use filter queries
! Use caching (FieldCache, QueryCache, Web Proxy Cache e.g. Varnish or Squid)
! Tune JVM properly
! Build search-layer hiding the usage of solr
SearchCommand cmd = new SearchCommand();
cmd.setKeywords(“BMW 323“);
...
SearchResult result = searchService.searchActiveAds(cmd);
"
List<Ad> ads = result.getAds();
! Create a QueryBuilder to ease query building
SolrQueryBuilder sqb = new SolrQueryBuilder();
sqb = sqb.freetext("freetext", "BMW").and().in("color", "RED", "BLACK“);
sqb = sqb.and().not().eq("fuel_type", "GAS").and().lt(“price“, "10000");
...
String query = sqb.build();
(Just an example. Normally filter queries should be used for a query like this!)
14
14. Problems
! Distance search including sorting
• Not supported in previous Solr versions
• LocalSolr
not working with Solr 1.4 final, GC issues, performance issues
• Solution:
Got rid of sort by distance. Implemented own distance search
based on bounding boxes and simple range queries.
• Solved in 3.1
! Real time updates
! Deep paging large result sets (SOLR-1726)
15
15. Outlook / Future Plans
! Migrate further applications to Solr
Most batch-jobs and customer support tool search against db
which is getting slower due to growth of data.
! Evaluate new features of Solr 3.1
• Spatial/distance search
• New auto-suggest component
• Extended dismax query parser
16