A discussion of the role of taxonomies and other controlled vocabularies in the managing of large amounts of data for researchers, focusing in particular on searchability and data visualization. Presented by Marjorie M.K. Hlava, president of Access Innovations, Inc., for the SLA Military Libraries Division 2013 Workshop, December 12, 2013.
A discussion of the role of taxonomies and other controlled vocabularies in the managing of large amounts of data for researchers, focusing in particular on searchability and data visualization. Presented by Marjorie M.K. Hlava, president of Access Innovations, Inc., for the SLA Military Libraries Division 2013 Workshop, December 12, 2013.
David Meza's slides from his talk at Connected Data London. David is a Chief Knowledge Architect at NASA, Johnson Space Centre. His keynote talk proposed how combining strategy, data science and information architecture can help transform data to knowledge.
Κnowledge Architecture: Combining Strategy, Data Science and Information Arch...Connected Data World
"The most important contribution management needs to make in the 21st Century is to increase the productivity of knowledge work and the knowledge worker", said Peter F. Drucker in 1999, and time has proven him right.
Even NASA is no exception, as it faces a number of challenges. NASA has hundreds of millions of documents, reports, project data, lessons learned, scientific research, medical analysis, geospatial data, IT logs, and all kinds of other data stored nation-wide.
The data is growing in terms of variety, velocity, volume, value and veracity. NASA needs to provide accessibility to engineering data sources, whose visibility is currently limited. To convert data to knowledge a convergence of Knowledge Management, Information Architecture and Data Science is necessary.
This is what David Meza, Acting Branch Chief - People Analytics, Sr. Data Scientist at NASA, calls "Knowledge Architecture": the people, processes, and technology of designing, implementing, and applying the intellectual infrastructure of organizations.
Knowledge Architecture: Graphing Your KnowledgeNeo4j
Ask any project manager and they will tell you the importance of reviewing lessons learned prior to starting a new project. The lesson learned databases are filled with nuggets of valuable information to help project teams increase the likelihood of project success. Why then do most lesson learned databases go unused by project teams? In my experience, they are difficult to search through and require hours of time to review the result set.
Recently I had a project engineer ask me if we could search our lessons learned using a list of 22 key terms the team was interested in. Our current keyword search engine would require him to enter each term individually, select the link, and save the document for review. Also, there was no way to search only the database, the query would search our entire corpus, close to 20 million URLs. This would not do. I asked our search team if they would run a special query against the lesson database only, using the terms provided. They returned a spreadsheet with a link to each document containing the terms. The engineer had his work cut out for him: over 1100 documents were on the list;.
I started thinking there had to be a better way. I had been experimenting with topic modeling, in particular to assist our users in connecting seemingly disparate documents through an easier visualization mechanism. Something better than a list of links on multiple pages. I gathered my toolbox: R/RStudio, for the topic modeling and exploring the data; Neo4j, for modeling and visualizing the topics; and Linkurious, a web front end for our users to search and visualize the graph database.
Sirris innovate2011 - Smart Products with smart data - introduction, Dr. Elen...Sirris
This lecture highlights current trends, challenges and opportunities related to the emergence of large amounts of data. It also presents Sirris’s recent research activities in this domain.
A look back at how the practice of data science has evolved over the years, modern trends, and where it might be headed in the future. Starting from before anyone had the title "data scientist" on their resume, to the dawn of the cloud and big data, and the new tools and companies trying to push the state of the art forward. Finally, some wild speculation on where data science might be headed.
Presentation given to Seattle Data Science Meetup on Friday July 24th 2015.
A discussion of the role of taxonomies and other controlled vocabularies in the managing of large amounts of data for researchers, focusing in particular on searchability and data visualization. Presented by Marjorie M.K. Hlava, president of Access Innovations, Inc., for the SLA Military Libraries Division 2013 Workshop, December 12, 2013.
David Meza's slides from his talk at Connected Data London. David is a Chief Knowledge Architect at NASA, Johnson Space Centre. His keynote talk proposed how combining strategy, data science and information architecture can help transform data to knowledge.
Κnowledge Architecture: Combining Strategy, Data Science and Information Arch...Connected Data World
"The most important contribution management needs to make in the 21st Century is to increase the productivity of knowledge work and the knowledge worker", said Peter F. Drucker in 1999, and time has proven him right.
Even NASA is no exception, as it faces a number of challenges. NASA has hundreds of millions of documents, reports, project data, lessons learned, scientific research, medical analysis, geospatial data, IT logs, and all kinds of other data stored nation-wide.
The data is growing in terms of variety, velocity, volume, value and veracity. NASA needs to provide accessibility to engineering data sources, whose visibility is currently limited. To convert data to knowledge a convergence of Knowledge Management, Information Architecture and Data Science is necessary.
This is what David Meza, Acting Branch Chief - People Analytics, Sr. Data Scientist at NASA, calls "Knowledge Architecture": the people, processes, and technology of designing, implementing, and applying the intellectual infrastructure of organizations.
Knowledge Architecture: Graphing Your KnowledgeNeo4j
Ask any project manager and they will tell you the importance of reviewing lessons learned prior to starting a new project. The lesson learned databases are filled with nuggets of valuable information to help project teams increase the likelihood of project success. Why then do most lesson learned databases go unused by project teams? In my experience, they are difficult to search through and require hours of time to review the result set.
Recently I had a project engineer ask me if we could search our lessons learned using a list of 22 key terms the team was interested in. Our current keyword search engine would require him to enter each term individually, select the link, and save the document for review. Also, there was no way to search only the database, the query would search our entire corpus, close to 20 million URLs. This would not do. I asked our search team if they would run a special query against the lesson database only, using the terms provided. They returned a spreadsheet with a link to each document containing the terms. The engineer had his work cut out for him: over 1100 documents were on the list;.
I started thinking there had to be a better way. I had been experimenting with topic modeling, in particular to assist our users in connecting seemingly disparate documents through an easier visualization mechanism. Something better than a list of links on multiple pages. I gathered my toolbox: R/RStudio, for the topic modeling and exploring the data; Neo4j, for modeling and visualizing the topics; and Linkurious, a web front end for our users to search and visualize the graph database.
Sirris innovate2011 - Smart Products with smart data - introduction, Dr. Elen...Sirris
This lecture highlights current trends, challenges and opportunities related to the emergence of large amounts of data. It also presents Sirris’s recent research activities in this domain.
A look back at how the practice of data science has evolved over the years, modern trends, and where it might be headed in the future. Starting from before anyone had the title "data scientist" on their resume, to the dawn of the cloud and big data, and the new tools and companies trying to push the state of the art forward. Finally, some wild speculation on where data science might be headed.
Presentation given to Seattle Data Science Meetup on Friday July 24th 2015.
Tim Estes - Information Systems in an Entity Centric WorldDigital Reasoning
Tim Estes, CEO of Digital Reasoning, talks about the use of Hadoop and other scalable technologies along with Digital Reasoning's analytics for automated understanding of cloud-scale text challenges.
This presentation was delivered at Hadoop World in New York in Oct 2010
Crowdsourcing Approaches to Big Data Curation - Rio Big Data MeetupEdward Curry
Data management efforts such as Master Data Management and Data Curation are a popular approach for high quality enterprise data. However, Data Curation can be heavily centralised and labour intensive, where the cost and effort can become prohibitively high. The concentration of data management and stewardship onto a few highly skilled individuals, like developers and data experts, can be a significant bottleneck. This talk explores how to effectively involving a wider community of users within big data management activities. The bottom-up approach of involving crowds in the creation and management of data has been demonstrated by projects like Freebase, Wikipedia, and DBpedia. The talk discusses how crowdsourcing data management techniques can be applied within an enterprise context.
Topics covered include:
- Data Quality And Data Curation
- Crowdsourcing
- Case Studies on Crowdsourced Data Curation
- Setting up a Crowdsourced Data Curation Process
- Linked Open Data Example
- Future Research Challenges
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la IglesiaMaria de la Iglesia
Según Hal Varian (experto en microeconomía y economía de la información y, desde el año 2002, Chief Economist de Google) “En los próximos años, el trabajo más atractivo será el de los estadísticos: La capacidad de recoger datos, comprenderlos, procesarlos, extraer su valor, visualizarlos, comunicarlos serán todas habilidades importantes en las próximas décadas. Ahora disponemos de datos gratuitos y omnipresentes. Lo que aún falta es la capacidad de comprender estos datos“.
Got Chaos? Extracting Business Intelligence from Email with Natural Language ...Digital Reasoning
In this presentation, O'Reilly author and Digital Reasoning CTO Matthew Russell along with Dr. Steve Kramer, founder and chief scientist at Paragon Science, discuss how Digital Reasoning processed the Enron corpus with its advanced Natural Language Processing (NLP) technology - effectively transforming it into building blocks that are viable for data science. Then, Paragon Science used dynamic graph analysis inspired from particle physics to tease out insights from the data in order to better understand whether an enterprise fiasco such as the Enron scandal could have been thwarted.
From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...Edward Curry
The Real-time Linked Dataspace (RLD) is an enabling platform for data management for intelligent systems within smart environments that combines the pay-as-you-go paradigm of dataspaces, linked data, and knowledge graphs with entity-centric real-time query capabilities.
The RLD contains all the relevant information within a data ecosystem including things, sensors, and data sources and has the responsibility for managing the relationships among these participants.
It manages sources without presuming a pre-existing semantic integration among them using specialised dataspace support services for loose administrative proximity and semantic integration for event and stream systems. Support services leverage approximate and best-effort techniques and operate under a 5 star model for “pay-as-you-go” incremental data management.
Mining the Social Web for Fun & Profit Within Your OrganizationDigital Reasoning
In this talk, Matthew Russell explores why it is imperative for organizations and companies to leverage social media and how they can do it. In today's world of massive, rapidly evolving data streams, it is very challenging to sift through the data and extract the hidden nuggets of critical business intelligence. With advances in machine learning and natural language processing, decision makers can now look at all of their data and see what's really important. Matthew presents examples of how companies like Digital Reasoning are using social media to answer questions like:
Who know whom, and what friends do they have in common?
How frequently are certain people communicating with one another?
Who are the quietest/chattiest people in a network?
Who are the most influential/popular people in a network?
What are people chatting about (and is it interesting)?
Christopher Gutteridge's slides form Connected Data London. Christopher, who is an Open Data Architect at the Univeristy of Southhampton presented why and how people should employ an Open Data strategy at their organisation.
"From Big Data to Smart data"
Jie (Jack) Yang, Associate Research Fellow, SMART Infrastructure Facility, presented a summary of his research as part of the SMART Seminar Series on 28 April 2016.
For more information, visit the event page at: http://smart.uow.edu.au/events/UOW212890.html.
Currently, the Yahoo EC Taiwan team provides business performance matrix to users by acquiring data from the Web production and Back office ERP systems. The reporting system is built using traditional BI technologies such as RDBMS, ETL tools, OLAP tools, home-made reporting tools, store procedures, web pages,?. With increasing usage growth of the user browsing data in the business decision on daily basis, The ability to provided data analytics on these Big Data is getting more and more important and needed. The traditional RDBMs have reaching its limit in process big data while connecting to OLAP tool. We started with the feasibility of connecting MicroStrategy with Hive 0.9 and created a prototype system to test in two scenarios – ad-hoc query to Hive and performance test of the predefined MicroStrategy Intelligent Cube for ad-hoc analytics. We did the performance test on Ad-hoc query via HiveQL and query from MicroStrategy cube, and will share the result in the session. Based on our test results, we will be able to provide the following applications to different types of users. A) Ad-hoc query running against Hadoop can allow well trained data analyst or power users to have deeper analysis on data within Hadoop. B) OLAP reports running against MicroStrategy Intelligent Cube can provide quicker response time on ad-hoc analytics with predefined data in Cube.
Tim Estes - Information Systems in an Entity Centric WorldDigital Reasoning
Tim Estes, CEO of Digital Reasoning, talks about the use of Hadoop and other scalable technologies along with Digital Reasoning's analytics for automated understanding of cloud-scale text challenges.
This presentation was delivered at Hadoop World in New York in Oct 2010
Crowdsourcing Approaches to Big Data Curation - Rio Big Data MeetupEdward Curry
Data management efforts such as Master Data Management and Data Curation are a popular approach for high quality enterprise data. However, Data Curation can be heavily centralised and labour intensive, where the cost and effort can become prohibitively high. The concentration of data management and stewardship onto a few highly skilled individuals, like developers and data experts, can be a significant bottleneck. This talk explores how to effectively involving a wider community of users within big data management activities. The bottom-up approach of involving crowds in the creation and management of data has been demonstrated by projects like Freebase, Wikipedia, and DBpedia. The talk discusses how crowdsourcing data management techniques can be applied within an enterprise context.
Topics covered include:
- Data Quality And Data Curation
- Crowdsourcing
- Case Studies on Crowdsourced Data Curation
- Setting up a Crowdsourced Data Curation Process
- Linked Open Data Example
- Future Research Challenges
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la IglesiaMaria de la Iglesia
Según Hal Varian (experto en microeconomía y economía de la información y, desde el año 2002, Chief Economist de Google) “En los próximos años, el trabajo más atractivo será el de los estadísticos: La capacidad de recoger datos, comprenderlos, procesarlos, extraer su valor, visualizarlos, comunicarlos serán todas habilidades importantes en las próximas décadas. Ahora disponemos de datos gratuitos y omnipresentes. Lo que aún falta es la capacidad de comprender estos datos“.
Got Chaos? Extracting Business Intelligence from Email with Natural Language ...Digital Reasoning
In this presentation, O'Reilly author and Digital Reasoning CTO Matthew Russell along with Dr. Steve Kramer, founder and chief scientist at Paragon Science, discuss how Digital Reasoning processed the Enron corpus with its advanced Natural Language Processing (NLP) technology - effectively transforming it into building blocks that are viable for data science. Then, Paragon Science used dynamic graph analysis inspired from particle physics to tease out insights from the data in order to better understand whether an enterprise fiasco such as the Enron scandal could have been thwarted.
From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...Edward Curry
The Real-time Linked Dataspace (RLD) is an enabling platform for data management for intelligent systems within smart environments that combines the pay-as-you-go paradigm of dataspaces, linked data, and knowledge graphs with entity-centric real-time query capabilities.
The RLD contains all the relevant information within a data ecosystem including things, sensors, and data sources and has the responsibility for managing the relationships among these participants.
It manages sources without presuming a pre-existing semantic integration among them using specialised dataspace support services for loose administrative proximity and semantic integration for event and stream systems. Support services leverage approximate and best-effort techniques and operate under a 5 star model for “pay-as-you-go” incremental data management.
Mining the Social Web for Fun & Profit Within Your OrganizationDigital Reasoning
In this talk, Matthew Russell explores why it is imperative for organizations and companies to leverage social media and how they can do it. In today's world of massive, rapidly evolving data streams, it is very challenging to sift through the data and extract the hidden nuggets of critical business intelligence. With advances in machine learning and natural language processing, decision makers can now look at all of their data and see what's really important. Matthew presents examples of how companies like Digital Reasoning are using social media to answer questions like:
Who know whom, and what friends do they have in common?
How frequently are certain people communicating with one another?
Who are the quietest/chattiest people in a network?
Who are the most influential/popular people in a network?
What are people chatting about (and is it interesting)?
Christopher Gutteridge's slides form Connected Data London. Christopher, who is an Open Data Architect at the Univeristy of Southhampton presented why and how people should employ an Open Data strategy at their organisation.
"From Big Data to Smart data"
Jie (Jack) Yang, Associate Research Fellow, SMART Infrastructure Facility, presented a summary of his research as part of the SMART Seminar Series on 28 April 2016.
For more information, visit the event page at: http://smart.uow.edu.au/events/UOW212890.html.
Currently, the Yahoo EC Taiwan team provides business performance matrix to users by acquiring data from the Web production and Back office ERP systems. The reporting system is built using traditional BI technologies such as RDBMS, ETL tools, OLAP tools, home-made reporting tools, store procedures, web pages,?. With increasing usage growth of the user browsing data in the business decision on daily basis, The ability to provided data analytics on these Big Data is getting more and more important and needed. The traditional RDBMs have reaching its limit in process big data while connecting to OLAP tool. We started with the feasibility of connecting MicroStrategy with Hive 0.9 and created a prototype system to test in two scenarios – ad-hoc query to Hive and performance test of the predefined MicroStrategy Intelligent Cube for ad-hoc analytics. We did the performance test on Ad-hoc query via HiveQL and query from MicroStrategy cube, and will share the result in the session. Based on our test results, we will be able to provide the following applications to different types of users. A) Ad-hoc query running against Hadoop can allow well trained data analyst or power users to have deeper analysis on data within Hadoop. B) OLAP reports running against MicroStrategy Intelligent Cube can provide quicker response time on ad-hoc analytics with predefined data in Cube.
Public and private organizations in all sectors are using their data to give them insight about their companies, as well as a competitive advantage. This session explores some of the key areas that organizations need to be considering in developing a Big Data management strategy: 1) Why are we collecting Big Data? 2) How can we mine our Big Data; 3) What measures are needed to govern Big Data? ; 4) How do we manage sensitive information and ensure compliance with relevant legislation?; and 5) How do we manage the balance the value, risks, and costs of Big Data?
Panel SXSW : The Internet of Me : Impact of biometrics and machine learning o...Pierre-Majorique Léger
Abstract : The explosion of consumer based sensors and machine learning is at the core of advancements in biometrics and neurophysiology, and never has so much data been available. Dedicated technologies and applications have started to disrupt the way we understand ourselves, our health and the way we interact with each other and Things. The Canadian Space Agency, and CAE Inc. will be discussing how these developments are impacting their technology roadmaps and vision, and how tech centers like the Tech3Lab qualify emotion and cognitive responses to allow for a better understanding of users health and performance. Mar 13, 2017 | 10:00am – 10:45am
Here are some tips on hiring and retaining top Big Data talent. Features : how to source candidates, how to interview them, interview techniques and mistakes.
Listen to video of presentation and download slides here : http://elephantscale.com/2017/03/building-successful-big-data-team-demand-webinar/
PAARL's 1st Marina G. Dayrit Lecture Series held at UP's Melchor Hall, 5F, Proctor & Gamble Audiovisual Hall, College of Engineering, on 3 March 2017, with Albert Anthony D. Gavino of Smart Communications Inc. as resource speaker on the topic "Using Big Data to Enhance Library Services"
an introductory course for Librarians on using Big Data and Data Science applications on the field of Library Science. The course is a 2 hour course module for basic fundamentals of applying DS work.
Have you ever wondered how search works while visiting an e-commerce site, internal website, or searching through other types of online resources? Look no further than this informative session on the ways that taxonomies help end-users navigate the internet! Hear from taxonomists and other information professionals who have first-hand experience creating and working with taxonomies that aid in navigation, search, and discovery across a range of disciplines.
Making AI Behave: Using Knowledge Domains to Produce Useful, Trustworthy ResultsAccess Innovations, Inc.
In today's highly charged atmosphere of anxiety and anticipation about AI, and especially LLMs,
one of the biggest concerns is how to ensure that it returns accurate results (meaning both true
and pertinent to its audience). This is particularly important to scholarly, scientific, and other
technical organizations, whose constituents are often in very specific domains, such as
medicine, engineering, history, biology, chemistry, etc. One extremely useful tool to incorporate in an AI-based process in such cases is a comprehensive and well-structured knowledge domain which is based on a controlled vocabulary.
Smart Submit and Client Support
Michael Millar, Junior Software Developer, and Frank Coates, Client Support Manager
Get a peek at the new and improved Smart Submit and learn about new, easier ways to contact the support team at Access Innovations.
How a Good Taxonomy Can Provide Valuable Business Insights
Kristen Monahan, Public Library of Science (PLOS)
Kristen is a business analyst and she won’t be talking about the PLOS taxonomy but rather how she uses that taxonomy to drill down into the massive amount of content, metadata, and usage and process data that is PLOS for deep, detailed analysis and to drive business decisions. Much of this work involves trend analysis. For example, trend analysis of submissions can look at the time it takes from submission to decision by subject (narrow subjects like Covid, broad subjects like biotechnology), or by institution, or by country, etc. to see not just the overall big picture but where in their submission and peer review workflows the bottlenecks might be. A trend analysis of topics over time can prompt them to issue a call for papers for a topic they think needs to be better covered–and then look at both short-term and long-term trends resulting from that call to papers. Their taxonomy doesn’t just make their content smarter–it makes how they publish that content smarter too.
Editor and Peer Reviewer Assignments Using Data Harmony
Andrew Smeall, Hindawi Publishing
Andrew will show how Hindawi, an open access publisher, applies their taxonomy to make editor and reviewer assignments for incoming submissions to their journals.
Cloud Deployment of Data Harmony
Jeffrey Gordon, Lead Developer, Access Innovations, Inc.
Jeffrey will describe the cloud deployment of the Data Harmony software.
Marjorie M. K. Hlava, President, Chair of the Board, and Chief Scientist, Access Innovations, Inc.
During this annual highlight of the DHUG meetings, Margie will discuss the exciting new changes and additions to the Data Harmony software. She will be joined by some members of our software development team to talk about specific initiatives we have worked on over the past year.
Access Innovations and Atypon: Beyond Content Tagging
Hong Zhou and Gerasimos Razis, Atypon
Gerasimos and Hong will discuss the changes to the Atypon platform since DHUG 2020.
Getting to the Point: Using AI and Taxonomies to Craft Meta -Titles
Travis Hicks, American Society of Clinical Oncology (ASCO)
Looking to better leverage SEO and include key terms in the url construct for research abstracts, ASCO is working with Access Innovations to evaluate how to programmatically create short titles for abstracts. The idea is to index titles against existing taxonomies as a way of producing a short title that succinctly identified what an abstract is about for purposes of constructing a new url configuration. Travis will discuss the need, challenges, and early results of the project.
Expanding the Use of MAIstro at ASCE
Xi Van Fleet, American Society for Civil Engineers
Using MAIstro, ASCE created the subject/topic taxonomies for their publications to enhance content discovery and business insight. After achieving their primary goal, they have been expanding its use for other applications.
Lessons Learned From Building a Taxonomy and Indexing 140 Years of Content
Michael Darr, Project Manager, D33 – American Chemical Society Pubs IT
Michael will talk about the things they would do differently if they were to build a new taxonomy and index a legacy file, and the things they did right the first time.
Bill’s talk is entitled “WHAT’S IN A NAME? How Kew helps drug regulators disambiguate the messy welter of medicinal plant names to shore up regulation and save lives”. It’s really eye-opening to realize how complicated and imprecise names can get, with multiple scientific, pharmaceutical and popular names for the same thing or with one name used for completely different things.
This has real-world consequences. For example, the EU mistakenly banned a useful plant we use every day when intending to ban a poisonous one because of a naming problem. How Kew is using semantic and taxonomic tools and technologies to bring order to this complexity (I almost said chaos) is really fascinating. They’re also helping to disambiguate nomenclature and provide links to authoritative information for botanical terms for use in journal articles, among other things.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
4. AccessInnovations,Inc..12/11/2013
Why do we care about Big Data?
• Data is the new oil – we have to learn how to
mine it! Qatar – European Commission Report
• $ 7 trillion economic value in 7 US sectors alone
• $90 B annually in sensitive devices
• An insurance firm with 5 terabytes of data on
share drives pays $1.5 m per year
• New McKinsey 4th factor of production
• Land, Labor, Capital, + Data
6. AccessInnovations,Inc..12/11/2013
Big Data Born
• Google, eBay, LinkedIn, and Facebook were built
around Big Data from the beginning.
• No need to reconcile or integrate Big Data with
more traditional sources of data and the
analytics performed upon them
• No merging Big Data technologies with their
traditional IT infrastructures
• Big Data could stand alone, Big Data analytics
could be the only focus of analytics
• Big Data technology architectures could be the
only architecture.
7. AccessInnovations,Inc..12/11/2013
Integrating Big Data
• Large, well-established orgs
• Must be integrated with everything else that’s
going on in the company.
• Analytics on Big Data have to coexist with
analytics on other types of data.
• Hadoop clusters have to do their work alongside
IBM mainframes.
• Data scientists must somehow get along and
work jointly with mere quantitative analysts.
8. AccessInnovations,Inc..12/11/2013
What is Big Data?
• Big Data is a term applied to data sets whose size
is beyond the ability of commonly used software
tools to capture, manage, and process the data
within a tolerable elapsed time. Big Data sizes
are a constantly moving target currently ranging
from a few dozen terabytes to many petabytes of
data in a single data set. – Wikipedia, May 2011
• “Unstructured”
• Terabytes, petabytes, zettabytes
• Streaming
10. AccessInnovations,Inc..12/11/2013
New Special Collections
• Volume, Velocity, Variety
• Ability to deal overwhelmed
• More about methods than data
• Location aware data
• Life streaming
• Insurance claims
• Hubble telescope
• CERN Collections
• Flight data
13. AccessInnovations,Inc..12/11/2013
Bit of a misnomer
• All data has some structure and more structure
possibilities
• PDF properties
• Word file properties
• File structures
14. AccessInnovations,Inc..12/11/2013
Property tables
• PDF - property tables
• Word files – property tables
• Files shares – implied structure in file names
• News feeds – headers have metadata
• Hubbell telescope feeds
• Images
18. AccessInnovations,Inc..12/11/2013
What are the problems?
• Data infrastructure challenges
• “taking diverse and heterogeneous data sets and
making them more homogeneous and usable”
• An opportunity?
• All that data – what can it tell us?
• Privacy
• Copyright
• Neurological impact
• Data collection methods
19. AccessInnovations,Inc..12/11/2013
New Government Initiative
The Big Data Senior Steering Group (BDSSG) was
formed to identify current Big Data research and
development activities across the Federal
government, offer opportunities for coordination,
and identify what the goal of a national initiative
in this area would look like. Subsequently, in
March 2012, The White House Big Data R&D
Initiative was launched and the BDSSG continues
to work in four main areas to facilitate and further
the goals of the Initiative.
20. AccessInnovations,Inc..12/11/2013
The National Big Data R&D Initiative
• Fast-growing volume of digital data of digital
data
• Advance state-of-the-art core technologies
needed to collect, store, preserve, manage,
analyze, and share huge quantities of data.
• Harness these technologies to accelerate the
pace of discovery in science and engineering,
strengthen our national security, and transform
teaching and learning; and
• Expand the workforce needed to develop and
use Big Data technologies.
21. AccessInnovations,Inc..12/11/2013
Data to Knowledge to Action
•Advance supporting technologies
•Big Data
•Data analytics;
•Educate and expand the Big Data workforce
•Improve key outcomes in economic growth,
job creation, education, health, energy,
sustainability, public safety, advanced
manufacturing, science and engineering, and
global development
22. AccessInnovations,Inc..12/11/2013
Data on Data
• January 22, 2013 - Data on Data: Presenting
Stakeholder Alignment Data on the
Cyberinfrastructure for Earth System Science
• Presentation and discussion with Professor Joel
Cutcher-Gershenfeld. Professor Cutcher-
Gershenfeld presented information on the NSF
EarthCube initiative including stakeholder survey
data (approximately 850 responses).
24. AccessInnovations,Inc..12/11/2013
Groups breakdown
• INTERAGENCY WORKING GROUPS
• Cyber Security and Information Assurance
• High End Computing
• COMMUNITY OF PRACTICE (CoP)
• · Faster Administration of Science and Technology
Education and Research
• COORDINATING GROUPS
• Human Computer Interaction and Information
Management
• High Confidence Software and Systems
• Large Scale Networking
• Software Design and Productivity
• Social, Economic, and Workforce Implications of IT
25. AccessInnovations,Inc..12/11/2013
SENIOR STEERING GROUPS (SSGs)
• Big Data
• Cyber Physical Systems
• Cyber Security and Information Assurance Research and
Development
• Health Information Technology Research and
Development
• Wireless Spectrum Research and Development
• SUBGROUP
• Health Information Technology Innovation and
Development Environments Subgroup
• TEAMS
• Joint Engineering Team
• Middleware and Grid Interagency Coordinating Team
28. AccessInnovations,Inc..12/11/2013
Data feeds
Web portal
Locally held
documents
Public
repositories
Commercial
data sources
Agency data
sources
INTERNET
(public)
spiders
Meta-Search Tool
Filtered
content
Search engineSearch engine
Search engine
Search engineSearch engine
TAXONOMY
Library
catalogs
Search engine
29. AccessInnovations,Inc..12/11/2013
Metadata options
• Structure unstructured data
• Create metadata
• Where to put it?
• Store metadata with the records
• HTML header
• Properties tables
• XML files
• Store metadata in a separate file
• Database
• Metadata repository
• Search system
• File structure
• SharePoint application
• Web interface
30. AccessInnovations,Inc..12/11/2013
Information retrieval starts with a
knowledge organization system
• Uncontrolled list
• Name authority file
• Synonym set / ring
• Controlled vocabulary
• Taxonomy
• Thesaurus
• Ontology
• Topic Map
• Semantic Network
Not complex - $
Highly complex - $$$$
LOTS OF OVERLAP!
31. AccessInnovations,Inc..12/11/2013
Structure of
controlled vocabularies
List of words Synonyms Taxonomy Thesaurus
INCREASING COMPLEXITY / RICHNESS
Ambiguity control Ambiguity control Ambiguity cont’l
Synonym control Synonym control Synonym cont’l
Hierarchical rel’s Hierarchical rel’s
Associative rel’s
32. AccessInnovations,Inc..12/11/2013
Taxonomy / thesaurus
• Main Term (MT) *
• Top Term (TT)
• Broader Terms (BT)
• Narrower Terms (NT)
• Narrower Term Instance
• Related Terms (RT)
• See also (SA)
• Non-Preferred Term (NP)
• Used for (UF), See (S)
• Scope Note (SN)
• History (H)
TAXONOMY
THESAURUS
ONTOLOGY
*a.k.a. subject term, heading,
node, index term, keyword,
category, descriptor, class
34. AccessInnovations,Inc..12/11/2013
Taxonomies in Context
A taxonomy aspires to be:
• a correlation of the different functional, regional
and (possibly) national languages used by a
community of practice
• a support mechanism for navigation
• a support tool for search engines and knowledge
maps
• an authority for tagging documents and other
information objects
• a knowledge base in its own right
Reference: “Taxonomies: the vital tool of
information architecture”, www.tfpl.com.
Courtesy of Lillian Gassie, Naval
Postgraduate School, Monterey, CA
35. AccessInnovations,Inc..12/11/2013
Where to use a taxonomy
• Link the taxonomy and indexing
• Always in sync with the industry
• Keep up to date with terminology
• Automatically index the old data
• Filter newsfeeds
• Search using the taxonomy
• File using the taxonomy
• Spell check using the taxonomy
• Link to translation system
• Catalog using the taxonomy
36. AccessInnovations,Inc..12/11/2013
Value of a Taxonomy
• Improves organization & structure
• Facilitates navigation
• Facilitates knowledge discovery
• Reduces effort
• Saves time
“Taxonomies are better created by
professional indexers or librarians
than by domain experts.”
Courtesy of Lillian Gassie, Naval
Postgraduate School, Monterey,
CA
37. AccessInnovations,Inc..12/11/2013
Taxonomies
• A well formed taxonomy is based on a thesaurus
• Provides a flexible platform for many views of
the taxonomy
• Allows fast deployment
• Is the basis for a good
• knowledge management system
• search retrieval system
• portal interface
38. AccessInnovations,Inc..12/11/2013
A quick look behind the scenes
Database
Management
System
Thesaurus
tool
Indexing
toolValidate terms
Add terms and rules
Change terms and rules
Delete terms and rules
Search thesaurus
Validate term entry
Block invalid terms
Record candidates
Establish rules for
term use
Suggest indexing
terms
41. AccessInnovations,Inc..12/11/2013
Workflow order
• Create the metadata structure
• Gather the locations of the records
• Index in place?
• Point to the data?
• Add metadata to the record?
• Store metadata in separate “table”?
• Use in full text?
• Use in search?
• Link databases and data sets with APIs
42. AccessInnovations,Inc..12/11/2013
APIs and Web Calls
• Link data cross platforms
• Not federated search
• Examples
• Was: Card catalog or OPACs to books on shelf
• Now: EBSCO host to Mendeley, Zotero, ResearchGate,
Academic.edu, etc.
• Use an API or web call
• Web calls
• Call to another web platform
• APIs, written handshakes
• Z39.50 is one standard for libraries
46. AccessInnovations,Inc..12/11/2013
QUERYAPI
CUSTOM
CONNECTOR
LOTUS NOTES
CONNECTOR
FAST ESP and Data Harmony
Architecture
Core Architectural Components
Pipeline
SEARCH
SERVER
QUERY
PROCESSOR
Query
Results
Vertical
Applications
Portals
Custom
Front-Ends
Mobile
DevicesContent
Push
DOCUMENT
PROCESSOR
Web
Content
Files,
Documents
Databases
Custom
Applications
CONTENTAPI
MANAGEMENT API
Index DB
DATABASE
CONNECTOR
FILE
TRAVERSER
WEB
CRAWLER
Pipeline
Email,
Groupware
Administrator’s
Dashboard
FILTER
SERVER
Agent DB
Alerts
Data Harmony Governance API
MAIstro
NavTree&MAIQuery
50. AccessInnovations,Inc..12/11/2013
Go – No Go
• Reach 85% precision to launch for productivity -
assisted
• Reach 85% for filtering or categorization
• Sorting for production
• Level of effort to get to 85%
• Integration into the workflow is efficient
51. AccessInnovations,Inc..12/11/2013
Learn to understand the parts
• Information infrastructure
• Information dynamics
• Tensions on data
• Design elements
• How and when to set the data framework
• Identifiers (taxonomies) Glue to hold data
together
53. AccessInnovations,Inc..12/11/2013
Benchmarks
• 15 – 20% irrelevant returns / noise
• Amount of work needed to achieve 85% level
• How good is good enough?
• Satisfice = satisfaction + suffice
• How good is good enough?
• How much error can you put up with?
• Hits, misses, noise
54. AccessInnovations,Inc..12/11/2013
ROI Calculations
• Assume – 5000 term thesaurus
• 1.5 synonyms per terms
• 7500 terms total
• Assume 85% accuracy
• Use assisted for indexing
• Use automatically for filtering
• Assume $55 per hour for staff
• Assume 10000 records for test batch
55. AccessInnovations,Inc..12/11/2013
Co-occurrence - Training sets
• Collect 20 items per thesaurus entry
• Preferred and non-preferred terms
• Find records – could be programmatic
• Need to collect and review about 60 to get 20 appropriate ones
• Review records – ensure they are correct
• 3 minutes average per final selected record
• = 1 hour per term entry
• 1 minute to review a record (20 resulting records)
• 7500 terms at one hour per term = 7500 hours
• Estimated at 7500 hours $55 / hour = $412,500
56. AccessInnovations,Inc..12/11/2013
Rulebuilding - rules
• 80% of rules built automatically
• 7500 x .8 = 6000
• 20% require complex rules
• Average rule takes 5 minutes
• (Actually MUCH faster using a rule building tool)
• 5 x 1500 = 7500 minutes
• 125 hours x $55 = $6875
57. AccessInnovations,Inc..12/11/2013
ROI - Segments
• Cost of auto system
• Cost of getting system ready
• Ongoing maintenance
• Increased efficiency
• Increased quality of retrieval
• Cost of legacy system maintenance
58. AccessInnovations,Inc..12/11/2013
ROI – Co-occurrence
• Cost of auto system- $500,000 (could be less, but the one used for this study
cost this amount)
• Cost of getting system ready
• Programming support and integration
• Estimated at 2 weeks programming $100 / hour = $8000
• Training sets
• Estimated at 7500 hours $55 / hour = $412,500
• Possible need to re-run training set several times
• Ongoing maintenance
• Estimated at 15% of purchase price for license = $75,000
• Quarterly retraining to keep up with new terms –
• Training sets for new terms 50 terms per quarter = 200 x $55=$11,000
• Increased efficiency
• Expected accuracy at 60%
• Increased quality of retrieval
• Cost of legacy system maintenance
59. AccessInnovations,Inc..12/11/2013
ROI – Rules system
• Cost of auto system- $60,000
• Cost of getting system ready
• Programming support and integration
• Estimated at 2 weeks programming at $100 / hour = $8000
• Rule building
• Estimated at 125 hours at $55 / hour = $6850
• Possible need to re run training set several times
• Ongoing maintenance
• Estimated at 15% of purchase price for license = $9000
• Rule building for new terms, 50 terms per quarter
• 200 terms x .8 = 160 automatic
• 40 at 5 minutes per term = 200 minutes /60 = 3.33 hours x $55 = $183
• Increased efficiency
• Expected accuracy at 60%
• Increased quality of retrieval
• Cost of legacy system maintenance
62. AccessInnovations,Inc..12/11/2013
Summary on ROI
• Novelty detection requires inference techniques
• Is needed to discover new terms
• Controlled vocabulary / thesaurus can be more
controlled and accurate approach
• Combination of the two would be best
• Controlled – use rules
• New terms - use inference
63. AccessInnovations,Inc..12/11/2013
Thesaurus Master
Machine Aided
Indexer (M.A.I.™)
Database
Repository
Search
Presentation
Layer
Increases
accuracy
Browse by Subject
Auto-completion
Broader Terms
Narrower Terms
Related Terms
Client Taxonomy
Inline Tagging
Metadata and
Entity Extractor
Automatic
Summarization
Search
Software
Client Data
Full Text
HTML, PDF,
Data Feeds, etc.
Client
taxonomy
Fully integrated with MOSS
The Workflow
Tag and
Create
metadata
Put in
data base
with tags
Build
Search
inverted index
Create
user
interface
Gather
source
data
65. AccessInnovations,Inc..12/11/2013
Semantic Process
Full text, HTML,
PDF, data feeds
Apply terms
Rules Base
User Interface
Web Portal
Client
Taxonomy
Data
Repository
Inline Tagging
Search
Software
Metadata
Extractor
Thesaurus
Master
Automatic
Summarization
66. AccessInnovations,Inc..12/11/2013
Database Plus Search Workflow
XIS
repository
Add
metadata
Taxonomy
Thesaurus
Master
MAI Rule
Base
MAI Concept
Extractor
Taxonomy
terms
Raw Full
text data
feeds
XIS Creation
Printed
source
materials
Load to
Search
Search
Harmony
Display
Search
Data
Crawls on
data
sources
SQL for
ecommerce
Source data
Clean and enhance data
Search data
75. AccessInnovations,Inc..12/11/2013
Data linked and served by metadata
Job Posting
for Expert
on Topic A
Author Networks
Social Networking
Journal
Article on
Topic A
Other Journal
Articles on
Topic A
Upcoming
Conference on
Topic A
Podcast Interview
with Researcher
Working on Topic A
Grant Available for
Researchers Working
on Topic A
CME Activity
on Topic A
Based on an image by Helen Atkins
83. AccessInnovations,Inc..12/11/2013
Data Mashups
• Drawing together information from several
sources
• Overlay on additional surfaces – like maps
• Fun distribution of data
• Cornell School of Ornithology
• Migration patterns
• YouTube
• Citizen Science
84. AccessInnovations,Inc..12/11/2013
Scientific social networking
based on metadata• Idea has been here - Who is citing who like
• ISI does it with references
• AIP’s UniPHY does it using semantics
• Expand your options using
• good metadata and descriptors
Map who is working in the
field and where
See the authors connections
86. AccessInnovations,Inc..12/11/2013
Future?
• Document systems replaced
• New infrastructure
• Institutional repositories not good long term for Big
Data
• Need to operate at scale
• Integrated, ecosystem to infrastructure
• Replace customized human mediated
• With interpretive layer – computer assisted
• Links to data
• Open data / supplemental data
• Too much is not enough!
87. AccessInnovations,Inc..12/11/2013
Differentiation
• Not by size of collection
• By the services they offer
• More services more competitive
• Spreadsheet science
• Go down hall and ask
• Big Data
• Too many people to ask
• Lose provenance
• Hard to differentiate the layers
88. AccessInnovations,Inc..12/11/2013
New ways of doing things
• People, data usage
• Preservation and discovery
• Publishers’ content, not format
• Visualization of Big Data sets
• Reproducibility of research
• Shared knowledge
• Open peer review
• Researcher.org
• Galaxy zoo
• Zooniverse
• 900,000 citizen scientists
• Wikipedia
90. AccessInnovations,Inc..12/11/2013
Skills we bring
• Librarians
• Weeding
• Redefining collections
• Collection development
• Skills to apply to Big Data
• Vocabulary development
• Search expertise
• Reference ability
• What to throw away
• What to keep for discoverability
• Need metadata to preserve
• Better discoverability
• Easier preservation
• Keep the data provenance
91. AccessInnovations,Inc..12/11/2013
We covered
• Big Data – What is it?
• New Government Initiative
• Content organization
• Discovery (Search)
• Management
• Skills we bring
• Examples of what we can do
92. AccessInnovations,Inc..12/11/2013
Who are we?
• Access Innovations –
• Semantic Enrichment Services Provider
• We change search to find!
• Creator of Data Harmony tools
• More than 600 taxonomies created
• More than 2000 engagements
• Financed by Sweat, Persistence, and Good Cash Flow
Management
• Accurate, on time, under budget!
Thanks to Helen Atkins of AACR for this illustration.The real power of this is that the links can all go in all directions, so we take advantage of having the user’s attention regardless of how they step into our “web”