The recent increase in the volume and variety of video content available online presents growing challenges for video search. Users face increased difficulty in formulating effective queries and search engines must deploy highly effective algorithms to provide relevant results. This talk addresses these challenges by introducing two novel frameworks and approaches. First, we discuss a principled framework for multimedia retrieval that moves beyond 'what' users are searching for also to encompass 'why' they search. This 'why' is understood as the reason, purpose or immediate goal behind a user information need, which is identified as the underlying 'user intent'. We identify useful intent categories for online video search, present validation experiments showing that these categories display enough invariance to be successfully modeled by a video search engine and demonstrate the potential for these categories to improve video retrieval with a large crowdsourcing user study. Second, we present a novel approach able to predict for which queries results optimization is most useful, i.e., predicting which queries will fail in the search session of a user on a video search engine. Being able to predict when a video search query would fail is likely to make the video search result optimization more efficient and deploy optimization techniques more effectively. This approach uses a combination of features derived from the search log of a video search engine (capturing users' behavior) and features derived from the video search results list (capturing the visual variance of search results), with the objective to predict whether a particular query is likely to fail in the context of a particular search session.
Presentation given by Chris Welty (IBM Research) at Knoesis. We get the permission to upload this presentation from Chris Welty. Event details are at: http://j.mp/Welty-at-Knoesis and the associate video is at: https://www.youtube.com/watch?v=grDKpicM5y0
An award-winning Hawaii-based engineering firm introduced MERCI, a mobile emergency response and command interface system. MERCI allows emergency managers to rapidly assess damage following disasters by collecting geospatial data, photos, and videos using mobile devices. This provides situational awareness and supports recovery efforts. The system was developed with Hawaii Civil Defense and follows FEMA standards. It has been tested in exercises and can integrate with other platforms. MERCI streamlines damage assessments and provides cost savings compared to traditional paper-based methods.
The document describes a framework called CUbRIK that uses human computation to improve multimedia search. It presents a case study on using the crowd to detect trademark logos in videos. Workers validate automatically detected logos and add new logos. The system matches tasks to workers based on their skills. An evaluation compares the logo detection accuracy of automatic methods, experts, and the crowd. While the crowd recall is higher, its precision is lower due to the workers' varied locations and expertise.
This document analyzes why the artist "Riley" would be suited to signing with the record label Aftermath Entertainment. It notes that Riley's style, including his clothing, hairstyle, and the themes and settings in his music videos, appeal to a wide target audience in a similar way to popular Aftermath artists like Kendrick Lamar and Eminem. Riley's music and image allow him to connect with audiences across social classes and demographics. The document also outlines details of Riley's upcoming music video for "Malcolm Middle" and how its conceptual nature and themes are aligned with Aftermath's style.
The document provides an analysis of the representation of people in a magazine through the use of various visual techniques and connotations. It examines how the lighting, typography, setting, facial expressions, colors, layout, fonts and other elements are used to portray themes of masculinity, urban lifestyle, hip hop culture, wealth, and the ego and ambition of the featured artist. The analysis focuses on how these visual features convey implied meanings and interpretations beyond the literal.
This shot list contains 80 shots for a music video. The shots include a variety of angles, compositions, and actions such as close-ups of Riley's hands and face, wide shots of Riley rapping alone and with others, tracking shots of Riley walking and rapping in tunnels, and long shots with slow zooms of locations marked with an X.
Presentation given by Chris Welty (IBM Research) at Knoesis. We get the permission to upload this presentation from Chris Welty. Event details are at: http://j.mp/Welty-at-Knoesis and the associate video is at: https://www.youtube.com/watch?v=grDKpicM5y0
An award-winning Hawaii-based engineering firm introduced MERCI, a mobile emergency response and command interface system. MERCI allows emergency managers to rapidly assess damage following disasters by collecting geospatial data, photos, and videos using mobile devices. This provides situational awareness and supports recovery efforts. The system was developed with Hawaii Civil Defense and follows FEMA standards. It has been tested in exercises and can integrate with other platforms. MERCI streamlines damage assessments and provides cost savings compared to traditional paper-based methods.
The document describes a framework called CUbRIK that uses human computation to improve multimedia search. It presents a case study on using the crowd to detect trademark logos in videos. Workers validate automatically detected logos and add new logos. The system matches tasks to workers based on their skills. An evaluation compares the logo detection accuracy of automatic methods, experts, and the crowd. While the crowd recall is higher, its precision is lower due to the workers' varied locations and expertise.
This document analyzes why the artist "Riley" would be suited to signing with the record label Aftermath Entertainment. It notes that Riley's style, including his clothing, hairstyle, and the themes and settings in his music videos, appeal to a wide target audience in a similar way to popular Aftermath artists like Kendrick Lamar and Eminem. Riley's music and image allow him to connect with audiences across social classes and demographics. The document also outlines details of Riley's upcoming music video for "Malcolm Middle" and how its conceptual nature and themes are aligned with Aftermath's style.
The document provides an analysis of the representation of people in a magazine through the use of various visual techniques and connotations. It examines how the lighting, typography, setting, facial expressions, colors, layout, fonts and other elements are used to portray themes of masculinity, urban lifestyle, hip hop culture, wealth, and the ego and ambition of the featured artist. The analysis focuses on how these visual features convey implied meanings and interpretations beyond the literal.
This shot list contains 80 shots for a music video. The shots include a variety of angles, compositions, and actions such as close-ups of Riley's hands and face, wide shots of Riley rapping alone and with others, tracking shots of Riley walking and rapping in tunnels, and long shots with slow zooms of locations marked with an X.
This document contains Latin words for kill, money/wealth, pale, put in charge, anyone/anything, thus, loosen/untie, hope, having suspected, will, as, and wind. It provides various forms of each Latin word including the nominative, infinitive, perfect, and supine forms where applicable.
This document discusses several art projects focused on self-expression and exploration of values through different mediums. It presents questions for student art projects involving auto-biographical still life drawings representing themselves using objects, creating abstract representations of important spaces, designing fantasy landscapes encompassing their values, and developing original characters placed in times/places that exemplify their own values to explore alternate histories and perspectives. The projects aim to have students express themselves and their values through visual art in open-ended creative ways.
The document discusses album advertisements for Jay-Z's "The Blueprint 3" and Kanye West's albums. It describes Jay-Z's ad as minimalist and art-house in its unconventional approach. It features a collage of musical instruments in an urban cityscape style. Kanye West also took an unconventional approach with a sharp minimalist ad of a blank CD and sticker that draws a wider audience while appealing to the core fan base through its authenticity. Both ads follow design principles to guide the eye while featuring urban-focused imagery and fonts to appeal to their target hip hop audiences.
Linux/Unix Night - (PEN) Testing Toolkits (English)Jelmer de Reus
The document compares the penetration testing toolkits BackBox Linux and Kali Linux. It discusses that these toolkits are used for enumeration, vulnerability scanning, penetration testing including wireless cracking, social engineering, and forensics. It provides an overview of the properties of each distro, including their menu structures and documentation. BackBox is based on Ubuntu while Kali uses a custom GNOME interface. The document demonstrates some of the tools and recommends using databases in Metasploit and focusing on tools with active communities.
This document provides an overview and introduction to database security. It discusses relevant security issues and developments in attacks and tools. It also covers the network ecosystem when deploying databases, including firewalling and VLAN configurations. Logging and event management are reviewed, including the use of SIEM software. Operating system hardening and patch management are addressed. Implementations of PostgreSQL and MySQL are demonstrated, including user and database creation, configuration files, and logging.
This document discusses social practices around personal videos online and opportunities to improve user experiences. It summarizes research in: 1) Understanding current user behaviors and challenges in sharing personal videos, 2) Using narrative structures and aggregation of community data to create more compelling automatically generated videos, and 3) Applying social science theories to build personalized tools that consider users' social relationships. The goal is to develop next generation video sharing tools.
Invited Talk OAGM Workshop Salzburg, May 2015dermotte
There is a gap between a user's information need and the queries they submit, known as the "intention gap". Bridging this gap is challenging due to the difficulty of translating intentions into search queries. Researchers have studied user intentions in various contexts like search, media production and sharing. However, fully understanding intentions is difficult as people have trouble expressing their own intentions and judging those of others. Future work should develop new techniques to relate content-based image retrieval to user intentions and take an interdisciplinary approach to better model intentions across domains.
CBMI 2013 Presentation: User Intentions in Multimediadermotte
This document discusses user intentions in visual information retrieval and multimedia information systems. It begins by introducing query by example search and different low-level visual features that work better for some domains than others. It then discusses how determining the right features and defining visual similarity is challenging. The document defines context and intention, and discusses how a user's intention relates to their information need. It reviews taxonomies of user intentions in web search and proposes intentions in multimedia may include search, production, sharing, archiving. The document proposes several open PhD theses around developing a general model of user intentions in multimedia, using games and human computation to infer intentions, bringing context to queries, and creating adaptable applications based on user intentions.
The document discusses audio and video fingerprinting techniques. It describes duplication recognition, categorization, and object/logo recognition as the main applications. It provides strengths, weaknesses, opportunities, and threats analyses for each application. It also summarizes preliminary results from experiments applying these techniques to a dataset from a Dutch television archive. Key challenges included transformations to videos, the diversity of content, and limitations of current commercial tools for this type of archival material.
Video & AI: capabilities and limitations of AI in detecting video manipulationsVasileiosMezaris
This document discusses capabilities and limitations of AI in detecting video manipulations such as deepfakes. It describes how AI can help identify manipulated videos through techniques like reverse video search, near duplicate detection, and video forensics. While AI shows promise in combating disinformation, current methods still have limitations and complete automation remains a challenge. A portfolio of tools is needed since no single technique can fully solve the problem.
SalesFUSION Webinar - Tracking Digital Body LanguageSalesfusion
This document summarizes a webinar on gathering and analyzing digital body language across the webinar lifecycle. The webinar featured an industry expert panel and discussed best practices for using metrics like polls, surveys, social media, and website visits to understand audiences and convert more leads. Attendees learned how to map the end-to-end sales process, ensure predictable outcomes, and maximize post-webinar follow up to nurture leads.
This webinar explored new and emerging ways to use online tools to assist those with legal problems and needs who are not able to secure the assistance of counsel. W feature initiatives in WA, TX, and CA, and Ohio.
Presenters:
Daniel Ediger, Northwest Justice Project
Colton Lawrence, Texas Legal Services Center
Neil Bowman-Davis, Napa Superior Court
Michael Walters, Pro Seniors, Inc.
This document describes a deep learning algorithm to classify videos as deepfakes or authentic. It discusses deepfakes, how they are created, the system architecture including data preprocessing, a ResNext-50 model architecture with LSTM and training workflow. Results show models trained on different datasets and frame sequences achieving accuracies from 84% to 98%. The project uses PyTorch and Django with Google Cloud Platform for computing.
Overview of the MediaEval 2014 Visual Privacy Task multimediaeval
This paper presents an overview of the Visual Privacy Task (VPT) of MediaEval 2014, its objectives, related dataset, and evaluation approaches. Participants in this task were required to implement a privacy filter or a combination of filters to protect various personal information regions in video sequences as provided. The
challenge was to achieve an adequate balance between the degree of privacy protection, intelligibility (how much useful information is retained post privacy filtering), and pleasantness (how minimal were the adverse effects of filtering on the appearance of the video frames). The submissions from the eight (8) teams who participated in this task were evaluated subjectively by surveillance experts, practitioners, data protection experts and by naïve viewers using a crowdsourcing approach.
http://ceur-ws.org/Vol-1263/mediaeval2014_submission_37.pdf
The document summarizes a business plan for Indico, a legal data extraction app. It outlines Indico's goal of attracting small to mid-size law practices by simplifying eDiscovery. It analyzes the $280B legal services market and growing $4B EDD sector. The plan details an initial strategy of piloting Indico before a marketing campaign. It identifies competitors and lists growth opportunities. Financial projections estimate $254M in first year revenue, growing to $34.4M by year 5, with corresponding profit increases from $104M to $21.5M. Key risks involve technology changes and privacy/security concerns.
Interactive films: lesson 1 - an introductioniain bruce
This document outlines an assignment to produce an interactive film with branching storylines and original sound design. It consists of 6 assignments across two units:
1) Digital video for interactive media, including essays on interactive video and planning/filming/editing a digital video sequence.
2) Soundtrack production, including essays on sound in film and recording/planning audio assets to produce a mastered soundtrack for the edited video sequence.
The document provides examples of interactive video types, discusses interactivity and its purpose, and assigns research and discussion tasks to understand principles of the interactive medium before planning and producing the student's own interactive film.
software construction and development pptxAdilIqbalAdil
The document discusses software design and development processes. It covers stakeholder identification, use case analysis, class diagram modeling, and documenting requirements through use cases. Stakeholders include anyone directly or indirectly affected by the system. Use cases describe tasks users can perform with the system. Class diagrams model the structure of information and responsibilities of the system. Requirements documentation includes use case descriptions with scenarios and extensions to guide system design.
Deep Learning Applications in the EnterpriseGanes Kesari
An introduction to Deep learning with a walkthrough of its application to business problems across a variety of areas. Case studies range from detection of animals to celebrities, classification of biodiversity to emotions and counting of crowds to coconuts.
Deck presented by Ganes Kesari B at the NJ Data Science Meeting, on 27th Oct 2018
Troy Janisch from American Family Insurance shares the do's and don'ts of a successful video campaign, and provides insight into some of the things his team is busy working on in the area of social video.
This document contains Latin words for kill, money/wealth, pale, put in charge, anyone/anything, thus, loosen/untie, hope, having suspected, will, as, and wind. It provides various forms of each Latin word including the nominative, infinitive, perfect, and supine forms where applicable.
This document discusses several art projects focused on self-expression and exploration of values through different mediums. It presents questions for student art projects involving auto-biographical still life drawings representing themselves using objects, creating abstract representations of important spaces, designing fantasy landscapes encompassing their values, and developing original characters placed in times/places that exemplify their own values to explore alternate histories and perspectives. The projects aim to have students express themselves and their values through visual art in open-ended creative ways.
The document discusses album advertisements for Jay-Z's "The Blueprint 3" and Kanye West's albums. It describes Jay-Z's ad as minimalist and art-house in its unconventional approach. It features a collage of musical instruments in an urban cityscape style. Kanye West also took an unconventional approach with a sharp minimalist ad of a blank CD and sticker that draws a wider audience while appealing to the core fan base through its authenticity. Both ads follow design principles to guide the eye while featuring urban-focused imagery and fonts to appeal to their target hip hop audiences.
Linux/Unix Night - (PEN) Testing Toolkits (English)Jelmer de Reus
The document compares the penetration testing toolkits BackBox Linux and Kali Linux. It discusses that these toolkits are used for enumeration, vulnerability scanning, penetration testing including wireless cracking, social engineering, and forensics. It provides an overview of the properties of each distro, including their menu structures and documentation. BackBox is based on Ubuntu while Kali uses a custom GNOME interface. The document demonstrates some of the tools and recommends using databases in Metasploit and focusing on tools with active communities.
This document provides an overview and introduction to database security. It discusses relevant security issues and developments in attacks and tools. It also covers the network ecosystem when deploying databases, including firewalling and VLAN configurations. Logging and event management are reviewed, including the use of SIEM software. Operating system hardening and patch management are addressed. Implementations of PostgreSQL and MySQL are demonstrated, including user and database creation, configuration files, and logging.
This document discusses social practices around personal videos online and opportunities to improve user experiences. It summarizes research in: 1) Understanding current user behaviors and challenges in sharing personal videos, 2) Using narrative structures and aggregation of community data to create more compelling automatically generated videos, and 3) Applying social science theories to build personalized tools that consider users' social relationships. The goal is to develop next generation video sharing tools.
Invited Talk OAGM Workshop Salzburg, May 2015dermotte
There is a gap between a user's information need and the queries they submit, known as the "intention gap". Bridging this gap is challenging due to the difficulty of translating intentions into search queries. Researchers have studied user intentions in various contexts like search, media production and sharing. However, fully understanding intentions is difficult as people have trouble expressing their own intentions and judging those of others. Future work should develop new techniques to relate content-based image retrieval to user intentions and take an interdisciplinary approach to better model intentions across domains.
CBMI 2013 Presentation: User Intentions in Multimediadermotte
This document discusses user intentions in visual information retrieval and multimedia information systems. It begins by introducing query by example search and different low-level visual features that work better for some domains than others. It then discusses how determining the right features and defining visual similarity is challenging. The document defines context and intention, and discusses how a user's intention relates to their information need. It reviews taxonomies of user intentions in web search and proposes intentions in multimedia may include search, production, sharing, archiving. The document proposes several open PhD theses around developing a general model of user intentions in multimedia, using games and human computation to infer intentions, bringing context to queries, and creating adaptable applications based on user intentions.
The document discusses audio and video fingerprinting techniques. It describes duplication recognition, categorization, and object/logo recognition as the main applications. It provides strengths, weaknesses, opportunities, and threats analyses for each application. It also summarizes preliminary results from experiments applying these techniques to a dataset from a Dutch television archive. Key challenges included transformations to videos, the diversity of content, and limitations of current commercial tools for this type of archival material.
Video & AI: capabilities and limitations of AI in detecting video manipulationsVasileiosMezaris
This document discusses capabilities and limitations of AI in detecting video manipulations such as deepfakes. It describes how AI can help identify manipulated videos through techniques like reverse video search, near duplicate detection, and video forensics. While AI shows promise in combating disinformation, current methods still have limitations and complete automation remains a challenge. A portfolio of tools is needed since no single technique can fully solve the problem.
SalesFUSION Webinar - Tracking Digital Body LanguageSalesfusion
This document summarizes a webinar on gathering and analyzing digital body language across the webinar lifecycle. The webinar featured an industry expert panel and discussed best practices for using metrics like polls, surveys, social media, and website visits to understand audiences and convert more leads. Attendees learned how to map the end-to-end sales process, ensure predictable outcomes, and maximize post-webinar follow up to nurture leads.
This webinar explored new and emerging ways to use online tools to assist those with legal problems and needs who are not able to secure the assistance of counsel. W feature initiatives in WA, TX, and CA, and Ohio.
Presenters:
Daniel Ediger, Northwest Justice Project
Colton Lawrence, Texas Legal Services Center
Neil Bowman-Davis, Napa Superior Court
Michael Walters, Pro Seniors, Inc.
This document describes a deep learning algorithm to classify videos as deepfakes or authentic. It discusses deepfakes, how they are created, the system architecture including data preprocessing, a ResNext-50 model architecture with LSTM and training workflow. Results show models trained on different datasets and frame sequences achieving accuracies from 84% to 98%. The project uses PyTorch and Django with Google Cloud Platform for computing.
Overview of the MediaEval 2014 Visual Privacy Task multimediaeval
This paper presents an overview of the Visual Privacy Task (VPT) of MediaEval 2014, its objectives, related dataset, and evaluation approaches. Participants in this task were required to implement a privacy filter or a combination of filters to protect various personal information regions in video sequences as provided. The
challenge was to achieve an adequate balance between the degree of privacy protection, intelligibility (how much useful information is retained post privacy filtering), and pleasantness (how minimal were the adverse effects of filtering on the appearance of the video frames). The submissions from the eight (8) teams who participated in this task were evaluated subjectively by surveillance experts, practitioners, data protection experts and by naïve viewers using a crowdsourcing approach.
http://ceur-ws.org/Vol-1263/mediaeval2014_submission_37.pdf
The document summarizes a business plan for Indico, a legal data extraction app. It outlines Indico's goal of attracting small to mid-size law practices by simplifying eDiscovery. It analyzes the $280B legal services market and growing $4B EDD sector. The plan details an initial strategy of piloting Indico before a marketing campaign. It identifies competitors and lists growth opportunities. Financial projections estimate $254M in first year revenue, growing to $34.4M by year 5, with corresponding profit increases from $104M to $21.5M. Key risks involve technology changes and privacy/security concerns.
Interactive films: lesson 1 - an introductioniain bruce
This document outlines an assignment to produce an interactive film with branching storylines and original sound design. It consists of 6 assignments across two units:
1) Digital video for interactive media, including essays on interactive video and planning/filming/editing a digital video sequence.
2) Soundtrack production, including essays on sound in film and recording/planning audio assets to produce a mastered soundtrack for the edited video sequence.
The document provides examples of interactive video types, discusses interactivity and its purpose, and assigns research and discussion tasks to understand principles of the interactive medium before planning and producing the student's own interactive film.
software construction and development pptxAdilIqbalAdil
The document discusses software design and development processes. It covers stakeholder identification, use case analysis, class diagram modeling, and documenting requirements through use cases. Stakeholders include anyone directly or indirectly affected by the system. Use cases describe tasks users can perform with the system. Class diagrams model the structure of information and responsibilities of the system. Requirements documentation includes use case descriptions with scenarios and extensions to guide system design.
Deep Learning Applications in the EnterpriseGanes Kesari
An introduction to Deep learning with a walkthrough of its application to business problems across a variety of areas. Case studies range from detection of animals to celebrities, classification of biodiversity to emotions and counting of crowds to coconuts.
Deck presented by Ganes Kesari B at the NJ Data Science Meeting, on 27th Oct 2018
Troy Janisch from American Family Insurance shares the do's and don'ts of a successful video campaign, and provides insight into some of the things his team is busy working on in the area of social video.
Workshop 4 - Storytelling with digital video (PPT)madhavi2011
This two-day workshop teaches participants how to create effective digital storytelling and videos for NGOs. Day 1 focuses on storytelling frameworks, basic video shooting techniques using Flip cameras, and creating initial videos. Day 2 covers editing videos in Movie Maker, setting up YouTube accounts, and sharing/distributing videos online. The goal is for participants to leave with the skills to create and disseminate informative videos that further their organization's mission.
Deepfakes and the authenticity of video material were discussed. Methods for creating and detecting deepfakes were presented. Challenges in detecting real-time deepfakes were highlighted. Research is ongoing to develop techniques like PRNU analysis and video-based neural networks to determine if videos have been manipulated. However, the arms race continues as deepfake generation becomes more accessible while detection becomes more difficult. Collaboration between universities and forensic institutes aims to address these challenges.
Techniques and Tools for fact-checking a presentation by Ochaya Jackson Amos in an online training session organised by 211 Check with support from the International Fact-checking Network (IFCN)
This document contains summaries of 15 mobile applications from NTechnosoft Private Limited. It includes the application name and description, screenshots, and links to the application pages on the iTunes and Google Play stores. Contact information is provided for Henry Flintoff, the business development manager at NTechnosoft Private Limited.
The Black Hole of Video Analytics- KISSmetrics / Viddler WebinarViddler Inc.
Eric McClatchy, Marketing Manager of Viddler, presented a video analytics webinar for KISSmetrics.
"The Black Hole of Video Analytics" prevents your ability to relate your video analytics to your website goals and metrics, severely limiting the insights your video analytics can bring.
Topics Covered:
- What is the "Black Hole"
- How to Avoid the "Black Hole"
- Advanced video reports and charts
- Experiments to improve video effectiveness
Similar to The User at the Wheel of the Online Video Search Engine (20)
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
The User at the Wheel of the Online Video Search Engine
1. The User at the Wheel of the
Online Video Search Engine
Christoph Kofler (c.kofler@tudelft.nl)
Delft University of Technology, Delft, The Netherlands
1
2. In this talk…
Two of our approaches presented at ACM Multimedia
2012, Nara, Japan:
1. User intent in video search
2. Query failure prediction in video search sessions
2
3. I.
Intent and its Discontent
ACM Multimedia 2012 Brave New Ideas
Work with Alan Hanjalic and Martha Larson
Slide credit: Martha Larson
3
5. Many results, but no satisfaction
Top ranked
results are about
koi ponds, but we
are discontent:
There is no
information
specifically about
the significance of
koi ponds.
5
6. Many queries, no satisfaction
Query suggestions
Refinement
strategies don’t
always work.
Query reformulation
6
7. Video Search Engine Workflow
Information
need
Query Results
“koi pond” list
Video search engine
• So what went wrong?
Openclipart.org: samukunai 7
10. Video Search Engine Workflow
Information
need
Query Results
“koi pond” list
Video search engine
• So what went wrong?
10
11. Video Search Engine Workflow
Information
need
Query Results
“koi pond” list
Video search engine
• So what went wrong?
We neglect the goal that the user is trying to reach…
…our video search is “blind” to user intent.
11
12. User information need
What Why
Query Results
“koi pond” list
Video search engine
User information need has two parts:
• Topic = What the user is searching for.
• Intent = Why the user is searching for it.
12
13. Removing the Intent Roadblock
The main research roadblock has been the question:
Which intent categories are
both useful to users
and technically within reach?
1. Categories of Intent: Which ones are useful to users?
2. Indexing Intent: Is intent technically feasible?
3. Impact of Intent: Could intent prevent discontent?
13
16. Natural Language Information Needs
• We harvested natural language information needs related to
video search from Yahoo! Answers.
• We analyzed 281 cases in which the user has clearly stated
the goal behind the information need.
16
17. User Search Intent Categories
• In an iterative process, we manually clustered the information
needs to identify the dominant user search intent categories
(using a card-sorting methodology).
Intent category Description
I. Information Obtain knowledge and/or gather information
II. Experience: Learning Learn something practically by experience
III. Experience: Exposure Experience a person, place, entity or event.
IV. Affect Change mood or affective state.
V. Object Video is its own goal.
17
19. Wider View on Video Intent
Search Intent: Creation Intent:
Video Intent
19
20. Is intent within our reach?
• We carry out a feasibility experiment using simple features from:
• Shot patterns
• Speech recognition transcripts
• User-contributed metadata: title, description, tags
v
e
r
s
u
s
Information Intent Affect Intent
20
21. Evaluating Classifiers for Intent
• Evaluate with two large sets of Internet video (from blip.tv)
• Train a classifier that assigns intent categories to videos.
• See paper for the experiment details; here selected results are
reported for the smaller, 350 hour set.
21
22. Features from shot patterns
• Shot patterns show promise.
• Weighted F-measure 0.53
• They are especially good in distinguishing
“Information” vs. “Affect”
Shot pattern from an “Information” video (correctly classified)
Shot pattern from an “Affect” video (correctly classified)
22
23. Features from ASR transcripts
• Speech recognition transcripts perform better (WFM 0.67)
• They don‟t reach the performance of tags (WFM 0.77)
“Egon comes packaged on a really nice looking blister cover that
features some great super natural colors and images from the
films. The back of the package features a really cool bio…”
Transcript excerpt from an “Experience: Exposure” video (correctly classified)
“It’s Thursday, April 10 2008. I am Robert Ellis, and this is your
Thursday snack. Welcome back to political lunch. Barack Obama
has painted himself in some ways,…”
Transcript excerpt from an “Information” video (correctly classified)
23
25. Experiment on User Perception of
Intent
• Workers were presented with a set of three videos returned by
YouTube in response to a query.
• The videos are about the same topic, i.e., “what”
• We ask if the videos have the same intent, i.e., “why”.
Short excerpt of the user study survey:
25
26. User Agreement on Video Intent
• Setup: For each of the 883 queries, three workers filled in
the survey (total 294 workers).
• Results: For 55% of the queries, 2/3 workers agreed that
the set contained videos representing at least two different
intent categories.
• Conclusions:
• If online video search engines become “intent-aware”,
users will indeed notice the difference.
26
27. Examples of Agreement on Intent
Query: „human metabolism Query: „motorcycle‟
glycolosis‟
Agreed on
Agreed on “Experience:
“Information” Learning”
Agreed on
Agreed on “Experience:
“Information” Learning”
Agreed on Agreed on
“Affect” “Affect”
27
29. Take-home message
• Intent can help us develop video search engines that get
users where they want to go.
• We have removed the video search intent roadblock: We
have shown which intent categories are important and that
they are in reach.
More challenges lie in the
road ahead.
29
30. Challenge 1: Evaluating Intent
• Quantifying the ability of intent to prevent discontent.
“My search engine
finds topics, but is it
getting me where I
want to go?”
Flickr: sean dreilinger 30
31. Challenge 2: Isolating Intent
• Addressing videos that fit multiple intents.
“I‟m not relaxing, I‟m
a biologist studying
fish feeding habits.”
31
32. Challenge 3: Implementing Intent
Query Results
“koi pond” list
Video search engine
• Implementing intent into the video search engine workflow.
“Intent fits anywhere and everywhere”
32
33. II.
When Video Search Goes Wrong
ACM Multimedia 2012 Multimedia Search and Retrieval
Work with Linjun Yang, Martha Larson, Tao Mei, Alan Hanjalic, Shipeng Li
Delft University of Technology, Delft, The Netherlands
Microsoft Research Asia, Beijing, China
33
34. Searching gets complex!
• Searching for videos on the Internet becomes increasingly
complex
• Users face increased difficulty in formulating effective and
successful text-based video search queries
34
37. Deployment of existing algorithms
Algorithms improving the performance of video search engines
have been developed for whole search pipeline
1. Not effectively deployed
2. “Expensive” for both user and search engine
37
38. How can we improve?
Predicting when users will fail in their search session…
…can help to more effectively deploy these algorithms
Focus of this
contribution!
Concept-based retrieval … Particular query suggestion
Better search results for user and “cheaper” for engine
38
39. Approach and Motivation
• Context-aware Query Failure Prediction
• Prediction of success or failure of a query at query time…
• …within a user‟s search session with the video search engine
Patterns of users’ interaction with the search engine
Visual features from search results list produced by query
• When does a query „fail‟? No search results click
39
40. Terminology: Query performance
prediction (QPP)
• Predict retrieval performance of query
• Correlates with precision
• How topically coherent are search results? (clear vs. ambigious)
• Statistics involve
• Query string
• Background collection
• Search results
• No search session context
40
43. Why QPP in Video Search is not
enough: User Perspective
0.5
(Almost) all fail (Almost) all successful
0.4
Frequency
0.3
0.2
0.1
0
0% 1-9% 10-19% 20-29% 30-39% 40-49% 50-59% 60-69% 70-79% 80-89% 90-100%
Proportion of success rate for queries
All engines YouTube Google video Bing video Yahoo! video
Example: koi history: 100K submitted, 60K successful
60% success rate
43
44. Why QPP in Video Search is not
enough: User Perspective
0.5
(Almost) all fail (Almost) all successful
0.4
Frequency
0.3
0.2
0.1
0
0% 1-9% 10-19% 20-29% 30-39% 40-49% 50-59% 60-69% 70-79% 80-89% 90-100%
Proportion of success rate for queries
All engines YouTube Google video Bing video Yahoo! video
Example: koi history: 100K submitted, 60K successful
60% success rate
Query performance prediction is not trivial in the majority of the cases,
since query success highly depends on the query‟s context.
44
45. Video Search Transaction Logs
Time Current URL Previous URL Query/Action Vertical
10:46:12 …search?q= - koi documentary video
koi+documentary
10:46:20 …search?q= …search?q= koi history video
koi+history koi+documentary
10:46:25 …q=koi+history&view=detail …search?q= <results click> video
&mid=E9589097DCE1DDD7D koi+history
17DE9589097DCE1DDD7D17
45
46. Context-aware
Query Failure Prediction
• Exploratory investigation of users’ search sessions,
stored in transaction log, to find characteristics indicative for
query failure
• Context is derived from query‟s context within a user‟s search
session
46
47. Context-aware
Query Failure Prediction
• Exploratory investigation of users’ search sessions,
stored in transaction log, to find characteristics indicative for
query failure
• Context is derived from query‟s context within a user‟s search
session
USER FEATURES:
QPP + Session Context
47
48. User Features (excerpt)
• General search session statistics
• Duration
• Number of interactions
• Search engine vertical switches
• Query formulation strategies and clarity
• Query reformulation types
• Differences between clarity of queries within session
• Overlapping query terms
• Mutually exclusive query topics
• Click-through data
• Click behavior in search results
• Dwell time on search results
48
49. Why QPP in Video Search is not
enough: Engine Perspective
49
50. Context-aware
Query Failure Prediction
• Exploit visual information of thumbnails of produced search
results list
• Consistency of visual content of search results on
conceptual level reflects topical focus of the results list
50
51. Context-aware
Query Failure Prediction
• Exploit visual information of thumbnails of produced search
results list
• Consistency of visual content of search results on
conceptual level reflects topical focus of the results list
ENGINE FEATURES:
QPP + Visual Search Results
51
52. Engine Features (excerpt)
• Show the potential of the visual information to be helpful
for query failure prediction
• Light-weight features to be
• Deployed during query time
• Covering the whole query space
• Higher-level representations are not scalable
• Video search results are represented by standard local and
global features
52
53. Model Training and Prediction
• Supervised learning trains generic classifiers on development
set using the extracted features
• One binary classifier for feature sets representing user and
engine features
53
54. Offline
User
Training
Features
Feature
Extraction
Engine
Features Model
Online
Context-
Engine features
Aware
Prediction
Q1 Q2 Q3 Q4
Feature
? Extraction
User features
54
56. Dataset
• Development set
• 24K search sessions
• 108K queries
• Test set
• 150K search sessions
• 1.1M queries
• 392K unique queries exclusively occur in the test set
• For each query, we collected information from 25 most-
relevant search results
• Textual information: titles of videos
• Visual information: static visual thumbnails
56
57. Baselines, Training, Evaluation
• Compare against a set of query performance prediction
baselines and the dominant class baseline
• Ground truth from clicks in search session
(from transaction log)
57
58. Performance
F (q. i. F (q. i.
Features WF
success) failure)
Best QPP baseline 0.6862 0.748 0.593
Feature combination from
0.7356 0.788 0.656
engine features
Feature combination from
0.7678 0.820 0.688
user features
Feature combination from
0.7744 0.830 0.690
user and engine features
• Engine features: +4% improvement
• User features: +8% improvement
• Combined features: +9% improvement
58
60. Discussion & Take home messages
1. Simple visual features from search results help to
extend query performance prediction
• Able to outperform conventional text-only query performance
prediction
• Performance increase (+4%) is quite modest, but promising
• Consistent with our expectations for our relatively simple
visual representations
• Can positively influence wrong predictions by user features-
only classifiers
60
61. Discussion & Take home messages
2. Features from the user context help the most for
query failure prediction
Three classes of query types benefited from our user features
(+8%)
1. User presumably wants recommendations over general
results, e.g., „youtube‟
2. Particular type of requested content is not available,
e.g., „free movies‟
3. Wrong video search engine usage (wrong vertical) or
misspellings, e.g., „yahoo mail‟, „micheal jackon‟
61
62. Discussion & Take home messages
2. Features from the user context help the most for
query failure prediction
• „Long tail‟ queries
• 36% of video queries in test set were submitted once
• Contribution of session context features is independent of the
frequency of query submission
• Challenge: „Cold start‟ queries do not have enough session
context
• Only very little information is needed to address the cold start
issue
62
63. Discussion & Take home messages
3. Context-aware Query Failure Prediction approach is
applicable using little session data
• Solely focuses on local search sessions
• No user profiles or global search patterns were involved in
the learning process
63
64. Future Work
1. Improvement of engine features using visual
information from the video search results list
• Higher-level representation of thumbnails
• Additional sources of visual information
2. Enhancing the performance of an entire range of
video search engine optimization techniques
3. Experimenting with additional definitions of query
failure (e.g., dwell time on search results)
64
65. The User at the Wheel of the
Online Video Search Engine
Christoph Kofler (c.kofler@tudelft.nl)
Delft University of Technology, Delft, The Netherlands
THANK YOU FOR YOUR ATTENTION!
65
Editor's Notes
Approaches: or combinations of these
Approaches: or combinations of these
Approaches: or combinations of these
Approaches: or combinations of these
We don’t know when which method is good and when to deploy a particular methodUser does not get proper search results (as heard yesterday in BNI) and engine has to do unnessacerycompution which might not influence user and which is therefore senseless.
Approaches: or combinations of these
Approaches: or combinations of these
Approaches: or combinations of these
Extreme cases are well predicted by qpp because in these cases no context is necessary to make a relateively good prediction. In (almost) all of the cases, when the particular query string is submitted to a search session, indepentenly where in the search session, then it will either be successful or failed. So qpp would do a good job here. However…
Extreme cases are well predicted by qpp because in these cases no context is necessary to make a relateively good prediction. In (almost) all of the cases, when the particular query string is submitted to a search session, indepentenly where in the search session, then it will either be successful or failed. So qpp would do a good job here. However…
One source to infer context are transaction logs…
We looked into queries which fall in the middle category on the plot before, i.e., which have a lot of successful and failed query instances throughout different search sessions. Then we manually investigated these search session in order to infer characteristics of the user which point to success or failure of these queries, dependent on the session context.
In the paper we came up with 5 observations pointing to query failure. These are related to the iterative search goal development throughout the session, the satisfaction of the user with the results thus far in the session and so on. Due to time limitations, I am refering you to the paper at this point and just want to mention some features which we extracted from these observations which are indicative for query failure.general Internet browser session and search session statistics, (ii) query (re)formulation behavior and clarity of search goal expressiveness, and (iii) click-through data in the video search results lists generated by the queries in the search sessionTwo types of pre-query session historiesSession query historyQuery-specific reformulation historyFeatures are extracted from these local search session histories relative to the current queryWe do not learn user profiles or global search patterns
We heard yesterday in the cbir session that it is not necessarily related that the more specific the queyr, the more visually consisten the search results. So visual features give additional information next to text-based search results which could be exploited w.r.t. query failure.
High consistency should then indicate that the search engine has achieved good performance on the query that generated the results list.
Both, NSCQ and QC baselines achieve a good balance between correctly classified instances of -qif and +qif, however QC outperforms NCSQ. The relatively strong performance of the conventional QPP baseline demonstrates the potential and the strength of the text-retrieval methods to transfer to video retrieval problems. For the remainder of the experiments we compare performance against the best-performing conventional QPP baseline achieved by the query clarity score.
Our user indicator-based query failure prediction methods statistically significantly outperform the conventional QPP baseline (QC in Table 2) and achieve an 8% improvement in absolute performance solely by taking local search context into account. The best-performing method is the classifier built on features derived from ‘User familiarity’. Another strong performer is ‘Previous dissatisfaction’, reflecting previous failures in the session. For the observation ‘Query iterations’, using local features from the query-specific reformulation region of the search session increases the performance compared to using the entire query history results, suggesting the value of using narrow local context. The relatively poor performance achieved by observation ‘Goal-directedness’ suggests that search goal clarity evolving over a search session is not consistent. Early and late fusions perform well but do not succeed in outperforming individual well-performing observations. Looking at F-measure values of individual classes shows that classifying +qif using the proposed classifiers is more conservative than classifying –qif instances. Observations clearly achieve a much better result for –qif than for +qif. The characteristics of successful queries are presumably more stable, most likely reflecting the relatively greater stability of the characteristics of the successful query.
is a clear sign that the visual component of video search results should not be ignored, but rather potentially makes an important contribution to query failure prediction
is a clear sign that the visual component of video search results should not be ignored, but rather potentially makes an important contribution to query failure prediction
is a clear sign that the visual component of video search results should not be ignored, but rather potentially makes an important contribution to query failure prediction
is a clear sign that the visual component of video search results should not be ignored, but rather potentially makes an important contribution to query failure prediction
is a clear sign that the visual component of video search results should not be ignored, but rather potentially makes an important contribution to query failure prediction