End to-end convolutional semantic embeddings
•
0 likes•5,523 views
This document discusses end-to-end convolutional semantic embeddings for relating images and text. It presents a model that uses a textual network of convolutional and recurrent layers to produce embeddings of sentences, and a visual network of convolutional layers to produce image embeddings. These embeddings are trained end-to-end using both global and local losses to relate images and sentences. The model achieves state-of-the-art results on tasks of sentence retrieval from images and image retrieval from sentences on datasets like MS-COCO and Flickr30K.
Report
Share
Report
Share
Download to read offline
Recommended
Context_Embedding_Network
DLゼミで発表した論文資料です
Context Embedding Networks
(https://arxiv.org/pdf/1710.01691.pdf)
Kun Ho Kim Oisin Mac Aodha Pietro Perona (California Institute of Technology)
Graph in Apache Cassandra. The World’s Most Scalable Graph Database
1. Building a graph database requires modeling the data, choosing a query language, and providing storage.
2. Existing distributed databases like Cassandra can be used for storage due to their scalability and reliability, though a native graph database provides more functionality.
3. Solving complex graph problems requires capabilities beyond basic queries, including search, analytics, and integration with machine learning, which graph databases are designed to support at scale.
Lessons learned from SearchD development
Replication, distributed processing, not killing Elasticsearch with 400M+ docs & a stupid frontend trick.
Analyze and Visualize Git Log for Fun and Profit
This document discusses analyzing and visualizing Git commit logs to gain insights from a codebase's history. It describes analyzing commits across dimensions like authors, file extensions, times, and dates to understand contributor patterns and identify opportunities for improvement. Visualizations are shown examining contributions by time zone, hour of day, and day of week to learn about developer workflows and prevent burnout.
Analyze and Visualize Git Log for Fun and Profit - DevTernity 2015
This document discusses analyzing and visualizing git commit logs to gain insights. It provides examples of visualizing commit data by time, authors, files changed, and other dimensions. Examples shown include contributions over time to Rails and Node.js, top contributors, and analyzing patterns in commit times. The document advocates using git logs for code metrics and "Mining" them to learn from a codebase's history.
#NetflixEverywhere Global Architecture
This document summarizes Netflix's journey to building a globally ubiquitous and failure-resilient architecture. It describes how Netflix evolved from a single data center architecture to a multi-region active-active design using microservices, Cassandra for data storage, EVCache for caching, and virtual DNS regions for traffic management. The architecture is designed to reliably serve customers from any region by replicating data and traffic across regions and implementing failover mechanisms.
AWS re:Invent 2016: From Resilience to Ubiquity - #NetflixEverywhere Global A...
Building and evolving a pervasive, global service requires a multi-disciplined approach that balances requirements with service availability, latency, data replication, compute capacity, and efficiency. In this session, we’ll follow the Netflix journey of failure, innovation, and ubiquity. We'll review the many facets of globalization and then delve deep into the architectural patterns that enable seamless, multi-region traffic management; reliable, fast data propagation; and efficient service infrastructure. The patterns presented will be broadly applicable to internet services with global aspirations.
CBOR - The Better JSON
JSON, by now, became a regular part of most applications and services. Do we, how ever, really want to transfer human readable information or are we looking for a binary protocol to be as debuggable as JSON? CBOR the Concise Binary Object Representation offers the best of JSON + an extremely efficient, binary representation.
http://www.cbor.io
Recommended
Context_Embedding_Network
DLゼミで発表した論文資料です
Context Embedding Networks
(https://arxiv.org/pdf/1710.01691.pdf)
Kun Ho Kim Oisin Mac Aodha Pietro Perona (California Institute of Technology)
Graph in Apache Cassandra. The World’s Most Scalable Graph Database
1. Building a graph database requires modeling the data, choosing a query language, and providing storage.
2. Existing distributed databases like Cassandra can be used for storage due to their scalability and reliability, though a native graph database provides more functionality.
3. Solving complex graph problems requires capabilities beyond basic queries, including search, analytics, and integration with machine learning, which graph databases are designed to support at scale.
Lessons learned from SearchD development
Replication, distributed processing, not killing Elasticsearch with 400M+ docs & a stupid frontend trick.
Analyze and Visualize Git Log for Fun and Profit
This document discusses analyzing and visualizing Git commit logs to gain insights from a codebase's history. It describes analyzing commits across dimensions like authors, file extensions, times, and dates to understand contributor patterns and identify opportunities for improvement. Visualizations are shown examining contributions by time zone, hour of day, and day of week to learn about developer workflows and prevent burnout.
Analyze and Visualize Git Log for Fun and Profit - DevTernity 2015
This document discusses analyzing and visualizing git commit logs to gain insights. It provides examples of visualizing commit data by time, authors, files changed, and other dimensions. Examples shown include contributions over time to Rails and Node.js, top contributors, and analyzing patterns in commit times. The document advocates using git logs for code metrics and "Mining" them to learn from a codebase's history.
#NetflixEverywhere Global Architecture
This document summarizes Netflix's journey to building a globally ubiquitous and failure-resilient architecture. It describes how Netflix evolved from a single data center architecture to a multi-region active-active design using microservices, Cassandra for data storage, EVCache for caching, and virtual DNS regions for traffic management. The architecture is designed to reliably serve customers from any region by replicating data and traffic across regions and implementing failover mechanisms.
AWS re:Invent 2016: From Resilience to Ubiquity - #NetflixEverywhere Global A...
Building and evolving a pervasive, global service requires a multi-disciplined approach that balances requirements with service availability, latency, data replication, compute capacity, and efficiency. In this session, we’ll follow the Netflix journey of failure, innovation, and ubiquity. We'll review the many facets of globalization and then delve deep into the architectural patterns that enable seamless, multi-region traffic management; reliable, fast data propagation; and efficient service infrastructure. The patterns presented will be broadly applicable to internet services with global aspirations.
CBOR - The Better JSON
JSON, by now, became a regular part of most applications and services. Do we, how ever, really want to transfer human readable information or are we looking for a binary protocol to be as debuggable as JSON? CBOR the Concise Binary Object Representation offers the best of JSON + an extremely efficient, binary representation.
http://www.cbor.io
KDD17Tutorial_final (1).pdf
This document summarizes research on learning representations of large-scale networks. It discusses how deep learning techniques can be used to learn low-dimensional embeddings that encode network structure and relationships between nodes. Convolutional and recurrent neural networks are proposed as methods to learn representations of entire networks by processing sequences of local neighborhood structures in an end-to-end fashion.
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang: two very promising beasts, made for the new parallel world, but still lingering in the wings. Not only are they addressing todays challenges but they are using parallel architectures as corner stone of their new and surprising approach to be faster and more productive. What are they good for? Why are we working to team them up?
Erlang promises faster implementation, way better maintenance and 4 times shorter code. VoltDB claims to be two orders of magnitude faster than its competitors. The two share many similarities: both are the result of scientific research and designed from scratch to address the new reality of parallel architectures with full force.
This talk presents the case for Erlang as server language, where it shines, how it looks, and how to get started. It details Erlang's secret sauce: microprocesses, actors, atoms, immutable variables, message passing and pattern matching. (Note: for a longer version of this treatment of Erlang only see: Why Erlang? http://www.slideshare.net/eonblast/why-erlang-gdc-online-2012)
VoltDB's inner workings are explained to understand why it can be so incredibly fast and still better than its NoSQL competitors. The well publicized Node.js benchmark clocking in at 695,000 transactions per second is described and the simple steps to get VoltDB up and running to see the prodigy from up close.
Source examples are presented that show Erlang and VoltDB in action.
The speaker is creator and maintainer of the Erlang VoltDB driver Erlvolt.
Mysql(2)
This document provides a summary of MySQL and PostgreSQL, two open-source database management systems. It compares their origins, features, platforms, and licensing. Key differences highlighted include that MySQL generally has simpler database design and replication while PostgreSQL has more complex rule sets and is faster. The document also provides contact information for getting further assistance.
Overview of Lincoln Paper Design
This set of slides has been presented to the Illinois Program for Research in the Humanities at the University of Illinois at Urbana-Champaign on 02-27-2009
#NetflixEverywhere Global Architecture
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1Rzjtjm.
Josh Evans talks about the Netflix journey of failure, innovation, and ubiquity. He reviews the many facets of globalization then delves deep into the architectural patterns that enable seamless, multi-region traffic management, reliable, fast data propagation, and efficient service infrastructure. The patterns presented are broadly applicable to Internet services with global aspirations. Filmed at qconlondon.com.
Josh Evans is Director of Operations Engineering at Netflix, with experience in e-commerce, playback control services, infrastructure, tools, testing, and operations.
JavaScript History
JavaScript was created in the 1990s at Netscape to enable interactivity in web browsers. It was influenced by languages like Self and Scheme. JavaScript was standardized by ECMA International to foster adoption beyond Netscape Navigator and Internet Explorer. The language has evolved through several standards over the years, with ES6/ES2015 being the latest version adding features like classes, modules, and iterators. New transpilers allow use of upcoming JavaScript features before full browser support.
OWF14 - Big Data : The State of Machine Learning in 2014
Olivier GRISEL
A broad overview of what technologies are usable now, what are just around the corner.
WCM-7 Surfing with CMIS
There is a high demand for companies to publish and promote their content on the web. To accommodate this demand Alfresco has provided a number of solutions covering editorial to web tier. As an example of this demand Ixxus was commissioned by a leading business information publisher to produce a microsite for ‘teaser’ content to increase subscriptions of their main site. To deliver this Ixxus utilized a number of features provided by Alfresco, such as services like the Transfer Service, the web scripts framework, and Surf. The majority of these features now make up the mainstay of Alfresco’s Web Quick Start WCM solution. The goal of this session is to demonstrate a real world example of how the combination of Alfresco, Surf and CMIS offers a great platform for developers to produce content-rich websites quickly. The session will cover: Using Spring Roo to construct a Surf application, Benefits of using Spring Surf, Using the Transfer Service, OpenCMIS in Surf, Varnish your Surf application, and What’s next
Historical aerial scanning project
This document summarizes a project by the Texas Water Development Board and TNRIS interns to digitize and georeference historical coastal aerial photographs from 1940-1990. The interns will scan over 5,000 photos per month to preserve them and prevent damage. The georeferenced scans will be made available online. The process involves scanning, georeferencing, mosaicking and enhancing the images. Stakeholders benefit from preserved history and access to coastal change data. Interns gain experience that improves employment prospects.
Cassandra Summit 2014: Active-Active Cassandra Behind the Scenes
Presenter: Roopa Tangirala, Senior Cloud Data Architect at Netflix
High availability is an important requirement for any online business and trying to architect around failures and expecting infrastructure to fail, and even then be highly available, is the key to success. One such effort here at Netflix was the Active-Active implementation where we provided region resiliency. This presentation will discuss the brief overview of the active-active implementation and how it leveraged Cassandra’s architecture in the backend to achieve its goal. It will cover our journey through A-A from Cassandra’s perspective, the data validation we did to prove the backend would work without impacting customer experience. The various problems we faced, like long repair times and gc_grace settings, plus lessons learned and what would we do differently next time around, will also be discussed.
Technologies For Appraising and Managing Electronic Records
This document summarizes technologies for appraising and managing electronic records, including discovering relationships among digital file collections and comparing document versions. It presents three technologies: file2learn to discover relationships between files based on metadata extraction and analysis; doc2learn for comprehensive document comparisons; and Polyglot for automated file format conversion and quality assessment.
Samsung voice intelligence.v5.5
This document discusses the development of Bixby, Samsung's intelligent virtual assistant. It describes Bixby as being fundamentally different from other assistants due to its ability to seamlessly switch between voice and touch modes, its context awareness, and its ability to understand incomplete commands. The document outlines some of the key challenges in developing Bixby, including managing its massive contextual input space and variable output capabilities across devices and versions. It discusses the use of deep learning and other techniques to address these challenges.
Devops at Netflix (re:Invent)
How Netflix operates for maximum freedom and agility.
Video here: https://www.youtube.com/watch?v=s0rCGFetdtM
CommCon 2018 - Realtime Machine Learning
This document discusses several applications of real-time machine learning including speech recognition, text-to-speech, image super resolution, noise suppression, network optimization for real-time communications, and selective video encoding. It also considers potential issues like voice copying, deep fakes, and privacy concerns that could arise from these technologies.
Closing Keynote
Dr. Jim Webber presented the closing keynote at a conference. He recapped Neo4j's recent releases, including version 3.0 which delivered new graph capabilities. He discussed future hardware trends and how native graph technology provides performance advantages. He also showed how Neo4j can handle workloads at massive scales of over 1 trillion relationships and 1 million writes per second on a single machine. Finally, he outlined Neo4j's new causal clustering architecture in version 3.1 which provides security, scalability, and fault tolerance for enterprise graph applications.
Be faster then rabbits
The document discusses various topics related to web development including Java principles, Spring frameworks, PHP, high-load web applications, mobile backend as a service (mBaas), web frameworks, Java web development frameworks like JSF and GWT, rendering on the server-side vs client-side, distribution of work between designers and developers, web browsers and their support for HTML5 and CSS3, programming languages, GUI frameworks, AngularJS, testing tools like JUnit, and build tools like Maven, Ant, and Ivy.
สร้างสรรค์กิจกรรมการเรียนรู้ที่มีความหมาย ด้วยสื่อ Learning Object
The document discusses various aspects of designing effective learning objects (LOs) for teaching mathematics at the secondary school level. It covers topics like LO metadata, instructional design approaches, storyboarding, feedback mechanisms, usability testing, and technical considerations. The key goals of LOs are to transition students from passive to active learning, bridge the digital divide, and close gaps in understanding through interactive, personalized instructional experiences.
Lessons Learnt From Working With Rails
As given at Scotland On Rails 2008.
(note: the highlighting and layering hasn't come through so well in Slideshares conversion process. I'll probably put a version to download from my site to solve this)
R for the semantic web, Quesada useR 2009
This document discusses using R to analyze and work with semantic web data represented as graphs. It introduces key concepts like ontologies, RDF, and linked data. It also describes how graph analysis techniques like centrality measures and PageRank can be applied to semantic web data to identify important nodes. Developing tools for semantic web data in R is challenging because current software uses an object-oriented approach while semantic data is represented as triples in graphs; an R package is needed to directly handle RDF.
AWS re:Invent 2016: Design Patterns for High Availability: Lessons from Amazo...
This document discusses design patterns for high availability used by Amazon CloudFront. It describes four key patterns: 1) FoodTasting which involves deploying changes incrementally to a small subset of servers first, 2) handling flash crowds by caching content, serving only necessary content, and using scheduled auto scaling, 3) implementing defense in depth strategies like multi-implementation and sharding to reduce the blast radius of failures, and 4) protecting against time bombs by jittering deployments and configurations across servers to avoid homogeneous outages. The document provides examples of how these patterns have been implemented in CloudFront and AWS services.
Generating Automatic Feedback on UI Mockups with Large Language Models
Generating Automatic Feedback on UI Mockups with Large Language Modelsを和訳紹介したものです
【DLゼミ】XFeat: Accelerated Features for Lightweight Image Matching
公開URL：https://arxiv.org/pdf/2404.19174
出典：Guilherme Potje, Felipe Cadar, Andre Araujo, Renato Martins, Erickson R. ascimento: XFeat: Accelerated Features for Lightweight Image Matching, Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
概要：リソース効率に優れた特徴点マッチングのための軽量なアーキテクチャ「XFeat（Accelerated Features）」を提案します。手法は、局所的な特徴点の検出、抽出、マッチングのための畳み込みニューラルネットワークの基本的な設計を再検討します。特に、リソースが限られたデバイス向けに迅速かつ堅牢なアルゴリズムが必要とされるため、解像度を可能な限り高く保ちながら、ネットワークのチャネル数を制限します。さらに、スパース下でのマッチングを選択できる設計となっており、ナビゲーションやARなどのアプリケーションに適しています。XFeatは、高速かつ同等以上の精度を実現し、一般的なラップトップのCPU上でリアルタイムで動作します。
More Related Content
Similar to End to-end convolutional semantic embeddings
KDD17Tutorial_final (1).pdf
This document summarizes research on learning representations of large-scale networks. It discusses how deep learning techniques can be used to learn low-dimensional embeddings that encode network structure and relationships between nodes. Convolutional and recurrent neural networks are proposed as methods to learn representations of entire networks by processing sequences of local neighborhood structures in an end-to-end fashion.
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang: two very promising beasts, made for the new parallel world, but still lingering in the wings. Not only are they addressing todays challenges but they are using parallel architectures as corner stone of their new and surprising approach to be faster and more productive. What are they good for? Why are we working to team them up?
Erlang promises faster implementation, way better maintenance and 4 times shorter code. VoltDB claims to be two orders of magnitude faster than its competitors. The two share many similarities: both are the result of scientific research and designed from scratch to address the new reality of parallel architectures with full force.
This talk presents the case for Erlang as server language, where it shines, how it looks, and how to get started. It details Erlang's secret sauce: microprocesses, actors, atoms, immutable variables, message passing and pattern matching. (Note: for a longer version of this treatment of Erlang only see: Why Erlang? http://www.slideshare.net/eonblast/why-erlang-gdc-online-2012)
VoltDB's inner workings are explained to understand why it can be so incredibly fast and still better than its NoSQL competitors. The well publicized Node.js benchmark clocking in at 695,000 transactions per second is described and the simple steps to get VoltDB up and running to see the prodigy from up close.
Source examples are presented that show Erlang and VoltDB in action.
The speaker is creator and maintainer of the Erlang VoltDB driver Erlvolt.
Mysql(2)
This document provides a summary of MySQL and PostgreSQL, two open-source database management systems. It compares their origins, features, platforms, and licensing. Key differences highlighted include that MySQL generally has simpler database design and replication while PostgreSQL has more complex rule sets and is faster. The document also provides contact information for getting further assistance.
Overview of Lincoln Paper Design
This set of slides has been presented to the Illinois Program for Research in the Humanities at the University of Illinois at Urbana-Champaign on 02-27-2009
#NetflixEverywhere Global Architecture
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1Rzjtjm.
Josh Evans talks about the Netflix journey of failure, innovation, and ubiquity. He reviews the many facets of globalization then delves deep into the architectural patterns that enable seamless, multi-region traffic management, reliable, fast data propagation, and efficient service infrastructure. The patterns presented are broadly applicable to Internet services with global aspirations. Filmed at qconlondon.com.
Josh Evans is Director of Operations Engineering at Netflix, with experience in e-commerce, playback control services, infrastructure, tools, testing, and operations.
JavaScript History
JavaScript was created in the 1990s at Netscape to enable interactivity in web browsers. It was influenced by languages like Self and Scheme. JavaScript was standardized by ECMA International to foster adoption beyond Netscape Navigator and Internet Explorer. The language has evolved through several standards over the years, with ES6/ES2015 being the latest version adding features like classes, modules, and iterators. New transpilers allow use of upcoming JavaScript features before full browser support.
OWF14 - Big Data : The State of Machine Learning in 2014
Olivier GRISEL
A broad overview of what technologies are usable now, what are just around the corner.
WCM-7 Surfing with CMIS
There is a high demand for companies to publish and promote their content on the web. To accommodate this demand Alfresco has provided a number of solutions covering editorial to web tier. As an example of this demand Ixxus was commissioned by a leading business information publisher to produce a microsite for ‘teaser’ content to increase subscriptions of their main site. To deliver this Ixxus utilized a number of features provided by Alfresco, such as services like the Transfer Service, the web scripts framework, and Surf. The majority of these features now make up the mainstay of Alfresco’s Web Quick Start WCM solution. The goal of this session is to demonstrate a real world example of how the combination of Alfresco, Surf and CMIS offers a great platform for developers to produce content-rich websites quickly. The session will cover: Using Spring Roo to construct a Surf application, Benefits of using Spring Surf, Using the Transfer Service, OpenCMIS in Surf, Varnish your Surf application, and What’s next
Historical aerial scanning project
This document summarizes a project by the Texas Water Development Board and TNRIS interns to digitize and georeference historical coastal aerial photographs from 1940-1990. The interns will scan over 5,000 photos per month to preserve them and prevent damage. The georeferenced scans will be made available online. The process involves scanning, georeferencing, mosaicking and enhancing the images. Stakeholders benefit from preserved history and access to coastal change data. Interns gain experience that improves employment prospects.
Cassandra Summit 2014: Active-Active Cassandra Behind the Scenes
Presenter: Roopa Tangirala, Senior Cloud Data Architect at Netflix
High availability is an important requirement for any online business and trying to architect around failures and expecting infrastructure to fail, and even then be highly available, is the key to success. One such effort here at Netflix was the Active-Active implementation where we provided region resiliency. This presentation will discuss the brief overview of the active-active implementation and how it leveraged Cassandra’s architecture in the backend to achieve its goal. It will cover our journey through A-A from Cassandra’s perspective, the data validation we did to prove the backend would work without impacting customer experience. The various problems we faced, like long repair times and gc_grace settings, plus lessons learned and what would we do differently next time around, will also be discussed.
Technologies For Appraising and Managing Electronic Records
This document summarizes technologies for appraising and managing electronic records, including discovering relationships among digital file collections and comparing document versions. It presents three technologies: file2learn to discover relationships between files based on metadata extraction and analysis; doc2learn for comprehensive document comparisons; and Polyglot for automated file format conversion and quality assessment.
Samsung voice intelligence.v5.5
This document discusses the development of Bixby, Samsung's intelligent virtual assistant. It describes Bixby as being fundamentally different from other assistants due to its ability to seamlessly switch between voice and touch modes, its context awareness, and its ability to understand incomplete commands. The document outlines some of the key challenges in developing Bixby, including managing its massive contextual input space and variable output capabilities across devices and versions. It discusses the use of deep learning and other techniques to address these challenges.
Devops at Netflix (re:Invent)
How Netflix operates for maximum freedom and agility.
Video here: https://www.youtube.com/watch?v=s0rCGFetdtM
CommCon 2018 - Realtime Machine Learning
This document discusses several applications of real-time machine learning including speech recognition, text-to-speech, image super resolution, noise suppression, network optimization for real-time communications, and selective video encoding. It also considers potential issues like voice copying, deep fakes, and privacy concerns that could arise from these technologies.
Closing Keynote
Dr. Jim Webber presented the closing keynote at a conference. He recapped Neo4j's recent releases, including version 3.0 which delivered new graph capabilities. He discussed future hardware trends and how native graph technology provides performance advantages. He also showed how Neo4j can handle workloads at massive scales of over 1 trillion relationships and 1 million writes per second on a single machine. Finally, he outlined Neo4j's new causal clustering architecture in version 3.1 which provides security, scalability, and fault tolerance for enterprise graph applications.
Be faster then rabbits
The document discusses various topics related to web development including Java principles, Spring frameworks, PHP, high-load web applications, mobile backend as a service (mBaas), web frameworks, Java web development frameworks like JSF and GWT, rendering on the server-side vs client-side, distribution of work between designers and developers, web browsers and their support for HTML5 and CSS3, programming languages, GUI frameworks, AngularJS, testing tools like JUnit, and build tools like Maven, Ant, and Ivy.
สร้างสรรค์กิจกรรมการเรียนรู้ที่มีความหมาย ด้วยสื่อ Learning Object
The document discusses various aspects of designing effective learning objects (LOs) for teaching mathematics at the secondary school level. It covers topics like LO metadata, instructional design approaches, storyboarding, feedback mechanisms, usability testing, and technical considerations. The key goals of LOs are to transition students from passive to active learning, bridge the digital divide, and close gaps in understanding through interactive, personalized instructional experiences.
Lessons Learnt From Working With Rails
As given at Scotland On Rails 2008.
(note: the highlighting and layering hasn't come through so well in Slideshares conversion process. I'll probably put a version to download from my site to solve this)
R for the semantic web, Quesada useR 2009
This document discusses using R to analyze and work with semantic web data represented as graphs. It introduces key concepts like ontologies, RDF, and linked data. It also describes how graph analysis techniques like centrality measures and PageRank can be applied to semantic web data to identify important nodes. Developing tools for semantic web data in R is challenging because current software uses an object-oriented approach while semantic data is represented as triples in graphs; an R package is needed to directly handle RDF.
AWS re:Invent 2016: Design Patterns for High Availability: Lessons from Amazo...
This document discusses design patterns for high availability used by Amazon CloudFront. It describes four key patterns: 1) FoodTasting which involves deploying changes incrementally to a small subset of servers first, 2) handling flash crowds by caching content, serving only necessary content, and using scheduled auto scaling, 3) implementing defense in depth strategies like multi-implementation and sharding to reduce the blast radius of failures, and 4) protecting against time bombs by jittering deployments and configurations across servers to avoid homogeneous outages. The document provides examples of how these patterns have been implemented in CloudFront and AWS services.
Similar to End to-end convolutional semantic embeddings (20)
OWF14 - Big Data : The State of Machine Learning in 2014
OWF14 - Big Data : The State of Machine Learning in 2014
Cassandra Summit 2014: Active-Active Cassandra Behind the Scenes
Cassandra Summit 2014: Active-Active Cassandra Behind the Scenes
Technologies For Appraising and Managing Electronic Records
Technologies For Appraising and Managing Electronic Records
สร้างสรรค์กิจกรรมการเรียนรู้ที่มีความหมาย ด้วยสื่อ Learning Object
สร้างสรรค์กิจกรรมการเรียนรู้ที่มีความหมาย ด้วยสื่อ Learning Object
AWS re:Invent 2016: Design Patterns for High Availability: Lessons from Amazo...
AWS re:Invent 2016: Design Patterns for High Availability: Lessons from Amazo...
More from harmonylab
Generating Automatic Feedback on UI Mockups with Large Language Models
Generating Automatic Feedback on UI Mockups with Large Language Modelsを和訳紹介したものです
【DLゼミ】XFeat: Accelerated Features for Lightweight Image Matching
公開URL：https://arxiv.org/pdf/2404.19174
出典：Guilherme Potje, Felipe Cadar, Andre Araujo, Renato Martins, Erickson R. ascimento: XFeat: Accelerated Features for Lightweight Image Matching, Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
概要：リソース効率に優れた特徴点マッチングのための軽量なアーキテクチャ「XFeat（Accelerated Features）」を提案します。手法は、局所的な特徴点の検出、抽出、マッチングのための畳み込みニューラルネットワークの基本的な設計を再検討します。特に、リソースが限られたデバイス向けに迅速かつ堅牢なアルゴリズムが必要とされるため、解像度を可能な限り高く保ちながら、ネットワークのチャネル数を制限します。さらに、スパース下でのマッチングを選択できる設計となっており、ナビゲーションやARなどのアプリケーションに適しています。XFeatは、高速かつ同等以上の精度を実現し、一般的なラップトップのCPU上でリアルタイムで動作します。
【修士論文】代替出勤者の選定業務における依頼順決定方法に関する研究 千坂知也
シフト勤務制の職場における従業員の急な欠勤は業務に支障をきたし，顧客などに対する信頼の失墜などにより，事業そのものに深刻な支障をきたす恐れがある．
このような急な欠勤に対し，管理者は代わりの出勤を他の従業員に依頼し，代替出勤者の選定を行う必要がある．この代替出勤者の選定業務における従来手法では，代替出勤者の選定後に発生した勤務表制約違反を解消するために勤務表全体を再作成するアプローチがとられてきた．このアプローチは厳しい勤務表制約条件が課される病院のような職場では有効とされてきたが，管理者は再作成した勤務表を再度全従業員に周知する必要があり，従業員側からしても自分以外の従業員の欠勤により自身の勤務予定が変更される可能性があるため，欠勤者が多発する職場では扱いにくい．そのため欠勤者が多発する職場では，代替出勤を行っても勤務表制約条件に違反しない従業員を事前に割り出しておき，それらの従業員に対し代替出勤依頼を行う手法がとられている．この手法では管理者の従業員に対する依頼順が業務の安定性，管理者や従業員の負担に直結する．
以上より本研究では管理者による従業員の代替出勤依頼の依頼順決定方法を提案し，代替出勤シミュレーターを構築して評価を行う．シミュレーターでは，従業員の休み希望のもと，複数の制約条件に基づき勤務表を作成し，勤務表上で確率的に欠勤を発生させ，管理者による代替出勤依頼をシミュレーションする．各従業員は管理者からの依頼に対し，確率的に可否を応答するものとする．本研究ではこのシミュレーター上で複数のパラメータセットに対し，従業員の受諾確率に基づいた方法，各従業員の代替出勤した回数に基づいた方法，代替出勤可能な日数に基づいた方法の提案、評価を行う．
【修士論文】経路探索のための媒介中心性に基づく道路ネットワーク階層化手法に関する研究
本研究では，ドライバーの細街路に対する選好を考慮した経路探索における道路ネットワークの階層化手法を提案する．提案手法では，ネットワーク分析の中心性指標の一つである媒介中心性の高いノードを特定して，各階層の非連結なサブネットワークを連結する．提案手法の有効性を検証するために，複数のドライバーの細街路への選好を用意し，札幌市の中心市街14km四方の道路ネットワークを対象とした計算機実験をおこなう．提案手法および先行手法における経路探索にかかる計算時間と得られた経路のコストを比較した結果，提案手法の優位性が確認された．
A Study on Decision Support System for Snow Removal Dispatch using Road Surfa...
This study focuses on addressing the challenges associated with decision-making in winter road snow removal operations, aiming to alleviate the burden on snow removal personnel. Specifically, we propose an approach to develop a system that collects and visualizes information on road snow conditions and weather data to support decision-making by personnel. Additionally, by sharing the collected information, we aim to facilitate the sharing of premonitions about changes in decision-making among snow removal personnel, reducing the need for physical inspections.We have validated the effectiveness of the system and confirmed its efficacy.
【卒業論文】印象タグを用いた衣服画像生成システムに関する研究
オーダーメイド服は個性を表現できる利点があるが、専門家以外は希望やこだわりが曖昧であったり、想像しているデザインの説明が難しかったりするために、デザインの注文が難しいという課題がある。
そこで、印象タグを元に衣服画像を作成する、対話型の衣服画像生成システムの開発を行い、個人の好みを反映した衣服のデザイン画を専門家以外が作成する場合の支援を試みた。
本研究ではシステムの構築を行い、性能評価として被験者に実際に操作してもらうことで、
ユーザーの発想やデザインの伝達に関する支援が可能であることを示した。
【卒業論文】大規模言語モデルを用いたマニュアル文章修正手法に関する研究
マニュアルに記載する文章では，読み手が内容を素早く明確に理解できることが重要である．そのための文章表現の技術を習得することは容易ではない．本論文では，マニュアルに適する文章を作成する支援のために，入力文章にライティングルールを適用する文章変換の手法を提案する．結果として，ライティングルールに基づく修正箇所の指摘と，修正候補の文章の生成ができた．これによって，マニュアルに適した文章を書きやすくなる．
DLゼミ: MobileOne: An Improved One millisecond Mobile Backbone
公開URL：https://openaccess.thecvf.com/content/CVPR2023/html/Vasu_MobileOne_An_Improved_One_Millisecond_Mobile_Backbone_CVPR_2023_paper.html
出典：Vasu, Pavan Kumar Anasosalu, et al.: MobileOne: An Improved One Millisecond Mobile Backbone, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)
概要：モバイル端末向けのニューラルネットワークは多くの場合、FLOPsやパラメータ数で最適化されています。しかし、これらの最適化は実際のモバイルデバイスで実行した場合のネットワークの応答時間に相関しない場合があります。我々は昨今のニューラルネットワークの最適化のボトルネックを特定・分析し、その結果をもとにした新たな効率的なバックボーンMobileOneを設計しました。結果はMobileFormerと同等の性能を得ながら、38倍高速であり、最先端の効率性を達成しました。
DLゼミ: Llama 2: Open Foundation and Fine-Tuned Chat Models
公開URL：https://arxiv.org/pdf/2307.09288.pdf
出典：Touvron Hugo, et al.: Llama 2: Open foundation and fine-tuned chat models, arXiv preprint arXiv:2307.09288 (2023)
概要：70億から700億のパラメータを持つ大規模言語モデル（LLM）の事前学習であるLlama 2を開発し、リリースしました。Llama 2-Chatと呼ばれるファインチューニングされたLLMは、対話のユースケースに最適化されています。提案モデルは、検証したほとんどのベンチマークにおいて、オープンソースのチャットモデルを凌駕しており、有用性と安全性に関する人間による評価に基づいて、クローズドソースのモデルの適切な代替となる可能性があります。コミュニティが我々の研究を基に、LLMの責任ある開発に貢献できるようにすることも目的にあります。
DLゼミ: ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation
公開URL：https://proceedings.neurips.cc/paper_files/paper/2022/file/fbb10d319d44f8c3b4720873e4177c65-Paper-Conference.pdf
出典：Yufei Xu, Jing Zhang, Qiming Zhang, Dacheng Tao : ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation, Advances in Neural Information Processing Systems 35 (NeurIPS 2022), pp. 38571-38584 (2022)
概要：姿勢推定はコンピュータビジョンの1タスクとして知られています。本論文ではプレーンなVision Transformerを使った姿勢推定手法「ViTPose」を提案します。ViTPoseは非階層的なVision Transformerを特徴抽出のために使用し、100Mから1Bのパラメータにスケールアップ可能です。このモデルは事前学習や、複数のポーズタスク対応などにおいて柔軟性を持ち、大規模モデルの知識を小規模モデルに転送することも可能です。実験結果は、ViTPoseがMS COCO Keypoint DetectionベンチマークにおいてSoTAを達成しました。
Voyager: An Open-Ended Embodied Agent with Large Language Models
公開URL：https://arxiv.org/abs/2305.16291
出典：Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar : Voyager: An Open-Ended Embodied Agent with Large Language Models, arXiv: 2305.16291 (2023)
概要：本論文では、Minecraftの世界を持続的に探索し、多様なスキルを獲得し、新たな発見を人間の介入なく行う最初の大規模言語モデル（LLM）を駆使した具現化エージェントであるVoyagerを紹介する。Voyagerは、探索を最大化する自動カリキュラム、複雑な行動を格納・取得するためのスキルライブラリ、そして環境のフィードバック、実行エラー、プログラム改善のための自己確認を組み込む新たな反復的なプロンプトメカニズムの3つの主要なコンポーネントから成り立つ。
DLゼミ: Ego-Body Pose Estimation via Ego-Head Pose Estimation
公開URL：https://arxiv.org/abs/2212.04636
出典：Jiaman Li, C. Karen Liu, Jiajun Wu
: Ego-Body Pose Estimation via Ego-Head Pose Estimation, arXiv preprint arXiv:2212.04636 (2022)
概要：人間の行動理解やVR/ARへの応用において、一人称映像から人の3次元姿勢 (3D human motion)を推定することは重要である。提案手法EgoEgoは、中間表現として頭部モーションを採用した新たな手法である。問題を2分割したことにより、既存のデータセットが活用でき、他のSoTA手法より優れた性能を示した。さらにベンチマークのための一人称映像と人の3次元推定データセットARESを開発した。
ReAct: Synergizing Reasoning and Acting in Language Models
公開URL：https://arxiv.org/abs/2210.03629
出典：Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao : ReAct: Synergizing Reasoning and Acting in Language Models, arXiv: 2210.03629 (2022)
概要：本論文では大規模言語モデルにおいて推論(Reasoning)と行動(Acting)を組合わせるprompt手法であるReActを提案した。知識集約型の推論タスク（QAタスク・事実検証タスク）と意思決定タスク（テキストゲーム、ウェブナビゲーション）でReActの性能を評価した。推論タスクにおいてReActはWikipedia APIとの対話によりChain of ThoughtのHallucinationを軽減し、意思決定タスクにおいては模倣学習や強化学習による手法を上回る性能を示した。
形態素解析を用いた帝国議会議事速記録の変遷に関する研究
対話形式議事録は，口語での会話内容が記録されており，言語研究において重要な資料となる．対話形式議事録の1つである帝国議会議事速記録は，記録期間が長く分量も多いという特徴があり，近代における言語表現の変化を分析するのに適した資料であるといえる．
このようなテキストデータに対して分析の第一段階である表現抽出を自動で行う手法については知見が少なく，抽出作業のほとんどが手作業で行われているのが現状である．大規模なデータに対して手作業での該当表現の抽出を行うのには多大な労力がかかり，見逃しも発生しやすく，分析を行う上でのボトルネックとなる．
本研究では，抽出段階の自動化を目的とし，可能の意味を表す可能表現について，品詞の組み合わせマッチングによる自動抽出を試みた．さらに，可能表現の出現頻度や変遷と会議の話題の関係性を分析するための先駆けとして，重要単語抽出による話題の自動タグ付けを行った．話題タグの自動付与においては，TF-IDFを用いたシンプルな特徴単語の抽出を試みた．可能表現の抽出においては，文型に注目した品詞のパターンマッチングによる抽出を試みた．
【卒業論文】深層生成モデルを用いたユーザ意図に基づく衣服画像の生成に関する研究
「かわいい」や「フェミニン」といった，印象を表す抽象的なファッション用語に基づいて，衣服画像の生成や編集をすることで，ユーザ意図の正確な伝達を支援することを目的とした研究．
灯油タンク内の液面高計測を用いた 灯油残量推定システムに関する研究
近年，家庭への灯油配送においてリモートセンシング化が進んでおり，灯油タンク内の残量計測が可能な通信型レーザセンサの開発が進んでいる．しかし，屋外にある灯油タンクにセンサを設置するため計測環境が不安定で，コスト面から導入可能なデバイスの性能にも制約があるため，センサの故障や異常値の計測が多く発生する．このような背景を踏まえて，本論文では，センサを管理・運用するための運用手法の確立を目的として，2つの異常検知手法と1つの残量推定手法を提案し，実データを用いた有効性の検証を行う．
深層自己回帰モデルを用いた俳句の生成と評価に関する研究
論文では，人工知能による創作の第一歩として俳句，その中でも特に一般的な有季定型句を対象とし，深層学習モデルを用いた生成と評価を行う.
既存の俳句や日本語テキストを用いてモデルを学習し，俳句生成器を構築する.
また，作成したモデルの俳句生成器としての性能を評価するために，俳人による評価済みの俳句からなる評価用データセットを作成して，有季定型句の生成における各モデルの性能を比較する.
評価用データセットの作成においてはまず，有季定型句の評価項目として句会における評価方式と俳句内の句節のつながりに着目し，これらを段階的評価の階層構造として定義した.
定義した階層構造に基づくラベル付を複数の俳人に依頼し，モデル間の性能評価に用いる.
また，各モデルが生成した文字列に対するアンケート調査を行い，現状のモデルの到達点を明らかにする.
競輪におけるレーティングシステムを用いた予想記事生成に関する研究
本研究では，競輪のレース結果予測の初心者が車券を購入する際の意思決定を補助することを目的として，レース結果を予測して，予測結果を用いた選手紹介文を生成するシステムを構築する．選手の戦績から的確に反映する特徴量を算出する手法として，多数人の同時対戦を考慮した拡張イロレーティングシステムを提案して，競輪のレース結果予測に適用する．実際の競輪のレース結果を用いた検証実験において，既存のレース結果予測手法に比べて，拡張イロレーティングシステムを用いることで予測精度を向上可能であることを確認した．さらに，拡張イロレーティングシステムを紹介文生成に用いることで，既存の手法に比べて，初心者の車券購入により有用な文章を生成可能であることを示す．
【卒業論文】B2Bオークションにおけるユーザ別 入札行動予測に関する研究
中古ブランド品の B2B オークションでは，多数のアイテム
のオークションが並列して実施されるため，参加者はすべてのアイテムの価値を検討することが困難であり，入札するアイテムの選択や，留保価格の適切な設定が課題である．本稿では，こうした課題を入札アイテムの推薦や留保価格に対する不落札率の推定により解決するため，オークションにおける多関係推薦モデル MultiRec を用いたユーザ別入札行動予測をオークションデータに適用し，その精度から利用可能性を検証した．
More from harmonylab (20)
Generating Automatic Feedback on UI Mockups with Large Language Models
Generating Automatic Feedback on UI Mockups with Large Language Models
【DLゼミ】XFeat: Accelerated Features for Lightweight Image Matching
【DLゼミ】XFeat: Accelerated Features for Lightweight Image Matching
A Study on Decision Support System for Snow Removal Dispatch using Road Surfa...
A Study on Decision Support System for Snow Removal Dispatch using Road Surfa...
DLゼミ：Primitive Generation and Semantic-related Alignment for Universal Zero-S...
DLゼミ：Primitive Generation and Semantic-related Alignment for Universal Zero-S...
DLゼミ: MobileOne: An Improved One millisecond Mobile Backbone
DLゼミ: MobileOne: An Improved One millisecond Mobile Backbone
DLゼミ: Llama 2: Open Foundation and Fine-Tuned Chat Models
DLゼミ: Llama 2: Open Foundation and Fine-Tuned Chat Models
DLゼミ: ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation
DLゼミ: ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation
Voyager: An Open-Ended Embodied Agent with Large Language Models
Voyager: An Open-Ended Embodied Agent with Large Language Models
DLゼミ: Ego-Body Pose Estimation via Ego-Head Pose Estimation
DLゼミ: Ego-Body Pose Estimation via Ego-Head Pose Estimation
ReAct: Synergizing Reasoning and Acting in Language Models
ReAct: Synergizing Reasoning and Acting in Language Models
Recently uploaded
Iron and Steel Technology Roadmap - Towards more sustainable steelmaking.pdf
Iron and Steel Technology towards Sustainable Steelmaking
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
Modelagem de um CSTR com reação endotermica.pdf
Modelagem em função de transferencia. CSTR não-linear.
bank management system in java and mysql report1.pdf
truth is high but higher still is truth full living
Advanced control scheme of doubly fed induction generator for wind turbine us...
This paper describes a speed control device for generating electrical energy on an electricity network based on the doubly fed induction generator (DFIG) used for wind power conversion systems. At first, a double-fed induction generator model was constructed. A control law is formulated to govern the flow of energy between the stator of a DFIG and the energy network using three types of controllers: proportional integral (PI), sliding mode controller (SMC) and second order sliding mode controller (SOSMC). Their different results in terms of power reference tracking, reaction to unexpected speed fluctuations, sensitivity to perturbations, and resilience against machine parameter alterations are compared. MATLAB/Simulink was used to conduct the simulations for the preceding study. Multiple simulations have shown very satisfying results, and the investigations demonstrate the efficacy and power-enhancing capabilities of the suggested control system.
6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024) will provide an excellent international forum for sharing knowledge and results in theory, methodology and applications of on Machine Learning & Applications.
Recycled Concrete Aggregate in Construction Part III
Using recycled concrete aggregates (RCA) for pavements is crucial to achieving sustainability. Implementing RCA for new pavement can minimize carbon footprint, conserve natural resources, reduce harmful emissions, and lower life cycle costs. Compared to natural aggregate (NA), RCA pavement has fewer comprehensive studies and sustainability assessments.
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMS
The smart irrigation system represents an innovative approach to optimize water usage in agricultural and landscaping practices. The integration of cutting-edge technologies, including sensors, actuators, and data analysis, empowers this system to provide accurate monitoring and control of irrigation processes by leveraging real-time environmental conditions. The main objective of a smart irrigation system is to optimize water efficiency, minimize expenses, and foster the adoption of sustainable water management methods. This paper conducts a systematic risk assessment by exploring the key components/assets and their functionalities in the smart irrigation system. The crucial role of sensors in gathering data on soil moisture, weather patterns, and plant well-being is emphasized in this system. These sensors enable intelligent decision-making in irrigation scheduling and water distribution, leading to enhanced water efficiency and sustainable water management practices. Actuators enable automated control of irrigation devices, ensuring precise and targeted water delivery to plants. Additionally, the paper addresses the potential threat and vulnerabilities associated with smart irrigation systems. It discusses limitations of the system, such as power constraints and computational capabilities, and calculates the potential security risks. The paper suggests possible risk treatment methods for effective secure system operation. In conclusion, the paper emphasizes the significant benefits of implementing smart irrigation systems, including improved water conservation, increased crop yield, and reduced environmental impact. Additionally, based on the security analysis conducted, the paper recommends the implementation of countermeasures and security approaches to address vulnerabilities and ensure the integrity and reliability of the system. By incorporating these measures, smart irrigation technology can revolutionize water management practices in agriculture, promoting sustainability, resource efficiency, and safeguarding against potential security threats.
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...University of Maribor
Slides from talk presenting:
Aleš Zamuda: Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapter and Networking.
Presentation at IcETRAN 2024 session:
"Inter-Society Networking Panel GRSS/MTT-S/CIS
Panel Session: Promoting Connection and Cooperation"
IEEE Slovenia GRSS
IEEE Serbia and Montenegro MTT-S
IEEE Slovenia CIS
11TH INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONIC AND COMPUTING ENGINEERING
3-6 June 2024, Niš, Serbia22CYT12-Unit-V-E Waste and its Management.ppt
Introduction- e - waste – definition - sources of e-waste– hazardous substances in e-waste - effects of e-waste on environment and human health- need for e-waste management– e-waste handling rules - waste minimization techniques for managing e-waste – recycling of e-waste - disposal treatment methods of e- waste – mechanism of extraction of precious metal from leaching solution-global Scenario of E-waste – E-waste in India- case studies.
International Conference on NLP, Artificial Intelligence, Machine Learning an...
International Conference on NLP, Artificial Intelligence, Machine Learning and Applications (NLAIM 2024) offers a premier global platform for exchanging insights and findings in the theory, methodology, and applications of NLP, Artificial Intelligence, Machine Learning, and their applications. The conference seeks substantial contributions across all key domains of NLP, Artificial Intelligence, Machine Learning, and their practical applications, aiming to foster both theoretical advancements and real-world implementations. With a focus on facilitating collaboration between researchers and practitioners from academia and industry, the conference serves as a nexus for sharing the latest developments in the field.
Recently uploaded (20)
ML Based Model for NIDS MSc Updated Presentation.v2.pptx
ML Based Model for NIDS MSc Updated Presentation.v2.pptx
Iron and Steel Technology Roadmap - Towards more sustainable steelmaking.pdf
Iron and Steel Technology Roadmap - Towards more sustainable steelmaking.pdf
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
bank management system in java and mysql report1.pdf
bank management system in java and mysql report1.pdf
Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...
6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)
Recycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part III
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMS
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMS
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
Manufacturing Process of molasses based distillery ppt.pptx
Manufacturing Process of molasses based distillery ppt.pptx
International Conference on NLP, Artificial Intelligence, Machine Learning an...
International Conference on NLP, Artificial Intelligence, Machine Learning an...
End to-end convolutional semantic embeddings
- 1. End-to-end Convolutional Semantic Embeddings DL 2018/11/1 M2
- 2. PAGE!! • • End-to-end Convolutional Semantic Embeddings • • Quanzeng You • Microsoft • Zhengyou Zhang • Tencent • Jiebo Luo • University of Rochester • • CVPR2018 1
- 3. PAGE!!2 Embedding Embedding CNN RNN CNN RNN Embedding Embedding RNN CNN Embedding CNN Embedding Embedding
- 4. PAGE!!3 • Textual Network • Global Intermediate • Intermediate Textual NetworkVisual Network
- 6. PAGE!! Textual Network 5 4 k 1D conv 1, 3, 4, 7 k-1
- 7. PAGE!! Textual Network 6 Highway Network Character- aware neural language models LSTM x z
- 8. PAGE!!7 Global Loss • • α CNN Embedding x Height x Width Flatten Global Loss Local Loss
- 9. PAGE!! Local Loss 8 ! " #$ #% Local Loss &% &$ '(#$) reshape
- 10. PAGE!! • • Sentence Retrieval • Image Retrieval • • MS-COCO • Train:82783, Validate:40504 • • Flicker30K • Train:29,783, Validate:1000, Test:1000 • MS-COCO • • Top-k accuracy k=1,5,10 • • α = 0.5 γ = 0 • Visual Network ImageNet ResNet 9 MS-COCO http://cocodataset.org Flicker30K http://web.engr.illinois.edu/~bplumme2/Flickr30kEntities/
- 11. PAGE!! MS-COCO 10 • Image Retrieval SoTA RRF-Net • Sentence Retrieval • Textual CNN RNN
- 12. PAGE!! Flicker30K 11 • DAN • DAN 448 x 448 256 x 256
- 13. PAGE!! Flicker30K 12 • RRF-Net Sentence Retrieval Image Retrieval • Visual Network Textual Network • RRF-Net HGLMM 18000 PCA 6000 End-to-End • 6000 CSE 1024
- 14. PAGE!!13 Embedding • • • Late Early • • Best • • Context !" # • Early (!" + &)/2 • Late #* & +* !" +,- * .* = (+* + +,- * )/2 • Best #* & !" 0 1* 1,- *
- 15. PAGE!!14 CSE Context • CSE Accuracy • Future Work
- 16. PAGE!!15 • Flicker30K Image Retrieval • CSE RRF-Net Flicker30K MS-COCO
- 17. PAGE!! • Embedding End-to-End NN • CNN Embedding • Textual Network • • • • NN SoTA • Image Retrieval SoTA • Sentence Retrieval • Future work • 16