Ir mcq-answering-system
•
3 likes•329 views
This document describes an MCQ answering system that uses textual entailment and text similarity measures. The system first preprocesses the question, passage and answer options to resolve references. It then retrieves relevant sentences from the passage and calculates similarity scores. The possible combinations of sentences and answer options are analyzed using textual entailment and similarity scoring. The option with the highest combined score is selected as the answer. System performance is evaluated using the c@1 metric which encourages reducing incorrect answers while maintaining correct ones.
Report
Share
Report
Share
Download to read offline
Recommended
IUI 2016 Presentation Slide
http://iui.acm.org/2016/program.html
AnalyticalInk: An Interactive Learning Environment for Math Word Problem Solving
Question Answering for Machine Reading Evaluation on Romanian and English
Question Answering for Machine Reading Evaluation on Romanian and EnglishFaculty of Computer Science
The document summarizes a group's participation in CLEF exercises for question answering tasks on Romanian and English documents from 2006-2010. It describes their system components, including background knowledge indexing, answer extraction, and results. Their 2011 system achieved an overall accuracy of 0.25 for Romanian and 0.21 for English, showing improvement over previous years but still needing ways to better select answers and handle numerical values and named entities.2013NCCMAT111 Syllabus
This document is a syllabus for a precalculus course titled "PreCalculus: Elementary Functions" that covers functions, including polynomial, rational, exponential, logarithmic and trigonometric functions. It will use the textbook "PreCalculus: Functions Modeling Change" and incorporate graphing calculator use. Students are expected to attend all classes, complete 15 homework assignments and 5 exams over the course of the summer semester. Daily classwork will involve the textbook, homework, notes, and a graphing calculator. Tutoring is available in the math learning center.
Mathematical Language Processing via Tree Embeddings
This document proposes a framework that uses tree embeddings to process mathematical language via encoding equations as trees. The framework includes a novel encoder-decoder architecture that learns representations of mathematical formulae. This approach achieves state-of-the-art performance on formula retrieval tasks by computing the similarity between embedding vectors of query and dataset equations. Future work will explore joint processing of math and text, deploying the system for textbook search, and using the embeddings for open-ended math problem solving.
Data Structures and Algorithm - Week 11 - Algorithm Analysis
This document discusses algorithm analysis and determining the time complexity of algorithms. It begins by defining an algorithm and noting that the efficiency of algorithms should be analyzed independently of specific implementations or hardware. The document then discusses analyzing the time complexity of various algorithms by counting the number of operations and expressing efficiency using growth functions. Common growth functions like constant, linear, quadratic, and exponential are introduced. The concept of asymptotic notation (Big O) for describing an algorithm's time complexity is also covered. Examples are provided to demonstrate how to determine the time complexity of iterative and recursive algorithms.
Paper Evaluation research methodology
Paper evaluation in research methodology subject to acquired the answer of preliminary questions by evaluation one example paper.
MCI-SIF PDA Thailand Project
This is one of three workshops conducted by the specialist team from MCI and SIF. This is part of MCI's community involvement programme. This is the workshop on secondary mathematics.
Data Structures and Algorithm - Week 4 - Trees, Binary Trees
This document discusses trees and binary trees. It covers tree terminology, tree traversals including preorder, inorder and postorder, implementing binary trees in Java, using binary trees to represent general trees, and applications of trees for data science. The next lecture will cover binary search trees, AVL trees, binary heaps and red black trees.
Recommended
IUI 2016 Presentation Slide
http://iui.acm.org/2016/program.html
AnalyticalInk: An Interactive Learning Environment for Math Word Problem Solving
Question Answering for Machine Reading Evaluation on Romanian and English
Question Answering for Machine Reading Evaluation on Romanian and EnglishFaculty of Computer Science
The document summarizes a group's participation in CLEF exercises for question answering tasks on Romanian and English documents from 2006-2010. It describes their system components, including background knowledge indexing, answer extraction, and results. Their 2011 system achieved an overall accuracy of 0.25 for Romanian and 0.21 for English, showing improvement over previous years but still needing ways to better select answers and handle numerical values and named entities.2013NCCMAT111 Syllabus
This document is a syllabus for a precalculus course titled "PreCalculus: Elementary Functions" that covers functions, including polynomial, rational, exponential, logarithmic and trigonometric functions. It will use the textbook "PreCalculus: Functions Modeling Change" and incorporate graphing calculator use. Students are expected to attend all classes, complete 15 homework assignments and 5 exams over the course of the summer semester. Daily classwork will involve the textbook, homework, notes, and a graphing calculator. Tutoring is available in the math learning center.
Mathematical Language Processing via Tree Embeddings
This document proposes a framework that uses tree embeddings to process mathematical language via encoding equations as trees. The framework includes a novel encoder-decoder architecture that learns representations of mathematical formulae. This approach achieves state-of-the-art performance on formula retrieval tasks by computing the similarity between embedding vectors of query and dataset equations. Future work will explore joint processing of math and text, deploying the system for textbook search, and using the embeddings for open-ended math problem solving.
Data Structures and Algorithm - Week 11 - Algorithm Analysis
This document discusses algorithm analysis and determining the time complexity of algorithms. It begins by defining an algorithm and noting that the efficiency of algorithms should be analyzed independently of specific implementations or hardware. The document then discusses analyzing the time complexity of various algorithms by counting the number of operations and expressing efficiency using growth functions. Common growth functions like constant, linear, quadratic, and exponential are introduced. The concept of asymptotic notation (Big O) for describing an algorithm's time complexity is also covered. Examples are provided to demonstrate how to determine the time complexity of iterative and recursive algorithms.
Paper Evaluation research methodology
Paper evaluation in research methodology subject to acquired the answer of preliminary questions by evaluation one example paper.
MCI-SIF PDA Thailand Project
This is one of three workshops conducted by the specialist team from MCI and SIF. This is part of MCI's community involvement programme. This is the workshop on secondary mathematics.
Data Structures and Algorithm - Week 4 - Trees, Binary Trees
This document discusses trees and binary trees. It covers tree terminology, tree traversals including preorder, inorder and postorder, implementing binary trees in Java, using binary trees to represent general trees, and applications of trees for data science. The next lecture will cover binary search trees, AVL trees, binary heaps and red black trees.
Computer Science XII - Hissan 2078
This document appears to be instructions for a computer science exam containing two sections - Group A with long answer questions and Group B with short answer questions. For Group A, students must answer 4 out of 5 questions, each worth 10 marks. The questions cover topics like strings, while loops, writing a program to find the sum and greatest number from user-input integers, differences between structures and unions, and classes in OOP. For Group B, students must answer 7 out of 15 questions, each worth 5 marks. The short answer questions cover topics such as system analysis in SDLC, differences between DDL and DML, network database models, normalization, computer networks and LANs, network topologies, OOP characteristics,
Data Structures and Algorithm - Week 9 - Search Algorithms
This document discusses search algorithms, including linear search and binary search. It covers:
1) Linear search sequentially scans an array to find a match, working for both sorted and unsorted arrays.
2) Binary search uses recursion to search a sorted array, eliminating half the search space each iteration, making it faster than linear search.
3) Implementation examples of linear and binary search in Java are provided, along with analysis of the time efficiency of each method.
Data Structures and Algorithm - Week 8 - Minimum Spanning Trees
The document discusses minimum spanning trees and provides examples of Prim's and Kruskal's algorithms. It includes:
- A definition of minimum spanning tree as a subgraph that spans all nodes with minimum total edge weight.
- Characteristics of Prim's and Kruskal's algorithms such as working with undirected, weighted/unweighted graphs and producing optimal solutions greedily.
- A walk-through example of Prim's algorithm on a graph and calculating the minimum spanning tree cost.
UMAP 2017 - Fine-Grained Open Learner Models
This document describes two studies that evaluated different versions of an open learner model (OLM) called a Rich-OLM. The goal was to balance complexity versus support for learners. Study 1 found the basic Rich-OLM motivated exploration and helped understand content relations. Study 2 compared a basic, gauge (added interpretation aid), and social comparison version. Results showed adding details increased complexity but a gauge helped by aiding interpretation. Social comparison decreased perceived usefulness. Overall, Rich-OLMs can provide fine-grained learner models but complexity must be addressed through visualization aids.
Comparisons of linear goal programming algorithms
This document compares different algorithms for solving linear goal programming problems:
1) Lee's modified simplex algorithm from 1972 and Ignizio's sequential algorithm from 1976 are two commonly used algorithms but require many columns and objective function rows, adding to computational time.
2) Orumie and Ebong developed a new algorithm in 2011 utilizing modified simplex procedures that has better computational times than existing algorithms for all problems tested.
3) The document reviews several other goal programming algorithms and finds that Orumie and Ebong's new method provides the best reduction in computational time for solving the problems.
NCC Math 122 Summer Session II Syllabus 2011
This syllabus outlines a 4 credit calculus course offered over a 5 week summer session. Topics covered include limits, derivatives, integrals, and differential equations. Students will use a TI-83/84 calculator and Hughes-Hallett 5th edition textbook. Requirements include 100% attendance, 15 homework assignments, and 5 cumulative exams. Students are also expected to utilize the math center for additional study. The course is divided into 5 weeks, with each week focusing on a new chapter and including a test on Fridays.
On e-Assessment
The document discusses e-assessment and the opportunities and challenges of using symbolic systems for mathematical input. It provides examples of how symbolic systems can assess student input for symbolic equivalence. While symbolic systems offer benefits like varied question types and assessing symbolic input, challenges remain in processing complex mathematical input. Analysis of student responses to test questions on specific mathematical concepts provides insights into their skills and knowledge in areas like algebra and fractions. Opportunities for e-assessment include use of digital pens, intelligent feedback, and diagrammatic or mathematical input.
Dms introduction Sharmila Chidaravalli
Discrete mathematics is the study of mathematical structures that are discrete rather than continuous. It has many applications in computer science, as computers use discrete structures to represent and manipulate data. Some key topics in discrete mathematics include logic and proofs, number theory, counting, and graph theory. These topics help with problem solving, algorithms, data structures, cryptography, and the design of efficient computer systems and networks. Examples of problems solved with discrete mathematics include the fastest sorting algorithms, map coloring, efficient data transmission, electronic health records, and Google Maps routing.
Natural Language Processing to Improve Student Engagement featuring Dr. Rebec...
Natural Language Processing to Improve Student Engagement featuring Dr. Rebec...Penn State EdTech Network
Dr. Rebecca (Becky) Passonneau, Professor at the Penn State School of Electrical Engineering and Computer Science, shares her research on the application of artificial intelligence through natural language processing (NLP) to analyze the content of students' written work. This work has potential to enhance instruction in a number of impactful ways.
Dr. Passonneau shares results presented at her recent workshop, Connecting Language, Interaction and Education in Digital Environments (CLIEDE 2017; Supported by NSF, Penn State Teaching and Learning with Technology, and the Waterbury Chair of the Penn State College of Education).SelQA: A New Benchmark for Selection-based Question Answering
This paper presents a new selection-based question answering dataset, SelQA. The dataset consists of questions generated through crowdsourcing and sentence length answers that are drawn from the ten most prevalent topics in the English Wikipedia. We introduce a corpus annotation scheme that enhances the generation of large, diverse, and challenging datasets by explicitly aiming to reduce word co-occurrences between the question and answers. Our annotation scheme is composed of a series of crowdsourcing tasks with a view to more effectively utilize crowdsourcing in the creation of question answering datasets in various domains. Several systems are compared on the tasks of answer sentence selection and answer triggering, providing strong baseline results for future work to improve upon.
Level-based Resume Classification on Nursing Positions
This document summarizes a thesis on classifying nursing resumes by level of experience using machine learning models. It describes collecting a dataset of over 6,000 real nursing resumes from Emory University across four experience levels. The resumes were preprocessed by extracting text sections. Classification approaches included string matching on experience and education vectors, bag-of-words models, and ensemble models. The best accuracy was 61.88% using gradient boosting on feature vectors of experience and education sections. Error analysis identified annotation and file format issues for future improvement.
Vjai paper reading201808-acl18-simple-and_effective multi-paragraph reading c...
This paper proposes two approaches for document-level question answering: a pipeline approach and a confidence-based approach. The pipeline approach selects a single paragraph and extracts an answer from it. The confidence-based approach assigns confidence scores to answers from multiple paragraphs and returns the highest scoring answer. The paper experiments with different training methods for the confidence model and evaluates on several datasets, finding the shared normalization and no-answer option methods perform best. Error analysis shows the model still struggles with connecting statements across sentences and paragraphs.
Generation of Assessment Questions from Textbooks Enriched with Knowledge Models
The document describes a system that automatically generates assessment questions from textbooks enriched with knowledge models. It extracts relevant information from knowledge models, preprocesses it, selects sentences, determines question types, and constructs questions in different formats like true/false, cloze, and multiple choice. The system was evaluated on statistics textbooks, with questions rated on wording, assessment value, and difficulty by experts. While wording was generally good, the automatically generated questions had lower assessment value than handcrafted ones, especially for true/false and multiple choice questions. Gap and distractor quality were mixed, indicating room for improvement in question construction. Overall, the approach shows promise for automatic question generation but would benefit from additional training data and features.
How AI Helps Students Solve Math Problems
Learn from ST Unitas, a Korean education technology company with over 60 brand names, including a recent merge with the Princeton Review, on how they leverage AWS machine learning (ML) services to enhance student learning.
LAK21 Data Driven Redesign of Tutoring Systems (Yun Huang)
This is the slides for our paper in LAK '21 conference:
Yun Huang, Nikki G. Lobczowski, J. Elizabeth Richey, Elizabeth A. McLaugh- lin, Michael W. Asher, Judith M. Harackiewicz, Vincent Aleven, and Kenneth R. Koedinger. 2021. A General Multi-method Approach to Data-Driven Re- design of Tutoring Systems. In LAK21: 11th International Learning Analytics and Knowledge Conference (LAK21), April 12–16, 2021, Irvine, CA, USA. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3448139.3448155
Abstract: Analytics of student learning data are increasingly important for continuous redesign and improvement of tutoring systems and courses. There is still a lack of general guidance on converting analytics into better system design, and on combining multiple methods to maximally improve a tutor. We present a multi-method approach to data-driven redesign of tutoring systems and its empirical evaluation. Our approach systematically combines existing and new learning analytics and instructional design methods. In particular, our methods involve identifying difficult skills and creating focused tasks for learning these difficult skills effectively following content redesign strategies derived from analytics. In our past work, we applied this approach to redesigning an algebraic modeling unit and found initial evidence of its effectiveness. In the current work, we extended this approach and applied it to redesigning two other tutor units in addition to a second iteration of redesigning the previously redesigned unit. We conducted a one-month classroom experiment with 129 high school students. Compared to the origi- nal tutor, the redesigned tutor led to significantly higher learning outcomes, with time mainly allocated to focused tasks rather than original full tasks. Moreover, it reduced over- and under-practice, yielded a more effective practice experience, and selected skills progressing from easier to harder to a greater degree. Our work provides empirical evidence of the effectiveness and generality of a multi-method approach to data-driven instructional redesign.
ML using MATLAB
This document provides an overview of machine learning clustering techniques, including agglomerative clustering and k-means clustering. It discusses how agglomerative clustering successively merges the closest pair of clusters until one cluster remains, while k-means clustering initializes cluster centroids randomly and alternates between data point assignment and centroid updates. The document also introduces the bag-of-words model for representing documents or images as histograms of words or visual words.
Towards Filling the Gap in Conversational Search: From Passage Retrieval to C...
Talk given at the 32nd ACM International Conference on Information and Knowledge Management (CIKM '23) in October 2023
A brief introduction to Searn Algorithm
SEARN is an algorithm for structured prediction that casts it as a sequence of cost-sensitive classification problems. It works by learning a policy to make incremental decisions that build up the full structured output. The policy is trained through an iterative process of generating cost-sensitive examples from sample outputs produced by the current policy, training a classifier on those examples, and interpolating the new policy with the previous one. This allows SEARN to learn the structured prediction task without requiring assumptions about the output structure, unlike approaches that make independence assumptions or rely on global prediction models.
[poster] A Compare-Aggregate Model with Latent Clustering for Answer Selection
[poster] A Compare-Aggregate Model with Latent Clustering for Answer SelectionSeoul National University
CIKM 2019
In this paper, we propose a novel method for a sentence-level answer-selection task that is one of the fundamental problems in natural language processing. First, we explore the effect of additional information by adopting a pretrained language model to compute the vector representation of the input text and by applying transfer learning from a large-scale corpus. Second, we enhance the compare-aggregate model by proposing a novel latent clustering method to compute additional information within the target corpus and by changing the objective function from listwise to pointwise. To evaluate the performance of the proposed approaches, experiments are performed with the WikiQA and TRECQA datasets. The empirical results demonstrate the superiority of our proposed approach, which achieve state-of-the-art performance on both datasets.
Machine Learning Fall, 2007 Course Information
This document provides information about a machine learning course including the instructor, prerequisite courses, grading policy, types of learning covered, and schedule. It also discusses well-posed learning problems, designing a learning system by choosing the training experience, target function, representation, learning algorithm, and issues in machine learning.
Machine Learning and Artificial Neural Networks.ppt
Machine learning and neural networks are discussed. Machine learning investigates how knowledge is acquired through experience. A machine learning model includes what is learned (the domain), who is learning (the computer program), and the information source. Techniques discussed include k-nearest neighbors algorithm, Winnow algorithm, naive Bayes classifier, decision trees, and reinforcement learning. Reinforcement learning involves an agent interacting with an environment to optimize outcomes through trial and error.
Active Learning on Question Answering with Dialogues
This thesis presents the design and architecture of an Active Learning system for Question Answering on Multiparty Dialogue. The goal of this system is to collect a robust Question Answering dataset and to improve the performance of the system on Question Answering challenges on Multiparty Dialogue. The system has an interactive web-based user interface which allows users to challenge the system with their own questions regarding a short passage of dialogues between multiple characters in a TV series. This system makes use of a state-of-art Machine Learning model to predict the answers to users' questions. In the same time, the system learns from users' responses and performs online update on the model. The system uses probability functions to guide user towards contributing data needed most for model improvement. The system is designed to handle heavy internet traffic by efficiently storing data and by carefully synchronizing the shared resources in the web system. The system has shown promising results in guiding users to contribute high quality data useful for model training.
More Related Content
What's hot
Computer Science XII - Hissan 2078
This document appears to be instructions for a computer science exam containing two sections - Group A with long answer questions and Group B with short answer questions. For Group A, students must answer 4 out of 5 questions, each worth 10 marks. The questions cover topics like strings, while loops, writing a program to find the sum and greatest number from user-input integers, differences between structures and unions, and classes in OOP. For Group B, students must answer 7 out of 15 questions, each worth 5 marks. The short answer questions cover topics such as system analysis in SDLC, differences between DDL and DML, network database models, normalization, computer networks and LANs, network topologies, OOP characteristics,
Data Structures and Algorithm - Week 9 - Search Algorithms
This document discusses search algorithms, including linear search and binary search. It covers:
1) Linear search sequentially scans an array to find a match, working for both sorted and unsorted arrays.
2) Binary search uses recursion to search a sorted array, eliminating half the search space each iteration, making it faster than linear search.
3) Implementation examples of linear and binary search in Java are provided, along with analysis of the time efficiency of each method.
Data Structures and Algorithm - Week 8 - Minimum Spanning Trees
The document discusses minimum spanning trees and provides examples of Prim's and Kruskal's algorithms. It includes:
- A definition of minimum spanning tree as a subgraph that spans all nodes with minimum total edge weight.
- Characteristics of Prim's and Kruskal's algorithms such as working with undirected, weighted/unweighted graphs and producing optimal solutions greedily.
- A walk-through example of Prim's algorithm on a graph and calculating the minimum spanning tree cost.
UMAP 2017 - Fine-Grained Open Learner Models
This document describes two studies that evaluated different versions of an open learner model (OLM) called a Rich-OLM. The goal was to balance complexity versus support for learners. Study 1 found the basic Rich-OLM motivated exploration and helped understand content relations. Study 2 compared a basic, gauge (added interpretation aid), and social comparison version. Results showed adding details increased complexity but a gauge helped by aiding interpretation. Social comparison decreased perceived usefulness. Overall, Rich-OLMs can provide fine-grained learner models but complexity must be addressed through visualization aids.
Comparisons of linear goal programming algorithms
This document compares different algorithms for solving linear goal programming problems:
1) Lee's modified simplex algorithm from 1972 and Ignizio's sequential algorithm from 1976 are two commonly used algorithms but require many columns and objective function rows, adding to computational time.
2) Orumie and Ebong developed a new algorithm in 2011 utilizing modified simplex procedures that has better computational times than existing algorithms for all problems tested.
3) The document reviews several other goal programming algorithms and finds that Orumie and Ebong's new method provides the best reduction in computational time for solving the problems.
NCC Math 122 Summer Session II Syllabus 2011
This syllabus outlines a 4 credit calculus course offered over a 5 week summer session. Topics covered include limits, derivatives, integrals, and differential equations. Students will use a TI-83/84 calculator and Hughes-Hallett 5th edition textbook. Requirements include 100% attendance, 15 homework assignments, and 5 cumulative exams. Students are also expected to utilize the math center for additional study. The course is divided into 5 weeks, with each week focusing on a new chapter and including a test on Fridays.
On e-Assessment
The document discusses e-assessment and the opportunities and challenges of using symbolic systems for mathematical input. It provides examples of how symbolic systems can assess student input for symbolic equivalence. While symbolic systems offer benefits like varied question types and assessing symbolic input, challenges remain in processing complex mathematical input. Analysis of student responses to test questions on specific mathematical concepts provides insights into their skills and knowledge in areas like algebra and fractions. Opportunities for e-assessment include use of digital pens, intelligent feedback, and diagrammatic or mathematical input.
Dms introduction Sharmila Chidaravalli
Discrete mathematics is the study of mathematical structures that are discrete rather than continuous. It has many applications in computer science, as computers use discrete structures to represent and manipulate data. Some key topics in discrete mathematics include logic and proofs, number theory, counting, and graph theory. These topics help with problem solving, algorithms, data structures, cryptography, and the design of efficient computer systems and networks. Examples of problems solved with discrete mathematics include the fastest sorting algorithms, map coloring, efficient data transmission, electronic health records, and Google Maps routing.
What's hot (8)
Data Structures and Algorithm - Week 9 - Search Algorithms
Data Structures and Algorithm - Week 9 - Search Algorithms
Data Structures and Algorithm - Week 8 - Minimum Spanning Trees
Data Structures and Algorithm - Week 8 - Minimum Spanning Trees
Similar to Ir mcq-answering-system
Natural Language Processing to Improve Student Engagement featuring Dr. Rebec...
Natural Language Processing to Improve Student Engagement featuring Dr. Rebec...Penn State EdTech Network
Dr. Rebecca (Becky) Passonneau, Professor at the Penn State School of Electrical Engineering and Computer Science, shares her research on the application of artificial intelligence through natural language processing (NLP) to analyze the content of students' written work. This work has potential to enhance instruction in a number of impactful ways.
Dr. Passonneau shares results presented at her recent workshop, Connecting Language, Interaction and Education in Digital Environments (CLIEDE 2017; Supported by NSF, Penn State Teaching and Learning with Technology, and the Waterbury Chair of the Penn State College of Education).SelQA: A New Benchmark for Selection-based Question Answering
This paper presents a new selection-based question answering dataset, SelQA. The dataset consists of questions generated through crowdsourcing and sentence length answers that are drawn from the ten most prevalent topics in the English Wikipedia. We introduce a corpus annotation scheme that enhances the generation of large, diverse, and challenging datasets by explicitly aiming to reduce word co-occurrences between the question and answers. Our annotation scheme is composed of a series of crowdsourcing tasks with a view to more effectively utilize crowdsourcing in the creation of question answering datasets in various domains. Several systems are compared on the tasks of answer sentence selection and answer triggering, providing strong baseline results for future work to improve upon.
Level-based Resume Classification on Nursing Positions
This document summarizes a thesis on classifying nursing resumes by level of experience using machine learning models. It describes collecting a dataset of over 6,000 real nursing resumes from Emory University across four experience levels. The resumes were preprocessed by extracting text sections. Classification approaches included string matching on experience and education vectors, bag-of-words models, and ensemble models. The best accuracy was 61.88% using gradient boosting on feature vectors of experience and education sections. Error analysis identified annotation and file format issues for future improvement.
Vjai paper reading201808-acl18-simple-and_effective multi-paragraph reading c...
This paper proposes two approaches for document-level question answering: a pipeline approach and a confidence-based approach. The pipeline approach selects a single paragraph and extracts an answer from it. The confidence-based approach assigns confidence scores to answers from multiple paragraphs and returns the highest scoring answer. The paper experiments with different training methods for the confidence model and evaluates on several datasets, finding the shared normalization and no-answer option methods perform best. Error analysis shows the model still struggles with connecting statements across sentences and paragraphs.
Generation of Assessment Questions from Textbooks Enriched with Knowledge Models
The document describes a system that automatically generates assessment questions from textbooks enriched with knowledge models. It extracts relevant information from knowledge models, preprocesses it, selects sentences, determines question types, and constructs questions in different formats like true/false, cloze, and multiple choice. The system was evaluated on statistics textbooks, with questions rated on wording, assessment value, and difficulty by experts. While wording was generally good, the automatically generated questions had lower assessment value than handcrafted ones, especially for true/false and multiple choice questions. Gap and distractor quality were mixed, indicating room for improvement in question construction. Overall, the approach shows promise for automatic question generation but would benefit from additional training data and features.
How AI Helps Students Solve Math Problems
Learn from ST Unitas, a Korean education technology company with over 60 brand names, including a recent merge with the Princeton Review, on how they leverage AWS machine learning (ML) services to enhance student learning.
LAK21 Data Driven Redesign of Tutoring Systems (Yun Huang)
This is the slides for our paper in LAK '21 conference:
Yun Huang, Nikki G. Lobczowski, J. Elizabeth Richey, Elizabeth A. McLaugh- lin, Michael W. Asher, Judith M. Harackiewicz, Vincent Aleven, and Kenneth R. Koedinger. 2021. A General Multi-method Approach to Data-Driven Re- design of Tutoring Systems. In LAK21: 11th International Learning Analytics and Knowledge Conference (LAK21), April 12–16, 2021, Irvine, CA, USA. ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3448139.3448155
Abstract: Analytics of student learning data are increasingly important for continuous redesign and improvement of tutoring systems and courses. There is still a lack of general guidance on converting analytics into better system design, and on combining multiple methods to maximally improve a tutor. We present a multi-method approach to data-driven redesign of tutoring systems and its empirical evaluation. Our approach systematically combines existing and new learning analytics and instructional design methods. In particular, our methods involve identifying difficult skills and creating focused tasks for learning these difficult skills effectively following content redesign strategies derived from analytics. In our past work, we applied this approach to redesigning an algebraic modeling unit and found initial evidence of its effectiveness. In the current work, we extended this approach and applied it to redesigning two other tutor units in addition to a second iteration of redesigning the previously redesigned unit. We conducted a one-month classroom experiment with 129 high school students. Compared to the origi- nal tutor, the redesigned tutor led to significantly higher learning outcomes, with time mainly allocated to focused tasks rather than original full tasks. Moreover, it reduced over- and under-practice, yielded a more effective practice experience, and selected skills progressing from easier to harder to a greater degree. Our work provides empirical evidence of the effectiveness and generality of a multi-method approach to data-driven instructional redesign.
ML using MATLAB
This document provides an overview of machine learning clustering techniques, including agglomerative clustering and k-means clustering. It discusses how agglomerative clustering successively merges the closest pair of clusters until one cluster remains, while k-means clustering initializes cluster centroids randomly and alternates between data point assignment and centroid updates. The document also introduces the bag-of-words model for representing documents or images as histograms of words or visual words.
Towards Filling the Gap in Conversational Search: From Passage Retrieval to C...
Talk given at the 32nd ACM International Conference on Information and Knowledge Management (CIKM '23) in October 2023
A brief introduction to Searn Algorithm
SEARN is an algorithm for structured prediction that casts it as a sequence of cost-sensitive classification problems. It works by learning a policy to make incremental decisions that build up the full structured output. The policy is trained through an iterative process of generating cost-sensitive examples from sample outputs produced by the current policy, training a classifier on those examples, and interpolating the new policy with the previous one. This allows SEARN to learn the structured prediction task without requiring assumptions about the output structure, unlike approaches that make independence assumptions or rely on global prediction models.
[poster] A Compare-Aggregate Model with Latent Clustering for Answer Selection
[poster] A Compare-Aggregate Model with Latent Clustering for Answer SelectionSeoul National University
CIKM 2019
In this paper, we propose a novel method for a sentence-level answer-selection task that is one of the fundamental problems in natural language processing. First, we explore the effect of additional information by adopting a pretrained language model to compute the vector representation of the input text and by applying transfer learning from a large-scale corpus. Second, we enhance the compare-aggregate model by proposing a novel latent clustering method to compute additional information within the target corpus and by changing the objective function from listwise to pointwise. To evaluate the performance of the proposed approaches, experiments are performed with the WikiQA and TRECQA datasets. The empirical results demonstrate the superiority of our proposed approach, which achieve state-of-the-art performance on both datasets.
Machine Learning Fall, 2007 Course Information
This document provides information about a machine learning course including the instructor, prerequisite courses, grading policy, types of learning covered, and schedule. It also discusses well-posed learning problems, designing a learning system by choosing the training experience, target function, representation, learning algorithm, and issues in machine learning.
Machine Learning and Artificial Neural Networks.ppt
Machine learning and neural networks are discussed. Machine learning investigates how knowledge is acquired through experience. A machine learning model includes what is learned (the domain), who is learning (the computer program), and the information source. Techniques discussed include k-nearest neighbors algorithm, Winnow algorithm, naive Bayes classifier, decision trees, and reinforcement learning. Reinforcement learning involves an agent interacting with an environment to optimize outcomes through trial and error.
Active Learning on Question Answering with Dialogues
This thesis presents the design and architecture of an Active Learning system for Question Answering on Multiparty Dialogue. The goal of this system is to collect a robust Question Answering dataset and to improve the performance of the system on Question Answering challenges on Multiparty Dialogue. The system has an interactive web-based user interface which allows users to challenge the system with their own questions regarding a short passage of dialogues between multiple characters in a TV series. This system makes use of a state-of-art Machine Learning model to predict the answers to users' questions. In the same time, the system learns from users' responses and performs online update on the model. The system uses probability functions to guide user towards contributing data needed most for model improvement. The system is designed to handle heavy internet traffic by efficiently storing data and by carefully synchronizing the shared resources in the web system. The system has shown promising results in guiding users to contribute high quality data useful for model training.
Active learning
This document discusses active learning. It begins with an overview and definition of active learning, contrasting it with passive learning. It then discusses applications of active learning in areas like speech recognition and medical diagnosis. Examples of active learning techniques like pool-based active learning are provided. The document examines whether active learning makes a difference compared to passive learning and discusses related human experiments. It explores active learning from multiple oracles, dealing with weak and strong oracles, and applications to crowdsourcing. The document concludes with discussing future directions and providing references.
Learning
The document discusses different types of machine learning, including supervised learning where algorithms are trained using labeled examples, unsupervised learning which explores unlabeled data to find structures, semi-supervised learning which uses both labeled and unlabeled data, and reinforcement learning where an agent learns through trial and error interactions with an environment. It also covers topics such as natural language processing, ensemble learning techniques like boosting and bagging, and applications of machine learning like image recognition, medical diagnosis, and fraud detection. The document provides an overview of key concepts in machine learning including how learning systems work and the different steps involved in natural language processing.
Naver learning to rank question answer pairs using hrde-ltc
The automatic question answering (QA) task has long been considered a primary objective of artificial intelligence.
Among the QA sub-systems, we focused on answer-ranking part. In particular, we investigated a novel neural network architecture with additional data clustering module to improve the performance in ranking answer candidates which are longer than a single sentence. This work can be used not only for the QA ranking task, but also to evaluate the relevance of next utterance with given dialogue generated from the dialogue model.
In this talk, I'll present our research results (NAACL 2018), and also its potential use cases (i.e. fake news detection). Finally, I'll conclude by introducing some issues on previous research, and by introducing recent approach in academic.
Stock market analysis using ga and neural network
its a presentation on stock market analysis using Genetic algorithm with Neural networks ,based on a scientific paper
,made in Cairo university under Supervision of prof.Dr. Magda
CS3114_09212011.ppt
This document provides an overview and introduction to the concepts taught in a data structures and algorithms course. It discusses the goals of reinforcing that every data structure has costs and benefits, learning commonly used data structures, and understanding how to analyze the efficiency of algorithms. Key topics covered include abstract data types, common data structures, algorithm analysis techniques like best/worst/average cases and asymptotic notation, and examples of analyzing the time complexity of various algorithms. The document emphasizes that problems can have multiple potential algorithms and that problems should be carefully defined in terms of inputs, outputs, and resource constraints.
[AIoTLab]attention mechanism.pptx
The document discusses attention mechanisms for encoder-decoder neural networks. It describes traditional encoder-decoder models that compress all input information into a fixed vector, which cannot encode long sentences. Attention mechanisms allow the decoder to access the entire encoded input sequence and assign weights to input elements based on their relevance to predicting the output. The core attention model uses an alignment function to calculate energy scores between the input and output, a distribution function to calculate attention weights from the energy scores, and a weighted sum to compute the context vector used by the decoder. Various alignment functions are discussed, including dot product, additive, and deep attention.
Similar to Ir mcq-answering-system (20)
Natural Language Processing to Improve Student Engagement featuring Dr. Rebec...
Natural Language Processing to Improve Student Engagement featuring Dr. Rebec...
SelQA: A New Benchmark for Selection-based Question Answering
SelQA: A New Benchmark for Selection-based Question Answering
Level-based Resume Classification on Nursing Positions
Level-based Resume Classification on Nursing Positions
Vjai paper reading201808-acl18-simple-and_effective multi-paragraph reading c...
Vjai paper reading201808-acl18-simple-and_effective multi-paragraph reading c...
Generation of Assessment Questions from Textbooks Enriched with Knowledge Models
Generation of Assessment Questions from Textbooks Enriched with Knowledge Models
LAK21 Data Driven Redesign of Tutoring Systems (Yun Huang)
LAK21 Data Driven Redesign of Tutoring Systems (Yun Huang)
Towards Filling the Gap in Conversational Search: From Passage Retrieval to C...
Towards Filling the Gap in Conversational Search: From Passage Retrieval to C...
[poster] A Compare-Aggregate Model with Latent Clustering for Answer Selection
[poster] A Compare-Aggregate Model with Latent Clustering for Answer Selection
Machine Learning and Artificial Neural Networks.ppt
Machine Learning and Artificial Neural Networks.ppt
Active Learning on Question Answering with Dialogues
Active Learning on Question Answering with Dialogues
Naver learning to rank question answer pairs using hrde-ltc
Naver learning to rank question answer pairs using hrde-ltc
More from Jishnu P
SinGAN - Learning a Generative Model from a Single Natural Image
SinGAN is a generative adversarial network (GAN) that can learn the distribution of a single natural image and generate new realistic samples from that image distribution. Unlike other GANs that require large datasets, SinGAN only needs a single image for training. It uses a multi-scale architecture with multiple generators and discriminators at different scales. SinGAN was shown to generate high quality samples for tasks like super resolution, image editing, and animation from a single image. It also has some failure cases like generating unrealistic samples at the boundaries.
Breaking CAPTCHAs using ML
The document discusses the evolution of CAPTCHAs from first generation distorted text to reCAPTCHAs that helped digitize books by using words humans could read but computers could not. It then discusses how NoCAPTCHA reCAPTCHA was developed to address issues like accessibility for those with disabilities. The document also summarizes a student project that used deep learning methods like CNNs and transfer learning to break single character CAPTCHAs with high accuracy, showing the need for more advanced CAPTCHAs.
Stencil computation research project presentation #1
The document discusses optimization techniques used in an auto-tuning framework for parallel multicore stencil computations. It describes loop unrolling, cache blocking, and arithmetic simplification implemented as AST transformations. Cache blocking exposes temporal locality and increases cache reuse by organizing data into blocks that fit in cache. The framework applies these serial optimizations and additional parallelization strategies by modifying the AST to reflect the chosen parallelization before code generation.
Btp 2017 presentation
This document describes a project that aims to maximize the power of hypothesis tests in single gene experiments with multiple cell types under a cost constraint. It formulates the problem and proposes a sequential sampling procedure to estimate the optimal sample size allocation across cell types. The procedure begins with an initial sample from each cell type and sequentially adds more samples until the estimated optimal sample size is reached for each cell type while ensuring the total cost is within the given budget. Graphical and tabular results from simulations evaluating the procedure are also presented.
Cs403 Parellel Programming Travelling Salesman Problem
This document discusses parallelizing the traveling salesman problem. It proposes two approaches: 1) dividing the permutations of routes among threads, having each calculate costs and return the minimum, and 2) sharing a variable for minimum cost across threads and updating it concurrently as threads evaluate routes. Pseudocode is provided for both approaches. Speedup calculations assume ideal parallelization but also account for reductions from unconnected cities.
Ansible Overview - System Administration and Maintenance
This presentation provides an overview of Ansible as an automation tool. Ansible has much more capabilities but here I have discussed its functionalities as an automation tool.
CS404 Pattern Recognition - Locality Preserving Projections
This document summarizes a presentation on Locality Preserving Projections (LPP), a dimensionality reduction technique. LPP preserves the local neighborhood structure of the data when projecting it to a lower-dimensional space. It constructs an adjacency graph representing the connections between nearby data points, chooses weights for these connections, and computes eigenmaps to perform the dimensionality reduction while maintaining locality. LPP is compared to Principal Component Analysis (PCA), the most common dimensionality reduction method, which aims to preserve global variance rather than local neighborhood structure.
More from Jishnu P (7)
SinGAN - Learning a Generative Model from a Single Natural Image
SinGAN - Learning a Generative Model from a Single Natural Image
Stencil computation research project presentation #1
Stencil computation research project presentation #1
Cs403 Parellel Programming Travelling Salesman Problem
Cs403 Parellel Programming Travelling Salesman Problem
Ansible Overview - System Administration and Maintenance
Ansible Overview - System Administration and Maintenance
CS404 Pattern Recognition - Locality Preserving Projections
CS404 Pattern Recognition - Locality Preserving Projections
Recently uploaded
Embedded machine learning-based road conditions and driving behavior monitoring
Car accident rates have increased in recent years, resulting in losses in human lives, properties, and other financial costs. An embedded machine learning-based system is developed to address this critical issue. The system can monitor road conditions, detect driving patterns, and identify aggressive driving behaviors. The system is based on neural networks trained on a comprehensive dataset of driving events, driving styles, and road conditions. The system effectively detects potential risks and helps mitigate the frequency and impact of accidents. The primary goal is to ensure the safety of drivers and vehicles. Collecting data involved gathering information on three key road events: normal street and normal drive, speed bumps, circular yellow speed bumps, and three aggressive driving actions: sudden start, sudden stop, and sudden entry. The gathered data is processed and analyzed using a machine learning system designed for limited power and memory devices. The developed system resulted in 91.9% accuracy, 93.6% precision, and 92% recall. The achieved inference time on an Arduino Nano 33 BLE Sense with a 32-bit CPU running at 64 MHz is 34 ms and requires 2.6 kB peak RAM and 139.9 kB program flash memory, making it suitable for resource-constrained embedded systems.
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMS
The smart irrigation system represents an innovative approach to optimize water usage in agricultural and landscaping practices. The integration of cutting-edge technologies, including sensors, actuators, and data analysis, empowers this system to provide accurate monitoring and control of irrigation processes by leveraging real-time environmental conditions. The main objective of a smart irrigation system is to optimize water efficiency, minimize expenses, and foster the adoption of sustainable water management methods. This paper conducts a systematic risk assessment by exploring the key components/assets and their functionalities in the smart irrigation system. The crucial role of sensors in gathering data on soil moisture, weather patterns, and plant well-being is emphasized in this system. These sensors enable intelligent decision-making in irrigation scheduling and water distribution, leading to enhanced water efficiency and sustainable water management practices. Actuators enable automated control of irrigation devices, ensuring precise and targeted water delivery to plants. Additionally, the paper addresses the potential threat and vulnerabilities associated with smart irrigation systems. It discusses limitations of the system, such as power constraints and computational capabilities, and calculates the potential security risks. The paper suggests possible risk treatment methods for effective secure system operation. In conclusion, the paper emphasizes the significant benefits of implementing smart irrigation systems, including improved water conservation, increased crop yield, and reduced environmental impact. Additionally, based on the security analysis conducted, the paper recommends the implementation of countermeasures and security approaches to address vulnerabilities and ensure the integrity and reliability of the system. By incorporating these measures, smart irrigation technology can revolutionize water management practices in agriculture, promoting sustainability, resource efficiency, and safeguarding against potential security threats.
ACEP Magazine edition 4th launched on 05.06.2024
This document provides information about the third edition of the magazine "Sthapatya" published by the Association of Civil Engineers (Practicing) Aurangabad. It includes messages from current and past presidents of ACEP, memories and photos from past ACEP events, information on life time achievement awards given by ACEP, and a technical article on concrete maintenance, repairs and strengthening. The document highlights activities of ACEP and provides a technical educational article for members.
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Batteries -Introduction – Types of Batteries – discharging and charging of battery - characteristics of battery –battery rating- various tests on battery- – Primary battery: silver button cell- Secondary battery :Ni-Cd battery-modern battery: lithium ion battery-maintenance of batteries-choices of batteries for electric vehicle applications.
Fuel Cells: Introduction- importance and classification of fuel cells - description, principle, components, applications of fuel cells: H2-O2 fuel cell, alkaline fuel cell, molten carbonate fuel cell and direct methanol fuel cells.
The Python for beginners. This is an advance computer language.
Python language is very important language at this time. we can easily understand this language by these notes.
Literature Review Basics and Understanding Reference Management.pptx
Three-day training on academic research focuses on analytical tools at United Technical College, supported by the University Grant Commission, Nepal. 24-26 May 2024
哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样
原版一模一样【微信：741003700 】【(csu毕业证书)查尔斯特大学毕业证硕士学历】【微信：741003700 】学位证，留信认证（真实可查，永久存档）offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原海外各大学 Bachelor Diploma degree, Master Degree Diploma
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
官方认证美国密歇根州立大学毕业证学位证书原版一模一样
原版一模一样【微信：741003700 】【美国密歇根州立大学毕业证学位证书】【微信：741003700 】学位证，留信认证（真实可查，永久存档）offer、雅思、外壳等材料/诚信可靠,可直接看成品样本，帮您解决无法毕业带来的各种难题！外壳，原版制作，诚信可靠，可直接看成品样本。行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备。十五年致力于帮助留学生解决难题，包您满意。
本公司拥有海外各大学样板无数，能完美还原海外各大学 Bachelor Diploma degree, Master Degree Diploma
1:1完美还原海外各大学毕业材料上的工艺：水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠。文字图案浮雕、激光镭射、紫外荧光、温感、复印防伪等防伪工艺。材料咨询办理、认证咨询办理请加学历顾问Q/微741003700
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
Understanding Inductive Bias in Machine Learning
This presentation explores the concept of inductive bias in machine learning. It explains how algorithms come with built-in assumptions and preferences that guide the learning process. You'll learn about the different types of inductive bias and how they can impact the performance and generalizability of machine learning models.
The presentation also covers the positive and negative aspects of inductive bias, along with strategies for mitigating potential drawbacks. We'll explore examples of how bias manifests in algorithms like neural networks and decision trees.
By understanding inductive bias, you can gain valuable insights into how machine learning models work and make informed decisions when building and deploying them.
New techniques for characterising damage in rock slopes.pdf
rock mass characterization and New techniques for characterising damage in rock slopes
A review on techniques and modelling methodologies used for checking electrom...
The proper function of the integrated circuit (IC) in an inhibiting electromagnetic environment has always been a serious concern throughout the decades of revolution in the world of electronics, from disjunct devices to today’s integrated circuit technology, where billions of transistors are combined on a single chip. The automotive industry and smart vehicles in particular, are confronting design issues such as being prone to electromagnetic interference (EMI). Electronic control devices calculate incorrect outputs because of EMI and sensors give misleading values which can prove fatal in case of automotives. In this paper, the authors have non exhaustively tried to review research work concerned with the investigation of EMI in ICs and prediction of this EMI using various modelling methodologies and measurement setups.
Harnessing WebAssembly for Real-time Stateless Streaming Pipelines
Traditionally, dealing with real-time data pipelines has involved significant overhead, even for straightforward tasks like data transformation or masking. However, in this talk, we’ll venture into the dynamic realm of WebAssembly (WASM) and discover how it can revolutionize the creation of stateless streaming pipelines within a Kafka (Redpanda) broker. These pipelines are adept at managing low-latency, high-data-volume scenarios.
Recycled Concrete Aggregate in Construction Part II
Using recycled concrete aggregates (RCA) for pavements is crucial to achieving sustainability. Implementing RCA for new pavement can minimize carbon footprint, conserve natural resources, reduce harmful emissions, and lower life cycle costs. Compared to natural aggregate (NA), RCA pavement has fewer comprehensive studies and sustainability assessments.
Recently uploaded (20)
Embedded machine learning-based road conditions and driving behavior monitoring
Embedded machine learning-based road conditions and driving behavior monitoring
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMS
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMS
The Python for beginners. This is an advance computer language.
The Python for beginners. This is an advance computer language.
Literature Review Basics and Understanding Reference Management.pptx
Literature Review Basics and Understanding Reference Management.pptx
ML Based Model for NIDS MSc Updated Presentation.v2.pptx
ML Based Model for NIDS MSc Updated Presentation.v2.pptx
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
Generative AI leverages algorithms to create various forms of content
Generative AI leverages algorithms to create various forms of content
New techniques for characterising damage in rock slopes.pdf
New techniques for characterising damage in rock slopes.pdf
A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...
basic-wireline-operations-course-mahmoud-f-radwan.pdf
basic-wireline-operations-course-mahmoud-f-radwan.pdf
Harnessing WebAssembly for Real-time Stateless Streaming Pipelines
Harnessing WebAssembly for Real-time Stateless Streaming Pipelines
Recycled Concrete Aggregate in Construction Part II
Recycled Concrete Aggregate in Construction Part II
Ir mcq-answering-system
- 1. MCQ Answering System By Vikash Saini (201352019) P Jishnu Jaykumar (201352005) Vivek Kumar Singh (201352015) INDIAN INSTITUTE OF INFORMATION TECHNOLOGY VADODARA Information Retrieval
- 2. Introduction ● QA System Using Textual Entailment and Text Similarity Measures
- 3. QA System with textual entailment and text similarity System Architecture
- 4. Corpus Reader ● Reads the passage. ● Scans all the questions and its corresponding options.
- 5. Preprocessing ● Annotate text ○ Identifying tokens and sentences in the text ○ Used to annotate parts of speech (PoS) - the constituent structure ○ Name-Entity recognition ○ Coreference resolution ● Example, ○ Question: What did John want? ○ Answer Option: He wanted a bike. ○ Resolved Answer: John wanted a bike. This process will result in a hypothesis set that will the basis for finding actual answer. Let us denote it as H={opt1,opt2,....optN} , where N = total number of options question has.
- 6. Sentence Retrieval ● For a particular question ○ Take each sentence from the passage. ○ Perform and get score ■ a=lexical analysis, n-gram Jaccard coefficient ■ b=Semantic similarity ■ c=Parts of speech analysis - POS ○ Now perform the linear combination of a,b,c and get the overall score. ○ Now select top K sentences out of the total measured -> Denote as set T = {s1,s2….sk}
- 7. Answer Selection ● We have T and H set available now. ● Making an ordered set of T*H will yield us all the possible combination of resolved answer and corresponding passage sentence. ● There are two main components in this module ○ Textual entailment ■ Machine learning component - logical reasoning ■ Will yield a binary value and a confident score - > score C ○ Answer Similarity ■ Statistical component ● Performing the same three analysis done in sentence retrieval ● Will yield a score by linear combination - score L ● Final score = a*(C) + b*(L) ● a,b are weight measures. Generally {3,2},{2,1},{1,0} are used. ● More weightage to logical score than statistical score.
- 8. Evaluation ● Evaluation is done based on c@1 score. ● It is calculated by below mentioned formula ○ c@1 = (U + (U*R/T)) / T ■ U = Unanswered question ■ R = Correctly answered question ■ T = Total question ● The main idea of this metric is to encourage systems to reduce the number of incorrect answers while maintaining the number of correct ones by leaving some questions unanswered.
- 9. References ● CLEF QA4MRE 1 main task [18] ● Solving Open-Domain Multiple Choice Questions with Textual Entailment and Text Similarity Measures , Neil Dhruva ‡ , Oliver Ferschke †‡ and Iryna Gurevych †‡ ● Graph-based Approach to the Question Answering Task Based on Entrance Exams Helena G ́omez-Adorno 1 , Grigori Sidorov 1 , David Pinto 2 , and Alexander Gelbukh 1
- 10. Any queries...
- 11. Thank you...