백인걸_Ask Optimal Question_alignings LLM with Retriever's preference in Conversational Search.pdf

1
1
Language Intelligence Lab, Chung-Ang University
Ask Optimal Questions: Aligning Large Language Models
with Retriever’s Preference in Conversational Search
Small Talk, 2024.03.13
백인걸
LILAB
Department of Artificial Intelligence
Chung-Ang University

2
• Rewriting in RAG
• Introduction, Abstract
• Method
• Experiment
• Conclusion, Limitation
Outline

3
Ask Optimal Questions
- Rewriting in RAG
LLM output
LLM input
Query rewriting

4
- Rewriting Methods
Query: what was the immediate impact of
the success of the manhattan project?
Keywords: immediate effects manhattan
project success
Query: what was the immediate impact of t
he success of the manhattan project?
Answer: The immediate impact of the Manhatt
an Project's success was the end of World War
II, marked by the use of atomic bombs on Hiros
hima and Nagasaki, leading to Japan's surrende
r. This event introduced the nuclear age, resha
ping global military, political landscapes, and s
parking ethical and scientific debates.
"passage_text": [ "The presence of
communication amid scientific mind
s was equally important to the suc
cess of the Manhattan Project as s
cientific intellect was. The only
cloud hanging over the impressive
achievement of the atomic research
ers and engineers is what their su
ccess truly meant; hundreds of tho
usands of innocent lives obliterat
ed.", "The Manhattan Project and i
ts atomic bomb helped bring an end
to World War II. Its legacy of pea
ceful uses of atomic energy contin
ues to have an impact on history a
nd science.",
…
Passage: The immediate impact of the su
ccess of the Manhattan Project was profound
and far-reaching, fundamentally altering the
course of world history. Upon its completion,
the project had developed the world's first at
omic bombs, which were subsequently used
on the Japanese …
Label

5
- Introduction, Abstract
기존 query하나에 대해서 다양한 방식으로 query rewriting 진행한 후,
retrieval system의 input으로 넣어 ranking을 바탕으로 데이터셋 구성.
언어 모델에 학습(SFT, DPO)시켜 최적 query를 생성해내는 언어모델을
만들자.
Generate Model
- Query Exploration: GPT-4
- LLM finetuning: Llama2-7b
Retrieval Model
- BM25(Sparse)
- ANCE(Dense)
Model
Summary
Evaluation
Conversation system - TopiOCQA, QReCC

6
- Methods - 1
만들자.
Summary
Method - 1

7
- QR(Query Rewriting= coreference resolution)
대명사를 기존 conversation에 있는 명사로 교체
- QR(Query Rewriting) w/Planning
Plan을 바탕으로 같은 의미를 가진 새로운 query를 생성하는 과정
- QE(Query Expansion)
기존 query에서 추가적인 정보를 더해 기존 query 확장
만들자.
Summary
- Methods - 1

8
- Method - 2
만들자.
Summary
Method - 2

9
Ask Optimal Questions 기존 query하나에 대해서 다양한 방식으로 query rewriting 진행한 후,
만들자.
Summary
- Method - 2
Keywords: immediate effects manhattan
project success
Query: what was the immediate impact of t
he success of the manhattan project?
Answer: The immediate impact of the Manhatt
an Project's success was the end of World War
II, marked by the use of atomic bombs on Hiros
hima and Nagasaki, leading to Japan's surrende
r. This event introduced the nuclear age, resha
ping global military, political landscapes, and s
parking ethical and scientific debates.
"passage_text": [ "The presence
of communication amid scientifi
c minds was equally important t
o the success of the Manhattan
Project as scientific intellect
was. The only cloud hanging ove
r the impressive achievement of
the atomic researchers and engi
neers is what their success tru
ly meant; hundreds of thousands
of innocent lives obliterated.",
"The Manhattan Project and its
atomic bomb helped bring an end
to World War II. Its legacy of
peaceful uses of atomic energy
continues to have an impact on
history and science.",
…
Passage: The immediate impact of the su
ccess of the Manhattan Project was profound
and far-reaching, fundamentally altering the
course of world history. Upon its completion,
the project had developed the world's first at
omic bombs, which were subsequently used
on the Japanese …
Label

10
Ask Optimal Questions 기존 query하나에 대해서 다양한 방식으로 query rewriting 진행한 후,
만들자.
Summary
- Method - 2
Rankss output
1
2
.
.
.
.
.
.

11
- Method - 3
만들자.
Summary
Method -3

12
- Method - 3
만들자.
Summary
1. Ranking Threshold 넘은 데이터에 대해서 SFT 학습
2. Ranking 을바탕으로 preference 정도를 알 수 있고,
이를 바탕으로 DPO 학습 진행

13
- Experiment
의미: 가져온 k개의 문서 중 관련된 문서가 얼마나 많은가
가져온 10개의 문서 중, 8개가 관련된 문서,
전체 관련 문서수가 9개 -> Recall@10은 8/9
Ref: https://bigdatamaster.tistory.com/178

14
- Experiment

15
- Conclusion, Limitation
Conclusion
- Query를 최적화 시키기는 RETPO 라는 framework 소개
- Retrieval system에서의 rank를 바탕으로 학습에 적용
- SFT, DPO 방식으로 학습된 모델이 QReCC, TopiOCQA 에서 baseline outperform
Limitation
- 학습데이터의 사용이 예산 제약으로 전체 TopiOCQA데이터에서 절반만 사용.
- 더 큰 파라미터에서의 모델은 query를 더 지속적으로 생성하는 경향이 있어
작은 모델 사용.

백인걸_Ask Optimal Question_alignings LLM with Retriever's preference in Conversational Search.pdf

Recommended

Recommended

More Related Content

Similar to 백인걸_Ask Optimal Question_alignings LLM with Retriever's preference in Conversational Search.pdf

Similar to 백인걸_Ask Optimal Question_alignings LLM with Retriever's preference in Conversational Search.pdf (20)

백인걸_Ask Optimal Question_alignings LLM with Retriever's preference in Conversational Search.pdf