百度也來卷RAG了，Self-Reasoning比Self-RAG最高提升11.8%
檢索增強(qiáng)型語言模型（Retrieval-Augmented Language Model, RALM）通過在推理過程中整合外部知識，減輕了LLM固有的事實幻覺問題。然而，RALMs仍面臨挑戰(zhàn)：檢索到的不相關(guān)文檔可能導(dǎo)致無效的回答生成、生成的輸出中缺乏適當(dāng)?shù)囊?，使得驗證模型的可信度變得復(fù)雜。
一個示例，展示了百度自我推理（Self-Reasoning）框架如何通過相關(guān)意識過程、證據(jù)意識選擇過程和軌跡分析過程生成推理軌跡。
為了解決上述問題，百度（Baidu Inc.）提出了一種新穎的自我推理框架（SELF-REASONING framework），旨在通過LLM自身生成的推理軌跡來提高RALMs的可靠性和可追溯性。它的創(chuàng)新之處在于通過端到端的框架直接增強(qiáng)LLMs的性能，而無需依賴外部的推理模型，從而提供了一種更有效和可擴(kuò)展的解決方案。
對提高RALMs的自我推理框架的說明圖。上層是基本的LLMs，它們通過固有知識回答問題。中層是標(biāo)準(zhǔn)的檢索增強(qiáng)型語言模型，它們使用檢索到的文檔幫助回答問題。底層是百度的自我推理框架，它使用自生成的推理軌跡來輸出答案。
- 相關(guān)性感知過程（Relevance-Aware Process, RAP）
選擇默認(rèn)的檢索器（如DPR和Contriever）來檢索與問題相關(guān)的文檔。
指導(dǎo)模型評估檢索到的文檔與問題的相關(guān)性，并生成解釋為何這些文檔被識別為相關(guān)的輸出。
如果所有檢索到的文檔都不相關(guān)，模型應(yīng)依據(jù)預(yù)訓(xùn)練階段獲得的內(nèi)部知識提供答案。
- 證據(jù)感知選擇過程（Evidence-Aware Selective Process, EAP）
要求模型從檢索到的文檔中明確選擇關(guān)鍵句子作為證據(jù)，并輸出選擇這些句子的原因。
模型需要自動選擇文檔中的關(guān)鍵句子片段，并解釋為何這些片段能夠回答問題。
- 軌跡分析過程（Trajectory Analysis Process, TAP）
將前兩個過程中生成的自我推理軌跡整合，形成推理片段鏈，以增強(qiáng)檢索增強(qiáng)生成的整體性能。
要求模型分析這些推理軌跡，并最終輸出簡潔的分析和簡短的答案。
此外，還包括數(shù)據(jù)生成和質(zhì)量控制的過程：
- 訓(xùn)練數(shù)據(jù)生成：使用GPT-4生成答案作為基準(zhǔn)真值，并創(chuàng)建正負(fù)樣本對。
- 數(shù)據(jù)質(zhì)量控制：通過自動驗證工具和過濾不正確答案的軌跡，確保數(shù)據(jù)生成的正確性和完整性。
模型訓(xùn)練：
- 使用標(biāo)準(zhǔn)的語言建模目標(biāo)進(jìn)行訓(xùn)練，通過最大化似然函數(shù)來訓(xùn)練自我推理生成模型。
- 提出了分階段掩碼策略的逐步訓(xùn)練方法，以逐步學(xué)習(xí)生成長推理軌跡。
通過在四個公共數(shù)據(jù)集（兩個短形式問答數(shù)據(jù)集、一個長形式問答數(shù)據(jù)集和一個事實驗證數(shù)據(jù)集）上的評估，證明了該方法的優(yōu)越性，能夠超越現(xiàn)有的最先進(jìn)模型，并且在僅使用2,000個訓(xùn)練樣本的情況下，就能與GPT-4的性能相媲美。
在兩個短形式問答數(shù)據(jù)集、一個長形式問答數(shù)據(jù)集和一個事實驗證數(shù)據(jù)集上，與不同基線模型的性能比較。用粗體黑色表示的數(shù)字代表除了GPT-4之外的最佳結(jié)果。結(jié)果是基于五次運(yùn)行的平均值，并且省略了標(biāo)準(zhǔn)差值（所有值均≤2%）。
SELF-REASONING框架通過利用LLM自身生成的推理軌跡，在不依賴外部模型或工具的情況下，有效提高了RALMs的魯棒性，并通過要求LLM明確生成文檔片段和引用，增強(qiáng)了RALMs的可解釋性和可追溯性。
在三個不同的數(shù)據(jù)集上的噪聲魯棒性實驗結(jié)果：(a) 左側(cè)是NQ數(shù)據(jù)集，(b) 中間是PopQA數(shù)據(jù)集，(c) 右側(cè)是FEVER數(shù)據(jù)集。Self-RAG和Vicuna是具有13B參數(shù)大小的模型。
附錄：
GPT-4生成短形式和長形式問答任務(wù)的自我推理軌跡的Prompt模版
Instructions
# Role
You are an experienced expert, skilled in answering various questions.
# Task
Please answer the question according to the provided reference evidence as
required.
# Reference Evidence
[1] Retrieved Document {{DOCUMENT 1}[2] Retrieved Document {{DOCUMENT 2}[3] Retrieved Document {{DOCUMENT 3}[4] Retrieved Document {{DOCUMENT 4}[5] Retrieved Document {{DOCUMENT 5}# Requirements
1. First, please judge whether the provided documents are relevant with the
question, and put it in the relevant field. If the provided content is irrelevant to
the question, explain the reason in the relevant reason field, then you can give
the answer with your internal knowledge.
2. If possible, answer the question in points and provide explanations.
3. If the content in the answer comes from different pieces of evidence, you
need to cite the sequence number of the evidence at the end of the sentence.
The citation format is shown below: [1], [1,3].
4. Place each cited piece of evidence in the cite_list field, cite content field to
store each paragraph of cited content (omitted words can be replaced by ...),
cite reason is used to store your thoughts and analysis of this content, how
this paragraph can answer the question.
5. Put the long answer content in the analysis field, and put the short
answer(no more than 10 words) in the answer field.
# Question
{{QUESTION}}GPT-4生成事實驗證任務(wù)的自我推理軌跡的Prompt模版
Instructions
# Role
You are an experienced expert, skilled in answering various questions.
# Task
Please answer the question according to the provided reference evidence as
required.
# Reference Evidence
[1] Retrieved Document {{DOCUMENT 1}}
[2] Retrieved Document {{DOCUMENT 2}}
[3] Retrieved Document {{DOCUMENT 3}}
[4] Retrieved Document {{DOCUMENT 4}}
[5] Retrieved Document {{DOCUMENT 5}}
# Requirements
1. First, please judge whether the provided documents are relevant with the
claim, and put it in the relevant field. If the provided content is irrelevant to the
question, explain the reason in the relevant reason field, then you can give the
answer with your internal knowledge.
2. If possible, answer the question in points and provide explanations.
3. If the content in the answer comes from different pieces of evidence, you
need to cite the sequence number of the evidence at the end of the sentence.
The citation format is shown below: [1], [1,3].
4. Place put each cited piece of evidence in the list, use cite content field to
store each paragraph of cited content (omitted words can be replaced by ...),
cite reason is used to store your thoughts and analysis of this content, how this
paragraph can answer the question.
5. Put the long answer content in the analysis field, and put the short answer
(SUPPORT/REFUTE/NOT ENOUGH INFO) in the answer field.
# Question
You should check whether the given claim is supported or refuted by above the
evidence. If supported，output SUPPORT, if controversial, output REFUTE; if
the evidence provided is not enough to make a decision or the provided
evidence is irrelevant, output NOT ENOUGH INFO.
{{CLAIM}}案例研究。展示了SELF-REASONING框架在推理過程中生成的自我推理軌跡，并展示了它們?nèi)绾芜壿嬌仙烧_答案。
Question
When was catch me if you can made?
Retrieved document #1: The film due to her busy schedule. The original start date was
January 2002, but was pushed to February 7 in Los Angeles, California. Locations included
Burbank, Downey, New York City, Quebec City and Montreal. …
Retrieved document #2: eventually turned to him for help in catching other check forgers.
Development for the film started in 1980, but did not progress until 1997, when Spielberg's
DreamWorks bought the film rights to Abagnale's book. David Fincher, Gore Verbinski, Lasse
Hallstr?m, Milo? Forman, and Cameron Crowe had all been possible candidates for director
before Spielberg decided to direct. …
Retrieved document #3: It was funded independently and had a budget of $800,000. The film
was Sommers's directorial debut and was written by him. When production finished, the studio
that had promised to distribute the movie had gone out of business, and it was eventually
released by MCA Inc. in July 1989. The movie…
(Other retrieved documents are omitted.)
Raw LLM Answer: 2000 ?
Standard RAG Answer: 1989 ?
Self-Reasoning Answer:
(Relevant Aware Process)
Relevant: True
Relevant Reason: The provided documents are relevant with question.
(Evidence Aware Selective Process)
Cite content: [1] … the original start date was January 2002, but was pushed to February 7 in
Los Angeles, …
Reason to cite: This piece provides information on the commencement and location of filming
for 'Catch Me If You Can', indicating that it started in April 2002
Cite content: [3] … When production finished, the studio that had promised to distribute the
movie had gone out of business, and it was eventually released by MCA Inc. in July 1989., …
Reason to cite: This piece provides information on the release date of 'Catch Me If You Can',
indicating that it was eventually released by MCA Inc. in July.
(Trajectory Analysis Process)
Analysis:（long answer）
The film 'Catch Me If You Can' was made in 2002. It started filming in April 2002 in Park Avenue,
just outside the Waldorf-Astoria Hotel, and moved to Orange, New Jersey, before returning to
Brooklyn for bank and courthouse scenes [1].
Final Answer：2002 ? （short answer）
https://arxiv.org/pdf/2407.19813
Improving Retrieval Augmented Language Model with Self-Reasoning
Baidu Inc., China.本文轉(zhuǎn)載自??PaperAgent??
