自拍偷在线精品自拍偷,亚洲欧美中文日韩v在线观看不卡

GraphRAG結(jié)合普通RAG,打造Hybrid RAG

發(fā)布于 2024-12-17 13:12
瀏覽
0收藏

RAG在生成式AI領(lǐng)域取得了重大進(jìn)展,用戶可以將自己的個(gè)人文檔,比如文本文件、PDF、視頻等,與大型語(yǔ)言模型(LLMs)連接起來進(jìn)行互動(dòng)。最近,RAG的進(jìn)階版GraphRAG也亮相了,它通過知識(shí)圖譜和LLMs來執(zhí)行RAG的檢索任務(wù)。

RAG和GraphRAG各有所長(zhǎng),也各有局限。RAG擅長(zhǎng)利用向量相似性技術(shù),而GraphRAG則依賴圖分析和知識(shí)圖譜來提供更精確的答案。那么,如果將兩者結(jié)合起來進(jìn)行檢索,會(huì)擦出怎樣的火花呢?

1 HybridRAG

HybridRAG是一個(gè)高級(jí)框架,它合并了RAG和GraphRAG。這種集成旨在提高信息檢索的準(zhǔn)確性和上下文相關(guān)性。簡(jiǎn)單來說,HybridRAG使用來自兩個(gè)檢索系統(tǒng)(RAG和GraphRAG)的上下文,最終輸出是兩個(gè)系統(tǒng)的混合。

2 HybridRAG的優(yōu)勢(shì)

  • 提高準(zhǔn)確性:通過利用結(jié)構(gòu)化推理和靈活檢索,HybridRAG提供的答案比單獨(dú)使用VectorRAG或GraphRAG更精確。
  • 增強(qiáng)上下文理解:通過整合不同系統(tǒng),HybridRAG能更深入地理解實(shí)體間的關(guān)系及其出現(xiàn)的上下文。
  • 動(dòng)態(tài)推理能力:知識(shí)圖譜可以動(dòng)態(tài)更新,使系統(tǒng)能夠適應(yīng)新信息的可用性。

3 使用LangChain來構(gòu)建HybridRAG系統(tǒng)

這里使用一個(gè)名為“Moon.txt”的文件進(jìn)行這個(gè)演示,這是一個(gè)超級(jí)英雄故事。請(qǐng)查看以下內(nèi)容。

In the bustling city of Lunaris, where the streets sparkled with neon lights and the moon hung low in the sky, lived an unassuming young man named Max. By day, he was a mild-mannered astronomer, spending his hours studying the stars and dreaming of adventures beyond Earth. But as the sun dipped below the horizon, Max transformed into something extraordinary—Moon Man, the guardian of the night sky.
Max’s transformation began with a mysterious encounter. One fateful evening, while gazing through his telescope, a brilliant flash of light erupted from the moon. A celestial being, shimmering with silver light, descended and bestowed upon him a magical amulet. “With this, you shall harness the power of the moon,” the being declared. “Use it wisely, for the night sky needs a hero.”
With the amulet around his neck, Max felt energy coursing through him. He could leap great distances, manipulate moonlight, and even communicate with nocturnal creatures. He vowed to protect his city from the shadows that lurked in the night.
As Moon Man, Max donned a sleek, silver suit adorned with celestial patterns that glimmered like the stars. With his newfound abilities, he patrolled the city, rescuing lost pets, helping stranded motorists, and even thwarting petty criminals. The citizens of Lunaris began to whisper tales of their mysterious hero, who appeared under the glow of the moon.
One night, as he soared through the sky, he encountered a gang of thieves attempting to steal a priceless artifact from the Lunaris Museum. With a flick of his wrist, he summoned a beam of moonlight that blinded the thieves, allowing him to swoop in and apprehend them. The city erupted in cheers, and Moon Man became a beloved figure.
However, peace in Lunaris was short-lived. A dark force emerged from the depths of the cosmos—an evil sorceress named Umbra, who sought to extinguish the moon’s light and plunge the world into eternal darkness. With her army of shadow creatures, she began to wreak havoc, stealing the moon’s energy and spreading fear among the citizens.
Moon Man knew he had to confront this new threat. He gathered his courage and sought the wisdom of the celestial being who had granted him his powers. “To defeat Umbra, you must harness the full power of the moon,” the being advised. “Only then can you restore balance to the night sky.”
With determination in his heart, Moon Man prepared for the ultimate battle. He climbed to the highest peak in Lunaris, where the moon shone brightest, and focused on channeling its energy. As Umbra and her shadow creatures descended upon the city, Moon Man unleashed a magnificent wave of moonlight, illuminating the darkness.
The battle raged on, with Umbra conjuring storms of shadows and Moon Man countering with beams of silver light. The clash of powers lit up the night sky, creating a dazzling display that captivated the citizens below. In a final, desperate move, Moon Man summoned all his strength and unleashed a powerful blast of moonlight that enveloped Umbra, banishing her to the farthest reaches of the cosmos.
With Umbra defeated, the moon’s light returned to its full glory, and the city of Lunaris rejoiced. Max, still in his Moon Man guise, stood atop the highest building, watching as the citizens celebrated their hero. They had learned the importance of hope and courage, even in the darkest of times.
From that day forward, Moon Man became a symbol of resilience and bravery. Max continued to protect Lunaris, knowing that as long as the moon shone brightly, he would always be there to guard the night sky. And so, the legend of Moon Man lived on, inspiring generations to look up at the stars and believe in the extraordinary.
As the years passed, stories of Moon Man spread beyond Lunaris, becoming a beacon of hope for those who felt lost in the darkness. Children would gaze at the moon, dreaming of adventures, and Max would smile, knowing that he had made a difference. For in the heart of every dreamer, the spirit of Moon Man lived on, reminding them that even the smallest light can shine brightly against the shadows.

導(dǎo)入包并設(shè)置LLM端嵌入模型(用于標(biāo)準(zhǔn)RAG)

import os
from langchain_experimental.graph_transformers import LLMGraphTransformer
from langchain_core.documents import Document
from langchain_community.graphs.networkx_graph import NetworkxEntityGraph
from langchain.chains import GraphQAChain
from langchain.text_splitter import CharacterTextSplitter
from langchain.document_loaders import TextLoader
from langchain.chains import RetrievalQA
from langchain.vectorstores import Chroma
from langchain_google_genai import GoogleGenerativeAI,GoogleGenerativeAIEmbeddings

GOOGLE_API_KEY=''

embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001",google_api_key=GOOGLE_API_KEY)
llm = GoogleGenerativeAI(model="gemini-pro",google_api_key=GOOGLE_API_KEY)

接者,為GraphRAG實(shí)現(xiàn)(鏈對(duì)象)創(chuàng)建函數(shù),覆蓋文件“Moon.txt”

def graphrag():
    with open('Moon.txt', 'r') as file:
        content = file.read()

    documents = [Document(page_content=content)]
    llm_transformer = LLMGraphTransformer(llm=llm)
    graph_documents = llm_transformer.convert_to_graph_documents(documents)

    graph = NetworkxEntityGraph()

    # 添加節(jié)點(diǎn)到圖
    for node in graph_documents[0].nodes:
        graph.add_node(node.id)

    # 添加邊到圖
    for edge in graph_documents[0].relationships:
        graph._graph.add_edge(
                edge.source.id,
                edge.target.id,
                relation=edge.type,
            )

        graph._graph.add_edge(
                edge.target.id,
                edge.source.id,
                relation=edge.type+" by",
            )

    chain = GraphQAChain.from_llm(
        llm=llm, 
        graph=graph, 
        verbose=True
    )
        
    return chain

同樣,為同一文件實(shí)現(xiàn)標(biāo)準(zhǔn)RAG創(chuàng)建函數(shù)

def rag():
    # 文檔加載器
    loader = TextLoader('Moon.txt')
    data = loader.load()

    # 文檔轉(zhuǎn)換器
    text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
    texts = text_splitter.split_documents(data)

    # 向量數(shù)據(jù)庫(kù)
    docsearch = Chroma.from_documents(texts, embeddings)

    # 需要知道的超參數(shù)
    retriever = docsearch.as_retriever(search_type='similarity_score_threshold',search_kwargs={"k": 7,"score_threshold":0.3})
    qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever)
        
    return qa

為兩種類型的RAG創(chuàng)建對(duì)象

standard_rag = rag()
graph_rag = graphrag()

現(xiàn)在是時(shí)候?qū)崿F(xiàn)HybridRAG了

def hybrid_rag(query,standard_rag,graph_rag):
    result1 = standard_rag.run(query)
    
    print("Standard RAG:",result1)
    result2 = graph_rag.run(query)
    
    
    print("Graph RAG:",result2)
    prompt = "Generate a final answer using the two context given : Context 1: {} \n Context 2: {} \n Question: {}".format(result1,result2,query)
    return llm(prompt)

query = "Some characteristics of Moon Man"
hybrid = hybrid_rag(query,standard_rag,graph_rag)
print("Hybrid:",hybrid)

如你所見,我們對(duì)給定的提示分別獨(dú)立執(zhí)行了標(biāo)準(zhǔn)RAG和GraphRAG。一旦找到答案,我們就會(huì)利用這兩個(gè)響應(yīng)作為上下文,來生成最終的答案。

談到輸出,最終的HybridRAG確實(shí)從兩次檢索中獲取了上下文,并產(chǎn)生了更好的結(jié)果。有些點(diǎn)被兩個(gè)RAG系統(tǒng) 遺漏了,但最終HybridRAG結(jié)合并給出了完美的答案。

STANDARD RAG: 
 Here are some characteristics of Moon Man, based on the story:

* **Brave:** He confronts danger and fights villains like Umbra.
* **Powerful:** He has superhuman abilities granted by the amulet.
* **Protective:** He safeguards Lunaris and its citizens.
* **Determined:** He doesn't give up, even when facing powerful enemies.
* **Compassionate:** He helps those in need, like rescuing lost pets.
* **Humble:** Despite his powers, he remains grounded and dedicated to his city. 



> Entering new GraphQAChain chain...
Entities Extracted:
Moon Man

Full Context:
Moon Man PROTECTS night sky
Moon Man WEARS silver suit
Moon Man PROTECTED Lunaris
Moon Man CAPTURED thieves
Moon Man DEFEATED Umbra
Moon Man INSPIRES hope
Moon Man INSPIRES courage

> Finished chain.

 @@ 
 GRAPH RAG: 
 Helpful Answer: 
* Protective (protects night sky, protected Lunaris)
* Courageous and Inspiring (inspires hope, inspires courage)
* Strong (captured thieves, defeated Umbra) 


 @@ 
 HYBRID RAG: 
 Moon Man is the **protective** champion of Lunaris, using his **strength** and **courage** to def

本文轉(zhuǎn)載自 ??AI科技論談??,作者: AI科技論談

標(biāo)簽
收藏
回復(fù)
舉報(bào)
回復(fù)
相關(guān)推薦