Logo

菜单

发布于 2026-01-06 0 阅读
0

RAG 中最难的问题……如何处理“未找到”答案 🔍🤔 由 Mux 呈现的 DEV 全球展示挑战赛：展示你的项目！

RAG 中最难的问题……如何处理“未找到”的答案🔍🤔

由 Mux 主办的 DEV 全球展示挑战赛：展示你的项目！

首先……什么是 RAG？🕵️‍♂️

检索增强生成（RAG）是一种自然语言处理方法，它通过引用外部文档来提供更准确、更符合上下文的答案。尽管RAG具有诸多优势，但也面临一些挑战，其中之一就是如何处理“未找到”的答案。解决这个问题对于开发一个高效可靠、人人都能使用的模型至关重要。

为什么“未找到”的答案可能令人担忧⛔️

有些模型在找不到答案时会产生“幻觉”，从而给出不准确的答案，误导用户。这会削弱用户对模型的信任，降低模型的可靠性和有效性。

我们该如何解决这个问题？🛠️

首先，模型最好是告知用户它找不到答案，而不是编造一个答案。

没关系

接下来，我们将深入探讨LLMWare有效处理“未找到”情况的一种方式。通过研究这些方法，我们可以更好地了解如何解决此问题，并提高RAG系统的整体性能和可靠性。

致视觉学习者……📺

这里有一个视频，讨论的主题与本文相同。建议您先观看视频，然后再按照本文中的步骤操作。

框架🖼️

LLMWare
对于我们的新读者来说，LLMWARE 是一个全面的开源框架，它为基于 LLM 的应用模式（包括检索增强生成 (RAG)）提供了一个统一的平台。

请pip3 install llmware在命令行中运行以下载软件包。

导入库并创建上下文📚

from llmware.models import ModelCatalog
from llmware.parsers import WikiParser
Enter fullscreen mode Exit fullscreen mode

ModelCatalog：一个用于llmware管理选择所需模型、加载模型和配置模型的类。

WikiParser：一个llmware用于处理从维基百科检索和打包内容的类。

text =("BEAVERTON, Ore.--(BUSINESS WIRE)--NIKE, Inc. (NYSE:NKE) today reported fiscal 2024 financial results for its "
      "third quarter ended February 29, 2024.) “We are making the necessary adjustments to drive NIKE’s next chapter "
      "of growth Post this Third quarter revenues were slightly up on both a reported and currency-neutral basis* "
      "at $12.4 billion NIKE Direct revenues were $5.4 billion, slightly up on a reported and currency-neutral basis "
      "NIKE Brand Digital sales decreased 3 percent on a reported basis and 4 percent on a currency-neutral basis "
      "Wholesale revenues were $6.6 billion, up 3 percent on a reported and currency-neutral basis Gross margin "
      "increased 150 basis points to 44.8 percent, including a detriment of 50 basis points due to restructuring charges "
      "Selling and administrative expense increased 7 percent to $4.2 billion, including $340 million of restructuring "
      "charges Diluted earnings per share was $0.77, including $0.21 of restructuring charges. Excluding these "
      "charges, Diluted earnings per share would have been $0.98* “We are making the necessary adjustments to "
      "drive NIKE’s next chapter of growth,” said John Donahoe, President & CEO, NIKE, Inc. “We’re encouraged by "
      "the progress we’ve seen, as we build a multiyear cycle of new innovation, sharpen our brand storytelling and "
      "work with our wholesale partners to elevate and grow the marketplace.")
Enter fullscreen mode Exit fullscreen mode

以下是我们提取的初始文本，其中详细介绍了知名运动品牌耐克。您可以根据需要修改此文本。

一切都在汇合

创建提取密钥🔐

extract_key = "company founding date"
dict_key = extract_key.replace(" ", "_")

company_founding_date = ""
Enter fullscreen mode Exit fullscreen mode

在这里，我们将公司成立日期设定为从文本中提取的目标信息。

运行初始提取🏃

model = ModelCatalog().load_model("slim-extract-tool", temperature=0.0, sample=False)
response = model.function_call(text, function="extract", params=[extract_key])
llm_response = response["llm_response"]
Enter fullscreen mode Exit fullscreen mode

模型：在此代码片段中，我们加载了 LLMWare 的 slim-extract-tool，这是一个 28 亿参数的 GGUF 模型，针对通用提取进行了微调（GGUF 是一种量化方法，可以加快推理速度并减小模型大小，但会降低准确性）。

温度：此参数控制输出的随机性。有效值介于 0 和 1 之间，值越低，模型越确定；值越高，模型越随机、越具创造性。

示例：确定输出是确定性生成还是概率性生成。False 生成确定性输出。True 生成概率性输出。

然后我们尝试使用该模型从文本中提取信息并将其存储在llm_response……

如果找到答案……✅

if dict_key in llm_response:

        company_founding_date = llm_response[dict_key]

        if len(company_founding_date) > 0:

            company_founding_date = company_founding_date[0]
            print(f"update: found the {extract_key} value - ", company_founding_date)
            return company_founding_date
Enter fullscreen mode Exit fullscreen mode

如果模型成功找到并提取出公司成立日期，我们将返回该信息。

如果找不到答案……❌

else:

    print(f"update: did not find the target value in the text - {company_founding_date}")
    print("update: initiating a secondary process to try to find the information")

    response = model.function_call(text, function="extract", params=["company name"])
Enter fullscreen mode Exit fullscreen mode

如果模型找不到公司成立日期，我们将运行第二次查询来查找公司名称，以便将来收集更多信息。

艾美奖

从维基百科获取信息📖

if "company_name" in response["llm_response"]:
    company_name = response["llm_response"]["company_name"][0]

    if company_name:
        print(f"\nupdate: found the company name - {company_name} - now using to lookup in secondary source")

        output = WikiParser().add_wiki_topic(company_name,target_results=1)
Enter fullscreen mode Exit fullscreen mode

从文本中提取出公司名称后，我们将从维基百科检索有关该公司的更多信息。

从检索到的文章数据生成摘要片段✍️

if output:

    supplemental_text = output["articles"][0]["summary"]

    if len(supplemental_text) > 150:
        supplemental_text_pp = supplemental_text[0:150] + " ... "
    else:
        supplemental_text_pp = supplemental_text

    print(f"update: using lookup - {company_name} - found secondary source article "
                              f"(extract displayed) - ", supplemental_text_pp)
Enter fullscreen mode Exit fullscreen mode

如果已成功从 Wiki 获取额外数据，则当响应长度超过 150 个字符时，我们会将其截断并设置supplemental_text_pp为

再次致电提取最新信息📞

new_response = model.function_call(supplemental_text,params=["company founding date"])

print("\nupdate: reviewed second source article - ", new_response["llm_response"])
Enter fullscreen mode Exit fullscreen mode

利用从维基百科检索到的新信息，我们再次对模型运行相同的提取过程。

找到则打印响应🖨️

if "company_founding_date" in new_response["llm_response"]:
    company_founding_date = new_response["llm_response"]["company_founding_date"]
    if company_founding_date:
        print("update: success - found the answer - ", company_founding_date)
Enter fullscreen mode Exit fullscreen mode

如果将新信息纳入考量后，我们找到了公司成立日期，并将结果打印出来。

完全集成代码🧑‍💻

from llmware.models import ModelCatalog
from llmware.parsers import WikiParser

text =("BEAVERTON, Ore.--(BUSINESS WIRE)--NIKE, Inc. (NYSE:NKE) today reported fiscal 2024 financial results for its "
      "third quarter ended February 29, 2024.) “We are making the necessary adjustments to drive NIKE’s next chapter "
      "of growth Post this Third quarter revenues were slightly up on both a reported and currency-neutral basis* "
      "at $12.4 billion NIKE Direct revenues were $5.4 billion, slightly up on a reported and currency-neutral basis "
      "NIKE Brand Digital sales decreased 3 percent on a reported basis and 4 percent on a currency-neutral basis "
      "Wholesale revenues were $6.6 billion, up 3 percent on a reported and currency-neutral basis Gross margin "
      "increased 150 basis points to 44.8 percent, including a detriment of 50 basis points due to restructuring charges "
      "Selling and administrative expense increased 7 percent to $4.2 billion, including $340 million of restructuring "
      "charges Diluted earnings per share was $0.77, including $0.21 of restructuring charges. Excluding these "
      "charges, Diluted earnings per share would have been $0.98* “We are making the necessary adjustments to "
      "drive NIKE’s next chapter of growth,” said John Donahoe, President & CEO, NIKE, Inc. “We’re encouraged by "
      "the progress we’ve seen, as we build a multiyear cycle of new innovation, sharpen our brand storytelling and "
      "work with our wholesale partners to elevate and grow the marketplace.")


def not_found_then_triage_lookup():

    print("\nNot Found Example - if info not found, then lookup in another source.\n")

    extract_key = "company founding date"
    dict_key = extract_key.replace(" ", "_")

    company_founding_date = ""

    model = ModelCatalog().load_model("slim-extract-tool", temperature=0.0, sample=False)
    response = model.function_call(text, function="extract", params=[extract_key])
    llm_response = response["llm_response"]

    print(f"update: first text reviewed for {extract_key} - llm response: ", llm_response)

    if dict_key in llm_response:

        company_founding_date = llm_response[dict_key]

        if len(company_founding_date) > 0:

            company_founding_date = company_founding_date[0]
            print(f"update: found the {extract_key} value - ", company_founding_date)
            return company_founding_date

        else:

            print(f"update: did not find the target value in the text - {company_founding_date}")
            print("update: initiating a secondary process to try to find the information")

            response = model.function_call(text, function="extract", params=["company name"])

            if "company_name" in response["llm_response"]:
                company_name = response["llm_response"]["company_name"][0]

                if company_name:
                    print(f"\nupdate: found the company name - {company_name} - now using to lookup in secondary source")

                    output = WikiParser().add_wiki_topic(company_name,target_results=1)

                    if output:

                        supplemental_text = output["articles"][0]["summary"]

                        if len(supplemental_text) > 150:
                            supplemental_text_pp = supplemental_text[0:150] + " ... "
                        else:
                            supplemental_text_pp = supplemental_text

                        print(f"update: using lookup - {company_name} - found secondary source article "
                              f"(extract displayed) - ", supplemental_text_pp)

                        new_response = model.function_call(supplemental_text,params=["company founding date"])

                        print("\nupdate: reviewed second source article - ", new_response["llm_response"])

                        if "company_founding_date" in new_response["llm_response"]:
                            company_founding_date = new_response["llm_response"]["company_founding_date"]
                            if company_founding_date:
                                print("update: success - found the answer - ", company_founding_date)

    return company_founding_date


if __name__ == "__main__":

    founding_date = not_found_then_triage_lookup()
Enter fullscreen mode Exit fullscreen mode

您也可以在我们的 GitHub 仓库中找到完整的集成代码，链接在此。

此外，笔记本版本（ipynb）可在此处获取。

结论🤖

穿苏格兰裙的男人

处理“未找到”结果是一项棘手的难题，但通过周密的设计可以有效缓解。LLMWare 通过实施更广泛的查找等技术，旨在提升其人工智能系统的整体用户体验和可靠性。

请访问我们的 GitHub 页面并点个赞！https://github.com/llmware-ai/llmware

请在此处加入我们的Discord服务器：https://discord.gg/MgRaZz2VAB

请访问我们的网站llmware.ai了解更多信息和最新动态。

文章来源：https://dev.to/llmware/the-hardest-problem-in-rag-handling-not-found-answers-7md
如何使用 CSS Grid 和 C...
为什么大多数编程语言的索引都从零开始？