片段汇总

本文介绍片段汇总的设计思路，不等同于当前问答中的会话摘要。当前实际执行的是证据核验与迭代检索；历史仅在 token 超预算时压缩，见上下文压缩。

片段汇总

功能背景

在 RAG 系统中，向量检索对于单个问题的检索准确度较高，但在处理复杂的汇总类问题时，检索的准确度会有所下降。这是因为大模型将多个片段添加到上下文中，导致信息的冗余和干扰。为了解决这一问题，本项目引入了片段汇总功能，通过对大量片段内容进行总结，提高复杂问题的回答准确率。

实现流程

  1. 文档拆分：将原始文档拆分为多个片段，每个片段约 1000 个 token。
  2. 片段总结：
    • 使用预定义的总结提示词对每个片段内容进行总结，生成简洁且结构化的摘要。
    • 将生成的摘要存储到 moss_kb_sentence 表中，类型设置为 2，表示为总结内容。
  3. 向量化处理：
    • 对总结后的片段内容进行向量化，以便后续的语义相似度计算和高效检索。
  4. 存储与管理：
    • 将总结内容及其向量化结果存储到数据库中，便于快速检索和管理。

提示词

ParagraphSummaryPrompt.txt 文件内容

Summarize the following text in a concise, structured manner, ensuring that it covers the core themes and key information, suitable for use in vector similarity calculations:
Summary of

代码解析

MossKbParagraphSummaryService 类

该类负责对段落内容进行总结，并将总结结果缓存到数据库中，以提高总结效率和避免重复请求。

package nexus.io.mosskb.service.kb;

import java.util.concurrent.locks.Lock;

import com.google.common.util.concurrent.Striped;
import nexus.io.db.activerecord.Db;
import nexus.io.mosskb.model.MossKbParagraphSummaryCache;
import nexus.io.openai.chat.OpenAiChatResponseVo;
import nexus.io.openai.client.OpenAiClient;
import nexus.io.template.PromptEngine;
import nexus.io.tio.utils.crypto.Md5Utils;
import nexus.io.tio.utils.snowflake.SnowflakeIdUtils;

public class MossKbParagraphSummaryService {

  private static final Striped<Lock> locks = Striped.lock(1024);
  String prompt = PromptEngine.renderToString("ParagraphSummaryPrompt.txt");
  String selectContentSql = "select content from moss_kb_paragraph_summary_cache where md5=?";

  public String summary(String paragraphContent) {
    String md5 = Md5Utils.getMD5(paragraphContent);
    String summaryContent = Db.queryStr(selectContentSql, md5);
    if (summaryContent != null) {
      return summaryContent;
    }

    prompt += "\r\n\r\n" + paragraphContent;
    Lock lock = locks.get(md5);
    lock.lock();
    try {
      summaryContent = Db.queryStr(selectContentSql, md5);
      if (summaryContent != null) {
        return summaryContent;
      }

      OpenAiChatResponseVo chat = null;
      long start = System.currentTimeMillis();
      try {
        chat = OpenAiClient.chat(prompt);
      } catch (Exception e) {
        try {
          chat = OpenAiClient.chat(prompt);
        } catch (Exception e1) {
          chat = OpenAiClient.chat(prompt);
        }
      }
      long end = System.currentTimeMillis();
      if (chat != null) {
        summaryContent = chat.getChoices().get(0).getMessage().getContent();
        MossKbParagraphSummaryCache model = new MossKbParagraphSummaryCache();
        model.setId(SnowflakeIdUtils.id()).setMd5(md5).setSrc(paragraphContent).setContent(summaryContent);
        model.setElapsed((end - start)).setSystemFingerprint(chat.getSystem_fingerprint()).setModel(chat.getModel())
            //
            .setCompletionTokens(chat.getUsage().getCompletion_tokens())
            //
            .setPromptTokens(chat.getUsage().getPrompt_tokens())
            //
            .setTotalTokens(chat.getUsage().getTotal_tokens());
        model.save();
        return summaryContent;
      }
    } finally {
      lock.unlock();
    }
    return null;
  }
}

代码说明：

  • 锁机制：使用 Striped<Lock> 确保在多线程环境下对同一段落内容的总结请求不会重复执行，避免资源浪费。
  • 摘要缓存：在对段落内容进行总结前，首先检查数据库中是否已存在该段落的摘要（通过 MD5 校验），若存在则直接返回，避免重复调用外部 API。
  • 外部 API 调用：若摘要不存在，则调用 OpenAI 的 Chat API 生成摘要，并将结果存储到 moss_kb_paragraph_summary_cache 表中，以供后续使用。

MossKbSentenceService 类

该类负责将段落摘要转换为句子级别的记录，并进行向量化处理，最终存储到数据库中。

package nexus.io.mosskb.service.kb;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletionService;
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.Future;

import org.postgresql.util.PGobject;

import nexus.io.db.activerecord.Db;
import nexus.io.db.activerecord.Row;
import nexus.io.jfinal.aop.Aop;
import nexus.io.mosskb.constant.TableNames;
import nexus.io.mosskb.model.MossKbSentence;
import nexus.io.mosskb.utils.ExecutorServiceUtils;
import nexus.io.tio.utils.crypto.Md5Utils;
import nexus.io.tio.utils.snowflake.SnowflakeIdUtils;

import dev.langchain4j.data.document.Document;
import dev.langchain4j.data.document.DocumentSplitter;
import dev.langchain4j.data.document.splitter.DocumentSplitters;
import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.openai.OpenAiTokenizer;
import lombok.extern.slf4j.Slf4j;

@Slf4j
public class MossKbSentenceService {
  MossKbEmbeddingService mossKbEmbeddingService = Aop.get(MossKbEmbeddingService.class);
  CompletionService<MossKbSentence> completionServiceSentence = new ExecutorCompletionService<>(ExecutorServiceUtils.getExecutorService());

  CompletionService<Row> completionServiceRow = new ExecutorCompletionService<>(ExecutorServiceUtils.getExecutorService());
  MossKbParagraphSummaryService mossKbParagraphSummaryService = Aop.get(MossKbParagraphSummaryService.class);

  public boolean summaryToSentenceAndSave(Long dataset_id, String modelName, List<Row> paragraphRecords, Long documentIdFinal) {
    boolean transactionSuccess;

    // Step 1: Generate summaries asynchronously
    List<Future<MossKbSentence>> summaryFutures = new ArrayList<>(paragraphRecords.size());

    for (Row paragraph : paragraphRecords) {
      Future<MossKbSentence> future = completionServiceSentence.submit(() -> {
        String paragraphContent = paragraph.getStr("content");
        // Generate summary for the paragraph
        String sentenceContent = mossKbParagraphSummaryService.summary(paragraphContent);

        // Create MossKbSentence object with the summary
        MossKbSentence mossKbSentence = new MossKbSentence();
        mossKbSentence.setId(SnowflakeIdUtils.id()).setType(2) // Assuming type 2 indicates a summary
            .setHitNum(0).setMd5(Md5Utils.getMD5(sentenceContent)).setContent(sentenceContent)
            //
            .setDatasetId(dataset_id).setDocumentId(documentIdFinal).setParagraphId(paragraph.getLong("id"));

        return mossKbSentence;
      });
      summaryFutures.add(future);
    }

    // Step 2: Retrieve summaries from futures
    List<MossKbSentence> sentences = new ArrayList<>();
    for (Future<MossKbSentence> future : summaryFutures) {
      try {
        MossKbSentence sentence = future.get(); // This will block until the summary is ready
        if (sentence != null) {
          sentences.add(sentence);
        }
      } catch (Exception e) {
        log.error("Error generating summary: {}", e.getMessage(), e);
      }
    }

    // Step 3: Generate vectors asynchronously for each summary
    List<Future<Row>> vectorFutures = new ArrayList<>(sentences.size());
    for (MossKbSentence sentence : sentences) {
      Future<Row> future = completionServiceRow.submit(() -> {
        // Generate vector for the summary content
        PGobject vector = mossKbEmbeddingService.getVector(sentence.getContent(), modelName);
        Row record = sentence.toRecord();
        record.set("embedding", vector);
        return record;
      });
      vectorFutures.add(future);
    }

    // Step 4: Retrieve vectors from futures
    List<Row> sentenceRows = new ArrayList<>();
    for (Future<Row> future : vectorFutures) {
      try {
        Row record = future.get(); // This will block until the vector is ready
        if (record != null) {
          sentenceRows.add(record);
        }
      } catch (Exception e) {
        log.error("Error generating vector: {}", e.getMessage(), e);
      }
    }

    // Step 5: Save all sentence records with embeddings to the database within a transaction
    transactionSuccess = Db.tx(() -> {
      // Delete existing sentences for the document to avoid duplicates
      Db.deleteById(TableNames.moss_kb_sentence, "document_id", documentIdFinal);
      // Batch save the new sentences with embeddings
      Db.batchSave(TableNames.moss_kb_sentence, sentenceRows, 2000);
      return true;
    });

    return transactionSuccess;
  }
}

代码说明：

  • 并行处理：利用 CompletionService 实现异步处理，提升系统的处理效率。
    • 生成摘要：对每个段落内容调用 MossKbParagraphSummaryService 生成摘要。
    • 生成向量：对生成的摘要内容调用 MossKbEmbeddingService 进行向量化处理。
  • 事务管理：在数据库操作中使用事务，确保数据的一致性和完整性。在保存新的句子记录前，删除已有的相关记录，避免数据重复。
  • 错误处理：对异步任务中的异常进行捕获和日志记录，确保系统的稳定性。

汇总前后的片段对比

为了直观展示片段汇总功能的效果，以下展示了在应用汇总功能前后，片段内容的对比。

汇总前 moss_kb_paragraph_summary_cache 表内容示例

He preferred to stick to the essential facts—birth, death, and emancipation.

## Conclusion

Sometime in 1865, a Union soldier approached Ella in the middle of her chores.

“A soldier rode up to my ma and told her she was free.”

The starkness of Ella’s story stunned me.

Her life consisted of two essential facts—slavery and freedom juxtaposed to mark the beginning and end of the chronicle.

But this was what slavery did: it stripped your history to bare facts and
# Family Stories

I don’t know if it was the bare bones of Ella’s story or the hopefulness and despair that lurked in Poppa’s words as he recounted it, as if he were weighing the promise of freedom against the vast stretches of stolen land before him, that made me eager to know more than what Poppa remembered or wanted to share.

Peter and I listened, silent.

We didn’t know what to say.

Poppa didn’t remember any kin before his grandmother, who smoked a corncob pipe.

He had inherited his love of pipes from her.

It was one of the things I adored about him.

He always smelled sweet like the maple tobacco smoke rising from his pipe.

What he knew about our family ended with his grandmother Ellen.

He remembered no other names.

When he spoke of these things, I saw how the sadness and anger of not knowing his people distorted the soft lines of Poppa’s face.

It surprised me; he had always seemed invincible, strapping, six foot two, and handsome, even at eighty-five.

I had seen this ache in others too.

At a barbecue at my grandmother’s, two of her cousins nearly came to blows disputing a grandfather’s name.

I was still too young then to recognize the same feelings inside me.

But I wondered about my great-great-grandmother’s mother, as well as all the others who had been forgotten.

If Poppa’s mother or grandmother shared any details about their lives in slavery, he didn’t share them with my brother and me. No doubt he was unwilling to disclose what he considered unspeakable. Still, he shared more with my brother and me than he had with my mother. Even now, he liked to call her “little girl.” When I returned home and asked her if Poppa had ever spoken to her about slavery or her great-grandmother Ella, the girl on the road, she replied, “When I was growing up we didn’t talk of such things.” Her great-grandmother had died before she was born, so my mother recalled nothing about her, not even her name.

At twelve I became obsessed with the maternal great-great-grandmother I had never known, endlessly constructing and rearranging the scene: her unease as the soldier advanced toward her, or the soldier on horseback looming over her and the smile inching across her face as she digested his words, or the peal of laughter trailing behind her as she turned upon her heels, or the war between disbelief and wonder that overcame Ella as she bolted toward her mother. Mulling over the details Poppa had shared with me, I tried to fill in the blank spaces of the story, but I never succeeded. Since that afternoon with my great-grandfather, I had been looking for relatives whose only proof of existence was fragments of stories and names that repeated themselves across generations.

Unlike friends who possessed a great trove of family photographs, I had no idea what my great-grandmother looked like or even my grandaunts when they were girls. All these things were gone; some of the photographs were given as tokens to dead relatives and buried with them; others were lost. The images I possessed of them were drawn from memory and imagination. My aunt Mosella, whose name itself was a memorial to my great-grandfather Moses and his mother Ella, once described a photograph taken of her mother and my great-great-grandmother Polly, whom everyone called Big Momma. In the photograph, her mother, Lou, was wearing a ruffled dress with bloomers and seated on Big Momma’s lap. She didn’t remember what my great-great-grandmother wore but only echoed my mother’s description of her: she was a big-boned woman with a round face the color of dark chocolate.

# Big Momma's Stories

Big Momma had never spoken of her life in slavery, nor had Ellen or Ella. Poppa could fill in only the bare outlines of their lives. The gaps and silences of my family were not unusual: slavery made the past a mystery, unknown and unspeakable.

## The Tales of the Past

汇总后

### Summary of "Afrotopia"

#### Core Themes:
1. **Strangerhood and Identity**: The author's feeling of being an outsider in Ghana and the broader implications of being labeled as an "obruni" highlight themes of alienation and cultural disconnection.
2. **Historical Legacy of Slavery**: The narrative emphasizes the long-lasting impact of slavery on identity and belonging, conveying personal stories that interweave familial history with the collective trauma of African descendants.
3. **Disillusionment with Idealization**: The text critiques the romantic notions surrounding a return to Africa, revealing the realities faced by expatriates, and contrasts the expectations of a utopian African experience with the complexities of its history and current socio-economic challenges.

#### Key Information:
- **Experiences in Ghana**: The author navigates her feelings of isolation and alienation, grappling with the term "obruni" as a constant reminder of her foreignness.
- **Family History and Memory**: Personal anecdotes reflect the struggle to connect with ancestors’ experiences of slavery, and a longing to reclaim lost identities and narratives, despite the silence surrounding these histories.
- **Cultural Critique**: The text discusses the dichotomy of freedom and slavery, exploring how the legacies of colonialism and neocolonialism shape modern Ghanaian society and the expatriate experience.
- **Reflections on Independence**: The celebration of Ghana’s independence is juxtaposed with the ongoing struggles faced by its people, revealing the unmet expectations that arose from decolonization.
- **Coup Experience**: The author's encounter with a perceived coup highlights the tension and fears surrounding political instability, serving as a metaphor for the broader struggles of identity and belonging in a post-colonial context.

#### Conclusion:
The narrative underscores the complexities of returning to one's roots, blending personal, historical, and socio-political reflections to articulate the challenges and disillusionments faced by those navigating their identities in the wake of slavery and colonialism.

对比分析：

  • 汇总前：原始内容较为冗长，涵盖了多个主题和细节，适合深入阅读和理解。然而，对于向量检索和生成模型来说，过长的文本可能导致上下文信息过载，影响检索准确性。
  • 汇总后：通过结构化的总结，提炼出核心主题和关键信息，使内容更加简洁明了。这不仅有助于提高向量检索的效率和准确性，也为生成模型提供了更为聚焦和相关的上下文信息，提升复杂汇总类问题的回答质量。

结论

本项目成功构建了一个功能完备的 RAG 系统，涵盖了从用户认证、模型管理到知识库和文档管理的各个环节。通过引入片段汇总功能，显著提升了系统在处理复杂汇总类问题时的准确性和效率。未来，随着系统的不断优化和功能的扩展，RAG 系统将能够更好地服务于各类用户，满足其在信息检索和问答方面的多样化需求。