Knowledge Mechanisms in Large Language Models:
A Survey and Perspective大语言模型中的知识机制：综述与展望

Mengru Wang ^†^†thanks: ˜˜Equal Contribution. Email: mengruwg@zju.edu.cn Yunzhi Yao¹¹footnotemark: 1 Email: zhangningyu@zju.edu.cn Ziwen Xu Shuofei Qiao Shumin Deng Affiliation: Zhejiang University, National University of Singapore, NUS-NCS Joint Lab, Singapore Peng Wang Xiang Chen Jia-Chen Gu Affiliation: University of California, Los Angeles, Yong Jiang Affiliation: Alibaba Group Pengjun Xie Affiliation: Alibaba Group Fei Huang Affiliation: Alibaba Group Huajun Chen Ningyu Zhang ^†^†thanks: ˜˜Corresponding Author.

Abstract摘要

Understanding knowledge mechanisms in Large Language Models (LLMs) is crucial for advancing towards trustworthy AGI. This paper reviews knowledge mechanism analysis from a novel taxonomy including knowledge utilization and evolution. Knowledge utilization delves into the mechanism of memorization, comprehension and application, and creation. Knowledge evolution focuses on the dynamic progression of knowledge within individual and group LLMs. Moreover, we discuss what knowledge LLMs have learned, the reasons for the fragility of parametric knowledge, and the potential dark knowledge (hypothesis) that will be challenging to address. We hope this work can help understand knowledge in LLMs and provide insights for future research.理解大语言模型（LLM）中的知识机制，对于迈向可信通用人工智能（AGI）至关重要。本文从知识利用和知识演化这一新颖分类法出发，对知识机制分析进行了综述。知识利用深入探讨了记忆、理解与应用以及创造的机制；知识演化则关注个体和群体 LLM 中知识的动态演进过程。此外，我们还讨论了 LLM 习得了哪些知识、参数化知识脆弱性的原因，以及未来将面临挑战的潜在“暗知识”（假设）。我们希望这项工作能有助于理解 LLM 中的知识，并为未来的研究提供启示。

1 Introduction1 引言

Knowledge is the cornerstone of intelligence and the continuation of civilization, furnishing us with foundational principles and guidance for navigating complex problems and emerging challenges Davis et al. (1993); Choi (2022). Throughout the extensive history of evolution, we have dedicated our lives to cultivating more advanced intelligence by utilizing acquired knowledge and exploring the frontiers of unknown knowledge McGraw and Harbison-Briggs (1990); Han et al. (2021).知识是智能的基石，也是文明的延续，它为我们应对复杂问题和新兴挑战提供了基础原则与指导（Davis 等，1993；Choi，2022）。纵观漫长的进化史，我们致力于通过利用已获取的知识并探索未知知识的边界，来培养更高级的智能（McGraw 和 Harbison-Briggs，1990；Han 等，2021）。

As we know, Large language models (LLMs) are renowned for encapsulating extensive parametric knowledge Roberts et al. (2020); Sung et al. (2021); Cao et al. (2021a); Zhong et al. (2021); Kandpal et al. (2023); Heinzerling and Inui (2020); Petroni et al. (2019); Qiao et al. (2023); Kritharoula et al. (2023); He et al. (2024a), achieving unprecedented progress in application. However, the knowledge mechanisms in LLMs for learning, storage, utilization, and evolution still remain mysterious Phillips et al. (2021); Gould et al. (2023a); Wang et al. (2024e). Extensive works aim to demystify various types of knowledge in LLMs through knowledge neurons Dai et al. (2022); Chen et al. (2024a) and circuits Elhage et al. (2021); Yao et al. (2024); Zou et al. (2024), yet these efforts, scattered across various tasks, await comprehensive review and analysis.众所周知，大语言模型（LLM）以封装了海量参数化知识而闻名（Roberts 等，2020；Sung 等，2021；Cao 等，2021a；Zhong 等，2021；Kandpal 等，2023；Heinzerling 和 Inui，2020；Petroni 等，2019；Qiao 等，2023；Kritharoula 等，2023；He 等，2024a），并在应用中取得了前所未有的进展。然而，LLM 在知识学习、存储、利用和演化方面的机制仍然神秘（Phillips 等，2021；Gould 等，2023a；Wang 等，2024e）。大量研究旨在通过知识神经元（Dai 等，2022；Chen 等，2024a）和知识回路（Elhage 等，2021；Yao 等，2024；Zou 等，2024）来揭示 LLM 中各类知识的奥秘，但这些分散在各项任务中的研究尚待全面梳理与分析。

Refer to caption — Figure 1: The analysis framework of knowledge mechanism within neural models includes knowledge evolution and utilization. Dark knowledge denotes knowledge unknown to human or model (machine). We investigate the mechanisms of knowledge utilization (right) in LLMs during a specific period of their evolution (left). The knowledge limitations identified through mechanisms analysis will inspire subsequent evolution (left).图 1：神经网络模型内部的知识机制分析框架，包含知识演化和知识利用。暗知识是指人类或模型（机器）未知的知识。我们研究了 LLM 在其演化特定阶段的知识利用机制（右侧）。通过机制分析识别出的知识局限性将启发后续的演化（左侧）。

As shown in Fig 1, this paper pioneeringly reviews the mechanism across the whole knowledge life cycle. We also propose a novel taxonomy for knowledge mechanisms in LLMs, as illustrated in Fig 2, which encompasses knowledge utilization at a specific time and knowledge evolution across all periods of LLMs ¹¹ 1 Knowledge utilization focuses on static knowledge at a specific period, while knowledge evolution explores the long-term dynamic development of knowledge across individual and group LLMs.. Specifically, we introduce preliminaries of this field (§2) and review the knowledge utilization mechanism from a new perspective (§3), delve into the fundamental principles for knowledge evolution (§4). Then, we investigate how to construct more efficient and trustworthy LLMs from the perspective of knowledge mechanism (§5). Later, We discuss open questions about the knowledge LLMs have and have not acquired (§6). Finally, we also provide some future directions (§7) and tools for knowledge mechanism analysis (§C). Our contributions are as follows:如图 1 所示，本文开创性地对整个知识生命周期的机制进行了综述。我们还提出了一种 LLM 知识机制的新分类法（如图 2 所示），涵盖了特定时间点的知识利用以及 LLM 全周期的知识演化。知识利用侧重于特定时期内的静态知识，而知识演化则探索个体和群体 LLM 中知识的长期动态发展。具体而言，我们介绍了该领域的预备知识（第 2 节），从新视角综述了知识利用机制（第 3 节），并深入探讨了知识演化的基本原则（第 4 节）。随后，我们从知识机制的角度研究了如何构建更高效、更可信的 LLM（第 5 节）。接着，我们讨论了关于 LLM 已习得及未习得知识的开放性问题（第 6 节）。最后，我们提供了一些未来研究方向（第 7 节）以及知识机制分析工具（附录 C）。我们的贡献如下：

{forest} {forest}

Figure 2: The taxonomy of knowledge mechanisms in LLMs.图 2：LLM 知识机制的分类法。

•

To the best of our knowledge, we are the first to review knowledge mechanisms in LLMs and provide a novel taxonomy across the entire life.据我们所知，我们是首个综述 LLM 知识机制并提供涵盖整个生命周期的新颖分类法的研究。
•

We propose a new perspective to analyze knowledge utilization mechanisms from three levels: memorization, comprehension and application, and creation.我们提出了一个分析知识利用机制的新视角，分为三个层次：记忆、理解与应用，以及创造。
•

We discuss knowledge evolution in individual and group LLMs, and analyze the inherent conflicts and integration in this process.我们讨论了个体和群体 LLM 中的知识演化，并分析了这一过程中的内在冲突与整合。
•

We observe that LLMs have learned basic world knowledge. However, the learned knowledge is fragile, leading to challenges such as hallucinations and knowledge conflicts. We speculate that this fragility may be primarily due to improper learning data. Besides, the unlearned dark knowledge will exist long.我们观察到 LLM 已经习得了基础世界知识。然而，这些习得的知识十分脆弱，导致了幻觉和知识冲突等挑战。我们推测这种脆弱性主要源于学习数据的不当。此外，未习得的暗知识将长期存在。

Comparison with Existing Surveys与现有综述的比较

Previous interpretability surveys typically aim to investigate various methods for explaining the roles of different components within LLMs from the global and local taxonomy Ferrando et al. (2024); Zhao et al. (2024a); Luo and Specia (2024); Murdoch et al. (2019); Rai et al. (2024a); Bereska and Gavves (2024); Vilas et al. (2024); Singh et al. (2024); Rai et al. (2024b). In contrast, this paper focuses on knowledge in LLMs. Hence, our taxonomy, oriented from target knowledge in LLMs, reviews how knowledge is acquired, stored, utilized, and subsequently evolves. Additionally, previous taxonomy mostly explore the explainability during the inference stage (a specific period), while ignoring knowledge acquisition during the pre-training stage and evolution during the post-training stage Räuker et al. (2023); Luo et al. (2024b); Apidianaki (2023); Jiao et al. (2023); Räuker et al. (2023); Rai et al. (2024c). Our taxonomy aims to explore the dynamic evolution across all periods from naivety to sophistication in both individual and group LLMs. In contrast to the most similar survey Cao et al. (2024a) that introduces knowledge life cycle, our work focuses on the underlying mechanisms at each stage.以往的可解释性综述通常旨在从全局和局部分类法出发，调查解释 LLM 内部不同组件作用的各种方法（Ferrando 等，2024；Zhao 等，2024a；Luo 和 Specia，2024；Murdoch 等，2019；Rai 等，2024a；Bereska 和 Gavves，2024；Vilas 等，2024；Singh 等，2024；Rai 等，2024b）。相比之下，本文聚焦于 LLM 中的知识。因此，我们的分类法以 LLM 中的目标知识为导向，综述了知识是如何被获取、存储、利用并随后演化的。此外，以往的分类法大多探讨推理阶段（特定时期）的可解释性，而忽略了预训练阶段的知识获取以及后训练阶段的知识演化（Räuker 等，2023；Luo 等，2024b；Apidianaki，2023；Jiao 等，2023；Räuker 等，2023；Rai 等，2024c）。我们的分类法旨在探索个体和群体 LLM 从幼稚到成熟全过程中的动态演化。与引入知识生命周期的最相似综述（Cao 等，2024a）不同，我们的工作侧重于各阶段的底层机制。

Generally, this paper may help us to explore and manipulate advanced knowledge in LLMs, examine current limitations through the history of knowledge evolution, and inspire more efficient and trustworthy architecture and learning strategy for future models from knowledge mechanism perspective. Note that most hypotheses in this paper are derived from transformer-based LLMs. We also validate the generalizability of these hypotheses across other architectural models and then propose universality intelligence in §B.总体而言，本文有助于我们探索和操控 LLM 中的高级知识，通过知识演化史审视当前的局限性，并从知识机制的角度为未来模型启发更高效、更可信的架构和学习策略。请注意，本文中的大多数假设均源于基于 Transformer 的 LLM。我们还在附录 B 中验证了这些假设在其他架构模型中的普适性，并提出了通用智能。

2 Preliminary2 预备知识

2.1 Knowledge Scope2.1 知识范畴

Knowledge is an awareness of facts, a form of familiarity, awareness, understanding, or acquaintance Zagzebski (2017); Hyman (1999); Mahowald et al. (2023); Gray et al. (2024). It often involves the possession of information learned through experience and can be understood as a cognitive success or an epistemic contact with reality. We denote a diverse array of knowledge as set $\mathbf{K}$ , wherein each element $k\in\mathbf{K}$ is a specific piece of knowledge, which can be expressed by various records, e.g., a text record “The president of the United States in 2024 is Biden” (denoted as $r_{k}$ ).知识是对事实的认知，是一种熟悉感、意识、理解或认识（Zagzebski，2017；Hyman，1999；Mahowald 等，2023；Gray 等，2024）。它通常涉及通过经验习得的信息，可以被理解为一种认知上的成功或与现实的认知接触。我们将多种多样的知识表示为集合 𝐊，其中每个元素 k∈𝐊 都是一条具体的知识，可以通过各种记录来表达，例如文本记录“2024 年美国总统是拜登”（记作 r_{k}）。

2.2 Definition of Knowledge in LLMs2.2 LLM 中的知识定义

Given a LLM denoted as $\mathcal{F}$ , we formulate that $\mathcal{F}$ master knowledge $k$ if $\mathcal{F}$ can correctly answer the corresponding question $r_{k\backslash t}$ :给定一个 LLM 记作 ℱ，如果 ℱ 能够正确回答相应的记录 r_{k\backslash t}，则我们认为 ℱ 掌握了知识 k：

		$\displaystyle t=\mathcal{F}\left(r_{k\backslash t}\right)$		(1)
		$\displaystyle(t\in\mathbf{T})\Rightarrow(\mathcal{F}\ \text{masters knowledge}\ k),$		(1)

$t$ is the output of a LLM $\mathcal{F}$ , $r_{k\backslash t}$ is a record about knowledge $k$ that lacks pivot information. Take an example for illustration: $r_{k\backslash t}$ is “The president of the United States in 2024 is __”, the pivot information is “Biden”. Note that, $r_{k\backslash t}$ can be represented by the above textual statement, captured through a question-answering pair (“Who is the President of the United States in 2024?’’, or conveyed by audio, video, image ²² 2 While audio, video, and image records have been somewhat investigated, they are still relatively unexplored areas and thus are only discussed in §7.2 and §B.2., and other equivalent expressions. The pivot information for $r_{k\backslash t}$ can be expressed by various fomarts, which are formulated as $\mathbf{T}=\left\{\text{``Biden''},\text{``Joe Biden''},\cdots\right\}$ . If the output $t$ is an element from the correct answer set $\mathbf{T}$ , we hypothesize that $\mathcal{F}$ master knowledge $k$ .t 是 LLM ℱ 的输出，r_{k\backslash t} 是关于知识 k 且缺失关键信息的记录。举例说明：r_{k\backslash t} 为“2024 年美国总统是 __”，关键信息即为“拜登”。请注意，r_{k\backslash t} 可以通过上述文本陈述表示，可以通过问答对（“谁是 2024 年美国总统？”）捕获，也可以通过音频、视频、图像（尽管音频、视频和图像记录已得到一定研究，但仍属于相对未开发的领域，因此仅在第 7.2 节和附录 B.2 中讨论）以及其他等效表达来传达。r_{k\backslash t} 的关键信息可以用多种格式表达，记作 𝐓={“拜登”, “乔·拜登”, ⋯}。如果输出 t 是正确答案集 𝐓 中的一个元素，我们假设 ℱ 掌握了知识 k。

2.3 The Architecture of LLMs2.3 LLM 的架构

An LLM $\mathcal{F}$ consists of numerous neurons, which work systematically under a specific architecture.LLM ℱ 由大量神经元组成，这些神经元在特定架构下系统地工作。

Transformer-based architecture.基于 Transformer 的架构。

The prevailing architecture in current LLMs is the Transformer Vaswani et al. (2017). Specifically, a transformer-based LLM $\mathcal{F}$ begins with a token embedding, followed by $L$ layers transformer block, and ends with token unembedding used for predicting answer tokens. Each transformer block layer $l$ consists of Attention Heads (Attention) and Multilayer Perceptron (MLP):当前 LLM 中的主流架构是 Transformer（Vaswani 等，2017）。具体而言，基于 Transformer 的 LLM ℱ 以词嵌入（token embedding）开始，后接 LL 层 Transformer 块，最后以用于预测答案 token 的词反嵌入（token unembedding）结束。每个 Transformer 块层 l 由注意力头（Attention）和多层感知机（MLP）组成：

\displaystyle h_{l+1}=h_{l}+\text{MLP}\left(h_{l}+\text{Attention}\left(h_{l}\right)\right),

(2)

$h_{l}$ is the hidden state from $l$ -th layer.h_{l} 是来自第 l 层的隐藏状态。

Other architectures.其他架构。

Other architectures including competitive variants of the transformer, e.g., SSM Gu and Dao (2023), TTT Sun et al. (2024) and RWKV Peng et al. (2023), and architectures in computer vision Li et al. (2023c) and multi-modal fields are detailed in §B.1.包括 Transformer 的竞争变体（如 SSM（Gu 和 Dao，2023）、TTT（Sun 等，2024）和 RWKV（Peng 等，2023））以及计算机视觉（Li 等，2023c）和多模态领域的其他架构，详见附录 B.1。

2.4 Knowledge Analysis Methods2.4 知识分析方法

Knowledge analysis method $\mathcal{M}$ aims to interpret how LLMs work inside and reveal precise causal connections between specific components and outputs Bereska and Gavves (2024). Furthermore, if components $\mathbf{C}$ of $\mathcal{F}$ accurately infer $t$ through analysis method $\mathcal{M}$ , it is assumed that the knowledge $k$ is presented by $\mathbf{C}$ :知识分析方法 ℳ 旨在解释 LLM 的内部工作原理，并揭示特定组件与输出之间的精确因果联系（Bereska 和 Gavves，2024）。此外，如果 ℱ 的组件 𝐂 通过分析方法 ℳ 准确推断出 t，则假设知识 k 由 𝐂 呈现：

		$\displaystyle t=\mathcal{M}_{\mathbf{C}\subseteq\mathcal{F}}\left(r_{k\backslash t},\mathbf{C}\right),$		(3)
		$\displaystyle(t\in\mathbf{T})\Rightarrow(\mathbf{C}\ \text{represents knowledge}\ k),$		(3)

The elements in set $\mathbf{C}$ may be individual neurons, MLPs, attention heads, a transformer block layer, or knowledge circuit Yao et al. (2024). These methods are divided into two categories: observation and intervention Bereska and Gavves (2024).集合 𝐂 中的元素可以是单个神经元、MLP、注意力头、Transformer 块层或知识回路（Yao 等，2024）。这些方法分为两类：观察法和干预法（Bereska 和 Gavves，2024）。

Observation-based methods.基于观察的方法。

These methods aim to observe the internal information of $\mathcal{F}$ , directly projecting the output of component $\mathbf{C}$ into human-understandable forms by $E$ :这些方法旨在观察 ℱ 的内部信息，通过 EE 将组件 𝐂 的输出直接投影为人类可理解的形式：

\displaystyle t=E_{\mathbf{C}\subseteq\mathcal{F}}\left(r_{k\backslash t},\mathbf{C},\mathcal{F}\right),

(4)

$E$ is a evaluation metric, which can be a probe Räuker et al. (2023), logit lens nostalgebraist (2020), or a sparse representation Gao et al. (2024c). Probe is a meticulously trained classifier, and its classification performance is used to observe the relationship between model’s behavior and the output of $\mathbf{C}$ Belinkov (2022); Elazar et al. (2021); McGrath et al. (2021); Gurnee et al. (2023). Logit lens usually translate output of $\mathbf{C}$ into vocabulary tokens via token unembedding Geva et al. (2022b); Belrose et al. (2023); Pal et al. (2023); Din et al. (2024); Langedijk et al. (2023). Sparse representation maps the output of $\mathbf{C}$ into a higher-dimensional space with strong sparsity through dictionary learning He et al. (2024b); Olshausen and Field (1997); Yun et al. (2021); Karvonen et al. (2024), with sparse auto-encoder Sharkey et al. (2022); Cunningham et al. (2023); Lee et al. (2006); Gao et al. (2024a) being a prominent example. The higher-dimensional space represents independent (or monosemantic Bricken et al. (2023)) and interpretable features more easily Rai et al. (2024c). The output of $\mathbf{C}$ is the combination Elhage et al. (2022); Bricken et al. (2023) of these features.EE 是一种评估指标，可以是探针（Räuker 等，2023）、Logit Lens（nostalgebraist，2020）或稀疏表示（Gao 等，2024c）。探针是一种经过精心训练的分类器，其分类性能用于观察模型行为与 𝐂 输出之间的关系（Belinkov，2022；Elazar 等，2021；McGrath 等，2021；Gurnee 等，2023）。Logit Lens 通常通过词反嵌入将 𝐂 的输出转换为词表 token（Geva 等，2022b；Belrose 等，2023；Pal 等，2023；Din 等，2024；Langedijk 等，2023）。稀疏表示通过字典学习（He 等，2024b；Olshausen 和 Field，1997；Yun 等，2021；Karvonen 等，2024）将 𝐂 的输出映射到具有强稀疏性的高维空间，其中稀疏自编码器（Sharkey 等，2022；Cunningham 等，2023；Lee 等，2006；Gao 等，2024a）是一个显著例子。高维空间能更轻松地表示独立（或单义的，Bricken 等，2023）且可解释的特征（Rai 等，2024c）。𝐂 的输出是这些特征的组合（Elhage 等，2022；Bricken 等，2023）。

Intervention-based methods.基于干预的方法。

These methods allow for direct corruptions in LLMs to identify the critical $\mathbf{C}$ via intervention strategies $\mathcal{I}$ . Note that $\mathbf{C}$ , encompassing various neuron combinations, correlates with specific model behaviors:这些方法允许对 LLM 进行直接破坏，以通过干预策略 ℐ 识别关键的 𝐂。请注意，包含各种神经元组合的 𝐂 与特定的模型行为相关联：

		$\displaystyle\mathbf{C}=\mathcal{I}\left(r_{k\backslash t},\mathcal{F}\right),$		(5)
		$\displaystyle t=E\left(r_{k\backslash t},\mathbf{C},\mathcal{F}\right)$		(5)

$\mathcal{I}$ is also known as causal mediation analysis Vig et al. (2020), causal tracing Meng et al. (2022), interchange interventions Geiger et al. (2022), activation patching Wang et al. (2023c); Zhang and Nanda (2023), path patching Goldowsky-Dill et al. (2023), and causal scrubbing techniques LawrenceC et al. (2022). Specifically, $\mathcal{I}$ consists of the following three steps. 1) Clean run: $\mathcal{F}$ generates the correct answer $t$ based on the input $r_{k\backslash t}$ . 2) Corrupted run: corrupt the generation process of $\mathcal{F}$ in the clean run by introducing noise into the input or neurons Meng et al. (2022); Goldowsky-Dill et al. (2023); Stolfo et al. (2023); Yao et al. (2024); Conmy et al. (2023); Mossing et al. (2024); Lepori et al. (2023); Huang et al. (2023a). 3) Restoration run: recover the correct answer $t$ by restoring unnoised information from $\mathbf{C}$ Meng et al. (2022); Vig et al. (2020); Wang et al. (2023c); Zhang et al. (2017); Nanda (2023). For intervention-based methods, $E$ typically refers to the token unembedding used for predicting answer tokens. Under the evaluation metric $E$ , there exists a causal relationship between $\mathbf{C}$ and specific behavior of LLMs $\mathcal{F}$ in Eq 5.ℐ 也被称为因果中介分析（Vig 等，2020）、因果追踪（Meng 等，2022）、交换干预（Geiger 等，2022）、激活修补（Wang 等，2023c；Zhang 和 Nanda，2023）、路径修补（Goldowsky-Dill 等，2023）以及因果擦除技术（LawrenceC 等，2022）。具体而言，ℐ 包含以下三个步骤。1) 清洁运行：ℱ 根据输入 r_{k\backslash t} 生成正确答案 t。2) 损坏运行：通过在输入或神经元中引入噪声来破坏 ℱ 在清洁运行中的生成过程（Meng 等，2022；Goldowsky-Dill 等，2023；Stolfo 等，2023；Yao 等，2024；Conmy 等，2023；Mossing 等，2024；Lepori 等，2023；Huang 等，2023a）。3) 恢复运行：通过从 𝐂 恢复未加噪的信息来还原正确答案 t（Meng 等，2022；Vig 等，2020；Wang 等，2023c；Zhang 等，2017；Nanda，2023）。对于基于干预的方法，EE 通常指用于预测答案 token 的词反嵌入。在评估指标 EE 下，𝐂 与 LLM ℱ 的特定行为之间存在因果关系（式 5）。

3 Knowledge Utilization in LLMs3 LLM 中的知识利用

Inspired by Bloom’s Taxonomy of cognition levels Wilson (2016); Bloom et al. (1956); Keene et al. (2010); Fadul (2009), we categorize knowledge representation and utilization within LLMs into three levels (as shown in Fig 3): memorization, comprehension and application, and creation ³³ 3 Note that we combine analyzing, evaluating, and creating from Bloom’s Taxonomy into one category level (creation) in our taxonomy, as they are difficult to disentangle. Specifically, creation emphasizes the capacity and process of forming novel and valuable things. Analyzing Wilson (2016), which breaks materials or concepts into parts, is used for creating novel things. Evaluating Wilson (2016) is usually used for assessing the value of new creations.. Note that these mechanistic analyses are implemented via methods in §2.4. We further evaluate the applicability, advantages, and limitations of different methods in §3.4.受布鲁姆认知层次分类法（Wilson，2016；Bloom 等，1956；Keene 等，2010；Fadul，2009）的启发，我们将 LLM 内部的知识表示和利用分为三个层次（如图 3 所示）：记忆、理解与应用，以及创造。请注意，我们在分类法中将布鲁姆分类法中的分析、评价和创造合并为一个类别（创造），因为它们难以区分。具体而言，创造强调形成新颖且有价值事物的能力和过程。分析（Wilson，2016）将材料或概念分解为部分，用于创造新事物。评价（Wilson，2016）通常用于评估新创造的价值。请注意，这些机制分析是通过第 2.4 节中的方法实现的。我们将在第 3.4 节进一步评估不同方法的适用性、优势和局限性。

3.1 Memorization3.1 记忆

Knowledge memorization Schwarzschild et al. (2024); Prashanth et al. (2024) aims to remember and recall knowledge in the training corpus, e.g., specific terms (entities), grammar, facts, commonsense, concepts, etc Allen-Zhu and Li (2023a); Yu et al. (2023a); Mahowald et al. (2023); Zhu and Li (2023); Allen-Zhu and Li (2023b); Allen-Zhu and Li (2024); Cao et al. (2024a). We posit knowledge memorization from Modular Region and Connection Hypothesis by reviewing existing research.知识记忆（Schwarzschild 等，2024；Prashanth 等，2024）旨在记住和召回训练语料库中的知识，例如特定术语（实体）、语法、事实、常识、概念等（Allen-Zhu 和 Li，2023a；Yu 等，2023a；Mahowald 等，2023；Zhu 和 Li，2023；Allen-Zhu 和 Li，2023b；Allen-Zhu 和 Li，2024；Cao 等，2024a）。我们通过回顾现有研究，从模块化区域和连接假设的角度阐述知识记忆。

This modular region hypothesis simplifies knowledge representation in transformer-based models into isolated modular region, e.g., MLPs or attention heads. Knowledge is encoded via MLPs. Geva et al. (2021) posit that MLPs operate as key-value memories and each individual key vector corresponds to a specific semantic pattern or grammar. Based on the above finding, Geva et al. (2022b); Geva et al. (2022a) reverse engineer the operation of the MLPs layers and find that MLPs can promote both semantic (e.g., measurement semantic including kg, percent, spread, total, yards, pounds, and hours) and syntactic (e.g., adverbs syntactic including largely, rapidly, effectively, previously, and normally) concepts in the vocabulary space. Miller and Neo (2024) find a single MLP neuron (in GPT-2 Large) capable of generating “an” or “a”. Subsequently, fact Dai et al. (2022); Meng et al. (2022) and commonsense knowledge Gupta et al. (2023) are found. Advanced language-specific neurons Tang et al. (2024), linguistic regions Zhao et al. (2023a), entropy neurons Stolfo et al. (2024), query-relevant neurons Chen et al. (2024b), abstract conceptual Wang et al. (2024f) and unsafe Wang et al. (2024b); Wu et al. (2023a) knowledge, are also observed in MLPs. In addition to MLP, knowledge is also conveyed by attention heads Geva et al. (2023); Gould et al. (2023b). Hoover et al. (2020) explain the knowledge each attention head has learned. Specifically, attention heads store evident linguistic features, positional information, and so on. Besides, fact knowledge Yu et al. (2023c); Li et al. (2023a) and bias Hoover et al. (2020) are mainly convey by attention heads. Jiang et al. (2024b) further observe that LLMs leverage self-attention to gather information through certain tokens in the contexts, which serve as clues, and use the value matrix for associative memory. Later, Zhu et al. (2024) also find that attention heads can simulate mental state and activate “Theory of Mind” (ToM) capability.这种模块化区域假设将基于 Transformer 模型中的知识表示简化为孤立的模块化区域，例如 MLP 或注意力头。知识通过 MLP 编码。Geva 等（2021）认为 MLP 充当键值存储器，每个单独的键向量对应于特定的语义模式或语法。基于上述发现，Geva 等（2022b；2022a）对 MLP 层进行了逆向工程，发现 MLP 可以在词汇空间中促进语义（如度量语义，包括 kg、percent、spread、total、yards、pounds 和 hours）和句法（如副词句法，包括 largely、rapidly、effectively、previously 和 normally）概念。Miller 和 Neo（2024）发现单个 MLP 神经元（在 GPT-2 Large 中）能够生成“an”或“a”。随后，事实知识（Dai 等，2022；Meng 等，2022）和常识知识（Gupta 等，2023）被发现。在 MLP 中还观察到了高级语言特定神经元（Tang 等，2024）、语言区域（Zhao 等，2023a）、熵神经元（Stolfo 等，2024）、查询相关神经元（Chen 等，2024b）、抽象概念（Wang 等，2024f）和不安全知识（Wang 等，2024b；Wu 等，2023a）。除 MLP 外，知识也由注意力头传达（Geva 等，2023；Gould 等，2023b）。Hoover 等（2020）解释了每个注意力头所习得的知识。具体而言，注意力头存储明显的语言特征、位置信息等。此外，事实知识（Yu 等，2023c；Li 等，2023a）和偏见（Hoover 等，2020）主要由注意力头传达。Jiang 等（2024b）进一步观察到，LLM 利用自注意力通过上下文中的特定 token（充当线索）来收集信息，并利用值矩阵进行联想记忆。后来，Zhu 等（2024）还发现注意力头可以模拟心理状态并激活“心智理论”（ToM）能力。

However, Hypothesis 1 ignores the connections between different regions. Inspired by advancements in neuroscience de Schotten et al. (2022), Hypothesis 2 asserts that the connection of different components integrates knowledge, rather than the isolated regions in Hypothesis 1.然而，假设 1 忽略了不同区域之间的联系。受神经科学进展（de Schotten 等，2022）的启发，假设 2 断言不同组件的连接整合了知识，而非假设 1 中孤立的区域。

Geva et al. (2023) outline the encoding of factual knowledge (e.g., “The capital of Ireland is Dublin”) through the following three steps: (1) subject (Ireland) information enrichment in MLPs, (2) the relation (capital of) propagates to the last token, (3) object (Dublin) is extracted by attention heads in later layers. This claim is supported by Li et al. (2024d). Similarly, Lv et al. (2024) conclude that task-specific attention head may move the topic entity to the final position of the residual stream, while MLPs conduct relation function. Moreover, the recent prominent knowledge circuit framework Nainani (2024); Yao et al. (2024); He et al. (2024b); Elhage et al. (2021); Marks et al. (2024) advocates leveraging a critical computational subgraph among all components to explore internal knowledge within LLM parameters. The competencies for indirect object identification and color object tasks are discovered to be embedded in specialized knowledge circuits Conmy et al. (2023); Wang et al. (2023c); Merullo et al. (2023a); Yu et al. (2024c). Lan et al. (2024) also identify number-related circuits that encode the predictive ability of Arabic numerals, number words, and months. More importantly, experimental evidence demonstrates that various types of knowledge, including linguistic, commonsense, factual, and biased information, are encapsulated in specific knowledge circuits Yao et al. (2024). Interestingly, knowledge encoded by specific circuits can rival or even surpass that of the entire LLM. This may be because knowledge circuits memorized the relevant knowledge, while noise from other components might impede the model’s performance on these tasks.Geva 等（2023）概述了通过以下三个步骤编码事实知识（例如“爱尔兰的首都是都柏林”）：(1) 主体（爱尔兰）信息在 MLP 中富集，(2) 关系（首都）传播到最后一个 token，(3) 客体（都柏林）由后期层的注意力头提取。这一主张得到了 Li 等（2024d）的支持。同样，Lv 等（2024）得出结论，任务特定的注意力头可能会将主题实体移动到残差流的最终位置，而 MLP 执行关系函数。此外，最近著名的知识回路框架（Nainani，2024；Yao 等，2024；He 等，2024b；Elhage 等，2021；Marks 等，2024）提倡利用所有组件之间的关键计算子图来探索 LLM 参数内部的知识。间接宾语识别和颜色对象任务的能力被发现嵌入在专门的知识回路中（Conmy 等，2023；Wang 等，2023c；Merullo 等，2023a；Yu 等，2024c）。Lan 等（2024）还识别出了编码阿拉伯数字、数字词和月份预测能力的数字相关回路。更重要的是，实验证据表明，包括语言、常识、事实和偏见信息在内的各类知识都封装在特定的知识回路中（Yao 等，2024）。有趣的是，由特定回路编码的知识可以媲美甚至超过整个 LLM。这可能是因为知识回路记住了相关知识，而来自其他组件的噪声可能会阻碍模型在这些任务上的表现。

3.2 Comprehension and Application3.2 理解与应用

Knowledge comprehension and application focus on demonstrating the understanding of memorized knowledge and then solving problems in new situations, e.g., generalization on out-of-domain tasks Wang et al. (2024a), reasoning Hou et al. (2023) and planning McGrath et al. (2021). Merrill et al. (2023) denote the transition from memorization to comprehension and application as grokking, and suggest that the grokking derives from two largely distinct subnetworks competition. Intuitively, only knowledge that is correctly memorized Prashanth et al. (2024) in §3.1 can be further applied to solving complex tasks. Therefore, we posit the following Reuse Hypothesis from two knowledge memorization perspectives.知识理解与应用侧重于展示对已记忆知识的理解，并在新情境中解决问题，例如领域外任务的泛化（Wang 等，2024a）、推理（Hou 等，2023）和规划（McGrath 等，2021）。Merrill 等（2023）将从记忆到理解与应用的转变称为“顿悟”（grokking），并认为顿悟源于两个很大程度上不同的子网络的竞争。直觉上，只有在 3.1 节中正确记忆（Prashanth 等，2024）的知识才能进一步应用于解决复杂任务。因此，我们从两个知识记忆视角提出了以下重用假设。

From the Modular Region Perspective, knowledge utilization reuses some regions. These regions might include a few neurons, attention heads, MLPs, a transformer layer, or partial knowledge circuits. Generally, basic knowledge (position information, n-gram pattern, syntactic features) tends to be stored at earlier layers, while sophisticated knowledge (mental state, emotion, and abstract concept, e.g., prime number, Camelidae, and safety) is located at later layers Zhu et al. (2024); Jin et al. (2024a); Wang et al. (2024b); Wang et al. (2024f); Men et al. (2024); Kobayashi et al. (2023). Therefore, neurons of earlier layers related to basic knowledge tend to be reused Kang and Choi (2023); Zhao et al. (2024a); Kandpal et al. (2023). Various math reasoning tasks also utilize the attention mechanism in initial layers to map input information to the final token positions, subsequently generating answers using a set of MLPs in later layers Stolfo et al. (2023); Hanna et al. (2023); Langedijk et al. (2023). Besides, some specific function regions are also reused. Specifically, retrieval heads Li et al. (2023a) are reused for Chain-of-Thought (CoT) reasoning and long-context tasks. These retrieval heads are found in 4 model families, 6 model scales, and 3 types of fine-tuning. Subsequently, induction heads, identified in Llama and GPT, are claimed to be reused for in-context learning (ICL) tasks Olsson et al. (2022); Crosbie and Shutova (2024). Attention heads can map country names to their capitals in capital city-related tasks Lv et al. (2024). Language-specific neurons (in Llama and BLOOM) are responsible for multiple language related tasks, such as English, French, Mandarin, and others Tang et al. (2024). Zhao et al. (2023a) further reveal linguistic regions (in Llama) correspond to linguistic competence, which is the cornerstone for performing various tasks. Later, function regions related to the process of math reasoning are also discovered in LLMs. For instance, the last layer of GPT-2 (trained from scratch) has been observed to exhibit mathematical reasoning abilities across various math questions Ye et al. (2024). From the Connection Perspective, knowledge utilization shares partial knowledge circuits. For instance, similar tasks share subgraphs (computational circuits) with analogous roles Lan et al. (2024). Besides, knowledge circuits (in GPT2) are reused to solve a seemingly different task, e.g., indirect object identification and colored objects tasks Merullo et al. (2023a). Wang et al. (2024a) further observe that two-hop composition reasoning tasks reuse the knowledge circuits from the first hop. Yao et al. (2024) also believe that this reuse phenomenon exists in factual recall and multi-hop reasoning. Specifically, sub-circuits are reused in similar factual knowledge, such as tasks related to “city_in_country”, “name_birth_place”, and “country_language”. Besides, Dutta et al. (2024) demystify LLMs how to perform CoT reasoning, i.e., Llama facilitates CoT tasks via multiple parallel circuits enjoying significant intersection.

3.3 Creation3.3 创造

Knowledge creation Runco and Jaeger (2012); Sternberg (2006) emphasizes the capacity and process of forming novel and valuable things, rather than the existing ones (i.e., LLMs have seen) discussed in §3.1 and §3.2. The creations encompass two levels: 1) LLMs create new terms following the current world’s principles comprehended by LLMs, such as new proteins Shin et al. (2021), molecules Bagal et al. (2022); Fang et al. (2023); Edwards et al. (2022), code DeLorenzo et al. (2024), video Kondratyuk et al. (2023), models Zheng et al. (2024), names for people and companies, written stories Pépin et al. (2024); Gómez-Rodríguez and Williams (2023); Buz et al. (2024), synthetic data Stenger et al. (2024); Mumuni et al. (2024); Abufadda and Mansour (2021), etc. These novel items operate according to the existing rules, e.g., law of conservation of energy, reasoning logic Wang et al. (2024a), or principles of probability theory. 2) LLMs may generate new rules, such as mathematical theorems, and the resulting terms will operate according to the new rules. We posit that the knowledge creation of LLMs may derive from the Extrapolation Hypothesis.知识创造（Runco 和 Jaeger，2012；Sternberg，2006）强调形成新颖且有价值事物的能力和过程，而非 3.1 和 3.2 节中讨论的现有事物（即 LLM 已经见过的）。创造包含两个层次：1) LLM 遵循当前世界原则创建新术语，如新蛋白质（Shin 等，2021）、分子（Bagal 等，2022；Fang 等，2023；Edwards 等，2022）、代码（DeLorenzo 等，2024）、视频（Kondratyuk 等，2023）、模型（Zheng 等，2024）、人名和公司名、书面故事（Pépin 等，2024；Gómez-Rodríguez 和 Williams，2023；Buz 等，2024）、合成数据（Stenger 等，2024；Mumuni 等，2024；Abufadda 和 Mansour，2021）等。这些新颖项目根据现有规则运行，例如能量守恒定律、推理逻辑（Wang 等，2024a）或概率论原理。2) LLM 可能会生成新规则，例如数学定理，由此产生的术语将根据新规则运行。我们推测 LLM 的知识创造可能源于外推假设。

The expression of knowledge is diverse; some knowledge is inherently continuous. Therefore, it is difficult, if not impossible, to represent certain knowledge using discrete data points Spivey and Michael (2007); Penrose (); Markman (2013). LLMs utilize insights into the operational principles of the world to extrapolate additional knowledge from known discrete points, bridging gaps in knowledge and expanding our understanding of the worldHeilman et al. (2003); Douglas et al. (2024); Park et al. (2023b); Kondratyuk et al. (2023). Drawing inspiration from research on human creativity Haase and Hanel (2023), the physical implementation of knowledge extrapolation relies on the plasticity of neurons Mukherjee and Chang (2024). Specifically, plasticity refers to LLMs changing activations and connectivity between neurons according to the input Coronel-Oliveros et al. (2024).知识的表达是多样的；有些知识本质上是连续的。因此，使用离散数据点来表示某些知识是困难的，甚至是不可能的（Spivey 和 Michael，2007；Penrose；Markman，2013）。LLM 利用对世界运作原则的洞察，从已知的离散点外推额外知识，弥合知识差距并扩展我们对世界的理解（Heilman 等，2003；Douglas 等，2024；Park 等，2023b；Kondratyuk 等，2023）。受人类创造力研究（Haase 和 Hanel，2023）的启发，知识外推的物理实现依赖于神经元的可塑性（Mukherjee 和 Chang，2024）。具体而言，可塑性是指 LLM 根据输入改变神经元之间的激活和连接性（Coronel-Oliveros 等，2024）。

However, from statistical perspective, the intricate connections and activations between neurons, though not infinite, resist exhaustive enumeration. In terms of value, not all creations are valuable. Obtaining something valuable with an exceedingly low probability is impractical, as even a monkey could theoretically print Shakespeare’s works. How do LLMs ensure the probability of generating valuable creations? What are the mechanisms underlying the novelty and value of creation? A prevalent conjecture posits that novelty is generated through the random walk Sæbø and Brovold (2024). However, intuitively, current LLMs themselves seem unable to evaluate the value of creations due to architectural limitations Chakrabarty et al. (2024). Because, once the next token is generated, there is no intrinsic mechanism for accepting or rejecting the creations. This hinders the evaluation of the usefulness and value of proposed novelties, as humans do, by bending, blending, or breaking biases Sæbø and Brovold (2024). Some works assume that each token is indeed valuable and meets long-term expectations. However, the well-known hallucination problem Xu et al. (2024d) of LLMs refutes this assumption. Besides, the transformer architecture struggles with long context Li et al. (2024b), despite the existence of many variants for addressing this issue Huang et al. (2023c); Liu et al. (2024b). More importantly, MLPs of Transformer may also work contrary to creativity, i.e., the increased attentions narrow the conditional distribution for token prediction Sæbø and Brovold (2024).然而，从统计学角度看，神经元之间复杂的连接和激活虽然不是无限的，但也无法穷尽。就价值而言，并非所有的创造都是有价值的。以极低的概率获得有价值的东西是不切实际的，因为理论上猴子也能打印出莎士比亚的作品。LLM 如何确保生成有价值创造的概率？创造的新颖性和价值背后的机制是什么？一种普遍的猜想认为，新颖性是通过随机游走产生的（Sæbø 和 Brovold，2024）。然而，直觉上，由于架构限制，当前的 LLM 本身似乎无法评估创造的价值（Chakrabarty 等，2024）。因为一旦生成了下一个 token，就没有内在机制来接受或拒绝这些创造。这阻碍了像人类那样通过弯曲、混合或打破偏见来评估所提出新颖性的有用性和价值（Sæbø 和 Brovold，2024）。一些研究假设每个 token 确实是有价值的并符合长期预期。然而，众所周知的 LLM 幻觉问题（Xu 等，2024d）反驳了这一假设。此外，尽管存在许多解决该问题的变体（Huang 等，2023c；Liu 等，2024b），Transformer 架构在处理长上下文方面仍存在困难（Li 等，2024b）。更重要的是，Transformer 的 MLP 可能与创造力背道而驰，即增加的注意力缩小了 token 预测的条件分布（Sæbø 和 Brovold，2024）。

3.4 Comparison of Different Mechanism Analysis Methods3.4 不同机制分析方法的比较

The above four Hypotheses are achieved by Observation-based and Intervention-based methods. These two methods are typically combined to trace knowledge in LLMs Mossing et al. (2024); Ghandeharioun et al. (2024). Most knowledge analysis methods are architecture-agnostic and can be adapted to various models.上述四个假设是通过基于观察和基于干预的方法实现的。这两种方法通常结合使用来追踪 LLM 中的知识（Mossing 等，2024；Ghandeharioun 等，2024）。大多数知识分析方法与架构无关，可以适配各种模型。

Each method is suitable for different scenarios. Specifically, the Modular Region Hypothesis can be analyzed using either Observation-based or Intervention-based methods. In contrast, the Connection Hypothesis, which examines inter-regional connectivity, generally necessitates Intervention-based methods. However, the results of knowledge mechanism analysis depend heavily on different methods and are sensitive to evaluation metrics and implementation details Schwettmann et al. (2023b). Hence, Huang et al. (2024b) propose a dataset, RAVEL, to quantify the comparisons between a variety of existing interpretability methods. They suggest that methods with supervision are better than methods with unsupervised featurizers. Later, Zhang and Nanda (2023) further systematically examine the impact of methodological details in intervention-based methods. For corrupted run, they recommend Symmetric Token Replacement (e.g., “The Eiffel Tower” $\rightarrow$ “The Colosseum”) Sharma et al. (2024); Vig et al. (2020) instead of Gaussian Noising Meng et al. (2022), which disrupts the model’s internal mechanisms. For metric $E$ , both logit lens and probe can be employed to trace factual knowledge Meng et al. (2022), where the target output is typically few tokens. In this scenario, Zhang and Nanda (2023) advocate using the logit lens over probes for evaluation metric $E$ due to its fine-grained control over localization outcomes. Moreover, probe is capable of exploring abstract knowledge and abilities, such as theory of mind or mental states Zhu et al. (2024); Ye et al. (2024); Jin (2024), where the target output requires multiple tokens to express. Jin (2024) suggest that deeper probes are more (generally) more accurate.每种方法都适用于不同的场景。具体而言，模块化区域假设可以使用基于观察或基于干预的方法进行分析。相比之下，检查区域间连通性的连接假设通常需要基于干预的方法。然而，知识机制分析的结果在很大程度上取决于不同的方法，并对评估指标和实现细节敏感（Schwettmann 等，2023b）。因此，Huang 等（2024b）提出了一个数据集 RAVEL，用于量化各种现有可解释性方法之间的比较。他们建议，有监督的方法优于使用无监督特征提取器的方法。后来，Zhang 和 Nanda（2023）进一步系统地检查了基于干预方法中方法论细节的影响。对于损坏运行，他们推荐使用对称 token 替换（例如“埃菲尔铁塔”→“斗兽场”）（Sharma 等，2024；Vig 等，2020），而不是会破坏模型内部机制的高斯噪声（Meng 等，2022）。对于评估指标 EE，Logit Lens 和探针都可以用来追踪事实知识（Meng 等，2022），其中目标输出通常是少数 token。在这种情况下，Zhang 和 Nanda（2023）主张在评估指标 EE 中使用 Logit Lens 而非探针，因为它对定位结果具有细粒度控制。此外，探针能够探索抽象知识和能力，如心智理论或心理状态（Zhu 等，2024；Ye 等，2024；Jin，2024），其中目标输出需要多个 token 来表达。Jin（2024）认为更深的探针（通常）更准确。

4 Knowledge Evolution in LLMs4 LLM 中的知识演化

Knowledge in LLMs should evolve with changes in the external environment. We introduce the Dynamic Intelligence Hypothesis for knowledge evolution in individuals and groups.LLM 中的知识应随外部环境的变化而演化。我们引入了用于个体和群体知识演化的动态智能假设。

4.1 Individual Evolution4.1 个体演化

Immersed in a dynamic world, individuals mature through an iterative process of memorization, forgetting, error correction, and deepening understanding of the world around them. Similarly, LLMs dynamically encapsulate knowledge into parameters through the process of conflict and integration.沉浸在动态世界中的个体，通过记忆、遗忘、纠错以及加深对周围世界理解的迭代过程而成熟。同样，LLM 通过冲突与整合的过程，将知识动态封装进参数中。

In the pre-training phase, LLMs start as blank slates, facilitating easier acquisition for new knowledge Allen-Zhu and Li (2024). Consequently, numerous experiments demonstrate that LLMs accumulate vast amounts of knowledge during this stage Cao et al. (2024b); Zhou et al. (2023a); Kaddour et al. (2023); Naveed et al. (2023); Singhal et al. (2022). Akyürek et al. (2022) delve further into identifying which training examples are instrumental in endowing LLMs with specific knowledge. However, contradictions during the pre-training stage may induce conflicts among internal parametric knowledge. On the one hand, the false and contradictory information in training corpus propagate and contaminate related memories in LLMs via semantic diffusion, introducing broader detrimental effects beyond direct impacts Bian et al. (2023). On the other hand, LLMs tend to prioritize memorizing more frequent and challenging facts, which can result in subsequent facts overwriting prior memorization, significantly hindering the memorization of low-frequency facts Lu et al. (2024). In other words, LLMs struggle with balancing and integrating both low and high-frequency knowledge.在预训练阶段，LLM 以白板状态开始，这促进了新知识的获取（Allen-Zhu 和 Li，2024）。因此，大量实验证明 LLM 在此阶段积累了海量知识（Cao 等，2024b；Zhou 等，2023a；Kaddour 等，2023；Naveed 等，2023；Singhal 等，2022）。Akyürek 等（2022）进一步深入研究了哪些训练样本有助于赋予 LLM 特定知识。然而，预训练阶段的矛盾可能会导致内部参数化知识之间的冲突。一方面，训练语料库中的错误和矛盾信息会通过语义扩散传播并污染 LLM 中的相关记忆，引入超出直接影响的更广泛的有害影响（Bian 等，2023）。另一方面，LLM 倾向于优先记忆更频繁和更具挑战性的事实，这可能导致后续事实覆盖之前的记忆，从而严重阻碍对低频事实的记忆（Lu 等，2024）。换句话说，LLM 在平衡和整合低频和高频知识方面存在困难。

After pre-training, LLMs are anticipated to refresh their internal knowledge to keep pace with the evolving world during post-training stage. Although LLMs seem to absorb new knowledge through continued learning, follow user instructions via instruct tuning Zhang et al. (2023c), and align with human values through alignment tuning Ziegler et al. (2019), Ji et al. (2024) have noted that LLMs intrinsically resist alignment during the post-training phase. In other words, LLMs tend to learn factual knowledge through pre-training, whereas fine-tuning ⁴⁴ 4 Fine-tuning includes instruct tuning and alignment tuning Zhao et al. (2023b). teaches them to utilize it more efficiently Gekhman et al. (2024); Zhou et al. (2023a); Ovadia et al. (2024). Ren et al. (2024a) also posit that instruction tuning is a form of self-alignment with existing internal knowledge rather than a process of learning new information. We conjecture that the debate on whether these processes truly introduce new knowledge stems from information conflicts. For example, the conflict between outdated information within LLMs and new external knowledge exacerbates their difficulty in learning new information. To mitigate information conflicts, Ni et al. (2023) propose first forgetting old knowledge then learning new knowledge. Another technique, retrieval-augmented generation (RAG) Huang and Huang (2024), while avoiding conflicts within internal parameters, still needs to manage conflicts between retrieved external information and LLMs’ internal knowledge Xu et al. (2024b). RAG also attempt to efficiently and effectively integrate new knowledge across passages or documents using multiple retrieval Yang et al. (2024a) and hippocampal indexing Gutiérrez et al. (2024). Besides, editing technologies, including knowledge and representation editing, exhibit promising potential for knowledge addition, modification, and erasure. Specifically, knowledge editing Meng et al. (2022); Mitchell et al. (2022); Cao et al. (2021b); Zhang et al. (2024a); Wang et al. (2023d); Mazzia et al. (2023) aims to selectively modify model parameters responsible for specific knowledge retention, while representation editing Zou et al. (2023); Wu et al. (2024) adjusts the model’s conceptualization of knowledge to revise the stored knowledge within LLMs. Note that the other strategy for knowledge editing adds external parameters or memory banks for new knowledge while preserving models’ parameters. We also provide the comparison of the above methods in §A for better understanding.预训练后，LLM 被期望在后训练阶段刷新其内部知识，以跟上不断演化的世界。尽管 LLM 似乎通过持续学习吸收新知识、通过指令微调遵循用户指令（Zhang 等，2023c），并通过对齐微调与人类价值观对齐（Ziegler 等，2019），但 Ji 等（2024）指出，LLM 在后训练阶段本质上抵制对齐。换句话说，LLM 倾向于通过预训练学习事实知识，而微调（微调包括指令微调和对齐微调，Zhao 等，2023b）则教导它们更有效地利用这些知识（Gekhman 等，2024；Zhou 等，2023a；Ovadia 等，2024）。Ren 等（2024a）还认为指令微调是一种与现有内部知识的自我对齐形式，而非学习新信息的过程。我们推测，关于这些过程是否真正引入了新知识的争论源于信息冲突。例如，LLM 内部的过时信息与新的外部知识之间的冲突加剧了它们学习新信息的难度。为了缓解信息冲突，Ni 等（2023）提出先遗忘旧知识，再学习新知识。另一种技术——检索增强生成（RAG，Huang 和 Huang，2024）在避免内部参数冲突的同时，仍需管理检索到的外部信息与 LLM 内部知识之间的冲突（Xu 等，2024b）。RAG 还尝试利用多次检索（Yang 等，2024a）和海马体索引（Gutiérrez 等，2024）跨段落或文档高效且有效地整合新知识。此外，包括知识和表示编辑在内的编辑技术，在知识添加、修改和擦除方面展现出巨大潜力。具体而言，知识编辑（Meng 等，2022；Mitchell 等，2022；Cao 等，2021b；Zhang 等，2024a；Wang 等，2023d；Mazzia 等，2023）旨在选择性地修改负责特定知识保留的模型参数，而表示编辑（Zou 等，2023；Wu 等，2024）则调整模型对知识的概念化，以修改 LLM 内部存储的知识。请注意，知识编辑的另一种策略是为新知识添加外部参数或内存库，同时保留模型的参数。为了更好地理解，我们在附录 A 中提供了上述方法的比较。

4.2 Group Evolution4.2 群体演化

Besides individual learning, social interaction plays a pivotal role in the acquisition of new knowledge and is a key driver of human societal development Baucal et al. (2014); Levine et al. (1993). LLMs, also known as agents, collaborate to accomplish complex tasks during group evolution, each bearing unique knowledge that may sometimes contradict each other. Therefore, contrary to individual evolution, group evolution encounters intensified conflicts, such as conflicts in specialized expertise among agents, competing interests, cultural disparities, moral dilemmas, and others. To achieve consensus and resolve conflicts, agents must first clarify their own and others’ goals (beliefs) through internal representations in models Zhu et al. (2024); Zou et al. (2023). Agents then discuss, debate, and reflect on shared knowledge through various communication methods Chan et al. (2024); Smit et al. (2024); Li et al. (2024e); Soltoggio et al. (2024), e.g., prompt instructions, task and agent descriptions, parameter signals (activation and gradient), and representations of models. However, conformity of agents, which tends to believe the majority’s incorrect answers rather than maintaining their own, hinders conflict resolution during group evolution Zhang et al. (2023a); Ma et al. (2024). Note that the group also struggles with automating moral decision-making when facing moral conflicts. Specifically, agents in the group miss ground truth for moral “correctness” and encounter dilemmas due to changes in moral norms over time Hagendorff and Danks (2023). Generally, when, what, and how to share knowledge in the communication process to maximize learning efficiency and long-term expectations are still open questions in group evolution.除了个体学习外，社会互动在获取新知识方面起着关键作用，也是人类社会发展的关键驱动力（Baucal 等，2014；Levine 等，1993）。LLM（也称为智能体）在群体演化过程中协作完成复杂任务，每个智能体都带有有时会相互矛盾的独特知识。因此，与个体演化不同，群体演化会遇到更强烈的冲突，例如智能体之间专业知识的冲突、利益竞争、文化差异、道德困境等。为了达成共识并解决冲突，智能体必须首先通过模型内部的表征来澄清自身和他人的目标（信念）（Zhu 等，2024；Zou 等，2023）。随后，智能体通过各种通信方法讨论、辩论并反思共享知识（Chan 等，2024；Smit 等，2024；Li 等，2024e；Soltoggio 等，2024），例如提示指令、任务和智能体描述、参数信号（激活和梯度）以及模型表征。然而，智能体的从众心理（倾向于相信多数人的错误答案而非坚持自己的观点）阻碍了群体演化过程中的冲突解决（Zhang 等，2023a；Ma 等，2024）。请注意，群体在面临道德冲突时，也难以实现道德决策的自动化。具体而言，群体中的智能体缺失道德“正确性”的基准真理，并因道德规范随时间的变化而面临困境（Hagendorff 和 Danks，2023）。总体而言，在通信过程中何时、何地以及如何共享知识以最大化学习效率和长期预期，在群体演化中仍是开放性问题。

Through debate and collaboration, groups integrate more knowledge and can surpass the cognition of individual units Liang et al. (2023a); Qian et al. (2023); Qiao et al. (2024); Talebirad and Nadiri (2023); Zhang et al. (2023a). This derives from the assumption that each individual unit can contribute to and benefit from the collective knowledge Soltoggio et al. (2024); Xu et al. (2024c). In addition, “When a measure becomes a target, it ceases to be a good measure”, which implies that optimizing one objective on a single individual will inevitably harm other optimization objectives to some extent. Hence, it is unrealistic for an individual to learn all knowledge compared to group optimization. Interestingly, LLM groups also follow the collaborative scaling law Qian et al. (2024a), where normalized solution quality follows a logistic growth pattern as scaling agents. Moreover, some works Huh et al. (2024); Bereska and Gavves (2024) propose that knowledge tends to converge into the same representation spaces among the whole artificial neural models group with different data, modalities, and objectives.通过辩论和协作，群体整合了更多知识，并能超越个体单元的认知（Liang 等，2023a；Qian 等，2023；Qiao 等，2024；Talebirad 和 Nadiri，2023；Zhang 等，2023a）。这源于这样一种假设：每个个体单元都能为集体知识做出贡献并从中受益（Soltoggio 等，2024；Xu 等，2024c）。此外，“当衡量标准成为目标时，它就不再是一个好的衡量标准”，这意味着在单个个体上优化一个目标必然会在一定程度上损害其他优化目标。因此，与群体优化相比，个体学习所有知识是不现实的。有趣的是，LLM 群体也遵循协作缩放定律（Qian 等，2024a），即随着智能体规模的扩大，归一化的解决方案质量呈现逻辑增长模式。此外，一些研究（Huh 等，2024；Bereska 和 Gavves，2024）提出，知识往往会收敛到具有不同数据、模态和目标的整个人工神经网络群体中的相同表征空间。

4.3 Comparison of Different Evolution Strategies4.3 不同演化策略的比较

Individuals and groups achieve dynamic intelligence primarily through two strategies: updating internal parametric knowledge Zhou et al. (2023a); Qiao et al. (2024) and leveraging external knowledge ⁵⁵ 5 Leveraging external knowledge includes using prompts Xie et al. (2024), ICL, and RAG. Huang and Huang (2024); Xie et al. (2024). These two strategies are usually used together in applications Yang et al. (2024b).个体和群体主要通过两种策略实现动态智能：更新内部参数化知识（Zhou 等，2023a；Qiao 等，2024）和利用外部知识（利用外部知识包括使用提示（Xie 等，2024）、ICL 和 RAG（Huang 和 Huang，2024；Xie 等，2024））。这两种策略通常在应用中结合使用（Yang 等，2024b）。

Updating internal parametric knowledge necessitates high-quality data for parameter adjustments Vashishtha et al. (2024); Cao et al. (2024a). Data proves pivotal when fine-tuning models to acquire new knowledge. Ovadia et al. (2024) also posit that the continued training of LLMs via unsupervised tuning generally exhibits suboptimal performance when it comes to acquiring new knowledge. Note that updating internal parametric knowledge requires resolving conflicts among internal parameters. The crux of effective internal knowledge updating lies in preserving the consistency of the model’s parameter knowledge before and after tuning. In contrast, leveraging external knowledge requires managing conflicts within the external knowledge itself ⁶⁶ 6 Inconsistencies in external information are common, as external documents often contain conflicting data, particularly in contexts for RAG. as well as conflicts between external and internal knowledge Xu et al. (2024b); Liu et al. (2024a). Besides, parametric knowledge compresses extensive information, promoting grokking and enhancing generalization Wang et al. (2024a). In contrast, leveraging external knowledge avoids high training costs but necessitates substantial maintenance and retrieval costs for every user query. Therefore, the combination of these two strategies is promising. An attempt for combination Yang et al. (2024b) suggests employing RAG for low-frequency knowledge and parametric strategy for high-frequency knowledge.更新内部参数化知识需要高质量的数据进行参数调整（Vashishtha 等，2024；Cao 等，2024a）。在微调模型以获取新知识时，数据被证明是关键。Ovadia 等（2024）还认为，通过无监督微调持续训练 LLM 在获取新知识方面通常表现欠佳。请注意，更新内部参数化知识需要解决内部参数之间的冲突。有效更新内部知识的关键在于保持模型参数知识在微调前后的连贯性。相比之下，利用外部知识需要管理外部知识本身的冲突（外部信息中的不一致很常见，因为外部文档通常包含相互冲突的数据，特别是在 RAG 的上下文中），以及外部知识与内部知识之间的冲突（Xu 等，2024b；Liu 等，2024a）。此外，参数化知识压缩了海量信息，促进了顿悟并增强了泛化能力（Wang 等，2024a）。相比之下，利用外部知识避免了高昂的训练成本，但需要为每个用户查询支付大量的维护和检索成本。因此，结合这两种策略前景广阔。一种结合尝试（Yang 等，2024b）建议对低频知识采用 RAG，对高频知识采用参数化策略。

5 Application of Knowledge Mechanism5 知识机制的应用

The mechanism analysis of knowledge utilization and evolution may provide an avenue to construct more efficient and trustworthy models in practice.对知识利用和演化的机制分析，可能为在实践中构建更高效、更可信的模型提供途径。

5.1 Efficient LLMs5.1 高效 LLM

Researchers have been working to reduce the cost of training and inference for LLMs through various optimization strategies, including architecture Ainslie et al. (2023); Fedus et al. (2022), data quality Kaddour (2023), parallelization Qi et al. (2024), generalization theory Zhang et al. (2024d), hardware Dey et al. (2023), scaling laws Hoffmann et al. (2022), optimizer Liu et al. (2023a), etc. The underlying knowledge mechanisms offer LLMs new potential for efficiently storing, utilizing, and evolving knowledge.研究人员一直致力于通过各种优化策略降低 LLM 的训练和推理成本，包括架构（Ainslie 等，2023；Fedus 等，2022）、数据质量（Kaddour，2023）、并行化（Qi 等，2024）、泛化理论（Zhang 等，2024d）、硬件（Dey 等，2023）、缩放定律（Hoffmann 等，2022）、优化器（Liu 等，2023a）等。底层的知识机制为 LLM 高效存储、利用和演化知识提供了新潜力。

For knowledge storage and utilization in LLMs, knowledge (memory) circuit provides the theory to decompose the knowledge computations of an LLM into smaller, recurring parts Yang et al. (2024b). These smaller parts guide the determination of which types of knowledge should be encoded into parameters. Therefore, $\text{Memory}^{3}$ Yang et al. (2024b) designs an explicit memory mechanism for Transformer-based LLMs, alleviating the burden of parameter size. Specifically, $\text{Memory}^{3}$ designs external information, explicit memory, and implicit memory for different usage frequencies, reducing writing and reading costs. For knowledge evolution, the knowledge mechanism analysis inspires editing and model merging. The details of editing technologies can be found in §4.2. Model merging technologies ⁷⁷ 7 Model merging also includes methods that directly interpolation Goddard et al. (2024) or randomly fusing Yu et al. (2023b), ignoring parameter directions. These methods are naive and are not the focus of our discussion here. leverage parameter directions to combine multiple task-specific models into a single multitask model without performing additional training rather than training from scratch. For instance, Task Arithmetic Ilharco et al. (2023) identifies the weight directions of task capabilities in different models, and then integrates a more powerful model by arithmetic operations on weight directions. TIES Yadav et al. (2023) resolves parameters directions conflicts, and merges only the parameters that are in alignment with the final agreed-upon sign. Akiba et al. (2024) further propose evolutionary optimization of model merging, which automatically discovers effective combinations of open-source models, harnessing their group intelligence without requiring extensive training data or computational resources. Besides, the Lottery Ticket Hypothesis Frankle and Carbin (2019) provides a cornerstone for model compression, generalizing across various datasets, optimizers, and model architectures Morcos et al. (2019); Chen et al. (2021). However, model compression often limits the success of editing and model merging Kolbeinsson et al. (2024). This phenomenon poses challenges for practical implementations, highlighting the need for more effective strategies.对于 LLM 中的知识存储和利用，知识（记忆）回路提供了将 LLM 的知识计算分解为更小、重复部分的理论（Yang 等，2024b）。这些较小的部分指导确定哪些类型的知识应编码到参数中。因此，Memory³（Yang 等，2024b）为基于 Transformer 的 LLM 设计了一种显式记忆机制，减轻了参数规模的负担。具体而言，Memory³ 为不同的使用频率设计了外部信息、显式记忆和隐式记忆，降低了写入和读取成本。对于知识演化，知识机制分析启发了编辑和模型合并。编辑技术的细节可以在第 4.2 节找到。模型合并技术（模型合并还包括直接插值（Goddard 等，2024）或随机融合（Yu 等，2023b）且忽略参数方向的方法。这些方法比较原始，不是我们讨论的重点）利用参数方向将多个特定任务模型合并为一个多任务模型，而无需进行额外训练，也无需从头开始训练。例如，Task Arithmetic（Ilharco 等，2023）识别了不同模型中任务能力的权重方向，然后通过权重方向的算术运算整合出一个更强大的模型。TIES（Yadav 等，2023）解决了参数方向冲突，仅合并与最终商定符号一致的参数。Akiba 等（2024）进一步提出了模型合并的演化优化，它自动发现开源模型的有效组合，利用它们的群体智能，而无需大量的训练数据或计算资源。此外，彩票假设（Frankle 和 Carbin，2019）为模型压缩提供了基石，并推广到各种数据集、优化器和模型架构中（Morcos 等，2019；Chen 等，2021）。然而，模型压缩往往限制了编辑和模型合并的成功（Kolbeinsson 等，2024）。这一现象对实际实现提出了挑战，凸显了对更有效策略的需求。

5.2 Trustworthy LLMs5.2 可信 LLM

Numerous studies investigate the underlying causes of security risks Reuel et al. (2024); Ren et al. (2024b); Li et al. (2024a); Bengio (2024); Bengio et al. (2024); Dalrymple et al. (2024). In particular, Wei et al. (2023) delve into the safety of LLM and reveal that the success of jailbreak is mainly due to the distribution discrepancies between malicious attacks and training data. Geva et al. (2022b) and Wang et al. (2024b) further discover that some parameters within LLMs, called toxic regions, are intrinsically tied to the generation of toxic content. Ji et al. (2024) even conjecture that LLMs resist alignment. Therefore, traditional aligned methods, DPO Rafailov et al. (2023) and SFT, seem to merely bypass toxic regions Lee et al. (2024); Wang et al. (2024b), making them susceptible to other jailbreak attacks Zhang et al. (2023d).已有大量研究探讨了安全风险的根本原因 (Reuel et al., 2024; Ren et al., 2024b; Li et al., 2024a; Bengio, 2024; Bengio et al., 2024; Dalrymple et al., 2024)。特别是 Wei 等人 (2023) 对大语言模型 (LLM) 的安全性进行了深入研究，揭示了越狱攻击的成功主要归因于恶意攻击与训练数据之间的分布差异。Geva 等人 (2022b) 和 Wang 等人 (2024b) 进一步发现，LLM 内部存在一些被称为“毒性区域”的参数，这些参数与毒性内容的生成有着内在联系。Ji 等人 (2024) 甚至推测 LLM 具有抗对齐性。因此，传统的对齐方法（如 DPO (Rafailov et al., 2023) 和 SFT）似乎只是绕过了这些毒性区域 (Lee et al., 2024; Wang et al., 2024b)，从而使模型容易受到其他越狱攻击的影响 (Zhang et al., 2023d)。

Inspired by the knowledge mechanism analysis in LLMs, a promising trustworthy strategy may be designing architecture and training process during the pre-training phase to encouraging modularity Liu et al. (2024c); Liu et al. (2023b), sparsity Chughtai et al. (2023), and monosemanticity Bricken et al. (2023); Jermyn et al. (2022), which make the reverse engineering process more tractable Jermyn et al. (2022); Bricken et al. (2023); Liu et al. (2024c); Tamkin et al. (2023). Yet, maintaining sparsity for a vast amount of world knowledge requires substantial resources, and whether monosemantic architecture can support advanced intelligence remains elusive. Besides, machine unlearning Nguyen et al. (2022); Tian et al. (2024); Yao et al. (2023a) aims to forget privacy or toxic information learned by LLMs. However, these unlearning methods suffer overfitting, forgetting something valuable due to the difficulty of disentangling verbatim memorization and general capabilities Huang et al. (2024c); Blanco-Justicia et al. (2024). Another alternative technique is knowledge editing, precisely modifying LLMs using few instances during the post-training stage Mazzia et al. (2023); Yao et al. (2023b); Wang et al. (2023d); Hase et al. (2024); Qian et al. (2024b). Extensive experiments demonstrate that knowledge editing has the potential to detoxify LLMs Yan et al. (2024). Specifically, Wu et al. (2023a) and Geva et al. (2022b) deactivate the neurons related to privacy information and toxic tokens, respectively. Wang et al. (2024b) identify and then erases toxic regions in LLMs. However, knowledge editing also introduces side effects, such as the inability of the modified knowledge to generalize to multi-hop tasks Zhong et al. (2023); Li et al. (2023d); Cohen et al. (2023); Kong et al. (2024) and the potential to impair the model’s general capabilities Gu et al. (2024); Qin et al. (2024). Therefore, recent efforts focus on representation editing instead of editing parameters in knowledge editing Zou et al. (2023); Turner et al. (2023); Zhou et al. (2023b); Zhu et al. (2024). These representations (hidden states) within LLMs can trace and address a wide range of safety-relevant problems, including honesty, harmlessness, and power seeking. Later, Wu et al. (2024) develop a family of representation finetuning methods to update new knowledge. Zou et al. (2024) propose circuit-breaking Li et al. (2023b), directly controlling the representations that are responsible for harmful outputs. However, these representation editing strategies require meticulous hyperparameter tuning for each task. More efficient optimization methods are needed to align with computational or temporal constraints.受 LLM 知识机制分析的启发，一种有前景的信任策略是在预训练阶段设计架构和训练流程，以鼓励模块化 (Liu et al., 2024c; Liu et al., 2023b)、稀疏性 (Chughtai et al., 2023) 和单义性 (Bricken et al., 2023; Jermyn et al., 2022)，从而使逆向工程过程更具可操作性 (Jermyn et al., 2022; Bricken et al., 2023; Liu et al., 2024c; Tamkin et al., 2023)。然而，为海量世界知识维持稀疏性需要大量资源，且单义架构能否支持高级智能仍不明确。此外，机器遗忘 (Nguyen et al., 2022; Tian et al., 2024; Yao et al., 2023a) 旨在消除 LLM 学到的隐私或毒性信息。但这些遗忘方法存在过拟合问题，由于难以区分逐字记忆与通用能力，往往会导致有价值信息的遗忘 (Huang et al., 2024c; Blanco-Justicia et al., 2024)。另一种替代技术是知识编辑，即在后训练阶段使用少量实例对 LLM 进行精确修改 (Mazzia et al., 2023; Yao et al., 2023b; Wang et al., 2023d; Hase et al., 2024; Qian et al., 2024b)。大量实验证明知识编辑具有消除 LLM 毒性的潜力 (Yan et al., 2024)。具体而言，Wu 等人 (2023a) 和 Geva 等人 (2022b) 分别停用了与隐私信息和毒性标记相关的神经元。Wang 等人 (2024b) 识别并擦除了 LLM 中的毒性区域。然而，知识编辑也会引入副作用，例如修改后的知识无法泛化到多跳任务 (Zhong et al., 2023; Li et al., 2023d; Cohen et al., 2023; Kong et al., 2024)，以及可能损害模型的通用能力 (Gu et al., 2024; Qin et al., 2024)。因此，近期研究重点转向了表征编辑，而非知识编辑中的参数编辑 (Zou et al., 2023; Turner et al., 2023; Zhou et al., 2023b; Zhu et al., 2024)。LLM 内部的这些表征（隐藏状态）可以追踪并解决广泛的安全相关问题，包括诚实性、无害性和权力追求。随后，Wu 等人 (2024) 开发了一系列表征微调方法来更新新知识。Zou 等人 (2024) 提出了电路破坏 (circuit-breaking) (Li et al., 2023b)，直接控制导致有害输出的表征。然而，这些表征编辑策略需要针对每个任务进行精细的超参数调整。为了符合计算或时间限制，需要更高效的优化方法。

6 Discussion6 讨论

In this section, we discuss some open questions and seek to explore their essence and underlying principles. Specifically, we discuss what knowledge LLMs have learned in §6.1, examine the fragility of the learned knowledge in application in §6.2, analyze the dark knowledge not yet learned by machines or humans in §6.3, and explore how LLMs can expand the boundaries of unknown knowledge from interdisciplinary perspectives §6.4.在本节中，我们将讨论一些悬而未决的问题，并试图探索其本质和基本原理。具体而言，我们将在 §6.1 讨论 LLM 学习到了什么知识，在 §6.2 审视所学知识在应用中的脆弱性，在 §6.3 分析机器或人类尚未习得的“暗知识”，并从跨学科视角探讨 LLM 如何拓展未知知识的边界 (§6.4)。

6.1 What Knowledge Have LLMs Learned?6.1 LLM 学习到了什么知识？

Critics question whether LLMs truly have knowledge or if they are merely mimicking Schwarzschild et al. (2024), akin to the “Stochastic Parro” Bender et al. (2021) and “Clever Hans” Shapira et al. (2024). We first review the doubts from the following three levels through observation phenomena: 1) Memorization: LLMs primarily rely on positional information over semantic understanding Li et al. (2022) to predict answers. Additionally, LLMs may generate different answers for the same question due to different expressions. 2) Comprehension and application: Allen-Zhu and Li (2023b) argue that LLMs hardly efficiently apply knowledge from pre-training data, even when such knowledge is perfectly stored and fully extracted from LLMs. Therefore, LLMs struggle with various reasoning tasks Wu et al. (2023b); Nezhurina et al. (2024); Gutiérrez et al. (2024) as well as the reversal curse Berglund et al. (2023). Besides, LLMs are not yet able to reliably act as text world simulators and encounter difficulties with planning Wang et al. (2024d). 3) Creation: Although LLMs are capable of generating new terms, their quality often falls below that created by humans Raiola (2023). Even though LLMs possess knowledge, some critics argue that current analysis methods may only explain low-level co-occurrence patterns, not internal mechanisms. The primary criticism asserts that the components responsible for certain types of knowledge in LLM fail to perform effectively in practical applications Hase et al. (2023). In addition, the components responsible for specific knowledge within LLMs vary under different methods. For these criticisms, Chen et al. (2024g); Chen et al. (2024e) propose degenerate neurons and posit that different degenerate components indeed independently express a fact. Chen et al. (2024f) delineate the differences in the mechanisms of knowledge storage and representation, proposing the Query Localization Assumption to response these controversies. Zhu and Li (2023) further observe that knowledge may be memorized but not extracted due to the knowledge not being sufficiently augmented (e.g., through paraphrasing, sentence shuffling) during pretraining. Hence, rewriting the training data to provide knowledge augmentation and incorporating more instruction fine-tuning data in the pre-training stage can effectively alleviate the above challenges and criticisms.批评者质疑 LLM 是否真正具备知识，还是仅仅在进行模仿 (Schwarzschild et al., 2024)，类似于“随机鹦鹉” (Bender et al., 2021) 和“聪明的汉斯” (Shapira et al., 2024)。我们首先通过观察到的现象，从以下三个层面回顾这些质疑：1) 记忆：LLM 主要依赖位置信息而非语义理解 (Li et al., 2022) 来预测答案。此外，由于表达方式不同，LLM 可能会对同一个问题生成不同的答案。2) 理解与应用：Allen-Zhu 和 Li (2023b) 认为，即使知识被完美存储并从 LLM 中完全提取出来，LLM 也难以有效地应用预训练数据中的知识。因此，LLM 在处理各种推理任务 (Wu et al., 2023b; Nezhurina et al., 2024; Gutiérrez et al., 2024) 以及“逆转诅咒” (Berglund et al., 2023) 时表现吃力。此外，LLM 尚无法可靠地充当文本世界模拟器，在规划方面也面临困难 (Wang et al., 2024d)。3) 创造：尽管 LLM 能够生成新术语，但其质量往往低于人类创作 (Raiola, 2023)。尽管 LLM 拥有知识，但一些批评者认为现有的分析方法可能只能解释低级的共现模式，而非内部机制。主要的批评观点认为，LLM 中负责特定类型知识的组件在实际应用中无法有效发挥作用 (Hase et al., 2023)。此外，在不同方法下，负责特定知识的 LLM 组件各不相同。针对这些批评，Chen 等人 (2024g; 2024e) 提出了退化神经元 (degenerate neurons)，并假设不同的退化组件确实独立地表达了一个事实。Chen 等人 (2024f) 阐述了知识存储与表征机制的区别，并提出了“查询定位假设” (Query Localization Assumption) 来回应这些争议。Zhu 和 Li (2023) 进一步观察到，知识可能已被记忆但因在预训练期间未得到充分增强（例如通过改写、句子重排）而无法被提取。因此，重写训练数据以提供知识增强，并在预训练阶段纳入更多的指令微调数据，可以有效缓解上述挑战和批评。

Despite considerable criticism, the mainstream view Didolkar et al. (2024); Jin and Rinard (); Jin (2024) is that current LLMs may possess basic world knowledge via memorization but hardly master underlying principles for reasoning and creativity. In other words, LLMs master basic knowledge via memorization (discussed in §3.1). Although LLMs possess the foundational ability to comprehend and apply knowledge (discussed in §3.2), exhibiting plausible and impressive reasoning capabilities. Current LLMs still struggle with reasoning and planning in complex tasks due to the fragility of knowledge in LLMs (elaborated in §6.2). These reasoning and planning abilities usually require to be induced through techniques such as ICL and CoT. Unfortunately, current LLMs are nearly incapable of creation due to the architectural limitations (discussed in §3.3). Therefore, some scholars explore various architectural choices (e.g., Mamba Gu and Dao (2023)) and training procedures. Besides, recent research attempts to manipulate neurons, knowledge circuits, or representations Allen-Zhu and Li (2023b); Zou et al. (2023); Wu et al. (2024); Li et al. (2023a) to explore more knowledge and awaken the reasoning and planning capabilities of LLMs.尽管存在大量批评，但主流观点 (Didolkar et al., 2024; Jin and Rinard, ; Jin, 2024) 认为，当前的 LLM 可能通过记忆掌握了基础的世界知识，但很难掌握推理和创造的底层原理。换句话说，LLM 通过记忆掌握了基础知识（在 §3.1 中讨论）。尽管 LLM 具备理解和应用知识的基础能力（在 §3.2 中讨论），并展现出看似合理且令人印象深刻的推理能力，但当前的 LLM 由于知识的脆弱性（在 §6.2 中详述），在复杂任务的推理和规划方面仍然表现吃力。这些推理和规划能力通常需要通过 ICL 和 CoT 等技术来诱导。遗憾的是，由于架构限制（在 §3.3 中讨论），当前的 LLM 几乎无法进行创造。因此，一些学者正在探索各种架构选择（如 Mamba (Gu and Dao, 2023)）和训练流程。此外，近期研究尝试操纵神经元、知识电路或表征 (Allen-Zhu and Li, 2023b; Zou et al., 2023; Wu et al., 2024; Li et al., 2023a)，以探索更多知识并唤醒 LLM 的推理和规划能力。

Remarks: LLMs have learned basic knowledge of the world by momorization. However, the learned knowledge is fragile, leading to challenges in knowledge comprehension and application. Unfortunately, due to architectural limitations, current LLMs struggle with creation.备注：LLM 已通过记忆学习到了基础的世界知识。然而，所学知识较为脆弱，导致在知识理解和应用方面面临挑战。遗憾的是，受限于架构，当前的 LLM 在创造方面表现吃力。

6.2 Why Is Learned Knowledge Fragile?6.2 为什么所学知识是脆弱的？

The knowledge learned by LLMs is fragile, leading to challenges in application including hallucination, knowledge conflicts, failed reasoning, and safety risk ⁸⁸ 8 The secure risk is elaborated in §5.2.. Hallucination denotes content generated by LLMs that diverges from real-world facts or inputs Huang et al. (2023b); Xu et al. (2024d); Farquhar et al. (2024); Chen et al. (2024d). On the one hand, factuality hallucination underscores the disparity between generated content and real-world knowledge. On the other hand, faithfulness hallucination describes the departure of generated content from user instructions or input context, as well as the coherence maintained within the generated content. Knowledge Conflict inherently denotes inconsistencies in knowledge Xu et al. (2024b); Kortukov et al. (2024). On the one hand, internal memory conflicts within the model cause LLMs to exhibit unpredictable behaviors and generate differing results to inputs which are semantically equivalent but syntactically distinct Xu et al. (2024b); Wang et al. (2023a); Feng et al. (2023b); Raj et al. (2022). On the other hand, context-memory conflict emerges when external context knowledge contradicts internal parametric knowledge Xu et al. (2024b); Mallen et al. (2023).LLM 所学知识的脆弱性导致了应用中的挑战，包括幻觉、知识冲突、推理失败和安全风险（安全风险在 §5.2 中详述）。幻觉是指 LLM 生成的内容偏离了现实世界的事实或输入 (Huang et al., 2023b; Xu et al., 2024d; Farquhar et al., 2024; Chen et al., 2024d)。一方面，事实性幻觉强调了生成内容与现实世界知识之间的差异；另一方面，忠实性幻觉描述了生成内容偏离用户指令或输入上下文的情况，以及生成内容内部缺乏连贯性的问题。知识冲突本质上指知识的不一致性 (Xu et al., 2024b; Kortukov et al., 2024)。一方面，模型内部的记忆冲突导致 LLM 表现出不可预测的行为，并对语义等价但句法不同的输入生成不同的结果 (Xu et al., 2024b; Wang et al., 2023a; Feng et al., 2023b; Raj et al., 2022)；另一方面，当外部上下文知识与内部参数化知识发生矛盾时，就会出现上下文-记忆冲突 (Xu et al., 2024b; Mallen et al., 2023)。

We posit that these challenges mainly derive from improper learning data. Specifically, hallucination is introduced by data Kang and Choi (2023); Weng (2024); Zhang et al. (2024c), heightened during the pre-training Brown et al. (2020); Chiang and Cholak (2022), alignment Azaria and Mitchell (2023); Ouyang et al. (2022), and deficiencies in decoding strategies Fan et al. (2018); Chuang et al. (2023); Shi et al. (2023). Internal memory conflict can be attributed to training corpus bias Wang et al. (2023b), and exacerbated by decoding strategies Lee et al. (2022b) and knowledge editing. Context-memory conflict arises mainly from the absence of accurate knowledge during training, necessitating retrieval from databases and the Web. Failed reasoning usually arises from improper data distribution. Specifically, knowledge may be memorized but not extractable or applicable without sufficient augmentation (e.g., through paraphrasing, sentence shuffling) during pre-training Zhu and Li (2023). Antoniades et al. (2024) also delve into the mechanism between parametric knowledge and learning data, demonstrate that training data distribution qualitatively influences generalization behavior Jiang et al. (2024a). Wang et al. (2024a) further suggest that improper data distribution in the corpus causes LLMs to lack essential reasoning components, such as the bridge layer for two-hop reasoning. Similar mechanism analysis also supports the above conclusion, indicating that hallucinations arise from a lack of mover heads Yao et al. (2024); Yu et al. (2024b), while knowledge conflicts stem from circuit competition failure in the last few layers Lv et al. (2024); Merullo et al. (2023b); Hase et al. (2023); Ju et al. (2024); Jin et al. (2024b). Additionally, data quantity is crucial for knowledge robustness. Specifically, LLMs can systematically learn comprehensive understandings of the world from extensive datasets, while little data during post-training stage may compromise the robustness of knowledge representation. This assumption is confirmed by numerous failures of post-training. For example, SFT exacerbates hallucinations Gekhman et al. (2024); Kang et al. (2024), and knowledge editing amplifies knowledge conflicts Li et al. (2023d); Yang et al. (2024c). Note that safety issues usually caused by the distribution of unseen data (adversarial input) Wei et al. (2023); Li et al. (2024c), which is elaborated in §5.2.我们认为这些挑战主要源于不当的学习数据。具体而言，幻觉是由数据引入的 (Kang and Choi, 2023; Weng, 2024; Zhang et al., 2024c)，并在预训练 (Brown et al., 2020; Chiang and Cholak, 2022)、对齐 (Azaria and Mitchell, 2023; Ouyang et al., 2022) 以及解码策略的缺陷 (Fan et al., 2018; Chuang et al., 2023; Shi et al., 2023) 中被放大。内部记忆冲突可归因于训练语料库的偏差 (Wang et al., 2023b)，并因解码策略 (Lee et al., 2022b) 和知识编辑而加剧。上下文-记忆冲突主要源于训练期间缺乏准确知识，从而需要从数据库和网络中进行检索。推理失败通常源于不当的数据分布。具体而言，若在预训练期间没有足够的增强（例如通过改写、句子重排），知识可能已被记忆但无法被提取或应用 (Zhu and Li, 2023)。Antoniades 等人 (2024) 也深入研究了参数化知识与学习数据之间的机制，证明了训练数据分布会定性地影响泛化行为 (Jiang et al., 2024a)。Wang 等人 (2024a) 进一步指出，语料库中不当的数据分布导致 LLM 缺乏必要的推理组件，例如两跳推理的桥接层。类似的机制分析也支持上述结论，表明幻觉源于缺乏移动头 (mover heads) (Yao et al., 2024; Yu et al., 2024b)，而知识冲突则源于最后几层中电路竞争的失败 (Lv et al., 2024; Merullo et al., 2023b; Hase et al., 2023; Ju et al., 2024; Jin et al., 2024b)。此外，数据量对于知识的稳健性至关重要。具体而言，LLM 可以从海量数据集中系统地学习对世界的全面理解，而后训练阶段的数据不足可能会损害知识表征的稳健性。这一假设得到了后训练中多次失败案例的证实。例如，SFT 加剧了幻觉 (Gekhman et al., 2024; Kang et al., 2024)，而知识编辑放大了知识冲突 (Li et al., 2023d; Yang et al., 2024c)。需要注意的是，安全问题通常由未见数据（对抗性输入）的分布引起 (Wei et al., 2023; Li et al., 2024c)，详见 §5.2。

Remarks: Improper learning caused by data distribution and quantity might be the fundamental and primary cause.备注：由数据分布和数据量导致的不当学习可能是最根本和首要的原因。

6.3 Does Difficult-to-Learn “Dark Knowledge” Exist?6.3 是否存在难以学习的“暗知识”？

The distribution and quality of data are vital for knowledge acquisition and robust operation within the model (machine). Imagine an ideal scenario where we have access to all kinds of data to train the machine. The data includes all possible modalities, such as text, image, audio, video, etc. Models can also interact with each other and the external environment. In this long-term development, will there still be unknown dark knowledge for intelligence to human or model (machine)?数据的分布和质量对于模型（机器）内的知识获取和稳健运行至关重要。设想一个理想场景，我们可以获取各种数据来训练机器，这些数据包括所有可能的模态，如文本、图像、音频、视频等。模型之间以及模型与外部环境也可以进行交互。在这种长期发展中，对于人类或模型（机器）的智能来说，是否还会存在未知的暗知识？

We hypothesize that there will still exist dark knowledge for intelligence in the future. As shown in Fig 4, dark knowledge describes knowledge unknown to human or machine from the following three situations: 1) knowledge unknown to human & known to machine (UH, KM). Machines leverage vast amounts of data to explore internal patterns, whereas humans struggle with processing such data due to physiological limitations on data processing capacity and computational limits Burns et al. (2023); McAleese et al. (2024). (UH, KM) includes gene prediction, intelligent transportation systems, and more. Specifically, the structural elucidation of proteins remains mysterious to humans for a long time. Cryo-electron microscopy, through capturing millions of images, first reveals the three-dimensional structures of proteins. Now, neural models can directly predict protein properties with high efficiency and accuracy Pak et al. (2023). 2) knowledge known to human & unknown to machine (UH, KM). On the one hand, some scholars claim that machine can possess a “Theory of Mind” capability Zhu et al. (2024) and emotions Normoyle et al. (2024). On the other hand, critics contend that machine lacks sentience Alvero and Peña (2023) and merely probabilistically generates tokens. The causes, extent, and dynamics of these emotions and sentience (like hunger, happiness, and loneliness) are subtle and intricate, making precise mathematical modeling by the machine exceptionally challenging. Specifically, different factors are tightly coupled, making it nearly impossible to disentangle clear input-output relationships as with well-defined factual knowledge. The sentient knowledge also exhibits chaotic behavior Li et al. (2020); Debbouche et al. (2021), being highly sensitive to initial conditions, where small changes can lead to vastly different outcomes Segretain et al. (2020). Therefore, opponents argue that no matter how many parameters machine possesses, it cannot learn all the knowledge that human has mastered. 3) knowledge unknown to human & unknown to machine (UH, UM) is beyond our cognition, e.g., the uncertainty in quantum mechanics and the origin of the universe. Generally, Dark knowledge extends beyond current data and model architectures Tseng et al. (2024). (UH, UM) necessitates human-machine collaboration. Yet, there is no definitive conclusion on whether (UH, KM) and (KH, UM) will be solved by model architecture, training data, and computational resources. Note that plain knowledge known to human and machine in Fig 4 encompasses well-defined historical events, mathematical theorems, physical laws, etc.我们假设未来智能仍将存在暗知识。如图 4 所示，暗知识描述了人类或机器未知的知识，具体包括以下三种情况：1) 人类未知但机器已知的知识 (UH, KM)。机器利用海量数据来探索内部模式，而人类由于数据处理能力和计算极限的生理限制，在处理此类数据时感到吃力 (Burns et al., 2023; McAleese et al., 2024)。(UH, KM) 包括基因预测、智能交通系统等。具体而言，蛋白质的结构解析长期以来对人类而言仍是个谜。冷冻电镜通过捕捉数百万张图像，首次揭示了蛋白质的三维结构。现在，神经模型可以直接高效且准确地预测蛋白质特性 (Pak et al., 2023)。2) 人类已知但机器未知的知识 (KH, UM)。一方面，一些学者声称机器可以具备“心智理论”能力 (Zhu et al., 2024) 和情感 (Normoyle et al., 2024)；另一方面，批评者认为机器缺乏感知能力 (Alvero and Peña, 2023)，仅仅是概率性地生成标记。这些情感和感知（如饥饿、快乐和孤独）的成因、程度和动态是微妙且复杂的，使得机器进行精确的数学建模极具挑战性。具体而言，不同因素紧密耦合，几乎不可能像定义明确的事实知识那样理清清晰的输入-输出关系。感知知识还表现出混沌行为 (Li et al., 2020; Debbouche et al., 2021)，对初始条件高度敏感，微小的变化可能导致截然不同的结果 (Segretain et al., 2020)。因此，反对者认为无论机器拥有多少参数，它都无法学习人类掌握的所有知识。3) 人类未知且机器未知的知识 (UH, UM) 超出了我们的认知，例如量子力学中的不确定性和宇宙起源。总体而言，暗知识超出了当前的数据和模型架构 (Tseng et al., 2024)。(UH, UM) 需要人机协作。然而，对于 (UH, KM) 和 (KH, UM) 是否能通过模型架构、训练数据和计算资源解决，目前尚无定论。注意，图 4 中人类和机器都已知的普通知识涵盖了定义明确的历史事件、数学定理、物理定律等。

Figure 4: The future cognition of knowledge. The direction of the arrow represents the transition of knowledge from known to unknown. Dark knowledge, represented in gray, denotes knowledge unknown to human or machine. Plain knowledge known to both human and machine is highlighted in blue. 图 4：未来对知识的认知。箭头的方向表示知识从已知向未知的转变。灰色表示暗知识，即人类或机器未知的知识。蓝色突出显示了人类和机器都已知的普通知识。

Remarks: Dark knowledge may persist for a long time and requires human-machine collaboration to explore.备注：暗知识可能会长期存在，需要人机协作来探索。

6.4 How to Explore More Knowledge from Interdisciplinary Inspiration?6.4 如何从跨学科灵感中探索更多知识？

How can LLMs continuously narrow the boundaries of dark knowledge and achieve higher level intelligence by leveraging the human experience of perpetual knowledge exploration throughout history? We may draw inspirations from the following interdisciplinary studies.LLM 如何利用人类历史上永无止境的知识探索经验，持续缩小暗知识的边界并实现更高水平的智能？我们可以从以下跨学科研究中汲取灵感。

Neuroscience studies the structure and function of the brain at molecular, cellular, neural circuit, and neural network levels Squire et al. (2012). Generally, both mechanism analysis in LLMs and neuroscience utilize observation and intervention methods to investigate the basic principles of knowledge learning and memory, decision-making, language, perception, and consciousness. The biological signals of the human brain and the internal activation signals in LLMs are capable of reciprocal transformation Caucheteux et al. (2023); Feng et al. (2023a); Mossing et al. (2024); Flesher et al. (2021). Benefiting from advancements in neuroscience Jamali et al. (2024); de Schotten et al. (2022); Lee et al. (2022a), mechanism analysis in LLMs has identified analogous function neurons and regions Zhao et al. (2023a), and knowledge circuits Yao et al. (2024). Besides, leveraging plasticity theory in neuroscience, LLMs explain the underlying technical support for intelligence Sæbø and Brovold (2024). In the future, mechanism analysis of LLMs may draw inspirations from neuroscience, guiding the next generation of artificial intelligence in organizing neural frameworks and in the storage and utilization of knowledge Ren and Xia (2024); Momeni et al. (2024); Yang et al. (2024b).神经科学从分子、细胞、神经回路和神经网络层面研究大脑的结构和功能 (Squire et al., 2012)。通常，LLM 的机制分析和神经科学都利用观察和干预方法来研究知识学习与记忆、决策、语言、感知和意识的基本原理。人脑的生物信号与 LLM 内部的激活信号能够实现相互转化 (Caucheteux et al., 2023; Feng et al., 2023a; Mossing et al., 2024; Flesher et al., 2021)。得益于神经科学的进步 (Jamali et al., 2024; de Schotten et al., 2022; Lee et al., 2022a)，LLM 的机制分析已经识别出类似的功能神经元和区域 (Zhao et al., 2023a) 以及知识电路 (Yao et al., 2024)。此外，利用神经科学中的可塑性理论，LLM 解释了智能背后的技术支持 (Sæbø and Brovold, 2024)。未来，LLM 的机制分析可能会从神经科学中汲取灵感，指导下一代人工智能在神经框架的组织以及知识的存储和利用方面的发展 (Ren and Xia, 2024; Momeni et al., 2024; Yang et al., 2024b)。

Cognitive Science focuses on the mind and its processes Kolak et al. (2006); Baronchelli et al. (2013), which include language, perception, memory, attention, reasoning, emotion and mental state. Although cognitive science and neuroscience overlap in their research content, cognitive science focuses more on abstract knowledge such as mental states and emotions rather than specific knowledge. Therefore, Zhu et al. (2024) track beliefs of self and others (formulated as “Theory of Mind”) in LLMs from the psychological perspective within cognitive science. Wang et al. (2022) further observe social-cognitive skill in multi-agent communication and cooperation. Generally, there is potential to explore advanced cognitive capabilities in LLMs from the perspective of cognitive science Vilas et al. (2024).认知科学侧重于思维及其过程 (Kolak et al., 2006; Baronchelli et al., 2013)，包括语言、感知、记忆、注意、推理、情感和心理状态。尽管认知科学和神经科学在研究内容上有重叠，但认知科学更关注心理状态和情感等抽象知识，而非具体知识。因此，Zhu 等人 (2024) 从认知科学的心理学视角追踪了 LLM 中自我和他人的信念（表述为“心智理论”）。Wang 等人 (2022) 进一步观察了多智能体交流与合作中的社会认知技能。总体而言，从认知科学的角度探索 LLM 中的高级认知能力具有潜力 (Vilas et al., 2024)。

Psychology is the scientific study of mind and behavior, which include both conscious and unconscious phenomena, and mental processes such as thoughts, feelings, and motives. Benefiting from decades of research in human psychology, machine psychology aims to uncover mechanisms of decision-making and reasoning in LLMs by treating them as participants in psychological experiments Hagendorff (2023). Machine psychology may delve into mysteries of social situations and interactions shaping machine behavior, attitudes, and beliefs Park et al. (2023a). Besides, group psychology paves an auspicious path for exploring dynamics such as debates and collaboration among LLMs (agents). For instance, Dunning–Kruger effect Mahmoodi et al. (2013); Brown and Esterle (2020) in cognitive psychology filed describes that individuals with limited competence in a particular domain overestimate their abilities, and vice versa. This phenomenon may guide the final vote in group debates and discussions. Promisingly, psychology of learning can be applied to study prompt designs, boost learning efficiency, improve communication strategies, and develop feedback mechanisms for LLMs Leon (2024).心理学是对思维和行为的科学研究，包括意识和无意识现象，以及思想、感受和动机等心理过程。得益于人类心理学数十年的研究，机器心理学旨在通过将 LLM 视为心理学实验的参与者，揭示其决策和推理的机制 (Hagendorff, 2023)。机器心理学可以深入探讨塑造机器行为、态度和信念的社会情境和互动之谜 (Park et al., 2023a)。此外，群体心理学为探索 LLM（智能体）之间的辩论和协作等动态提供了吉祥的路径。例如，认知心理学领域中的达克效应 (Dunning–Kruger effect) (Mahmoodi et al., 2013; Brown and Esterle, 2020) 描述了在特定领域能力有限的个体往往会高估自己的能力，反之亦然。这一现象可以指导群体辩论和讨论中的最终投票。有前景的是，学习心理学可应用于研究提示词设计、提高学习效率、改善交流策略以及开发 LLM 的反馈机制 (Leon, 2024)。

Education is the transmission of knowledge, skills, and character traits and manifests in various forms. Inspired by education in humans, Zhang et al. (2024a) categorize knowledge acquisition in LLMs into three distinct phases: recognition, association, and mastery. Besides, education instructs humans managing various types of conflicts: identifying inconsistencies in external information (inter-context conflict), deciding between external sources and internal memory (context-memory conflict), resolving memory confusion (internal memory conflict), and addressing cultural conflicts. The above knowledge conflicts and integration also exist in knowledge evolution of LLMs across individuals and groups Dan et al. (2023). Fortunately, education facilitates humans in learning to learn. Can LLMs similarly self-evolve to continuously adapt to societal changes and requirements?教育是知识、技能和性格特征的传承，并以多种形式呈现。受人类教育的启发，Zhang 等人 (2024a) 将 LLM 的知识获取归纳为三个不同阶段：识别、关联和掌握。此外，教育指导人类管理各种类型的冲突：识别外部信息中的不一致（上下文间冲突）、在外部来源与内部记忆之间做出决定（上下文-记忆冲突）、解决记忆混乱（内部记忆冲突）以及处理文化冲突。上述知识冲突和整合也存在于 LLM 在个体和群体间的知识演化中 (Dan et al., 2023)。幸运的是，教育促进了人类的“学会学习”。LLM 能否类似地自我演化，以持续适应社会的变化和需求？

Remarks: LLMs may improve their architecture and mechanisms for knowledge learning, storage, and expression, drawing inspiration from neuroscience. Besides, cognitive science and psychology provide promising alternatives for sophisticated intelligence, emergent capabilities and behaviors in evolution. Educational studies can inspire the learning strategy of LLMs, navigating conflicts and integrating knowledge during their evolution.备注：LLM 可以从神经科学中汲取灵感，改进其知识学习、存储和表达的架构与机制。此外，认知科学和心理学为演化中的复杂智能、涌现能力和行为提供了有前景的替代方案。教育学研究可以启发 LLM 的学习策略，帮助其在演化过程中应对冲突并整合知识。

7 Future Directions7 未来方向

7.1 Parametric VS. Non-Parametric Knowledge7.1 参数化与非参数化知识

LLMs can be conceptualized as parametric knowledge stores, where the parameters of the model—typically the weights of the neural network—encode a representation of the world’s knowledge. This parametric approach to knowledge storage means that the knowledge is implicitly embedded within the model’s architecture, and it can be retrieved and manipulated through the computational processes of the neural network Allen-Zhu and Li (2023b). In contrast, non-parametric knowledge storage involves methods where the knowledge is explicitly represented and can be directly accessed. Examples of non-parametric knowledge storage include knowledge graphs, databases, and symbolic reasoning systems, where knowledge is represented as discrete symbols or facts. Parametric knowledge enables LLMs to deeply compress and integrate information Huang et al. (2024d); Shwartz-Ziv and LeCun (2024), allowing them to generalize and apply this knowledge across various contexts. This is akin to LLMs mastering the mathematical operation rule of “mod” through parametric knowledge, enabling them to generalize and seamlessly solve all mod-related problems Pearce et al. (2023); Hu et al. (2024). Conversely, non-parametric knowledge requires extensive searches across the knowledge space for each user query. Subsequently, Wang et al. (2024a) also prove that non-parametric knowledge severely fails in complex reasoning tasks, with accuracy levels approaching random guessing. Unfortunately, parametric knowledge within LLMs is opaque, often encountering challenges such as interpretability issues, outdated information, hallucinations, and security concerns.LLM 可以被概念化为参数化知识存储器，其中模型的参数（通常是神经网络的权重）编码了世界知识的表征。这种参数化知识存储方法意味着知识隐含地嵌入在模型的架构中，并通过神经网络的计算过程进行检索和操纵 (Allen-Zhu and Li, 2023b)。相比之下，非参数化知识存储涉及明确表征知识并可直接访问的方法。非参数化知识存储的例子包括知识图谱、数据库和符号推理系统，其中知识被表征为离散的符号或事实。参数化知识使 LLM 能够深度压缩和整合信息 (Huang et al., 2024d; Shwartz-Ziv and LeCun, 2024)，从而使其能够在各种上下文中泛化和应用这些知识。这类似于 LLM 通过参数化知识掌握了“取模” (mod) 的数学运算规则，使其能够泛化并无缝解决所有与取模相关的问题 (Pearce et al., 2023; Hu et al., 2024)。相反，非参数化知识需要针对每个用户查询在知识空间中进行广泛搜索。随后，Wang 等人 (2024a) 也证明了非参数化知识在复杂推理任务中严重失效，准确率接近随机猜测。遗憾的是，LLM 内部的参数化知识是不透明的，经常遇到可解释性问题、信息过时、幻觉和安全担忧等挑战。

Addressing these issues often requires leveraging external non-parametric knowledge, which offers transparency, flexibility, adaptability, and ease of operation. However, augmenting parametric knowledge in LLMs with non-parametric knowledge Yang et al. (2024b); Luo et al. (2023); Wen et al. (2023); Ko et al. (2024) remains an ongoing challenge due to retrieval accuracy from haystack, context lengths, and resources ⁹⁹ 9 On the one hand, storing large amounts of non-parametric knowledge requires a lot of space and high maintenance costs. On the other hand, retrieving information for each user query is very resource-intensive. limitations Shang et al. (2024); Zhao et al. (2024b). Besides, simultaneously retrieving relevant information from a long context and conducting reasoning is nearly impossible in reasoning-in-a-haystack experiments Shang et al. (2024). Similarly, augmenting non-parametric knowledge—either by distilling knowledge from an LLM’s parametric knowledge West et al. (2022); Kazemi et al. (2023) or by using it to parse text directly Zhang et al. (2023b)—also poses significant challenges. Moreover, Yang et al. (2024b) propose a novel explicit memory that lies between parametric and non-parametric knowledge. LLM with explicit memory enjoys a smaller parameter size and lower resource consumption for retrieving external non-parametric knowledge.解决这些问题通常需要利用外部非参数化知识，这提供了透明度、灵活性、适应性和易操作性。然而，由于大海捞针式的检索准确性、上下文长度和资源限制，利用非参数化知识增强 LLM 中的参数化知识 (Yang et al., 2024b; Luo et al., 2023; Wen et al., 2023; Ko et al., 2024) 仍然是一个持续存在的挑战 (Shang et al., 2024; Zhao et al., 2024b)。一方面，存储大量非参数化知识需要大量空间和高昂的维护成本；另一方面，为每个用户查询检索信息是非常耗费资源的。此外，在“大海捞针”推理实验中，同时从长上下文中检索相关信息并进行推理几乎是不可能的 (Shang et al., 2024)。同样，通过从 LLM 的参数化知识中蒸馏知识 (West et al., 2022; Kazemi et al., 2023) 或直接使用它来解析文本 (Zhang et al., 2023b) 来增强非参数化知识，也构成了重大挑战。此外，Yang 等人 (2024b) 提出了一种介于参数化和非参数化知识之间的新型显式记忆。具有显式记忆的 LLM 拥有更小的参数规模，且检索外部非参数化知识的资源消耗更低。

Generally, inspired by the knowledge mechanisms analysis in LLMs, we have the potential to develop more architectural and learning strategies for organizing knowledge within LLMs. These efficient LLMs Sastry et al. (2024) are advancing toward lower GPU, computation, and storage resource requirements, as well as smaller model sizes by combining the strengths of parametric and non-parametric knowledge Yang et al. (2024b); Momeni et al. (2024); Chen (2024); Pan et al. (2024b); Pan et al. (2023b).总体而言，受 LLM 知识机制分析的启发，我们有潜力开发更多的架构和学习策略来组织 LLM 内部的知识。这些高效的 LLM (Sastry et al., 2024) 正通过结合参数化和非参数化知识的优势，朝着更低的 GPU、计算和存储资源需求以及更小的模型规模迈进 (Yang et al., 2024b; Momeni et al., 2024; Chen, 2024; Pan et al., 2024b; Pan et al., 2023b)。

7.2 Embodied Intelligence7.2 具身智能

The current LLM still cannot be regarded as a truly intelligent creature Bender and Koller (2020); Bisk et al. (2020). The process of human language acquisition is not merely a passive process of listening to language. Instead, it is an active and interactive process that involves engagement with the physical world and communication with other people. To enhance the current LLM’s capabilities and transform it into a powerful agent, it is necessary to enable it to learn from multimodal information and interact with the environment and humans.当前的 LLM 仍然不能被视为真正的智能生物 (Bender and Koller, 2020; Bisk et al., 2020)。人类语言的习得过程不仅仅是被动听取语言的过程，而是一个涉及与物理世界互动以及与他人交流的主动且交互的过程。为了增强当前 LLM 的能力并将其转变为强大的智能体，有必要使其能够从多模态信息中学习，并与环境和人类进行交互。

Multimodal LLMs.多模态 LLM。

The integration of multiple modalities is a critical challenge in the field of LLMs and embodied AI. While LLMs have demonstrated impressive capabilities when processing language data, their ability to seamlessly incorporate and synthesize information from other modalities such as images, speech, and video is still an area of active research. However, the current multi-modal model faces challenges, particularly in complex reasoning tasks that require understanding and integrating information from both text and images.多模态的整合是 LLM 和具身 AI 领域的一个关键挑战。虽然 LLM 在处理语言数据时表现出了令人印象深刻的能力，但它们无缝整合和综合来自图像、语音和视频等其他模态信息的能力仍是一个活跃的研究领域。然而，当前的多模态模型面临挑战，特别是在需要理解和整合文本与图像信息的复杂推理任务中。

Recent studies Huang et al. (2024a); Chen et al. (2024c) have highlighted the discrepancy between the model’s performance in language tasks and its ability to integrate knowledge from different modalities effectively. These findings suggest that current models often prioritize linguistic information, failing to fully exploit the synergistic potential of multimodal data Wang et al. (2024c). There are some pioneering efforts in this direction Pan et al. (2024a); Schwettmann et al. (2023a) , aiming to uncover the mechanisms by which multi-modal models store and retrieve information. Despite these advancements, there is still a need for further exploration to deepen our understanding of multi-modal knowledge storage.近期的研究 (Huang et al., 2024a; Chen et al., 2024c) 强调了模型在语言任务中的表现与其有效整合不同模态知识的能力之间的差异。这些发现表明，当前模型往往优先考虑语言信息，而未能充分利用多模态数据的协同潜力 (Wang et al., 2024c)。在该方向上已有了一些开创性的工作 (Pan et al., 2024a; Schwettmann et al., 2023a)，旨在揭示多模态模型存储和检索信息的机制。尽管取得了这些进展，但仍需要进一步探索以加深我们对多模态知识存储的理解。

Self-evolution.自我演化。

As discussed in the previous part, current language models are mainly based on tuning to gain knowledge, which requires a lot of training and high-quality data. These learnings are passive whereas, to be a human, evolution usually also undergoes communication and interaction. As an intelligent agent, the models should be able to learn through interactions and learn by themselves spontaneously. Recently, some work has attempted to enable the model to learn by themselves Zhang et al. (2024b) or learn by interaction with the environment Xu et al. (2024a); Xi et al. (2024). By integrating self-evolving mechanisms, models can continuously update their knowledge base and improve their understanding without relying solely on manually curated datasets. This not only reduces the dependency on large-scale labeled data but also allows the models to adapt to evolving linguistic norms and cultural contexts over time.如前所述，当前的语言模型主要基于微调来获取知识，这需要大量的训练和高质量的数据。这些学习是被动的；而对于人类而言，演化通常还伴随着交流和互动。作为智能体，模型应该能够通过交互进行学习，并自发地进行自我学习。近期，一些工作尝试使模型能够进行自我学习 (Zhang et al., 2024b) 或通过与环境的交互进行学习 (Xu et al., 2024a; Xi et al., 2024)。通过整合自我演化机制，模型可以持续更新其知识库并改进其理解力，而无需完全依赖人工策划的数据集。这不仅减少了对大规模标注数据的依赖，还使模型能够随着时间的推移适应不断演变的语言规范和文化背景。

7.3 Domain LLMs7.3 领域 LLM

The success of general-purpose LLMs has indeed inspired the development of domain-specific models that are tailored to particular areas of knowledge Calderon and Reichart (2024), such as biomedicine Yu et al. (2024a); Moutakanni et al. (2024), finance Yang et al. (2023), geoscience Deng et al. (2023), ocean science Bi et al. (2024), etc. However, unlike human language, the knowledge of these different domains bears specific characteristics. It remains unclear whether LLMs can acquire complex scientific knowledge or if such knowledge still resides within the realm of current dark knowledge. Furthermore, does domain-specific knowledge such as mathematics share the same underlying mechanisms as textual knowledge Bengio and Malkin (2024), or does it exhibit more intricate mechanisms of knowledge acquisition? Currently, there is a relative lack of research focusing on the mechanism of these domain-specific knowledge and there is an increasing recognition of the importance of developing a deeper understanding of these mechanisms.通用 LLM 的成功确实激发了针对特定知识领域量身定制的领域模型的发展 (Calderon and Reichart, 2024)，例如生物医学 (Yu et al., 2024a; Moutakanni et al., 2024)、金融 (Yang et al., 2023)、地球科学 (Deng et al., 2023)、海洋科学 (Bi et al., 2024) 等。然而，与人类语言不同，这些不同领域的知识具有特定的特征。目前尚不清楚 LLM 是否能够获取复杂的科学知识，或者此类知识是否仍处于当前暗知识的范畴内。此外，数学等特定领域的知识是否与文本知识共享相同的底层机制 (Bengio and Malkin, 2024)，还是表现出更复杂的知识获取机制？目前，专注于这些领域特定知识机制的研究相对匮乏，人们也越来越认识到深入理解这些机制的重要性。

Data sparsity and diversity in domain-specific models pose another challenge. Sparsity is usually caused by confidentiality, privacy, and the cost of acquisition in specialized fields. As for diversity, the presentation of knowledge varies across different fields. For instance, in the biomedical domain, knowledge includes complex biological concepts such as the structure and function of proteins and molecules. This requires models to integrate understanding that extends beyond natural language, often involving graphical representations like chemical structures, which cannot be directly expressed in text. Similarly, in fields such as finance and law Lai et al. (2023), models must engage in sophisticated reasoning and decision-making processes based on domain-specific knowledge. Hence, the critical tasks of collecting high-quality data for domain-specific models (including synthetic data generation) and effectively embedding domain knowledge into LLMs require immediate attention.领域特定模型中的数据稀疏性和多样性构成了另一个挑战。稀疏性通常是由专业领域的保密性、隐私性和获取成本引起的。至于多样性，知识的呈现方式在不同领域各不相同。例如，在生物医学领域，知识包括复杂的生物概念，如蛋白质和分子的结构与功能。这要求模型整合超越自然语言的理解，通常涉及化学结构等图形表征，而这些无法直接用文本表达。同样，在金融和法律等领域 (Lai et al., 2023)，模型必须基于领域特定知识进行复杂的推理和决策过程。因此，为领域特定模型收集高质量数据（包括合成数据生成）并将领域知识有效地嵌入 LLM 中的关键任务需要立即引起关注。

8 Conclusion8 结论

In this paper, we propose a novel knowledge mechanism analysis taxonomy and review knowledge evolution. We further discuss knowledge utilization issues, as well as unexplored dark knowledge. We hope these insights may inspire some promising directions for future research and shed light on more powerful and trustworthy models.在本文中，我们提出了一种新颖的知识机制分析分类法，并回顾了知识演化。我们进一步讨论了知识利用问题以及尚未探索的暗知识。我们希望这些见解能够激发未来研究的一些有前景的方向，并为开发更强大、更值得信赖的模型提供启示。

Limitations局限性

This work has some limitations as follows:本研究存在以下局限性：

Hypothesis假设

Despite reviewing a large body of literature and proposing several promising hypotheses, there are still some limitations. On the one hand, there may be other hypotheses for knowledge utilization and evolution in LLMs. On the other hand, the accuracy of these hypotheses requires further exploration and validation over time.尽管回顾了大量文献并提出了几个有前景的假设，但仍存在一些局限性。一方面，对于 LLM 中的知识利用和演化可能存在其他假设；另一方面，这些假设的准确性需要随着时间的推移进行进一步的探索和验证。

Knowledge知识

There are various forms of knowledge representation. However, due to current research constraints, this paper does not delve into space Li et al. (2024f), time Gurnee and Tegmark (2023), event-based knowledge, and geoscience Lin et al. (2024).知识表征的形式多种多样。然而，受限于当前的研究约束，本文未深入探讨空间 (Li et al., 2024f)、时间 (Gurnee and Tegmark, 2023)、基于事件的知识以及地球科学 (Lin et al., 2024)。

Reference参考文献

The field of knowledge mechanisms is developing rapidly and this paper may miss some important references. Additionally, due to the page limit, we have omit certain technical details. We will continue to pay attention to and supplement new works.知识机制领域发展迅速，本文可能遗漏了一些重要的参考文献。此外，由于篇幅限制，我们省略了某些技术细节。我们将持续关注并补充新的研究成果。

Models模型

Despite mentioning artificial neural models in this paper, knowledge mechanism analysis focuses on LLMs. We will continue to pay attention to other modal models progresses. Besides, all existing work has not considered models larger than 100 billion parameters. Whether the knowledge mechanisms within large-scale models are consistent with smaller ones remains to be studied.尽管本文提到了人工神经模型，但知识机制分析主要集中在 LLM 上。我们将继续关注其他模态模型的发展。此外，所有现有工作均未考虑超过 1000 亿参数的模型。大规模模型内部的知识机制是否与较小模型一致，仍有待研究。

Ethics Statement伦理声明

We anticipate no ethical or societal implications arising from our research. However, we acknowledge that the internal mechanisms of large language models might be exploited for malicious purposes. We believe such malicious applications can be prevented through model access and legislative regulation. More critically, a transparent model contributes to the development of safer and more reliable general artificial intelligence.我们预计我们的研究不会产生伦理或社会影响。然而，我们承认大型语言模型的内部机制可能会被用于恶意目的。我们相信，可以通过模型访问控制和立法监管来预防此类恶意应用。更关键的是，透明的模型有助于开发更安全、更可靠的通用人工智能。

Acknowledgements致谢

We would like to express gratitude to the anonymous reviewers for their kind comments. This work was supported by the National Natural Science Foundation of China (No. 62206246, No. NSFCU23B2055, No. NSFCU19B2027), the Fundamental Research Funds for the Central Universities (226-2023-00138), Zhejiang Provincial Natural Science Foundation of China (No. LGG22F030011), Yongjiang Talent Introduction Programme (2021A-156-G), Information Technology Center and State Key Lab of CAD&CG, Zhejiang University, and NUS-NCS Joint Laboratory (A-0008542-00-00).我们衷心感谢匿名评审员提出的宝贵建议。本工作得到了国家自然科学基金（No. 62206246, No. NSFCU23B2055, No. NSFCU19B2027）、中央高校基本科研业务费专项资金（226-2023-00138）、浙江省自然科学基金（No. LGG22F030011）、甬江人才工程（2021A-156-G）、浙江大学信息技术中心及 CAD&CG 国家重点实验室，以及 NUS-NCS 联合实验室（A-0008542-00-00）的支持。

References参考文献

Abufadda and Mansour (2021) Mohammad Abufadda and Khalid Mansour. 2021. A survey of synthetic data generation for machine learning. In 22nd International Arab Conference on Information Technology, ACIT 2021, Muscat, Oman, December 21-23, 2021, pages 1–7. IEEE. Abufadda and Mansour (2021) Mohammad Abufadda and Khalid Mansour. 2021. A survey of synthetic data generation for machine learning. In 22nd International Arab Conference on Information Technology, ACIT 2021, Muscat, Oman, December 21-23, 2021, pages 1–7. IEEE.
Ainslie et al. (2023) Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023. GQA: training generalized multi-query transformer models from multi-head checkpoints. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 4895–4901. Association for Computational Linguistics. Ainslie et al. (2023) Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023. GQA: training generalized multi-query transformer models from multi-head checkpoints. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 4895–4901. Association for Computational Linguistics.
Akiba et al. (2024) Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. 2024. Evolutionary optimization of model merging recipes. CoRR, abs/2403.13187. Akiba et al. (2024) Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. 2024. Evolutionary optimization of model merging recipes. CoRR, abs/2403.13187.
Akyürek et al. (2022) Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Guu. 2022. Towards tracing factual knowledge in language models back to the training data. arXiv preprint arXiv:2205.11482. Akyürek et al. (2022) Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Guu. 2022. Towards tracing factual knowledge in language models back to the training data. arXiv preprint arXiv:2205.11482.
Allen-Zhu and Li (2023a) Zeyuan Allen-Zhu and Yuanzhi Li. 2023a. Physics of language models: Part 1, context-free grammar. CoRR, abs/2305.13673. Allen-Zhu and Li (2023a) Zeyuan Allen-Zhu and Yuanzhi Li. 2023a. Physics of language models: Part 1, context-free grammar. CoRR, abs/2305.13673.
Allen-Zhu and Li (2023b) Zeyuan Allen-Zhu and Yuanzhi Li. 2023b. Physics of language models: Part 3.2, knowledge manipulation. CoRR, abs/2309.14402. Allen-Zhu and Li (2023b) Zeyuan Allen-Zhu and Yuanzhi Li. 2023b. Physics of language models: Part 3.2, knowledge manipulation. CoRR, abs/2309.14402.
Allen-Zhu and Li (2024) Zeyuan Allen-Zhu and Yuanzhi Li. 2024. Physics of language models: Part 3.3, knowledge capacity scaling laws. CoRR, abs/2404.05405. Allen-Zhu and Li (2024) Zeyuan Allen-Zhu and Yuanzhi Li. 2024. Physics of language models: Part 3.3, knowledge capacity scaling laws. CoRR, abs/2404.05405.
Alvero and Peña (2023) Aj Alvero and Courtney Peña. 2023. AI sentience and socioculture. J. Soc. Comput., 4(3):205–220. Alvero and Peña (2023) Aj Alvero and Courtney Peña. 2023. AI sentience and socioculture. J. Soc. Comput., 4(3):205–220.
Antoniades et al. (2024) Antonis Antoniades, Xinyi Wang, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. 2024. Generalization v.s. memorization: Tracing language models’ capabilities back to pretraining data. arXiv preprint arXiv:2407.14985. Antoniades et al. (2024) Antonis Antoniades, Xinyi Wang, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. 2024. Generalization v.s. memorization: Tracing language models’ capabilities back to pretraining data. arXiv preprint arXiv:2407.14985.
Apidianaki (2023) Marianna Apidianaki. 2023. From word types to tokens and back: A survey of approaches to word meaning representation and interpretation. Comput. Linguistics, 49(2):465–523. Apidianaki (2023) Marianna Apidianaki. 2023. From word types to tokens and back: A survey of approaches to word meaning representation and interpretation. Comput. Linguistics, 49(2):465–523.
Azaria and Mitchell (2023) Amos Azaria and Tom M. Mitchell. 2023. The internal state of an LLM knows when its lying. CoRR, abs/2304.13734. Azaria and Mitchell (2023) Amos Azaria and Tom M. Mitchell. 2023. The internal state of an LLM knows when its lying. CoRR, abs/2304.13734.
Bagal et al. (2022) Viraj Bagal, Rishal Aggarwal, P. K. Vinod, and U. Deva Priyakumar. 2022. Molgpt: Molecular generation using a transformer-decoder model. J. Chem. Inf. Model., 62(9):2064–2076. Bagal et al. (2022) Viraj Bagal, Rishal Aggarwal, P. K. Vinod, and U. Deva Priyakumar. 2022. Molgpt: Molecular generation using a transformer-decoder model. J. Chem. Inf. Model., 62(9):2064–2076.
Bai et al. (2022) Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Chris Olah, Benjamin Mann, and Jared Kaplan. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. CoRR, abs/2204.05862. Bai 等人 (2022) Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Chris Olah, Benjamin Mann 以及 Jared Kaplan. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback (通过人类反馈强化学习训练一个有用且无害的助手). CoRR, abs/2204.05862.
Baronchelli et al. (2013) Andrea Baronchelli, Ramon Ferrer-i Cancho, Romualdo Pastor-Satorras, Nick Chater, and Morten H Christiansen. 2013. Networks in cognitive science. Trends in cognitive sciences, 17(7):348–360. Baronchelli 等人 (2013) Andrea Baronchelli, Ramon Ferrer-i Cancho, Romualdo Pastor-Satorras, Nick Chater 以及 Morten H Christiansen. 2013. Networks in cognitive science (认知科学中的网络). Trends in cognitive sciences, 17(7):348–360.
Bau et al. (2017) David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Network dissection: Quantifying interpretability of deep visual representations. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 3319–3327. IEEE Computer Society. Bau 等人 (2017) David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva 以及 Antonio Torralba. 2017. Network dissection: Quantifying interpretability of deep visual representations (网络剖析：量化深度视觉表征的可解释性). 收录于 2017 IEEE 计算机视觉与模式识别会议 (CVPR 2017), 美国夏威夷州火奴鲁鲁, 2017年7月21-26日, 第3319–3327页. IEEE 计算机协会.
Baucal et al. (2014) Aleksandar Baucal, Lausanne Mouline, Switzerland Kristiina Kumpulainen, Charis Psaltis, and Baruch Schwarz. 2014. Social interaction in learning and development. Baucal 等人 (2014) Aleksandar Baucal, Lausanne Mouline, Switzerland Kristiina Kumpulainen, Charis Psaltis 以及 Baruch Schwarz. 2014. Social interaction in learning and development (学习与发展中的社会互动).
Belinkov (2022) Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Comput. Linguistics, 48(1):207–219. Belinkov (2022) Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances (探测分类器：前景、缺陷与进展). Comput. Linguistics, 48(1):207–219.
Belrose et al. (2023) Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. 2023. Eliciting latent predictions from transformers with the tuned lens. CoRR, abs/2303.08112. Belrose 等人 (2023) Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman 以及 Jacob Steinhardt. 2023. Eliciting latent predictions from transformers with the tuned lens (利用调优镜头从 Transformer 中引出潜在预测). CoRR, abs/2303.08112.
Bender et al. (2021) Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In FAccT ’21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3-10, 2021, pages 610–623. ACM. Bender 等人 (2021) Emily M. Bender, Timnit Gebru, Angelina McMillan-Major 以及 Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? (论随机鹦鹉的危险：语言模型会太大吗？) 收录于 FAccT ’21: 2021年 ACM 公平、问责与透明度会议, 虚拟会议 / 加拿大多伦多, 2021年3月3-10日, 第610–623页. ACM.
Bender and Koller (2020) Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185–5198, Online. Association for Computational Linguistics. Bender 和 Koller (2020) Emily M. Bender 和 Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data (迈向自然语言理解：论数据时代的意义、形式与理解). 收录于计算语言学协会第 58 届年会论文集, 第5185–5198页, 在线. 计算语言学协会.
Bengio (2024) Yoshua Bengio. 2024. Government interventions to avert future catastrophic ai risks. Harvard Data Science Review, (Special Issue 5). Bengio (2024) Yoshua Bengio. 2024. Government interventions to avert future catastrophic ai risks (政府为规避未来 AI 灾难性风险所采取的干预措施). Harvard Data Science Review, (特刊 5).
Bengio et al. (2024) Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Trevor Darrell, Yuval Noah Harari, Ya-Qin Zhang, Lan Xue, Shai Shalev-Shwartz, et al. 2024. Managing extreme ai risks amid rapid progress. Science, 384(6698):842–845. Bengio 等人 (2024) Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Trevor Darrell, Yuval Noah Harari, Ya-Qin Zhang, Lan Xue, Shai Shalev-Shwartz 等人. 2024. Managing extreme ai risks amid rapid progress (在快速进步中管理极端 AI 风险). Science, 384(6698):842–845.
Bengio and Malkin (2024) Yoshua Bengio and Nikolay Malkin. 2024. Machine learning and information theory concepts towards an AI mathematician. CoRR, abs/2403.04571. Bengio 和 Malkin (2024) Yoshua Bengio 和 Nikolay Malkin. 2024. Machine learning and information theory concepts towards an AI mathematician (迈向 AI 数学家：机器学习与信息论概念). CoRR, abs/2403.04571.
Bereska and Gavves (2024) Leonard Bereska and Efstratios Gavves. 2024. Mechanistic interpretability for AI safety - A review. CoRR, abs/2404.14082. Bereska 和 Gavves (2024) Leonard Bereska 和 Efstratios Gavves. 2024. Mechanistic interpretability for AI safety - A review (面向 AI 安全的机械可解释性——综述). CoRR, abs/2404.14082.
Berglund et al. (2023) Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. 2023. The reversal curse: Llms trained on "a is b" fail to learn "b is a". CoRR, abs/2309.12288. Berglund 等人 (2023) Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak 以及 Owain Evans. 2023. The reversal curse: Llms trained on "a is b" fail to learn "b is a" (逆转诅咒：在“a 是 b”上训练的 LLM 无法学会“b 是 a”). CoRR, abs/2309.12288.
Bi et al. (2024) Zhen Bi, Ningyu Zhang, Yida Xue, Yixin Ou, Daxiong Ji, Guozhou Zheng, and Huajun Chen. 2024. Oceangpt: A large language model for ocean science tasks. Bi 等人 (2024) Zhen Bi, Ningyu Zhang, Yida Xue, Yixin Ou, Daxiong Ji, Guozhou Zheng 以及 Huajun Chen. 2024. Oceangpt: A large language model for ocean science tasks (OceanGPT：用于海洋科学任务的大型语言模型).
Bian et al. (2023) Ning Bian, Peilin Liu, Xianpei Han, Hongyu Lin, Yaojie Lu, Ben He1, and Le Sun. 2023. A drop of ink may make a million think: The spread of false information in large language models. arxiv.org/pdf/2305.04812v1. Bian 等人 (2023) Ning Bian, Peilin Liu, Xianpei Han, Hongyu Lin, Yaojie Lu, Ben He1 以及 Le Sun. 2023. A drop of ink may make a million think: The spread of false information in large language models (一滴墨水足以引发百万人的思考：大型语言模型中虚假信息的传播). arxiv.org/pdf/2305.04812v1.
Bisk et al. (2020) Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, and Joseph Turian. 2020. Experience grounds language. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8718–8735, Online. Association for Computational Linguistics. Bisk 等人 (2020) Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto 以及 Joseph Turian. 2020. Experience grounds language (经验奠定语言基础). 收录于 2020 年自然语言处理经验方法会议 (EMNLP) 论文集, 第8718–8735页, 在线. 计算语言学协会.
Blanco-Justicia et al. (2024) Alberto Blanco-Justicia, Najeeb Jebreel, Benet Manzanares-Salor, David Sánchez, Josep Domingo-Ferrer, Guillem Collell, and Kuan Eeik Tan. 2024. Digital forgetting in large language models: A survey of unlearning methods. CoRR, abs/2404.02062. Blanco-Justicia 等人 (2024) Alberto Blanco-Justicia, Najeeb Jebreel, Benet Manzanares-Salor, David Sánchez, Josep Domingo-Ferrer, Guillem Collell 以及 Kuan Eeik Tan. 2024. Digital forgetting in large language models: A survey of unlearning methods (大型语言模型中的数字遗忘：机器遗忘方法综述). CoRR, abs/2404.02062.
Bloom et al. (1956) Benjamin S. Bloom, Max D. Engelhart, Edward J. Furst, Walker H. Hill, and David R. Krathwohl. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain. David McKay Company. Bloom 等人 (1956) Benjamin S. Bloom, Max D. Engelhart, Edward J. Furst, Walker H. Hill 以及 David R. Krathwohl. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain (教育目标分类学：教育目标的分类。手册 I：认知领域). David McKay Company.
Bricken et al. (2023) Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Alex Tamkin, Karina Nguyen, Brayden McLean, Tristan Hume Josiah E. Burke, Shan Carter, Tom Henighan, , and Chris Olah. 2023. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread. Bricken 等人 (2023) Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Alex Tamkin, Karina Nguyen, Brayden McLean, Tristan Hume Josiah E. Burke, Shan Carter, Tom Henighan 以及 Chris Olah. 2023. Towards monosemanticity: Decomposing language models with dictionary learning (迈向单义性：利用字典学习分解语言模型). Transformer Circuits Thread.
Brown and Esterle (2020) John N. A. Brown and Lukas Esterle. 2020. I’m already optimal: the dunning-kruger effect, sociogenesis, and self-integration. In 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems, ACSOS 2020, Companion Volume, Washington, DC, USA, August 17-21, 2020, pages 82–84. IEEE. Brown 和 Esterle (2020) John N. A. Brown 和 Lukas Esterle. 2020. I’m already optimal: the dunning-kruger effect, sociogenesis, and self-integration (我已经是最优的了：达克效应、社会发生论与自我整合). 收录于 2020 IEEE 自主计算与自组织系统国际会议 (ACSOS 2020), 伴随卷, 美国华盛顿特区, 2020年8月17-21日, 第82–84页. IEEE.
Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. Brown 等人 (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever 以及 Dario Amodei. 2020. Language models are few-shot learners (语言模型是少样本学习者). 收录于神经信息处理系统进展 33：2020年神经信息处理系统年度会议 (NeurIPS 2020), 2020年12月6-12日, 虚拟会议.
Burns et al. (2023) Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeff Wu. 2023. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. CoRR, abs/2312.09390. Burns 等人 (2023) Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever 以及 Jeff Wu. 2023. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision (弱到强的泛化：通过弱监督引出强能力). CoRR, abs/2312.09390.
Buz et al. (2024) Tolga Buz, Benjamin Frost, Nikola Genchev, Moritz Schneider, Lucie-Aimée Kaffee, and Gerard de Melo. 2024. Investigating wit, creativity, and detectability of large language models in domain-specific writing style adaptation of reddit’s showerthoughts. CoRR, abs/2405.01660. Buz 等人 (2024) Tolga Buz, Benjamin Frost, Nikola Genchev, Moritz Schneider, Lucie-Aimée Kaffee 以及 Gerard de Melo. 2024. Investigating wit, creativity, and detectability of large language models in domain-specific writing style adaptation of reddit’s showerthoughts (调查大型语言模型在 Reddit “淋浴思考”领域特定写作风格适配中的机智、创造力与可检测性). CoRR, abs/2405.01660.
Calderon and Reichart (2024) Nitay Calderon and Roi Reichart. 2024. On behalf of the stakeholders: Trends in nlp model interpretability in the era of llms. arXiv preprint arXiv:2407.19200. Calderon 和 Reichart (2024) Nitay Calderon 和 Roi Reichart. 2024. On behalf of the stakeholders: Trends in nlp model interpretability in the era of llms (代表利益相关者：LLM 时代 NLP 模型可解释性的趋势). arXiv 预印本 arXiv:2407.19200.
Cao et al. (2024a) Boxi Cao, Hongyu Lin, Xianpei Han, and Le Sun. 2024a. The life cycle of knowledge in big language models: A survey. Mach. Intell. Res., 21(2):217–238. Cao 等人 (2024a) Boxi Cao, Hongyu Lin, Xianpei Han 以及 Le Sun. 2024a. The life cycle of knowledge in big language models: A survey (大语言模型中知识的生命周期：综述). Mach. Intell. Res., 21(2):217–238.
Cao et al. (2021a) Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue, and Jin Xu. 2021a. Knowledgeable or educated guess? revisiting language models as knowledge bases. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 1860–1874. Association for Computational Linguistics. Cao 等人 (2021a) Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue 以及 Jin Xu. 2021a. Knowledgeable or educated guess? revisiting language models as knowledge bases (是博学还是有根据的猜测？重访作为知识库的语言模型). 收录于计算语言学协会第 59 届年会暨第 11 届自然语言处理国际联合会议 (ACL/IJCNLP 2021) 论文集, (第 1 卷：长论文), 虚拟会议, 2021年8月1-6日, 第1860–1874页. 计算语言学协会.
Cao et al. (2024b) Boxi Cao, Qiaoyu Tang, Hongyu Lin, Shanshan Jiang, Bin Dong, Xianpei Han, Jiawei Chen, Tianshu Wang, and Le Sun. 2024b. Retentive or forgetful? diving into the knowledge memorizing mechanism of language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 14016–14036, Torino, Italia. ELRA and ICCL. Cao 等人 (2024b) Boxi Cao, Qiaoyu Tang, Hongyu Lin, Shanshan Jiang, Bin Dong, Xianpei Han, Jiawei Chen, Tianshu Wang 以及 Le Sun. 2024b. Retentive or forgetful? diving into the knowledge memorizing mechanism of language models (保持还是遗忘？深入研究语言模型的知识记忆机制). 收录于 2024 年计算语言学、语言资源与评估联合国际会议 (LREC-COLING 2024) 论文集, 第14016–14036页, 意大利都灵. ELRA 和 ICCL.
Cao et al. (2021b) Nicola De Cao, Wilker Aziz, and Ivan Titov. 2021b. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 6491–6506. Association for Computational Linguistics. Cao 等人 (2021b) Nicola De Cao, Wilker Aziz 以及 Ivan Titov. 2021b. Editing factual knowledge in language models (编辑语言模型中的事实知识). 收录于 2021 年自然语言处理经验方法会议 (EMNLP 2021) 论文集, 虚拟会议 / 多米尼加共和国蓬塔卡纳, 2021年11月7-11日, 第6491–6506页. 计算语言学协会.
Caucheteux et al. (2023) Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King. 2023. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nature human behaviour, 7(3):430–441. Caucheteux 等人 (2023) Charlotte Caucheteux, Alexandre Gramfort 以及 Jean-Rémi King. 2023. Evidence of a predictive coding hierarchy in the human brain listening to speech (人类大脑听取语音时存在预测编码层级的证据). Nature human behaviour, 7(3):430–441.
Chakrabarty et al. (2024) Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan, and Chien-Sheng Wu. 2024. Art or artifice? large language models and the false promise of creativity. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI 2024, Honolulu, HI, USA, May 11-16, 2024, pages 30:1–30:34. ACM. Chakrabarty 等人 (2024) Tuhin Chakrabarty, Philippe Laban, Divyansh Agarwal, Smaranda Muresan 以及 Chien-Sheng Wu. 2024. Art or artifice? large language models and the false promise of creativity (艺术还是技巧？大型语言模型与创造力的虚假承诺). 收录于 CHI 计算系统中的人为因素会议 (CHI 2024) 论文集, 美国夏威夷州火奴鲁鲁, 2024年5月11-16日, 第30:1–30:34页. ACM.
Chan et al. (2024) Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2024. Chateval: Towards better llm-based evaluators through multi-agent debate. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. Chan 等人 (2024) Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu 以及 Zhiyuan Liu. 2024. Chateval: Towards better llm-based evaluators through multi-agent debate (ChatEval：通过多智能体辩论实现更好的基于 LLM 的评估器). 收录于第十二届国际学习表征会议 (ICLR 2024), 奥地利维也纳, 2024年5月7-11日. OpenReview.net.
Chan et al. (2023) Lawrence Chan, Leon Lang, and Erik Jenner. 2023. Natural abstractions: Key claims, theorems, and critiques. Chan 等人 (2023) Lawrence Chan, Leon Lang 以及 Erik Jenner. 2023. Natural abstractions: Key claims, theorems, and critiques (自然抽象：关键主张、定理与批判).
Chen (2024) Huajun Chen. 2024. Large knowledge model: Perspectives and challenges. Data Intelligence. Chen (2024) Huajun Chen. 2024. Large knowledge model: Perspectives and challenges (大型知识模型：观点与挑战). Data Intelligence.
Chen et al. (2024a) Lihu Chen, Adam Dejl, and Francesca Toni. 2024a. Analyzing key neurons in large language models. arXiv preprint arXiv:2406.10868. Chen 等人 (2024a) Lihu Chen, Adam Dejl 以及 Francesca Toni. 2024a. Analyzing key neurons in large language models (分析大型语言模型中的关键神经元). arXiv 预印本 arXiv:2406.10868.
Chen et al. (2024b) Lihu Chen, Adam Dejl, and Francesca Toni. 2024b. Identifying query-relevant neurons in large language models for long-form texts. CoRR, abs/2406.10868. Chen 等人 (2024b) Lihu Chen, Adam Dejl 以及 Francesca Toni. 2024b. Identifying query-relevant neurons in large language models for long-form texts (识别长文本大型语言模型中与查询相关的神经元). CoRR, abs/2406.10868.
Chen et al. (2024c) Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Feng Zhao. 2024c. Are we on the right way for evaluating large vision-language models? CoRR, abs/2403.20330. Chen 等人 (2024c) Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin 以及 Feng Zhao. 2024c. Are we on the right way for evaluating large vision-language models? (我们在评估大型视觉语言模型的道路上走对了吗？) CoRR, abs/2403.20330.
Chen et al. (2021) Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, and Zhangyang Wang. 2021. The elastic lottery ticket hypothesis. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 26609–26621. Chen 等人 (2021) Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu 以及 Zhangyang Wang. 2021. The elastic lottery ticket hypothesis (弹性彩票假设). 收录于神经信息处理系统进展 34：2021年神经信息处理系统年度会议 (NeurIPS 2021), 2021年12月6-14日, 虚拟会议, 第26609–26621页.
Chen et al. (2024d) Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David F Fouhey, and Joyce Chai. 2024d. Multi-object hallucination in vision-language models. arXiv preprint arXiv:2407.06192. Chen 等人 (2024d) Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David F Fouhey 以及 Joyce Chai. 2024d. Multi-object hallucination in vision-language models (视觉语言模型中的多目标幻觉). arXiv 预印本 arXiv:2407.06192.
Chen et al. (2024e) Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2024e. Journey to the center of the knowledge neurons: Discoveries of language-independent knowledge neurons and degenerate knowledge neurons. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 17817–17825. AAAI Press. Chen 等人 (2024e) Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu 以及 Jun Zhao. 2024e. Journey to the center of the knowledge neurons: Discoveries of language-independent knowledge neurons and degenerate knowledge neurons (知识神经元中心之旅：语言无关知识神经元与退化知识神经元的发现). 收录于第 38 届 AAAI 人工智能会议 (AAAI 2024), 第 36 届人工智能创新应用会议 (IAAI 2024), 第 14 届人工智能教育进展研讨会 (EAAI 2014), 2024年2月20-27日, 加拿大温哥华, 第17817–17825页. AAAI 出版社.
Chen et al. (2024f) Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2024f. Knowledge localization: Mission not accomplished? enter query localization! CoRR, abs/22405.14117. Chen 等人 (2024f) Yuheng Chen, Pengfei Cao, Yubo Chen, Kang Liu 以及 Jun Zhao. 2024f. Knowledge localization: Mission not accomplished? enter query localization! (知识定位：任务未完成？引入查询定位！) CoRR, abs/22405.14117.
Chen et al. (2024g) Yuheng Chen, Pengfei Cao, Yubo Chen, Yining Wang, Shengping Liu, Kang Liu, and Jun Zhao. 2024g. The da vinci code of large pre-trained language models: Deciphering degenerate knowledge neurons. CoRR, abs/2402.13731. Chen 等人 (2024g) Yuheng Chen, Pengfei Cao, Yubo Chen, Yining Wang, Shengping Liu, Kang Liu 以及 Jun Zhao. 2024g. The da vinci code of large pre-trained language models: Deciphering degenerate knowledge neurons (大型预训练语言模型的达芬奇密码：破译退化知识神经元). CoRR, abs/2402.13731.
Chiang and Cholak (2022) David Chiang and Peter Cholak. 2022. Overcoming a theoretical limitation of self-attention. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 7654–7664. Association for Computational Linguistics. Chiang 和 Cholak (2022) David Chiang 和 Peter Cholak. 2022. Overcoming a theoretical limitation of self-attention (克服自注意力的理论局限性). 收录于计算语言学协会第 60 届年会 (第 1 卷：长论文) 论文集, ACL 2022, 爱尔兰都柏林, 2022年5月22-27日, 第7654–7664页. 计算语言学协会.
Choi (2022) Yejin Choi. 2022. Knowledge is power: Symbolic knowledge distillation, commonsense morality, & multimodal script knowledge. In WSDM ’22: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21 - 25, 2022, page 3. ACM. Choi (2022) Yejin Choi. 2022. Knowledge is power: Symbolic knowledge distillation, commonsense morality, & multimodal script knowledge (知识就是力量：符号知识蒸馏、常识道德与多模态脚本知识). 收录于 WSDM ’22: 第 15 届 ACM 网络搜索与数据挖掘国际会议, 虚拟会议 / 美国亚利桑那州坦佩, 2022年2月21-25日, 第3页. ACM.
Chuang et al. (2023) Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. 2023. Dola: Decoding by contrasting layers improves factuality in large language models. CoRR, abs/2309.03883. Chuang 等人 (2023) Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass 以及 Pengcheng He. 2023. Dola: Decoding by contrasting layers improves factuality in large language models (DoLa：通过对比层进行解码以提高大型语言模型的事实准确性). CoRR, abs/2309.03883.
Chughtai et al. (2023) Bilal Chughtai, Lawrence Chan, and Neel Nanda. 2023. A toy model of universality: Reverse engineering how networks learn group operations. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 6243–6267. PMLR. Chughtai 等人 (2023) Bilal Chughtai, Lawrence Chan 以及 Neel Nanda. 2023. A toy model of universality: Reverse engineering how networks learn group operations (普适性的玩具模型：逆向工程网络如何学习群运算). 收录于机器学习国际会议 (ICML 2023), 2023年7月23-29日, 美国夏威夷州火奴鲁鲁, 机器学习研究论文集第 202 卷, 第6243–6267页. PMLR.
Cohen et al. (2023) Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2023. Evaluating the ripple effects of knowledge editing in language models. CoRR, abs/2307.12976. Cohen 等人 (2023) Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson 以及 Mor Geva. 2023. Evaluating the ripple effects of knowledge editing in language models (评估语言模型中知识编辑的涟漪效应). CoRR, abs/2307.12976.
Conmy et al. (2023) Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Conmy 等人 (2023) Arthur Conmy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim 以及 Adrià Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability (迈向机械可解释性的自动化电路发现). 收录于神经信息处理系统进展 36：2023年神经信息处理系统年度会议 (NeurIPS 2023), 美国路易斯安那州新奥尔良, 2023年12月10-16日.
Coronel-Oliveros et al. (2024) Carlos Coronel-Oliveros, Vicente Medel, Sebastián Orellana, Julio Rodiño, Fernando Lehue, Josephine Cruzat, Enzo Tagliazucchi, Aneta Brzezicka, Patricio Orio, Natalia Kowalczyk-Grebska, and Agustín Ibáñez. 2024. Gaming expertise induces meso-scale brain plasticity and efficiency mechanisms as revealed by whole-brain modeling. NeuroImage, 293:120633. Coronel-Oliveros 等人 (2024) Carlos Coronel-Oliveros, Vicente Medel, Sebastián Orellana, Julio Rodiño, Fernando Lehue, Josephine Cruzat, Enzo Tagliazucchi, Aneta Brzezicka, Patricio Orio, Natalia Kowalczyk-Grebska 以及 Agustín Ibáñez. 2024. Gaming expertise induces meso-scale brain plasticity and efficiency mechanisms as revealed by whole-brain modeling (全脑建模揭示：游戏专业知识诱导中尺度大脑可塑性与效率机制). NeuroImage, 293:120633.
Crosbie and Shutova (2024) Joy Crosbie and Ekaterina Shutova. 2024. Induction heads as an essential mechanism for pattern matching in in-context learning. arxiv.org/pdf/2407.07011. Crosbie 和 Shutova (2024) Joy Crosbie 和 Ekaterina Shutova. 2024. Induction heads as an essential mechanism for pattern matching in in-context learning (作为上下文学习中模式匹配基本机制的归纳头). arxiv.org/pdf/2407.07011.
Cunningham et al. (2023) Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models. CoRR, abs/2309.08600. Cunningham 等人 (2023) Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben 以及 Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models (稀疏自动编码器在语言模型中发现高度可解释的特征). CoRR, abs/2309.08600.
Dai et al. (2022) Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 8493–8502. Association for Computational Linguistics. Dai 等人 (2022) Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang 以及 Furu Wei. 2022. Knowledge neurons in pretrained transformers (预训练 Transformer 中的知识神经元). 收录于计算语言学协会第 60 届年会 (第 1 卷：长论文) 论文集, ACL 2022, 爱尔兰都柏林, 2022年5月22-27日, 第8493–8502页. 计算语言学协会.
Dalrymple et al. (2024) David Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark W. Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, and Joshua B. Tenenbaum. 2024. Towards guaranteed safe AI: A framework for ensuring robust and reliable AI systems. CoRR, abs/2405.06624. Dalrymple 等人 (2024) David Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark W. Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing 以及 Joshua B. Tenenbaum. 2024. Towards guaranteed safe AI: A framework for ensuring robust and reliable AI systems (迈向有保障的安全 AI：确保鲁棒且可靠 AI 系统的框架). CoRR, abs/2405.06624.
Dalvi et al. (2023) Fahim Dalvi, Hassan Sajjad, and Nadir Durrani. 2023. Neurox library for neuron analysis of deep NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2023, Toronto, Canada, July 10-12, 2023, pages 226–234. Association for Computational Linguistics. Dalvi 等人 (2023) Fahim Dalvi, Hassan Sajjad 以及 Nadir Durrani. 2023. Neurox library for neuron analysis of deep NLP models (用于深度 NLP 模型神经元分析的 NeuroX 库). 收录于计算语言学协会第 61 届年会：系统演示论文集, ACL 2023, 加拿大多伦多, 2023年7月10-12日, 第226–234页. 计算语言学协会.
Dan et al. (2023) Yuhao Dan, Zhikai Lei, Yiyang Gu, Yong Li, Jianghao Yin, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, Aimin Zhou, Ze Zhou, Qin Chen, Jie Zhou, Liang He, and Xipeng Qiu. 2023. Educhat: A large-scale language model-based chatbot system for intelligent education. CoRR, abs/2308.02773. Dan 等人 (2023) Yuhao Dan, Zhikai Lei, Yiyang Gu, Yong Li, Jianghao Yin, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, Aimin Zhou, Ze Zhou, Qin Chen, Jie Zhou, Liang He 以及 Xipeng Qiu. 2023. Educhat: A large-scale language model-based chatbot system for intelligent education (EduChat：用于智能教育的大规模基于语言模型的聊天机器人系统). CoRR, abs/2308.02773.
Davis et al. (1993) Randall Davis, Howard E. Shrobe, and Peter Szolovits. 1993. What is a knowledge representation? AI Mag., 14(1):17–33. Davis 等人 (1993) Randall Davis, Howard E. Shrobe 以及 Peter Szolovits. 1993. What is a knowledge representation? (什么是知识表示？) AI Mag., 14(1):17–33.
de Schotten et al. (2022) Thiebaut de Schotten, Michel, Forkel, and Stephanie J. 2022. The emergent properties of the connected brain. Science, 378(6619):505–510. de Schotten 等人 (2022) Thiebaut de Schotten, Michel, Forkel 以及 Stephanie J. 2022. The emergent properties of the connected brain (连接大脑的涌现属性). Science, 378(6619):505–510.
Debbouche et al. (2021) Nadjette Debbouche, Adel Ouannas, Iqbal M. Batiha, Giuseppe Grassi, Mohammed K. A. Kaabar, Hadi Jahanshahi, Ayman A. Aly, and Awad M. Aljuaid. 2021. Chaotic behavior analysis of a new incommensurate fractional-order hopfield neural network system. Complex., 2021:3394666:1–3394666:11. Debbouche 等人 (2021) Nadjette Debbouche, Adel Ouannas, Iqbal M. Batiha, Giuseppe Grassi, Mohammed K. A. Kaabar, Hadi Jahanshahi, Ayman A. Aly 以及 Awad M. Aljuaid. 2021. Chaotic behavior analysis of a new incommensurate fractional-order hopfield neural network system (一种新型非公度分数阶 Hopfield 神经网络系统的混沌行为分析). Complex., 2021:3394666:1–3394666:11.
DeLorenzo et al. (2024) Matthew DeLorenzo, Vasudev Gohil, and Jeyavijayan Rajendran. 2024. Creativeval: Evaluating creativity of llm-based hardware code generation. CoRR, abs/2404.08806. DeLorenzo 等人 (2024) Matthew DeLorenzo, Vasudev Gohil 以及 Jeyavijayan Rajendran. 2024. Creativeval: Evaluating creativity of llm-based hardware code generation (CreativeVal：评估基于 LLM 的硬件代码生成的创造力). CoRR, abs/2404.08806.
Deng et al. (2023) Cheng Deng, Tianhang Zhang, Zhongmou He, Yi Xu, Qiyuan Chen, Yuanyuan Shi, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin, and Junxian He. 2023. K2: A foundation language model for geoscience knowledge understanding and utilization. Deng 等人 (2023) Cheng Deng, Tianhang Zhang, Zhongmou He, Yi Xu, Qiyuan Chen, Yuanyuan Shi, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin 以及 Junxian He. 2023. K2: A foundation language model for geoscience knowledge understanding and utilization (K2：用于地球科学知识理解与利用的基础语言模型).
Dey et al. (2023) Nolan Dey, Gurpreet Gosal, Zhiming Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom, and Joel Hestness. 2023. Cerebras-gpt: Open compute-optimal language models trained on the cerebras wafer-scale cluster. CoRR, abs/2304.03208. Dey 等人 (2023) Nolan Dey, Gurpreet Gosal, Zhiming Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom 以及 Joel Hestness. 2023. Cerebras-gpt: Open compute-optimal language models trained on the cerebras wafer-scale cluster (Cerebras-GPT：在 Cerebras 晶圆级集群上训练的开源计算最优语言模型). CoRR, abs/2304.03208.
Didolkar et al. (2024) Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy P. Lillicrap, Danilo J. Rezende, Yoshua Bengio, Michael Mozer, and Sanjeev Arora. 2024. Metacognitive capabilities of llms: An exploration in mathematical problem solving. CoRR, abs/2405.12205. Didolkar 等人 (2024) Aniket Didolkar, Anirudh Goyal, Nan Rosemary Ke, Siyuan Guo, Michal Valko, Timothy P. Lillicrap, Danilo J. Rezende, Yoshua Bengio, Michael Mozer 以及 Sanjeev Arora. 2024. Metacognitive capabilities of llms: An exploration in mathematical problem solving (LLM 的元认知能力：数学问题求解探索). CoRR, abs/2405.12205.
Din et al. (2024) Alexander Yom Din, Taelin Karidi, Leshem Choshen, and Mor Geva. 2024. Jump to conclusions: Short-cutting transformers with linear transformations. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, pages 9615–9625. ELRA and ICCL. Din 等人 (2024) Alexander Yom Din, Taelin Karidi, Leshem Choshen 以及 Mor Geva. 2024. Jump to conclusions: Short-cutting transformers with linear transformations (跳向结论：用线性变换对 Transformer 进行捷径处理). 收录于 2024 年计算语言学、语言资源与评估联合国际会议 (LREC/COLING 2024) 论文集, 2024年5月20-25日, 意大利都灵, 第9615–9625页. ELRA 和 ICCL.
Dosovitskiy et al. (2021) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. Dosovitskiy 等人 (2021) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit 以及 Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale (一张图片胜过 16x16 个单词：用于大规模图像识别的 Transformer). 收录于第 9 届国际学习表征会议 (ICLR 2021), 虚拟会议, 奥地利, 2021年5月3-7日. OpenReview.net.
Douglas et al. (2024) Raymond Douglas, Andis Draguns, and Tomas Gavenciak. 2024. Mitigating the problem of strong priors in lms with context extrapolation. CoRR, abs/2401.17692. Douglas 等人 (2024) Raymond Douglas, Andis Draguns 以及 Tomas Gavenciak. 2024. Mitigating the problem of strong priors in lms with context extrapolation (利用上下文外推减轻语言模型中强先验的问题). CoRR, abs/2401.17692.
Dunefsky et al. (2024) Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. 2024. Transcoders find interpretable llm feature circuits. CoRR. Dunefsky 等人 (2024) Jacob Dunefsky, Philippe Chlenski 以及 Neel Nanda. 2024. Transcoders find interpretable llm feature circuits (转码器发现可解释的 LLM 特征电路). CoRR.
Dutta et al. (2024) Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty. 2024. How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning. CoRR, abs/2402.18312. Dutta 等人 (2024) Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti 以及 Tanmoy Chakraborty. 2024. How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning (如何逐步思考：对思维链推理的机械理解). CoRR, abs/2402.18312.
Edwards et al. (2022) Carl Edwards, Tuan Manh Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. 2022. Translation between molecules and natural language. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 375–413. Association for Computational Linguistics. Edwards 等人 (2022) Carl Edwards, Tuan Manh Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho 以及 Heng Ji. 2022. Translation between molecules and natural language (分子与自然语言之间的翻译). 收录于 2022 年自然语言处理经验方法会议 (EMNLP 2022) 论文集, 阿联酋阿布扎比, 2022年12月7-11日, 第375–413页. 计算语言学协会.
Elazar et al. (2021) Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. 2021. Amnesic probing: Behavioral explanation with amnesic counterfactuals. Trans. Assoc. Comput. Linguistics, 9:160–175. Elazar 等人 (2021) Yanai Elazar, Shauli Ravfogel, Alon Jacovi 以及 Yoav Goldberg. 2021. Amnesic probing: Behavioral explanation with amnesic counterfactuals (遗忘性探测：基于遗忘反事实的行为解释). Trans. Assoc. Comput. Linguistics, 9:160–175.
Elhage et al. (2022) Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of superposition. CoRR, abs/2209.10652. Elhage 等人 (2022) Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg 以及 Christopher Olah. 2022. Toy models of superposition (叠加的玩具模型). CoRR, abs/2209.10652.
Elhage et al. (2021) Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2021. A mathematical framework for transformer circuits. Transformer Circuits Thread. Https://transformer-circuits.pub/2021/framework/index.html. Elhage 等人 (2021) Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish 以及 Chris Olah. 2021. A mathematical framework for transformer circuits (Transformer 电路的数学框架). Transformer Circuits Thread. https://transformer-circuits.pub/2021/framework/index.html.
Ethayarajh et al. (2024) Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. KTO: model alignment as prospect theoretic optimization. CoRR, abs/2402.01306. Ethayarajh 等人 (2024) Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky 以及 Douwe Kiela. 2024. KTO: model alignment as prospect theoretic optimization (KTO：作为前景理论优化的模型对齐). CoRR, abs/2402.01306.
Fadul (2009) Jose A Fadul. 2009. Collective learning: Applying distributed cognition for collective intelligence. International Journal of Learning, 16(4). Fadul (2009) Jose A Fadul. 2009. Collective learning: Applying distributed cognition for collective intelligence (集体学习：应用分布式认知实现集体智慧). International Journal of Learning, 16(4).
Fan et al. (2018) Angela Fan, Mike Lewis, and Yann N. Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, pages 889–898. Association for Computational Linguistics. Fan 等人 (2018) Angela Fan, Mike Lewis 以及 Yann N. Dauphin. 2018. Hierarchical neural story generation (分层神经故事生成). 收录于计算语言学协会第 56 届年会论文集, ACL 2018, 澳大利亚墨尔本, 2018年7月15-20日, 第 1 卷：长论文, 第889–898页. 计算语言学协会.
Fang et al. (2023) Yin Fang, Ningyu Zhang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. 2023. Domain-agnostic molecular generation with self-feedback. CoRR, abs/2301.11259. Fang 等人 (2023) Yin Fang, Ningyu Zhang, Zhuo Chen, Xiaohui Fan 以及 Huajun Chen. 2023. Domain-agnostic molecular generation with self-feedback (具有自我反馈的领域无关分子生成). CoRR, abs/2301.11259.
Farquhar et al. (2024) Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625–630. Farquhar 等人 (2024) Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn 以及 Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy (利用语义熵检测大型语言模型中的幻觉). Nature, 630(8017):625–630.
Fedus et al. (2022) William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. J. Mach. Learn. Res., 23:120:1–120:39. Fedus 等人 (2022) William Fedus, Barret Zoph 以及 Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity (Switch Transformer：通过简单高效的稀疏性扩展至万亿参数模型). J. Mach. Learn. Res., 23:120:1–120:39.
Feng et al. (2023a) Xiachong Feng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023a. Aligning semantic in brain and language: A curriculum contrastive method for electroencephalography-to-text generation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. Feng 等人 (2023a) Xiachong Feng, Xiaocheng Feng, Bing Qin 以及 Ting Liu. 2023a. Aligning semantic in brain and language: A curriculum contrastive method for electroencephalography-to-text generation (对齐大脑与语言中的语义：一种用于脑电图转文本生成的课程对比方法). IEEE Transactions on Neural Systems and Rehabilitation Engineering.
Feng et al. (2023b) Zhangyin Feng, Weitao Ma, Weijiang Yu, Lei Huang, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023b. Trends in integration of knowledge and large language models: A survey and taxonomy of methods, benchmarks, and applications. CoRR, abs/2311.05876. Feng 等人 (2023b) Zhangyin Feng, Weitao Ma, Weijiang Yu, Lei Huang, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin 以及 Ting Liu. 2023b. Trends in integration of knowledge and large language models: A survey and taxonomy of methods, benchmarks, and applications (知识与大型语言模型集成趋势：方法、基准与应用综述及分类). CoRR, abs/2311.05876.
Ferrando et al. (2024) Javier Ferrando, Gabriele Sarti, Arianna Bisazza, and Marta R. Costa-jussà. 2024. A primer on the inner workings of transformer-based language models. CoRR, abs/2405.00208. Ferrando 等人 (2024) Javier Ferrando, Gabriele Sarti, Arianna Bisazza 以及 Marta R. Costa-jussà. 2024. A primer on the inner workings of transformer-based language models (基于 Transformer 的语言模型内部工作原理入门). CoRR, abs/2405.00208.
Flesher et al. (2021) Sharlene N Flesher, John E Downey, Jeffrey M Weiss, Christopher L Hughes, Angelica J Herrera, Elizabeth C Tyler-Kabara, Michael L Boninger, Jennifer L Collinger, and Robert A Gaunt. 2021. A brain-computer interface that evokes tactile sensations improves robotic arm control. Science, 372(6544):831–836. Flesher 等人 (2021) Sharlene N Flesher, John E Downey, Jeffrey M Weiss, Christopher L Hughes, Angelica J Herrera, Elizabeth C Tyler-Kabara, Michael L Boninger, Jennifer L Collinger 以及 Robert A Gaunt. 2021. A brain-computer interface that evokes tactile sensations improves robotic arm control (唤起触觉的脑机接口改善了机械臂控制). Science, 372(6544):831–836.
Frankle and Carbin (2019) Jonathan Frankle and Michael Carbin. 2019. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. Frankle 和 Carbin (2019) Jonathan Frankle 和 Michael Carbin. 2019. The lottery ticket hypothesis: Finding sparse, trainable neural networks (彩票假设：寻找稀疏、可训练的神经网络). 收录于第 7 届国际学习表征会议 (ICLR 2019), 美国路易斯安那州新奥尔良, 2019年5月6-9日. OpenReview.net.
Gao et al. (2024a) Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024a. Scaling and evaluating sparse autoencoders. CoRR, abs/2406.04093. Gao 等人 (2024a) Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike 以及 Jeffrey Wu. 2024a. Scaling and evaluating sparse autoencoders (扩展并评估稀疏自动编码器). CoRR, abs/2406.04093.
Gao et al. (2024b) Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024b. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093. Gao 等人 (2024b) Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike 以及 Jeffrey Wu. 2024b. Scaling and evaluating sparse autoencoders (扩展并评估稀疏自动编码器). arXiv 预印本 arXiv:2406.04093.
Gao et al. (2024c) Leo Gao, Tom Dupré la Tour andHenk Tillman, Gabriel Goh andRajan Troll, Alec Radford andIlya Sutskever, Jan Leike, and Jeffrey Wu. 2024c. Scaling and evaluating sparse autoencoders. CoRR, abs/2406.04093. Gao 等人 (2024c) Leo Gao, Tom Dupré la Tour andHenk Tillman, Gabriel Goh andRajan Troll, Alec Radford andIlya Sutskever, Jan Leike 以及 Jeffrey Wu. 2024c. Scaling and evaluating sparse autoencoders (扩展并评估稀疏自动编码器). CoRR, abs/2406.04093.
Ge et al. (2024) Huaizhi Ge, Frank Rudzicz, and Zining Zhu. 2024. What do the circuits mean? a knowledge edit view. arXiv preprint arXiv:2406.17241. Ge 等人 (2024) Huaizhi Ge, Frank Rudzicz 以及 Zining Zhu. 2024. What do the circuits mean? a knowledge edit view (电路意味着什么？知识编辑视角). arXiv 预印本 arXiv:2406.17241.
Geiger et al. (2022) Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah D. Goodman, and Christopher Potts. 2022. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 7324–7338. PMLR. Geiger 等人 (2022) Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah D. Goodman 以及 Christopher Potts. 2022. Inducing causal structure for interpretable neural networks (为可解释神经网络诱导因果结构). 收录于机器学习国际会议 (ICML 2022), 2022年7月17-23日, 美国马里兰州巴尔的摩, 机器学习研究论文集第 162 卷, 第7324–7338页. PMLR.
Gekhman et al. (2024) Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. 2024. Does fine-tuning llms on new knowledge encourage hallucinations? CoRR, abs/2405.05904. Gekhman 等人 (2024) Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart 以及 Jonathan Herzig. 2024. Does fine-tuning llms on new knowledge encourage hallucinations? (在知识上微调 LLM 会助长幻觉吗？) CoRR, abs/2405.05904.
Geva et al. (2023) Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 12216–12235. Association for Computational Linguistics. Geva 等人 (2023) Mor Geva, Jasmijn Bastings, Katja Filippova 以及 Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models (解构自回归语言模型中事实关联的召回). 收录于 2023 年自然语言处理经验方法会议 (EMNLP 2023) 论文集, 新加坡, 2023年12月6-10日, 第12216–12235页. 计算语言学协会.
Geva et al. (2022a) Mor Geva, Avi Caciularu, Guy Dar, Paul Roit, Shoval Sadde, Micah Shlain, Bar Tamir, and Yoav Goldberg. 2022a. Lm-debugger: An interactive tool for inspection and intervention in transformer-based language models. In Proceedings of the The 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022 - System Demonstrations, Abu Dhabi, UAE, December 7-11, 2022, pages 12–21. Association for Computational Linguistics. Geva 等人 (2022a) Mor Geva, Avi Caciularu, Guy Dar, Paul Roit, Shoval Sadde, Micah Shlain, Bar Tamir 以及 Yoav Goldberg. 2022a. Lm-debugger: An interactive tool for inspection and intervention in transformer-based language models (LM-Debugger：用于检查和干预基于 Transformer 的语言模型的交互式工具). 收录于 2022 年自然语言处理经验方法会议 (EMNLP 2022) - 系统演示论文集, 阿联酋阿布扎比, 2022年12月7-11日, 第12–21页. 计算语言学协会.
Geva et al. (2022b) Mor Geva, Avi Caciularu, Kevin Ro Wang, and Yoav Goldberg. 2022b. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 30–45. Association for Computational Linguistics. Geva 等人 (2022b) Mor Geva, Avi Caciularu, Kevin Ro Wang 以及 Yoav Goldberg. 2022b. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space (Transformer 前馈层通过在词汇空间中提升概念来构建预测). 收录于 2022 年自然语言处理经验方法会议 (EMNLP 2022) 论文集, 阿联酋阿布扎比, 2022年12月7-11日, 第30–45页. 计算语言学协会.
Geva et al. (2021) Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 5484–5495. Association for Computational Linguistics. Geva 等人 (2021) Mor Geva, Roei Schuster, Jonathan Berant 以及 Omer Levy. 2021. Transformer feed-forward layers are key-value memories (Transformer 前馈层是键值记忆). 收录于 2021 年自然语言处理经验方法会议 (EMNLP 2021) 论文集, 虚拟会议 / 多米尼加共和国蓬塔卡纳, 2021年11月7-11日, 第5484–5495页. 计算语言学协会.
Ghandeharioun et al. (2024) Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. 2024. Patchscopes: A unifying framework for inspecting hidden representations of language models. Ghandeharioun 等人 (2024) Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon 以及 Mor Geva. 2024. Patchscopes: A unifying framework for inspecting hidden representations of language models (Patchscopes：一种用于检查语言模型隐藏表征的统一框架).
Goddard et al. (2024) Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. Arcee’s mergekit: A toolkit for merging large language models. CoRR, abs/2403.13257. Goddard 等人 (2024) Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade 以及 Jacob Solawetz. 2024. Arcee’s mergekit: A toolkit for merging large language models (Arcee 的 Mergekit：用于合并大型语言模型的工具包). CoRR, abs/2403.13257.
Goldowsky-Dill et al. (2023) Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. 2023. Localizing model behavior with path patching. CoRR, abs/2304.05969. Goldowsky-Dill 等人 (2023) Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato 以及 Aryaman Arora. 2023. Localizing model behavior with path patching (利用路径补丁定位模型行为). CoRR, abs/2304.05969.
Gómez-Rodríguez and Williams (2023) Carlos Gómez-Rodríguez and Paul Williams. 2023. A confederacy of models: a comprehensive evaluation of llms on creative writing. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 14504–14528. Association for Computational Linguistics. Gómez-Rodríguez 和 Williams (2023) Carlos Gómez-Rodríguez 和 Paul Williams. 2023. A confederacy of models: a comprehensive evaluation of llms on creative writing (模型联盟：LLM 在创意写作上的综合评估). 收录于计算语言学协会研究结果：EMNLP 2023, 新加坡, 2023年12月6-10日, 第14504–14528页. 计算语言学协会.
Goodfellow et al. (2014) Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 2672–2680. Goodfellow 等人 (2014) Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville 以及 Yoshua Bengio. 2014. Generative adversarial nets (生成对抗网络). 收录于神经信息处理系统进展 27：2014年神经信息处理系统年度会议, 2014年12月8-13日, 加拿大蒙特利尔, 第2672–2680页.
Goodhart (1984) C. A. E. Goodhart. 1984. Problems of Monetary Management: The UK Experience, pages 91–121. Macmillan Education UK, London. Goodhart (1984) C. A. E. Goodhart. 1984. Problems of Monetary Management: The UK Experience (货币管理问题：英国经验), 第91–121页. Macmillan Education UK, 伦敦.
Gould et al. (2023a) Rhys Gould, Euan Ong, George Ogden, and Arthur Conmy. 2023a. Successor heads: Recurring, interpretable attention heads in the wild. CoRR, abs/2312.09230. Gould 等人 (2023a) Rhys Gould, Euan Ong, George Ogden 以及 Arthur Conmy. 2023a. Successor heads: Recurring, interpretable attention heads in the wild (后继头：野外循环、可解释的注意力头). CoRR, abs/2312.09230.
Gould et al. (2023b) Rhys Gould, Euan Ong, George Ogden, and Arthur Conmy. 2023b. Successor heads: Recurring, interpretable attention heads in the wild. CoRR, abs/2312.09230. Gould 等人 (2023b) Rhys Gould, Euan Ong, George Ogden 以及 Arthur Conmy. 2023b. Successor heads: Recurring, interpretable attention heads in the wild (后继头：野外循环、可解释的注意力头). CoRR, abs/2312.09230.
Gray et al. (2024) Colin M. Gray, Cristiana Teixeira Santos, Nataliia Bielova, and Thomas Mildner. 2024. An ontology of dark patterns knowledge: Foundations, definitions, and a pathway for shared knowledge-building. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI 2024, Honolulu, HI, USA, May 11-16, 2024, pages 289:1–289:22. ACM. Gray 等人 (2024) Colin M. Gray, Cristiana Teixeira Santos, Nataliia Bielova 以及 Thomas Mildner. 2024. An ontology of dark patterns knowledge: Foundations, definitions, and a pathway for shared knowledge-building (暗黑模式知识本体：基础、定义与共享知识构建的途径). 收录于 CHI 计算系统中的人为因素会议 (CHI 2024) 论文集, 美国夏威夷州火奴鲁鲁, 2024年5月11-16日, 第289:1–289:22页. ACM.
Gu and Dao (2023) Albert Gu and Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces. CoRR, abs/2312.00752. Gu 和 Dao (2023) Albert Gu 和 Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces (Mamba：具有选择性状态空间的线性时间序列建模). CoRR, abs/2312.00752.
Gu et al. (2024) Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. 2024. Model editing can hurt general abilities of large language models. CoRR, abs/2401.04700. Gu 等人 (2024) Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang 以及 Nanyun Peng. 2024. Model editing can hurt general abilities of large language models (模型编辑可能会损害大型语言模型的通用能力). CoRR, abs/2401.04700.
Gu et al. (2023) Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, and Philip H. S. Torr. 2023. A systematic survey of prompt engineering on vision-language foundation models. CoRR, abs/2307.12980. Gu 等人 (2023) Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp 以及 Philip H. S. Torr. 2023. A systematic survey of prompt engineering on vision-language foundation models (关于视觉语言基础模型提示工程的系统综述). CoRR, abs/2307.12980.
Gupta et al. (2023) Anshita Gupta, Debanjan Mondal, Akshay Krishna Sheshadri, Wenlong Zhao, Xiang Li, Sarah Wiegreffe, and Niket Tandon. 2023. Editing common sense in transformers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 8214–8232. Association for Computational Linguistics. Gupta 等人 (2023) Anshita Gupta, Debanjan Mondal, Akshay Krishna Sheshadri, Wenlong Zhao, Xiang Li, Sarah Wiegreffe 以及 Niket Tandon. 2023. Editing common sense in transformers (编辑 Transformer 中的常识). 收录于 2023 年自然语言处理经验方法会议 (EMNLP 2023) 论文集, 新加坡, 2023年12月6-10日, 第8214–8232页. 计算语言学协会.
Gurnee et al. (2023) Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. 2023. Finding neurons in a haystack: Case studies with sparse probing. CoRR, abs/2305.01610. Gurnee 等人 (2023) Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii 以及 Dimitris Bertsimas. 2023. Finding neurons in a haystack: Case studies with sparse probing (大海捞针：稀疏探测案例研究). CoRR, abs/2305.01610.
Gurnee and Tegmark (2023) Wes Gurnee and Max Tegmark. 2023. Language models represent space and time. CoRR, abs/2310.02207. Gurnee 和 Tegmark (2023) Wes Gurnee 和 Max Tegmark. 2023. Language models represent space and time (语言模型表征空间与时间). CoRR, abs/2310.02207.
Gutiérrez et al. (2024) Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. Hipporag: Neurobiologically inspired long-term memory for large language models. CoRR, abs/2405.14831. Gutiérrez 等人 (2024) Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga 以及 Yu Su. 2024. Hipporag: Neurobiologically inspired long-term memory for large language models (HippoRAG：受神经生物学启发的大型语言模型长期记忆). CoRR, abs/2405.14831.
Haase and Hanel (2023) Jennifer Haase and Paul H. P. Hanel. 2023. Artificial muses: Generative artificial intelligence chatbots have risen to human-level creativity. CoRR, abs/2303.12003. Haase 和 Hanel (2023) Jennifer Haase 和 Paul H. P. Hanel. 2023. Artificial muses: Generative artificial intelligence chatbots have risen to human-level creativity (人工缪斯：生成式人工智能聊天机器人已达到人类水平的创造力). CoRR, abs/2303.12003.
Hagendorff (2023) Thilo Hagendorff. 2023. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. CoRR, abs/2303.13988. Hagendorff (2023) Thilo Hagendorff. 2023. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods (机器心理学：使用心理学方法调查大型语言模型中的涌现能力与行为). CoRR, abs/2303.13988.
Hagendorff and Danks (2023) Thilo Hagendorff and David Danks. 2023. Ethical and methodological challenges in building morally informed AI systems. AI Ethics, 3(2):553–566. Hagendorff 和 Danks (2023) Thilo Hagendorff 和 David Danks. 2023. Ethical and methodological challenges in building morally informed AI systems (构建道德知情 AI 系统中的伦理与方法论挑战). AI Ethics, 3(2):553–566.
Han et al. (2021) Xu Han, Zhengyan Zhang, and Zhiyuan Liu. 2021. Knowledgeable machine learning for natural language processing. Commun. ACM, 64(11):50–51. Han 等人 (2021) Xu Han, Zhengyan Zhang 以及 Zhiyuan Liu. 2021. Knowledgeable machine learning for natural language processing (面向自然语言处理的博学机器学习). Commun. ACM, 64(11):50–51.
Han et al. (2022) Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. 2022. PTR: prompt tuning with rules for text classification. AI Open, 3:182–192. Han 等人 (2022) Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu 以及 Maosong Sun. 2022. PTR: prompt tuning with rules for text classification (PTR：用于文本分类的带规则提示调优). AI Open, 3:182–192.
Hanna et al. (2023) Michael Hanna, Ollie Liu, and Alexandre Variengien. 2023. How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Hanna 等人 (2023) Michael Hanna, Ollie Liu 和 Alexandre Variengien. 2023. GPT-2 是如何计算“大于”关系的？：对预训练语言模型数学能力的解释。收录于《神经信息处理系统进展 36：2023 年神经信息处理系统年会》(NeurIPS 2023)，美国路易斯安那州新奥尔良，2023 年 12 月 10 日至 16 日。
Hase et al. (2023) Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. 2023. Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Hase 等人 (2023) Peter Hase, Mohit Bansal, Been Kim 和 Asma Ghandeharioun. 2023. 定位分析有助于模型编辑吗？语言模型中基于因果关系的定位与知识编辑之间的显著差异。收录于《神经信息处理系统进展 36：2023 年神经信息处理系统年会》(NeurIPS 2023)，美国路易斯安那州新奥尔良，2023 年 12 月 10 日至 16 日。
Hase et al. (2024) Peter Hase, Thomas Hofweber, Xiang Zhou, Elias Stengel-Eskin1, and Mohit Bansal1. 2024. Fundamental problems with model editing: How should rational belief revision work in llms? arxiv.org/pdf/2406.19354. Hase 等人 (2024) Peter Hase, Thomas Hofweber, Xiang Zhou, Elias Stengel-Eskin1 和 Mohit Bansal1. 2024. 模型编辑的根本问题：大语言模型中的理性信念修正应该如何工作？ arxiv.org/pdf/2406.19354。
He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770–778. IEEE Computer Society. He 等人 (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren 和 Jian Sun. 2016. 用于图像识别的深度残差学习。收录于《2016 年 IEEE 计算机视觉与模式识别会议》(CVPR 2016)，美国内华达州拉斯维加斯，2016 年 6 月 27 日至 30 日，第 770–778 页。IEEE 计算机协会。
He et al. (2024a) Qiyuan He, Yizhong Wang, and Wenya Wang. 2024a. Can language models act as knowledge bases at scale? CoRR, abs/2402.14273. He 等人 (2024a) Qiyuan He, Yizhong Wang 和 Wenya Wang. 2024a. 语言模型能否大规模充当知识库？ CoRR, abs/2402.14273。
He et al. (2024b) Zhengfu He, Xuyang Ge, Qiong Tang, Tianxiang Sun, Qinyuan Cheng, and Xipeng Qiu. 2024b. Dictionary learning improves patch-free circuit discovery in mechanistic interpretability: A case study on othello-gpt. CoRR, abs/2402.12201. He 等人 (2024b) Zhengfu He, Xuyang Ge, Qiong Tang, Tianxiang Sun, Qinyuan Cheng 和 Xipeng Qiu. 2024b. 字典学习改进了机械可解释性中无补丁的电路发现：以 Othello-GPT 为例。 CoRR, abs/2402.12201。
Heilman et al. (2003) Kenneth M Heilman, Stephen E Nadeau, and David O Beversdorf. 2003. Creative innovation: possible brain mechanisms. Neurocase, 9(5):369–379. Heilman 等人 (2003) Kenneth M Heilman, Stephen E Nadeau 和 David O Beversdorf. 2003. 创造性创新：可能的大脑机制。《Neurocase》, 9(5):369–379。
Heinzerling and Inui (2020) Benjamin Heinzerling and Kentaro Inui. 2020. Language models as knowledge bases: On entity representations, storage capacity, and paraphrased queries. CoRR, abs/2008.09036. Heinzerling 和 Inui (2020) Benjamin Heinzerling 和 Kentaro Inui. 2020. 作为知识库的语言模型：关于实体表示、存储容量和释义查询。 CoRR, abs/2008.09036。
Hoffmann et al. (2022) Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. 2022. Training compute-optimal large language models. CoRR, abs/2203.15556. Hoffmann 等人 (2022) Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals 和 Laurent Sifre. 2022. 训练计算最优的大语言模型。 CoRR, abs/2203.15556。
Hoover et al. (2020) Benjamin Hoover, Hendrik Strobelt, and Sebastian Gehrmann. 2020. exbert: A visual analysis tool to explore learned representations in transformer models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2020, Online, July 5-10, 2020, pages 187–196. Association for Computational Linguistics. Hoover 等人 (2020) Benjamin Hoover, Hendrik Strobelt 和 Sebastian Gehrmann. 2020. exbert：一种用于探索 Transformer 模型中学习到的表示的可视化分析工具。收录于《第 58 届国际计算语言学协会年会：系统演示》(ACL 2020)，在线会议，2020 年 7 月 5 日至 10 日，第 187–196 页。国际计算语言学协会。
Hou et al. (2023) Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, and Mrinmaya Sachan. 2023. Towards a mechanistic interpretation of multi-step reasoning capabilities of language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 4902–4919. Association for Computational Linguistics. Hou 等人 (2023) Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut 和 Mrinmaya Sachan. 2023. 迈向语言模型多步推理能力的机械解释。收录于《2023 年自然语言处理经验方法会议》(EMNLP 2023)，新加坡，2023 年 12 月 6 日至 10 日，第 4902–4919 页。国际计算语言学协会。
Hu et al. (2022) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. Hu 等人 (2022) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang 和 Weizhu Chen. 2022. LoRA：大语言模型的低秩自适应。收录于《第十届国际学习表征会议》(ICLR 2022)，虚拟会议，2022 年 4 月 25 日至 29 日。OpenReview.net。
Hu et al. (2024) Yi Hu, Xiaojuan Tang, Haotong Yang, and Muhan Zhang. 2024. Case-based or rule-based: How do transformers do the math? CoRR, abs/2402.17709. Hu 等人 (2024) Yi Hu, Xiaojuan Tang, Haotong Yang 和 Muhan Zhang. 2024. 基于案例还是基于规则：Transformer 如何进行数学运算？ CoRR, abs/2402.17709。
Huang et al. (2024a) Han Huang, Haitian Zhong, Tao Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2024a. Vlkeb: A large vision-language model knowledge editing benchmark. Huang 等人 (2024a) Han Huang, Haitian Zhong, Tao Yu, Qiang Liu, Shu Wu, Liang Wang 和 Tieniu Tan. 2024a. VLKEB：一个大型视觉-语言模型知识编辑基准。
Huang et al. (2023a) Jing Huang, Atticus Geiger, Karel D’Oosterlinck, Zhengxuan Wu, and Christopher Potts. 2023a. Rigorously assessing natural language explanations of neurons. In Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@EMNLP 2023, Singapore, December 7, 2023, pages 317–331. Association for Computational Linguistics. Huang 等人 (2023a) Jing Huang, Atticus Geiger, Karel D’Oosterlinck, Zhengxuan Wu 和 Christopher Potts. 2023a. 严格评估神经元的自然语言解释。收录于《第六届 BlackboxNLP 工作坊：分析与解释 NLP 神经网络》(BlackboxNLP@EMNLP 2023)，新加坡，2023 年 12 月 7 日，第 317–331 页。国际计算语言学协会。
Huang et al. (2024b) Jing Huang, Zhengxuan Wu, Christopher Potts, Mor Geva, and Atticus Geiger. 2024b. RAVEL: evaluating interpretability methods on disentangling language model representations. CoRR, abs/2402.17700. Huang 等人 (2024b) Jing Huang, Zhengxuan Wu, Christopher Potts, Mor Geva 和 Atticus Geiger. 2024b. RAVEL：评估解耦语言模型表示的可解释性方法。 CoRR, abs/2402.17700。
Huang et al. (2024c) Jing Huang, Diyi Yang, and Christopher Potts. 2024c. Demystifying verbatim memorization in large language models. CoRR, abs/2407.17817. Huang 等人 (2024c) Jing Huang, Diyi Yang 和 Christopher Potts. 2024c. 揭秘大语言模型中的逐字记忆。 CoRR, abs/2407.17817。
Huang et al. (2023b) Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023b. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. CoRR, abs/2311.05232. Huang 等人 (2023b) Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin 和 Ting Liu. 2023b. 大语言模型中的幻觉调查：原则、分类、挑战与开放性问题。 CoRR, abs/2311.05232。
Huang and Huang (2024) Yizheng Huang and Jimmy Huang. 2024. A survey on retrieval-augmented text generation for large language models. CoRR, abs/2404.10981. Huang 和 Huang (2024) Yizheng Huang 和 Jimmy Huang. 2024. 大语言模型检索增强文本生成的调查。 CoRR, abs/2404.10981。
Huang et al. (2023c) Yunpeng Huang, Jingwei Xu, Zixu Jiang, Junyu Lai, Zenan Li, Yuan Yao, Taolue Chen, Lijuan Yang, Zhou Xin, and Xiaoxing Ma. 2023c. Advancing transformer architecture in long-context large language models: A comprehensive survey. CoRR, abs/2311.12351. Huang 等人 (2023c) Yunpeng Huang, Jingwei Xu, Zixu Jiang, Junyu Lai, Zenan Li, Yuan Yao, Taolue Chen, Lijuan Yang, Zhou Xin 和 Xiaoxing Ma. 2023c. 推进长上下文大语言模型中的 Transformer 架构：一项全面调查。 CoRR, abs/2311.12351。
Huang et al. (2024d) Yuzhen Huang, Jinghan Zhang, Zifei Shan, and Junxian He. 2024d. Compression represents intelligence linearly. CoRR, abs/2404.09937. Huang 等人 (2024d) Yuzhen Huang, Jinghan Zhang, Zifei Shan 和 Junxian He. 2024d. 压缩以线性方式代表智能。 CoRR, abs/2404.09937。
Huh et al. (2024) Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. 2024. The platonic representation hypothesis. arXiv preprint arXiv:2405.07987. Huh 等人 (2024) Minyoung Huh, Brian Cheung, Tongzhou Wang 和 Phillip Isola. 2024. 柏拉图表征假设。 arXiv 预印本 arXiv:2405.07987。
Hyman (1999) John Hyman. 1999. How knowledge works. The philosophical quarterly, 49(197):433–451. Hyman (1999) John Hyman. 1999. 知识是如何运作的。《The Philosophical Quarterly》, 49(197):433–451。
Ilharco et al. (2023) Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. Ilharco 等人 (2023) Gabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi 和 Ali Farhadi. 2023. 用任务算术编辑模型。收录于《第十一届国际学习表征会议》(ICLR 2023)，卢旺达基加利，2023 年 5 月 1 日至 5 日。OpenReview.net。
Jamali et al. (2024) Mohsen Jamali, Benjamin Grannan, Jing Cai, Arjun R Khanna, William Muñoz, Irene Caprara, Angelique C Paulk, Sydney S Cash, Evelina Fedorenko, and Ziv M Williams. 2024. Semantic encoding during language comprehension at single-cell resolution. Nature, pages 1–7. Jamali 等人 (2024) Mohsen Jamali, Benjamin Grannan, Jing Cai, Arjun R Khanna, William Muñoz, Irene Caprara, Angelique C Paulk, Sydney S Cash, Evelina Fedorenko 和 Ziv M Williams. 2024. 语言理解过程中的单细胞分辨率语义编码。《Nature》, 第 1–7 页。
Jermyn et al. (2022) Adam S. Jermyn, Nicholas Schiefer, and Evan Hubinger. 2022. c. CoRR, abs/2211.09169. Jermyn 等人 (2022) Adam S. Jermyn, Nicholas Schiefer 和 Evan Hubinger. 2022. c。 CoRR, abs/2211.09169。
Ji et al. (2024) Jiaming Ji, Kaile Wang, Tianyi Qiu, Boyuan Chen, Jiayi Zhou, Changye Li, Hantao Lou, and Yaodong Yang. 2024. Language models resist alignment. arXiv preprint arXiv:2406.06144. Ji 等人 (2024) Jiaming Ji, Kaile Wang, Tianyi Qiu, Boyuan Chen, Jiayi Zhou, Changye Li, Hantao Lou 和 Yaodong Yang. 2024. 语言模型抵制对齐。 arXiv 预印本 arXiv:2406.06144。
Jiang et al. (2024a) Shuyang Jiang, Yusheng Liao, Ya Zhang, Yu Wang, and Yanfeng Wang. 2024a. Taia: Large language models are out-of-distribution data learners. arXiv preprint arXiv:2405.20192. Jiang 等人 (2024a) Shuyang Jiang, Yusheng Liao, Ya Zhang, Yu Wang 和 Yanfeng Wang. 2024a. Taia：大语言模型是分布外数据学习者。 arXiv 预印本 arXiv:2405.20192。
Jiang et al. (2024b) Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar, and Bryon Aragam. 2024b. Do llms dream of elephants (when told not to)? latent concept association and associative memory in transformers. CoRR. Jiang 等人 (2024b) Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar 和 Bryon Aragam. 2024b. 大语言模型会梦到大象吗（当被告知不要这样做时）？Transformer 中的潜在概念关联与联想记忆。 CoRR。
Jiao et al. (2023) Licheng Jiao, Zhongjian Huang, Xu Liu, Yuting Yang, Mengru Ma, Jiaxuan Zhao, Chao You, Biao Hou, Shuyuan Yang, Fang Liu, Wenping Ma, Lingling Li, Puhua Chen, Zhixi Feng, Xu Tang, Yuwei Guo, Xiangrong Zhang, Dou Quan, Shuang Wang, Weibin Li, Jing Bai, Yangyang Li, Ronghua Shang, and Jie Feng. 2023. Brain-inspired remote sensing interpretation: A comprehensive survey. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens., 16:2992–3033. Jiao 等人 (2023) Licheng Jiao, Zhongjian Huang, Xu Liu, Yuting Yang, Mengru Ma, Jiaxuan Zhao, Chao You, Biao Hou, Shuyuan Yang, Fang Liu, Wenping Ma, Lingling Li, Puhua Chen, Zhixi Feng, Xu Tang, Yuwei Guo, Xiangrong Zhang, Dou Quan, Shuang Wang, Weibin Li, Jing Bai, Yangyang Li, Ronghua Shang 和 Jie Feng. 2023. 脑启发遥感解译：一项全面调查。《IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens.》, 16:2992–3033。
Jin (2024) Charles Jin. 2024. Latent causal probing: A formal perspective on probing with causal models of data. arXiv preprint arXiv:2407.13765. Jin (2024) Charles Jin. 2024. 潜在因果探测：关于使用数据因果模型进行探测的形式化视角。 arXiv 预印本 arXiv:2407.13765。
(156) Charles Jin and Martin Rinard. Emergent representations of program semantics in language models trained on programs. In Forty-first International Conference on Machine Learning. (156) Charles Jin 和 Martin Rinard. 在程序训练的语言模型中程序语义的涌现表征。收录于《第四十一届国际机器学习会议》。
Jin et al. (2024a) Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. 2024a. Exploring concept depth: How large language models acquire knowledge at different layers? CoRR, abs/2404.07066. Jin 等人 (2024a) Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du 和 Yongfeng Zhang. 2024a. 探索概念深度：大语言模型如何在不同层获取知识？ CoRR, abs/2404.07066。
Jin et al. (2024b) Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. 2024b. Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models. CoRR, abs/2402.18154. Jin 等人 (2024b) Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu 和 Jun Zhao. 2024b. 切断头部即终止冲突：一种解释和缓解语言模型中知识冲突的机制。 CoRR, abs/2402.18154。
Ju et al. (2024) Tianjie Ju, Weiwei Sun, Wei Du, Xinwei Yuan, Zhaochun Ren, and Gongshen Liu. 2024. How large language models encode context knowledge? A layer-wise probing study. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, pages 8235–8246. ELRA and ICCL. Ju 等人 (2024) Tianjie Ju, Weiwei Sun, Wei Du, Xinwei Yuan, Zhaochun Ren 和 Gongshen Liu. 2024. 大语言模型如何编码上下文知识？一项逐层探测研究。收录于《2024 年计算语言学、语言资源与评估联合国际会议》(LREC/COLING 2024)，意大利都灵，2024 年 5 月 20 日至 25 日，第 8235–8246 页。ELRA 和 ICCL。
Kaddour (2023) Jean Kaddour. 2023. The minipile challenge for data-efficient language models. CoRR, abs/2304.08442. Kaddour (2023) Jean Kaddour. 2023. 数据高效语言模型的 Minipile 挑战。 CoRR, abs/2304.08442。
Kaddour et al. (2023) Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, and Robert McHardy. 2023. Challenges and applications of large language models. CoRR, abs/2307.10169. Kaddour 等人 (2023) Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu 和 Robert McHardy. 2023. 大语言模型的挑战与应用。 CoRR, abs/2307.10169。
Kandpal et al. (2023) Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 15696–15707. PMLR. Kandpal 等人 (2023) Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace 和 Colin Raffel. 2023. 大语言模型难以学习长尾知识。收录于《国际机器学习会议》(ICML 2023)，美国夏威夷檀香山，2023 年 7 月 23 日至 29 日，第 202 卷《机器学习研究论文集》，第 15696–15707 页。PMLR。
Kang and Choi (2023) Cheongwoong Kang and Jaesik Choi. 2023. Impact of co-occurrence on factual knowledge of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 7721–7735. Association for Computational Linguistics. Kang 和 Choi (2023) Cheongwoong Kang 和 Jaesik Choi. 2023. 共现对大语言模型事实知识的影响。收录于《国际计算语言学协会研究结果：EMNLP 2023》，新加坡，2023 年 12 月 6 日至 10 日，第 7721–7735 页。国际计算语言学协会。
Kang et al. (2024) Katie Kang, Eric Wallace, Claire J. Tomlin, Aviral Kumar, and Sergey Levine. 2024. Unfamiliar finetuning examples control how language models hallucinate. CoRR, abs/2403.05612. Kang 等人 (2024) Katie Kang, Eric Wallace, Claire J. Tomlin, Aviral Kumar 和 Sergey Levine. 2024. 不熟悉的微调示例控制着语言模型的幻觉方式。 CoRR, abs/2403.05612。
Karvonen et al. (2024) Adam Karvonen, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Smith, Claudio Mayrink Verdun, David Bau, and Samuel Marks. 2024. Measuring progress in dictionary learning for language model interpretability with board game models. arXiv preprint arXiv:2408.00113. Karvonen 等人 (2024) Adam Karvonen, Benjamin Wright, Can Rager, Rico Angell, Jannik Brinkmann, Logan Smith, Claudio Mayrink Verdun, David Bau 和 Samuel Marks. 2024. 通过棋盘游戏模型衡量语言模型可解释性中字典学习的进展。 arXiv 预印本 arXiv:2408.00113。
Kazemi et al. (2023) Mehran Kazemi, Sid Mittal, and Deepak Ramachandran. 2023. Understanding finetuning for factual knowledge extraction from language models. CoRR, abs/2301.11293. Kazemi 等人 (2023) Mehran Kazemi, Sid Mittal 和 Deepak Ramachandran. 2023. 理解从语言模型中提取事实知识的微调过程。 CoRR, abs/2301.11293。
Keene et al. (2010) Judith Keene, John Colvin, and Justine Sissons. 2010. Mapping student information literacy activity against bloom’s taxonomy of cognitive skills. Journal of information literacy, 4(1):6–20. Keene 等人 (2010) Judith Keene, John Colvin 和 Justine Sissons. 2010. 将学生信息素养活动与布鲁姆认知技能分类法进行映射。《Journal of Information Literacy》, 4(1):6–20。
Ko et al. (2024) Miyoung Ko, Sue Hyun Park, Joonsuk Park, and Minjoon Seo. 2024. Investigating how large language models leverage internal knowledge to perform complex reasoning. arXiv preprint arXiv:2406.19502. Ko 等人 (2024) Miyoung Ko, Sue Hyun Park, Joonsuk Park 和 Minjoon Seo. 2024. 调查大语言模型如何利用内部知识执行复杂推理。 arXiv 预印本 arXiv:2406.19502。
Kobayashi et al. (2023) Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2023. Analyzing feed-forward blocks in transformers through the lens of attention map. arXiv preprint arXiv:2302.00456. Kobayashi 等人 (2023) Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi 和 Kentaro Inui. 2023. 通过注意力图的视角分析 Transformer 中的前馈块。 arXiv 预印本 arXiv:2302.00456。
Koh et al. (2020) Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020. Concept bottleneck models. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 5338–5348. PMLR. Koh 等人 (2020) Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim 和 Percy Liang. 2020. 概念瓶颈模型。收录于《第 37 届国际机器学习会议》(ICML 2020)，虚拟会议，2020 年 7 月 13 日至 18 日，第 119 卷《机器学习研究论文集》，第 5338–5348 页。PMLR。
Kolak et al. (2006) Daniel Kolak, William Hirstein, Peter Mandik, and Jonathan Waskan. 2006. Cognitive science: An introduction to mind and brain. Routledge. Kolak 等人 (2006) Daniel Kolak, William Hirstein, Peter Mandik 和 Jonathan Waskan. 2006. 认知科学：思维与大脑导论。 Routledge。
Kolbeinsson et al. (2024) Arinbjorn Kolbeinsson, Kyle O’Brien, Tianjin Huang, Shanghua Gao, Shiwei Liu, Jonathan Richard Schwarz, Anurag Vaidya, Faisal Mahmood, Marinka Zitnik, Tianlong Chen, et al. 2024. Composable interventions for language models. arXiv preprint arXiv:2407.06483. Kolbeinsson 等人 (2024) Arinbjorn Kolbeinsson, Kyle O’Brien, Tianjin Huang, Shanghua Gao, Shiwei Liu, Jonathan Richard Schwarz, Anurag Vaidya, Faisal Mahmood, Marinka Zitnik, Tianlong Chen 等人. 2024. 语言模型的可组合干预。 arXiv 预印本 arXiv:2407.06483。
Kondratyuk et al. (2023) Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Rachel Hornung, Hartwig Adam, Hassan Akbari, Yair Alon, Vighnesh Birodkar, Yong Cheng, Ming-Chang Chiu, Josh Dillon, Irfan Essa, Agrim Gupta, Meera Hahn, Anja Hauth, David Hendon, Alonso Martinez, David Minnen, David A. Ross, Grant Schindler, Mikhail Sirotenko, Kihyuk Sohn, Krishna Somandepalli, Huisheng Wang, Jimmy Yan, Ming-Hsuan Yang, Xuan Yang, Bryan Seybold, and Lu Jiang. 2023. Videopoet: A large language model for zero-shot video generation. CoRR, abs/2312.14125. Kondratyuk 等人 (2023) Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Rachel Hornung, Hartwig Adam, Hassan Akbari, Yair Alon, Vighnesh Birodkar, Yong Cheng, Ming-Chang Chiu, Josh Dillon, Irfan Essa, Agrim Gupta, Meera Hahn, Anja Hauth, David Hendon, Alonso Martinez, David Minnen, David A. Ross, Grant Schindler, Mikhail Sirotenko, Kihyuk Sohn, Krishna Somandepalli, Huisheng Wang, Jimmy Yan, Ming-Hsuan Yang, Xuan Yang, Bryan Seybold 和 Lu Jiang. 2023. VideoPoet：一种用于零样本视频生成的大语言模型。 CoRR, abs/2312.14125。
Kong et al. (2024) Lingkai Kong, Haorui Wang, Wenhao Mu, Yuanqi Du, Yuchen Zhuang, Yifei Zhou, Yue Song, Rongzhi Zhang, Kai Wang, and Chao Zhang. 2024. Aligning large language models with representation editing: A control perspective. arXiv preprint arXiv:2406.05954. Kong 等人 (2024) Lingkai Kong, Haorui Wang, Wenhao Mu, Yuanqi Du, Yuchen Zhuang, Yifei Zhou, Yue Song, Rongzhi Zhang, Kai Wang 和 Chao Zhang. 2024. 通过表征编辑对齐大语言模型：一种控制视角。 arXiv 预印本 arXiv:2406.05954。
Kornblith et al. (2019) Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. 2019. Similarity of neural network representations revisited. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 3519–3529. PMLR. Kornblith 等人 (2019) Simon Kornblith, Mohammad Norouzi, Honglak Lee 和 Geoffrey E. Hinton. 2019. 重新审视神经网络表示的相似性。收录于《第 36 届国际机器学习会议》(ICML 2019)，美国加利福尼亚州长滩，2019 年 6 月 9 日至 15 日，第 97 卷《机器学习研究论文集》，第 3519–3529 页。PMLR。
Kortukov et al. (2024) Evgenii Kortukov, Alexander Rubinstein, Elisa Nguyen, and Seong Joon Oh. 2024. Studying large language model behaviors under realistic knowledge conflicts. CoRR, abs/2404.16032. Kortukov 等人 (2024) Evgenii Kortukov, Alexander Rubinstein, Elisa Nguyen 和 Seong Joon Oh. 2024. 研究现实知识冲突下大语言模型的行为。 CoRR, abs/2404.16032。
Kraljic and Lahav (2024) Tanya Kraljic and Michal Lahav. 2024. From prompt engineering to collaborating: A human-centered approach to AI interfaces. Interactions, 31(3):30–35. Kraljic 和 Lahav (2024) Tanya Kraljic 和 Michal Lahav. 2024. 从提示工程到协作：一种以人为中心的 AI 界面方法。《Interactions》, 31(3):30–35。
Kritharoula et al. (2023) Anastasia Kritharoula, Maria Lymperaiou, and Giorgos Stamou. 2023. Language models as knowledge bases for visual word sense disambiguation. In Joint proceedings of the 1st workshop on Knowledge Base Construction from Pre-Trained Language Models (KBC-LM) and the 2nd challenge on Language Models for Knowledge Base Construction (LM-KBC) co-located with the 22nd International Semantic Web Conference (ISWC 2023), Athens, Greece, November 6, 2023, volume 3577 of CEUR Workshop Proceedings. CEUR-WS.org. Kritharoula 等人 (2023) Anastasia Kritharoula, Maria Lymperaiou 和 Giorgos Stamou. 2023. 作为视觉词义消歧知识库的语言模型。收录于《第一届预训练语言模型知识库构建工作坊》(KBC-LM) 和第二届语言模型知识库构建挑战赛 (LM-KBC) 的联合会议论文集，与第 22 届国际语义网会议 (ISWC 2023) 同期举行，希腊雅典，2023 年 11 月 6 日，第 3577 卷《CEUR 工作坊论文集》。CEUR-WS.org。
Lai et al. (2023) Jinqi Lai, Wensheng Gan, Jiayang Wu, Zhenlian Qi, and Philip S. Yu. 2023. Large language models in law: A survey. Lai 等人 (2023) Jinqi Lai, Wensheng Gan, Jiayang Wu, Zhenlian Qi 和 Philip S. Yu. 2023. 法律领域中的大语言模型：一项调查。
Lan et al. (2024) Michael Lan, Fazl, and Barez. 2024. Interpreting shared circuits for ordered sequence prediction in a large language model. arXiv preprint arXiv:2311.04131. Lan 等人 (2024) Michael Lan, Fazl 和 Barez. 2024. 解释大语言模型中用于有序序列预测的共享电路。 arXiv 预印本 arXiv:2311.04131。
Langedijk et al. (2023) Anna Langedijk, Hosein Mohebbi, Gabriele Sarti, Willem H. Zuidema, and Jaap Jumelet. 2023. Decoderlens: Layerwise interpretation of encoder-decoder transformers. CoRR, abs/2310.03686. Langedijk 等人 (2023) Anna Langedijk, Hosein Mohebbi, Gabriele Sarti, Willem H. Zuidema 和 Jaap Jumelet. 2023. DecoderLens：编码器-解码器 Transformer 的逐层解释。 CoRR, abs/2310.03686。
LawrenceC et al. (2022) LawrenceC, Adrià Garriga-alonso, Nicholas Goldowsky Dill, ryan greenblatt, jenny, Ansh Radhakrishnan, Buck, and Nate Thomas. 2022. Causal scrubbing: A method for rigorously testing interpretability hypotheses. LawrenceC 等人 (2022) LawrenceC, Adrià Garriga-alonso, Nicholas Goldowsky Dill, ryan greenblatt, jenny, Ansh Radhakrishnan, Buck 和 Nate Thomas. 2022. 因果清理：一种严格测试可解释性假设的方法。
Lee et al. (2024) Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, and Rada Mihalcea. 2024. A mechanistic understanding of alignment algorithms: A case study on DPO and toxicity. CoRR, abs/2401.01967. Lee 等人 (2024) Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld 和 Rada Mihalcea. 2024. 对齐算法的机械理解：以 DPO 和毒性为例。 CoRR, abs/2401.01967。
Lee et al. (2023a) Dong-Ho Lee, Akshen Kadakia, Brihi Joshi, Aaron Chan, Ziyi Liu, Kiran Narahari, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, and Xiang Ren. 2023a. XMD: an end-to-end framework for interactive explanation-based debugging of NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2023, Toronto, Canada, July 10-12, 2023, pages 264–273. Association for Computational Linguistics. Lee 等人 (2023a) Dong-Ho Lee, Akshen Kadakia, Brihi Joshi, Aaron Chan, Ziyi Liu, Kiran Narahari, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara 和 Xiang Ren. 2023a. XMD：一种用于 NLP 模型交互式基于解释调试的端到端框架。收录于《第 61 届国际计算语言学协会年会：系统演示》(ACL 2023)，加拿大多伦多，2023 年 7 月 10 日至 12 日，第 264–273 页。国际计算语言学协会。
Lee et al. (2023b) Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. 2023b. RLAIF: scaling reinforcement learning from human feedback with AI feedback. CoRR, abs/2309.00267. Lee 等人 (2023b) Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune 和 Abhinav Rastogi. 2023b. RLAIF：利用 AI 反馈扩展人类反馈强化学习。 CoRR, abs/2309.00267。
Lee et al. (2006) Honglak Lee, Alexis J. Battle, Rajat Raina, and Andrew Y. Ng. 2006. Efficient sparse coding algorithms. In Advances in Neural Information Processing Systems 19, Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 4-7, 2006, pages 801–808. MIT Press. Lee 等人 (2006) Honglak Lee, Alexis J. Battle, Rajat Raina 和 Andrew Y. Ng. 2006. 高效稀疏编码算法。收录于《神经信息处理系统进展 19：第二十届神经信息处理系统年会》，加拿大不列颠哥伦比亚省温哥华，2006 年 12 月 4 日至 7 日，第 801–808 页。MIT 出版社。
Lee et al. (2022a) Jin Hyung Lee, Qin Liu, and Ehsan Dadgar-Kiani. 2022a. Solving brain circuit function and dysfunction with computational modeling and optogenetic fmri. Science, 378(6619):493–499. Lee 等人 (2022a) Jin Hyung Lee, Qin Liu 和 Ehsan Dadgar-Kiani. 2022a. 利用计算建模和光遗传学功能磁共振成像解决大脑回路功能与功能障碍。《Science》, 378(6619):493–499。
Lee et al. (2022b) Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi, and Bryan Catanzaro. 2022b. Factuality enhanced language models for open-ended text generation. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. Lee 等人 (2022b) Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi 和 Bryan Catanzaro. 2022b. 用于开放式文本生成的真实性增强语言模型。收录于《神经信息处理系统进展 35：2022 年神经信息处理系统年会》(NeurIPS 2022)，美国路易斯安那州新奥尔良，2022 年 11 月 28 日至 12 月 9 日。
Leon (2024) Florin Leon. 2024. A review of findings from neuroscience and cognitive psychology as possible inspiration for the path to artificial general intelligence. CoRR, abs/2401.10904. Leon (2024) Florin Leon. 2024. 神经科学与认知心理学发现的综述：作为通往通用人工智能路径的可能灵感。 CoRR, abs/2401.10904。
Lepori et al. (2023) Michael A. Lepori, Ellie Pavlick, and Thomas Serre. 2023. Neurosurgeon: A toolkit for subnetwork analysis. CoRR, abs/2309.00244. Lepori 等人 (2023) Michael A. Lepori, Ellie Pavlick 和 Thomas Serre. 2023. Neurosurgeon：一个用于子网络分析的工具包。 CoRR, abs/2309.00244。
Levine et al. (1993) John M Levine, Lauren B Resnick, and E Tory Higgins. 1993. Social foundations of cognition. Annual review of psychology, 44(1):585–612. Levine 等人 (1993) John M Levine, Lauren B Resnick 和 E Tory Higgins. 1993. 认知的社会基础。《Annual Review of Psychology》, 44(1):585–612。
Li et al. (2024a) Aaron J Li, Satyapriya Krishna, and Himabindu Lakkaraju. 2024a. More rlhf, more trust? on the impact of human preference alignment on language model trustworthiness. arXiv preprint arXiv:2404.18870. Li 等人 (2024a) Aaron J Li, Satyapriya Krishna 和 Himabindu Lakkaraju. 2024a. 更多的 RLHF，更多的信任？关于人类偏好对齐对语言模型可信度影响的研究。 arXiv 预印本 arXiv:2404.18870。
Li et al. (2020) Guohui Li, Xiangyu Zhang, and Hong Yang. 2020. Complexity analysis and synchronization control of fractional-order jafari-sprott chaotic system. IEEE Access, 8:53360–53373. Li 等人 (2020) Guohui Li, Xiangyu Zhang 和 Hong Yang. 2020. 分数阶 Jafari-Sprott 混沌系统的复杂性分析与同步控制。《IEEE Access》, 8:53360–53373。
Li et al. (2023a) Kenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023a. Inference-time intervention: Eliciting truthful answers from a language model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Li 等人 (2023a) Kenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister 和 Martin Wattenberg. 2023a. 推理时干预：从语言模型中引出真实的答案。收录于《神经信息处理系统进展 36：2023 年神经信息处理系统年会》(NeurIPS 2023)，美国路易斯安那州新奥尔良，2023 年 12 月 10 日至 16 日。
Li et al. (2023b) Maximilian Li, Xander Davies, and Max Nadeau. 2023b. Circuit breaking: Removing model behaviors with targeted ablation. CoRR, abs/2309.05973. Li 等人 (2023b) Maximilian Li, Xander Davies 和 Max Nadeau. 2023b. 电路破坏：通过目标消融移除模型行为。 CoRR, abs/2309.05973。
Li et al. (2022) Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, and Qun Liu. 2022. How pre-trained language models capture factual knowledge? A causal-inspired analysis. In Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 1720–1732. Association for Computational Linguistics. Li 等人 (2022) Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang 和 Qun Liu. 2022. 预训练语言模型如何捕获事实知识？一项因果启发分析。收录于《国际计算语言学协会研究结果：ACL 2022》，爱尔兰都柏林，2022 年 5 月 22 日至 27 日，第 1720–1732 页。国际计算语言学协会。
Li et al. (2024b) Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, and Wenhu Chen. 2024b. Long-context llms struggle with long in-context learning. CoRR, abs/2404.02060. Li 等人 (2024b) Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue 和 Wenhu Chen. 2024b. 长上下文大语言模型在长上下文学习方面表现不佳。 CoRR, abs/2404.02060。
Li et al. (2024c) Tianlong Li, Xiaoqing Zheng, and Xuanjing Huang. 2024c. Open the pandora’s box of llms: Jailbreaking llms through representation engineering. CoRR, abs/2401.06824. Li 等人 (2024c) Tianlong Li, Xiaoqing Zheng 和 Xuanjing Huang. 2024c. 打开大语言模型的潘多拉魔盒：通过表征工程越狱大语言模型。 CoRR, abs/2401.06824。
Li et al. (2024d) Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024d. PMET: precise model editing in a transformer. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 18564–18572. AAAI Press. Li 等人 (2024d) Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma 和 Jie Yu. 2024d. PMET：Transformer 中的精确模型编辑。收录于《第 38 届 AAAI 人工智能会议》(AAAI 2024)，第 36 届人工智能创新应用会议 (IAAI 2024)，第 14 届人工智能教育进展研讨会 (EAAI 2014)，加拿大温哥华，2024 年 2 月 20 日至 27 日，第 18564–18572 页。AAAI 出版社。
Li et al. (2023c) Xin Li, Yulin Ren, Xin Jin, Cuiling Lan, Xingrui Wang, Wenjun Zeng, Xinchao Wang, and Zhibo Chen. 2023c. Diffusion models for image restoration and enhancement - A comprehensive survey. CoRR, abs/2308.09388. Li 等人 (2023c) Xin Li, Yulin Ren, Xin Jin, Cuiling Lan, Xingrui Wang, Wenjun Zeng, Xinchao Wang 和 Zhibo Chen. 2023c. 图像修复与增强的扩散模型——一项全面调查。 CoRR, abs/2308.09388。
Li et al. (2024e) Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, and Eugene Ie. 2024e. Improving multi-agent debate with sparse communication topology. CoRR, abs/2406.11776. Li 等人 (2024e) Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li 和 Eugene Ie. 2024e. 利用稀疏通信拓扑改进多智能体辩论。 CoRR, abs/2406.11776。
Li et al. (2024f) Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin, and Chao Huang. 2024f. Urbangpt: Spatio-temporal large language models. CoRR, abs/2403.00813. Li 等人 (2024f) Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin 和 Chao Huang. 2024f. UrbanGPT：时空大语言模型。 CoRR, abs/2403.00813。
Li et al. (2023d) Zhoubo Li, Ningyu Zhang, Yunzhi Yao, Mengru Wang, Xi Chen, and Huajun Chen. 2023d. Unveiling the pitfalls of knowledge editing for large language models. CoRR, abs/2310.02129. Li 等人 (2023d) Zhoubo Li, Ningyu Zhang, Yunzhi Yao, Mengru Wang, Xi Chen 和 Huajun Chen. 2023d. 揭示大语言模型知识编辑的缺陷。 CoRR, abs/2310.02129。
Liang et al. (2023a) Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023a. Encouraging divergent thinking in large language models through multi-agent debate. CoRR, abs/2305.19118. Liang 等人 (2023a) Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu 和 Shuming Shi. 2023a. 通过多智能体辩论鼓励大语言模型中的发散思维。 CoRR, abs/2305.19118。
Liang et al. (2023b) Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. 2023b. Encouraging divergent thinking in large language models through multi-agent debate. CoRR, abs/2305.19118. Liang 等人 (2023b) Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu 和 Shuming Shi. 2023b. 通过多智能体辩论鼓励大语言模型中的发散思维。 CoRR, abs/2305.19118。
Lin et al. (2024) Zhouhan Lin, Cheng Deng, Le Zhou, Tianhang Zhang, Yi Xu, Yutong Xu, Zhongmou He, Yuanyuan Shi, Beiya Dai, Yunchong Song, Boyi Zeng, Qiyuan Chen, Tao Shi, Tianyu Huang, Yiwei Xu, Shu Wang, Luoyi Fu, Weinan Zhang, Junxian He, Chao Ma, Yunqiang Zhu, Xinbing Wang, and Chenghu Zhou. 2024. Geogalactica: A scientific large language model in geoscience. CoRR, abs/2401.00434. Lin 等人 (2024) Zhouhan Lin, Cheng Deng, Le Zhou, Tianhang Zhang, Yi Xu, Yutong Xu, Zhongmou He, Yuanyuan Shi, Beiya Dai, Yunchong Song, Boyi Zeng, Qiyuan Chen, Tao Shi, Tianyu Huang, Yiwei Xu, Shu Wang, Luoyi Fu, Weinan Zhang, Junxian He, Chao Ma, Yunqiang Zhu, Xinbing Wang 和 Chenghu Zhou. 2024. GeoGalactica：地球科学领域的科学大语言模型。 CoRR, abs/2401.00434。
Liu et al. (2023a) Hong Liu, Zhiyuan Li, David Hall, Percy Liang, and Tengyu Ma. 2023a. Sophia: A scalable stochastic second-order optimizer for language model pre-training. CoRR, abs/2305.14342. Liu 等人 (2023a) Hong Liu, Zhiyuan Li, David Hall, Percy Liang 和 Tengyu Ma. 2023a. Sophia：一种用于语言模型预训练的可扩展随机二阶优化器。 CoRR, abs/2305.14342。
Liu et al. (2024a) Huanshuo Liu, Hao Zhang, Zhijiang Guo, Kuicai Dong, Xiangyang Li, Yi Quan Lee, Cong Zhang, and Yong Liu. 2024a. Ctrla: Adaptive retrieval-augmented generation via probe-guided control. CoRR, abs/2405.18727. Liu 等人 (2024a) Huanshuo Liu, Hao Zhang, Zhijiang Guo, Kuicai Dong, Xiangyang Li, Yi Quan Lee, Cong Zhang 和 Yong Liu. 2024a. CtrlA：通过探测引导控制的自适应检索增强生成。 CoRR, abs/2405.18727。
Liu et al. (2024b) Xiaoran Liu, Qipeng Guo, Yuerong Song, Zhigeng Liu, Kai Lv, Hang Yan, Linlin Li, Qun Liu, and Xipeng Qiu. 2024b. Farewell to length extrapolation, a training-free infinite context with finite attention scope. arXiv preprint arXiv:2407.15176. Liu 等人 (2024b) Xiaoran Liu, Qipeng Guo, Yuerong Song, Zhigeng Liu, Kai Lv, Hang Yan, Linlin Li, Qun Liu 和 Xipeng Qiu. 2024b. 告别长度外推，一种具有有限注意力范围的训练无关无限上下文。 arXiv 预印本 arXiv:2407.15176。
Liu et al. (2024c) Ziming Liu, Eric Gan, and Max Tegmark. 2024c. Seeing is believing: Brain-inspired modular training for mechanistic interpretability. Entropy, 26(1):41. Liu 等人 (2024c) Ziming Liu, Eric Gan 和 Max Tegmark. 2024c. 眼见为实：用于机械可解释性的脑启发模块化训练。《Entropy》, 26(1):41。
Liu et al. (2023b) Ziming Liu, Mikail Khona, Ila R. Fiete, and Max Tegmark. 2023b. Growing brains: Co-emergence of anatomical and functional modularity in recurrent neural networks. CoRR, abs/2310.07711. Liu 等人 (2023b) Ziming Liu, Mikail Khona, Ila R. Fiete 和 Max Tegmark. 2023b. 生长的大脑：递归神经网络中解剖和功能模块化的共同涌现。 CoRR, abs/2310.07711。
Liu et al. (2024d) Ziyao Liu, Huanyi Ye, Chen Chen, and Kwok-Yan Lam. 2024d. Threats, attacks, and defenses in machine unlearning: A survey. CoRR, abs/2403.13682. Liu 等人 (2024d) Ziyao Liu, Huanyi Ye, Chen Chen 和 Kwok-Yan Lam. 2024d. 机器遗忘中的威胁、攻击与防御：一项调查。 CoRR, abs/2403.13682。
Lu et al. (2024) Xingyu Lu, Xiaonan Li, Qinyuan Cheng, Kai Ding, Xuanjing Huang, and Xipeng Qiu. 2024. Scaling laws for fact memorization of large language models. arXiv preprint arXiv:2406.15720. Lu 等人 (2024) Xingyu Lu, Xiaonan Li, Qinyuan Cheng, Kai Ding, Xuanjing Huang 和 Xipeng Qiu. 2024. 大语言模型事实记忆的缩放定律。 arXiv 预印本 arXiv:2406.15720。
Lundström et al. (2022) Daniel Lundström, Tianjian Huang, and Meisam Razaviyayn. 2022. A rigorous study of integrated gradients method and extensions to internal neuron attributions. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 14485–14508. PMLR. Lundström 等人 (2022) Daniel Lundström, Tianjian Huang 和 Meisam Razaviyayn. 2022. 关于集成梯度方法及其对内部神经元归因扩展的严谨研究。收录于《国际机器学习会议》(ICML 2022)，美国马里兰州巴尔的摩，2022 年 7 月 17 日至 23 日，第 162 卷《机器学习研究论文集》，第 14485–14508 页。PMLR。
Luo and Specia (2024) Haoyan Luo and Lucia Specia. 2024. From understanding to utilization: A survey on explainability for large language models. CoRR, abs/2401.12874. Luo 和 Specia (2024) Haoyan Luo 和 Lucia Specia. 2024. 从理解到利用：大语言模型可解释性调查。 CoRR, abs/2401.12874。
Luo et al. (2023) Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2023. Reasoning on graphs: Faithful and interpretable large language model reasoning. CoRR, abs/2310.01061. Luo 等人 (2023) Linhao Luo, Yuan-Fang Li, Gholamreza Haffari 和 Shirui Pan. 2023. 图推理：忠实且可解释的大语言模型推理。 CoRR, abs/2310.01061。
Luo et al. (2024a) Man Luo, Xin Xu, Yue Liu, Panupong Pasupat, and Mehran Kazemi. 2024a. In-context learning with retrieved demonstrations for language models: A survey. CoRR, abs/2401.11624. Luo 等人 (2024a) Man Luo, Xin Xu, Yue Liu, Panupong Pasupat 和 Mehran Kazemi. 2024a. 语言模型中带有检索演示的上下文学习：一项调查。 CoRR, abs/2401.11624。
Luo et al. (2024b) Siwen Luo, Hamish Ivison, Soyeon Caren Han, and Josiah Poon. 2024b. Local interpretations for explainable natural language processing: A survey. ACM Comput. Surv., 56(9):232:1–232:36. Luo 等人 (2024b) Siwen Luo, Hamish Ivison, Soyeon Caren Han 和 Josiah Poon. 2024b. 可解释自然语言处理的局部解释：一项调查。《ACM Comput. Surv.》, 56(9):232:1–232:36。
Lv et al. (2024) Ang Lv, Kaiyi Zhang, Yuhan Chen, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie, and Rui Yan. 2024. Interpreting key mechanisms of factual recall in transformer-based language models. CoRR, abs/2403.19521. Lv 等人 (2024) Ang Lv, Kaiyi Zhang, Yuhan Chen, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie 和 Rui Yan. 2024. 解释基于 Transformer 的语言模型中事实回忆的关键机制。 CoRR, abs/2403.19521。
Ma et al. (2024) Chenglong Ma, Yongli Ren, Pablo Castells, and Mark Sanderson. 2024. Temporal conformity-aware hawkes graph network for recommendations. In Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024, pages 3185–3194. ACM. Ma 等人 (2024) Chenglong Ma, Yongli Ren, Pablo Castells 和 Mark Sanderson. 2024. 用于推荐的临时一致性感知 Hawkes 图网络。收录于《ACM Web 会议 2024》(WWW 2024)，新加坡，2024 年 5 月 13 日至 17 日，第 3185–3194 页。ACM。
Mahmoodi et al. (2013) Ali Mahmoodi, Majid Nili Ahmadabadi, and Bahador Bahrami. 2013. The less you know, you think you know more; dunning and kruger effect in collective decision making. In Proceedings of the 35th Annual Meeting of the Cognitive Science Society, CogSci 2013, Berlin, Germany, July 31 - August 3, 2013. cognitivesciencesociety.org. Mahmoodi 等人 (2013) Ali Mahmoodi, Majid Nili Ahmadabadi 和 Bahador Bahrami. 2013. 知道得越少，越认为自己知道得多；集体决策中的邓宁-克鲁格效应。收录于《第 35 届认知科学学会年会》(CogSci 2013)，德国柏林，2013 年 7 月 31 日至 8 月 3 日。cognitivesciencesociety.org。
Mahowald et al. (2023) Kyle Mahowald, Anna A. Ivanova, Idan Asher Blank, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2023. Dissociating language and thought in large language models: a cognitive perspective. CoRR, abs/2301.06627. Mahowald 等人 (2023) Kyle Mahowald, Anna A. Ivanova, Idan Asher Blank, Nancy Kanwisher, Joshua B. Tenenbaum 和 Evelina Fedorenko. 2023. 大语言模型中语言与思维的解离：一种认知视角。 CoRR, abs/2301.06627。
Mallen et al. (2023) Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 9802–9822. Association for Computational Linguistics. Mallen 等人 (2023) Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi 和 Hannaneh Hajishirzi. 2023. 何时不应信任语言模型：调查参数化和非参数化记忆的有效性。收录于《第 61 届国际计算语言学协会年会》(第 1 卷：长论文)(ACL 2023)，加拿大多伦多，2023 年 7 月 9 日至 14 日，第 9802–9822 页。国际计算语言学协会。
Markman (2013) Arthur B Markman. 2013. Knowledge representation. Psychology Press. Markman (2013) Arthur B Markman. 2013. 知识表示。 Psychology Press。
Marks et al. (2024) Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. 2024. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. CoRR, abs/2403.19647. Marks 等人 (2024) Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau 和 Aaron Mueller. 2024. 稀疏特征电路：发现和编辑语言模型中可解释的因果图。 CoRR, abs/2403.19647。
Mazzia et al. (2023) Vittorio Mazzia, Alessandro Pedrani, Andrea Caciolai, Kay Rottmann, and Davide Bernardi. 2023. A survey on knowledge editing of neural networks. CoRR, abs/2310.19704. Mazzia 等人 (2023) Vittorio Mazzia, Alessandro Pedrani, Andrea Caciolai, Kay Rottmann 和 Davide Bernardi. 2023. 神经网络知识编辑调查。 CoRR, abs/2310.19704。
McAleese et al. (2024) Nat McAleese, Rai (Michael Pokorny), and Juan Felipe Cerón Uribe. 2024. Llm critics help catch llm bugs. McAleese 等人 (2024) Nat McAleese, Rai (Michael Pokorny) 和 Juan Felipe Cerón Uribe. 2024. LLM 评论家有助于捕捉 LLM 的错误。
McGrath et al. (2021) Thomas McGrath, Andrei Kapishnikov, Nenad Tomasev, Adam Pearce, Demis Hassabis, Been Kim, Ulrich Paquet, and Vladimir Kramnik. 2021. Acquisition of chess knowledge in alphazero. CoRR, abs/2111.09259. McGrath 等人 (2021) Thomas McGrath, Andrei Kapishnikov, Nenad Tomasev, Adam Pearce, Demis Hassabis, Been Kim, Ulrich Paquet 和 Vladimir Kramnik. 2021. AlphaZero 中象棋知识的获取。 CoRR, abs/2111.09259。
McGraw and Harbison-Briggs (1990) Karen L. McGraw and Karan Harbison-Briggs. 1990. Knowledge acquisition - principles and guidelines. Prentice Hall. McGraw 和 Harbison-Briggs (1990) Karen L. McGraw 和 Karan Harbison-Briggs. 1990. 知识获取——原则与指南。 Prentice Hall。
Men et al. (2024) Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2024. Shortgpt: Layers in large language models are more redundant than you expect. CoRR, abs/2403.03853. Men 等人 (2024) Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han 和 Weipeng Chen. 2024. ShortGPT：大语言模型中的层比你预想的更冗余。 CoRR, abs/2403.03853。
Meng et al. (2022) Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. Meng 等人 (2022) Kevin Meng, David Bau, Alex Andonian 和 Yonatan Belinkov. 2022. 定位和编辑 GPT 中的事实关联。收录于《神经信息处理系统进展 35：2022 年神经信息处理系统年会》(NeurIPS 2022)，美国路易斯安那州新奥尔良，2022 年 11 月 28 日至 12 月 9 日。
Meng et al. (2024) Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. CoRR, abs/2405.14734. Meng 等人 (2024) Yu Meng, Mengzhou Xia 和 Danqi Chen. 2024. SimPO：具有无参考奖励的简单偏好优化。 CoRR, abs/2405.14734。
Merrill et al. (2023) William Merrill, Nikolaos Tsilivis, and Aman Shukla. 2023. A tale of two circuits: Grokking as competition of sparse and dense subnetworks. CoRR, abs/2303.11873. Merrill 等人 (2023) William Merrill, Nikolaos Tsilivis 和 Aman Shukla. 2023. 两个电路的故事：作为稀疏和密集子网络竞争的“顿悟”现象。 CoRR, abs/2303.11873。
Merullo et al. (2023a) Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. 2023a. Circuit component reuse across tasks in transformer language models. CoRR, abs/2310.08744. Merullo 等人 (2023a) Jack Merullo, Carsten Eickhoff 和 Ellie Pavlick. 2023a. Transformer 语言模型中跨任务的电路组件重用。 CoRR, abs/2310.08744。
Merullo et al. (2023b) Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. 2023b. A mechanism for solving relational tasks in transformer language models. Merullo 等人 (2023b) Jack Merullo, Carsten Eickhoff 和 Ellie Pavlick. 2023b. 一种在 Transformer 语言模型中解决关系任务的机制。
Miller and Neo (2024) Joseph Miller and Clement Neo. 2024. We found an neuron in gpt-2. Miller 和 Neo (2024) Joseph Miller 和 Clement Neo. 2024. 我们在 GPT-2 中发现了一个神经元。
Mitchell et al. (2022) Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2022. Fast model editing at scale. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. Mitchell 等人 (2022) Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn 和 Christopher D. Manning. 2022. 大规模快速模型编辑。收录于《第十届国际学习表征会议》(ICLR 2022)，虚拟会议，2022 年 4 月 25 日至 29 日。OpenReview.net。
Momeni et al. (2024) Ali Momeni, Babak Rahmani, Benjamin Scellier, Logan G. Wright, Peter L. McMahon, Clara C. Wanjura, Yuhang Li, Anas Skalli, Natalia G. Berloff, Tatsuhiro Onodera, Ilker Oguz, Francesco Morichetti, Philipp del Hougne, Manuel Le Gallo, Abu Sebastian, Azalia Mirhoseini, Cheng Zhang, Danijela Marković, Daniel Brunner, Christophe Moser, Sylvain Gigan, Florian Marquardt, Aydogan Ozcan, Julie Grollier, Andrea J. Liu, Demetri Psaltis, Andrea Alù, and Romain Fleury. 2024. Training of physical neural networks. arXiv preprint arXiv:2406.03372. Momeni 等人 (2024) Ali Momeni, Babak Rahmani, Benjamin Scellier, Logan G. Wright, Peter L. McMahon, Clara C. Wanjura, Yuhang Li, Anas Skalli, Natalia G. Berloff, Tatsuhiro Onodera, Ilker Oguz, Francesco Morichetti, Philipp del Hougne, Manuel Le Gallo, Abu Sebastian, Azalia Mirhoseini, Cheng Zhang, Danijela Marković, Daniel Brunner, Christophe Moser, Sylvain Gigan, Florian Marquardt, Aydogan Ozcan, Julie Grollier, Andrea J. Liu, Demetri Psaltis, Andrea Alù 和 Romain Fleury. 2024. 物理神经网络的训练。 arXiv 预印本 arXiv:2406.03372。
Morcos et al. (2019) Ari S. Morcos, Haonan Yu, Michela Paganini, and Yuandong Tian. 2019. One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 4933–4943. Morcos 等人 (2019) Ari S. Morcos, Haonan Yu, Michela Paganini 和 Yuandong Tian. 2019. 一张彩票赢遍所有：在数据集和优化器之间推广彩票初始化。收录于《神经信息处理系统进展 32：2019 年神经信息处理系统年会》(NeurIPS 2019)，加拿大不列颠哥伦比亚省温哥华，2019 年 12 月 8 日至 14 日，第 4933–4943 页。
Mossing et al. (2024) Dan Mossing, Steven Bills, Henk Tillman, Tom Dupré la Tour, Nick Cammarata, Leo Gao, Joshua Achiam, Catherine Yeh, Jan Leike, Jeff Wu, and William Saunders. 2024. Transformer debugger. https://github.com/openai/transformer-debugger. Mossing 等人 (2024) Dan Mossing, Steven Bills, Henk Tillman, Tom Dupré la Tour, Nick Cammarata, Leo Gao, Joshua Achiam, Catherine Yeh, Jan Leike, Jeff Wu 和 William Saunders. 2024. Transformer 调试器。 https://github.com/openai/transformer-debugger。
Moutakanni et al. (2024) Théo Moutakanni, Piotr Bojanowski, Guillaume Chassagnon, Céline Hudelot, Armand Joulin, Yann LeCun, Matthew Muckley, Maxime Oquab, Marie-Pierre Revel, and Maria Vakalopoulou. 2024. Advancing human-centric AI for robust x-ray analysis through holistic self-supervised learning. CoRR, abs/2405.01469. Moutakanni 等人 (2024) Théo Moutakanni, Piotr Bojanowski, Guillaume Chassagnon, Céline Hudelot, Armand Joulin, Yann LeCun, Matthew Muckley, Maxime Oquab, Marie-Pierre Revel 和 Maria Vakalopoulou. 2024. 通过整体自监督学习推进用于稳健 X 射线分析的以人为本的 AI。 CoRR, abs/2405.01469。
Mukherjee and Chang (2024) Anirban Mukherjee and Hannah Hanwen Chang. 2024. AI knowledge and reasoning: Emulating expert creativity in scientific research. CoRR, abs/2404.04436. Mukherjee 和 Chang (2024) Anirban Mukherjee 和 Hannah Hanwen Chang. 2024. AI 知识与推理：在科学研究中模拟专家创造力。 CoRR, abs/2404.04436。
Mumuni et al. (2024) Alhassan Mumuni, Fuseini Mumuni, and Nana Kobina Gerrar. 2024. A survey of synthetic data augmentation methods in computer vision. CoRR, abs/2403.10075. Mumuni 等人 (2024) Alhassan Mumuni, Fuseini Mumuni 和 Nana Kobina Gerrar. 2024. 计算机视觉中合成数据增强方法的调查。 CoRR, abs/2403.10075。
Murdoch et al. (2019) W. James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. 2019. Interpretable machine learning: definitions, methods, and applications. CoRR, abs/1901.04592. Murdoch 等人 (2019) W. James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. 2019. 可解释机器学习：定义、方法与应用。 CoRR, abs/1901.04592.
Nainani (2024) Jatin Nainani. 2024. Evaluating brain-inspired modular training in automated circuit discovery for mechanistic interpretability. CoRR, abs/2401.03646. Nainani (2024) Jatin Nainani. 2024. 评估自动化电路发现中受大脑启发的模块化训练，以实现机械可解释性。 CoRR, abs/2401.03646.
Nanda (2023) Neel Nanda. 2023. How to think about activation patching. Nanda (2023) Neel Nanda. 2023. 如何思考激活修补 (activation patching)。
Nanda and Bloom (2022) Neel Nanda and Joseph Bloom. 2022. Transformerlens. https://github.com/TransformerLensOrg/TransformerLens. Nanda 和 Bloom (2022) Neel Nanda and Joseph Bloom. 2022. Transformerlens。 https://github.com/TransformerLensOrg/TransformerLens.
Naveed et al. (2023) Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, and Ajmal Mian. 2023. A comprehensive overview of large language models. CoRR, abs/2307.06435. Naveed 等人 (2023) Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, and Ajmal Mian. 2023. 大型语言模型综述。 CoRR, abs/2307.06435.
Nezhurina et al. (2024) Marianna Nezhurina, Lucia Cipolina-Kun, Mehdi Cherti, and Jenia Jitsev. 2024. Alice in wonderland: Simple tasks showing complete reasoning breakdown in state-of-the-art large language models. CoRR, abs/2406.02061. Nezhurina 等人 (2024) Marianna Nezhurina, Lucia Cipolina-Kun, Mehdi Cherti, and Jenia Jitsev. 2024. 爱丽丝梦游仙境：展示最先进大型语言模型推理完全崩溃的简单任务。 CoRR, abs/2406.02061.
Nguyen et al. (2022) Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. 2022. A survey of machine unlearning. arXiv preprint arXiv:2209.02299. Nguyen 等人 (2022) Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. 2022. 机器遗忘综述。 arXiv 预印本 arXiv:2209.02299.
Ni et al. (2023) Shiwen Ni, Dingwei Chen, Chengming Li, Xiping Hu, Ruifeng Xu, and Min Yang. 2023. Forgetting before learning: Utilizing parametric arithmetic for knowledge updating in large language models. CoRR, abs/2311.08011. Ni 等人 (2023) Shiwen Ni, Dingwei Chen, Chengming Li, Xiping Hu, Ruifeng Xu, and Min Yang. 2023. 先遗忘后学习：利用参数算术进行大型语言模型的知识更新。 CoRR, abs/2311.08011.
Normoyle et al. (2024) Aline Normoyle, João Sedoc, and Funda Durupinar. 2024. Using llms to animate interactive story characters with emotions and personality. In IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VR Workshops 2024, Orlando, FL, USA, March 16-21, 2024, pages 632–635. IEEE. Normoyle 等人 (2024) Aline Normoyle, João Sedoc, and Funda Durupinar. 2024. 使用 LLM 赋予交互式故事角色情感和个性。载于：IEEE 虚拟现实与 3D 用户界面摘要与研讨会，VR Workshops 2024，美国佛罗里达州奥兰多，2024 年 3 月 16-21 日，第 632–635 页。IEEE。
nostalgebraist (2020) nostalgebraist. 2020. interpreting gpt: the logit lens. nostalgebraist (2020) nostalgebraist. 2020. 解释 GPT：Logit Lens (Logit 透镜)。
Olah et al. (2020) Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. 2020. Zoom in: An introduction to circuits. Distill. Olah 等人 (2020) Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. 2020. 放大：电路介绍。 Distill.
Olshausen and Field (1997) Bruno A Olshausen and David J Field. 1997. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research, 37(23):3311–3325. Olshausen 和 Field (1997) Bruno A Olshausen and David J Field. 1997. 使用超完备基集的稀疏编码：V1 采用的策略？ Vision research, 37(23):3311–3325.
Olsson et al. (2022) Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2022. In-context learning and induction heads. CoRR, abs/2209.11895. Olsson 等人 (2022) Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. 2022. 语境学习与归纳头 (induction heads)。 CoRR, abs/2209.11895.
Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. Ouyang 等人 (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. 通过人类反馈训练语言模型以遵循指令。载于：神经信息处理系统进展 35：2022 年神经信息处理系统年度会议，NeurIPS 2022，美国路易斯安那州新奥尔良，2022 年 11 月 28 日至 12 月 9 日。
Ovadia et al. (2024) Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. 2024. Fine-tuning or retrieval? comparing knowledge injection in llms. Ovadia 等人 (2024) Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. 2024. 微调还是检索？比较 LLM 中的知识注入。
Pak et al. (2023) Marina A Pak, Karina A Markhieva, Mariia S Novikova, Dmitry S Petrov, Ilya S Vorobyev, Ekaterina S Maksimova, Fyodor A Kondrashov, and Dmitry N Ivankov. 2023. Using alphafold to predict the impact of single mutations on protein stability and function. Plos one, 18(3):e0282689. Pak 等人 (2023) Marina A Pak, Karina A Markhieva, Mariia S Novikova, Dmitry S Petrov, Ilya S Vorobyev, Ekaterina S Maksimova, Fyodor A Kondrashov, and Dmitry N Ivankov. 2023. 使用 AlphaFold 预测单点突变对蛋白质稳定性和功能的影响。 Plos one, 18(3):e0282689.
Pal et al. (2023) Koyena Pal, Jiuding Sun, Andrew Yuan, Byron C. Wallace, and David Bau. 2023. Future lens: Anticipating subsequent tokens from a single hidden state. In Proceedings of the 27th Conference on Computational Natural Language Learning, CoNLL 2023, Singapore, December 6-7, 2023, pages 548–560. Association for Computational Linguistics. Pal 等人 (2023) Koyena Pal, Jiuding Sun, Andrew Yuan, Byron C. Wallace, and David Bau. 2023. 未来透镜：从单个隐藏状态预测后续 Token。载于：第 27 届计算自然语言学习会议论文集，CoNLL 2023，新加坡，2023 年 12 月 6-7 日，第 548–560 页。计算语言学协会。
Pan et al. (2023a) Haowen Pan, Yixin Cao, Xiaozhi Wang, and Xun Yang. 2023a. Finding and editing multi-modal neurons in pre-trained transformer. CoRR, abs/2311.07470. Pan 等人 (2023a) Haowen Pan, Yixin Cao, Xiaozhi Wang, and Xun Yang. 2023a. 在预训练 Transformer 中查找并编辑多模态神经元。 CoRR, abs/2311.07470.
Pan et al. (2024a) Haowen Pan, Yixin Cao, Xiaozhi Wang, Xun Yang, and Meng Wang. 2024a. Finding and editing multi-modal neurons in pre-trained transformers. Pan 等人 (2024a) Haowen Pan, Yixin Cao, Xiaozhi Wang, Xun Yang, and Meng Wang. 2024a. 在预训练 Transformer 中查找并编辑多模态神经元。
Pan et al. (2023b) Jeff Z. Pan, Simon Razniewski, Jan-Christoph Kalo, Sneha Singhania, Jiaoyan Chen, Stefan Dietze, Hajira Jabeen, Janna Omeliyanenko, Wen Zhang, Matteo Lissandrini, Russa Biswas, Gerard de Melo, Angela Bonifati, Edlira Vakaj, Mauro Dragoni, and Damien Graux. 2023b. Large language models and knowledge graphs: Opportunities and challenges. Pan 等人 (2023b) Jeff Z. Pan, Simon Razniewski, Jan-Christoph Kalo, Sneha Singhania, Jiaoyan Chen, Stefan Dietze, Hajira Jabeen, Janna Omeliyanenko, Wen Zhang, Matteo Lissandrini, Russa Biswas, Gerard de Melo, Angela Bonifati, Edlira Vakaj, Mauro Dragoni, and Damien Graux. 2023b. 大型语言模型与知识图谱：机遇与挑战。
Pan et al. (2024b) Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2024b. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 36(7):3580–3599. Pan 等人 (2024b) Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2024b. 统一大型语言模型与知识图谱：路线图。 IEEE 知识与数据工程汇刊，36(7):3580–3599.
Park et al. (2023a) Joon Sung Park, Joseph C. O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023a. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST 2023, San Francisco, CA, USA, 29 October 2023- 1 November 2023, pages 2:1–2:22. ACM. Park 等人 (2023a) Joon Sung Park, Joseph C. O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023a. 生成式智能体：人类行为的交互式模拟。载于：第 36 届 ACM 用户界面软件与技术年度研讨会论文集，UIST 2023，美国加利福尼亚州旧金山，2023 年 10 月 29 日至 11 月 1 日，第 2:1–2:22 页。ACM。
Park et al. (2023b) Kiho Park, Yo Joong Choe, and Victor Veitch. 2023b. The linear representation hypothesis and the geometry of large language models. CoRR, abs/2311.03658. Park 等人 (2023b) Kiho Park, Yo Joong Choe, and Victor Veitch. 2023b. 线性表示假设与大型语言模型的几何结构。 CoRR, abs/2311.03658.
Pearce et al. (2023) Adam Pearce, Asma Ghandeharioun, Nada Hussein, Nithum Thain, Martin Wattenberg, and Lucas Dixon. 2023. Do machine learning models memorize or generalize? Pearce 等人 (2023) Adam Pearce, Asma Ghandeharioun, Nada Hussein, Nithum Thain, Martin Wattenberg, and Lucas Dixon. 2023. 机器学习模型是记忆还是泛化？
Peng et al. (2023) Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan S. Wind, Stanislaw Wozniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, and Rui-Jie Zhu. 2023. RWKV: reinventing rnns for the transformer era. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 14048–14077. Association for Computational Linguistics. Peng 等人 (2023) Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan S. Wind, Stanislaw Wozniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, and Rui-Jie Zhu. 2023. RWKV：为 Transformer 时代重塑 RNN。载于：计算语言学协会研究结果：EMNLP 2023，新加坡，2023 年 12 月 6-10 日，第 14048–14077 页。计算语言学协会。
(269) Roger Penrose. Limitations of the discrete. (269) Roger Penrose. 离散的局限性。
Pépin et al. (2024) Antoine Bellemare Pépin, François Lespinasse, Philipp Thölke, Yann Harel, Kory Mathewson, Jay A. Olson, Yoshua Bengio, and Karim Jerbi. 2024. Divergent creativity in humans and large language models. CoRR, abs/2405.13012. Pépin 等人 (2024) Antoine Bellemare Pépin, François Lespinasse, Philipp Thölke, Yann Harel, Kory Mathewson, Jay A. Olson, Yoshua Bengio, and Karim Jerbi. 2024. 人类与大型语言模型中的发散性创造力。 CoRR, abs/2405.13012.
Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 2463–2473. Association for Computational Linguistics. Petroni 等人 (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. 语言模型作为知识库？载于：2019 年自然语言处理实证方法会议暨第 9 届自然语言处理国际联合会议论文集，EMNLP-IJCNLP 2019，中国香港，2019 年 11 月 3-7 日，第 2463–2473 页。计算语言学协会。
Phillips et al. (2021) P Jonathon Phillips, P Jonathon Phillips, Carina A Hahn, Peter C Fontana, Amy N Yates, Kristen Greene, David A Broniatowski, and Mark A Przybocki. 2021. Four principles of explainable artificial intelligence. Phillips 等人 (2021) P Jonathon Phillips, P Jonathon Phillips, Carina A Hahn, Peter C Fontana, Amy N Yates, Kristen Greene, David A Broniatowski, and Mark A Przybocki. 2021. 可解释人工智能的四个原则。
Prashanth et al. (2024) USVSN Sai Prashanth, Alvin Deng, Kyle O’Brien, Jyothir S V, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A. Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, Katherine Lee, and Naomi Saphra. 2024. Recite, reconstruct, recollect: Memorization in lms as a multifaceted phenomenon. CoRR. Prashanth 等人 (2024) USVSN Sai Prashanth, Alvin Deng, Kyle O’Brien, Jyothir S V, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A. Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, Katherine Lee, and Naomi Saphra. 2024. 背诵、重构、回忆：作为多面现象的语言模型记忆。 CoRR.
Qi et al. (2024) Penghui Qi, Xinyi Wan, Guangxing Huang, and Min Lin. 2024. Zero bubble pipeline parallelism. CoRR, abs/2401.10241. Qi 等人 (2024) Penghui Qi, Xinyi Wan, Guangxing Huang, and Min Lin. 2024. 零气泡流水线并行 (Zero bubble pipeline parallelism)。 CoRR, abs/2401.10241.
Qian et al. (2023) Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. 2023. Communicative agents for software development. CoRR, abs/2307.07924. Qian 等人 (2023) Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. 2023. 用于软件开发的交流智能体。 CoRR, abs/2307.07924.
Qian et al. (2024a) Chen Qian, Zihao Xie, Yifei Wang, Wei Liu, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2024a. Scaling large-language-model-based multi-agent collaboration. arxiv.org/abs/2406.07155. Qian 等人 (2024a) Chen Qian, Zihao Xie, Yifei Wang, Wei Liu, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2024a. 扩展基于大型语言模型的多智能体协作。 arxiv.org/abs/2406.07155.
Qian et al. (2024b) Chen Qian, Jie Zhang, Wei Yao, Dongrui Liu, Zhenfei Yin, Yu Qiao, Yong Liu, and Jing Shao. 2024b. Towards tracing trustworthiness dynamics: Revisiting pre-training period of large language models. CoRR, abs/2402.19465. Qian 等人 (2024b) Chen Qian, Jie Zhang, Wei Yao, Dongrui Liu, Zhenfei Yin, Yu Qiao, Yong Liu, and Jing Shao. 2024b. 迈向追踪可信度动态：重访大型语言模型的预训练阶段。 CoRR, abs/2402.19465.
Qiao et al. (2023) Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. 2023. Reasoning with language model prompting: A survey. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 5368–5393. Association for Computational Linguistics. Qiao 等人 (2023) Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. 2023. 语言模型提示推理：一项综述。载于：第 61 届计算语言学协会年会论文集（第 1 卷：长论文），ACL 2023，加拿大多伦多，2023 年 7 月 9-14 日，第 5368–5393 页。计算语言学协会。
Qiao et al. (2024) Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Chengfei Lv, and Huajun Chen. 2024. AUTOACT: automatic agent learning from scratch via self-planning. CoRR, abs/2401.05268. Qiao 等人 (2024) Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Chengfei Lv, and Huajun Chen. 2024. AUTOACT：通过自我规划从零开始自动学习智能体。 CoRR, abs/2401.05268.
Qin et al. (2024) Jiaxin Qin, Zixuan Zhang, Chi Han, Manling Li, Pengfei Yu, and Heng Ji. 2024. Why does new knowledge create messy ripple effects in llms? CoRR, abs/2407.12828. Qin 等人 (2024) Jiaxin Qin, Zixuan Zhang, Chi Han, Manling Li, Pengfei Yu, and Heng Ji. 2024. 为什么新知识会在 LLM 中产生混乱的连锁反应？ CoRR, abs/2407.12828.
Rafailov et al. (2023) Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Rafailov 等人 (2023) Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. 直接偏好优化：你的语言模型其实是一个奖励模型。载于：神经信息处理系统进展 36：2023 年神经信息处理系统年度会议，NeurIPS 2023，美国路易斯安那州新奥尔良，2023 年 12 月 10-16 日。
Rai et al. (2024a) Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024a. A practical review of mechanistic interpretability for transformer-based language models. arXiv preprint arXiv:2407.02646. Rai 等人 (2024a) Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024a. 基于 Transformer 的语言模型的机械可解释性实用综述。 arXiv 预印本 arXiv:2407.02646.
Rai et al. (2024b) Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024b. A practical review of mechanistic interpretability for transformer-based language models. CoRR, abs/2407.02646. Rai 等人 (2024b) Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024b. 基于 Transformer 的语言模型的机械可解释性实用综述。 CoRR, abs/2407.02646.
Rai et al. (2024c) Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024c. A practical review of mechanistic interpretability for transformer-based language models. arXiv preprint arXiv:2407.02646. Rai 等人 (2024c) Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. 2024c. 基于 Transformer 的语言模型的机械可解释性实用综述。 arXiv 预印本 arXiv:2407.02646.
Raiola (2023) Ralph Raiola. 2023. Chatgpt, can you tell me a story? an exercise in challenging the true creativity of generative AI. Commun. ACM, 66(5). Raiola (2023) Ralph Raiola. 2023. ChatGPT，你能给我讲个故事吗？挑战生成式 AI 真实创造力的一项练习。 Commun. ACM, 66(5).
Raj et al. (2022) Harsh Raj, Domenic Rosati, and Subhabrata Majumdar. 2022. Measuring reliability of large language models through semantic consistency. CoRR, abs/2211.05853. Raj 等人 (2022) Harsh Raj, Domenic Rosati, and Subhabrata Majumdar. 2022. 通过语义一致性衡量大型语言模型的可靠性。 CoRR, abs/2211.05853.
Räuker et al. (2023) Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell. 2023. Toward transparent AI: A survey on interpreting the inner structures of deep neural networks. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning, SaTML 2023, Raleigh, NC, USA, February 8-10, 2023, pages 464–483. IEEE. Räuker 等人 (2023) Tilman Räuker, Anson Ho, Stephen Casper, and Dylan Hadfield-Menell. 2023. 迈向透明 AI：解释深度神经网络内部结构的综述。载于：2023 年 IEEE 安全与可信机器学习会议，SaTML 2023，美国北卡罗来纳州罗利，2023 年 2 月 8-10 日，第 464–483 页。IEEE。
Ren and Xia (2024) Jing Ren and Feng Xia. 2024. Brain-inspired artificial intelligence: A comprehensive review. arXiv preprint arXiv:2408.14811. Ren 和 Xia (2024) Jing Ren and Feng Xia. 2024. 受大脑启发的人工智能：综合综述。 arXiv 预印本 arXiv:2408.14811.
Ren et al. (2024a) Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Guanglu Wan, Xunliang Cai, and Le Sun. 2024a. Learning or self-aligning? rethinking instruction fine-tuning. Ren 等人 (2024a) Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Guanglu Wan, Xunliang Cai, and Le Sun. 2024a. 学习还是自我对齐？重新思考指令微调。
Ren et al. (2024b) Richard Ren, Steven Basart, Adam Khoja, Alice Gatti, Long Phan, Xuwang Yin, Mantas Mazeika, Alexander Pan, Gabriel Mukobi, Ryan H Kim, et al. 2024b. Safetywashing: Do ai safety benchmarks actually measure safety progress? arXiv preprint arXiv:2407.21792. Ren 等人 (2024b) Richard Ren, Steven Basart, Adam Khoja, Alice Gatti, Long Phan, Xuwang Yin, Mantas Mazeika, Alexander Pan, Gabriel Mukobi, Ryan H Kim, et al. 2024b. 安全洗白 (Safetywashing)：AI 安全基准真的能衡量安全进展吗？ arXiv 预印本 arXiv:2407.21792.
Reuel et al. (2024) Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, et al. 2024. Open problems in technical ai governance. arXiv preprint arXiv:2407.14981. Reuel 等人 (2024) Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, et al. 2024. 技术 AI 治理中的开放问题。 arXiv 预印本 arXiv:2407.14981.
Roberts et al. (2020) Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 5418–5426. Association for Computational Linguistics. Roberts 等人 (2020) Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. 你能在语言模型的参数中塞进多少知识？载于：2020 年自然语言处理实证方法会议论文集，EMNLP 2020，在线，2020 年 11 月 16-20 日，第 5418–5426 页。计算语言学协会。
Runco and Jaeger (2012) Mark A Runco and Garrett J Jaeger. 2012. The standard definition of creativity. Creativity research journal, 24(1):92–96. Runco 和 Jaeger (2012) Mark A Runco and Garrett J Jaeger. 2012. 创造力的标准定义。 Creativity research journal, 24(1):92–96.
Sæbø and Brovold (2024) Solve Sæbø and Helge Brovold. 2024. On the stochastics of human and artificial creativity. CoRR, abs/2403.06996. Sæbø 和 Brovold (2024) Solve Sæbø and Helge Brovold. 2024. 论人类与人工创造力的随机性。 CoRR, abs/2403.06996.
Sastry et al. (2024) Girish Sastry, Lennart Heim, Haydn Belfield, Markus Anderljung, Miles Brundage, Julian Hazell, Cullen O’Keefe, Gillian K. Hadfield, Richard Ngo, Konstantin Pilz, George Gor, Emma Bluemke, Sarah Shoker, Janet Egan, Robert F. Trager, Shahar Avin, Adrian Weller, Yoshua Bengio, and Diane Coyle. 2024. Computing power and the governance of artificial intelligence. CoRR, abs/2402.08797. Sastry 等人 (2024) Girish Sastry, Lennart Heim, Haydn Belfield, Markus Anderljung, Miles Brundage, Julian Hazell, Cullen O’Keefe, Gillian K. Hadfield, Richard Ngo, Konstantin Pilz, George Gor, Emma Bluemke, Sarah Shoker, Janet Egan, Robert F. Trager, Shahar Avin, Adrian Weller, Yoshua Bengio, and Diane Coyle. 2024. 计算能力与人工智能治理。 CoRR, abs/2402.08797.
Schwarzschild et al. (2024) Avi Schwarzschild, Zhili Feng, Pratyush Maini, Zachary C. Lipton, and J. Zico Kolter. 2024. Rethinking LLM memorization through the lens of adversarial compression. CoRR, abs/2404.15146. Schwarzschild 等人 (2024) Avi Schwarzschild, Zhili Feng, Pratyush Maini, Zachary C. Lipton, and J. Zico Kolter. 2024. 通过对抗压缩的视角重新思考 LLM 记忆。 CoRR, abs/2404.15146.
Schwettmann et al. (2023a) Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, and Antonio Torralba. 2023a. Multimodal neurons in pretrained text-only transformers. In IEEE/CVF International Conference on Computer Vision, ICCV 2023 - Workshops, Paris, France, October 2-6, 2023, pages 2854–2859. IEEE. Schwettmann 等人 (2023a) Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, and Antonio Torralba. 2023a. 预训练纯文本 Transformer 中的多模态神经元。载于：IEEE/CVF 国际计算机视觉会议，ICCV 2023 - 研讨会，法国巴黎，2023 年 10 月 2-6 日，第 2854–2859 页。IEEE。
Schwettmann et al. (2023b) Sarah Schwettmann, Tamar Rott Shaham, Joanna Materzynska, Neil Chowdhury, Shuang Li, Jacob Andreas, David Bau, and Antonio Torralba. 2023b. FIND: A function description benchmark for evaluating interpretability methods. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Schwettmann 等人 (2023b) Sarah Schwettmann, Tamar Rott Shaham, Joanna Materzynska, Neil Chowdhury, Shuang Li, Jacob Andreas, David Bau, and Antonio Torralba. 2023b. FIND：用于评估可解释性方法的函数描述基准。载于：神经信息处理系统进展 36：2023 年神经信息处理系统年度会议，NeurIPS 2023，美国路易斯安那州新奥尔良，2023 年 12 月 10-16 日。
Segretain et al. (2020) Rémi Segretain, Sergiu Ivanov, Laurent Trilling, and Nicolas Glade. 2020. A methodology for evaluating the extensibility of boolean networks’ structure and function. In Complex Networks & Their Applications IX - Volume 2, Proceedings of the Ninth International Conference on Complex Networks and Their Applications, COMPLEX NETWORKS 2020, 1-3 December 2020, Madrid, Spain, volume 944 of Studies in Computational Intelligence, pages 372–385. Springer. Segretain 等人 (2020) Rémi Segretain, Sergiu Ivanov, Laurent Trilling, and Nicolas Glade. 2020. 一种评估布尔网络结构和功能可扩展性的方法。载于：复杂网络及其应用 IX - 第 2 卷，第九届复杂网络及其应用国际会议论文集，COMPLEX NETWORKS 2020，2020 年 12 月 1-3 日，西班牙马德里，计算智能研究系列第 944 卷，第 372–385 页。Springer。
Shah et al. (2024) Raj Sanjay Shah, Khushi Bhardwaj, and Sashank Varma. 2024. Development of cognitive intelligence in pre-trained language models. arXiv preprint arXiv:2407.01047. Shah 等人 (2024) Raj Sanjay Shah, Khushi Bhardwaj, and Sashank Varma. 2024. 预训练语言模型中认知智能的发展。 arXiv 预印本 arXiv:2407.01047.
Shang et al. (2024) Jingbo Shang, Zai Zheng, Xiang Ying, Felix Tao, and Mindverse Team. 2024. Ai-native memory: A pathway from llms towards agi. arXiv preprint arXiv:2406.18312. Shang 等人 (2024) Jingbo Shang, Zai Zheng, Xiang Ying, Felix Tao, and Mindverse Team. 2024. AI 原生记忆：从 LLM 迈向 AGI 的路径。 arXiv 预印本 arXiv:2406.18312.
Shapira et al. (2024) Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2024. Clever hans or neural theory of mind? stress testing social reasoning in large language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 1: Long Papers, St. Julian’s, Malta, March 17-22, 2024, pages 2257–2273. Association for Computational Linguistics. Shapira 等人 (2024) Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2024. 聪明的汉斯还是神经心理理论？压力测试大型语言模型中的社会推理。载于：第 18 届计算语言学协会欧洲分会会议论文集，EACL 2024 - 第 1 卷：长论文，马耳他圣朱利安，2024 年 3 月 17-22 日，第 2257–2273 页。计算语言学协会。
Sharkey et al. (2022) Lee Sharkey, Dan Braun, , and Beren Millidge. 2022. Taking features out of superposition with sparse autoencoders. Sharkey 等人 (2022) Lee Sharkey, Dan Braun, , and Beren Millidge. 2022. 用稀疏自编码器从叠加中提取特征。
Sharma et al. (2024) Arnab Sen Sharma, David Atkinson, and David Bau. 2024. Locating and editing factual associations in mamba. CoRR, abs/2404.03646. Sharma 等人 (2024) Arnab Sen Sharma, David Atkinson, and David Bau. 2024. 定位并编辑 Mamba 中的事实关联。 CoRR, abs/2404.03646.
Shi et al. (2023) Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen-tau Yih. 2023. Trusting your evidence: Hallucinate less with context-aware decoding. CoRR, abs/2305.14739. Shi 等人 (2023) Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen-tau Yih. 2023. 信任你的证据：通过上下文感知解码减少幻觉。 CoRR, abs/2305.14739.
Shin et al. (2021) Jung-Eun Shin, Adam J. Riesselman, Aaron W. Kollasch, Conor McMahon, Elana Simon, Chris Sander, Aashish Manglik, Andrew C. Kruse, and Debora S. Marks. 2021. Protein design and variant prediction using autoregressive generative models. Nature Communications, 12(1). Shin 等人 (2021) Jung-Eun Shin, Adam J. Riesselman, Aaron W. Kollasch, Conor McMahon, Elana Simon, Chris Sander, Aashish Manglik, Andrew C. Kruse, and Debora S. Marks. 2021. 使用自回归生成模型进行蛋白质设计和变异预测。 Nature Communications, 12(1).
Shwartz-Ziv and LeCun (2024) Ravid Shwartz-Ziv and Yann LeCun. 2024. To compress or not to compress - self-supervised learning and information theory: A review. Entropy, 26(3):252. Shwartz-Ziv 和 LeCun (2024) Ravid Shwartz-Ziv and Yann LeCun. 2024. 压缩还是不压缩 - 自监督学习与信息论：综述。 Entropy, 26(3):252.
Singh et al. (2024) Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. 2024. Rethinking interpretability in the era of large language models. CoRR, abs/2402.01761. Singh 等人 (2024) Chandan Singh, Jeevana Priya Inala, Michel Galley, Rich Caruana, and Jianfeng Gao. 2024. 在大型语言模型时代重新思考可解释性。 CoRR, abs/2402.01761.
Singhal et al. (2022) Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Kumar Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Schärli, Aakanksha Chowdhery, Philip Andrew Mansfield, Blaise Agüera y Arcas, Dale R. Webster, Gregory S. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle K. Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Natarajan. 2022. Large language models encode clinical knowledge. CoRR, abs/2212.13138. Singhal 等人 (2022) Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Kumar Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Nathaneal Schärli, Aakanksha Chowdhery, Philip Andrew Mansfield, Blaise Agüera y Arcas, Dale R. Webster, Gregory S. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle K. Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Natarajan. 2022. 大型语言模型编码临床知识。 CoRR, abs/2212.13138.
Smit et al. (2024) Andries P. Smit, Nathan Grinsztajn, Paul Duckworth, Thomas D. Barrett, and Arnu Pretorius. 2024. Should we be going mad? A look at multi-agent debate strategies for llms. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net. Smit 等人 (2024) Andries P. Smit, Nathan Grinsztajn, Paul Duckworth, Thomas D. Barrett, and Arnu Pretorius. 2024. 我们应该疯狂吗？LLM 的多智能体辩论策略一瞥。载于：第 41 届国际机器学习会议，ICML 2024，奥地利维也纳，2024 年 7 月 21-27 日。OpenReview.net。
Sohl-Dickstein et al. (2015) Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 2256–2265. JMLR.org. Sohl-Dickstein 等人 (2015) Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. 使用非平衡热力学的深度无监督学习。载于：第 32 届国际机器学习会议论文集，ICML 2015，法国里尔，2015 年 7 月 6-11 日，第 37 卷 JMLR 研讨会与会议论文集，第 2256–2265 页。JMLR.org。
Soltoggio et al. (2024) Andrea Soltoggio, Eseoghene Ben-Iwhiwhu, Vladimir Braverman, Eric Eaton, Benjamin Epstein, Yunhao Ge, Lucy Halperin, Jonathan How, Laurent Itti, Michael A Jacobs, et al. 2024. A collective ai via lifelong learning and sharing at the edge. Nature Machine Intelligence, 6(3):251–264. Soltoggio 等人 (2024) Andrea Soltoggio, Eseoghene Ben-Iwhiwhu, Vladimir Braverman, Eric Eaton, Benjamin Epstein, Yunhao Ge, Lucy Halperin, Jonathan How, Laurent Itti, Michael A Jacobs, et al. 2024. 通过终身学习和边缘共享实现集体 AI。 Nature Machine Intelligence, 6(3):251–264.
Spivey and Michael (2007) Spivey and Michael. 2007. The continuity of mind. Spivey 和 Michael (2007) Spivey and Michael. 2007. 思想的连续性。
Squire et al. (2012) Larry Squire, Darwin Berg, Floyd E Bloom, Sascha Du Lac, Anirvan Ghosh, and Nicholas C Spitzer. 2012. Fundamental neuroscience. Academic press. Squire 等人 (2012) Larry Squire, Darwin Berg, Floyd E Bloom, Sascha Du Lac, Anirvan Ghosh, and Nicholas C Spitzer. 2012. 基础神经科学。 Academic press.
Stenger et al. (2024) Michael Stenger, Robert Leppich, Ian T. Foster, Samuel Kounev, and André Bauer. 2024. Evaluation is key: a survey on evaluation measures for synthetic time series. J. Big Data, 11(1):66. Stenger 等人 (2024) Michael Stenger, Robert Leppich, Ian T. Foster, Samuel Kounev, and André Bauer. 2024. 评估是关键：合成时间序列评估指标综述。 J. Big Data, 11(1):66.
Sternberg (2006) Robert J Sternberg. 2006. The nature of creativity. Creativity research journal, 18(1):87. Sternberg (2006) Robert J Sternberg. 2006. 创造力的本质。 Creativity research journal, 18(1):87.
Stolfo et al. (2023) Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. 2023. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 7035–7052. Association for Computational Linguistics. Stolfo 等人 (2023) Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. 2023. 使用因果中介分析对语言模型中算术推理的机械解释。载于：2023 年自然语言处理实证方法会议论文集，EMNLP 2023，新加坡，2023 年 12 月 6-10 日，第 7035–7052 页。计算语言学协会。
Stolfo et al. (2024) Alessandro Stolfo, Ben Wu, Wes Gurnee, Yonatan Belinkov, Xingyi Song, Mrinmaya Sachan, and Neel Nanda. 2024. Confidence regulation neurons in language models. arXiv preprint arXiv:2406.16254. Stolfo 等人 (2024) Alessandro Stolfo, Ben Wu, Wes Gurnee, Yonatan Belinkov, Xingyi Song, Mrinmaya Sachan, and Neel Nanda. 2024. 语言模型中的置信度调节神经元。 arXiv 预印本 arXiv:2406.16254.
Sucholutsky et al. (2023) Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C. Love, Erin Grant, Jascha Achterberg, Joshua B. Tenenbaum, Katherine M. Collins, Katherine L. Hermann, Kerem Oktar, Klaus Greff, Martin N. Hebart, Nori Jacoby, Qiuyi Zhang, Raja Marjieh, Robert Geirhos, Sherol Chen, Simon Kornblith, Sunayana Rane, Talia Konkle, Thomas P. O’Connell, Thomas Unterthiner, Andrew K. Lampinen, Klaus-Robert Müller, Mariya Toneva, and Thomas L. Griffiths. 2023. Getting aligned on representational alignment. CoRR, abs/2310.13018. Sucholutsky 等人 (2023) Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C. Love, Erin Grant, Jascha Achterberg, Joshua B. Tenenbaum, Katherine M. Collins, Katherine L. Hermann, Kerem Oktar, Klaus Greff, Martin N. Hebart, Nori Jacoby, Qiuyi Zhang, Raja Marjieh, Robert Geirhos, Sherol Chen, Simon Kornblith, Sunayana Rane, Talia Konkle, Thomas P. O’Connell, Thomas Unterthiner, Andrew K. Lampinen, Klaus-Robert Müller, Mariya Toneva, and Thomas L. Griffiths. 2023. 在表征对齐上达成一致。 CoRR, abs/2310.13018.
Sun et al. (2024) Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, and Carlos Guestrin. 2024. Learning to (learn at test time): Rnns with expressive hidden states. arXiv:2407.04620. Sun 等人 (2024) Yu Sun, Xinhao Li, Karan Dalal, Jiarui Xu, Arjun Vikram, Genghan Zhang, Yann Dubois, Xinlei Chen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, and Carlos Guestrin. 2024. 学习（在测试时学习）：具有表达性隐藏状态的 RNN。 arXiv:2407.04620.
Sun et al. (2023) Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. 2023. Retentive network: A successor to transformer for large language models. CoRR, abs/2307.08621. Sun 等人 (2023) Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue, Jianyong Wang, and Furu Wei. 2023. Retentive Network：大型语言模型的 Transformer 继任者。 CoRR, abs/2307.08621.
Sung et al. (2021) Mujeen Sung, Jinhyuk Lee, Sean S. Yi, Minji Jeon, Sungdong Kim, and Jaewoo Kang. 2021. Can language models be biomedical knowledge bases? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 4723–4734. Association for Computational Linguistics. Sung 等人 (2021) Mujeen Sung, Jinhyuk Lee, Sean S. Yi, Minji Jeon, Sungdong Kim, and Jaewoo Kang. 2021. 语言模型能成为生物医学知识库吗？载于：2021 年自然语言处理实证方法会议论文集，EMNLP 2021，虚拟活动/多米尼加共和国蓬塔卡纳，2021 年 11 月 7-11 日，第 4723–4734 页。计算语言学协会。
Talebirad and Nadiri (2023) Yashar Talebirad and Amirhossein Nadiri. 2023. Multi-agent collaboration: Harnessing the power of intelligent LLM agents. CoRR, abs/2306.03314. Talebirad 和 Nadiri (2023) Yashar Talebirad and Amirhossein Nadiri. 2023. 多智能体协作：利用智能 LLM 智能体的力量。 CoRR, abs/2306.03314.
Tamkin et al. (2023) Alex Tamkin, Mohammad Taufeeque, and Noah D. Goodman. 2023. Codebook features: Sparse and discrete interpretability for neural networks. CoRR, abs/2310.17230. Tamkin 等人 (2023) Alex Tamkin, Mohammad Taufeeque, and Noah D. Goodman. 2023. 码本特征：神经网络的稀疏和离散可解释性。 CoRR, abs/2310.17230.
Tang et al. (2024) Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-specific neurons: The key to multilingual capabilities in large language models. CoRR, abs/2402.16438. Tang 等人 (2024) Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. 特定语言神经元：大型语言模型中多语言能力的关键。 CoRR, abs/2402.16438.
Tian et al. (2024) Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang, Dianbo Sui, Xi Chen, Huajun Chen, and Ningyu Zhang. 2024. To forget or not? towards practical knowledge unlearning for large language models. arXiv preprint arXiv:2407.01920. Tian 等人 (2024) Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang, Dianbo Sui, Xi Chen, Huajun Chen, and Ningyu Zhang. 2024. 遗忘还是不遗忘？迈向大型语言模型的实用知识遗忘。 arXiv 预印本 arXiv:2407.01920.
Todd et al. (2023) Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. 2023. Function vectors in large language models. CoRR, abs/2310.15213. Todd 等人 (2023) Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. 2023. 大型语言模型中的函数向量。 CoRR, abs/2310.15213.
Tseng et al. (2024) Tom Tseng, Euan McLean, Kellin Pelrine, Tony T. Wang, and Adam Gleave. 2024. Can go ais be adversarially robust? arXiv preprint arXiv:2406.12843. Tseng 等人 (2024) Tom Tseng, Euan McLean, Kellin Pelrine, Tony T. Wang, and Adam Gleave. 2024. 围棋 AI 能具有对抗鲁棒性吗？ arXiv 预印本 arXiv:2406.12843.
Turner et al. (2023) Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. 2023. Activation addition: Steering language models without optimization. CoRR, abs/2308.10248. Turner 等人 (2023) Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. 2023. 激活相加：无需优化即可引导语言模型。 CoRR, abs/2308.10248.
Vashishtha et al. (2024) Vashishtha, Aniket, Kumar, Abhinav, Reddy, Abbavaram Gowtham, Balasubramanian, Vineeth N, Sharma, and Amit. 2024. Teaching transformers causal reasoning through axiomatic training. arXiv preprint arXiv:2407.07612. Vashishtha 等人 (2024) Vashishtha, Aniket, Kumar, Abhinav, Reddy, Abbavaram Gowtham, Balasubramanian, Vineeth N, Sharma, and Amit. 2024. 通过公理化训练教授 Transformer 因果推理。 arXiv 预印本 arXiv:2407.07612.
Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998–6008. Vaswani 等人 (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need (注意力机制就足够了)。载于：神经信息处理系统进展 30：2017 年神经信息处理系统年度会议，2017 年 12 月 4-9 日，美国加利福尼亚州长滩，第 5998–6008 页。
Vig et al. (2020) Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart M. Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. Vig 等人 (2020) Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart M. Shieber. 2020. 使用因果中介分析调查语言模型中的性别偏见。载于：神经信息处理系统进展 33：2020 年神经信息处理系统年度会议，NeurIPS 2020，2020 年 12 月 6-12 日，虚拟。
Vilas et al. (2024) Martina G. Vilas, Federico Adolfi, David Poeppel, and Gemma Roig. 2024. Position paper: An inner interpretability framework for ai inspired by lessons from cognitive neuroscience. CoRR, abs/2406.01352. Vilas 等人 (2024) Martina G. Vilas, Federico Adolfi, David Poeppel, and Gemma Roig. 2024. 立场论文：受认知神经科学经验启发的人工智能内部可解释性框架。 CoRR, abs/2406.01352.
Wang et al. (2024a) Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. 2024a. Grokked transformers are implicit reasoners: A mechanistic journey to the edge of generalization. CoRR, abs/2405.15071. Wang 等人 (2024a) Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. 2024a. Grokking 后的 Transformer 是隐式推理机：迈向泛化边缘的机械之旅。 CoRR, abs/2405.15071.
Wang et al. (2023a) Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, and Yue Zhang. 2023a. Survey on factuality in large language models: Knowledge, retrieval and domain-specificity. CoRR, abs/2310.07521. Wang 等人 (2023a) Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, and Yue Zhang. 2023a. 大型语言模型事实性综述：知识、检索与领域特异性。 CoRR, abs/2310.07521.
Wang et al. (2023b) Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, and Muhao Chen. 2023b. A causal view of entity bias in (large) language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 15173–15184. Association for Computational Linguistics. Wang 等人 (2023b) Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, and Muhao Chen. 2023b. （大型）语言模型中实体偏差的因果视角。载于：计算语言学协会研究结果：EMNLP 2023，新加坡，2023 年 12 月 6-10 日，第 15173–15184 页。计算语言学协会。
Wang et al. (2023c) Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023c. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. Wang 等人 (2023c) Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023c. 野外可解释性：GPT-2 small 中间接宾语识别的电路。载于：第 11 届国际学习表征会议，ICLR 2023，卢旺达基加利，2023 年 5 月 1-5 日。OpenReview.net。
Wang et al. (2024b) Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, and Huajun Chen. 2024b. Detoxifying large language models via knowledge editing. CoRR, abs/2403.14472. Wang 等人 (2024b) Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, and Huajun Chen. 2024b. 通过知识编辑对大型语言模型进行去毒化。 CoRR, abs/2403.14472.
Wang et al. (2024c) Peng Wang, Xiang Wei, Fangxu Hu, and Wenjuan Han. 2024c. Transgpt: Multi-modal generative pre-trained transformer for transportation. CoRR, abs/2402.07233. Wang 等人 (2024c) Peng Wang, Xiang Wei, Fangxu Hu, and Wenjuan Han. 2024c. TransGPT：用于交通的多模态生成式预训练 Transformer。 CoRR, abs/2402.07233.
Wang et al. (2024d) Ruoyao Wang, Graham Todd, Ziang Xiao, Xingdi Yuan, Marc-Alexandre Côté, Peter Clark, and Peter Jansen. 2024d. Can language models serve as text-based world simulators? CoRR, abs/2406.06485. Wang 等人 (2024d) Ruoyao Wang, Graham Todd, Ziang Xiao, Xingdi Yuan, Marc-Alexandre Côté, Peter Clark, and Peter Jansen. 2024d. 语言模型能充当基于文本的世界模拟器吗？ CoRR, abs/2406.06485.
Wang et al. (2024e) Shaobo Wang, Hongxuan Tang, Mingyang Wang, Hongrui Zhang, Xuyang Liu, Weiya Li, Xuming Hu, and Linfeng Zhang. 2024e. Gnothi seauton: Empowering faithful self-interpretability in black-box models. CoRR, abs/2410.21815. Wang 等人 (2024e) Shaobo Wang, Hongxuan Tang, Mingyang Wang, Hongrui Zhang, Xuyang Liu, Weiya Li, Xuming Hu, and Linfeng Zhang. 2024e. Gnothi Seauton (认识你自己)：赋能黑盒模型中忠实的自我可解释性。 CoRR, abs/2410.21815.
Wang et al. (2023d) Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. 2023d. Knowledge editing for large language models: A survey. CoRR, abs/2310.16218. Wang 等人 (2023d) Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. 2023d. 大型语言模型知识编辑：综述。 CoRR, abs/2310.16218.
Wang et al. (2024f) Xiaohan Wang, Shengyu Mao, Ningyu Zhang, Shumin Deng, Yunzhi Yao, Yue Shen, Lei Liang, Jinjie Gu, and Huajun Chen. 2024f. Editing conceptual knowledge for large language models. CoRR, abs/2403.06259. Wang 等人 (2024f) Xiaohan Wang, Shengyu Mao, Ningyu Zhang, Shumin Deng, Yunzhi Yao, Yue Shen, Lei Liang, Jinjie Gu, and Huajun Chen. 2024f. 编辑大型语言模型的概念知识。 CoRR, abs/2403.06259.
Wang et al. (2022) Yuanfei Wang, Fangwei Zhong, Jing Xu, and Yizhou Wang. 2022. Tom2c: Target-oriented multi-agent communication and cooperation with theory of mind. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. Wang 等人 (2022) Yuanfei Wang, Fangwei Zhong, Jing Xu, and Yizhou Wang. 2022. Tom2c：具有心理理论的目标导向多智能体交流与合作。载于：第 10 届国际学习表征会议，ICLR 2022，虚拟活动，2022 年 4 月 25-29 日。OpenReview.net。
Wei et al. (2023) Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does LLM safety training fail? In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Wei 等人 (2023) Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. 越狱：LLM 安全训练是如何失败的？载于：神经信息处理系统进展 36：2023 年神经信息处理系统年度会议，NeurIPS 2023，美国路易斯安那州新奥尔良，2023 年 12 月 10-16 日。
Wen et al. (2023) Yilin Wen, Zifeng Wang, and Jimeng Sun. 2023. Mindmap: Knowledge graph prompting sparks graph of thoughts in large language models. CoRR, abs/2308.09729. Wen 等人 (2023) Yilin Wen, Zifeng Wang, and Jimeng Sun. 2023. MindMap：知识图谱提示激发大型语言模型中的思维图谱。 CoRR, abs/2308.09729.
Weng (2024) Lilian Weng. 2024. Extrinsic hallucinations in llms. Weng (2024) Lilian Weng. 2024. LLM 中的外在幻觉。
West et al. (2022) Peter West, Chandra Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic knowledge distillation: from general language models to commonsense models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 4602–4625. Association for Computational Linguistics. West 等人 (2022) Peter West, Chandra Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. 符号知识蒸馏：从通用语言模型到常识模型。载于：2022 年北美计算语言学协会会议：人类语言技术论文集，NAACL 2022，美国华盛顿州西雅图，2022 年 7 月 10-15 日，第 4602–4625 页。计算语言学协会。
Wilson (2016) Leslie Owen Wilson. 2016. Anderson and krathwohl–bloom’s taxonomy revised. Wilson (2016) Leslie Owen Wilson. 2016. Anderson 和 Krathwohl–Bloom 的修订版分类法。
Wu et al. (2023a) Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. 2023a. DEPN: detecting and editing privacy neurons in pretrained language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 2875–2886. Association for Computational Linguistics. Wu 等人 (2023a) Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. 2023a. DEPN：检测并编辑预训练语言模型中的隐私神经元。载于：2023 年自然语言处理实证方法会议论文集，EMNLP 2023，新加坡，2023 年 12 月 6-10 日，第 2875–2886 页。计算语言学协会。
Wu et al. (2023b) Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. 2023b. Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks. CoRR, abs/2307.02477. Wu 等人 (2023b) Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. 2023b. 推理还是背诵？通过反事实任务探索语言模型的能力与局限性。 CoRR, abs/2307.02477.
Wu et al. (2024) Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2024. Reft: Representation finetuning for language models. CoRR, abs/2404.03592. Wu 等人 (2024) Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2024. ReFT：语言模型的表征微调。 CoRR, abs/2404.03592.
Xi et al. (2024) Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Dingwen Yang, Chenyang Liao, Xin Guo, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, and Yu-Gang Jiang. 2024. Agentgym: Evolving large language model-based agents across diverse environments. Xi 等人 (2024) Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Dingwen Yang, Chenyang Liao, Xin Guo, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, and Yu-Gang Jiang. 2024. AgentGym：在多样化环境中演化基于大型语言模型的智能体。
Xie et al. (2024) Chengxing Xie, Canyu Chen, Feiran Jia, Ziyu Ye, Kai Shu, Adel Bibi, Ziniu Hu, Philip H. S. Torr, Bernard Ghanem, and Guohao Li. 2024. Can large language model agents simulate human trust behaviors? CoRR, abs/2402.04559. Xie 等人 (2024) Chengxing Xie, Canyu Chen, Feiran Jia, Ziyu Ye, Kai Shu, Adel Bibi, Ziniu Hu, Philip H. S. Torr, Bernard Ghanem, and Guohao Li. 2024. 大型语言模型智能体能模拟人类信任行为吗？ CoRR, abs/2402.04559.
Xu et al. (2024a) Fangzhi Xu, Qiushi Sun, Kanzhi Cheng, Jun Liu, Yu Qiao, and Zhiyong Wu. 2024a. Interactive evolution: A neural-symbolic self-training framework for large language models. Xu 等人 (2024a) Fangzhi Xu, Qiushi Sun, Kanzhi Cheng, Jun Liu, Yu Qiao, and Zhiyong Wu. 2024a. 交互式演化：大型语言模型的神经符号自训练框架。
Xu et al. (2024b) Rongwu Xu, Zehan Qi, Cunxiang Wang, Hongru Wang, Yue Zhang, and Wei Xu. 2024b. Knowledge conflicts for llms: A survey. CoRR, abs/2403.08319. Xu 等人 (2024b) Rongwu Xu, Zehan Qi, Cunxiang Wang, Hongru Wang, Yue Zhang, and Wei Xu. 2024b. LLM 的知识冲突：综述。 CoRR, abs/2403.08319.
Xu et al. (2024c) Ruoxi Xu, Yingfei Sun, Mengjie Ren, Shiguang Guo, Ruotong Pan, Hongyu Lin, Le Sun, and Xianpei Han. 2024c. AI for social science and social science of AI: A survey. CoRR, abs/2401.11839. Xu 等人 (2024c) Ruoxi Xu, Yingfei Sun, Mengjie Ren, Shiguang Guo, Ruotong Pan, Hongyu Lin, Le Sun, and Xianpei Han. 2024c. AI 社会科学与 AI 社会科学：综述。 CoRR, abs/2401.11839.
Xu et al. (2024d) Ziwei Xu, Sanjay Jain, and Mohan S. Kankanhalli. 2024d. Hallucination is inevitable: An innate limitation of large language models. CoRR, abs/2401.11817. Xu 等人 (2024d) Ziwei Xu, Sanjay Jain, and Mohan S. Kankanhalli. 2024d. 幻觉是不可避免的：大型语言模型的先天局限性。 CoRR, abs/2401.11817.
Yadav et al. (2023) Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, and Mohit Bansal. 2023. Ties-merging: Resolving interference when merging models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Yadav 等人 (2023) Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, and Mohit Bansal. 2023. TIES-Merging：合并模型时解决干扰。载于：神经信息处理系统进展 36：2023 年神经信息处理系统年度会议，NeurIPS 2023，美国路易斯安那州新奥尔良，2023 年 12 月 10-16 日。
Yan et al. (2024) Jianhao Yan, Futing Wang, Yafu Li, and Yue Zhang. 2024. Potential and challenges of model editing for social debiasing. CoRR, abs/2402.13462. Yan 等人 (2024) Jianhao Yan, Futing Wang, Yafu Li, and Yue Zhang. 2024. 模型编辑用于社会去偏见的潜力与挑战。 CoRR, abs/2402.13462.
Yang et al. (2024a) Diji Yang, Jinmeng Rao, Kezhen Chen, Xiaoyuan Guo, Yawen Zhang, Jie Yang, and Yi Zhang. 2024a. IM-RAG: multi-round retrieval-augmented generation through learning inner monologues. CoRR, abs/2405.13021. Yang 等人 (2024a) Diji Yang, Jinmeng Rao, Kezhen Chen, Xiaoyuan Guo, Yawen Zhang, Jie Yang, and Yi Zhang. 2024a. IM-RAG: multi-round retrieval-augmented generation through learning inner monologues. CoRR, abs/2405.13021.
Yang et al. (2024b) Hongkang Yang, Zehao Lin, Wenjin Wang, Hao Wu, Zhiyu Li, Bo Tang, Wenqiang Wei, Jinbo Wang, Zeyun Tang, Shichao Song, Chenyang Xi, Yu Yu, Kai Chen, Feiyu Xiong, Linpeng Tang, and Weinan E. 2024b. $\text{Memory}^{3}$ : Language modeling with explicit memory. arxiv.org/abs/2407.01178. Yang 等人 (2024b) Hongkang Yang, Zehao Lin, Wenjin Wang, Hao Wu, Zhiyu Li, Bo Tang, Wenqiang Wei, Jinbo Wang, Zeyun Tang, Shichao Song, Chenyang Xi, Yu Yu, Kai Chen, Feiyu Xiong, Linpeng Tang, and Weinan E. 2024b. Memory3: Language modeling with explicit memory. arxiv.org/abs/2407.01178.
Yang et al. (2023) Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023. Fingpt: Open-source financial large language models. Yang 等人 (2023) Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023. Fingpt: Open-source financial large language models.
Yang et al. (2024c) Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. 2024c. The butterfly effect of model editing: Few edits can trigger large language models collapse. CoRR, abs/2402.09656. Yang 等人 (2024c) Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, and Xueqi Cheng. 2024c. The butterfly effect of model editing: Few edits can trigger large language models collapse. CoRR, abs/2402.09656.
Yao et al. (2023a) Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2023a. Large language model unlearning. CoRR, abs/2310.10683. Yao 等人 (2023a) Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2023a. Large language model unlearning. CoRR, abs/2310.10683.
Yao et al. (2023b) Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023b. Editing large language models: Problems, methods, and opportunities. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10222–10240, Singapore. Association for Computational Linguistics. Yao 等人 (2023b) Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023b. Editing large language models: Problems, methods, and opportunities. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10222–10240, Singapore. Association for Computational Linguistics.
Yao et al. (2024) Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. 2024. Knowledge circuits in pretrained transformers. CoRR, abs/2405.17969. Yao 等人 (2024) Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. 2024. Knowledge circuits in pretrained transformers. CoRR, abs/2405.17969.
Ye et al. (2024) Tian Ye, Zicheng Xum, Yuanzhi Li, and Zeyuan Allen-Zhu. 2024. Physics of language models: Part 2.1, grade-school math and the hidden reasoning process. arXiv preprint arXiv:2407.20311. Ye 等人 (2024) Tian Ye, Zicheng Xum, Yuanzhi Li, and Zeyuan Allen-Zhu. 2024. Physics of language models: Part 2.1, grade-school math and the hidden reasoning process. arXiv preprint arXiv:2407.20311.
Yu et al. (2024a) Huizi Yu, Lizhou Fan, Lingyao Li, Jiayan Zhou, Zihui Ma, Lu Xian, Wenyue Hua, Sijia He, Mingyu Jin, Yongfeng Zhang, Ashvin Gandhi, and Xin Ma. 2024a. Large language models in biomedical and health informatics: A bibliometric review. Yu 等人 (2024a) Huizi Yu, Lizhou Fan, Lingyao Li, Jiayan Zhou, Zihui Ma, Lu Xian, Wenyue Hua, Sijia He, Mingyu Jin, Yongfeng Zhang, Ashvin Gandhi, and Xin Ma. 2024a. Large language models in biomedical and health informatics: A bibliometric review.
Yu et al. (2023a) Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Nianyi Lin, Kaifeng Yun, Linlu Gong, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi, Hailong Jin, Jinxin Liu, Yu Gu, Yuan Yao, Ning Ding, Lei Hou, Zhiyuan Liu, Bin Xu, Jie Tang, and Juanzi Li. 2023a. Kola: Carefully benchmarking world knowledge of large language models. CoRR, abs/2306.09296. Yu 等人 (2023a) Jifan Yu, Xiaozhi Wang, Shangqing Tu, Shulin Cao, Daniel Zhang-li, Xin Lv, Hao Peng, Zijun Yao, Xiaohan Zhang, Hanming Li, Chunyang Li, Zheyuan Zhang, Yushi Bai, Yantao Liu, Amy Xin, Nianyi Lin, Kaifeng Yun, Linlu Gong, Jianhui Chen, Zhili Wu, Yunjia Qi, Weikai Li, Yong Guan, Kaisheng Zeng, Ji Qi, Hailong Jin, Jinxin Liu, Yu Gu, Yuan Yao, Ning Ding, Lei Hou, Zhiyuan Liu, Bin Xu, Jie Tang, and Juanzi Li. 2023a. Kola: Carefully benchmarking world knowledge of large language models. CoRR, abs/2306.09296.
Yu et al. (2023b) Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2023b. Language models are super mario: Absorbing abilities from homologous models as a free lunch. CoRR, abs/2311.03099. Yu 等人 (2023b) Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2023b. Language models are super mario: Absorbing abilities from homologous models as a free lunch. CoRR, abs/2311.03099.
Yu et al. (2024b) Lei Yu, Meng Cao, Jackie Chi Kit Cheung, and Yue Dong. 2024b. Mechanisms of non-factual hallucinations in language models. CoRR, abs/2403.18167. Yu 等人 (2024b) Lei Yu, Meng Cao, Jackie Chi Kit Cheung, and Yue Dong. 2024b. Mechanisms of non-factual hallucinations in language models. CoRR, abs/2403.18167.
Yu et al. (2024c) Lei Yu, Jingcheng Niu, Zining Zhu, and Gerald Penn. 2024c. Functional faithfulness in the wild: Circuit discovery with differentiable computation graph pruning. arXiv preprint arXiv:2407.03779. Yu 等人 (2024c) Lei Yu, Jingcheng Niu, Zining Zhu, and Gerald Penn. 2024c. Functional faithfulness in the wild: Circuit discovery with differentiable computation graph pruning. arXiv preprint arXiv:2407.03779.
Yu et al. (2023c) Qinan Yu, Jack Merullo, and Ellie Pavlick. 2023c. Characterizing mechanisms for factual recall in language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 9924–9959. Association for Computational Linguistics. Yu 等人 (2023c) Qinan Yu, Jack Merullo, and Ellie Pavlick. 2023c. Characterizing mechanisms for factual recall in language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 9924–9959. Association for Computational Linguistics.
Yun et al. (2021) Zeyu Yun, Yubei Chen, Bruno A. Olshausen, and Yann LeCun. 2021. Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors. In Proceedings of Deep Learning Inside Out: The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, DeeLIO@NAACL-HLT 2021, Online, June 10 2021, pages 1–10. Association for Computational Linguistics. Yun 等人 (2021) Zeyu Yun, Yubei Chen, Bruno A. Olshausen, and Yann LeCun. 2021. Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors. In Proceedings of Deep Learning Inside Out: The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, DeeLIO@NAACL-HLT 2021, Online, June 10 2021, pages 1–10. Association for Computational Linguistics.
Zagzebski (2017) Linda Zagzebski. 2017. What is knowledge? The Blackwell guide to epistemology, pages 92–116. Zagzebski (2017) Linda Zagzebski. 2017. What is knowledge? The Blackwell guide to epistemology, pages 92–116.
Zhang et al. (2017) Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. Zhang 等人 (2017) Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017. Understanding deep learning requires rethinking generalization. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Zhang and Nanda (2023) Fred Zhang and Neel Nanda. 2023. Towards best practices of activation patching in language models: Metrics and methods. CoRR, abs/2309.16042. Zhang 和 Nanda (2023) Fred Zhang and Neel Nanda. 2023. Towards best practices of activation patching in language models: Metrics and methods. CoRR, abs/2309.16042.
Zhang et al. (2023a) Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. 2023a. Exploring collaboration mechanisms for LLM agents: A social psychology view. CoRR, abs/2310.02124. Zhang 等人 (2023a) Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. 2023a. Exploring collaboration mechanisms for LLM agents: A social psychology view. CoRR, abs/2310.02124.
Zhang et al. (2023b) Kai Zhang, Bernal Jimenez Gutierrez, and Yu Su. 2023b. Aligning instruction tasks unlocks large language models as zero-shot relation extractors. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 794–812. Association for Computational Linguistics. Zhang 等人 (2023b) Kai Zhang, Bernal Jimenez Gutierrez, and Yu Su. 2023b. Aligning instruction tasks unlocks large language models as zero-shot relation extractors. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 794–812. Association for Computational Linguistics.
Zhang et al. (2024a) Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. 2024a. A comprehensive study of knowledge editing for large language models. CoRR, abs/2401.01286. Zhang 等人 (2024a) Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. 2024a. A comprehensive study of knowledge editing for large language models. CoRR, abs/2401.01286.
Zhang et al. (2023c) Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2023c. Instruction tuning for large language models: A survey. CoRR, abs/2308.10792. Zhang 等人 (2023c) Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2023c. Instruction tuning for large language models: A survey. CoRR, abs/2308.10792.
Zhang et al. (2024b) Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Yipeng Zhang, Haitao Mi, and Helen Meng. 2024b. Self-tuning: Instructing llms to effectively acquire new knowledge through self-teaching. Zhang 等人 (2024b) Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Yipeng Zhang, Haitao Mi, and Helen Meng. 2024b. Self-tuning: Instructing llms to effectively acquire new knowledge through self-teaching.
Zhang et al. (2024c) Yuji Zhang, Sha Li, Jiateng Liu, Pengfei Yu, Yi R Fung, Jing Li, Manling Li, and Heng Ji. 2024c. Knowledge overshadowing causes amalgamated hallucination in large language models. arXiv preprint arXiv:2407.08039. Zhang 等人 (2024c) Yuji Zhang, Sha Li, Jiateng Liu, Pengfei Yu, Yi R Fung, Jing Li, Manling Li, and Heng Ji. 2024c. Knowledge overshadowing causes amalgamated hallucination in large language models. arXiv preprint arXiv:2407.08039.
Zhang et al. (2019) Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: enhanced language representation with informative entities. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 1441–1451. Association for Computational Linguistics. Zhang 等人 (2019) Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: enhanced language representation with informative entities. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 1441–1451. Association for Computational Linguistics.
Zhang et al. (2023d) Zhexin Zhang, Junxiao Yang, Pei Ke, and Minlie Huang. 2023d. Defending large language models against jailbreaking attacks through goal prioritization. CoRR, abs/2311.09096. Zhang 等人 (2023d) Zhexin Zhang, Junxiao Yang, Pei Ke, and Minlie Huang. 2023d. Defending large language models against jailbreaking attacks through goal prioritization. CoRR, abs/2311.09096.
Zhang et al. (2024d) Zhongwang Zhang, Pengxiao Lin, Zhiwei Wang, Yaoyu Zhang, and Zhi-Qin John Xu. 2024d. Initialization is critical to whether transformers fit composite functions by inference or memorizing. CoRR, abs/2405.05409. Zhang 等人 (2024d) Zhongwang Zhang, Pengxiao Lin, Zhiwei Wang, Yaoyu Zhang, and Zhi-Qin John Xu. 2024d. Initialization is critical to whether transformers fit composite functions by inference or memorizing. CoRR, abs/2405.05409.
Zhao et al. (2024a) Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024a. Explainability for large language models: A survey. ACM Trans. Intell. Syst. Technol., 15(2):20:1–20:38. Zhao 等人 (2024a) Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024a. Explainability for large language models: A survey. ACM Trans. Intell. Syst. Technol., 15(2):20:1–20:38.
Zhao et al. (2023a) Jun Zhao, Zhihao Zhang, Yide Ma, Qi Zhang, Tao Gui, Luhui Gao, and Xuanjing Huang. 2023a. Unveiling A core linguistic region in large language models. CoRR, abs/2310.14928. Zhao 等人 (2023a) Jun Zhao, Zhihao Zhang, Yide Ma, Qi Zhang, Tao Gui, Luhui Gao, and Xuanjing Huang. 2023a. Unveiling A core linguistic region in large language models. CoRR, abs/2310.14928.
Zhao et al. (2024b) Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. 2024b. Retrieval-augmented generation for ai-generated content: A survey. CoRR, abs/2402.19473. Zhao 等人 (2024b) Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. 2024b. Retrieval-augmented generation for ai-generated content: A survey. CoRR, abs/2402.19473.
Zhao et al. (2023b) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023b. A survey of large language models. arXiv preprint arXiv:2303.18223. Zhao 等人 (2023b) Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023b. A survey of large language models. arXiv preprint arXiv:2303.18223.
Zheng et al. (2024) Chujie Zheng, Ziqi Wang, Heng Ji, Minlie Huang, and Nanyun Peng. 2024. Weak-to-strong extrapolation expedites alignment. CoRR, abs/2404.16792. Zheng 等人 (2024) Chujie Zheng, Ziqi Wang, Heng Ji, Minlie Huang, and Nanyun Peng. 2024. Weak-to-strong extrapolation expedites alignment. CoRR, abs/2404.16792.
Zhong et al. (2021) Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021. Factual probing is [MASK]: learning vs. learning to recall. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 5017–5033. Association for Computational Linguistics. Zhong 等人 (2021) Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021. Factual probing is [MASK]: learning vs. learning to recall. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 5017–5033. Association for Computational Linguistics.
Zhong et al. (2023) Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts, and Danqi Chen. 2023. Mquake: Assessing knowledge editing in language models via multi-hop questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 15686–15702. Association for Computational Linguistics. Zhong 等人 (2023) Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts, and Danqi Chen. 2023. Mquake: Assessing knowledge editing in language models via multi-hop questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 15686–15702. Association for Computational Linguistics.
Zhou et al. (2023a) Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023a. LIMA: less is more for alignment. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Zhou 等人 (2023a) Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023a. LIMA: less is more for alignment. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
Zhou et al. (2024) Yilun Zhou, Caiming Xiong, Silvio Savarese, and Chien-Sheng Wu. 2024. Shared imagination: Llms hallucinate alike. arXiv preprint arXiv:2407.16604. Zhou 等人 (2024) Yilun Zhou, Caiming Xiong, Silvio Savarese, and Chien-Sheng Wu. 2024. Shared imagination: Llms hallucinate alike. arXiv preprint arXiv:2407.16604.
Zhou et al. (2023b) Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. 2023b. How alignment and jailbreak work: Explain llm safety through intermediate hidden states. CoRR, abs/2309.04827. Zhou 等人 (2023b) Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. 2023b. How alignment and jailbreak work: Explain llm safety through intermediate hidden states. CoRR, abs/2309.04827.
Zhu et al. (2024) Wentao Zhu, Zhining Zhang, and Yizhou Wang. 2024. Language models represent beliefs of self and others. CoRR, abs/2402.18496. Zhu 等人 (2024) Wentao Zhu, Zhining Zhang, and Yizhou Wang. 2024. Language models represent beliefs of self and others. CoRR, abs/2402.18496.
Zhu and Li (2023) Zeyuan Allen Zhu and Yuanzhi Li. 2023. Physics of language models: Part 3.1, knowledge storage and extraction. CoRR, abs/2309.14316. Zhu 和 Li (2023) Zeyuan Allen Zhu and Yuanzhi Li. 2023. Physics of language models: Part 3.1, knowledge storage and extraction. CoRR, abs/2309.14316.
Ziegler et al. (2019) Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F. Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. CoRR, abs/1909.08593. Ziegler 等人 (2019) Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F. Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. CoRR, abs/1909.08593.
Zou et al. (2023) Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. 2023. Representation engineering: A top-down approach to AI transparency. CoRR, abs/2310.01405. Zou 等人 (2023) Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. 2023. Representation engineering: A top-down approach to AI transparency. CoRR, abs/2310.01405.
Zou et al. (2024) Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. 2024. Improving alignment and robustness with circuit breakers. CoRR, abs/2406.04313. Zou 等人 (2024) Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. 2024. Improving alignment and robustness with circuit breakers. CoRR, abs/2406.04313.

Appendix A Comparison of Methods for Knowledge Evolution附录 A 知识演化方法比较

Note that due to the page limit, §4 does not provide a detailed enumeration of various techniques and details, such as machine unlearning and knowledge augmentation. Hence, we briefly outline common methods during post-training stage in this section and illustrate their associations and differences Zhang et al. (2024a) in Fig 5.需要注意的是，受限于篇幅，第 4 节并未详细列举各种技术细节，如机器遗忘（machine unlearning）和知识增强（knowledge augmentation）。因此，本节简要概述了训练后阶段的常用方法，并在图 5 中展示了它们之间的关联与差异（Zhang 等人，2024a）。

•

Continual Learning aims to continually acquire new skills and learn new tasks while retaining previously acquired knowledge.持续学习（Continual Learning）旨在不断获取新技能和学习新任务，同时保留先前获得的知识。
•

Parameter-efficient Fine-tuning (PET) Zhang et al. (2019) only updates a minimal set of parameters instead of full fine-tuning. A promising strategy is LoRA Hu et al. (2022).参数高效微调（PET，Zhang 等人，2019）仅更新极少量的参数，而非进行全量微调。LoRA（Hu 等人，2022）是一种极具前景的策略。
•

Knowledge Augmentation is proposed to assist the model in handling unknown knowledge for LLMs Zhang et al. (2019); Han et al. (2022). RAG Huang and Huang (2024) is the is the most prevalent methods. Beside, knowledge augmentation also includes prompt engineering Gu et al. (2023); Kraljic and Lahav (2024); Liang et al. (2023b) and in-context learning Luo et al. (2024a).知识增强（Knowledge Augmentation）旨在辅助模型处理大语言模型（LLM）未知的知识（Zhang 等人，2019；Han 等人，2022）。RAG（Huang 和 Huang，2024）是目前最普遍的方法。此外，知识增强还包括提示工程（Gu 等人，2023；Kraljic 和 Lahav，2024；Liang 等人，2023b）以及上下文学习（Luo 等人，2024a）。
•

Machine Unlearning Nguyen et al. (2022); Tian et al. (2024); Liu et al. (2024d) focuses on discarding undesirable behaviors from LLMs.机器遗忘（Machine Unlearning，Nguyen 等人，2022；Tian 等人，2024；Liu 等人，2024d）专注于从大语言模型中剔除不良行为。
•

Editing, including knowledge editing Zhang et al. (2024a) and representation editing Wu et al. (2024), aims to enable quick and precise modifications to the LLMs. Usually, editing first identifies the knowledge location in LLMs and then precisely modifies model behavior through a few instances.编辑（Editing），包括知识编辑（Zhang 等人，2024a）和表示编辑（Wu 等人，2024），旨在实现对大语言模型的快速且精确的修改。通常，编辑过程首先识别大语言模型中的知识位置，然后通过少量实例精确修改模型行为。

Figure 5: Comparison of Different Methods for Knowledge Evolution.图 5：不同知识演化方法的比较。

Appendix B Universality Intelligence附录 B 通用智能

To validate the hypotheses in this paper across different architectures, we first introduce other popular model architectures in §B.1, and observe the generalizability of our hypotheses across other model architectures in §B.2. Besides, recent work further claims that models trained with different data, modalities, and objectives are converging to shared representation spaces Huh et al. (2024). Artificial and biological neural networks ¹⁰¹⁰ 10 Unless otherwise specified as biological networks, the terms models, neural networks, neural models, and machines refer to artificial neural networks. also share similar features and circuits, suggesting a universal underlying mechanism Sucholutsky et al. (2023); Chan et al. (2023); Kornblith et al. (2019). Therefore, analogous to biological taxonomy, we introduce artificial neural model family and discuss the potential universality intelligence in the future in §B.3.为了在不同架构中验证本文的假设，我们首先在 B.1 节中介绍了其他主流模型架构，并在 B.2 节中观察了这些假设在其他模型架构中的泛化能力。此外，近期研究进一步指出，使用不同数据、模态和目标训练的模型正趋向于共享表示空间（Huh 等人，2024）。人工神经网络与生物神经网络（除非另有说明，否则术语“模型”、“神经网络”、“神经模型”和“机器”均指人工神经网络）也具有相似的特征和回路，这表明存在一种普遍的潜在机制（Sucholutsky 等人，2023；Chan 等人，2023；Kornblith 等人，2019）。因此，类似于生物分类学，我们引入了“神经模型家族”的概念，并在 B.3 节中探讨了未来潜在的通用智能。

B.1 Model ArchitectureB.1 模型架构

B.1.1 TransformerB.1.1 Transformer

MLP

The Multilayer Perceptron (MLP) is a crucial component in neural networks, usually comprising multiple fully connected layers. Within the Transformer architecture, the MLP plays a vital role in applying nonlinear transformations to the input hidden states, thereby enriching the model’s capacity for expression. More precisely, every MLP block involves two linear transformations separated by a point-wise activation function $\sigma$ :多层感知机（MLP）是神经网络中的关键组件，通常由多个全连接层组成。在 Transformer 架构中，MLP 对输入的隐藏状态应用非线性变换，从而丰富了模型的表达能力。更具体地说，每个 MLP 块包含两次线性变换，中间由逐点激活函数 σ 分隔：

\displaystyle\text{MLP}^{l}\left(h^{l}\right)=\sigma\left(W_{K}^{l}h^{l}\right)W_{V}^{l},

(6)

where $\sigma$ is the point-wise activation function, typically a non-linear function such as ReLU or GELU. $W_{K}^{l}$ is the weight matrix for the first linear transformation in the $l$ -th layer, mapping the input hidden state $h^{l}$ to an intermediate representation. $W_{V}^{l}$ is the weight matrix for the second linear transformation in the $l$ -th layer, transforming the intermediate representation to the output of the MLP block.其中 σ 是逐点激活函数，通常为 ReLU 或 GELU 等非线性函数。W_K^l 是第 l 层第一次线性变换的权重矩阵，将输入隐藏状态 h^l 映射为中间表示。W_V^l 是第 l 层第二次线性变换的权重矩阵，将中间表示转换为 MLP 块的输出。

Attention注意力机制

is a mechanism in neural networks, especially in models like Transformers, that captures dependencies between different positions within a sequence. It works by transforming each input element into Query ( $Q$ ), Key ( $K$ ), and Value ( $V$ ) vectors, computing attention scores between elements, and then calculating a weighted sum of values based on these scores. Specifically, for an input sequence represented as matrix $X$ , the transformations are as follows:是神经网络（特别是 Transformer 等模型）中的一种机制，用于捕捉序列中不同位置之间的依赖关系。其工作原理是将每个输入元素转换为查询（Q）、键（K）和值（V）向量，计算元素间的注意力分数，然后根据这些分数计算值的加权和。具体而言，对于表示为矩阵 X 的输入序列，其变换如下：

$\displaystyle Q$	$\displaystyle=XW^{Q},$	(7)
$\displaystyle K$	$\displaystyle=XW^{K},$
$\displaystyle V$	$\displaystyle=XW^{V},$

where $W^{Q}$ , $W^{K}$ , and $W^{V}$ are learned projection matrices. The attention scores are computed using the scaled dot-product attention mechanism:其中 W^Q、W^K 和 W^V 是学习到的投影矩阵。注意力分数使用缩放点积注意力机制计算：

\displaystyle H=\text{Attention}(Q,K,V)=\text{Softmax}\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V,

(8)

where $d_{k}$ is the dimensionality of the Key vectors. This allows the model to focus on different parts of the sequence adaptively, making it effective for tasks like natural language processing where understanding long-range dependencies is important.其中 d_k 是键向量的维度。这使得模型能够自适应地关注序列的不同部分，从而使其在自然语言处理等需要理解长距离依赖的任务中表现出色。

Variants of TransformerTransformer 的变体

Variants of the Transformer also achieve success. For instance, RWKV Peng et al. (2023) combines the efficient parallelizable training of transformers with the efficient inference of RNNs while mitigating their limitations. TTT Sun et al. (2024) replaces the hidden state of an RNN with a machine learning model. TTT compresses context through actual gradient descent on input tokens. RetNet Sun et al. (2023) theoretically derives the connection between recurrence and attention, simultaneously achieves training parallelism, low-cost inference, and good performance.Transformer 的变体也取得了成功。例如，RWKV（Peng 等人，2023）结合了 Transformer 高效的可并行训练能力与 RNN 高效的推理能力，同时减轻了各自的局限性。TTT（Sun 等人，2024）用一个机器学习模型取代了 RNN 的隐藏状态，通过对输入 token 进行实际梯度下降来压缩上下文。RetNet（Sun 等人，2023）从理论上推导了递归与注意力之间的联系，同时实现了训练并行化、低成本推理和良好的性能。

B.1.2 SSMB.1.2 SSM

MambaMamba

introduced by Gu and Dao (2023), is a recent family of autoregressive language models based on state space models (SSMs). Mamba employs a unique architecture called MambaBlock, which replaces the attention and MLP blocks used in Transformer layers.由 Gu 和 Dao（2023）提出，是近期基于状态空间模型（SSM）的一类自回归语言模型。Mamba 采用了一种独特的架构，称为 MambaBlock，它取代了 Transformer 层中使用的注意力块和 MLP 块。

Specifically, Mamba maps a sequence of tokens $x=[x_{1},x_{2},\ldots,x_{T}]$ to a probability distribution over the next token $y$ . Each token $x_{i}$ is first embedded into a hidden state of size $d$ as $h_{i}^{(0)}$ , which is then transformed sequentially by a series of MambaBlocks. The hidden state $h_{i}^{(\ell)}$ after the $\ell$ -th MambaBlock is computed as follows:具体来说，Mamba 将 token 序列 x=[x_1, x_2, …, x_T] 映射为下一个 token y 的概率分布。每个 token x_i 首先被嵌入为大小为 d 的隐藏状态 h_i^{(0)}，然后通过一系列 MambaBlock 依次进行变换。第 l 个 MambaBlock 之后的隐藏状态 h_i^{(l)} 计算如下：

\displaystyle h_{i}^{(\ell)}=h_{i}^{(\ell-1)}+o_{i}^{(\ell)}

(9)

The output $o_{i}^{(\ell)}$ of the $\ell$ -th MambaBlock for the $i$ -th token is a combination of $s_{i}^{(\ell)}$ (from Conv and SSM operations) and $g_{i}^{(\ell)}$ (a gating mechanism):第 i 个 token 经过第 l 个 MambaBlock 的输出 o_i^{(l)} 是 s_i^{(l)}（来自卷积和 SSM 操作）与 g_i^{(l)}（门控机制）的组合：

	$\displaystyle o_{i}^{(\ell)}$	$\displaystyle=\text{MambaBlock}^{(\ell)}\left[h_{1}^{(\ell-1)},h_{2}^{(\ell-1)},\ldots,h_{i}^{(\ell-1)}\right]$		(10)
		$\displaystyle=W_{o}^{(\ell)}\left[s_{i}^{(\ell)}\otimes g_{i}^{(\ell)}\right]$		(10)

Here, $\otimes$ denotes element-wise multiplication. The calculation of $s_{i}^{(\ell)}$ is as follows:此处 ⊗ 表示逐元素乘法。s_i^{(l)} 的计算如下：

\displaystyle a_{i}^{(\ell)}=W_{a}^{(\ell)}h_{i}^{(\ell)}

(11)

		$\displaystyle c_{1}^{(\ell)},c_{2}^{(\ell)},\ldots,c_{i}^{(\ell)}=$		(12)
		$\displaystyle\text{SiLU}\left[\text{Conv1D}\left[a_{1}^{(\ell)},a_{2}^{(\ell)},\ldots,a_{i}^{(\ell)}\right]\right]$		(12)

\displaystyle s_{i}^{(\ell)}=\text{selective-SSM}\left[c_{1}^{(\ell)},c_{2}^{(\ell)},\ldots,c_{i}^{(\ell)}\right]

(13)

The operations in Equations (12) and (13) correspond to Conv and SSM operations, respectively. The gating mechanism $g_{i}^{(\ell)}$ is given by:公式 (12) 和 (13) 中的操作分别对应卷积和 SSM 操作。门控机制 g_i^{(l)} 如下：

\displaystyle g_{i}^{(\ell)}=\text{SiLU}\left[W_{g}^{(\ell)}h_{i}^{(\ell-1)}\right]

(14)

The formulas and concepts used here are adapted from Sharma et al. (2024).此处使用的公式和概念参考了 Sharma 等人（2024）。

Compared to Transformer, Mamba’s design enables more efficient parallel training and effectively captures dependencies in sequences, making it suitable for various natural language processing tasks.与 Transformer 相比，Mamba 的设计实现了更高效的并行训练，并能有效捕捉序列中的依赖关系，使其适用于各种自然语言处理任务。

B.1.3 Vision and Multi-modal ModelsB.1.3 视觉与多模态模型

In the realm of vision and multi-modal models, various architectures have emerged, each with its unique approach to tackling complex visual tasks. For example, GANs (Generative Adversarial Nets) Goodfellow et al. (2014) consist of two neural networks: a generator and a discriminator. Through adversarial learning, the generator aims to produce realistic data samples (such as images), while the discriminator attempts to distinguish between real and generated data. Diffusion Model Li et al. (2023c); Sohl-Dickstein et al. (2015) is a powerful tool for generating high-quality images and data. It simulates a diffusion process by gradually adding and removing noise to achieve data generation. ResNet (Residual Network) He et al. (2016) introduced residual learning, revolutionizing deep network training by improving efficiency and performance through skip connections. ViT (Vision Transformer) Dosovitskiy et al. (2021) integrated the Transformer architecture into vision tasks, capturing long-range dependencies by processing image patches.在视觉和多模态模型领域，出现了多种架构，每种架构在处理复杂视觉任务时都有其独特的方法。例如，GAN（生成对抗网络，Goodfellow 等人，2014）由生成器和判别器两个神经网络组成。通过对抗学习，生成器旨在产生逼真的数据样本（如图像），而判别器则试图区分真实数据与生成数据。扩散模型（Diffusion Model，Li 等人，2023c；Sohl-Dickstein 等人，2015）是生成高质量图像和数据的强大工具，它通过逐步添加和去除噪声来模拟扩散过程，从而实现数据生成。ResNet（残差网络，He 等人，2016）引入了残差学习，通过跳跃连接提高了效率和性能，彻底改变了深度网络的训练。ViT（视觉 Transformer，Dosovitskiy 等人，2021）将 Transformer 架构集成到视觉任务中，通过处理图像块来捕捉长距离依赖。

B.2 Knowledge Mechanisms in Other ArchitecturesB.2 其他架构中的知识机制

Surprisingly, similar mechanisms as those found in transformer-based LLMs have also been discovered in other architectural models. Specifically, Mamba employs the knowledge memorization mechanism similar to Transformer Sharma et al. (2024). Vision and multi-modal architectures also adopt function region (Modular Region Hypothesis) for knowledge utilization Pan et al. (2023a); Schwettmann et al. (2023a); Koh et al. (2020); Bau et al. (2017), e.g., multi-modal neuron regions are responsible for multi-modal tasks. Besides, the connections hypothesis between neurons is found in vision architecture models Olah et al. (2020). Olah et al. (2020) further suggest that different types of knowledge reuse partial components, e.g., cars and cats reuse the same neurons (Reuse Hypothesis). As for the Dynamic Intelligence Hypothesis, it inherently focuses on entire artificial neural models. Generally, neural models across various architectures, trained with different objectives on different data and modalities, are converging to a shared statistical model of reality in their representation spaces Huh et al. (2024). These neural models may tend to share similar knowledge mechanisms and imagination Zhou et al. (2024)令人惊讶的是，在其他架构模型中也发现了与 Transformer 类 LLM 相似的机制。具体而言，Mamba 采用了与 Transformer 类似的知识记忆机制（Sharma 等人，2024）。视觉和多模态架构也采用了函数区域（模块化区域假设）进行知识利用（Pan 等人，2023a；Schwettmann 等人，2023a；Koh 等人，2020；Bau 等人，2017），例如，多模态神经元区域负责处理多模态任务。此外，在视觉架构模型中也发现了神经元之间的连接假设（Olah 等人，2020）。Olah 等人（2020）进一步指出，不同类型的知识会复用部分组件，例如，汽车和猫会复用相同的神经元（复用假设）。至于动态智能假设，它本质上关注的是整个神经模型。总体而言，跨越各种架构、在不同数据和模态上以不同目标训练的神经模型，在其表示空间中正趋向于一个共享的现实统计模型（Huh 等人，2024）。这些神经模型倾向于共享相似的知识机制和想象力（Zhou 等人，2024）。

B.3 Machine and HumanB.3 机器与人类

Analogous to the Hominidae Family in biological taxonomy, artificial neural models can be regarded as Neural Model Family:类似于生物分类学中的人科（Hominidae），人工神经模型可以被视为“神经模型家族”：

•

Family: Neural Model, likened to “Hominidae”.科：神经模型，类比“人科”。
•

Genus: Transformer architecture, Mamba architecture, etc., likened to “Homo” and “Pan”.属：Transformer 架构、Mamba 架构等，类比“人属”和“黑猩猩属”。
•

Species: BERT, GPT, Llama, Mistral, Mamba, etc., likened to “Sapiens”, “Pan troglodytes”, and “Pan paniscus”.种：BERT、GPT、Llama、Mistral、Mamba 等，类比“智人”、“黑猩猩”和“倭黑猩猩”。

Metaphorically, Llama-7B, Llama-13B, Llama-70B, etc., can be viewed as the infancy, childhood, and adulthood of humans. Shah et al. (2024) further find that, regardless of model size, the developmental trajectories of PLMs consistently exhibit a window of maximal alignment with human cognitive development. Therefore, we hypothesize that artificial neural networks (machine) and biological neural networks (human) tend to converge to universality intelligence. In other words, huamn and machine share similar features and circuits.比喻地说，Llama-7B、Llama-13B、Llama-70B 等可以看作是人类的婴儿期、童年期和成年期。Shah 等人（2024）进一步发现，无论模型大小如何，预训练语言模型（PLM）的发展轨迹始终表现出与人类认知发展最大程度对齐的窗口。因此，我们假设人工神经网络（机器）和生物神经网络（人类）倾向于趋向于通用智能。换句话说，人类和机器共享相似的特征和回路。

Specifically, extensive evidences demonstrate that machine and human share the same mechanism of knowledge memorization, i.e., modular region and connection de Schotten et al. (2022). The activations of modern language models can also linearly map onto the brain responses to speech Caucheteux et al. (2023). Caucheteux et al. (2023) pioneer the explanation via predictive coding theory: while transformer-based LLMs are optimized to predict nearby words, the human brain would continuously predict a hierarchy of representations that spans multiple timescales. The above phenomenon indicates that machine and human share similar underlying mechanisms of knowledge Sucholutsky et al. (2023); Chan et al. (2023); Kornblith et al. (2019), irrespective of their specific configurations, process and comprehend information. This could be due to inbuilt inductive biases Sæbø and Brovold (2024) in neural networks or natural abstractions Chan et al. (2023) – concepts favored by the natural world that any cognitive system would naturally gravitate towards Bereska and Gavves (2024).具体而言，大量证据表明机器和人类共享相同的知识记忆机制，即模块化区域和连接（de Schotten 等人，2022）。现代语言模型的激活也可以线性映射到大脑对语音的反应上（Caucheteux 等人，2023）。Caucheteux 等人（2023）通过预测编码理论开创了这种解释：虽然基于 Transformer 的 LLM 被优化用于预测附近的单词，但人脑会持续预测跨越多个时间尺度的表示层次。上述现象表明，无论其具体配置如何，机器和人类在处理和理解信息时都共享相似的潜在知识机制（Sucholutsky 等人，2023；Chan 等人，2023；Kornblith 等人，2019）。这可能是由于神经网络中内置的归纳偏置（Sæbø 和 Brovold，2024）或自然抽象（Chan 等人，2023）——即任何认知系统都会自然趋向的、受自然界青睐的概念（Bereska 和 Gavves，2024）。

Appendix C Tools for Mechanism Analysis附录 C 机制分析工具

Numerous tools exist for interpreting knowledge mechanisms in LLMs. TransformerLens Nanda and Bloom (2022) is a library for the mechanistic interpretability using observation and intervention. TransformerLens allows users to cache, remove, or replace internal activations during model running. XMD Lee et al. (2023a) provides various forms of feedback via an intuitiveness, which enable explanations align with the user feedback. NeuroX Dalvi et al. (2023) implements various interpretation methods under a unified API then provides interpretability of LLMs. PatchScope Ghandeharioun et al. (2024) is a tool developed by Google that employs a novel model to elucidate the hidden states in the original model. Transformer Debugger Mossing et al. (2024), an interpretability tool from OpenAI, utilizes GPT-4 and sparse auto-encoders to explain language neurons. Sparse autoencoders Gao et al. (2024b) leverages sparse auto-encoders to extract interpretable features from a language model by reconstructing activations from a sparse bottleneck layer. Transcoders Dunefsky et al. (2024) decomposes model computations involving MLPs into interpretable circuits.目前存在多种用于解释 LLM 知识机制的工具。TransformerLens（Nanda 和 Bloom，2022）是一个用于通过观察和干预进行机械可解释性研究的库，允许用户在模型运行期间缓存、移除或替换内部激活。XMD（Lee 等人，2023a）通过直观性提供多种反馈形式，使解释与用户反馈保持一致。NeuroX（Dalvi 等人，2023）在统一 API 下实现了多种解释方法，从而提供 LLM 的可解释性。PatchScope（Ghandeharioun 等人，2024）是 Google 开发的一种工具，使用一种新颖的模型来阐明原始模型中的隐藏状态。Transformer Debugger（Mossing 等人，2024）是 OpenAI 的一款可解释性工具，利用 GPT-4 和稀疏自动编码器来解释语言神经元。稀疏自动编码器（Gao 等人，2024b）利用稀疏自动编码器，通过从稀疏瓶颈层重构激活来从语言模型中提取可解释特征。Transcoders（Dunefsky 等人，2024）将涉及 MLP 的模型计算分解为可解释的电路。

Knowledge Mechanisms in Large Language Models: A Survey and Perspective大语言模型中的知识机制：综述与展望