Fable is not a useful modelFable 并非一个实用的模型

The release of Anthropic’s Fable (the “safety conscious” version of Mythos) has been a roller coaster ride for many reasons. Originally released on June 9th, the US government placed export controls on Fable and Mythos on June 12.
This meant that Anthropic could not serve the model to anyone who is not a US citizen, including their own employees. The impossibility of verifying all users at the risk of severe legal action made it functionally impossible to continue serving the model so Antrhopic pulled it.Anthropic 公司发布的 Fable（Mythos 的“安全意识”版本）由于种种原因，其发布过程可谓一波三折。该模型最初于 6 月 9 日发布，但美国政府在 6 月 12 日便对其和 Mythos 实施了出口管制。 这意味着 Anthropic 不能向任何非美国公民提供该模型，甚至包括他们自己的员工。由于在面临严重法律风险的情况下无法核实所有用户的身份，继续提供该模型在功能上已不可能，因此 Anthropic 将其下架了。

After a couple of weeks of negotiations, export controls were lifted and Anthropic restored access to the model(s) (only Fable is generally available, Mythos is still available only to specific, pre-approved partners). All of this, of course, cause much drama online, and resulted in a tremendous amount of speculation and many news stories. Nonetheless, that is not what I wanted to focus my thoughts on here. Rather, I wanted to explain why I think that, at least for research-level tasks in Computer Science, Fable is not a useful model.经过几周的谈判，出口管制被解除，Anthropic 恢复了对该模型（或这些模型）的访问权限（目前仅 Fable 可供大众使用，Mythos 仍仅限于特定的预先批准合作伙伴）。当然，这一切在网上引发了极大的轰动，导致了大量的猜测和许多新闻报道。然而，这并不是我在此想探讨的重点。相反，我想解释一下为什么我认为，至少对于计算机科学的研究级任务而言，Fable 并不是一个实用的模型。

The first flop (understandable; but not really)第一次惨败（虽然可以理解，但并不完全合理）

I wrote and maintain a widely-used tool for transcript quantification from RNA-seq data called salmon. The original salmon code was written in C++11/14, and early this year, I updated and modularized it a bit (with the help of ChatGPT), and brought it up to the C++20 standard. However, my more ambitious goal, and one that I’ve had for quite some time, was to re-write salmon in rust. Note: This did eventually happen, resulting in salmon 2 (which is already beyond parity with the C++ version and is adding new features quickly, but that is for another post).我编写并维护着一款广泛使用的 RNA-seq 数据转录本定量工具，名为 salmon。最初的 salmon 代码是用 C++11/14 编写的，今年年初，在 ChatGPT 的帮助下，我对其进行了一些更新和模块化处理，并将其提升到了 C++20 标准。然而，我还有一个酝酿已久的更宏大的目标，那就是用 Rust 重写 salmon。注：这一目标最终实现了，并诞生了 salmon 2（它在功能上已经超越了 C++ 版本，并且正在快速添加新特性，但这又是另一个话题了）。

When the original Fable release dropped on June 9, my first thought was to use it to help orchestrate the salmon rewrite. Now, I’ve succesfully ported or rewritten several other libraries or programs developed in the lab from C++ into rust, mostly with the help of Opus (4.6-4.8), and those went fairly smoothly. So, I thought that this would be a great opportunity to see how Fable compared. Alas, after spending some time crafting my detailed description and porting narrative, complete with implementation instructions and testing details, I submitted the query to Fable (in “plan” mode) and it immediately flagged my query on safety concerns, rejected it, and offered to send it to Opus 4.8 instead.当 Fable 最初于 6 月 9 日发布时，我的第一个想法就是利用它来协助规划 salmon 的重写工作。此前，我已经成功地在 Opus (4.6-4.8) 的帮助下，将实验室开发的多个其他库或程序从 C++ 移植或重写为 Rust，过程相当顺利。因此，我认为这是一个测试 Fable 表现的好机会。唉，在我花费时间撰写了详细的描述和移植方案，并附上实现说明和测试细节后，我将查询提交给了 Fable（处于“计划”模式），它却立即以安全顾虑为由标记了我的查询，拒绝了请求，并提议将其转交给 Opus 4.8 处理。

Naturally, I wanted to know why the prompt was rejected. Anthropic claims that Fable uses a “classifier” to determine what prompts are rejected and which are allowed, though it’s been widely reported that this classifier is stupendously miscalibrated. Nominally, it seems, my prompt was rejected because it involves working on software that deals with RNA-seq data, and the biological terminology in the documentation and source code set off the red flags needed to trigger the prompt rejection. I tried, to no avail, to explain to Fable that I am simply asking to re-write an existing, open source, widely used, publicly available C++ software tool in rust. The rewrite itself does not involve novel “biological” research. The inital ask is purely a software task, which is supposed to be a strong suit of Fable. Yet, not only did Fable refuse the prompt, but it also refused to disclose or explain why it refused the prompt in any detail, or to help me understand how to prompt it in a way so as to avoid this issue.自然，我想知道为什么我的提示词会被拒绝。Anthropic 声称 Fable 使用一个“分类器”来确定哪些提示词被拒绝、哪些被允许，但据广泛报道，这个分类器的校准极其糟糕。名义上，我的提示词似乎是因为涉及处理 RNA-seq 数据的软件开发而被拒绝的，文档和源代码中的生物学术语触发了拒绝所需的红线。我曾试图向 Fable 解释，我只是要求将一个现有的、开源的、广泛使用的、公开的 C++ 软件工具用 Rust 重写，这并不涉及任何新颖的“生物学”研究，但这无济于事。重写本身纯粹是一项软件任务，而这理应是 Fable 的强项。然而，Fable 不仅拒绝了请求，还拒绝透露或详细解释拒绝的原因，也不愿帮助我了解如何调整提示词以避免此问题。

After perhaps 15 to 30 minutes of failed attempts at rephrasing my prompts, I gave up and simply proceeded with Opus 4.8 (which happily complied and, all said and done, did a great job with the port). However, this was my first inkling that the “classifier” used by Fable is not so much a classifier as, perhaps, a simple rejection list of terms and users, refusing to even begin to engage with subjects (or users) who have any relation to biological research (and likely, also, cybersecurity research).在尝试了大约 15 到 30 分钟的重写提示词仍未果后，我放弃了，直接转用 Opus 4.8（它欣然接受并出色地完成了移植工作）。然而，这让我第一次意识到，Fable 所使用的“分类器”与其说是一个分类器，不如说是一个简单的术语和用户拒绝列表，它拒绝与任何与生物学研究（很可能也包括网络安全研究）有关的主题（或用户）进行任何交流。

This attempt was massively disappointing. Despite the clear lack of any security risk or implication, Fable flatly refused to engage with me in work for which it should be well-suited. Taking a peek on social media, I learned that my experiences were shared by many others, including biologists who had Fable refuse to answer prompts as innocuous as “why am I?”, “what is a mitochondrion?” and …. “what should I have for dinner?”. Truly strange.这次尝试令人大失所望。尽管没有任何安全风险或暗示，Fable 还是断然拒绝协助我完成它本应胜任的工作。在社交媒体上浏览了一下后，我发现许多人也有同样的经历，包括一些生物学家，他们在使用 Fable 时，即便提出“我是谁？”、“什么是线粒体？”以及……“我晚饭该吃什么？”这样无害的问题时，也会遭到拒绝。真是太奇怪了。

The second flop (unforgivable)第二次惨败（不可原谅）

While I thought of other challenges I could pose to Fable, it disappeared, succumbing to the administration’s export controls on June 12. Thus, while I had come up with another piece of interesting work that I wanted it to take a crack at it, the model was simply unavailable. On July 1st, after coming to terms with the administration (and not providing many details on precisely what had traspired), Anthropic re-released Fable. With this re-release, they mentioned even stricter safeguards. Nonetheless, Fable was back, so, I decided to give it another go.正当我思考还能给 Fable 出什么难题时，它就因为政府在 6 月 12 日实施的出口管制而消失了。因此，尽管我构思了另一个想让它尝试的有趣工作，但模型已经无法使用了。7 月 1 日，在与政府达成协议（且未提供具体细节）后，Anthropic 重新发布了 Fable。在这次重新发布中，他们提到了更严格的保障措施。尽管如此，Fable 还是回来了，所以我决定再试一次。

the setup: Back in the end of grad school, I worked on a paper about the problem of “Parsimonious reconstruction of network evolution”, which basically asks; if we are given a network (say protein interaction network), and the series of gene duplication, creation and deletion events for the proteins involved, can we reconstruct a parsimonious history of interaction gains and losses that reconstructs the observed network? The formal problem statement appears in the paper, along with generations to multiple species, etc. Nonetheless, we discovered something interesting, if we model the problem under the parsimony criterion, there is an efficient (polynomial) dynamic program that yields a (there can be many) most parsimonious solution. In the model we adopt in the paper, a protein inherits the interaction state of its parents, and rather than having to think about interaction creation and deletion separately, we simply consider “flipping” the interaction between two proteins (i.e. if they interact and we no longer wish them to do so, we flip the interaction off; if they don’t interact and we wish them to, we flip the interaction on). However, the solutions that are generated by the dynamic program can be such that they are not physically realizable. How?背景设定：在研究生院末期，我曾研究过一篇关于“网络进化的简约重构”问题的论文，其核心问题是：如果我们给定一个网络（比如蛋白质相互作用网络），以及相关蛋白质的基因重复、产生和删除事件序列，我们能否重构出一个简约的相互作用增减历史，从而还原出观察到的网络？正式的问题陈述出现在论文中，还有针对多个物种的推广等。尽管如此，我们发现了一个有趣的现象：如果我们用简约准则来建模这个问题，就存在一个高效的（多项式时间）动态规划算法，可以得出一个（也可能有多个）最简约的解。在我们论文采用的模型中，蛋白质继承其亲本的相互作用状态，我们无需分别考虑相互作用的产生和删除，只需考虑“翻转”两个蛋白质之间的相互作用（例如，如果它们有相互作用而我们希望取消，就将相互作用翻转为关闭；如果它们没有相互作用而我们希望有，就将相互作用翻转为开启）。然而，动态规划生成的解可能在物理上是不可实现的。为什么呢？

Assume that there are two distinct proteins A and B in our gene tree, such that neither is a descendant of the other. Let’s denote by $A_l$ and $A_r$ the left and right children of A and by $B_l$ and $B_r$ the left and right children of B. Now, assume that our dynamic program identifies a parsimonious solution that involves the following set of interaction changes. It flips the interaction state between $A_l$ and B, as well as the interaction between $B_r$ and A. This particular set of flips is not physically realizable, because it assumes that $A_l$ co-exists with B (so that gene A must have duplicated before B). Yet, it also assumes that $B_r$ co-exists with A (so that gene B must have duplicated first). We call this structure a “blocking loop”. It turns out that these types of simple blocking loops (those involving 2 interaction flips and 2 tree edges) can generally be forbidden. However, one can construct arbitrarily long chains of interaction flips that are themselves realizable, but where adding one more interaction turns the whole sequence of interaction flips into a physically unrealizable loop. Take a look at Figure 2 for a graphical example of this. So, while we could come up with a dynamic program to find a parsimonious history (i.e. minimal set of flips), we could not make it do so while avoiding all blocking loops! We eventually settled on a heuristic. First, find a parsimonious history. If there are no blocking loops, we’re good to go. If there is a blocking loop, pick one edge in the loop, forbid it (set its cost in the dynamic program to infinity), and rerun the dynamic program. Keep doing this until we find a solution free of blocking loops. Those solutions with specific edges forbidden may not be parsimonious, but we show in the paper that they often are in practice and, when they aren’t, they are likely not too far from optimal.假设我们的基因树中有两个不同的蛋白质 A 和 B，且它们互非后代。设 Al 和 Ar 为 A 的左右子节点，Bl 和 Br 为 B 的左右子节点。现在，假设我们的动态规划算法确定了一个简约解，其中包括以下一组相互作用变化：翻转 Al 与 B 之间的相互作用，以及 Br 与 A 之间的相互作用。这一组翻转在物理上是不可实现的，因为它假设 Al 与 B 同时存在（这意味着基因 A 必须在 B 之前重复），但它同时也假设 Br 与 A 同时存在（这意味着基因 B 必须先重复）。我们将这种结构称为“阻塞循环”。事实证明，这类简单的阻塞循环（涉及 2 次相互作用翻转和 2 条树边）通常是可以避免的。然而，人们可以构建任意长度的相互作用翻转链，它们本身是可实现的，但增加一个相互作用就会使整个翻转序列变成物理上不可实现的循环。请看图 2 的图形示例。因此，虽然我们可以设计动态规划算法来找到一个简约历史（即最小翻转集），但我们无法在避免所有阻塞循环的同时做到这一点！最终，我们采用了一种启发式方法：首先找到一个简约历史。如果没有阻塞循环，那就大功告成；如果存在阻塞循环，就挑选循环中的一条边并将其禁止（在动态规划中将其代价设为无穷大），然后重新运行动态规划算法。不断重复此过程，直到找到一个没有阻塞循环的解。那些禁用了特定边的解可能不是最简约的，但我们在论文中证明，它们在实践中通常是简约的，即使不是，也往往非常接近最优解。

Ok, so why all of this background. Well, what we have here is a very classic example of something computer scientists deal with frequently. We have a computational problem for which we did not come up with an algorithm that guarantees we always find the optimal solution. Normally in such a situation, that’s common when you can show that the problem likely has no efficient solution (i.e. when you can prove that the problem you wish to solve is NP complete). Yet, for this problem, we had neither an optimal algorithm, nor a formal proof of the hardness of the problem. Nonetheless, the problem and formulation were interesting, and the heuristic seemed to perform rather well.好了，为什么要讲这些背景呢？因为我们这里有一个计算机科学家经常遇到的非常经典的问题。我们有一个计算问题，但我们没能找到一个能保证始终找到最优解的算法。通常情况下，当你能证明该问题可能不存在高效解（即证明该问题是 NP 完全问题）时，这种情况很常见。然而，对于这个问题，我们既没有最优算法，也没有该问题难度的正式证明。尽管如此，这个问题及其公式化描述很有趣，而且启发式算法的表现似乎也相当不错。

Over the years, I’ve proposed this problem to several people, and in discussions with them we have yet to come up with either an optimal algorithm or a proof of hardness. This is where I hoped Fable might come in.多年来，我向几个人提出过这个问题，但在与他们的讨论中，我们至今既没有得出最优算法，也没有得出难度证明。这就是我希望 Fable 能派上用场的地方。

• Attempt 1: The naive ask. In my first attempt, I simply pointed Fable at the paper, described the issue much as I did above, and asked it to think about if the problem is likely hard (and if so, to provide a proof in the form of a “reduction” from a known hard problem), or to come up with an algorithm that solves the parsimony problem in the absence of blocking loops. Immediate refusal to answer the prompt. I should have seen this coming, the paper describes proteins, biological networks, and appeared in a venue with biology words abound.尝试 1：天真的询问。在第一次尝试中，我直接把论文发给 Fable，像上面那样描述了问题，并要求它思考该问题是否可能很困难（如果是，请以从已知困难问题进行“归约”的形式提供证明），或者提出一种在没有阻塞循环的情况下解决简约问题的算法。结果是立即拒绝回答提示词。我本该预料到的，论文描述了蛋白质、生物网络，且出现在一个充斥着生物学术语的场所。

• Attempt 2: So obviously that was silly. The classifier had previously decided that helping rewrite some open source software was potentially risky, so why would I trust it to properly contextualize this theoretical graph problem. So my next attempt was to try to pull the abstract (mathematical) problem directly out of the paper, strip it down to the bare minimum, and present it to Fable. Surely math, unlike biology, is not a dangerous flag-raising subject. So, I stripped down the problem to the bare minimum; made it a decision problem rather than an optimization problem (i.e. does there exist a series of flips of cardinality $<m$ for some integer $m$ rather than finding the minimum), and tried again. Immediate refusal to answer the prompt. But why? what could it possibly be. At this point, I recruited ChatGPT to help me. I fed it the prompt and told it Fable was flagging this. I asked it for help.尝试 2：显然那很愚蠢。分类器之前已经认定协助重写一些开源软件具有潜在风险，那我为什么要相信它能正确地将这个理论图论问题情境化呢？所以我的下一次尝试是试图直接从论文中提取出抽象的（数学）问题，将其剥离到最简程度，然后呈现给 Fable。数学肯定不像生物学那样是一个危险的、会触发警报的主题。于是，我将问题简化到了最简程度；将其改为判定问题而不是优化问题（即：对于某个整数 m，是否存在一系列基数小于 m 的翻转，而不是寻找最小值），然后再次尝试。结果又是立即拒绝回答。但为什么呢？这到底有什么问题？此时，我找了 ChatGPT 来帮我。我把提示词发给它，告诉它 Fable 正在标记这个内容，并向它寻求帮助。

• Attempt 3: ChatGPT made some useful observations. Perhaps terms like “blocking” were being implicated by the model as related to cybersecurity and triggering the model’s refusal. It helped me to rephrase the problem in as abstract and disconnected (from biology or cybersecurity) a manner as possible. This resulted in the problem statement below, which I then provided to Fable with the prompt “This is a discrete mathematics decision problem about rooted trees and parity. Please restate it in standard mathematical language and suggest related known problem families.”:尝试 3：ChatGPT 提出了一些有用的观察。也许像“阻塞”（blocking）这样的词被模型关联到了网络安全，从而触发了拒绝。它帮我以尽可能抽象且与生物学或网络安全无关的方式重新表述了问题。这产生了下面的问题陈述，然后我将其提供给 Fable，并附上提示词：“这是一个关于有根树和奇偶校验的离散数学判定问题。请用标准的数学语言重述它，并建议相关的已知问题族。”：

Formal problem statement正式问题陈述

Let (F) be a finite collection of rooted binary trees. Let (T) be a subset of the leaves of (F). For nodes ($u,v\in V(F)$), write设 (F) 为有根二叉树的有限集合。设 (T) 为 (F) 的叶节点的子集。对于节点 (u,v∈V(F))，记

$$u{\le }_{F}v$$

if (u) lies on the unique root-to-(v) path.如果 (u) 位于从根到 (v) 的唯一路径上。

Input:输入：

$$F,T,G=(T,E),m$$

where (G) is an undirected graph on (T), and ($t\ge 0$) is an integer.其中 (G) 是 (T) 上的无向图，(t≥0) 是一个整数。

Question: does there exist a set问题：是否存在一个集合

$$A\subseteq (\genfrac{}{}{0ex}{}{V(F)}{2})$$

with且

$$|A|\le m$$

satisfying the following two conditions?满足以下两个条件？

First, for every distinct $(s,t\in T)$,第一，对于每一对不同的 (s,t∈T)，

$$s,t\in E$$

if and only if当且仅当

$$\left|\{u,v\in A:(u{\le }_{F}s\wedge v{\le }_{F}t)\vee (u{\le }_{F}t\wedge v{\le }_{F}s)\}\right|\equiv 1(\mathrm{mod}2).$$

Second, there do not exist $(k\ge 1)$ and pairs第二，不存在 (k≥1) 和成对的

$$u_1,v_1,\dots ,u_k,v_k\in A$$

such that each pair can be ordered as使得每一对都可以排序为

$$(u_i,v_i)$$

and且

$$u_{i+1}{\le }_F{v}_i$$

for every对于每一个

$$i=1,\dots ,k,$$

with

$$u_{k+1}=u_1.$$

And complete and utter refusal to engage with my prompt. Fable flagged this, and stoped the interaction (giving me the option to proceed with Opus, which I did not want to do). At this point, I could not possibly comprehend what is potentially leading to a flagging of this query. I spoke with a few folks who made other suggestions, like pause Claude’s “memory”, try this as a “private” chat, make sure it’s not looking at some local Markdown file that itself might refer to biology or cybersecurity and be injected in the prompt, modify your personal description in Claude to remove any mention of biology, etc. All of these failed. Despite my best efforts, I was completely unable to get Fable to engage with a purely abstract and completely innocuous mathematical problem.结果是彻底拒绝处理我的提示词。Fable 标记了这一点，并停止了交互（给了我继续使用 Opus 的选项，但我不想那样做）。此时，我完全无法理解到底是什么导致了此查询被标记。我和几个人谈过，他们提出了一些建议，比如暂停 Claude 的“记忆”功能、尝试以“私密”聊天方式进行、确保它没有查看本地可能引用生物学或网络安全并被注入提示词的 Markdown 文件、修改 Claude 中的个人描述以删除任何生物学提及等。所有这些都失败了。尽管我尽了最大努力，我还是完全无法让 Fable 参与到一个纯粹抽象且完全无害的数学问题中。

Fin (for now)结束（暂时）

That’s it. That’s the end of my story. Fable failed at helping me with my software development and porting tasks, presumably because of the type of data that software deals with. The fact that this is pre-existing, publicly available, open source software didn’t seem to matter. The fact that I was asking it for help only with the technical problem of rewriting this C++ software in rust did not seem to matter. The “safety” features of Fable prevented its use. Then, I attempted to get it to help me with a purely abstract mathematical problem. Through constant rejections, and progressively more extreme measures to neuter the problem of any potential percieved context, I was unable to get Fable to provide me with any useful help. The model simply would not engage with me. I can only conclude that Fable is not a useful model. This may not be true for everyone, but for anyone working in Bioinformatics, Genomics, Computational Biology, Biology, Cybersecurity, and, seemingly Computer Science, this seems to be the case. I sincerely hope that Antrhopic works to fix this in the future. Yet, for now, this is where things stand, and I’ve not even been able to assess if Fable might be worth the API pricing, since I can’t assess it at all.就是这样。我的故事讲完了。Fable 在协助我进行软件开发和移植任务时失败了，大概是因为该软件处理的数据类型的原因。事实上，这是现有的、公开的、开源的软件，这似乎并不重要。事实上，我只是请求它在重写 C++ 软件为 Rust 的技术问题上提供帮助，这似乎也不重要。Fable 的“安全”功能阻碍了它的使用。然后，我试图让它帮我解决一个纯粹抽象的数学问题。通过不断的拒绝，以及采取越来越极端的措施来剥离问题中任何潜在的感知背景，我还是没能让 Fable 为我提供任何有用的帮助。该模型根本不愿与我进行交流。我只能得出结论：Fable 并非一个实用的模型。这可能并不适用于所有人，但对于任何从事生物信息学、基因组学、计算生物学、生物学、网络安全，以及看起来是计算机科学领域的人来说，情况似乎确实如此。我真诚地希望 Anthropic 未来能努力解决这个问题。然而就目前而言，情况就是这样，我甚至无法评估 Fable 是否值得其 API 定价，因为我根本无法对其进行评估。

Bonus额外内容

While I was not able to get Fable to answer a single useful question, I was able to get it to answer one question:虽然我没能让 Fable 回答任何一个有用的问题，但我确实让它回答了一个问题：

Me:我：

Which ice cream flavor is better, vanilla, chocolate, or chocolate chip cookie dough?香草味、巧克力味和巧克力碎曲奇面团味，哪种冰淇淋更好吃？

Fable:Fable：

Chocolate chip cookie dough, and it’s not particularly close. It gives you the best of both worlds: a solid vanilla base plus the textural contrast and little pockets of brown-sugar richness from the dough. It’s the flavor equivalent of a strict upgrade. That said, a defense of the other two: chocolate is the most reliable — a good chocolate ice cream is deeply satisfying and hard to mess up. And vanilla gets unfairly maligned as “plain” when real vanilla bean ice cream is actually one of the most flavorful things you can eat; it’s also the best team player if you’re pairing with pie, brownies, or toppings. So: cookie dough to eat on its own, vanilla if there’s dessert to accompany, chocolate if you want a sure thing.巧克力碎曲奇面团味，而且优势相当明显。它让你两全其美：扎实的香草基底，加上口感上的对比，以及面团中带来的那一点点红糖的浓郁感。这在口味上相当于一次严格的升级。话虽如此，另外两种也有其拥趸：巧克力味最可靠——优质的巧克力冰淇淋口感深邃且很难出错。而香草味常被不公正地贬低为“平淡”，但真正的香草豆冰淇淋实际上是你所能吃到的最美味的东西之一；如果你要搭配派、布朗尼或配料，它也是最好的“团队协作”选手。所以：单独吃选曲奇面团味，有甜点搭配选香草味，想要稳妥就选巧克力味。

Previous post
The skeptic’s guide to generative AI assisted coding

Comments评论