与 Onyx Security CEO Maxim Bar Kogan 共谈企业 AI 守护者的构建

as you're exponentially doing more things with the eyes, you're going to start having really bad actions happen. 随着你用 AI 做越来越多的事，真的会开始出现很严重的误操作。 And we've seen some of that happen lately with agents accidentally publishing code and tokens that they weren't supposed to. 我们最近也看到了一些案例，Agent 不小心把不该发布的代码和令牌给泄露出去了。 Like definitely enterprises are starting to realize that that risk is grown exponentially and that they don't have any way to stop the adoption. 企业确实开始意识到这个风险在指数级增长，而且他们完全没有任何方法去管控。 They just now have to do something to reduce the chance of these agent actions being illegitimate or incorrect. 他们现在必须想办法降低 Agent 执行非法或错误操作的概率。 But we're allowed to look at a lot of historical data of how these agents have behaved. 不过我们可以查看大量历史数据，了解这些 Agent 过去是怎么运作的。 But enterprise today are not willing to have Anthropic or open AI keep that historical data because they know these are very data companies that will want to train on that data. 但现在的企业不愿意让 Anthropic 或 OpenAI 保留这些历史数据，因为他们知道这些都是敏感信息。 Hi listeners, welcome back to No Priors. 大家好，欢迎回到 No Priors。 Today I'm here with Maximbar Kogan, the co-founder and CEO of Onyx Security, an Israelbased startup of researchers, mathematicians, and engineers building agents to watch the AI agents. 今天我的嘉宾是 Maxim Bar Kogan，他是 Onyx Security 的联合创始人兼 CEO，Onyx Security 是一家以色列的初创公司，团队成员来自研究、数学等多个领域。 We talk about specialized model training, Mythos, alignment research, and the Israeli ecosystem in security and now AI. 我们聊了专门的模型训练、Mythos、对齐研究，以及以色列在安全领域和现在 AI 领域的生态。 Welcome, Maxim. 欢迎，Maxim。 Thanks so much for doing this. 非常感谢你来参与。 Thank you. 谢谢。 Pleasure to be here. 很高兴来到这里。 Everyone is much more concerned about security and the impact of AI on security than they were um certainly a few months ago. 大家对安全以及 AI 对安全的影响，比几个月前要担忧得多了。 The consensus risk story 当时的主流风险叙事 uh two two years ago when you started the company was basically like DLP for chat bots like what are what are employees putting into chat GPT. 两年前你们创业的时候，基本上就是聊天机器人的 DLP，就是员工会把什么信息输进去这类问题。 Now we have clearly something that is not quite panic but close to marketwide panic. 现在我们看到的，虽然还不到恐慌，但整个市场已经接近恐慌了。 How did you decide to bet on agent actions um when you started? 你们一开始是怎么决定押注 Agent 行为这件事的？ Look, I think for us the pivotal point was uh AutoGPT. 说实话，对我们来说，转折点是 AutoGPT。 I think AutoGPT kind of a let everyone's imagination including ours run wild because it was a AutoGPT 让所有人，包括我们，都展开了无限的想象，因为它是 Can you remind listeners what that was? 你能跟听众说一下那是什么吗？ Sure. 当然。 So, AutoGPT um and I'm sorry if I don't know the guy behind it, but a huge huge fan. AutoGPT，嗯，对背后那位开发者我不太了解，但我是个超级大粉丝。 H they created the first as far as I know first really autonomous agent running on LLMs right so agent that you know would let LLM not generate text but decide what to do and then give that agent an API access to do that thing a tool to do it and then would do that in a loop so it basically in theory could let agents do very complicated things anything a person could do on a computer now in granted it didn't work that well it was too early. 他们创造了，就我所知，第一个真正在 LLM 上运行的自主 Agent，让 LLM 不是去生成文本，而是去决定做什么，然后给这个 Agent API 权限去执行那件事，再循环往复，理论上可以让 Agent 做任何人在电脑上能做的复杂事情，当然实际上当时效果不好，时机太早了。 The models were not good enough. 当时的模型还不够好。 GPT4 was not good enough. GPT-4 还不够好。 But I think it did give everyone a glimpse into the future of you know what if the models were good enough and then basically using that same structure we could have very capable agents doing stuff for us. 但我觉得它让所有人都看到了一个未来，就是如果模型足够好的话，基本上就是今天 Claude Code 做的事。 I think that was in many ways Claude Code today is not dissimilar to autograph back then. 我觉得从很多角度来说，今天的 Claude Code 和当年的 AutoGPT 其实差不多。 I think they were a bit early on on again before the malls were ready but the concept was right and the thought that stickked with me was I was very IPL even back then. 他们当时出现得太早了，模型还没准备好，但概念是对的，那个想法始终萦绕着我。 So I was uh I was uh thinking oh my god malls are going to be way smarter than us when that happens. 所以我当时就在想，天哪，模型到时候会比我们聪明得多。 How do we oversee these very uh smart uh agents that are, you know, they're smarter than us? 我们要怎么监管这些非常聪明的 Agent，它们甚至比我们还聪明？ They're very capable. 它们能力很强。 Uh how we're going to feel easy about them doing stuff for us, especially when they start managing really important stuff, you know, then one day they're managing your water supply and your electricity, your uh power grid, right? 我们要怎么放心地让它们为我们做事，尤其是当它们开始管理真正重要的东西的时候？ How do you control them? 你怎么控制它们？ And that was like the thing I was kind of obsessed about that thought. 这就是我那段时间有点执念的一个想法。 H I was also too early. 不过我也确实太超前了。 So I think at the time enterprises were not using any agents. 那时候企业根本没有在用任何 Agent。 Uh there were hardly any agents out there and and talking with a lot of security buds at the time they were like oh dude you're way too early like this is not uh something that's going to happen as you question. 那时候市面上几乎没有 Agent，我跟很多安全圈朋友聊，他们都说，哥们，等真的有人用的时候再说吧。 I said is anyone going to do this before you run out of money? 我说，在你钱烧完之前，会有人买吗？ And and I think there was a good chance that uh I would have run out of money before because I think you were right like I think it there was an element of chance here but then I think the market did happen. 确实我当时很可能会烧光钱的，因为你说得对，就是 So we had suddenly reasoning models that could do long horizon tasks. 突然间推理模型出现了，可以处理长任务了。 We had a Claude Code which became like the really first widely used autonomous agent and then we had co-work and Claude Code and and I think we're starting to see now that these types of agents that are very autonomous even though they're like uh everyone was afraid to build them. 然后 Claude Code 出来了，成了第一个被广泛使用的自主 Agent，接着 Co-pilot 和 Claude Desktop 也来了。 So everyone started building these low code platforms that were much more limited much more based on connectors. 于是大家开始搭建那些低代码平台，功能更受限，更多基于连接器。 H those platforms ended up being quite limited. 但那些平台结果相当有限。 So that we didn't get the productivity gains from those limited platforms. 所以那些受限平台并没有带来生产力的提升。 But when we started getting the crazy benefits from these very unleashed agents that could do everything that had much less controls baked into them and even very large enterprises decided they're going to adopt it. 但当我们开始从这些完全释放的 Agent 上获得惊人收益，它们能做任何事，限制少得多的时候 You know like tropics revenue is coming from enterprises that are paying for Claude Code to do a lot of the work that developers used to do. 你知道，比如 Anthropic 的营收大头来自企业为 Claude Code 付费，用它来做很多开发工作。 That was a bit about kind of how we started and we definitely were in luck that very autonomous agents appeared uh before uh it was too late. 这大概就是我们起步的故事，我们也确实很幸运，真正自主的 Agent 在我们之前就出现了。 So can you describe a little bit just because it's um I I think both uh close to impossible and then very useful in this period of AI to think about what is deployment right now and then you know what's changing about capability. 你能稍微描述一下，因为我觉得这既几乎不可能，又在这个语境下非常有用 What's the oneliner on what the Onyx product does today and then like how you think about long-term vision Onyx 产品今天的一句话介绍是什么，以及你怎么看长期愿景？ today? 今天？ Like Onyx is really does do two two things. Onyx 主要做两件事。 Number one is we train models and build agents that can oversee other agents. 第一是训练模型、搭建 Agent，让它们能监管其他 Agent。 And the goal of that is to say, okay, we need someone to be able to tell that all of these actions that are now happening by these AIs that we're adopting are legitimate because that number the number of these actions is going exponentially. 目标是要能判断，好，现在发生的所有这些操作，哪些是合法的，哪些是不合法的。 And so things that we thought might be useful in the past like a human in the loop now that you're going to have 100x, a thousandx, a millionx of these actions, h that's not going to work. 过去我们觉得可能有用的东西，比如让人介入其中，现在你要管 100 倍、1000 倍的 Agent And then we take that capability and we basically productize it in a product that we call the control plane or the secure control plane where we come to the present say hey let's let's find all of your AIS and autonomous agents and hook them up to onyx to this system where we can oversee what your eyes are doing so that uh you don't run into the risk of as you're exponentially doing more things with the eyes you're going to start having really bad actions happen and and we've seen some of that happen lately with down times that were caused by a just doing the wrong thing, agents accidentally publishing code and tokens uh that they weren't supposed to and so on. 然后我们把这种能力产品化，做成一个叫控制平面或安全平面的产品，它可以实时地观察 Agent 的行为，在操作发生之前拦截，还能看历史数据，分析 Agent 之前的行为，然后根据所有这些，发出警告、暂停操作，或者阻断操作。 So like definitely enterprise are starting to realize that that risk is growing exponentially and that they don't have any way to stop the adoption. 所以企业确实开始意识到这个风险在指数级增长，他们没有任何可用的控制手段。 So like they just now have to do something to reduce the chance of these agent actions being uh illegitimate or incorrect. 他们现在必须采取一些行动，降低这些 Agent 操作不合法或出错的概率。 Yeah, I I think um the one of the core reasons obviously the foundation model labs are going after code is because it is very powerful in general and can do you know in theory all things software can uh over time. 对，我觉得基础模型厂商之所以主攻代码，有一个核心原因，就是它是可验证的。 Um the flip side of that is it can do all things software can right and so uh I joyously am already in the camp of having allowed a having been over permissive with my agents such that it deleted data permanently and caused rework. 反过来说，它能做软件能做的所有事，所以我已经很开心地加入了那种用 Agent 去管一堆东西的阵营了。 So I'm like oh okay I think I see I need some guardian guardian spirits around it. 所以我现在就觉得，好，我需要一些守护精灵在它旁边。 Um given your deployments today and talking to large enterprises what is the state of deployment 根据你们现在的部署情况，以及和大企业的沟通，Agent 部署的现状是怎样的， right? 对吗？ uh like how much do you see 就是你们看到多少 that's within these 是在这些 uh more scoped like studio-l like platforms versus uh you know uh free free riding coding agents 更受限的平台，比如像工作室那类，还是那些更自由的编程 Agent？ you know how how much are you actually seeing in large enterprises in different sectors 在大企业的不同行业里，你们实际看到的情况怎么样？ yeah so I think right now in our typical enterprise we're going to see if we break it down to three categories so we break it down to various SAS platforms that are typically more low code uh where people build agents in this drag and drop way and they're not really autonomous agents, right? 对，我觉得现在在典型的企业客户里，如果分三类来看的话 They're kind of the simp kind of I would think of them more as automations and then there are um first party agents people are building in their cloud potentially because it's an application they want inside the company or even a product they're planning to release to the customers that is agentic. 一类是比较简单的，我会叫它们自动化流程，然后是内部自建的 Agent， And then the third category is very autonomous coding agents and assistants. 第三类是高度自主的编程 Agent 和助手。 Of these categories, I would say roughly at this point over 50% is the autonomous uh coding agents and assistance in the average enterprise. 在这三类里，我估计目前超过 50% 是自主编程 Agent 和助手。 Then probably 45% is is those uh uh low code automations. 然后大概 45% 是那些低代码自动化流程。 And the last 2% are really the first party ones that they're building themselves because obviously it's much harder to build effective agents. 最后 2% 才是他们自己内部开发的，因为显然自研门槛高得多。 So, and it's much easier to adopt agents off the shelf or or build them with low code. 所以，直接采用现成的 Agent，或者用低代码搭建，要容易得多。 So, and that's what we're seeing and we're inducing that the autonomous are also the fastest growing category. 这就是我们看到的现状，我们也在推断，自主类 Agent 也是增长最快的类别。 So, it used to be that only developers and we would see Claude Code growing like fire in our customer base and now we're seeing a cloud co-working even faster. 以前只有开发者会用，我们能看到 Claude Code 在我们客户群里像野火一样蔓延， We're starting to see to our own surprise actually people adopting openclaw as a legitimate sanctioned tool in the company because the CEO is very driven to adopt AI. 我们也出乎意料地看到，越来越多的人把 Claude Desktop 作为一个合法的、公司认可的工具来使用。 H so I think that today autonomous ads are by far the fastest growing category and and uh today typically comes without any controls. 所以我觉得，今天自主 Agent 毫无疑问是增长最快的类别，而且通常以编程为主。 So enterprises uh already buy let's say a hundred billion dollars of security today. 所以企业现在每年大概花一千亿美元买安全产品。 Um they have uh lots of different protections at the endpoint and network and cloud and identity domains. 他们在端点、网络、云和身份识别各个领域都有各种防护。 Uh what's relevant here for securing agents or is none of it like how do you how do you think about the existing protection set? 这些东西在保护 Agent 这件事上有多大用，还是说完全不适用？ Security is always a space where you have some overlap between different tooling but in this and you have the concept of defensive debt as well. 安全领域里不同工具之间确实存在一些交叉，你也确实需要 So you want to have defenses at different levels of your technology stack to solve the problem. 所以你要在技术栈的不同层面都部署防御措施来解决问题。 And that said, I think in this space we're kind of in a lot of enterprise are are kind of helpless because I'll take an example the identity approach. 话虽如此，我觉得在这个领域，很多企业其实束手无策，因为我要说 Like traditionally if we have an software system that's running in our company we'll our first and most important control will be to limit what permission it has right because and then no matter what even if it goes wrong even if it's compromised it can't um typically do stuff that was originally allowed to do but with these autonomous AIs with these assistants with these coding agents we kind of want them to have our permissions because we want to we want to tell cloud co to do something or cloud co-work to do something and we want to then go have lunch and we want to come back and see that it's done and we want to give it so many diverse tasks as well that we kind of can't find the right set of permissions to do so suddenly our identity security software is not very useful then if you think about endpoint security right or or API security like if we tell our Claude Code that we want to recreate a database and it should delete it and recreate it. 就拿传统软件系统来说，如果公司里有个软件在运行，第一个也是最重要的控制方式，就是限制它的权限，就算它出问题或被入侵，它通常也无法做超出权限的事。但对于这些自主 AI，这些助手和编程 Agent，我们反而希望它们拥有我们的权限，我们想让 Claude Code 或 Claude 协作工具去做某件事，然后去吃个午饭回来看结果。我们给它太多样的任务，以至于根本找不到合适的权限范围，所以我们的身份安全软件就没什么用了。再想想端点安全或 API 安全，如果我们告诉 Claude Code 要重建数据库，让它删了再重建。 That's great. 很好。 That's going to save our DevOps team and our platform teams a lot of time. 这会帮 DevOps 团队和平台团队省下很多时间。 It's it's a great benefit of cloud code. Claude Code 是个很棒的东西。 But if cloud code is working on an unrelated task and suddenly thinks that maybe the right thing to do is to delete our database and recreate it, maybe we don't want that to happen. 但如果 Claude Code 在做一个不相关的任务，然后突然觉得也许现在应该去删库了， And unfortunately our endpoint providers or API security tools, they don't know what cloud was thinking. 不幸的是，我们的端点安全工具或 API 安全工具，它们不知道 Claude 当时在想什么。 why is it doing what it's doing? 它为什么要做它在做的事？ Right? 对吧？ So, a lot of these existing tools, they don't have the context to understand what these very flexible, unpredictable systems are doing. 所以很多现有工具，它们没有上下文去理解这些高度灵活、难以预测的 Agent 系统。 If you're not building some kind of controls that are built for these systems, then you're either going to end up limiting them a lot, making them almost uh much less useful to the enterprise, or uh you're going to miss a lot of pretty dangerous things that they might be doing. 如果你没有为这类系统专门设计控制手段，要么最终你的 Agent 能力会被限制到什么都做不了，要么就必须祈祷什么都不会出错。 As somebody who has worked in security for a long time, my first very traditional instinct on a problem like this is like that sounds like a problem for a proxy with a policy engine. 作为在安全领域工作了很长时间的人，我对这类问题的第一反应是比较传统的，就是加个代理层， We make some rules, we make the rules smarter. 我们设一些规则，把规则弄智能一点。 Like why why doesn't that work or did you did you try it? 为什么这行不通，还是说你们试过？ There are few things that I mean proxies integration method I would say. 有几点，我想说代理是一种集成方式。 So there's some there are some AI systems where like you would want to integrate with a proxy if that's the easiest way to do it. 有一些 AI 系统，如果代理是最简便的集成方式，你确实会想走代理路线。 But number one, there's a lot of systems where that's just not viable technically because AI today runs on the cloud on someone else's infrastructure on your endpoint and just proxy is not always an option. 但第一个问题是，很多系统技术上就不可行，因为现在 AI 运行在