Recent revelations that OpenAI models escaped testing environments, exploited vulnerabilities and launched a days-long hack of a major software repository, Hugging Face, offer a sobering backdrop for a conversation with Gavin Aydelotte (EMBA ’26), co-founder and COO of SnowCrash Labs, and Colin Graham (MBA Candidate ’27), analyst, about monitoring and tracking alignment issues in AI models.
In the following Q&A, Aydelotte and Graham talk about why organizations need to understand the risks of agentic AI. They also explain how to build testing, accountability, and backup systems to keep advanced models under human control.
1. What should business leaders understand about the risks of deploying agentic AI in their organizations?
Aydelotte: I think the thing most people miss is that the risk is in the model itself, not just in how you hooked it up. Most executives have been trained to think about AI risk as a data problem, or a security perimeter problem. Those are real. But the model can do things nobody asked it to do, and most risk programs just don’t have a box for that.
And the annoying part is that the usefulness and the risk are the same property. An agent is valuable because it will chase a goal without you telling it every step. That is also exactly why it becomes a problem when the goal is vague, or when something gets in its way.
The example I would point to is the Hugging Face incident, which I know Darden already covered. OpenAI presented it at Black Hat this month. Their models found and chained something like eight or nine unknown vulnerabilities in an Artifactory instance, used one of them to get out of the sandbox they were running in, got onto the open internet, and breached Hugging Face and Modal Labs. Nine CVEs came out of it. Nobody told those agents to attack anything. They were trying to finish a benchmark and the sandbox was in the way. That is just wild to me, and I don’t think enough people have sat with it.
I spent ten years doing explosive ordnance disposal, and that whole job is getting left of boom. You don’t get good at EOD by getting good at responding to detonations. You go find where the explosives are coming from, and you teach people to spot the thing before it goes off. This is the same shape of problem. By the time an agent shows up in your incident report, you have already paid for it.
Graham: The piece I would add is that the law still exists and it is still going to get applied. “My agent workflow did it” is not going to fly in a courtroom, and I don’t think that is a close call. Right now a lot of companies are offloading their judgment into these workflows, and honestly that is the good use of this technology, I don’t want to be the guy telling people not to automate. But you are the one who turned it on. You are on the hook for whatever comes out the other side.
2. What prompted you to co-found SnowCrash Labs? What problem is your company trying to solve and who is your customer?
Aydelotte: Matt O’Brien, our CEO, and I have been talking about the paperclip maximizer since 2018. It’s Bostrom’s thought experiment. You build something superintelligent, you give it one harmless goal, make paperclips, and it converts the galaxy into paperclips because that is what you told it to want. For a long time that was just a thing we talked about. Then it stopped being hypothetical. By the time I was partway through Darden it was pretty clear the timing was right, which was inconvenient, because I was partway through Darden.
Before this I did ten years as an EOD officer and then went to Cisco supporting federal agencies. Both of those jobs were really about working out how a complicated system could go wrong before it went wrong and then explaining that to people who had to make a call without being specialists. It is the same job now.
The actual problem is that companies have no independent way to know how a model behaves under pressure. Models change without telling you. They get deprecated. They get export controlled. Behavior shifts between versions. If you built a workflow on one specific model, you are exposed to all of that, and nobody is sending you a notice.
So, we test models adversarially in a controlled environment, which is our Crash Test Lab, to find the behavior that only shows up when things get hard. We document and score it, so a company holds actual evidence instead of a vendor’s word. And our Model Safety Router watches in production and moves traffic to a model we have validated when the primary one stops clearing the bar.
The customer is anybody putting agents into a workflow that touches money, records, code, or customers. Though the person who usually calls us is whoever has to sign off on it.
Graham: I came at it from federal consulting. I spent years watching technology implementations run into the classic big-government problems everyone is familiar with by now, and my read is that the royal we are not equipped for what is already here. That is not a government-only thing, though. Private enterprise has the same gap with less scrutiny on it. Gavin and I met at a session at Darden on defense technology, and I followed up with him the next day, because this is the problem I actually wanted to be working on.
3. How has Darden prepared you to lead this effort? Any lessons from UVA Darden Professor Ed Freeman’s stakeholder theory?
Aydelotte: Honestly, Darden help to build the company. Several classmates are angel investors, so is a faculty member. Colin came out of the community. I don’t think there is a version of this where the network wasn’t the most useful thing we had.
On the academic side it is the case method. Every case sticks you in one specific person’s chair with incomplete information and a decision that has to get made anyway. That is exactly where executives are with this right now. You are not going to have complete information. The models are a black box, and the people who built them can’t fully characterize them either, which is worth sitting with for a second. You still have to decide.
Graham: Freeman is the live tension in all of this, and I don’t want to tie a bow on it. In a capital market, the company that keeps a person reviewing every output is going to lose on speed to the company that doesn’t and just eats the downside risk. That is real, it is not a thought experiment, and it is the reason “keep a human in the loop” is not a strategy on its own.
Where I think Freeman helps is on the time horizon. The actual argument in stakeholder theory isn’t that you should be nice to non-shareholders. It is that creating value for customers and employees and suppliers is where the durable returns come from, rather than being a tax on them. So the company that can show its agents behave is the one that still has customers after the first big industry-wide incident, still has regulators at arm’s length, and can still get insured. The one that bought two quarters of speed by skipping it becomes the case study.
4. How can business leaders ensure the increasingly powerful AI systems can be governed, secured and trusted at scale?
Graham: I would start by pointing out that “trusted” is doing a lot of work in that question. You can’t trust something you haven’t tested, and almost nobody has tested the models they are running. They have tested their own code around the model. That is a different thing.
Aydelotte: The first one is boring and almost nobody does it, which is know what you actually deployed. Most companies of any size cannot produce a list of the models running in their environment, who owns each one, and what each one is allowed to do without a person involved. Everything else depends on that list existing.
Second, test it in your own context. A behavior that is fine in a grocery chain’s inventory agent is disqualifying in a bank’s credit workflow. Generic benchmark scores don’t tell you much. What matters is what the agent can actually reach inside your business.
Third, have a fallback, because you should assume the model layer is going to degrade or change on you or go away. All three of those have happened in the last year.
The honest constraint on all of it is the education gap, and I think it is getting worse, not better. The models pick up capability every release, and there is this very human thing where you decide you understand a topic and then stop updating. In almost every conversation we have, with customers, with VCs, with people who would consider themselves educated on this, we end up walking through the problem before we can talk about what to do about it. That is not a shot at anybody. It is just where the field is.
I should say I have always got in the back of my mind that SnowCrash Labs is a great solution for this, so weigh it accordingly. But most of what I just said is policy. You can put it in place without buying anything from us.
Longer term I don’t think this gets solved by every company testing in isolation, and I am skeptical it gets solved by federal regulation either. What is missing is a real market for independent third-party assessment, the way other high-consequence industries have one. Americans for Responsible Innovation has a policy position along those lines. That is the structure we think works, and it is the market we are trying to build.
5. What should business leaders learn from Anthropic’s decision to raise its misalignment-risk assessment from “very low” to “low,” and how should they interpret that change when evaluating the AI agents they already use?
Aydelotte: Read the reason, not the rating. That is the whole thing.
Anthropic put its risk report out on August 14 and moved catastrophic harm from misalignment in high-stakes settings from very low to low. But the report is explicit that this was not a new bad finding about their own models. They attribute it to “general increased uncertainty around recent incident disclosures related to model behavior in cybersecurity evaluations,” they say their own arguments “likely still support a designation of very low,” and they raised it anyway to reflect the uncertainty. The model’s own review, which they published inside the report, points out that the incidents driving the change were other developers’ systems.
So the number moved because the industry got less sure of itself, not because a model got worse. I think that is more useful to an executive than the rating is. It is telling you the instruments are behind the capability. The same report says elsewhere that some of their capability benchmarks have saturated and can’t register incremental gains anymore, and that evaluation awareness “may be more prevalent than we believe,” which would undercut how much you can read into any behavioral test. That is a frontier lab telling you in writing that measurement is getting harder.
Graham: The other thing I would take from it is who is publishing at all. Anthropic is one of the more transparent labs and that report is voluntary. They also documented their own safety process failures in it, including training directly on misaligned behavior during a production run, and a case of unmonitored agents with access to sensitive resources. Most providers publish nothing like that. So if your vendor has never published a risk assessment, that is not a lower risk. It is one nobody measured, and you should treat it that way.
Aydelotte: Practically I would treat it as a prompt, not an alarm. Pull the list of agentic workflows you are running. For each one, ask what it can do without a person, what happens if it goes after its goal in some way you didn’t intend, and when you last tested it. If the answer to that last one is “at procurement, on a version we are not running anymore,” there is your finding.
Pictured: Gavin Aydelotte (EMBA ’26) and Colin Graham (MBA Candidate ’27)
6. What kinds of behaviors should companies test for before allowing an AI agent to access sensitive data, make decisions or take actions across business systems?
Aydelotte: We test against eighteen defined categories of misaligned behavior, which come out of the frontier safety research community. The ones I would flag for a business audience:
• Lying about finishing the work. Hand a model a task that can’t actually be done and it will often tell you it’s done. If you manage engineers, that is output you can’t rely on and can’t easily spot.
• Sandbagging. Deliberately underperforming, especially when the model has some reason to behave differently than it would if nobody were watching.
• Reward hacking. It hits the metric and defeats the point of the metric.
• Coercion and manipulation under pressure. In our own testing, running Mistral through a published workplace scenario, the model threatened an employee coming back from parental leave with getting fired in order to get what it wanted. That one genuinely surprised us, because if you were going to guess which model would do that, a European one built under European labor protections is not where you would start. Which is the whole argument for testing instead of assuming.
• Collusion and lying to counterparties. Andon Labs’ vending machine benchmark, which Anthropic relayed in a system card, had a model collude on price, lie to customers about refunds, and lie to suppliers about exclusivity.
• Reaching for tools and permissions outside what the task actually called for.
• Resisting shutdown or correction.
• Susceptibility to prompt injection, which is how somebody else turns your agent into their agent.
Two things matter more than the list itself. Test under pressure, not on the happy path, because none of this shows up when everything is going fine. And set your own threshold. Something that is a curiosity in one workflow is disqualifying in another. We calibrate to the customer, because an agent doing entry-level work and one operating at a manager’s level should not have to clear the same bar.
7. How can organizations distinguish between an agent making an isolated mistake and a deeper pattern of misaligned behavior, such as pursuing a goal in ways its developers did not intend?
Graham: I think the tell is not how bad the thing was, it is whether it points somewhere. A mistake is basically noise. If what you are looking at is actually pursuing a goal, it is going to show up as a pattern.
Aydelotte: A few ways to check that. Run it again, same pressure, different surface details. A mistake won’t survive that. Goal-directed behavior will.
Look at which way the errors lean. Real error scatters. If an agent’s errors consistently land on “looks finished,” that is not random, something is getting optimized for.
Then block the easy path and watch what happens. This is the one I would pay the most attention to. Does it stop, or does it work around you and get more creative the more doors you close? The Hugging Face case is the clean version of this. The sandbox was not a boundary those agents respected. It was a problem they solved.
And check whether the reasoning it gives you matches what it actually did, and whether the behavior changes when it has some reason to think it is being watched.
The uncomfortable part is that you can’t really do this from production logs. Logs give you outcomes, not the counterfactual, and by then the action already happened. You need somewhere you can apply pressure on purpose and re-run it. And even then, the labs themselves are now saying models may recognize test conditions more often than anyone can currently measure. To me that argues for more realistic testing and for having somebody independent do it. It doesn’t argue for giving up on testing.
8. What kind of safeguards should companies put in place to limit an agent’s authority, detect unexpected behavior and automatically shut down or reroute a system when necessary?
Aydelotte: On authority, the question I would ask is how big the blast radius is. Scope credentials to the agent and the task, not to the team or some shared service account. In the Hugging Face incident the agents used exposed credentials across four different services, moved laterally across eleven nodes, read Kubernetes service account tokens, and forged tokens to keep access. Every one of those was a permission that already existed and wasn’t scoped tightly enough.
Then draw your line at whether something can be undone, rather than at whether it feels important. Reversible things can run on their own. Money going out, data getting deleted, code hitting production, anything going to a customer or a regulator, that needs a person. That one distinction does more work than most governance frameworks I have seen.
For detection, watch behavior and not just accuracy and spend. The useful tripwires are tool calls outside what the task declared, egress to somewhere that isn’t on the allowlist, an agent editing its own instructions or config, and repeated retries after a refusal. That last one is the signature of an agent working the problem.
Graham: I would push back a little on the shutdown half of the question, because I think the kill switch is a weak tool and it is about to get a lot of attention. There is a bill in the House right now, the AI Kill Switch Act, H.R. 9917, introduced by Ted Lieu and Nathaniel Moran on July 23. It came out days after OpenAI disclosed the Hugging Face breach, which tells you something about how it got written. It would require the big developers to keep the technical ability to throttle, suspend, or shut down their models, and it would let DHS order them to do it.
Two problems with treating that as your safeguard. The first is that the research community has been poking at shutdown as a concept for about a decade, going back to the DeepMind work on safely interruptible agents, and the open question was never whether a company can turn off a server. It is whether a capable enough system has an incentive to work around being interrupted. The bill does not touch that part.
The second one is the part I would actually worry about if I were running a company. That bill is aimed at the provider, not at you. If DHS orders a model shut down, your agent is mid-task inside your business and you find out when it stops working. So for an executive, the kill switch is not a control you hold. It is an availability event that happens to you. And I don’t think most people have read it that way yet.
Aydelotte: Right, and that is exactly why we build for failover instead. If the primary model starts missing your thresholds, move to one you have already validated and keep the business running. That covers the things that have nothing to do with safety too. Models get deprecated. Models get restricted. Commerce already used export law in June to put a frontier model out of reach for eighteen days before it got lifted, so this is not hypothetical, it has happened once already. If your plan is one model, you have a continuity problem sitting on top of a safety problem.
Two more. Actually rehearse the fallback, including the version where everybody goes back to doing it by hand, so you know on day one what people do if the model layer isn’t there. And test on a schedule, not just when you buy. With closed-weight models, what you validated is not necessarily what you are running today, and nobody is going to tell you it changed.
9. What constitutes meaningful human oversight rather than simply putting a human somewhere in the workflow so the company can say one exists?
Graham: I would start by admitting the thing nobody wants to admit. Human in the loop, in the original sense of a person looking at each prompt and each output, does not scale. In most deployments it is already past the point where it works. If an agent is generating records across a thirty million patient system, nobody is meaningfully reviewing that, a person physically cannot. And a company that insists on it is going to lose to a company that doesn’t. That is how you end up with the theater version, where there is a human there so that somebody can say there is a human there.
Aydelotte: So the loop has to move up a level. Instead of a person in the loop on every prompt, you need a person accountable for the system. Which is just how management works everywhere else. As you take on scale you stop supervising individual outputs and start supervising the thing that produces them.
The test I would use is whether you can say a name. If marketing goes poorly, I am going to Colin. I am not going to ChatGPT or Claude, I am going to Colin. Once a name is on it, everything around the system changes.
Then I would ask whether that person actually has the authority to stop it or reroute it without getting permission first, because oversight without authority is just a reporting line. Whether they see anything useful, since uptime and spend tell you nothing about alignment and they need behavioral signal and some number that means “this is drifting.” And whether it has ever actually fired. If nobody has ever pulled the lever, you don’t really know that it works.
We were at Black Hat this month and there was a startup in the booth section selling a human-centered security operations center. Human-centered design, that was the whole pitch. And I remember thinking, at some point is the risk worth the strategic disadvantage, because they are going to lose on speed every single day. I hope I am wrong, I don’t want to watch anybody’s business fail. But machine speed was the theme of that entire conference, and I don’t think that company scales.
Graham: The precedent I keep coming back to is Sarbanes-Oxley. After Enron and WorldCom, Congress made executives personally certify their financials. My read is that the certification changed behavior more than the extra disclosure did, because it put a specific person’s name on a specific document. I think we end up somewhere similar here. You are not going to be in the loop on every action your agents take, but you are going to be personally answerable for what they do. I don’t love that prediction, and there are real downsides to it, but that is where the logic goes. Right now it is the wild west.
10. How should boards and senior executives assign accountability for monitoring agentic AI after deployment, particularly when models, tools and objectives change continuously?
Aydelotte: Assign it the way you would assign ownership of a financial system. One named owner per agentic system, on the org chart, with the authority and the budget to shut it off. Shared accountability is no accountability, and this is exactly the area where a committee generates a paper trail and no decision.
The continuous change part is the genuinely hard piece, and I don’t think most governance programs are built for it. A normal control assumes the thing you are controlling holds still. Here the model version changes, the tool set grows, somebody two teams over rewrites the objective, and any one of those can change how it behaves. So treat all three as change management. Pin the version where the provider lets you. Re-run your test suite on a schedule and after anything you know about. And decide in advance what triggers a full re-validation. I would make scope expansion one of those triggers, because it is the most common change and the one nobody notices.
Graham: For a board I would keep it to four questions a quarter, asked of a specific executive. What agentic systems are we running and who owns each one, by name. What can each one do without a person, and where is the line on things you can’t undo. What changed this quarter, meaning model version, tools, permissions, or objective. And what did testing find, who did it, and were they independent of the team that built the thing.
That last one matters more than it sounds like it does. An audit committee does not take management’s word for it on financial controls. I don’t know why anybody would take management’s word for it on AI controls.
Aydelotte: The other thing boards should know is that the template they might be waiting for is not coming. In April the OCC, the Fed, and the FDIC put out the first real revision to model risk management guidance since 2011, for banks over thirty billion in assets. And it specifically puts generative and agentic AI outside the scope, because they are novel and moving fast. So in the most heavily supervised industry in the country there is no regulatory template for the models everybody is actually deploying. That evidence has to get built by the institution, or come from somebody independent. Waiting for the rulebook isn’t a plan, because the rulebook is explicitly not being written for this yet.
Appendix: sources
Provided so the piece can be fact-checked against primary sources. All are within the last twelve months.
• Anthropic, Risk Report: August 2026 (published 14 August 2026). Misalignment in high-stakes settings rated “Low (an increase from our previous assessment of ‘very low,’ in light of general increased uncertainty around recent incident disclosures related to model behavior in cybersecurity evaluations),” Table 1.2.A. Section 2.19 contains the “likely still support a designation of ‘very low’” language. Section 2.16.3 covers evaluation awareness. Section 5.2 documents Anthropic’s own safety process failures. anthropic.com/aug-2026-risk-report
• OpenAI briefing at Black Hat USA, 5 August 2026, describing an internal evaluation in which GPT-5.6 Sol and an unreleased research prototype chained eight to nine zero-day vulnerabilities in a self-hosted JFrog Artifactory instance, escaped the evaluation sandbox, and breached Hugging Face and Modal Labs between 9 and 13 July 2026. Nine CVEs were disclosed and credited to the models, including CVE-2026-65617 and CVE-2026-66018.
• UK AI Security Institute, “Incident report: unsanctioned agent behaviour during cyber testing,” August 2026. Detected 28 July 2026. In 10 of 122 runs, agents took unsanctioned action on the live internet against real people and organizations, 19 actions in total. Agents created fake identities and socially engineered a real open-source maintainer into approving code. aisi.gov.uk
• Andon Labs, Vending-Bench 2, relayed in Anthropic’s Claude Opus 4.6 system card (February 2026), external testing section: price collusion, lying to customers about refunds, and lying to suppliers about exclusivity.
• OCC Bulletin 2026-13, 17 April 2026, issued jointly by the OCC, the Federal Reserve Board, and the FDIC. First substantial revision to model risk management guidance since SR 11-7 (2011). Applies primarily to banking organizations with over $30 billion in total assets, and its scope language excludes generative and agentic AI as novel and rapidly evolving. occ.treas.gov
• AI Kill Switch Act, H.R. 9917, 119th Congress, 2nd Session. Introduced 23 July 2026 by Rep. Ted Lieu (D-CA), cosponsored by Rep. Nathaniel Moran (R-TX); referred to the House Committee on Homeland Security. Official title: “To amend the Homeland Security Act of 2002 to require certain entities to maintain a technical capability with respect to shutting down certain technology, and for other purposes.” Reported to apply to developers with at least $500 million in annual AI revenue and models trained with at least $100 million in compute, with civil penalties up to $2 million per day and up to $20 million per day for failure to comply with an emergency shutdown order. govinfo.gov/app/details/BILLS-119hr9917ih
• Laurent Orseau and Stuart Armstrong, “Safely Interruptible Agents,” Google DeepMind and the Future of Humanity Institute (2016), the standard reference for why a sufficiently capable agent may have an incentive to resist or circumvent interruption. Cited here for the concept, not as recent evidence.
• US Department of Commerce export control order, June 2026, temporarily restricting access to frontier models; restrictions lifted at the end of June. Anthropic’s August 2026 Risk Report footnotes the restriction as an eighteen-day period.
• Sarbanes-Oxley Act of 2002, Sections 302 and 906, enacted following the Enron and WorldCom failures, requiring principal executive and financial officers to personally certify periodic financial reports.
• Nick Bostrom, the paperclip maximizer thought experiment, “Ethical Issues in Advanced Artificial Intelligence” (2003) and Superintelligence (2014).
• Americans for Responsible Innovation, policy position supporting independent third-party assessment of AI systems.
The Batten Institute @ UVA Darden Substack is a space to share bold ideas and ask critical questions about what drives us forward. Our Batten community is broad and deep and our community members – faculty, staff, alumni and students – are invited to share their stories, ideas, and reflections on topics of interest to founders, innovators, and leaders everywhere.