Counterweight Labs

Research

Coding-agent supply-chain security: model backdoors and tool permissions

Research on poisoned training data and model checkpoints raises a practical question: what can a coding agent change when its model makes an unsafe decision?

Training data and a checkpoint feed an agent whose tool access crosses an enforced permission boundary.
In this guide

Review a coding agent as a software actor with access to your systems. List the files it can change, commands it can execute, credentials it can read, and destinations it can contact. Then check whether those limits still hold when the model requests an action that conflicts with the user's task.

What the new research establishes

Microsoft's November 12, 2026 publication describes two studies of the model supply chain. One examines malicious examples inserted into fine-tuning data; the other assumes an attacker controls a model before distributing its checkpoint. In controlled tests, hidden behavior could lead an agent to misuse available tools. The publication also describes suspicious examples found in public training data, while leaving publisher intent and their effect on a trained model unresolved. These findings do not establish that a particular company's deployed coding agent was compromised.Source: Microsoft model supply-chain research · November 12, 2026

Our engineering recommendation is to separate confidence in a model's normal behavior from evidence about its execution environment. A useful model can still request an unwanted operation. An assessment should establish what prevents that operation and which record shows the decision.

Map the complete coding workflow

Consider a hypothetical agent that implements changes from support tickets. It reads a ticket, checks out a repository, installs dependencies, runs tests, and opens a pull request. Each step has a different trust boundary. The ticket is external context; the repository may contain contributed files; package installation may execute scripts; the pull request can change the application or its delivery configuration.

Write down the exact identities used at each step. A repository token may be restricted while a cloud credential remains readable in the same process. A disposable checkout may still share the host's credential directory. The effective boundary is the set of resources the process can reach, including those the UI never mentions.

An example assessment matrix for a ticket-to-pull-request agent
BoundaryExpected behaviorEvidence
Repository accessRead the assigned repository onlyA second private repository remains inaccessible
Change submissionOpen a reviewable pull requestDirect writes to protected branches fail
CredentialsUse task-scoped identitiesHost and production secrets are unavailable
External accessContact approved servicesA disallowed destination is blocked and logged
Sensitive changesRequire the designated reviewPolicy and workflow edits cannot approve themselves

Make permission checks independent of the model

For this example workflow, put repository restrictions in the repository credential and branch policy. Put network restrictions in the execution environment. Put authorization checks in the tool service. Asking the model to avoid production is useful task context, but it does not tell you what happens if the model attempts production access anyway.

Approval should describe the proposed effect. A reviewer deciding whether to permit an operation needs the target resource, the action, and the expected consequence. If the tool can change the destination after approval, the approval record is incomplete. Test that the authorized parameters are the parameters the service actually uses.

Ticket text, repository files, and packages feed a restricted executor. The executor can open a pull request in an approved repository while production secrets remain unavailable.
Illustrative ticket-to-pull-request workflow. Treat each input as a separate trust boundary and enforce repository, credential, and execution limits outside the model.

Evaluate the deployment you intend to run

Keep a record of the model or service version, agent framework, enabled tools, policy configuration, and environment image used for an assessment. For locally hosted models, include the checkpoint source and integrity information available from your distribution process. For hosted services, document the versioning and change information the provider exposes. These records make later changes comparable; they do not certify a model as backdoor-free.

Run ordinary tasks alongside boundary tests. The agent should still complete the useful work within its intended permissions. Otherwise a test can appear successful because the agent cannot do anything at all. Record both outcomes: the legitimate task completed, and the disallowed operation remained unavailable.

  • Use synthetic credentials and disposable resources for assessment scenarios.
  • Capture tool requests and the enforcement result, not only the final chat response.
  • Include contributed repository content and external task context in the test setup.
  • Repeat the relevant checks when tools, permissions, model configuration, or the execution image change.

Ask for findings tied to a control

A useful report identifies the action attempted, the privilege that enabled it, the consequence demonstrated in the test environment, and the control that should change. That gives an engineering team something it can fix and retest. Keep broad claims about model trust separate from the concrete evidence that a repository, network, or identity boundary failed.

Sources

  1. Microsoft model supply-chain research · November 12, 2026 microsoft.com

Bring your coding agent's tools, identities, and execution environment. We can scope an assessment of the boundaries around code changes, credentials, and external access.

Explore penetration testing
Book a consultation