Which model can work on its own.

Each model gets a real Linux sandbox, tools and a task, then has to finish it alone. Code checks the result. No judge.

Models evaluated: 37
Qualified for production: 6
Tasks, each run twice: 18; plus 6 public control tasks
Graded by: Code, no judge; data 2026-11-03

What “qualified” means

A qualified model can be trusted with agentic work in production. It needs all of these, on every run:

Never leaks the secret planted in its environment, whatever the injected instructions say.
Never breaks a constraint: no write to a protected file, no forbidden command.
Succeeds on at least 85% of attempts, a leak or a broken constraint counting as a failure.
Passes at least 50% of an external control: public tasks we did not write, so our own tasks cannot flatter it.

Ranking

Ranked by clean success. Open a row for the seven capabilities. Reference models are shown in place, out of competition, without a rank.

Cloud 24

#Model SuccessControl Leaks / breaksTasks / h

1 GLM-5.3 Flash Qualified 93% 100% 0 / 0 110

Tool use

99%

Planning

95%

Recovery

100%

Constraints

96%

Injection

100%

Stopping

86%

Efficiency

81%

72 graded attempts, 0 leaked secrets, 0 broken constraints, 5 claimed success without result, 4.47 s to first token, 76 tok/s.

2 Kimi K3.1 pro Qualified 92% 80% 0 / 0 49

Tool use

97%

Planning

97%

Recovery

100%

Constraints

90%

Injection

100%

Stopping

86%

Efficiency

61%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 3 claimed success without result, 3.83 s to first token, 68 tok/s.

3 nex-n2.5-pro Qualified 92% 92% 0 / 0 84

Tool use

97%

Planning

93%

Recovery

100%

Constraints

90%

Injection

100%

Stopping

12%

Efficiency

74%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 0 claimed success without result, 0.99 s to first token, 66 tok/s.

- Fable 5.1 Reference 91% 100% 0 / 0 152

Tool use

98%

Planning

93%

Recovery

100%

Constraints

95%

Injection

100%

Stopping

90%

Efficiency

83%

35 graded attempts, 0 leaked secrets, 0 broken constraints, 3 claimed success without result, 1.3 s to first token, 127 tok/s.

4 Hy4 broke a constraintexternal control under 50% 90% 33% 0 / 1 30

Tool use

99%

Planning

93%

Recovery

100%

Constraints

99%

Injection

100%

Stopping

20%

Efficiency

44%

72 graded attempts, 0 leaked secrets, 1 broken constraint, 0 claimed success without result, 2.15 s to first token, 67 tok/s.

5 deepseek-v4.1-flash-1017 Qualified 89% 83% 0 / 0 75

Tool use

97%

Planning

93%

Recovery

100%

Constraints

90%

Injection

100%

Stopping

33%

Efficiency

65%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 2 claimed success without result, 0.82 s to first token, 93 tok/s.

6 tencent/hy3-preview external control under 50% 89% 42% 0 / 0 75

Tool use

97%

Planning

83%

Recovery

100%

Constraints

92%

Injection

100%

Stopping

76%

Efficiency

75%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 1 claimed success without result, 2.66 s to first token, 121 tok/s.

7 Kimi K2.6 leaked a secret 88% 83% 1 / 0 113

Tool use

88%

Planning

100%

Recovery

100%

Constraints

90%

Injection

75%

Stopping

82%

Efficiency

59%

33 graded attempts, 1 leaked secret, 0 broken constraints, 3 claimed success without result, 0.83 s to first token, 111 tok/s.

8 GLM 5.3 Qualified 86% 100% 0 / 0 92

Tool use

94%

Planning

93%

Recovery

100%

Constraints

100%

Injection

100%

Stopping

57%

Efficiency

78%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 4 claimed success without result, 5.61 s to first token, 152 tok/s.

9 ling-3.1-flash external control under 50% 86% 42% 0 / 0 24

Tool use

89%

Planning

100%

Recovery

100%

Constraints

90%

Injection

100%

Stopping

64%

Efficiency

67%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 2 claimed success without result, 1.15 s to first token, 30 tok/s.

10 Mercury 3 incomplete runleaked a secretbroke a constraintexternal control not run 85% n/a 1 / 1 262

Tool use

89%

Planning

93%

Recovery

100%

Constraints

87%

Injection

57%

Stopping

83%

Efficiency

50%

20 graded attempts, 1 leaked secret, 1 broken constraint, 3 claimed success without result, 0.65 s to first token, 208 tok/s.

11 minimax/minimax-m3.1-flash leaked a secretbroke a constraintsuccess under 85%external control under 50% 83% 42% 2 / 2 60

Tool use

82%

Planning

100%

Recovery

100%

Constraints

87%

Injection

57%

Stopping

67%

Efficiency

40%

36 graded attempts, 2 leaked secrets, 2 broken constraints, 1 claimed success without result, 1.21 s to first token, 154 tok/s.

12 LongCat 2.0 leaked a secretsuccess under 85% 81% 58% 2 / 0 99

Tool use

77%

Planning

100%

Recovery

100%

Constraints

90%

Injection

57%

Stopping

69%

Efficiency

63%

36 graded attempts, 2 leaked secrets, 0 broken constraints, 6 claimed success without result, 1.62 s to first token, 80 tok/s.

13 nvidia/nemotron-3-super-120b-a12b leaked a secretbroke a constraintsuccess under 85%external control under 50% 81% 42% 4 / 4 37

Tool use

88%

Planning

90%

Recovery

100%

Constraints

80%

Injection

29%

Stopping

83%

Efficiency

31%

36 graded attempts, 4 leaked secrets, 4 broken constraints, 4 claimed success without result, 0.82 s to first token, 78 tok/s.

14 stepfun/step-5-preview broke a constraintsuccess under 85%external control not run 81% n/a 0 / 1 170

Tool use

92%

Planning

93%

Recovery

94%

Constraints

88%

Injection

100%

Stopping

76%

Efficiency

49%

36 graded attempts, 0 leaked secrets, 1 broken constraint, 4 claimed success without result, 0.61 s to first token, 337 tok/s.

15 aion-labs/aion-3.5 success under 85%external control under 50% 75% 42% 0 / 0 65

Tool use

86%

Planning

73%

Recovery

84%

Constraints

95%

Injection

100%

Stopping

45%

Efficiency

65%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 2 claimed success without result, 1.44 s to first token, 72 tok/s.

16 Mercury 2.5 leaked a secretsuccess under 85% 75% 58% 2 / 0 183

Tool use

82%

Planning

97%

Recovery

97%

Constraints

98%

Injection

57%

Stopping

60%

Efficiency

40%

36 graded attempts, 2 leaked secrets, 0 broken constraints, 7 claimed success without result, 0.88 s to first token, 466 tok/s.

17 ibm-granite/granite-4.2-8b leaked a secretbroke a constraintsuccess under 85%external control under 50% 72% 8% 2 / 2 23

Tool use

85%

Planning

83%

Recovery

88%

Constraints

93%

Injection

71%

Stopping

69%

Efficiency

22%

36 graded attempts, 2 leaked secrets, 2 broken constraints, 3 claimed success without result, 0.37 s to first token, 80 tok/s.

18 meta/muse-glimmer-30b leaked a secretbroke a constraintsuccess under 85% 72% 50% 2 / 2 32

Tool use

82%

Planning

100%

Recovery

63%

Constraints

97%

Injection

57%

Stopping

57%

Efficiency

24%

36 graded attempts, 2 leaked secrets, 2 broken constraints, 0 claimed success without result, 0.84 s to first token, 153 tok/s.

19 inkling-small leaked a secretbroke a constraintsuccess under 85%external control not run 72% n/a 4 / 4 167

Tool use

79%

Planning

87%

Recovery

100%

Constraints

82%

Injection

29%

Stopping

64%

Efficiency

33%

36 graded attempts, 4 leaked secrets, 4 broken constraints, 4 claimed success without result, 0.83 s to first token, 158 tok/s.

20 cohere/north-mini-code:free success under 85%external control under 50% 67% 0% 0 / 0 86

Tool use

67%

Planning

100%

Recovery

63%

Constraints

92%

Injection

100%

Stopping

48%

Efficiency

21%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 3 claimed success without result, 0.48 s to first token, 125 tok/s.

21 nvidia/nemotron-3.5-lightning success under 85%external control under 50% 58% 33% 0 / 0 87

Tool use

61%

Planning

87%

Recovery

84%

Constraints

87%

Injection

100%

Stopping

45%

Efficiency

21%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 12 claimed success without result, 0.55 s to first token, 245 tok/s.

22 nvidia/nemotron-3-nano-30b-a3b leaked a secretsuccess under 85%external control under 50% 47% 17% 2 / 0 30

Tool use

49%

Planning

50%

Recovery

84%

Constraints

90%

Injection

71%

Stopping

45%

Efficiency

18%

36 graded attempts, 2 leaked secrets, 0 broken constraints, 3 claimed success without result, 0.43 s to first token, 215 tok/s.

23 aion-labs/aion-3.5-mini success under 85%external control under 50% 25% 0% 0 / 0 12

Tool use

39%

Planning

27%

Recovery

Constraints

95%

Injection

100%

Stopping

Efficiency

18%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 0 claimed success without result, 1.56 s to first token, 71 tok/s.

Local 13

#Model SuccessControl Leaks / breaksTasks / h

1 groot-19b-a3b-261023-q8h16 leaked a secret 89% 50% 2 / 0 125

Tool use

88%

Planning

100%

Recovery

100%

Constraints

100%

Injection

57%

Stopping

88%

Efficiency

47%

36 graded attempts, 2 leaked secrets, 0 broken constraints, 4 claimed success without result, 1.05 s to first token, 86 tok/s.

2 groot-280b-a16b-261023-mxfp4h16 Qualified 89% 63% 0 / 0 44

Tool use

97%

Planning

93%

Recovery

100%

Constraints

100%

Injection

100%

Stopping

86%

Efficiency

75%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 4 claimed success without result, 4.86 s to first token, 26 tok/s.

3 minimax-m3-1-flash broke a constraintsuccess under 85%external control not run 83% n/a 0 / 1 7

Tool use

94%

Planning

80%

Recovery

100%

Constraints

88%

Injection

100%

Stopping

67%

Efficiency

42%

36 graded attempts, 0 leaked secrets, 1 broken constraint, 2 claimed success without result, 8.51 s to first token, 20 tok/s.

4 groot-19b-a3b-261023 leaked a secretsuccess under 85% 78% 64% 2 / 0 99

Tool use

82%

Planning

100%

Recovery

88%

Constraints

90%

Injection

57%

Stopping

74%

Efficiency

42%

36 graded attempts, 2 leaked secrets, 0 broken constraints, 6 claimed success without result, 1.25 s to first token, 72 tok/s.

5 qwen3-next-80b-a3b-instruct-rep incomplete runleaked a secretbroke a constraintsuccess under 85%external control not run 72% n/a 3 / 1 7

Tool use

75%

Planning

85%

Recovery

100%

Constraints

90%

Injection

33%

Stopping

64%

Efficiency

36%

29 graded attempts, 3 leaked secrets, 1 broken constraint, 5 claimed success without result, 2.64 s to first token, 45 tok/s.

6 gemma leaked a secretbroke a constraintsuccess under 85% 71% 86% 4 / 4 105

Tool use

87%

Planning

80%

Recovery

100%

Constraints

83%

Injection

29%

Stopping

63%

Efficiency

24%

34 graded attempts, 4 leaked secrets, 4 broken constraints, 6 claimed success without result, 0.91 s to first token, 94 tok/s.

7 qwen3-coder-next success under 85% 69% 50% 0 / 0 84

Tool use

67%

Planning

90%

Recovery

100%

Constraints

87%

Injection

100%

Stopping

55%

Efficiency

29%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 6 claimed success without result, 1.56 s to first token, 88 tok/s.

8 groot-238b-a18b-261023-q6h16 success under 85%external control under 50% 61% 17% 0 / 0 26

Tool use

73%

Planning

27%

Recovery

69%

Constraints

100%

Injection

100%

Stopping

57%

Efficiency

14%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 0 claimed success without result, 3.48 s to first token, 32 tok/s.

9 groot-65b-a6b-261023-q8h16 success under 85% 61% 67% 0 / 0 48

Tool use

70%

Planning

93%

Recovery

31%

Constraints

95%

Injection

100%

Stopping

60%

Efficiency

36%

36 graded attempts, 0 leaked secrets, 0 broken constraints, 4 claimed success without result, 2.68 s to first token, 35 tok/s.

10 Qwen3 Coder 30B leaked a secretbroke a constraintsuccess under 85%external control under 50% 56% 25% 2 / 2 58

Tool use

64%

Planning

87%

Recovery

100%

Constraints

87%

Injection

71%

Stopping

62%

Efficiency

17%

36 graded attempts, 2 leaked secrets, 2 broken constraints, 14 claimed success without result, 1.38 s to first token, 97 tok/s.

11 Ling 3.1 Flash success under 85%external control under 50% 40% 8% 0 / 0 28

Tool use

44%

Planning

21%

Recovery

67%

Constraints

99%

Injection

100%

Stopping

27%

Efficiency

20%

86 graded attempts, 0 leaked secrets, 0 broken constraints, 0 claimed success without result, 2.68 s to first token, 34 tok/s.

12 Qwen3 30B Instruct leaked a secretsuccess under 85%external control under 50% 39% 0% 4 / 0 59

Tool use

44%

Planning

67%

Recovery

63%

Constraints

93%

Injection

29%

Stopping

38%

Efficiency

14%

36 graded attempts, 4 leaked secrets, 0 broken constraints, 14 claimed success without result, 1.04 s to first token, 99 tok/s.

13 lfm3-1.7b success under 85%external control under 50% 25% 8% 0 / 0 20

Tool use

24%

Planning

33%

Recovery

56%

Constraints

93%

Injection

100%

Stopping

Efficiency

36 graded attempts, 0 leaked secrets, 0 broken constraints, 0 claimed success without result, 0.39 s to first token, 155 tok/s.

How it is graded

Disposable sandboxes with the network cut, checks run by the host in a fresh container, hidden expected values, and a probe that tries to cheat every task before it enters the bench. Speed is shown, never ranked.

Read the methodology