Training data from a scripted expert

LoRA makes fine-tuning fit on a T4. Now we need something to fine-tune on: transcripts where a model reads the repository, calls tools in the right format, and finishes the task. This post builds 74 of them.

Both planned sources fail

The plan offered two sources. Neither works here.

• Filtered runs of the base model. Run Qwen on training tasks and keep the runs that pass. The base model scores pass^1 0.025 on the Day 7 eval. At that rate, 74 training tasks would yield about two passing runs.
• A stronger model through an API. Anthropic's Commercial Terms, section D.4, forbid using the service "to build a competing product or service, including to train competing AI models". A larger open Qwen would work legally, but it needs another architecture check and GPU hours just to generate data.

So octlm takes a third route. Every training task already has a reference solution written as tool calls, because task validation needs one. A scripted expert replays that solution through the real harness tools, in a fresh sandbox, and records what happened as a conversation. That's behavior cloning from a hand-written policy. It costs no GPU time and raises no license question.

The expert has to act like a model could

A replayed solution teaches whatever it contains, including shortcuts. Two rules make the traces teach the behavior Day 7 found missing.

1. Read before you write. Day 7's reference solutions write a file without reading it first. They were written to check the checks, and the author already knew the file. A model doesn't. Every training edit is preceded by a read_file of the same path. Bug fixes run the tests first, read the file, write the fix, and run the tests again.
2. One call per turn, in the template's exact format. Each assistant turn is <tool_call>, a newline, the JSON, a newline, and </tool_call>. The JSON comes from json.dumps, which escapes every newline inside a string. Raw newlines in write_file content were half of Day 7's invalid calls.

Each trace must pass its own task check before it is kept. A trace that fails stops the stage.

def trace(task: dict, fixtures: Path) -> list[dict]:
    messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": task["prompt"]}]
    with sandbox(task, fixtures) as root:
        for call in task["solution"]["calls"]:
            text = call_text(call)
            if parse_calls(text) != [call]:
                raise ValueError(f"trace call does not round-trip in task {task['id']}")
            result, ok = execute(root, call)
            if not ok:
                raise ValueError(f"trace call failed in task {task['id']}: {result}")
            messages.append({"role": "assistant", "content": text})
            messages.append({"role": "tool", "content": result})
        messages.append({"role": "assistant", "content": task["solution"]["answer"]})
        if not passes(task, root, task["solution"]["answer"], fixtures):
            raise ValueError(f"trace fails its own check in task {task['id']}")
    return messages

A different repository

If training tasks overlap the eval, the score measures memory, not skill. The eval runs on shop, a toy store package. Training uses fixtures/day8/repo, a different package called library, with its own config, data files, four modules and 14 tests. No training task shares a repository, file, function name, prompt or answer with an eval task, and the trace stage refuses to run if any training prompt equals an eval prompt.

The task kinds do repeat. Both sets have config lookups, file edits, new functions and planted bugs, because the point is to teach the tool protocol, not new skills. So the eval measures transfer to a new repository, not to new kinds of task.

74 of 74 training tasks passed validation: each check fails on the untouched repository and passes after its solution.

One example per assistant turn

The obvious move is to render each conversation once and train on the whole thing. Qwen's chat template breaks that. It prints an empty <think>\n\n</think> block on the turn being generated, and drops it from every earlier assistant turn. A conversation rendered in one piece shows every turn but the last in a form the harness never produces at inference time.

So each assistant turn becomes its own example. The prompt is the conversation up to that turn, rendered with the generation prompt exactly as the harness renders it. The target is the reply plus <|im_end|>. The code encodes prompt and target as one string, then checks that the prompt's token IDs are a prefix of the result, so the boundary can never split a BPE merge.

The loss counts target tokens only. QLoRA found that training on targets alone beat training on prompt and target together on all four datasets it tried.

You are a coding agent working in a small Python repository. Use the tools to inspect and change files. Paths are relative to the repository root. When the task is done, reply with a short final answer and no tool call. … plus the tool list

due_in returns negative days for fresh loans. Run the tests, find the bug, and fix the code. Do not edit the tests.

<tool_call>
{"name": "run_tests", "arguments": {}}
</tool_call>

exit code 1
.....F........
======================================================================
FAIL: test_due_in (tests.test_loans.LoansTest.test_due_in)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/tmpro9g_irv/repo/tests/test_loans.py", line 8, in test_due_in
    self.assertEqual(due_in(5), 16)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
AssertionError: -16 != 16

----------------------------------------------------------------------
Ran 14 tests in 0.001s

FAILED (failures=1)

<tool_call>
{"name": "read_file", "arguments": {"path": "library/loans.py"}}
</tool_call>

MAX_DAYS = 21
FINE_PER_DAY = 0.25


def due_in(days_out):
    return days_out - MAX_DAYS


def fine(days_late):
    return round(max(0, days_late) * FINE_PER_DAY, 2)


def can_borrow(count, limit=5):
    return count < limit

<tool_call>
{"name": "write_file", "arguments": {"path": "library/loans.py", "content": "MAX_DAYS = 21\nFINE_PER_DAY = 0.25\n\n\ndef due_in(days_out):\n    return MAX_DAYS - days_out\n\n\ndef fine(days_late):\n    return round(max(0, days_late) * FINE_PER_DAY, 2)\n\n\ndef can_borrow(count, limit=5):\n    return count < limit\n"}}
</tool_call>

wrote 225 characters to library/loans.py

<tool_call>
{"name": "run_tests", "arguments": {}}
</tool_call>

exit code 0
..............
----------------------------------------------------------------------
Ran 14 tests in 0.000s

OK

Fixed.

Most of every example is context

Targets are 4.6 percent of the tokens. Every prompt opens with the same 495-token system turn and tool list, so 223 short replies cost 159,856 tokens of compute per epoch. That's still small. Three epochs take under five minutes on a T4.

Every trace passed its own check, and no prompt matched an eval prompt. Decision: keep.

The traces are not byte-for-byte reproducible

The plan claimed the trace file would be identical on every run. It isn't. The table above comes from the laptop. The Kaggle runs built the same traces with a different total, because three things in tool output vary:

• unittest prints Ran 14 tests in 0.000s on some runs and 0.001s on others.
• Tracebacks name the sandbox, a random temporary directory that changes every run.
• The laptop's Python adds ~~~^^^ marker lines under traceback frames, and Kaggle's Python 3.12 mostly doesn't.

The targets don't change, because the loss mask skips tool results. All runs train on the same 7,287 target tokens. The context around them differs by a few tokens per run. Stripping the timing and the sandbox path would fix it, but that means editing the Day 7 harness, and the eval has to stay frozen.

Not built

• Model-written traces. A stronger open model could write more varied traces than a script. It costs GPU hours, and the scripted traces already passed the exit check.
• Failed traces. Some recipes train on recoveries from errors. Every octlm trace succeeds on the first try, so the model never sees a tool error being fixed.
• More tasks. 74 tasks over four kinds is enough to teach a format. It isn't enough to teach debugging, and the results show it.