# The ICPP Manual

ICPP is a C-like scripting language interpreted by `kcpp.exe -icpp <file>`. It shares kcpp's parser and
type system, adds exactly one new piece of syntax (`$var`, a dynamically-typed local), and exposes a large
library of native and script-composed builtin functions — database access, text/regex, image/OCR/biometric
utilities, and a full tensor/neural-network/autodiff engine.

This manual documents the language as it stands after extensive real-world testing. Every gotcha listed
under [Known limitations](#known-limitations) was found via an actual crash or wrong-answer repro, not
theorized — treat that section as load-bearing, not optional reading.

---

## 1. Running a script

```
kcpp.exe -icpp myscript.c
```

The file extension is conventionally `.c` (kcpp's normal parser handles it) or `.icpp` for
template-embedded use. A script needs a `main()`:

```c
int main(void)
{
    iprint("hello, world\n");
    return 0;
}
```

Diagnostics: set the environment variable `KCPP_DIAG_ICPP=1` before running to get a detailed trace of
parsing, identifier resolution, and expression evaluation — the single most useful tool for debugging a
script that produces the wrong number or crashes.

Teaching mode: `kcpp -icpp -llmtrace script.c` (or set `KCPP_LLMTRACE=1`) prints a detailed educational
walkthrough of every LLM-pipeline builtin as it runs — tokenizing, embedding lookup, positional encoding,
linear layers/weights, attention (per-head scores, causal masking, softmax weights), loss, the backward
pass, and the SGD update — including **why** each step exists, in student-facing language. Matrices are
printed up to 8×8 (corner + ellipsis beyond that). Designed for teaching how an LLM is built, using the
same scripts from §9 — output lines are prefixed `[llmtrace]` so they're easy to filter.

Multiple files: `#include "otherfile.c"` splices another ICPP source file in at that point (works for
both function definitions and top-level statements). This is how the reusable `icpp_autodiff.c` library
(§8) is normally pulled into a training script.

---

## 2. Language basics

ICPP is plain C syntax (functions, `int`/`double`/`char *` locals and parameters, `if`/`for`/`while`,
arithmetic, `#include`) plus one addition:

### `$var` — a dynamically-typed local

```c
$x = 5;
$y = tensor_get(handle, 0, 0);
```

A `$var` doesn't need a declaration and infers its type from whatever it's assigned. Internally it has a
single generic 8-byte storage slot, wide enough for an int, a double, or a handle.

**When to use `$var` vs. a plain typed local**: ICPP interprets the full C type system — `int`,
`double`, `char *`, arrays, structs, every declaration C allows — and that is not going anywhere. A
`$var` is an addition to it, not a replacement for it.

For the ICPP scripts **you** write, use `$var`. It is an ordinary mutable variable: assignments happen at
run time, in statement order, so it works anywhere a typed local does — reassignment, self-reference, a
loop counter, accumulation across iterations, assignment inside an `if` branch, and inside a function
body using that function's own parameters. Writing `$n` rather than `int n` is one less declaration to
keep in step with what the variable actually holds.

Reach for a typed local when the type is the point: a specific width or signedness, a pointer you intend
to do pointer arithmetic on, an array, or a struct. Function signatures are unaffected —
`int twice(int n)` is how a typed function is declared, and `function` declares a typeless one.

> **This advice changed.** Earlier versions of this manual said the opposite — use typed locals inside
> any function body or loop, and reserve `$var` for top-level scratch in `main()`. That was correct at
> the time and is no longer. A `$var` used to be written only by its **parse-time** initializer, so the
> first assignment won and every later one was silently dropped; `$v = f();` ran its call during
> parsing, before the statements above it had executed. That defect is fixed.
>
> `test/icpp_conformance/10_dollar_var.icpp` pins the current semantics with 16 checks covering exactly
> the cases the old advice steered around. Older ICPP that uses typed locals throughout still works and
> does not need changing, but new code should not copy the pattern.

`KamrangaC++/tools/icpplint.py` checks a file against this rule and reports anything that departs from
it.

### `function` — a typeless function definition

```c
function scale(x, f)
{
    return x * f;
}
```

In addition to ordinary typed C/C++ functions (which keep working exactly as before), ICPP scripts can
define **typeless functions** with the `function` keyword: no return type, no parameter types. Both are
determined **at runtime, per call** — the same call target can serve doubles on one call and ints on the
next:

```c
d = scale(2.5, 4.0);   /* double path: d = 10.0 */
i = scale(6, 7);       /* SAME function, int path: i = 42 */
m = scale(1.25, 2);    /* mixed args promote like C: 2.5 */
```

Rules and behavior:

- Top-level definitions only, and only in `-icpp` mode (in a plain kcpp compile the keyword is a clean
  error). `subroutine` is accepted as a synonym keyword.
- Parameter names are bare identifiers: `function f(a, b)`. Each parameter is the same generic 8-byte
  runtime-typed slot a `$var` uses, and its actual type (int/handle vs double) is stamped from whatever
  each call's argument evaluates to.
- `return expr;` returns whatever type `expr` evaluated to at runtime; a function with no `return` can be
  called as a plain statement.
- Typeless functions compose: they can call each other (including passing arithmetic expressions of their
  own typeless parameters as arguments) and recurse.
- Define before use, same as typed functions.
- Supported value kinds in v1: ints, handles, and doubles (the same scalar set typed ICPP functions
  support). Strings as typeless parameters/returns are not covered yet.
- For recursion, compute the recursive step through a local (`result = n * tfact(n - 1); return result;`)
  rather than `return n * tfact(n - 1);` — the single-expression form hits a separate, PRE-EXISTING
  interpreter bug that affects typed functions identically (see Known limitations).

### `rtlbegin` / `rtlend` — embedded verbatim Verilog

```c
rtlbegin
module hand_counter #(parameter WIDTH = 8) (
    input logic clk, input logic rst_n, output logic [WIDTH-1:0] count);
    always_ff @(posedge clk or negedge rst_n) begin
        if (!rst_n) count <= '0; else count <= count + 1'b1;
    end
endmodule
rtlend
```

Everything between the two keywords is **passed straight through** — the ICPP parser never tokenizes it,
so it may contain anything Verilog allows (`@`, `posedge`, `<=`, `32'sd1`, `always_ff`, `` `define ``,
`$display`) without the script caring, and the interpreter ignores the block completely. `kcpp -rtl`
copies it byte-for-byte into the `.v` output, **ahead of** the generated modules so those can instantiate
it.

That lets you hand-write the RTL that matters and use ICPP for everything around it: generated
combinational and FSM modules, plus the loops, file handling and assertions that drive the whole thing.
Multiple blocks are concatenated in source order. Both keywords are `-icpp`-mode only, so existing C code
using the name `rtlbegin` still compiles. An unterminated block is a real error naming the opening line.

### Handles

Nearly everything that isn't a plain number or string in this library — a tensor, a database connection,
a query result — is an **opaque handle**: a `long` returned by a constructor function, passed to every
function that operates on that object, and released with a matching `_free`/`_disconnect` call. Handles
are never dereferenced directly in script code; they're just numbers you pass around.

```c
int t;
t = tensor_new(2, 2);      /* t is now a handle */
tensor_set(t, 0, 0, 5.0);  /* operate on it via the handle */
tensor_free(t);            /* release it when done */
```

---

## 3. Script conventions

Everything in this chapter is a **convention**, not a syntax rule. ICPP interprets the full C type
system, and a script written entirely in typed C locals compiles and runs correctly. These are the habits
that make hand-written ICPP behave the way it reads, and each one exists because the alternative fails
quietly rather than loudly — which is the expensive kind.

`KamrangaC++/tools/icpplint.py` checks a file against all four. Its exit code is the number of problems,
so it drops into a build step or a pre-commit hook:

```sh
python KamrangaC++/tools/icpplint.py myscript.icpp
python KamrangaC++/tools/icpplint.py "www/*.icpp.html"
```

It reads only ICPP code — the whole file for a script, the `<?icpp ... ?>` blocks for a template — and
blanks comments and string bodies first, so an `int` inside a message cannot trip a rule.

### 3.1 Declare variables as `$var`

```c
$count = 0;
$name  = icpp_sql_get($cur, "display_name");
```

rather than:

```c
int count;
char *name;
count = 0;
```

A `$var` needs no declaration and takes its type from what it is assigned, so there is no second place to
keep in step when what the variable holds changes. See §2 for the semantics.

**Not flagged**, because the type carries meaning a `$var` cannot:

| | why |
|---|---|
| `int twice(int n)` | a function signature; `function` declares the typeless form |
| parameters | same |
| `int buf[64];` | an array has no `$var` form |
| `struct point p;` | names its own type, not a builtin one |
| a specific width, signedness, or a pointer you do arithmetic on | the type *is* the point |

### 3.2 Never assign to a bare identifier

```c
total = 5;        /* no */
$total = 5;       /* yes */
```

A bare name on the left of `=` is neither a declaration nor a `$var`. What it does is an interpreter
detail rather than something the script asked for, so it is worth never writing.

### 3.3 A `$var` you read is a `$var` you assigned

```c
$rows = 0;
iprint($rowz);    /* empty, and no error */
```

A misspelled `$var` is **not** an error in ICPP. It is simply empty. The script runs, the output has a
hole in it, and nobody notices until someone asks why a column is blank. The checker reports any `$var`
that is read but never assigned anywhere in the file, which catches the typo at the point it was made.

### 3.4 A `$var` is function-scoped

Its first assignment declares it, wherever that assignment happens, and it is
readable from there to the end of the function:

```c
if (ready) { $state = 1; }
iprint($state);            /* fine - $state outlived the block */
```

> Declaring one inside braces used to **segfault** — a plain block, an `if`
> branch, a loop body, a `switch` arm, all of them. The symbol went into the
> block's own table while only the function's locals get stack addresses, so
> the variable resolved with no address and the assignment wrote through NULL.
> Fixed, and pinned by six checks in `test/icpp_conformance/10_dollar_var.icpp`.
>
> Seeding a `$var` at the top of the function was the habit that avoided this,
> which is why the advice below is older than the explanation.

### 3.5 In a template, seed a `$var` at the top

This one applies to `<?icpp ... ?>` pages, where `icpptpl` folds every block on the page into a single
generated `main()`.

```c
$support = "support@far-reach.com";      /* seeded at the top */
$cur = icpp_sql_query($db, "select ...");
while (icpp_sql_next($cur))
{
    $support = icpp_sql_get($cur, "string_value");
}
```

Assigning inside a block works — §2 says so, and the conformance suite pins it. But a `$var` that only
comes into existence down one branch renders as nothing on every other branch, and a page is exactly the
place where that is easy to miss: there is no error, just a gap in the HTML. Seeding it at the top costs
one line and removes the question.

The checker reports this as a `note:` rather than a problem, and it does not affect the exit code.

### 3.6 Two things to know about `$var` in loops

Both are cheap to avoid and neither is caught by the checker, because both are
about ICPP's parser rather than about style.

**Seed a `$var` before a `for` uses it.** A `for` init cannot introduce one:

```c
for ($i = 0; $i < 4; $i++) { ... }        /* Error 69 if $i is new here */

$i = 0;                                    /* seed it first */
for ($i = 0; $i < 4; $i++) { ... }         /* fine */
```

This is the parser being helpful — it says `Misuse of '$' in 'for' statement`
rather than doing something surprising — and it is the same habit as §3.5.

**An unbraced body may be a `$var` statement**, in any of `for`, `while`,
`do`, `if` and `else`, and in a `switch` case arm:

```c
for ($i = 0; $i < 4; $i++) $sum = $sum + $i;
while ($i < 4) $i = $i + 1;
if (ready) $state = 1; else $state = 0;
switch ($kind) { case 1: $label = "one"; break; }
```

> Until recently none of these parsed — the single-statement body switches
> accepted `IDENTIFIER` and had no case for `DOLLAR`, so a `$` body was
> rejected with `Misuse of '$' in 'for' statement`, naming the enclosing
> statement rather than the body that caused it. Fixing that then uncovered
> two crashes underneath, in `while`/`do` and in `switch`, which had nothing
> to do with `$var` and took down plain typed C as well. All three are fixed;
> `01_core_language.icpp` pins them.

### 3.7 A `$var` may change type

Assigning a value of a different type replaces the old one completely — string
to number, number to string, double to int and back:

```c
$v = icpp_concat("ab", "cd");
$v = 0;                        /* 0 */
$v = 2.5;                      /* 2.5 */
$v = icpp_concat("x", "y");    /* "xy" */
```

> This used to be broken, and badly. The slot is a generic 8 bytes but a
> number was copied in at its own width, so the upper half of whatever was
> there before survived. A `$var` that had held a string then read back as a
> truncated pointer rather than as the number just assigned to it. Fixed; the
> five checks at the end of `test/icpp_conformance/10_dollar_var.icpp` pin it.

Reusing one name for genuinely different things is still worth avoiding, but
as a readability matter rather than a correctness one: a name that means a
string in one paragraph and a counter in the next is hard to follow in any
language.

### 3.8 What the checker does not decide

It has nothing to say about naming, spacing, brace style or comment density, and it will not rewrite a
file. It reports; you decide. Generated code and `test/icpp_conformance/*.icpp` are reasonable things to
leave out of it — the conformance suite exercises typed C deliberately, and a test that documents the
interpreter should not be edited into agreement with a style checker.

---

## 4. Known limitations

These are real, confirmed interpreter behaviors — not stylistic suggestions. Each was found via an actual
failing repro. (Two limitations formerly documented here — a braceless `if` body crashing inside a loop,
and a standalone `{ }` block silently not executing — have since been fixed; see the note below.)

### A bare `if` body inside a loop, and a standalone `{ }` block — FIXED

Both of the following used to be real bugs and are now fixed and covered by permanent regression tests
(`bare_if_in_loop_check.c`, `bare_block_check.c` in `test/PhaseA/`):

```c
for (i=0; i<3; i=i+1)
{
    if (i==0)
        x = 1;          /* used to crash the interpreter, no error message - now fine */
    iprint(x);
}
```

```c
int main(void)
{
    x = 5;
    {
        y = x + 1;      /* used to silently never execute - now fine, y is 6 */
    }
    iprint(y);
}
```

**Root causes** (both classic instances of this codebase's recurring "twin implementation" pattern, where
a codegen-mode function and its interpret-mode counterpart drift out of sync):

- The braceless-if-in-a-loop crash: `InterpretExecThenItem`/`InterpretExecElseItem` (`InternetCPP.c`)
  unconditionally called `ExecStatement` — the **codegen**-mode dispatcher, which for a bare
  (`STATEMENT_TYPE_PLAIN`) then/else body calls the codegen `EvalExpression` directly, relying on
  TAC/register-allocator state that's never initialized in `-icpp` mode — instead of branching on
  interpret mode the way the already-correct sibling `InterpretExecForItem` did. A *braced* body
  (`STATEMENT_TYPE_NEWBLOCK`) happened to survive because its handler, `ExecNewBlock`, has its own
  internal interpret/codegen split; a bare body had no such split anywhere in the call chain. Fixed by
  giving both `Item` functions the same interpret-mode branch `InterpretExecForItem` already had.
- The standalone-block no-op: `PROC_ParseCodeBlock` and `BLOCK_ParseCodeBlock` (`parse.c`) — the two
  parser functions for a bare `{ }` (at a function body's top level, and nested inside another block,
  respectively) — created their `STATEMENT_BLOCK` but, unlike every sibling `*_ParseIf`/`*_ParseFor`/
  `*_ParseWhile`, never set `statement_type` to `STATEMENT_TYPE_NEWBLOCK`. It defaulted to
  `STATEMENT_TYPE_PLAIN` (0), so the interpreter's statement dispatcher ran `InterpretEvalExpression` on
  the block's (`NULL`, since this statement represents a nested block, not an expression) `ExprStack` — a
  silent no-op — instead of ever calling `InterpretExecNewBlock` to walk the block's real contents. Fixed
  by adding the same `statement_type = STATEMENT_TYPE_NEWBLOCK` assignment every sibling function already
  had.

Bare (braceless) `if`/`for`/`while` bodies and standalone `{ }` scoping blocks are both fully supported
now — no style workaround needed.

### A recursive call inside a `return` expression returns garbage

```c
int fact(int n)
{
    if (n <= 1) { return 1; }
    return n * fact(n - 1);    /* fact(5) = 12960000, not 120 - typed AND typeless alike */
}
```

Evaluating a call to the same function inside a `return` expression clobbers the already-resolved sibling
operand (`n` here). This is a long-standing interpreter bug, NOT introduced by (and not specific to) the
typeless `function` feature — confirmed against a plain typed `int fact(int n)` with the identical wrong
result. Workaround (used by both recursion regression tests): compute through a local first —
`result = n * fact(n - 1); return result;`.

### A `char *` local assigned a string literal — FIXED

```c
char *p;
p = "hello world";
iprint(icpp_strlen(p));          /* 11 - correct */
iprint(icpp_strcmp(p, "hello world"));  /* 0 */
p = "changed";                   /* reassignment works */
iprint(isprint(p, "!"));         /* changed! */
```

This used to lose the string entirely (`icpp_strlen(p)` returned 0, and every file builtin reported
missing — which misleadingly looks like a *file* problem rather than a variable problem). Two root causes,
both fixed and covered by `charptr_string_check.c`:

- The interpret-mode ASSIGN fast path copied `number_info` bytes, but a string literal carries its text in
  `conststring_or_identifer` — and for a pointer target it sized the copy with
  `GetBasicTypeSize(CHAR_TYPE)` = **1**, i.e. a single byte. It now stores the string's address at
  pointer width.
- The read path tagged the result `NUMBER`, invisible to every string-consuming builtin. The string check
  also has to sit *above* the indirection-type switch, since a pointer never arrives as
  `INDIRECTION_TYPE_VALUE`.

`$var`s assigned a string work the same way now (the runtime-type stamp records `STRING`). String
literals themselves were never length-limited (120+ characters verified).

### `$var` inside a function body that uses its own parameters

```c
int f(int x)
{
    $y = x * 2;
    return $y;
}

function g(a, b)
{
    $z = a + b;
    return $z;
}
```

Both work. `f(21)` returns 42 and `g(3, 4)` returns 7.

> **This section used to say the opposite.** It documented the above as a parse error —
> `Initializing symbol 'y' must be constant number` — on the grounds that a `$var`'s initializer had to
> be resolvable at parse time, which a parameter's value never is, and it told you to use a typed local
> instead. That was true then. A `$var` is now assigned at run time like any other variable, so a
> parameter on the right-hand side is ordinary.

### Numeric arithmetic and argument passing — all now working correctly

Earlier in this project's history, `double` arithmetic (`+ - * /`), `double` function parameters/return
values, and decimal-literal handling all had real bugs (silently truncating to garbage or hanging on
division). **All of these are fixed as of this manual's writing** — real `double` arithmetic, parameters,
and return values all work correctly, including passing a `double` *variable* (not just a literal) as a
function argument. They're listed here only so a future contributor knows this ground has already been
covered and re-verified, not because they still need a workaround.

### Real function calls repeated in a loop, when very large

A large, deeply-nested user-defined function (many locals, many internal calls) called repeatedly inside a
loop can exhaust interpreter stack space. If a training loop or similar segfaults on the second iteration
after working fine once: **inline the loop body directly instead of wrapping it in one large function
call**, and double-check every `tensor_free()` for accidental double-frees (a much more common real cause
of the same symptom — verify with `grep -c "tensor_free(name)"` per variable, expecting exactly 1 each).

---

## 5. Core builtins reference

Every function below is a native sysfunc unless marked *(script)*, meaning it lives in `icpp_autodiff.c`
and is pulled in via `#include`. Argument types are handles unless noted otherwise.

### Output

| Function | Signature | Notes |
|---|---|---|
| `iprint` | `iprint(a, b, c, ...)` → void | Prints each argument concatenated, no format string. Strings pass through as-is; numbers auto-format (`%g` for doubles, `%d` for ints). The general-purpose print statement for ICPP scripts. |
| `ifprint` | `ifprint(path, a, b, ...)` → void | `iprint`'s print-to-**file** sibling: same argument formatting, appended to `path` as one line (file created if absent). Good for building transcripts/logs from a script. |
| `isprint` | `isprint(a, b, c, ...)` → `char *` | `iprint`'s build-a-**string** sibling (same implementation as `icpp_concat` — two names, one handler). |
| `printf` / `sprintf` | standard C | Only `%d`/`%s` actually substitute; other format specifiers are parsed but silently no-op — prefer `iprint`. |

### Strings + file comparison (RTL-verification helpers)

Built for verification workflows — comparing a ModelSim transcript or simulation log against an
expected-output file — but general-purpose.

| Function | Signature | Notes |
|---|---|---|
| `icpp_strcmp` | `(char *a, char *b)` → `int` | −1/0/1 (clamped) |
| `icpp_strstr` | `(char *hay, char *needle)` → `int` | 0-based index of first occurrence, −1 if absent |
| `icpp_strlen` | `(char *s)` → `int` | |
| `icpp_starts_with` / `icpp_ends_with` | `(char *s, char *fix)` → `int` | 1/0 |
| `icpp_file_read` | `(char *path)` → `char *` | whole file as text (bounded ~4KB — for bigger logs use the two below, which stream) |
| `icpp_file_contains` | `(char *path, char *needle)` → `int` | 1/0, −1 unreadable; streams, so huge sim logs are fine |
| `icpp_file_diff` | `(char *a, char *b)` → `int` | 0 = identical; else 1-based first differing line; −1 unopenable. Line endings normalized (CRLF vs LF compare equal). |

### ISQL (database) bridge

| Function | Signature |
|---|---|
| `icpp_sql_connect` | `(char *dsn)` → `int` connection handle |
| `icpp_sql_query` | `(int conn, char *sql)` → `int` result handle |
| `icpp_sql_next` | `(int result)` → `int` (1 = row available, 0 = done) |
| `icpp_sql_get` | `(int result, char *colname)` → `char *` |
| `icpp_sql_count` | `(int result)` → `int` row count |
| `icpp_sql_free` | `(int result)` → void |
| `icpp_sql_disconnect` | `(int conn)` → void |
| `icpp_sql_status` | `(int conn)` → `char *` last status/error message |

### Files, hashing, images, OCR, biometrics

| Function | Signature | Notes |
|---|---|---|
| `icpp_file_size` | `(char *path)` → `int` |
| `icpp_file_hash` | `(char *path)` → `char *` (FNV-1a hex digest) |
| `icpp_file_fingerprint` | `(char *path)` → `char *` |
| `icpp_fingerprint_similarity` | `(char *a, char *b)` → `int` |
| `icpp_fingerprint_match` | `(char *scan_path, char *template_path)` → `int` (NIST NBIS minutiae matcher) |
| `icpp_afis_extract` | `(char *image_path)` → `int` handle (0 = failure) — SourceAFIS-C port |
| `icpp_afis_minutiae` | `(int handle)` → `int` minutiae count (−1 = bad handle) |
| `icpp_afis_match` | `(int probe, int candidate)` → `double` score (≥ 40 accepts) |
| `icpp_afis_match_files` | `(char *p1, char *p2)` → `double` one-shot convenience |
| `icpp_afis_free` | `(int handle)` → void |
| `icpp_image_phash` | `(char *path)` → `char *` perceptual hash |
| `icpp_phash_similarity` | `(char *a, char *b)` → `int` |
| `icpp_image_crop` | `(char *in, int x, int y, int w, int h, char *out)` → `int` success |
| `icpp_ocr_extract` | `(char *image_path)` → `char *` (shells to Tesseract) |
| `icpp_re` | `(char *input, char *directive)` → `char *` — `"m/PATTERN/"` to match, `"s/PATTERN/REPLACE/"` to substitute |
| `icpp_concat` | `(a, b, c, ...)` → `char *` — builds a string from mixed strings/numbers |
| `icpp_getparam` | `(char *name)` → `char *` — reads a CGI `QUERY_STRING` parameter (iwebserver context) |

**Two independent fingerprint matchers.** `icpp_fingerprint_match` uses NIST NBIS (mindtct + bozorth3);
`icpp_afis_*` uses the in-tree SourceAFIS-to-C port. Their scores are **not comparable to each other** —
different algorithms, unrelated scales — but each separates genuine from impostor pairs on its own scale
(measured on real fingerprints: SourceAFIS 235.6 vs 3.2, NBIS 170 vs 5). The AFIS form is handle-based so
1:N identification extracts once and matches many times; the probe's edge hash is built on first use and
cached on that handle:

```c
int probe;
int candidate;
probe = icpp_afis_extract("probe.png");
candidate = icpp_afis_extract("candidate.png");
if (icpp_afis_match(probe, candidate) >= 40.0) { iprint("MATCH"); }
icpp_afis_free(probe);
icpp_afis_free(candidate);
```

### Running tools + test harness (RTL verification driving)

These make an ICPP script a drop-in replacement for a Perl/Python/shell regression driver: run a tool,
inspect what it produced, assert, and exit non-zero so CI fails.

| Function | Signature | Notes |
|---|---|---|
| `icpp_exec` | `(char *cmdline)` → `int` | Runs it, child output goes to your console, returns its exit code (−1 = could not start). |
| `icpp_exec_capture` | `(char *cmdline)` → `char *` | Returns the child's stdout **and stderr** as a string (compiler errors live on stderr). |
| `icpp_exec_status` | `()` → `int` | Exit code of the last `icpp_exec_capture` (which returns text, not a code). |
| `icpp_exec_to_file` | `(char *cmdline, char *path)` → `int` | Both streams into a file — use for big logs, then search them with the *streamed* `icpp_file_contains`/`icpp_file_diff`. |
| `icpp_assert` | `(int cond, char *msg)` → `int` | Prints `PASS:`/`FAIL:`, counts it. |
| `icpp_assert_eq_int` | `(int actual, int expected, char *msg)` → `int` | Prints **both** values on mismatch. |
| `icpp_assert_eq_str` | `(char *actual, char *expected, char *msg)` → `int` | Same, for strings. |
| `icpp_test_summary` | `()` → `int` | Prints `N checks: P passed, F failed`, returns **F**. |
| `icpp_exit` | `(int code)` → void | Terminates the run immediately with that status. |

### The `i` utilities — grep/cat/cut without leaving ICPP

A script should not have to shell out to `grep` or `sed` just to look
inside a file. Every string-returning one of these allocates a result
sized to what it produced, so `icat` of a 200 KB log returns 200 KB —
there is no 4 KB ceiling.

| Builtin | Signature | Does |
|---|---|---|
| `icat` | `(char *path)` → `char *` | The whole file. |
| `ihead` | `(char *path, int n)` → `char *` | First `n` lines. |
| `itail` | `(char *path, int n)` → `char *` | Last `n` lines — keeps a ring, so it works on a multi-GB transcript. |
| `igrep` | `(char *path, char *text)` → `char *` | The lines containing `text`. |
| `igrep_count` | `(char *path, char *text)` → `int` | How many lines contain it. |
| `icut` | `(char *text, char *delim, int field)` → `char *` | That 1-based field of every line. Takes **text**, so it composes: `icut(icat(f),":",3)`. |
| `ils` | `(char *dir)` → `char *` | One entry per line; directories end in `/`. |
| `ifind` | `(char *dir, char *pattern)` → `char *` | Matching paths, searched recursively (`*` and `?`). |
| `ireplace` | `(char *path, char *from, char *to)` → `int` | Byte-exact in-file replace; returns the count. |
| `ibash` | `(char *cmdline)` → `char *` | Runs the line under `bash -c` and returns its output. |
| `imake` | `(char *args)` → `int` | Runs **kmake** (not nmake, not mingw32-make); build output streams to the console. |
| `irunning` | `(char *who)` → `int` | How many matching processes are alive. `who` is a pid or an image name. |
| `ikill` | `(char *who)` → `int` | Kills them; returns how many. |

```c
/* did the simulation pass, and how many warnings? */
if (icpp_strstr(itail("vsim.log", 20), "ALL PASSED") >= 0)
    iprint("passed with ", igrep_count("vsim.log", "Warning"), " warnings\n");

/* the third colon-separated field of every matching line */
iprint(icut(igrep("parts.txt", "MRR"), ":", 3));

/* don't start a second simulator on top of a wedged one */
if (irunning("vsim.exe")) ikill("vsim.exe");
imake("makefile");
```

`icut` takes text while the others take a path — that is what lets you
chain them. `ibash` needs a bash on PATH (Git for Windows, WSL, MSYS2);
with none the result is empty and `icpp_exec_status()` is `-1`. `imake`
finds kmake via `$KMAKE_EXE`, then beside the running executable, then
PATH. `ikill` terminates without letting the target clean up — right for
a hung simulator, wrong for anything holding a file you care about, and
killing by NAME kills every copy.

### Threads and CPU affinity

Run several tools at once and pin them to cores — eight simulator runs, a
fitter sweep, a parallel build.

| Builtin | Signature | Does |
|---|---|---|
| `ithread_create` | `(char *cmdline)` → `int` | Starts the command on a thread; returns a handle, or `-1`. |
| `ithread_join` | `(int handle)` → `int` | Waits, returns **that command's** exit code. |
| `ithread_barrier` | `()` → `int` | Waits for every outstanding thread; returns how many. |
| `ithread_affinity_set` | `(int handle, int mask)` → `int` | `handle >= 0` pins that worker thread; `handle < 0` sets the **process** affinity. |
| `CPU_ZERO` / `CPU_SET` | `()` / `(int cpu, int mask)` → `int` | Build a mask. `CPU_SET` **returns** it — ICPP has no output parameters. |
| `icpu_count` | `()` → `int` | Logical processors, so a script can size its own fan-out. |
| `isleep` | `(int ms)` → `int` | Blocks for milliseconds. |

```c
$m = CPU_ZERO(); $m = CPU_SET(0, $m); $m = CPU_SET(1, $m);
ithread_affinity_set(-1, $m);        /* children inherit PROCESS affinity */

$a = ithread_create("vsim.exe -c -do run_a.do");
$b = ithread_create("vsim.exe -c -do run_b.do");
icpp_assert_eq_int(ithread_join($a), 0, "run A passed");
ithread_barrier();                    /* wait for the rest of the batch */
```

**A thread runs an external command, never an interpreted ICPP function.**
That is a deliberate limit: the interpreter is not reentrant — a call
writes `argument_stack_offset` and `MemberOf` into the *shared* function
prototype, so two threads inside the same ICPP function would corrupt each
other's parameter addresses silently. Running interpreted code on threads
needs per-thread interpreter state first.

To pin a **tool** to cores, set the *process* affinity (`handle < 0`)
before launching it — a child process inherits the process mask, never the
creating thread's. Pinning a worker thread only helps if that thread is
itself burning CPU, and here it mostly waits. Use `ithread_join`'s return
value rather than `icpp_exec_status()` with threads: that status is one
global and the threads race for it.

### Shell redirection in a command line

A command line may use the shell's own operators — `2>&1`, `>`, `>>`, `<`,
`|`, `&`, `&&`. When one is present the command is handed to the command
processor, so it means exactly what it means at a prompt:

```c
icpp_exec("vsim.exe -c -do run.do > sim.log 2>&1");  /* stderr into the log too */
icpp_exec("vlog.exe design.v 2> errors.txt");        /* stderr only              */
icpp_exec("kcpp.exe -rtl d.c | findstr Error");      /* pipes work as well       */
```

Without an operator the program is started directly, which is faster and
keeps a bare relative program name resolving against the current
directory. Note that `icpp_exec_capture` and `icpp_exec_to_file` **already**
merge `stderr` into what they collect, so `2>&1` is only needed there if
you want the shell to do something further with it.

A shell builtin (`dir`, `echo`, `copy`) is not a program, so it needs the
shell: give it an operator, or spell it `cmd /c dir ...`.

A complete ModelSim regression driver — this exact script is in `test/PhaseA/rtl_verify.c` and passes:

```c
rc = icpp_exec_to_file("kcpp.exe -rtl design.c", "gen.log");
icpp_assert_eq_int(rc, 0, "RTL generated");
rc = icpp_exec_to_file("vlog.exe -quiet -sv design.v design_tb.v", "vlog.log");
icpp_assert_eq_int(rc, 0, "vlog compiled");
icpp_assert_eq_int(icpp_file_contains("vlog.log", "** Error"), 0, "no compile errors");
icpp_exec_to_file("vsim.exe -c -quiet -do \"run -all; quit -f\" tb", "vsim.log");
icpp_assert(icpp_file_contains("vsim.log", "ALL PASSED"), "simulation self-check passed");
icpp_exit(icpp_test_summary());     /* exit status = failure count */
```

The idiomatic last line is `icpp_exit(icpp_test_summary());` — the process exit code becomes the number
of failed checks, which is exactly what a CI job needs.

### Lists, maps, split/join

Handle-based, like every other ICPP object. Stored strings are copied, so you may reuse your buffers.

| Function | Signature | Notes |
|---|---|---|
| `icpp_list_new` / `icpp_list_free` | `()` → handle / `(h)` → void | |
| `icpp_list_add` | `(h, char *s)` → `int` | returns the new count |
| `icpp_list_get` | `(h, int i)` → `char *` | out-of-range gives `""` |
| `icpp_list_count` / `icpp_list_sort` | `(h)` → `int` | sort is lexicographic, in place |
| `icpp_split` | `(char *text, char *sep)` → list handle | `sep` is a literal **string**, so `"a::b"` split on `"::"` gives 2 items; the trailing field is kept |
| `icpp_join` | `(h, char *sep)` → `char *` | |
| `icpp_map_new` / `icpp_map_free` | `()` → handle / `(h)` → void | linear-scan association array — sized for script use |
| `icpp_map_set` | `(h, char *key, char *val)` → `int` | **replaces** an existing key rather than duplicating it |
| `icpp_map_get` / `icpp_map_has` | `(h, char *key)` → `char *` / `int` | absent key gives `""` — use `has` to distinguish |
| `icpp_map_count` / `icpp_map_keys` | `(h)` → `int` / list handle | keys come back in insertion order; free the list |

### Hex and bit manipulation

| Function | Signature | Notes |
|---|---|---|
| `icpp_hex` | `(int v, int digits)` → `char *` | lowercase, zero-padded, **no** `0x` prefix; `digits<=0` means "as needed". `icpp_hex(-1,8)` → `"ffffffff"` |
| `icpp_parse_hex` | `(char *s)` → `int` | accepts `ff`, `FF`, `0xff`, leading spaces |
| `icpp_bits` | `(int v, int hi, int lo)` → `int` | Verilog's `v[hi:lo]`, so the script and the RTL describe a field identically |
| `icpp_bin` | `(int v, int width)` → `char *` | MSB-first `0`/`1` text |

Parsing a simulation transcript — the job normally written in Perl:

```c
fields = icpp_split(icpp_file_line("sim.log", i), " ");     /* "# 100 ns SIG=top.dut.addr VAL=deadbeef" */
kv     = icpp_split(icpp_list_get(fields, 4), "=");
icpp_map_set(seen, name, icpp_list_get(kv, 1));
val = icpp_parse_hex(icpp_map_get(seen, "top.dut.addr"));   /* 0xdeadbeef */
iprint("upper nibble=", icpp_hex(icpp_bits(val, 31, 28), 1));   /* d */
iprint("ctrl[7:4]=",    icpp_bits(ctrl, 7, 4));                 /* 10 */
```

### MCP client + JSON (connect ICPP to LLM tool servers)

MCP (Model Context Protocol) is the standard way LLM applications talk to tool servers. ICPP is an MCP
**client** over the stdio transport: it spawns the server as a child process and speaks newline-delimited
JSON-RPC 2.0 over its stdin/stdout (initialize handshake included — the handle returned by
`icpp_mcp_connect` is immediately ready to use).

| Function | Signature | Notes |
|---|---|---|
| `icpp_mcp_connect` | `(char *cmdline)` → `int` handle | Spawns the server (`"my_server.exe --flag"`), performs the MCP `initialize` handshake. 0 = failure. A bare relative exe name is resolved against the current directory. |
| `icpp_mcp_list_tools` | `(int h)` → `char *` | The server's raw `tools/list` JSON — drill in with `icpp_json_get`. |
| `icpp_mcp_call` | `(int h, char *tool, char *argsjson)` → `char *` | Calls a tool; returns `result.content[0].text` (or the raw response if the server used another shape). If `argsjson` contains no `"` you may write it with single quotes — `"{'a':3,'b':4}"` — converted automatically (avoids escaping quotes in script literals). |
| `icpp_mcp_disconnect` | `(int h)` → void | Closes the server's stdin (EOF), waits, terminates a straggler. |
| `icpp_json_get` | `(char *json, char *path)` → `char *` | Extract any value by dotted path, e.g. `"result.tools.0.name"` (numeric segment = array index). Strings come back unescaped; objects/arrays come back as raw JSON for further drilling. Works on any JSON, not just MCP. |
| `icpp_openai_chat` | `(char *base_url, char *api_key, char *model, char *prompt)` → `char *` | One-call OpenAI-compatible chat completion: POSTs `{base_url}/v1/chat/completions` (WinHTTP — the OS handles TLS), returns the assistant's reply (`choices[0].message.content`), or the raw error JSON, or `""` on transport failure. Works against `https://api.openai.com` (key required) and local OpenAI-compatible servers — `icpp_openai_chat("http://localhost:11434", "", "llama3", "hi")` for Ollama, LM Studio on `:1234`, etc. Empty key = no Authorization header. |

```c
int h;
h = icpp_mcp_connect("test_mcp_server.exe");
iprint(icpp_json_get(icpp_mcp_list_tools(h), "result.tools.0.name")); /* add */
iprint(icpp_mcp_call(h, "add", "{'a':3,'b':4}"));                     /* 7   */
icpp_mcp_disconnect(h);
```

### Chatbot APIs (`icpp_chatbot.c` script library + native support)

`#include "icpp_chatbot.c"` gives you a customer-support bot's three legs. The library itself is written
with ICPP's typeless `function` feature; these native builtins support it:

| Function | Signature | Notes |
|---|---|---|
| `icpp_text_embed` | `(char *text, int dim)` → tensor `[1,dim]` | Hashed character-bigram embedding (deterministic, training-free; lowercased, whitespace-collapsed). Use dim 256. |
| `tensor_cosine` | `(a, b)` → `double` | Cosine similarity of two same-shaped tensors, the ranking half of retrieval. |
| `tensor_save` / `tensor_load` | `(handle, char *path)` → int / `(char *path)` → handle | Persist any tensor as inspectable text (`ICPPTENSOR rows cols` + one `%.17g` per line). **This is how an LLM trained by one ICPP script is used by another.** |
| `icpp_file_line` / `icpp_file_lines` | `(char *path, int n)` → `char *` / `(char *path)` → `int` | 1-based line access / line count — the KB store and general log inspection. |

Library functions (all typeless `function`s):

- **RAG retrieval** — KB is `bot_kb.txt`, one fact per line, appended with `ifprint`;
  `bot_kb_best(qvec)` returns the best-matching line number:
  ```c
  ifprint("bot_kb.txt", "We open at 9am and close at 6pm every day");
  best = bot_kb_best(icpp_text_embed(question, 256));
  iprint("bot: ", icpp_file_line("bot_kb.txt", best));
  ```
- **Talk to an ICPP-created LLM** — `bot_llm_generate(emb, w, b, seed_id, count, out_ids)` runs the
  next-char generation loop (embedding_lookup → linear → argmax) over weights another script trained and
  `tensor_save`d; decode `out_ids` with `tensor_detokenize`.
- **ISQL tickets** — log exchanges / pull account facts with the existing `icpp_sql_*` bridge, composing
  SQL with `isprint(...)`.
- **Remote-LLM brain when wanted** — `icpp_openai_chat(url, key, model, isprint("Context: ", fact, " Question: ", q))`.

### Graphics canvas (software rendering → PNG)

Handle-based RGB canvas; colors are `0xRRGGBB` ints (red `16711680`, green `65280`, blue `255`). Save as
a real PNG (vendored `stb_image_write`) and serve it from an iwebserver page as `<img src=...>`.

| Function | Signature |
|---|---|
| `icpp_img_new` | `(int w, int h)` → handle (white canvas) |
| `icpp_img_set` / `icpp_img_get` | `(h, x, y, rgb)` → int / `(h, x, y)` → rgb (−1 out of bounds) |
| `icpp_img_line` | `(h, x0, y0, x1, y1, rgb)` — Bresenham |
| `icpp_img_rect` / `icpp_img_fill_rect` | `(h, x, y, w, ht, rgb)` |
| `icpp_img_save_png` | `(h, char *path)` → 1/0 |
| `icpp_img_free` | `(h)` → void |

(`icpp_img_text` is deliberately not in v1 — a bitmap font is its own task; see plan Part 20.)

---

## 6. Tensor engine — core operations

A tensor is a 2D, row-major array of `double`, referenced by an opaque `long` handle. All shapes are
`[rows, cols]`.

| Function | Signature | Notes |
|---|---|---|
| `tensor_new` | `(int rows, int cols)` → handle | Zero-filled. |
| `tensor_free` | `(handle)` → void | |
| `tensor_get` | `(handle, int row, int col)` → `double` | |
| `tensor_set` | `(handle, int row, int col, double value)` → `int` (1=ok) | |
| `tensor_rows` / `tensor_cols` | `(handle)` → `int` | |
| `tensor_matmul` | `(A, B)` → handle | Standard matrix multiply, `A.cols` must equal `B.rows`. |
| `tensor_add` / `tensor_sub` | `(A, B)` → handle | Elementwise, identical shapes required (no broadcasting). |
| `tensor_scale` | `(handle, double factor)` → handle | Elementwise scalar multiply. |
| `tensor_transpose` | `(handle)` → handle | `[r,c]` → `[c,r]`. |
| `tensor_relu` | `(handle)` → handle | Elementwise `max(0,x)`. |
| `tensor_softmax` | `(handle)` → handle | Row-wise, numerically stable. |
| `tensor_argmax` | `(handle, int row)` → `int` | Column index of the largest value in that row — "which class." |
| `tensor_linear` | `(x, weight, bias)` → handle | `x @ weight + bias`, bias `[1,out]` broadcast down every row. The standard trainable projection layer. |

---

## 7. Tensor engine — neural network / LLM layers

| Function | Signature | Notes |
|---|---|---|
| `tensor_layernorm` | `(x, gamma, beta, double eps)` → handle | Row-wise normalize to mean 0/var 1, then scale/shift. `gamma`/`beta` are `[1,cols]`. |
| `tensor_embedding_lookup` | `(table, ids)` → handle | `table` is `[vocab,d_model]`, `ids` is a `[1,n]` row of token IDs (doubles, rounded). Gathers `n` rows. |
| `tensor_causal_mask` | `(scores)` → handle | Forces upper-triangular (future) positions to a large negative number, for autoregressive attention. |
| `tensor_positional_encoding` | `(int seq_len, int d_model)` → handle | Fixed sinusoidal encoding, no learned parameters. |
| `tensor_multihead_attention` | `(Q, K, V, int num_heads, int causal)` → handle | Real per-head column-slicing. Q's row count may differ from K/V's (cross-attention). |
| `tensor_self_attention` | `(x, wq,bq,wk,bk,wv,bv,wo,bo, int num_heads, int causal)` → handle | Full trainable layer: Q/K/V/output projections + attention, all from one input `x`. |
| `tensor_cross_attention` | `(query_x, kv_x, wq,bq,wk,bk,wv,bv,wo,bo, int num_heads, int causal)` → handle | Same as above, but Q comes from `query_x` and K/V from a different sequence `kv_x`. |
| `tensor_tokenize` | `(char *text)` → handle | Byte-level: a `[1,strlen(text)]` tensor of raw byte values (vocab_size=256, no OOV case). |
| `tensor_detokenize` | `(handle)` → `char *` | Exact inverse of `tensor_tokenize`. |
| `tensor_cross_entropy_loss` | `(logits, targets)` → `double` | `logits` is `[seq_len,vocab]`, `targets` is `[1,seq_len]`. Returns the **mean** per-token loss. |

**Scope note**: attention's *backward* pass (§8) is only mathematically correct for `num_heads=1`. Forward
attention supports any `num_heads`; if you need to train a true multi-head model, a native backward op
mirroring the per-head column-slice loop still needs to be written.

---

## 8. Autodiff (backward / gradient rules)

This library implements **explicit backprop**, not an automatic-differentiation graph: you save whatever
forward tensors a backward call needs, and call the backward functions yourself, in reverse order. Every
rule below was verified by numerical gradient checking (perturb an input, compare the finite-difference
slope to the analytical gradient) — not just formula inspection.

### Native (in `icpptensor.c`)

| Function | Signature | Gradient of |
|---|---|---|
| `tensor_relu_backward` | `(x, dy)` → `dx` | `relu(x)`. Needs the forward **input** `x`. |
| `tensor_softmax_backward` | `(y, dy)` → `dx` | `softmax(x)`. Needs the forward **output** `y`. |
| `tensor_layernorm_backward_x` | `(x, gamma, dy, double eps)` → `dx` | |
| `tensor_layernorm_backward_gamma` | `(x, dy, double eps)` → `dgamma` | |
| `tensor_layernorm_backward_beta` | `(dy)` → `dbeta` | Pure sum-over-rows; also reused for any broadcast-bias gradient. |
| `tensor_embedding_lookup_backward` | `(table, ids, dy)` → `dtable` | Scatter-**add** — a repeated token ID correctly accumulates gradient from every occurrence. |
| `tensor_cross_entropy_loss_backward` | `(logits, targets)` → `dlogits` | `(softmax(logits) - one_hot(target)) / N`. No upstream gradient argument — this is always the last op in a graph. |

### Script-composed *(in `icpp_autodiff.c`, `#include` it)*

These need no native code — they compose entirely from the primitives above.

| Function | Signature | Gradient of |
|---|---|---|
| `tensor_matmul_backward_a` | `(A, B, dC)` → `dA` | `C = A @ B`. `dA = dC @ B^T`. |
| `tensor_matmul_backward_b` | `(A, B, dC)` → `dB` | `dB = A^T @ dC`. |
| `tensor_add_backward` | `(dC)` → a fresh copy of `dC` | `C = A + B` — identity gradient to both operands (call once per operand needed). |
| `tensor_linear_backward_x` | `(x, w, dy)` → `dx` | `tensor_linear`'s input gradient (= `matmul_backward_a`; the bias term drops out). |
| `tensor_linear_backward_w` | `(x, w, dy)` → `dw` | (= `matmul_backward_b`). |
| `tensor_linear_backward_b` | `(dy)` → `db` | (= `layernorm_backward_beta`, reused). |
| `tensor_attention_backward_q/k/v` | `(q, k, v, dout, double scale)` → `dq`/`dk`/`dv` | Single-head scaled-dot-product attention. Recomputes the forward pass internally (no saved intermediates needed from the caller). |

---

## 9. Optimizer

| Function | Signature | Notes |
|---|---|---|
| `tensor_sgd_step` | `(param, grad, double lr)` → void | **Mutates `param` in place** — the one deliberate exception to every other `tensor_*` op's "always return a fresh handle" rule. A parameter handle is meant to persist across thousands of training steps, exactly like every real ML framework's `optimizer.step()`. |

There is no `zero_grad` — this library has no persistent per-parameter gradient accumulator to zero.
Gradient accumulation across a mini-batch is just ordinary `tensor_add` into a fresh (already
zero-filled) `tensor_new` accumulator.

---

## 10. Worked example — a tiny trainable classifier

A complete, tested, 2-layer MLP that trains to classify three 8×8 patterns (top-lit / bottom-lit /
checkerboard) via real backprop and SGD:

```c
#include "icpp_autodiff.c"

int init_small(int rows, int cols, int seedoffset)
{
    int t; int i; int j; int idx; double value;
    t = tensor_new(rows,cols);
    for (i=0; i<rows; i=i+1)
    {
        for (j=0; j<cols; j=j+1)
        {
            idx = (i*13+j*29+seedoffset*7) % 17;
            value = (double)(idx-8) * 0.02;
            tensor_set(t,i,j,value);
        }
    }
    return t;
}

int main(void)
{
    int images; int labels;
    int w1; int b1; int w2; int b2;
    int hidden1; int hidden1_relu; int logits;
    double loss;
    int dlogits; int dhidden1_relu; int dw2; int db2; int dhidden1_lin; int dw1; int db1;
    int step;

    /* ... build `images` [3,64] and `labels` [1,3] here ... */

    w1 = init_small(64,16,1); b1 = tensor_new(1,16);
    w2 = init_small(16,3,2);  b2 = tensor_new(1,3);

    for (step=0; step<100; step=step+1)
    {
        hidden1 = tensor_linear(images,w1,b1);
        hidden1_relu = tensor_relu(hidden1);
        logits = tensor_linear(hidden1_relu,w2,b2);
        loss = tensor_cross_entropy_loss(logits,labels);

        dlogits = tensor_cross_entropy_loss_backward(logits,labels);
        dhidden1_relu = tensor_linear_backward_x(hidden1_relu,w2,dlogits);
        dw2 = tensor_linear_backward_w(hidden1_relu,w2,dlogits);
        db2 = tensor_linear_backward_b(dlogits);
        dhidden1_lin = tensor_relu_backward(hidden1,dhidden1_relu);
        dw1 = tensor_linear_backward_w(images,w1,dhidden1_lin);
        db1 = tensor_linear_backward_b(dhidden1_lin);

        tensor_sgd_step(w1,dw1,0.02); tensor_sgd_step(b1,db1,0.02);
        tensor_sgd_step(w2,dw2,0.02); tensor_sgd_step(b2,db2,0.02);

        tensor_free(hidden1); tensor_free(hidden1_relu); tensor_free(logits);
        tensor_free(dlogits); tensor_free(dhidden1_relu); tensor_free(dw2); tensor_free(db2);
        tensor_free(dhidden1_lin); tensor_free(dw1); tensor_free(db1);
    }
    return 0;
}
```

The full version (with data setup) is at `KamrangaC++/test/PhaseA/dnn_mlp_check.c` — it's also the
project's own regression test, converging to loss `0.0002` and 100% classification accuracy by step 70.

For a full transformer training loop (tokenizer → embeddings → transformer block → LM head → loss →
backward → SGD, over multiple steps with the loss printed and verified to decrease), see
`KamrangaC++/test/PhaseA/training_loop_check.c`.

---

## 11. Verifying your own changes

If you add a new backward rule or numeric op, verify it with a numerical gradient check, not just
eyeballing the formula — perturb one input by a small epsilon (`0.0001` works well), recompute the
forward pass, and confirm `(f(x+eps) - f(x-eps)) / (2*eps)` matches your analytical gradient. Every
backward rule in §8 was verified this way; it's the only reliable way to catch a sign error or an
off-by-one in an index.

The full regression suite lives at `KamrangaC++/test/PhaseA/run_regression.sh` — run it after any change
touching the interpreter, sysfuncs, or the tensor engine.
