Layers and optimizers

On top of the raw tape, Talario ships an ergonomic layer API (defined in php/talario.php) with device-resident, trainable Parameters.

Layers#

• Linear($dev, $in, $out, $bias = true) -- dense y = x @ W (+ b), He-initialized. invokeReLU() runs the fused relu(x@W + b) in a single dispatch.
• Conv2d($dev, $cin, $cout, $kh, $kw, $stride, $pad) -- 2-D convolution via im2col + GEMM.
• MaxPool2d($kh, $kw, $stride, $pad) -- spatial max pooling over NHWC maps.
• Activation($kind) -- a pointwise relu/sigmoid/tanh as a callable layer.
• Sequential([...]) -- a feed-forward stack that fuses Linear followed by ReLU.

use Talario\{Sequential, Linear, Activation, Adam};

$net = new Sequential([
    new Linear($dev, 784, 128),
    new Activation('relu'),
    new Linear($dev, 128, 10),
]);
$opt = new Adam($net->parameters(), 1e-3);

Optimizers#

• SGD($layers, $lr) -- batched in-place w -= lr * grad.
• Adam($layers, $lr, $beta1, $beta2, $eps) -- batched in-place adaptive-moment update.

Both collect the parameters from the layers you pass and apply one batched, on-device update per step(), so the optimizer is a single submission rather than one per parameter.

foreach ($batches as [$x, $y]) {
    $t = new Tape($dev);
    $logits = $net($t, $t->tensor($x, [$n, 784]));
    $loss = $t->softmaxCrossEntropy($logits, $t->tensor($y, [$n, 10]));
    $t->backward($loss);
    $opt->step();
}

Saving and restoring#

Sequential::stateDict() collects every layer's weights as plain arrays; loadState() restores them into a same-architecture model.

Edit this page on GitHub