Hello, World

This is a test post to make sure the MDX pipeline works. Mostly as a means for me to check if anything went wrong while I work on the website.

LaTeX

The website supports inline math $E = mc^2$ latex rendering with katex and kmath

Display math:

$$\int_{-\infty}^{\infty} e^{-x^2} \, dx = \sqrt{\pi}$$

References

This paragraph has a reference11. Hello. This is a Tufte style layout. Any reference in the paragraph show up like this.22. Hi. Another footnote here. This layout is also known as Marginalia. in it but you might not be able to see them if you’re on a phone so you can just tap on them to open them!

Code

Inline console.log("hello") and some code blocks:

from jaxtyping import Float, Int, Bool
from torch import Tensor


def batched_gather(
    x: Float[Tensor, "bsz sl emb"],
    indices: Int[Tensor, "bsz idxs"],
) -> Float[Tensor, "bsz idxs emb"]:
    """Gather along sequence dimension."""
    return x.gather(1, indices.unsqueeze(-1).expand(-1, -1, x.size(-1)))


class ResidualBlock(nn.Module):
    """Pre-norm residual block with feed-forward."""

    def __init__(self, dim: int, d_ff: int = 3072):
        super().__init__()
        assert dim % 2 == 0, f"{dim} must be even"
        self.ln = nn.LayerNorm(dim)
        self.ff = nn.Sequential(
            nn.Linear(dim, d_ff), # proj-cast
            nn.GELU(),
            nn.Linear(d_ff, dim), # undo cast
        )

    def forward(
        self,
        x: Float[Tensor, "bsz seq emb"],
        mask: Bool[Tensor, "bsz sl"] | None = None,
    ) -> Float[Tensor, "bsz sl emb"]:
        return self.ln(x + self.ff(self.ln(x)))

@torch.compile(fullgraph=False)
def train_step(xb, yb, model, optimizer):
    logits = model(xb)
    ce_loss = F.cross_entropy(
        logits.view(-1, vocab_size),
        yb.view(-1),
    )
    return ce_loss

Lists

• Item one
• Item two
• Item three

This is a blockquote. It should render with a left border and italic text.