Articles on AI workflows, MCP patterns, and what we have learned building Synapse.
Most explanations cover one piece at a time. Here's the full data flow — from your prompt to the next generated token — traced through every component in order.
Every token an LLM generates reuses Keys and Values from everything that came before. The KV cache is what makes that reuse cheap. Here's how it works — and why inference slows down with longer context.
Read moreSelf-attention is blind to order. Shuffle the words in a sentence and you get identical attention scores. Positional embeddings solve this — but the way they do it determines whether your model can handle long contexts at inference time.
Read moreAttention gets all the press. But a transformer block is more than attention — there's a feedforward network that holds most of the parameters, two residual connections, and a normalisation design that determines whether large-scale training is stable. Here's all of it, in order.
Read moreMost developers treat LLMs as black boxes that write answers. The reality is stranger and more mechanical — and understanding it changes how you build.
Read moreA single attention pass can only ask one question at a time. Multi-head attention runs several in parallel — each head specialising in a different type of relationship. Here's what that means in practice.
Read moreStatic embeddings can't tell 'bank' the financial institution from 'bank' the riverbank. Self-attention is how language models fix that — by rewriting each token's meaning based on what surrounds it.
Read moreStay in the loop
It asks the kind of questions senior engineers do, sketches diagrams with you, and forces you to defend every architectural choice so your ideas are ready for interviews or production reviews.