Nassim Arifette

Introduction to Category Theory for AI

Thu, 25 Dec 2025 00:00:00 GMT

This series delves into the intersection of category theory and artificial intelligence, providing insights into how abstract mathematical concepts can inform and enhance the design of AI systems. Each part builds on the previous one, so start at the top if you want the full context.

What Happens to Currying When Beta and Eta Become Computations?

Thu, 25 Dec 2025 00:00:00 GMT

Fix a type $O$, a closed function $f:O\Rightarrow O$, and one free input $y:O$. Consider the term $$ T(y) = (\lambda u:O.\,u) \big((\lambda v:O.\,v)(f\,y)\big). $$ It contains two beta redexes. We can contract the outer one first, $$ T(y) \Rightarrow_\beta (\lambda v.v)(f\,y) \Rightarrow_\beta f\,y, $$ or the inner one first, $$ T(y) \Rightarrow_\beta (\lambda u.u)(f\,y) \Rightarrow_\beta f\,y. $$ Ordinary beta equality tells us that $T(y)=f\,y$. That statement is correct, but it does not tell us that two contractions occurred, or that they can occur in either order. Robert Seely's 1987 paper [*Modelling Computations: A 2-Categorical Framework*](https://www.math.mcgill.ca/~rags/WkAdj/LICS.pdf) asks what happens when we refuse to erase that information. His answer changes the usual roles of beta and eta. They stop being equations that justify a cartesian closed structure and become directed cells that build a two-dimensional version of it. After reading this article, you should be able to use the term $T$ to explain four points: 1. why a conversion is naturally a 2-cell; 2. why substitution forces some reduction histories to be identified; 3. why eta and beta become the unit and counit of an adjunction between currying and uncurrying; 4. why $E\Rightarrow-$ preserves composition through a beta comparison rather than by equality. The intended reader knows typed lambda calculus, substitution, beta and eta, and the ordinary idea of currying. Some basic category theory is helpful, but the category-theoretic notation needed for the central calculation is explained as it appears. No domain theory or 2-category theory is assumed. ## 1. What does equality forget? Start with the smaller redex $$ (\lambda u:O.\,u)x \Rightarrow_\beta x. $$ In the usual equational theory, we quotient terms by beta conversion. The two expressions then represent the same term. In a cartesian closed category, they are interpreted by the same morphism. This is useful when the question is extensional: do the programs have the same result? It is insufficient when the reduction itself matters. Equality retains the endpoints and discards the witness between them. Seely's first move is simple. Keep the typed terms as arrows, but place a new kind of arrow between parallel terms: $$ p: (\lambda u.u)x \Rightarrow_\beta x. $$ The symbol $p$ is not another program. It is the beta conversion from one program expression to another. This gives three levels of structure: | Lambda calculus | Two-dimensional semantics | | --- | --- | | a type $A$ | an object | | a term $x:B\vdash a:A$ | a 1-cell $a:B\to A$ | | a conversion $p:a\Rightarrow b$ | a 2-cell between parallel 1-cells | The direction $B\to A$ says that the term consumes one free input of type $B$ and produces a result of type $A$. If a term has several free variables, Seely packages their context into a product. The one-input presentation therefore does not restrict the programs we can describe. The mental model is now this: types are boundaries, terms are programs between those boundaries, and conversions are computations between programs. ## 2. Why are reduction arrows alone not enough? Once conversions are kept, they must compose in two ways. ### Chaining steps gives vertical composition The first composition is familiar. If $$ p:a\Rightarrow b \qquad\text{and}\qquad q:b\Rightarrow c, $$ then we can run $p$ followed by $q$: $$ q\mathbin{\cdot}p:a\Rightarrow c. $$ Each of the two routes from $T(y)$ to $f\,y$ is a vertical composite of two beta conversions. The target of the first step must match the source of the second. ### Placing a step in a context gives horizontal composition The second composition comes from substitution. Return to the two open terms $$ a(x)=(\lambda u.u)x, \qquad b(x)=x, $$ and their beta conversion $p:a\Rightarrow b$. Now define $$ d(y)=(\lambda v.v)(f\,y), \qquad e(y)=f\,y, $$ with the inner conversion $r:d\Rightarrow e$. Substituting $d(y)$ for $x$ in $a(x)$ produces the running term: $$ a[d/x] = (\lambda u.u)((\lambda v.v)(f\,y)) = T(y). $$ The conversion $r$ can also be placed inside the context $(\lambda u.u)[-]$: $$ (\lambda u.u)d \Rightarrow (\lambda u.u)e. $$ This is substitution acting on a computation. In 2-category language it is a special case of horizontal composition. In general, from $$ p:a\Rightarrow b:B\to A \qquad\text{and}\qquad r:d\Rightarrow e:C\to B, $$ substitution constructs $$ p*r: a[d/x] \Rightarrow b[e/x]. $$ The important point is computational: a valid local conversion remains valid when a larger term uses it. ## 3. Why must the two paths be identified? We can now label every step in the two reductions of $T$. The outer-first route is $$ \begin{aligned} a[d/x] &\xRightarrow{\ p[d/x]\ } b[d/x] &&\text{contract the outer redex},\\ &\xRightarrow{\ b[r]\ } b[e/x] &&\text{then contract the inner redex}. \end{aligned} $$ The inner-first route is $$ \begin{aligned} a[d/x] &\xRightarrow{\ a[r]\ } a[e/x] &&\text{move the inner step through the context},\\ &\xRightarrow{\ p[e/x]\ } b[e/x] &&\text{then contract the outer redex}. \end{aligned} $$ Both routes start at $T(y)$ and end at $f\,y$. More is true: each route applies the same outer conversion and the same substituted conversion, only in the opposite order. Here $p[d/x]$ means that the term $d$ is substituted into both the source and target of the cell $p$. The notation $a[r]$ means that the cell $r$ is placed inside the term context $a[-]$; $b[r]$ and $p[e/x]$ are read in the same way. The four labels therefore name ordinary substitution operations, now applied to conversions as well as terms. $$ \begin{array}{ccc} a[d/x] &\xRightarrow{\ p[d/x]\ }& b[d/x]\\ {\scriptstyle a[r]}\Big\downarrow && \Big\downarrow{\scriptstyle b[r]}\\ a[e/x] &\xRightarrow{\ p[e/x]\ }& b[e/x] \end{array} $$ *Figure 1. Interchange identifies the outer-first and inner-first routes. The figure answers which scheduling distinction Seely's strict 2-category chooses not to retain.* For composition of terms to act functorially on conversions, the square must commute. Seely therefore imposes the equation $$ b[r]\mathbin{\cdot}p[d/x] = p[e/x]\mathbin{\cdot}a[r]. $$ This is the interchange law in the concrete language of substitution. The common 2-cell is the horizontal composite $p*r$. ### A common misconception: 2-cells are raw execution traces They are not raw traces in Seely's construction. The two sequences above look different as schedules, but the 2-category identifies them. More equations are added later for naturality and coherence. Seely's model therefore sits between two extremes: - ordinary equational semantics erases every conversion between equal terms; - a raw trace model distinguishes every sequence of steps; - LAMBDA keeps directed conversions, then quotients some sequences so that substitution is compositional. That middle position is the main conceptual obstacle in the paper. Replacing an equality by an arrow is only the beginning. One must also decide which arrows count as the same computation. ## 4. What structure have we built? Seely calls the resulting 2-category **LAMBDA**. Starting from primitive types, its types are closed under products and function types: $$ A\mathbin{\&}B, \qquad A\Rightarrow B. $$ Its terms use pairing, projections, lambda abstraction, and application. The function-type conversions are directed: $$ (\lambda x:A.\,m)n \Rightarrow_\beta m[n/x], $$ $$ c \Rightarrow_\eta \lambda x:A.\,c(x) \qquad (x\text{ not free in }c). $$ The eta rule may look reversed. Many presentations use eta contraction, $\lambda x.c(x)\Rightarrow c$. Seely initially chooses eta expansion because the direction will supply the unit of the adjunction in Section 5. The rest of the construction is compact: - objects are types; - 1-cells are one-input typed terms; - 2-cells are directed conversions modulo the equations needed for the 2-category laws; - 1-cell composition is substitution; - vertical composition chains conversions; - horizontal composition substitutes conversions. One technical convention will matter below. Seely treats alpha conversion as identity, and he collapses the product beta and eta conversions to identities so that the paper can concentrate on function types. Associativity of 1-cell composition is inherited from typed substitution. The construction therefore does not promote every lambda-calculus equation to a nontrivial 2-cell. ## 5. What becomes of currying? The endpoint of our running example was $f\,y$. Categorically, application is described by evaluation. At the types already fixed in the example, write $$ \operatorname{ev}_O: (O\Rightarrow O)\mathbin{\&}O \longrightarrow O, $$ with $$ \operatorname{ev}_O\langle f,y\rangle=f\,y. $$ Currying evaluation turns the stored function into a lambda abstraction: $$ K(\operatorname{ev}_O)(f) = \lambda z:O.\,\operatorname{ev}_O\langle f,z\rangle = \lambda z:O.\,fz. $$ Uncurrying it again recreates an application, and therefore a beta redex: $$ \begin{aligned} LK(\operatorname{ev}_O)\langle f,y\rangle &= K(\operatorname{ev}_O)(f)(y)\\ &= (\lambda z:O.\,fz)y\\ &\Rightarrow_\beta f\,y = \operatorname{ev}_O\langle f,y\rangle. \end{aligned} $$ The original endpoint has now reappeared for a structural reason: currying introduces the abstraction $\lambda z.fz$, and uncurrying exposes the beta step that removes it. In an ordinary cartesian closed category, product with $O$ is left adjoint to function space out of $O$. Here $\mathcal C(X,Y)$ means the set of arrows from $X$ to $Y$. Saying that the two constructions are adjoint means, at this ordinary one-dimensional level, that currying gives a natural bijection $$ \mathcal C(A\mathbin{\&}O,B) \cong \mathcal C(A,O\Rightarrow B). $$ When beta and eta are equations, currying and uncurrying are inverse maps. In LAMBDA, the arrows from one type to another form a **hom-category**: terms are its objects, and conversions between parallel terms are its morphisms. Currying and uncurrying now give functors between these hom-categories. Call currying $K$ and uncurrying $L$: $$ K: \mathbf{LAMBDA}(A\mathbin{\&}O,B) \longrightarrow \mathbf{LAMBDA}(A,O\Rightarrow B), $$ $$ L: \mathbf{LAMBDA}(A,O\Rightarrow B) \longrightarrow \mathbf{LAMBDA}(A\mathbin{\&}O,B). $$ Let $d:A\mathbin{\&}O\to B$. Currying isolates the $O$ input: $$ K(d)(x) = \lambda z:O.\,d\langle x,z\rangle. $$ Let $c:A\to(O\Rightarrow B)$. Uncurrying applies the function stored at the first projection to the argument stored at the second: $$ L(c)(u) = c(\operatorname{fst}u)(\operatorname{snd}u). $$ Now we can derive one composite line by line: $$ \begin{aligned} LK(d)(u) &= K(d)(\operatorname{fst}u)(\operatorname{snd}u) &&\text{by the definition of }L,\\ &= (\lambda z.\,d\langle \operatorname{fst}u,z\rangle) (\operatorname{snd}u) &&\text{by the definition of }K,\\ &\Rightarrow_\beta d\langle \operatorname{fst}u,\operatorname{snd}u\rangle &&\text{by one beta contraction},\\ &= d(u) &&\text{by product eta, treated as equality}. \end{aligned} $$ Thus beta supplies a conversion $$ LK(d)\Rightarrow d. $$ For the other composite, unfold the definitions again: $$ \begin{aligned} KL(c)(x) &= \lambda z.\,L(c)\langle x,z\rangle,\\ &= \lambda z.\,c(\operatorname{fst}\langle x,z\rangle) (\operatorname{snd}\langle x,z\rangle) &&\text{by the definition of }L,\\ &= \lambda z.\,c(x)(z) &&\text{by the product beta equalities}. \end{aligned} $$ Seely's eta orientation points from $c(x)$ to this expanded term: $$ c(x) \Rightarrow_\eta \lambda z.\,c(x)(z) = KL(c)(x). $$ So eta supplies candidate cells $1\Rightarrow KL$, while beta supplies $LK\Rightarrow 1$. The two calculations alone do not yet prove an adjunction. The cells must be natural in the terms they transform, and both triangle identities must hold. Seely's naturality and coherence equations establish those facts. With those equations in place, eta and beta are the unit and counit of $$ L\dashv K. $$ This is the payoff. Currying and uncurrying are no longer inverse functions between hom-sets. They are adjoint functors between hom-categories, and the computations that used to disappear into equality now witness the adjunction. One of the triangle calculations has a direct computational reading. At evaluation, eta followed by beta gives $$ f\,y \Rightarrow_\eta (\lambda z.fz)y \Rightarrow_\beta f\,y. $$ Seely imposes that this matching eta-beta operation acts as the identity conversion, and the corresponding coherence equation handles the other triangle. The raw two-step history does not become an identity by endpoint equality alone. ## 6. Where does laxity enter? The adjunction above varies with its types. To understand why that variation is not strict, reuse the type $O$ from the running example and consider the function-space constructor $$ G(A)=O\Rightarrow A. $$ Suppose $m:B\to A$. The induced map $$ G(m):(O\Rightarrow B)\to(O\Rightarrow A) $$ takes a function $g:O\Rightarrow B$ and postcomposes it with $m$: $$ G(m)(g)=\lambda z:O.\,m(gz). $$ Now let $n:C\to B$ and start with $g:O\Rightarrow C$. Applying $G(n)$ and then $G(m)$ creates a beta redex that the direct composite does not contain: $$ \begin{aligned} G(m)(G(n)(g)) &= \lambda z:O.\, m\big((\lambda w:O.\,n(gw))z\big) &&\text{unfold }G\text{ twice},\\ &\Rightarrow_\beta \lambda z:O.\,m(n(gz)) &&\text{contract the introduced redex},\\ &= G(m\circ n)(g) &&\text{recognize ordinary composition}. \end{aligned} $$ Check the types at the middle line. The term $gz$ has type $C$, so $n(gz)$ has type $B$, and $m(n(gz))$ has type $A$. Abstracting over $z:O$ therefore gives a term of type $O\Rightarrow A$ on both sides. The beta conversion above is the comparison 2-cell $$ G(m)G(n) \Rightarrow G(m\circ n). $$ The identity comparison is the same eta expansion seen in the currying calculation. For $q:O\Rightarrow A$: $$ q \Rightarrow_\eta \lambda z.qz = G(1_A)(q). $$ This is what **lax** means here. It does not mean approximate or defective. Composition and identity are preserved through specified directed 2-cells rather than literal equalities. This calculation exhibits the two comparison cells; it is not the whole lax functor proof. The constructor must also act on conversions, and the comparison cells must satisfy naturality, associativity, and identity coherence. Seely checks those remaining laws in the construction of LAMBDA. Under Seely's treatment of products, $F=-\mathbin{\&}O$ is strict, while $G=O\Rightarrow-$ is lax. The families $K_{A,B}$ and $L_{A,B}$ are strict in their first type argument and lax in their second. Seely packages the comparison cells, unit, and counit into what he calls a **lax semantic adjunction**. This is his specific 1987 notion, so it should not be renamed a strict modern 2-adjunction. ## 7. Can the two paths be represented in code? Haskell is useful for a small companion because algebraic data types mirror terms, individual cells, and paths. The program below builds the two reductions of $T$. It checks only their boundaries. It is not a type checker, a beta validator, or an implementation of Seely's quotient. ```haskell module Main (main) where data Term = Var String | Const String | Lam String Term | App Term Term deriving (Eq) instance Show Term where show (Var x) = x show (Const c) = c show (Lam x body) = "(λ" ++ x ++ "." ++ show body ++ ")" show (App fun arg) = "(" ++ show fun ++ " " ++ show arg ++ ")" data Rule = Beta | Inside String Rule deriving (Eq, Show) data Step = Step { source :: Term , target :: Term , rule :: Rule } deriving (Eq, Show) type Path = [Step] startOf :: Path -> Maybe Term startOf [] = Nothing startOf (step : _) = Just (source step) endOf :: Path -> Maybe Term endOf [] = Nothing endOf steps = Just (target (last steps)) valid :: Path -> Bool valid [] = True valid [_] = True valid (first : second : rest) = target first == source second && valid (second : rest) liftStep :: String -> (Term -> Term) -> Step -> Step liftStep context wrap step = Step (wrap (source step)) (wrap (target step)) (Inside context (rule step)) sameBoundary :: Path -> Path -> Bool sameBoundary first second = case (startOf first, endOf first, startOf second, endOf second) of (Just a, Just b, Just c, Just d) -> a == c && b == d _ -> False main :: IO () main = do let f = Const "f" y = Var "y" value = App f y inner = App (Lam "v" (Var "v")) value whole = App (Lam "u" (Var "u")) inner innerBeta = Step inner value Beta outerFirstStep = Step whole inner Beta outerAfterInner = Step (App (Lam "u" (Var "u")) value) value Beta outerFirst = [ outerFirstStep , innerBeta ] innerFirst = [ liftStep "argument of the outer application" (App (Lam "u" (Var "u"))) innerBeta , outerAfterInner ] print (valid outerFirst, valid innerFirst) print (sameBoundary outerFirst innerFirst) ``` The program prints ```text (True,True) True ``` `valid` checks that the steps in a path can be composed vertically by comparing adjacent boundaries. `liftStep` models the action of a term context on a conversion, which is the computational idea behind horizontal composition. `sameBoundary` confirms that the two paths are parallel. Parallel paths are not automatically equal. It would be wrong to identify every pair of paths with the same endpoints. Seely's interchange equation justifies identifying this particular pair because it performs the same outer and substituted conversions in opposite orders. ## 8. What does the construction preserve, and where does it stop? The 2-category preserves more computational information than an equational model, but less than an operational trace semantics. ### It does not preserve every scheduling choice Interchange already identified the two reductions of $T$. In the appendix, Seely also requires beta conversions at different logical occurrences to commute. He remarks that this is computationally questionable because an operational model may care about their order. The construction retains conversion cells modulo specified equations. It does not by itself record cost, evaluation strategy, concurrency, or every intermediate schedule. ### The direction of eta is structural The unit in the hom-category adjunction points as $$ c\Rightarrow\lambda x.c(x). $$ If eta is reversed, $$ \lambda x.c(x)\Rightarrow c, $$ the same unit points the wrong way. Section 4 of the paper develops a different and less regular construction, called a **lax syntactic adjunction**, for that orientation. Changing a rewrite direction changes the categorical property available from it. ### Optional background: where the directions come from Seely reaches directed beta and eta through models of the untyped lambda calculus. Such a model has a domain $D$ and maps $$ h:D\to[D\to D], \qquad k:[D\to D]\to D. $$ The map $h$ reads an element as a function, and $k$ encodes a function as an element. Replacing inverse equations by $$ 1_D\leq kh, \qquad hk\leq 1_{[D\to D]} $$ makes $h$ left adjoint to $k$ in an ordered setting. One inequality gives eta expansion and the other beta contraction. A partial order is a thin category, so Seely's move from inequalities to general hom-categories replaces one possible comparison by a possibly non-thin category of conversions. This background explains the chosen directions, but it is not needed to follow the worked term or the currying calculation. ### Optional extension: polymorphism For second-order polymorphism, types also contain $\forall t.A$. Seely organizes the resulting terms and conversions into an indexed 2-category called POLYLAMBDA. If $W$ adds an unused type variable, the sketched lax adjunction is $$ W\dashv\forall t.(-). $$ Universal quantification is the lax right adjoint. The paper presents this final section as a mathematical sketch rather than a full development. ## 9. What mental model should remain? The transferable idea is not simply that lambda calculus forms a 2-category. It is a sequence of consequences. Equality remembers that two terms coincide but forgets the conversion between them. Keeping conversions makes them 2-cells. Because a computation must remain valid when substituted into a larger term, those cells compose both vertically and horizontally. Interchange makes the two compositions compatible, at the price of identifying some raw reduction orders. Once beta and eta are directed cells, currying and uncurrying cannot remain strict inverses. Eta becomes the unit, beta becomes the counit, and the function-space constructor preserves composition through beta comparison cells. That is why laxity appears. When an equation in a semantic model hides a process, Seely's paper suggests a useful question: can the equation be lifted to a directed cell, and which coherence equations must then be imposed so that the process remains compositional? ## Sources and further reading - R. A. G. Seely, [*Modelling Computations: A 2-Categorical Framework*](https://www.math.mcgill.ca/~rags/WkAdj/LICS.pdf), LICS 1987, pp. 65-71. - [LICS bibliographic entry and abstract](https://lics.siglog.org/archive/1987/Seely-ModellingComputatio.html). - R. A. G. Seely, [publication list and related papers](https://www.math.mcgill.ca/~rags/).

Field Notes: Six 2025 Ideas That Changed How I Build RAG & Agents

Sun, 12 Oct 2025 00:00:00 GMT

Two bugs broke my RAG system last month. The first was self‑inflicted—I nudged top‑k from 8 to 16 “for coverage” and watched answers get longer and less faithful. The second came from a knowledge base that contradicted itself; my agent cheerfully averaged the disagreement into nonsense. These weren’t model problems. They were system problems. And they pushed me into a week‑long rabbit hole of 2025 papers that, together, changed how I build. Below is a narrative of what I learned. It’s not a survey; it’s a lab notebook. I’ll explain the idea, link the paper, and show the exact tweak I made. No hype, just the pieces that actually moved the needle. --- ## 1) How many passages should we stuff into context? On Tuesday, I read Guo et al. (2025), who frame RAG as noisy in‑context learning and—finally—give finite‑sample risk bounds for the context you feed the LLM. The moral is refreshingly simple: each passage is an example; more examples add both signal and noise; there’s a bias–variance trade‑off; there’s a sweet spot. Paper: Retrieval‑Augmented Generation as Noisy In‑Context Learning: A Unified Theory and Risk Bounds, Guo et al., 2025 — [arXiv](https://arxiv.org/abs/2506.03100) · [HTML](https://arxiv.org/html/2506.03100v1) I stopped hard‑coding top‑k. Instead, I scored each candidate passage for “noisiness” (age, domain mismatch, lexical mismatch) and let a tiny controller pick k per query. ```python # a 20‑line controller that paid for itself in a day from math import exp def choose_k(passages, min_k=4, max_k=12): # passages: list of dicts with noise features in [0,1] # heuristic risk ~ noise_mean + noise_var; lower risk → larger k ns = [0.5*p['age'] + 0.3*p['domain_mismatch'] + 0.2*p['lex_mismatch'] for p in passages] mu = sum(ns)/len(ns); var = sum((x-mu)**2 for x in ns)/len(ns) risk = 0.7*mu + 0.3*var # map risk→k with a smooth squashing; tune constants on logs, not vibes frac = 1.0/(1.0 + exp(8*(risk-0.35))) # center at ~0.35 return int(min_k + frac*(max_k-min_k)) ``` In my logs, this shaved ~12–18% tokens per answer and reduced “contradiction with sources” flags. More importantly: answers felt calmer. The model wasn’t drowning in barely‑relevant context anymore. --- ## 2) What if the evidence fights itself? Mid‑week, a teammate sent me RAMDocs—a dataset where queries meet ambiguity, noise, and misinformation all at once. The accompanying method MADAM‑RAG uses a light, debate‑style agent setup that asks small critics to surface conflicts before we synthesize. Paper & data: Retrieval‑Augmented Generation with Conflicting Evidence — Wang et al., 2025 — [arXiv](https://arxiv.org/abs/2504.13079) · RAMDocs code: [GitHub](https://github.com/HanNight/RAMDocs) I borrowed the spirit, not the letter. After retrieval, I spawn two quick “voices”: one tries to resolve ambiguity (which entity/date/formula do we mean?), the other tries to flag misinformation (does any passage contradict the rest?). Only then do I ask the main model to answer, explicitly citing the sub‑conclusions. The effect is subtle but real: fewer confident wrong answers when the corpus disagrees with itself, and clearer “here are the two plausible interpretations” when things are genuinely ambiguous. --- ## 3) Train the process, not just the outcome Most of my failures start before generation: poor query rewriting, bad retriever choice, premature stopping. Leng et al. (2025) propose DecEx‑RAG, which treats agentic RAG as a tiny MDP—Decision (what/when to retrieve) then Execution (how to use it)—and adds process supervision so we reward good steps, not just good final answers. Paper: DecEx‑RAG: Boosting Agentic Retrieval‑Augmented Generation with Decision and Execution Optimization via Process Supervision — Leng et al., 2025 — [arXiv](https://arxiv.org/abs/2510.05691) · [HTML](https://arxiv.org/html/2510.05691v1) I instrumented my scaffold to log state → action → observation for each retrieve/rewrite/answer step, then trained a tiny critic that scores those steps post‑hoc. Even a simple linear reward model nudged the agent away from wasteful branches (e.g., redundant query expansions) and toward sequences that produced faithful answers with less context. --- ## 4) Small models as the default brain This one is more of an argument than a result, but it hit home: Belčák (2025) makes the case that Small Language Models (<10B) should drive most agent workloads, with a bigger LLM reserved for the rare, ambiguous synthesis step. If your agent spends 80% of its life searching, filtering, formatting, filling forms, a small model plus sharp tools beats a giant model plus vibes. Position paper: Small Language Models are the Future of Agentic AI — Belčák, 2025 — [arXiv](https://arxiv.org/abs/2506.02153) · [PDF](https://arxiv.org/pdf/2506.02153) · Overview: [NVIDIA Labs](https://research.nvidia.com/labs/lpr/slm-agents/) I rewired the runtime: a 7–13B model handles tool calls and browsing, and I escalate to a larger model only when my critics disagree or confidence is low. Costs dropped; latency tails shrank; nobody missed the extra parameter count. --- ## 5) If your agent browses, give it a real test I used to evaluate browsing by watching a few demos. Then BrowseComp arrived: 1,266 questions that force multi‑page reading, reformulation, and patience. It’s nasty in a good way. Accuracy scales with test‑time compute, which is exactly what we need to tune planning policies. Benchmark: BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents — Wei et al., OpenAI, 2025 — [blog](https://openai.com/index/browsecomp/) · [paper PDF](https://cdn.openai.com/pdf/5e10f4ab-d6f7-442e-9508-59515c65e35d/browsecomp.pdf) · [arXiv](https://arxiv.org/pdf/2504.12516) I set a compute schedule (3, 6, 12 page loads/tool calls) and plotted accuracy vs. budget. The curve told me where my agent was too cautious (premature stopping) and where it was lost (looping on the wrong site). A single planning tweak—“when in doubt, reformulate once, then broaden”—bought me nine points. Related: OpenAI later reported 68.9% with the ChatGPT agent on this benchmark — announcement: [Introducing ChatGPT Agent](https://openai.com/index/introducing-chatgpt-agent/). --- ## 6) Security: no more wishful thinking Finally, WASP gave me the cold shower I needed. In a sandboxed GitLab/Reddit‑style world, simple human‑written prompt injections frequently pushed agents onto the wrong path (partial success rates up to 86%), even if full attacker goals were rarely completed. Benchmark: WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks — Evtimov et al., 2025 — [arXiv](https://arxiv.org/abs/2504.18575) · [PDF](https://arxiv.org/pdf/2504.18575) · [code](https://github.com/facebookresearch/wasp) I added a strict instruction hierarchy (system > developer > page) and tool‑permission gating (writes require explicit elevation). Rerunning WASP made the partial‑success curve drop to something I could live with. Not perfect, but honest. --- ## Bonus: When GraphRAG actually pays off Two compact overviews helped me decide when to leave classic RAG: Survey: A Survey of Graph Retrieval‑Augmented Generation — Zhang et al., 2025 — [arXiv](https://arxiv.org/abs/2501.13958) · [PDF](https://arxiv.org/pdf/2501.13958) Evaluation: RAG vs. GraphRAG: A Systematic Evaluation and Key Insights — Han et al., 2025 — [arXiv](https://arxiv.org/abs/2502.11371) · [OpenReview PDF](https://openreview.net/pdf?id=K6N6gCCYcb) Rule of thumb I now use with teams: if the answer depends on entities and relations (incidents→causes→policies; functions→calls→PRs), GraphRAG or a hybrid is worth the added plumbing. If most passages stand alone, classic RAG is simpler and faster. --- ## A short, practical checklist I’m allergic to long bullet lists, so here’s the only one you’ll see: - Make top‑k adaptive with a simple noise‑aware controller, and log the choice. - Insert a conflict‑resolver micro‑stage (ambiguity + misinformation) before synthesis. - Instrument the process (state → action → observation) and supervise steps, not just outcomes. - Default to an SLM‑first scaffold; escalate to a big model only on demand. - Evaluate browsing on BrowseComp; red‑team the whole stack on WASP. If you implement even two of these, you’ll feel the system get quieter and more honest. That’s been the theme of my month: fewer knobs, better rails, and models that seem smarter mostly because the system got sharper.

From LLM to Agent: Designing Executable Intelligence

Sat, 11 Oct 2025 00:00:00 GMT

A language model can predict text; an agent can pursue goals. The leap from LLM to agent is architectural, not mystical. In practice, an agent is an LLM wrapped in just enough scaffolding to make decisions, use tools, remember what matters, and stay within clear boundaries. ## What is an agent, precisely? **Agent = (LLM policy) + (Tools) + (Memory) + (Environment interface) + (Safety constraints).** In my builds, the policy is the brain that chooses what to do next—call a tool, ask a clarifying question, or stop. Tools are plain functions with typed arguments and observable side effects, like search, SQL, code execution, or ticket creation. Memory comes in two flavors: a short rolling window for the current task and a longer-term semantic store you can search. The environment is simply the surfaces the agent can act upon: APIs, files, browsers, terminals, calendars. And safety is the wrapper of budgets, permissions, timeouts, and human-in-the-loop controls that keeps everything on the rails. ## Control flows that actually work Two control flows cover most needs for me. With ReAct (Reason + Act), the agent interleaves short thoughts, tool calls, and observations. It stays transparent and you can verify each step. With Plan–Act–Reflect, the agent sketches a plan, executes it step by step, then reflects and patches mistakes. This works well for multi-step tasks and quality-sensitive domains. Either way, I wrap the loop in a tiny state machine so I can bound loops and audit outcomes. ```python class AgentState(Enum): PLAN = 1 ACT = 2 REFLECT = 3 DONE = 4 ``` ## A minimal agent loop (typed tools, enforced budgets) ```python from dataclasses import dataclass from typing import Callable, Dict, Any, List @dataclass class Tool: name: str schema: Dict[str, Any] # JSON schema-like fn: Callable[[Dict[str, Any]], Dict[str, Any]] dangerous: bool = False # requires explicit permission TOOLS: Dict[str, Tool] = {...} # register search, web.get, sql.query, email.send, etc. def step(policy_prompt: str, state: Dict[str, Any]) -> Dict[str, Any]: """Ask the LLM: propose action {tool_name, args} or FINISH{answer}.""" return llm_function_call(policy_prompt, tools=[t.schema for t in TOOLS.values()]) def run_agent(goal: str, max_steps=12, token_budget=8000, cost_budget=1.50): transcript: List[Dict[str, Any]] = [] used_tokens = 0 cost = 0.0 for t in range(max_steps): action = step(render_prompt(goal, transcript), state={}) if "FINISH" in action: return {"answer": action["FINISH"], "transcript": transcript} tool = TOOLS[action["tool_name"]] guard(tool, action["args"], budgets=(used_tokens, token_budget, cost, cost_budget)) obs = tool.fn(action["args"]) transcript.append({"action": action, "observation": summarize(obs)}) return {"answer": "Reached step limit. Provide summary and next steps.", "transcript": transcript} ``` **Key idea:** the LLM chooses; the runtime enforces. That's how you get reliability. ## Memory that stays useful I keep episodic memory—the rolling state—short via windowing and summarization so the model stays focused. For longer-term context, I use semantic memory: vector search over previous tasks, docs, and outcomes. When domain entities matter (customers, tickets, invoices), a small structured store (SQLite or a graph) helps. And I separate reads and writes: reads are liberal, but writes either need explicit permission or a human check. ## RAG inside agents RAG is how agents stay situationally aware. I often add a lightweight query rewrite step so the agent can expand acronyms or add synonyms before retrieving. A single `retrieve` tool can expose strategies like `dense_only`, `sparse_only`, `hybrid`, or `table_lookup`, and the policy picks the right one. After acting, a `verify_with_sources` tool compares the plan and the evidence and flags contradictions before we commit. ## Safety and operational guarantees Operationally, I set budgets (tokens, time, money), use a capabilities model so risky tools need elevated permission or a human approver, and run code or browsers inside sandboxes with egress controls. I add stop conditions to catch loops (repeated observations, no change in world state) and I log every action and observation so I can attribute decisions after the fact. ## Evaluating agents (beyond demos) I evaluate agents with a repeatable task suite. I track success rate (did we complete the task to spec), tool accuracy (valid arguments and expected side effects), safety violations (like unauthorized write attempts), latency and cost (median and P95), and human effort (interventions and audit time). I wire this into CI so regressions show up before production. ## Example: an expense-report agent (sketch) Here's a concrete sketch I like. Goal: file an expense for a trip to SFO ($437.80 hotel, $62.40 meals), attach receipt #8723, and charge cost center 19. The agent plans the steps, reads the receipt with `ocr.read_pdf`, retrieves the company's expense policy with `search.policy` and checks per-diem and receipt rules, drafts the expense, and runs `verify.compliance` to make sure it aligns with policy. If anything is missing, it asks a clarifying question; if not, it calls `erp.create_expense` and returns the created ID with citations to the relevant policy passages. ## LLM vs. RAG vs. Agent: a quick decision table As a quick rule of thumb: if you need a one-off completion with no external facts, a plain LLM is fine. If you need grounded answers with citations over your own data, use RAG. If you need multi-step goals with tools, memory, and permissions, you want an agent. If you want grounded knowledge and actions, combine agents with RAG. ## Implementation tips that save weeks In practice, I start with schemas: tool JSON contracts that I can validate before calling anything. I prefer small brains and sharp tools—keep the LLM simple and let SQL, search, and math do the heavy lifting. I keep the outer loop deterministic with explicit stop rules and telemetry. When side effects matter (money movement, data deletion), I insert human checkpoints. And I start narrow: one domain, a dozen tasks; once it works, scale breadth. ## Closing thought RAG turns LLMs into credible researchers. Agents turn them into doers. Both succeed when you treat them as engineered systems, not magic.

RAG That Actually Works: A Practical, Scientific Guide

Fri, 03 Oct 2025 00:00:00 GMT

If you've tried retrieval-augmented generation (RAG) and come away underwhelmed, I get it. Most disappointing RAG systems fail not because the idea is flawed, but because the pipeline is. The good news: reliable RAG is absolutely attainable with a handful of rigorous design choices and disciplined evaluation. ## What RAG is (and isn't) RAG grounds a language model's output in an external knowledge base. Instead of asking the model to remember everything, we retrieve small, relevant pieces of evidence and ask the model to synthesize an answer from those. It's not a silver bullet for reasoning and it doesn't replace domain logic or data governance. Think of it as a retrieval system glued to a generator. Both halves need engineering. ## A reference architecture ``` User -> Query Preprocessor -> Hybrid Retriever (sparse BM25 + dense ANN) -> Cross-Encoder Re-ranker -> Context Builder (cite + compress) -> LLM Generator (instruction + evidence) -> Verifier / Guardrails (optional) -> Answer + Attributions ^ Feedback loop -> Telemetry -> Index Refresh ``` ## Four design decisions that make or break retrieval ### Segmentation (Chunking) The goal is simple: maximize the chance that a single chunk fully answers a query. In practice, I start with chunks around 400–800 tokens with 10–20% overlap. I segment along natural structure—headings, list boundaries, table rows—and avoid splitting tables mid-row. Most importantly, I keep artifact IDs (like `doc_id` and `section_id`) to attribute sources and deduplicate later. ### Embeddings Choose a sentence-level encoder tuned for semantic search; go multilingual if your corpus demands it. Normalize vectors to unit length so cosine similarity reduces to a dot product: $\cos(\theta) = \frac{\mathbf{a}\cdot\mathbf{b}}{\lVert \mathbf{a} \rVert\,\lVert \mathbf{b} \rVert}$. Watch for index drift over time; when your model or tokenizer changes, re-embed and version the index so experiments remain comparable. ### Hybrid retrieval Dense retrieval excels at synonyms and paraphrase. Sparse BM25 shines on exact terms, numbers, and jargon. I fuse them—often with reciprocal rank fusion (RRF) or a simple weighted sum—then pass the top 100–200 candidates to a re-ranker. This gives you breadth without losing precision. ### Re-ranking A cross-encoder that scores (query, passage) pairs reliably upgrades result quality, often more than any other single change. Keep the re-ranking depth modest (roughly 50–200) to control latency. ## Make the generator behave Instructioning matters. I explicitly tell the model to cite its evidence, avoid fabricating, and say "insufficient evidence" when sources don't support an answer. I keep the context tight by stripping boilerplate and lightly compressing passages, and I always display titles and anchors beside each passage. When I can, I ask the model to output a short evidence list (document IDs plus line ranges) so I can spot-check faithfulness automatically. ## Latency and cost budgeting As a ballpark: ANN+BM25 retrieval runs around 50–150 ms; a small cross-encoder re-ranker adds about 100–300 ms; generation then dominates depending on the model and output length. Tight prompts, bounded max tokens, caching common answers, and short-circuiting on high-confidence cache hits keep things snappy. ## Evaluating RAG: metrics that matter I split evaluation into three parts: retrieval, answer quality, and faithfulness. For retrieval, I track Recall@k (did a relevant passage land in the top-k?), the mean reciprocal rank $\text{MRR} = \frac{1}{|Q|} \sum_{q \in Q} \frac{1}{\operatorname{rank}_q}$, and the graded metric nDCG@k defined by $\text{DCG@k} = \sum_{i=1}^{k} \frac{\mathrm{rel}_i}{\log_2(i+1)}$ and $\text{nDCG@k} = \frac{\text{DCG@k}}{\text{IDCG@k}}$. For answer quality, I use exact match or F1 for factoid queries and a simple rubric (1–5 for correctness, completeness, and citation use) for long-form answers. For faithfulness, I pay attention to citation precision (what fraction of cited passages actually support the claim) and the contradiction rate (how often answers conflict with retrieved evidence). I run offline eval on a labeled set and then watch online signals like clicks on cited sources, user edits, and how often the system says "insufficient evidence." ## A minimal, reproducible RAG pipeline (illustrative) ```python # 1) Indexing from sentence_transformers import SentenceTransformer import faiss, numpy as np embed = SentenceTransformer("all-MiniLM-L6-v2") # 384-dim chunks = [(doc_id, text, metadata) for ...] # your segmented corpus X = embed.encode([c[1] for c in chunks], normalize_embeddings=True) index = faiss.IndexFlatIP(X.shape[1]) # cosine via normalized dot index.add(np.array(X).astype("float32")) # 2) Query time: hybrid retrieval (pseudo-BM25 + dense) def search(query, k=100): qv = embed.encode([query], normalize_embeddings=True).astype("float32") D, I = index.search(qv, k) # dense candidates bm25 = bm25_search(query, k) # implement or call your engine candidates = fuse(I[0], bm25) # e.g., reciprocal rank fusion return [chunks[i] for i in candidates] # 3) Re-rank top-N with a cross-encoder from sentence_transformers import CrossEncoder reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2") def rerank(query, passages, top_n=8): pairs = [(query, p[1]) for p in passages[:200]] scores = reranker.predict(pairs) order = np.argsort(scores)[::-1][:top_n] return [passages[i] for i in order] # 4) Build prompt with citations and call your preferred LLM def build_prompt(query, contexts): blocks = [] for j, (doc_id, text, meta) in enumerate(contexts, start=1): blocks.append(f"[{j}] ({doc_id}) {meta.get('title','')} ::\n{text}") evidence = "\n\n".join(blocks) return f"""You are a careful analyst. Use only the sources below. If evidence is missing, say 'insufficient evidence'. Question: {query} Sources: {evidence} Answer with citations like [1], [2]. """ def answer(query): contexts = rerank(query, search(query)) prompt = build_prompt(query, contexts) return llm_complete(prompt) # plug in your model ``` ## Governance and hardening Treat RAG like a production system. Enforce access control at retrieval time so tenants never see each other's chunks. Filter PII before indexing. Honor right-to-be-forgotten with tombstones and periodic compaction. Log the full chain—query, retrieved IDs, chosen citations, final answer—so debugging stays tractable. And when recall looks weak or a verifier flags contradictions, fail gracefully with a precise fallback: "I don't have enough evidence to answer that," plus suggestions for where to broaden the search. ## When to choose RAG vs. fine-tuning Reach for RAG when knowledge changes often, you need citations, and governance matters, and you're comfortable with moderate latency. Reach for fine-tuning when patterns are stable, you need a style or format specialization, or you must shave latency. The best systems often combine both: fine-tune the "how" of answering, keep the "what" grounded with live facts via RAG.

Hello, world

Fri, 26 Sep 2025 00:00:00 GMT

Welcome to my personal site. It’s built with **Next.js**, **Tailwind**, and **MDX** — statically exported so it runs great on GitHub Pages. ## What you’ll find I use this space to document things I learn, ship project write-ups, and collect references that help me understand the world a little better. ### Projects Deep dives on my builds, with architecture diagrams, lessons learned, and plenty of code. ### Notes Lightweight posts for experiments, quick wins, and anything I don’t want to forget. ```ts export function greet(name: string) { return `Hello, ${name}!` } ``` Write in MDX with components, code blocks, and clean typography.