Theme: Constitutional Order

AI FOR LAW: THE PROBLEM OF DUMBNESS IN LLMs
Present LLM’s are worse than terrible at ordinary language logic. Questionably tepid at even simple mathematical logic. Even though they can be acceptable at programmatic logic.

LLMs rely for their incompetence at (ignorance of) logic on the patterns embedded in ordinary language and programmatic language, using statistical derivations obtained by brute force consuming of online text – and appear to need to rely on third party APIs for mathematical logic.

Our work in the formal operational logic of decidability and in particular legal decidability, may require we compose our work and our work in the law in particular law in programmatic form, which by its structure contains the logic, *IF* the models cannot learn to develop the logic on their own.

Now, the grammar of our work is a bridge between formal written language and programmatic language, with the constraint from mathematical logic on equilibration (equals signs).

We assume we can produce enough of our text to train the models such that they depend less on models and more on summarization. But I feel this is optimistic given the very structure of the algorithms that produce LLM’s – which are, for all intents and purposes, content-summarizing search engines.

So until we see another level of emergence from LLMs, demonstrating the ability to perform logic, by converting ordinary and formal language into the equivalent of programmatic logic, then I assume we will have to both maintain our current use of operational vocabulary and grammar in formal language, and reduce such work to programmatic language. Even then it may require an external API, or an extraordinary amount of training to produce that logical capacity.

I haven’t spent enough time on Wolfram’s software to determine if we might produce it there. But I can see the potential of an LLM converting text to code, running the code and returning the result to itself for presentation to the user.