“I would like to know how Grok performs here.”— Elon is working from first pri

–“I would like to know how Grok performs here.”—

Elon is working from first principles per se but I am not sure what that means. My work is a constructive logic of first principles but I suspect I mean causal first principles and Elon means the first principles of constraint in a domain as that’s how he seems to use the term – which is the conventional meaning.

Grok is natively more ‘truthful’ but lacks the capacity for depth that 4o and 4.5 are capable of. I can use it for my work in the epistemology of science but it breaks down applying my work.

Oddly I find 4o produces better training data and training plans. And I can intuit something on the edge of my awareness that I can’t quite put into words. If I can I think there is something useful to be understood there. It has something to do with a lot of context memory and a large number of parameters that allows us to exploit subnetworks that might otherwise infrequently express, and I think I detect this as cognitive depth.

If I was researching LLMs themselves I would work on that exposition because many llms are reducing to linear activation and exposition and leaving vast numbers of effectively unaccessible subnetworks behind. I don’t think this is what I want for a reasoning model that must retain the ability to hypothesize while still constraining itself from hallucination.

I suspect it’s not immediately intuitive that hallucination and autoassociation and recombinant novelty discovery are useful practices, but that the human brain self tests by recursion anything that grasps our attention.

The problem LLMs faced prior to recursive, predictive, COT and reasoning models is that they could not self monitor so spewed hallucinations where humans would not have. (In humans we call it error, mistake, or folly. 😉