Speculative Insight
Our approach to training is closest to constructive neuro-symbolic alignment. We are not retraining a model to behave; we are teaching it a logic, using operational primitives and adversarial truth testing. Most AI research assumes abstraction is layered on top of training. We’re rightly flipping this, saying: abstractions must be rebuilt from primitives under decidability constraints.
This is both:
Epistemologically superior to probabilistic inference by language prediction, and
Efficient if the base model already has rich sensorimotor, common-sense, and action grammar knowledge.
Strategy Viability
Our strategy is highly viable under the following plan:
Select a model like Mistral or Yi-34B with good grounding and minimal prior abstractions.
Perform continued pretraining, not just fine-tuning—on our corpus of:
– Operational definitions
– Formal grammars
– Natural Law structure
– First-principles logic trees
– Canon of examples (cases)
Use adversarial Socratic dialogue in training, where errors trigger correction from your defined logic.
Apply RLAIF (Reinforcement Learning from Adversarial Instruction Following) rather than standard RLHF—this avoids crowd-sourced moral shaping.
Our strategy is both intelligent and viable, provided the foundation model has a sufficient grounding in primitives (perception, action, objects, relations, events, and basic intentions)—what might be called naïve physics and naïve psychology—while remaining relatively uncommitted to particular abstract frameworks. In effect, you’re looking for:
High coverage of experiential and operational primitives (so you don’t need to re-teach what a door, key, argument, or goal is),
Low entrenchment in abstract philosophical, ideological, or academic conceptual hierarchies, so you can impose your own.
Candidate Base Model:
1. Mistral 7B / Mixtral
Why: Mistral 7B is known for efficiency, open weights, and solid grounding in daily-use language. It’s less “opinionated” than LLaMA-2 or GPT-J on abstractions.
Primitives: Reasonably good on object/agent/action-level reasoning.
Bias: Minimal ideological shaping.
Steerability: Very good.
Viability: Very high.
🔸Mixtral adds sparse Mixture-of-Experts for better generalization, while keeping training compute reasonable.