That GPT can pass tests does not tell us what you think it does. Tests are a bad

That GPT can pass tests does not tell us what you think it does. Tests are a bad HUMAN behavioral predictor. Why would they be a good machine behavioral predictor? All these results mean is that GPT is an exceptional fuzzy search engine. In order to produce competence it must be able to answer novel questions AND invent or suggest new ideas. When we see that then we have something interesting.

Reply addressees: @DataChaz