Okay maybe I need to make myself more clear, and start from your claim:
> It has to do with mental capacities to "bring objects under a concept", partition experience into its conceptual structure ("conceptualise"), simulate scenarios based on compositions of concepts ("the imagination") and so on. These are mental capabilities a wide class of animals possess, who know no language; that LLMs do not possess.
Assume it is true that humans are capable of these capabilities, why do you think LLMs are not capable of them? We don't know if they are capable of these capabilities, and that's what explainable ai is.
Take a step back and assume that LLMs are not capable of these capabilities, how do you prove that these are the fundamental concepts in the universe, instead of projections of higher level concepts from higher dimensional space that humans and animals are not aware of? What if there exist a more universal set of concepts that contains all the concepts we know and others, in a higher dimension, and both LLMs and humans are just using the lower dimension projections of such concepts?
> It has to do with mental capacities to "bring objects under a concept", partition experience into its conceptual structure ("conceptualise"), simulate scenarios based on compositions of concepts ("the imagination") and so on. These are mental capabilities a wide class of animals possess, who know no language; that LLMs do not possess.
Assume it is true that humans are capable of these capabilities, why do you think LLMs are not capable of them? We don't know if they are capable of these capabilities, and that's what explainable ai is.
Take a step back and assume that LLMs are not capable of these capabilities, how do you prove that these are the fundamental concepts in the universe, instead of projections of higher level concepts from higher dimensional space that humans and animals are not aware of? What if there exist a more universal set of concepts that contains all the concepts we know and others, in a higher dimension, and both LLMs and humans are just using the lower dimension projections of such concepts?