I wouldn't say that Deep Neural Networks are too much different than a Human brains JedRothwell from my understanding.
They are very different in many important ways. The reasons are complicated and beyond the scope of the discussion. They are very different because the idea of neural networks was inspired by biological networks in 1958. Little was known about biological networks back then. As far as I know, computer artificial neural networks (ANN) were developed independently since then, with little reference to biology. Interestingly, AI knowledge might now be used in the other direction:
Deep learning comes full circle
Artificial intelligence drew much inspiration from the human brain but went off in its own direction. Now, AI has come full circle and is helping neuroscientists better understand how our own brains work.
Note they use the word "inspiration." Not imitation or "modeled from the brain."
However, for the sake of argument, suppose AI and brains were very similar. Take a real world example. Brains and neurons are similar in many species, ranging from earthworms, to bees, to humans. But, as I said, the way bees think and the way we think are radically different. The same neuron hardware works (thinks, that is) in very different ways. The way an LLM uses its neurons is radically different from both bees and people. It is alien to any form of life on earth. The algorithms are different. In some cases, they are loosely modeled on human thought processes. There are now efforts underway to fix some of the problems with AI. Some of the solutions are modeled on human thought processes. Some are based on logic, which is to say, ancient understanding of thought processes. If these new methods work, AI may become a little more like us. A little less alien. Some examples:
There is tremendous redundancy and waste in LLM because they cannot categorize effectively. They will have two separate categories for "blue" in the examples of "blue shirt" and "blue car" (an example given by a programmer). There are efforts underway to improve tokenization and to generalize tokens. That would bring "blue" into one adjective-like category that can be applied to wide range of nouns. This technique is borrowed from human thought. (They may have already done this. I read about this effort some time ago.)
Another example: there is tremendous effort going into the problem of training. An LLM has to look at thousands of images of cats before it develops a generalized, codified set of parameters that tell it an image probably shows a cat. A small child, on the other hand, can see one or two cats and she will quickly learn to identify a cat in real life or in a photo. Compared to LLM, the human brain and thought processes are thousands to millions of times better at learning from examples, and learning to identify objects in the real world. Or identifying concepts. Efforts are now underway to understand how the human brain learns, and what kinds of algorithms and shortcuts we can borrow from the human brain and apply to LLM, to improve their efficiency and reduce overhead, which takes a lot of computer power and physical energy. This problem is called "data inefficiency." ChatGPT tells me that researchers are looking at human cogitation methods to fix this:
"The problem you're referring to, where AI systems require a large number of examples to learn and generalize from visual data, is commonly known as "data inefficiency" or "data hunger" in AI research. It highlights the disparity between the way humans, especially children, can learn to recognize objects and concepts from just a few examples (a process known as "few-shot learning" or "one-shot learning") and the data-intensive nature of most machine learning models, particularly deep learning models, which often require vast amounts of labeled data to achieve similar levels of recognition accuracy.
Addressing data inefficiency is a significant challenge in the field of artificial intelligence because reducing the amount of labeled data needed for training can make AI systems more practical, cost-effective, and adaptable in real-world applications. Researchers are continually working on techniques like transfer learning, meta-learning, and few-shot learning algorithms to help AI systems generalize from limited data, similar to how humans do."