Summarizer

Neural Networks vs Biology

Comparison between artificial neural networks and biological brains, noting differences in learning mechanisms and questioning whether deep learning parallels biological intelligence

← Back to There Will Be a Scientific Theory of Deep Learning

The debate over whether deep learning parallels biological intelligence suggests that while artificial neural networks utilize powerful end-to-end optimization, they may be fundamentally orthogonal to the brain's real-time, "reservoir-like" processing. Commenters highlight that biological systems benefit from structures hard-coded by natural selection and the ability to learn and act simultaneously, whereas AI remains limited by a rigid separation between training and inference. Furthermore, the lack of coherent internal world models in current AI makes it difficult for machines to replicate the flexible, object-oriented reasoning that allows even a two-year-old to navigate physical reality. Ultimately, this gap raises the question of whether deep learning is a mastered engineering feat or merely an exploitation of mysterious mathematical principles that we have yet to scientifically theorize.

7 comments tagged with this topic

View on HN · Topics
Comparing Deep Learning with neuroscience may turn out to be erroneous. They may be orthogonal. The brain likely has more in common with Reservoir Computing (sans the actual learning algorithm) than Deep Learning. Deep Learning relies on end to end loss optimization, something which is much more powerful than anything the brain can be doing. But the end-to-end limitation is restricting, credit assignment is a big problem. Consider how crazy the generative diffusion models are, we generate the output in its entirety with a fixed number of steps - the complexity of the output is irrelevant. If only we could train a model to just use Photoshop directly, but we can't. Interestingly, there are some attempts at a middle ground where a variable number of continuous variables describe an image: < https://visual-gen.github.io/semanticist/ >
View on HN · Topics
If you think a 2 year old is doing deep learning, you're probably wrong. But if you think natural selection was providing end to end loss optimization, you might be closer to right. An _awful lot_ of our brain structure and connectivity is born, vs learned, and that goes for Mice and Men.
View on HN · Topics
> If only we could train a model to just use Photoshop directly, but we can't. It is probably coming, I get the impression - just from following the trend of the progress - that internal world models are the hardest part. I was playing with Gemma 4 and it seemed to have a remarkable amount of trouble with the idea of going from its house to another house, collecting something and returning; starting part-way through where it was already at house #2. It figured it out but it seemed to be working very hard with the concept to a degree that was really a bit comical. It looks like that issue is solving itself as text & image models start to unify and they get more video-based data that makes the object-oriented nature of physical reality obvious. Understanding spatial layouts seems like it might be a prerequisite to being able to consistently set up a scene in Photoshop. It is a bit weird that it seems pulling an image fully formed from the aether is statistically easier than putting it together piece by piece.
View on HN · Topics
Ive yet to see a model that trains AND applies the trained data real-time. Thats basically every living being, from bacteria to plants to mammals. Even PID loops have a training phase separate from recitation phase.
View on HN · Topics
I suspect all Minsky did was reinforce what many people were already thinking. I experimented with neural nets in the late 80s and they seemed super interesting, but also very limited. My sense at the time was that the general thinking was, they might be useful if you could approach the number of neurons and connections in the human brain, but that seemed like a very far off, effectively impossible goal at the time.
View on HN · Topics
It's actually really fascinating that there isn't a scientific theory of deep learning, especially as it's a product of human engineering as opposed to e.g. biology or particle physics.
View on HN · Topics
Calling it “a product of human engineering” is misleading. Deep learning exploits principles we don’t fully understand. We didn’t engineer those principles. It’s not fundamentally any different than particle physics or biology, which are both similarly consequences of rules that we didn’t invent and can’t control.