Summarizer

Neurosymbolic AI Approaches

References to traditional neurosymbolic computing debates, with some dismissing this as 'old neurosymbolic garbage restated' while others see potential in embedding computational primitives into LLMs.

← Back to Executing programs inside transformers with exponentially faster inference

While some critics dismiss modern neurosymbolic efforts as a repackaging of "old garbage," others find the prospect of embedding hardcoded computational primitives into transformer weights to be a fascinating ideological shift. The debate centers on whether a system must be able to execute symbolic logic to truly "internalize" and comprehend information, a claim that remains polarizing even among those who find the research conceptually "cool." Despite this intrigue, practical skepticism persists due to the lack of released weights and compilers necessary for independent experimentation. Ultimately, the discourse reflects a deep-seated tension between purely neural architectures and the belief that formal logic is a prerequisite for genuine machine intelligence.

4 comments tagged with this topic

View on HN · Topics
This seems like it has some potential, but is pretty much useless as it is. Shame there are no weights released - let alone the "compiler" tool they used to actually synthesize computational primitives into model weights. It seems like a "small model" system that's amenable to low budget experiments, and I would love to see what this approach can be pushed towards. I disagree with the core premise, it's basically the old neurosymbolic garbage restated, but embedding predefined computational primitives into LLMs could have some uses nonetheless.
View on HN · Topics
If you want to experiment with hardcoding small programs into transformer weights, maybe try ALTA: https://arxiv.org/abs/2410.18077v2
View on HN · Topics
>This shows the downside of using AI to write up your project. I see the eloquent sentences, but don't get the message. Not really sure what this obsession with calling things you don't like AI generated is but it's poor form. If you have something to say about the text then say it. Otherwise leave baseless accusations out of it. >What's the benefit? Is it speed? Where are the benchmarks? Is it that you can backprop through this computation? Do you do so?.... It's pretty clearly an ideological thing. Some people are firmly on the 'some sort of symbolic logic is necessary' camp. From the article, 'A system that cannot compute cannot truly internalize what computation is.' Some things are just interesting for the sake of it. This is one of those things. I don't agree with the authors on the above and I'm still glad they shared. It's a very interesting read regardless.
View on HN · Topics
one of the most interesting pieces I've read recently. Not sure I agree with all the statements there (e.g. without execution the system has no comprehension) - but extremely cool