There's a lot of value in the implementation of many strong and fast algeorithms in computer algebra in proprietary tools such as Maple, Wolfram, Matlab. However, I (though of course believe that such work needs to be compensated) find it against the spirit of science to keep them from the general public. I think it would be good service to use AI tools to bring open source alternatives like sympy and sage and macaulay to par. There's really A LOT of cool algorithms missing (most familiar to me are some in computational algebraic geometry)
Additionally I think because of how esoteric some algorithms are, they are not always implemented in the most efficient way for today's computers. It would be really nice to have better software written by strong software engineers who also understands the maths for mathematicians. I hope to see an application of AI here to bring more SoTA tools to mathematicians--I think it is much more value than formalization brings to be completely honest.
There's a great discussion with Stephen Wolfram on the Sean Carroll podcast. Listening to it made me think very highly of Wolfram. He's a free thinking, eccentric, mathematician, scientist; who got started doing serious work at a very young age. He still has a youthful creative approach to thought and science. I hope LLMs do pair well with his tools.
I'm a fan of his work and person too. Not a fanatic or evangelical level, but I do think he's one of the more historically relevant computer scientists and philosophers working today. I can overlook his occasional arrogance, and recognize that there's a genuine and original thinker who's been pursuing truth and knowledge for decades.
I tried using wolfram alpha as a tool for an llm research agent, and I couldn't find any tasks it could solve with it, that it couldn't solve with just Google and Python.
Well sure, in theory any mathematical problem can be solved with any Turing complete programming language. I think the idea here is that for certain problem domains Mathematica might be more efficient or easier for humans to understand than Python.
I like Mathematica and use it regularly. But I did not see any benefits of using it over python as a tool that Claude Code can use. Every script it produced in wolfram was slower with worse answers than python. Wolfram people are really trying but so far the results are not very good.
sympy is good enough for typical uses. the user interface is worse but that doesn't matter to Claude. I imagine if you have some really weird symbolic or numeric integrals, Mathematica may have some highly sophisticated algorithms where it would have an edge.
however, even this advantage is eaten away somewhat because the models themselves are decent at solving hard integrals.
I like to think of Claude as enjoying himself more when working with good tools rather than bad ones. But metaphysics aside, tools that have the functions you would expect, by the names you would expect, with the behavior you would expect, do seem to be just as important when the users are LLMs.
I think the problem is just not enough training on that specific language because it's proprietary. Most useful Mathematica code is on someone's personal computer, not GitHub. They can build up a useful set of training data, some benchmarks, a contest for the AI companies to score high on, because they do love that kind of thing.
But for most internet applications (as opposed to "math" stuff) I would think Python is still a better language choice.
CAG sounds like fake solution for LLM's. Math problems are not custom data, they are limited in amount, and do not refresh like product manuals.
Hence math can always be part either generic llm or math fine tuned llm, without weird layer made for human ( entire wolfram) and dependencies.
Wolfram alpha was always an extra translation layer between machine and human. LLM's are a universal translation layer that can also solve problems, verify etc.
You wouldn't use an LLM to solve a big Linear Programming problem, because it would cost way more than using the Simplex Method, and you'd be worried that it might be wrong.
Aside, I hate the fact that I read posts like these and just subconsciously start counting the em-dashes and the "it's not just [thing], it's [other thing]" phrasing. It makes me think it's just more AI.
The other day I formatted a sentence out loud in the "it's not just x it's y" structure and immediately felt gross, despite having done it probably a million times in my lifetime. That was an out-of-body feeling.
In George Orwell's essay "Politics and the English Language," [0] one of his primary recommendations for writing well is to "Never use a metaphor, simile, or other figure of speech which you are used to seeing in print."
"It's not just X, it's Y" definitely seems to qualify today. It's a stale way to express an idea.
I hadn't revisited that essay since LLMs became a thing, but boy was it prescient:
> By using stale metaphors, similes, and idioms [and LLMs], you save much mental effort, at the cost of leaving your meaning vague, not only for your reader but for yourself ... But you are not obliged to go to all this trouble. You can shirk it by simply throwing your mind open and letting the ready-made phrases come crowding in. They will construct your sentences for you — even think your thoughts for you, to a certain extent — and at need they will perform the important service of partially concealing your meaning even from yourself.
The em-dash metric is silly. Some people (including me) have always used them and plan to continue to do so. I just pulled up some random articles by Wolfram from the before-LLM days and guess what: em-dashes everywhere. One sample from 2018 had 89 of them. Wolfram has always written in the same style (which, admittedly, can be a bit self-aggrandizing and verbose). It’s kinda weird to see people just blowing it off as AI slop just because of a —.
LLMs use the em-dash excessively but correctly. This post is littered with them in places they don't belong which makes it look decidedly human, as if written by someone who believes that random em-dashes make their writing look more professional, while actually having the opposite effect.
If you really want to know: more than one emmy-dash per paragraph is probably excessive.
> LLMs don’t—and can’t—do everything. What they do is very impressive—and useful. It’s broad. And in many ways it’s human-like. But it’s not precise. And in the end it’s not about deep computation.
This is a mess. What is the flow here? Two abrupt interrupts (and useful) followed by stubby sentences. Yucky.
Idk about the grammatical correctness of the punctuation, but I really enjoyed reading his writing. Never read something by him before, it was genuinely refreshing, specially given it was a glorified ad.
>"But an approach that’s immediately and broadly applicable today—and for which we’re releasing several new products—is based on what we call
computation-augmented generation, or CAG.
The key idea of CAG is to inject in real time capabilities from our foundation tool into the stream of content that LLMs generate. In traditional retrieval-augmented generation, or RAG, one is injecting content that has been retrieved from existing documents.
CAG is like an infinite extension of RAG
, in which an infinite amount of content can be generated on the fly—using computation—to feed to an LLM."
We welcome CAG -- to the list of LLM-related technologies!
Additionally I think because of how esoteric some algorithms are, they are not always implemented in the most efficient way for today's computers. It would be really nice to have better software written by strong software engineers who also understands the maths for mathematicians. I hope to see an application of AI here to bring more SoTA tools to mathematicians--I think it is much more value than formalization brings to be completely honest.
Unfortunately, the bank doesn't accept spirit of science dollars, and neither does the restaurant down the street from me either.
Stephen Wolfram on Computation, Hypergraphs, and Fundamental Physics - https://podbay.fm/p/sean-carrolls-mindscape-science-society-... (2hr 40min)
I'm a fan of his work and person too. Not a fanatic or evangelical level, but I do think he's one of the more historically relevant computer scientists and philosophers working today. I can overlook his occasional arrogance, and recognize that there's a genuine and original thinker who's been pursuing truth and knowledge for decades.
however, even this advantage is eaten away somewhat because the models themselves are decent at solving hard integrals.
But for most internet applications (as opposed to "math" stuff) I would think Python is still a better language choice.
Even the documentation search is available:
```bash
/Applications/Wolfram.app/Contents/MacOS/WolframKernel -noprompt -run '
Needs["DocumentationSearch`"];
result = SearchDocumentation["query term"];
Print[Column[Take[result, UpTo[10]]]];
Exit[]'
```
Hence math can always be part either generic llm or math fine tuned llm, without weird layer made for human ( entire wolfram) and dependencies.
Wolfram alpha was always an extra translation layer between machine and human. LLM's are a universal translation layer that can also solve problems, verify etc.
Aside, I hate the fact that I read posts like these and just subconsciously start counting the em-dashes and the "it's not just [thing], it's [other thing]" phrasing. It makes me think it's just more AI.
e.g. https://writings.stephenwolfram.com/2014/07/launching-mathem...
"It's not just X, it's Y" definitely seems to qualify today. It's a stale way to express an idea.
I hadn't revisited that essay since LLMs became a thing, but boy was it prescient:
> By using stale metaphors, similes, and idioms [and LLMs], you save much mental effort, at the cost of leaving your meaning vague, not only for your reader but for yourself ... But you are not obliged to go to all this trouble. You can shirk it by simply throwing your mind open and letting the ready-made phrases come crowding in. They will construct your sentences for you — even think your thoughts for you, to a certain extent — and at need they will perform the important service of partially concealing your meaning even from yourself.
[0]: https://bioinfo.uib.es/~joemiro/RecEscr/PoliticsandEngLang.p...
Does he speak the same way - pausing for emphasis?
Somehow I don't think "trying to make my writing look professional" is very high on the priority list.
> LLMs don’t—and can’t—do everything. What they do is very impressive—and useful. It’s broad. And in many ways it’s human-like. But it’s not precise. And in the end it’s not about deep computation.
This is a mess. What is the flow here? Two abrupt interrupts (and useful) followed by stubby sentences. Yucky.
computation-augmented generation, or CAG.
The key idea of CAG is to inject in real time capabilities from our foundation tool into the stream of content that LLMs generate. In traditional retrieval-augmented generation, or RAG, one is injecting content that has been retrieved from existing documents.
CAG is like an infinite extension of RAG
, in which an infinite amount of content can be generated on the fly—using computation—to feed to an LLM."
We welcome CAG -- to the list of LLM-related technologies!