I think the confusion is because strictly speaking $f(x) = O(g(x))$ is an abuse of notation. $O(g(n)), \Theta(g(n))$ and friends are sets. We can't say that a function equals a set, or that a function "is less" than another function, but notoriously mathematics runs on javascript, so we try to do something instead of giving a type error.
Here "is less" is interpreted as "eventually less for all values" and "plus a set" is interpreted as "plus any function of that set".
I never liked this notation for asymptotics and I always preferred the $f(x) \in O(g(x))$ style, but it's just notation in the end.
The reason it's preferred to use "=" instead of "\in" is because the way that Landau notation is generally used in practice is as a kind of ellipsis or placeholder. For example, the Taylor expansion e^x = 1 + x + O(x^2). I could just as well write e^x = 1 + x + ..., but the former conveys more meaning about what is hidden behind the ellipsis. It's an abuse of notation, but in the contexts that it's used, it's not clear what additional clarity using "\in" would bring over "=". Maybe also that big O is mainly used as a notation to facilitate doing calculations, less describing what family a function belongs to. Here are Knuth's thoughts, which I agree with: https://micromath.wordpress.com/2008/04/14/donald-knuth-calc...
To me it seems similar to the + C on an antiderivative (or more generally, quotient objects). Technically, you are dealing with an equivalence class of functions, so a set. But it's usually counterproductive to think of it that way (and when you study this stuff properly, one of the first things you do is prove that you (usually) don't need to, and can instead use an arbitrary representative as a stand-in for the set), so you write F(x)+C.
I think the Landau notation is a bit more finicky with the details. When it's really a quotient (like modular arithmetic) I'm with you, but here $O()$ morally means "at most this" and often you have to use the "direction of the inequality" to prove complexity bounds, so I'm more comfortable with the set notation. But again, it's just notation, I could use either.
It's actually a linear (more generally, abstract) algebra thing. (All, Differentiable, Smooth, or all sorts of other sets of) functions form a vector space. The derivative is a linear operator (generalized matrix). If you have a linear equation Ax=b, then if you can find some solution X, the general solution set is X+kerA, where kerA (the kernel or nullspace) is the set of all v where Av=0. What's the kernel of the derivative operator (i.e. what has 0 derivative)? Constant functions. So the general solution is whatever particular antiderivative you find plus any constant function.
You can do this sort of "particular solution plus kernel" analysis on any linear operator, which gives one strategy for solving linear differential equations. e.g. (aD^2+bD+cI) is a linear operator (weighted sums and compositions of linear operators are linear), so you can potentially do that analysis to solve problems like af''+bf'+cf=g. In that context you say the general solution is to add a homogeneous solution (af''+bf'+cf=0) to a particular solution (my intro differential equations class covered this but didn't have linear algebra as a prereq so naturally at the time it was just magic, like everything else presented in intro diffeq).
Although, when I learned foundations of mathematics, every function was a set, and if you wanted them, you'd get plenty of junk theorems from that fact.
I feel its not that bad an abuse of notation as kinda consistent with other areas of mathematics -
A coset, quotients r + I, affine subspaces v + W, etc. Not literal sets but comparing some representative with a class label, and the `=, +` is defined not just on the actual objects but on some other structure used to make some comparison too.
> computer science students should be familiar with the standard f(x)=O(g(x)) notation
I have always thought that expressing it like that instead of f(x) ∈ O(g(x)) is very confusing. I understand the desire to apply arithmetic notation of summation to represent the factors, but "concluding" this notation with equality, when it's not an equality... Is grounds for confusion.
The easiest way to read it is "there exists a function h in O(1) such that f(x) <= g(x) + h(x)."
I think first we should teach "f in O(g)" notation, then teach the above, then observe that a special case of the above is the "abuse of notation" f(x) = O(g(x)).
You do things a bit different on an actual computer, right. As x cannot tend to infinity (so everything is O(1) by that measure) so common sense is applied.
O(1) just means "a bounded function (on a neighborhood of infinity)". Unlike f(x), which refers to some function by name, O(1) refers to some function by a property it has. It's the same principle at work in "even + odd = odd".
Programmers wringing their hands over the meaning of f(x)=O(g(x)) never seem to have manipulated any expression more complex than f(x)=O(g(x)).
Here "is less" is interpreted as "eventually less for all values" and "plus a set" is interpreted as "plus any function of that set".
I never liked this notation for asymptotics and I always preferred the $f(x) \in O(g(x))$ style, but it's just notation in the end.
You can do this sort of "particular solution plus kernel" analysis on any linear operator, which gives one strategy for solving linear differential equations. e.g. (aD^2+bD+cI) is a linear operator (weighted sums and compositions of linear operators are linear), so you can potentially do that analysis to solve problems like af''+bf'+cf=g. In that context you say the general solution is to add a homogeneous solution (af''+bf'+cf=0) to a particular solution (my intro differential equations class covered this but didn't have linear algebra as a prereq so naturally at the time it was just magic, like everything else presented in intro diffeq).
Lean is much more notorious for mathematics.
A coset, quotients r + I, affine subspaces v + W, etc. Not literal sets but comparing some representative with a class label, and the `=, +` is defined not just on the actual objects but on some other structure used to make some comparison too.
I have always thought that expressing it like that instead of f(x) ∈ O(g(x)) is very confusing. I understand the desire to apply arithmetic notation of summation to represent the factors, but "concluding" this notation with equality, when it's not an equality... Is grounds for confusion.
it's a notation for "some element of that set"
f(x) = g(x) + O(1)
f(x) - g(x) = O(1)
You get:
Now, if you already know that means "f and g eventually differ by no more than a constant", then must mean "f eventually stops exceeding g by a constant".I think first we should teach "f in O(g)" notation, then teach the above, then observe that a special case of the above is the "abuse of notation" f(x) = O(g(x)).
Programmers wringing their hands over the meaning of f(x)=O(g(x)) never seem to have manipulated any expression more complex than f(x)=O(g(x)).