What are Basis Functions?

It is simpler to explain a basis function if we move out of the realm of analog (functions) and into the realm of digital (vectors) (*). Every two-dimensional vector (x,y) is a combination of the vector (1,0) and (0,1). These two vectors are the basis vectors for (x,y). Why? Notice that x multiplied by (1,0) is the vector (x,0), and y multiplied by (0,1) is the vector (0,y). The sum is (x,y).

The best basis vectors have the valuable extra property that the vectors are perpendicular, or orthogonal to each other. For the basis (1,0) and (0,1), this criteria is satisfied.

Now let's go back to the analog world, and see how to relate these concepts to basis functions. Instead of the vector (x,y), we have a function f(x). Imagine that f(x) is a musical tone, say the note A in a particular octave. We can construct A by adding sines and cosines using combinations of amplitudes and frequencies. The sines and cosines are the basis functions in this example, and the elements of Fourier synthesis. For the sines and cosines chosen, we can set the additional requirement that they be orthogonal. How? By choosing the appropriate combination of sine and cosine function terms whose inner product add up to zero. The particular set of functions that are orthogonal and that construct f(x) are our orthogonal basis functions for this problem.