The Shannon Sampling Theorem was used in another thread of an example of an area in which there are misconceptions that are widely held. The question was raised as just what the Shannon Sampling Theorem is. This thread is a response to that question.
In many applications data that is fundamentally analog in nature, as is the case with most real physical data, is obtained by sampling a data channel at intervals. That results in what is called 'sampled data'. Nearly all data is obtained in this manner, as a practical issue. So what you have is a sequence of data points that represents, in some manner yer to be determined, what is in reality a continuous curve.
The question then arises as to how that set of data points should be interpreted so as to accurately represent the original continuous curve. The question also arises as to how rapidly the data needs to be sampled in order to obtain an accurate representation.
What is typically done, and most people have done this in some sort of laboratory class, is to simply plot the data points on a sheet of paper and "connect the dots".. Sometimes the connecting is done with a French curve, or on a computer using a spline function. But the technique is ultimately just connecting dots. In this process there is relatively thought given to the rate at which data is sampled, beyon the "more is better" realization.
Claude Shannon, in his studies that led to the development of modern informatin theory, addressed the problem of sampled data approximation more sysematically. He proved what is presented in electrical engineering classes on signal theory or information theory as the Shannon Sampling Theorem. That theorem shows that given a band-limited time signal, with maximum frequency W, that if one samples the signal at a frequency of at least 2W that the signal can be reconstructed exactly using only the sampled data points. The reconstruction is effected through an infinite series of so-called sync funtions (function that look like six(x)/x up to scaling and shifts along the time axis) whith coefficients taken from the sampled data points.
Note two things: 1) The reconstructin is NOT done by "connecting the dots", but rather with an infinite series of fundtions of a specific type. 2) It is critical to theproof that the function be band-limited.
What is a band-limited function? Start with an ordinary function of time, f. Then for "nice" functions f one can define what is called the Fourier transform of f, call it F (nice here generally means that the absolute value of f has a finite integral). F is defined from f via an integral against a complex exponential function. Fourier transforms have some nice properties relative to operations like differentiation and convolution that are exploited in many applications, electrical engineering communication theory being one such application. f is called band limited if F is zero outside of some interval, and the largest value for which it is not zero is the limit W to which reference is made in the Shannon Sampling theorem. (This could be made more clear if this forum supported the necessary mathematical symbology -- i.e. if we could post graphics or if we could compose in Tex).
So Shannon's theorem says that if you have a band-limited signal and if you sample at at least twice the highest frequency then you have enough information to completely determine the original signal from the samples. It also tells you how to go about reconstructing that signal. This theorem is widely used in engineering applications for handling data. This theorem is one reason why data acquisition systems usually have what are called "anti-aliasing" filters in the signal flow ahead of the actual sampling in order to limit the frequency band of the incoming signal. That avoids distortion that occurs when the sampling rate is too low relative to the frequency content of the signal. So you don't really samply the actual signal but only a filtered version of it. But as is the case with all physically realizable filters, the band limitation is not perfect.
What is discussed above is a fairly complete synopsis of the Shannon Sampling Theorem as it is usually presented in university classes. In only a few classes is the following point made: All practical time signals are time limited (they are zero outside of some finite time interval) since there are no practical signals that extend to either positive or negative infinity in time, and no time-limited signal is also band-limited unless it is zero everywhere. . The fact that a time-limited signal cannot be band-limted unless itis identically zero is a consequence of the Paley-Weiner theorem in harmonic analysis. It is well-known in the mathematical community, but only seems to be recognized in the engineering community by a few people. In many texts one simply sees the assumption made that a given signal is band-limited as the presentation proceeds from that point, with no mention of the fact that no such practical signals exist -- that strikes me as a rather cavalier attitude.
The remainder of what I will discuss seems to be limited to a very few people, and is not at all widely known.
The question arises as to what sort of error one might expect if one tries to apply the Shannon Sampling Theorem to a real world signal, one that is not truly band-limited. This should be of interest since the proof of the theorem relies very heavily on the hypothesis that the signal is indeed band-limited. The answer is that if you "pretend" that the signal is band-limited, play the sampling and reconstruction game as per the theorem then the error in reconstructing the signal permits a maximum error in reconstruction, uniformly, of no more than twice the absolute value of the integral of the Fourier transform outside the assumed band. So as I know this result is published only in an old (1967) volume of the IBM Journal. I had to work it out for myself, but a colleague found this paper while he wasd checking my work.
The question also arises as to how big the error might be if one "connects the dots" rather than uses the infinite series of sync functions requred by the theorem. The answer here is rather troubling. Even if the signal is band-limited (it won't in general be time-limited for this construction) you can construct a function for which "connecting the dots" results in any arbitrary pre-selected error, no matter now large. Thus the usual practice of simply joining data points is fraught with peril.
In many applications data that is fundamentally analog in nature, as is the case with most real physical data, is obtained by sampling a data channel at intervals. That results in what is called 'sampled data'. Nearly all data is obtained in this manner, as a practical issue. So what you have is a sequence of data points that represents, in some manner yer to be determined, what is in reality a continuous curve.
The question then arises as to how that set of data points should be interpreted so as to accurately represent the original continuous curve. The question also arises as to how rapidly the data needs to be sampled in order to obtain an accurate representation.
What is typically done, and most people have done this in some sort of laboratory class, is to simply plot the data points on a sheet of paper and "connect the dots".. Sometimes the connecting is done with a French curve, or on a computer using a spline function. But the technique is ultimately just connecting dots. In this process there is relatively thought given to the rate at which data is sampled, beyon the "more is better" realization.
Claude Shannon, in his studies that led to the development of modern informatin theory, addressed the problem of sampled data approximation more sysematically. He proved what is presented in electrical engineering classes on signal theory or information theory as the Shannon Sampling Theorem. That theorem shows that given a band-limited time signal, with maximum frequency W, that if one samples the signal at a frequency of at least 2W that the signal can be reconstructed exactly using only the sampled data points. The reconstruction is effected through an infinite series of so-called sync funtions (function that look like six(x)/x up to scaling and shifts along the time axis) whith coefficients taken from the sampled data points.
Note two things: 1) The reconstructin is NOT done by "connecting the dots", but rather with an infinite series of fundtions of a specific type. 2) It is critical to theproof that the function be band-limited.
What is a band-limited function? Start with an ordinary function of time, f. Then for "nice" functions f one can define what is called the Fourier transform of f, call it F (nice here generally means that the absolute value of f has a finite integral). F is defined from f via an integral against a complex exponential function. Fourier transforms have some nice properties relative to operations like differentiation and convolution that are exploited in many applications, electrical engineering communication theory being one such application. f is called band limited if F is zero outside of some interval, and the largest value for which it is not zero is the limit W to which reference is made in the Shannon Sampling theorem. (This could be made more clear if this forum supported the necessary mathematical symbology -- i.e. if we could post graphics or if we could compose in Tex).
So Shannon's theorem says that if you have a band-limited signal and if you sample at at least twice the highest frequency then you have enough information to completely determine the original signal from the samples. It also tells you how to go about reconstructing that signal. This theorem is widely used in engineering applications for handling data. This theorem is one reason why data acquisition systems usually have what are called "anti-aliasing" filters in the signal flow ahead of the actual sampling in order to limit the frequency band of the incoming signal. That avoids distortion that occurs when the sampling rate is too low relative to the frequency content of the signal. So you don't really samply the actual signal but only a filtered version of it. But as is the case with all physically realizable filters, the band limitation is not perfect.
What is discussed above is a fairly complete synopsis of the Shannon Sampling Theorem as it is usually presented in university classes. In only a few classes is the following point made: All practical time signals are time limited (they are zero outside of some finite time interval) since there are no practical signals that extend to either positive or negative infinity in time, and no time-limited signal is also band-limited unless it is zero everywhere. . The fact that a time-limited signal cannot be band-limted unless itis identically zero is a consequence of the Paley-Weiner theorem in harmonic analysis. It is well-known in the mathematical community, but only seems to be recognized in the engineering community by a few people. In many texts one simply sees the assumption made that a given signal is band-limited as the presentation proceeds from that point, with no mention of the fact that no such practical signals exist -- that strikes me as a rather cavalier attitude.
The remainder of what I will discuss seems to be limited to a very few people, and is not at all widely known.
The question arises as to what sort of error one might expect if one tries to apply the Shannon Sampling Theorem to a real world signal, one that is not truly band-limited. This should be of interest since the proof of the theorem relies very heavily on the hypothesis that the signal is indeed band-limited. The answer is that if you "pretend" that the signal is band-limited, play the sampling and reconstruction game as per the theorem then the error in reconstructing the signal permits a maximum error in reconstruction, uniformly, of no more than twice the absolute value of the integral of the Fourier transform outside the assumed band. So as I know this result is published only in an old (1967) volume of the IBM Journal. I had to work it out for myself, but a colleague found this paper while he wasd checking my work.
The question also arises as to how big the error might be if one "connects the dots" rather than uses the infinite series of sync functions requred by the theorem. The answer here is rather troubling. Even if the signal is band-limited (it won't in general be time-limited for this construction) you can construct a function for which "connecting the dots" results in any arbitrary pre-selected error, no matter now large. Thus the usual practice of simply joining data points is fraught with peril.
Facebook
Twitter
Instagram