by: Dave Robinson
date: 7th November, 2011
For information only
Please comment on the forum: http://setiquest.org/forum/topic/alternative-display-method-waterfall-plot
Currently the methodology used for finding any signals within the spectra generated by Fourier Transforming the data from the ATA and stacking each spectrum into an array which has frequency along the X axis, and Time along the Y axis, such a diagram is known as a Waterfall diagram. Any signal shows up as a line down this array. A static tone will generate a vertical straight line down the array, as its position is time invariant. However in practice such a signal is probably the least interesting. Those from most astronomical sources exhibit what we can describe as a Doppler Gradient, where the frequency actually changes with time, often in a linear manner. This type of signal will generate a sloped straight line down the Waterfall diagram. The following picture shows a synthetic and ideal Waterfall image.
In practise, due to noise, the diagram will look more like the following: -
Here it can be seen that some of the signals have virtually vanished into the background noise, and are hardly visible at all, indeed in practice the signals may be so well below the noise level, that they are completely invisible.
The author noted that this sort of display spreads the information regarding its existence over many lines, and was interested to see if there was some form of transformation that could be applied to this data format to compact the information in the same way that a sine wave spread all over the time domain can be compressed into a single point using the Fourier Transform. To this end the author has decided to investigate the Radon Transform, and a modified version of the Hough Transform – both of which appear to offer the compaction that is required.
This document describes the initial experiments carried out to investigate the suitability of the Radon transform. It must be emphasized that the author has yet to try it on any real ATA data, and the results were generated using the above synthetic Waterfall data. This is in the form of a 1024 x 1024 sample array. The processing has been hacked together using MathCad, and has been reported here in order to ascertain whether such a process would be interesting to the SETI community bearing in mind the very positive results that came out of this experiment.
One technique that is applicable in these circumstances is that of column integration; this simply finds the sum of all of the samples/pixels along a column, and as the noise is bipolar, it tends to integrate to zero, whereas the signal itself continues to add. However this simple methodology has a major drawback, which is illustrated in the next diagram.
It is clear that this technique has made a really good job of extracting the two zero Doppler Gradient signals, but has lost all of the others; and considering that these are the least interesting signals in the Waterfall, as they are probably not from an astronomical source, the process appears to be rather academic, without much use.
With the development of modern CAT scanners has come much work on the so called Radon transform; essentially what it does is undertake a column integral, in much the same way that has just been illustrated, but then rotates the detectors, and repeats the process at another angle. The this results in an output array of column integrals taken over the complete circuit of the detectors. Does this have any application to the above problem? Well consider the case where we twist the Waterfall array slightly and redo the column integration.
This shows that we are now extracting signals that are at another Doppler Gradient. (Note that for clarity this process was done using the ideal noise free data, in order to demonstrate exactly what is going on).
The following image shows a limited Radon Transform made of the noisy image over a 10degree angle range at 0.1 degree increments. This result has been taken with the noisy image, to illustrate its noise reducing capabilities.
From this diagram, it is clearly possible to identify the position of the signals, from the very bright patches. What is also obvious that the information regarding the Doppler Gradient is available from the 'Y' position of these bright spots, so in this respect this image is providing more information than the standard Waterfall display, and has significant signal extraction advantages – condensing the signal into a single point has improved the visibility of signal considerably despite the almost overwhelming noise in the original Waterfall. Note that the intensity of the signal also gets mapped into the intensity of the bright spot – the smaller the signal, the dimmer the spot.
It is possible to post process this Radon Plane by noticing that each spot has a characteristic 'bow-tie' structure. Correlating this image with a template consisting of this bow-tie would truly condense the signal into a small point, which is the ultimate task of this research. But that is 'tomorrows' task.
Any feedback on the usefulness of this methodology would be gratefully received. My next task is to code it in Python/Scipy, but I don't want to go through this process if, for example, it has already been tried and found to unusable
Please comment about this on the forum: http://setiquest.org/forum/topic/alternative-display-method-waterfall-plot