Discussion Forums

Algorithm Performance Data

14 replies [Last post]
tapas1982
Offline
Joined: 2011-05-16
Posts: 11

Inspired by the SetiLive project I have an idea  to detect the signals from the Waterfall plot images.As I have read from the SETI web site that an human eye can search for signal than any other software programs.My idea is to apply the concepts of Computer vision to detect signals in the waterfall plots.

 

I am trying to develop some programs with the idea.But I am sure SETI with so many years of experience might have better performing algorithms. The only way I can know that I have found a better algorithm is by comparing my results with SETI's algorithm performance data.Is there any way I can find these data?Is there was I can find the information that given a waterfall plot image how fast the current system is able to detect a signal.

 

If I have the information then I can compare my algorithms and see if I am able to find anything better to serve the purpose.

tapas1982
Offline
Joined: 2011-05-16
Posts: 11
Example

Input Waterfal Plot :

Input

Signal Detected by My program :

Time Taken :  1.371 Seconds

We will be needing such test data to verify if the algorith is better than the existing standard algorithms.It will be also great if we can classify such data based on types of signals.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
can you describe your algorithm a little?

Looks like you've cleaned up the original using an averaging or median filter of some sort.  Or are you doing something a little more sophisticated like a MRF?  Also it seems that your detection algorithm still requires a human in the loop to detect the line. Is your plan to run the cleaned up version through a classifier to assert a detection?

Also, what are the "types of signals" that you wish to classify?

tapas1982
Offline
Joined: 2011-05-16
Posts: 11
It is an naive approach.Just

It is an naive approach.Just trying to figure out the requirements and be ready with the test data.You are right I have cleaned up the unwanted dots.

Right now it is just identifying the straigh line signals acurately.But I have plans to make it more sofisticated.For this I need more data.

The line is drawn by my program.That means I know the exact co-ordinates in the plot.That means no human interaction is required.In this example the program draws a line but I can report it in any other required format.

Thanks for your response.Will appreciate any further comments in details so that I can think of improving my algorithm.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
here's a link to data

seti data  there's a lot files and some example spectrum to calibrate your algorithm.

My comment for you is how do you know there is a signal to begin with.  Seems like your program will create a line from the input file regardless to the presence/absence of a signal.  The example you showed seems to have a signal, so the output looks good.  But what if the file contained only noise?  My guess is that it will still produce some kind of line.

Seems like an automated system would first do a coarse signal detection like a simple energy detector and then run a stronger algorithm like the one you are proposing.  This should help reduce the computation time to evaluate a large number of files.  Of course, you have to deal with the trade-off of probability-of-detect versus probablity-false-alarm.

tapas1982
Offline
Joined: 2011-05-16
Posts: 11
Thanks. My approach is still

Thanks.

My approach is still not perfect.But I am working on it.But it is not generating random lines.It is generating lines only if there is a singnal.As it is not perfect it doesn't generate a line sometimes even if there is a singnal.I have to fix that part.

tapas1982
Offline
Joined: 2011-05-16
Posts: 11
I saw the data link.But I am

I saw the data link.But I am not able to figure out how automatically I can download the data.For example the following link has some data :

http://setiquest.sigblips.com/analysis/2010-11-06-dorothy_etaareitus_142...

But what is the logic to get the next set of plots? Is there a naming format?

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
that's part of the fun i suppose

each collect has a data directory

the *.dat files are the IQ sample files.  there's a readme file to tell you how to parse the data. then you can create your own spectrograms. or maybe right-click and save the images available?

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
I'm not trying to discourage you.

Just asking questions that I would think about if I was developing this algorithm.

I believe you that it isn't generating random lines, but I am curious how you know there is a signal in the first place?

What would happen if you created an image file of just Gaussian noise and ran that file through your algorithm? How would your algorithm know there is not a signal present?

tapas1982
Offline
Joined: 2011-05-16
Posts: 11
Ok.I understand now.I took

Ok.I understand now.I took the image from the SETI live website.Where they allow users to mark signals.I took a screen shot and using it as a test data to develop a process which can automatically mark these signals.Out inputs might be different.Let me know.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
i haven't looked at setilive stuff.

Do they tell you apriori that the file contains a signal and you are suppose to mark it?
Like I said, I haven't looked at the SETI live stuff, but seems like it wouldn't know in advance (kinda the point).
Ultimately, if you don't know in advance that an image contains a signal, then how do you know if there is a signal?

tapas1982
Offline
Joined: 2011-05-16
Posts: 11
The goal of SETI live is to

The goal of SETI live is to let common users mark the signals.As humans can see the signals easily they can mark it.

You will get a good idea once you try to see the SETI live website.They have a very nice and simple tutorial.

I started my study from the SETI live website.So I am not knowledgeable about how to generate Waterfall plots (Images) from the .dat files.Will appreciate any tips regarding this.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
ah so there is still a human in the loop

the human would have to select which file to run your algorithm on.

basically it's a plot of frequency versus time ... i'm unsure why they are called waterfall plots here. in the communications field, watefall plots refer to bit-error rates to see how close you are to channel capacity.  I would google spectrogram.

i don't know how familiar you are with signal processing, so it might not be worth my time to explain them further at the moment.

tapas1982
Offline
Joined: 2011-05-16
Posts: 11
Thanks for the input.I am

Thanks for the input.I am naive to this and trying to find out my ways.I don't know much about waterfall plots.But I will google as you suggested.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
i'd suggest reading up on the fourier transform

the DFT (more general) or the FFT (fast implementation of DFT).

this understanding should help you with the waterfall plots.