I did a detailed signal analysis of the Exoplanet 060 data file and posted it to my baudline blog. There are a couple interesting drifting-random-walking and pulsing signals in this data file. Hydrogen even has sidebands and there are also a number of signal collection data flaws:
Like my Kepler-Exo4 blog post this is also a work-in-progress so I will be adding and updating things as I find them.
* Switched order of Zero Gaps and Power Fluctuation sections.
* Added a power fluctuation graph (dB vs. seconds).
I wonder what the ATA was doing to create that graph? It's like the target moved out of the beam and then back in.
This is outstanding analysis!
The "hydrogen sidebands" and "25600 Hz sidebands" should not appear in data recorded May 07 and later. Those sidebands were associated with a problem in later stages of the beamformer FPGA code that we believe to be fixed.
The "gaps" are zero-filled packets purposely inserted in the data where we find missing packets indicated by sequence number jumps while post-processing the recorded data. This problem should be fixed, or significantly reduced, for the latest data as well.
Unfortunately, the power fluctuation problem is still with us. It correlates with signal-processing room temperature and appears to be isolated for the most part to analog gain blocks in the RF-to-IF downconverters. There are a few good ideas for how to compensate for this, but no action has yet been taken.
You appear to have missed the 60 Hz and harmonics amplitude modulation :-) You will most likely have to look at the data in a different manner to see it.
"You appear to have missed the 60 Hz and harmonics amplitude modulation :-)"
Ha ha, Rob, that's pretty funny! Especially since we only discovered this amplitude modulation today... Rob beat me to the punch, but I can give a little background.
I want to be up front and say that our system for fast data recording (which generates these data) is new and still in the debugging stages. This system was developed for setiQuest to allow citizens access to our raw data streams. We don't pretend that the system functions perfectly yet, and we appreciate the work that ambitious scientists do to help us understand our system. We'll let you know about problems as we find them and we'll do our best to understand problems found by you.
About 60 Hz. A 60 Hz (line voltage) gain modulation in the amplifiers is a common defect in radio telescope data. There are about a dozen analog amplifiers in the signal chain from LNA (low noise amplifier in telescope receiver) to signal output. These amplifiers are very linear, but at the -70 dB level you can see nonlinear behavior which causes a "mixing" of the ever present 60 Hz signal with any strong signal that comes in. We've seen similar behavior at Arecibo and other radio telescopes.
In data previous to May 7, we were also seeing artifacts or "images" of strong features with a regular spacing of 1.455 MHz. As Rob mentioned, repairing a bug in the FPGA code fixed this, but it took several weeks for us to track down. Again, we're ramping up with a new instrument and these kinds of bugs are slightly embarassing but not unexpected.
Thanks again and good hunting!
Don't worry, bugs in the system, it is very understandable, that's just part of the engineering process. It is good to know that you are working on fixing them.
I looked around for the 60 Hz modulation you and Rob mentioned and I couldn't find any sign of it. I tried decimating way down and looking at plus and minus 60 Hz and its harmonics (120, 180, 240, ... ). No 60 Hz sidebands on the strong tones. Nothing. Does the Exo 060 have this flaw? If so, can you give me a hint to how I might go about seeing it?
Detect the signal power envelope (square the complex samples) and then pass the envelope as a real time-series into baudline. A plot of the 60 Hz modulation and harmonics was generated during our analysis using the following Octave (Matlab) code:
octave:5> envelope .-= std(envelope);
octave:7> title "2010-03-19-exo060 Envelope Spectrum"
octave:8> ylabel "log power"
octave:9> xlabel "frequency (Hz)"
Thanks, I didn't think of looking at the quadrature data like that. Setting baudline's Input Mapping time domain operation to magnitude now shows me the 60 Hz and a bunch of its harmonics. It's 59.864 Hz actually (: and yes that's 3 decimal points but it's only 4 significant digits since the last one is moving due to the power line frequency drifting! I'm generating a 60 Hz frequency fluctuation plot right now and I'm going to add it to the Advanced Analysis section on my Exo 060 blog post.
Just to be thorough here and in case you missed it; when you look at the magnitude power envelope do you see the huge elephants at 1/3 and 2/3 F? They have the 25600 Hz sidebands too. You've got some wild quadrature phase problem going on and I've never seen anything like it before. It is crazy yet incredibly fascinating at the same time. When I move the down mixer frequency the biggest 2/3 F tone stays stationary! I mean it doesn't move on my spectral display. This is analogous to changing the tuner on your radio and every station is playing the same polka song! And your radio isn't broken either. Wild, no it's paradoxically mind-bending! Decimating by 8 makes the elephant go away but it is visible in /2 and /4.
Just curious. Does line 5 "envelope .-= std(envelope);" mean that you are subtracting the standard deviation from the one second of data? I guess you're doing this to remove the DC component? If so then why not just use the mean or even better just leave the 0 Hz bin alone?
I just checked the AMC-7 and Kepler-Exo4 data files and they both exhibit the same quadrature phase problem ("elephant polka") I mentioned previously. The spike shapes are different but the stationary-while-down-mixing effect is the same. I hope this helps you track the bug down.
I am still in awe of this phenomena so I decided to try to do some reverse engineering. I looked at the cross-correlation and the phase of the transfer function for all three of the files. I see your filter and a bunch of phase errors. Very interesting and not what I expected. It is completely different for each file so I'm guessing different DSP code or filter parameters were used for each data collection run?
I figured out what was causing the stationary-while-down-mixing effect. It was nothing magical, just a unique sort of signal distortion but still a very interesting phenomena nonetheless. DSP is beautiful like that.
Funny thing is when I said "I had never seen anything like this before", I in fact had but I just didn't realize it. It's actually even funnier than that, the punchline is just one word and anyone reading this thread has seen it, so email me if you know the answer. (;
The "huge elephants" are found at multiples of ~ 1.45 MHz which are associated with the "hydrogen sidebands" in your previous analysis. We expect these to not appear in the latest data.
Yes those hydrogen sidebands are huge elephants but the ones I was referring to are even bigger and they are invisible! To see them you need to use the quadrature magnitude time-domain operation like you suggested above. It will be a while until I finish my Advanced Analysis blog section so here is a quick picture:
For this plot in baudline I average paste collected 8 seconds of data. The dB axis is the same for both curves so this is to scale. The green curve is the standard real Fourier view of the Q channel, the small bump near 250 kHz is hydrogen. The red curve is the real Fourier of the quadrature magnitude time-domain operation. The biggest spikes are at exactly 1/3 and 2/3 F, they also happen to have 25600 Hz sidebands. The spikes near 0 Hz are 25600 Hz and 8+ of it's harmonics.
We refer to the same artifacts in the same space. See this image.
Sorry, I didn't realize that. I thought you were still talking about hydrogen's sidebands in the complex frequency domain. Looks like you had a higher sampling rate for that image because your plot goes past Nyquist.
Did your solution also fix the 25600 Hz harmonics problem?
"It's 59.864 Hz actually (: and yes that's 3 decimal points but it's only 4 significant digits since the last one is moving due to the power line frequency drifting!"
That's weird. In past work I have often looked at high spectral resolution at 60 Hz when testing synthesizers etc., and it's always bang on. I don't know how the power companies achieve this, but they pretty much have to since many wall clocks rely on the 60 Hz as their frequency reference; ditto synchronous LP record players. Then again, Hat Creek (location of the ATA) has pretty funky power, being rural and at the end of a long run. But a 0.2% frequency error seems very odd. All the observatory gear is locked to an atomic time standard that is disciplined to agree with GPS. I wonder if the sample rate we provided you with could be off the actual rate somehow, although that seems very unlikely.
Dang nab it! Should have checked Wikipedia first, where I found:
"In North America, ..., a correction of ±0.02 Hz (0.033%) is applied. Time error corrections start and end either on the hour or on the half hour.
My bad :-(
Yup because of old fashioned wall clocks they have to make corrections so it's exactly 60 Hz. I always thought that the corrections were over 24 hours so that it averages to 60 Hz. Hourly corrections work too but what is important is that time doesn't drift long term, like all my quartz crystal clocks! Old fashioned technology is actually superior in this sense. (:
The AC line frequency actually drifts and jumps all over the place. It is quite fun to watch! Maybe I'll post more pictures of this, phase shift would be interesting, but here is something from a couple years ago:
What hardware did you use to record the signal. It looks similar to a recording I did some time ago of the 50Hz power line, similar drifts. But I concluded that it was soundcard drifts, no powerline drifts (other signals present had similar drifts).
Yes, that is what I thought at first too. So I decided to test this theory. In the Mystery Signal 11 answer section I discuss conducting a test using two different computers with two different sound cards with two different Linux versions in two different rooms. I remotely ran baudline over X11 and the 60 Hz wandering matched perfectly. So other than the baudline software, the only thing in common was the 60 Hz AC line power.
Now you are correct that ADC clock drift can cause a similar sort of phenomena. A couple years ago I conducted a Sample Rate Stability study that used NTP. If you like you can read more about that here:
The conclusion is that the ADC clocks in soundcards are of extremely low quality and that there are several techniques that can be used to help deal with this problem. Another item of interest is the soundcard clock errors are on the order of 2 ppm while the 60 Hz wander I measured was more on the order of 200+ ppm.
I found this picture of a page from Rob's lab book about the 60 Hz artifact in this exo060 dataset:
Nice lab book Rob. It reminds me of my Physics labs back in school but we attached our printed plots with glue sticks instead of tape! (:
Interesting mention of the NTSC vertical sync being a potential cause of the ~60 Hz artifact. The FCC turned off analog NTSC broadcasting back on February 17 2009 but the flyback of a nearby analog TV could emit a signal picked up by the ATA. I see NTSC artifacts in music recordings all the time with baudline. Even in recent recordings. NTSC just bleeds into everything.
I've heard that the ATA is a radio transmitter free zone. No cell phones and no wifi. It may be impractical but old analog NTSC TV should probably be added to that list. I read somewhere that the Parkes Observatory in Australia uses only diesel powered vehicles to eliminate the potential interference from spark ignition systems.
I subsequently demonstrated that the "60Hz" signal is from the cryogenic compressors in the feeds. My theory is that there is inductive coupling into the signal path. I don't remember where I posted the research results that led to this conclusion. Most likely to log.hcro.org or internal staff email. Perhaps one of the remaining staff can provide a reference. It is possible, using low-level commands issued to the rim-box control board to read monitor and control parameters, such as power, frequency, and temperature, from the compressor control board (also in the rim box). I showed that the compressors were running at the same frequency as the measured strong RFI signal.
How did you demonstrate that the cryogenic compressors were the cause? By turning them off and the 60 Hz signal going away?
Are the cryogenic coolers in the feed of each antenna? Other than a redesign of the coolers how would you fix it? With better shielding?
Thanks, glad you liked it. Your explanations make a lot of sense. Good luck on getting those problems fixed for the next data set. (:
There is two non-drifting signals in sigblip's analysis. Can't these be safely disregarded as terrestrial interference? Won't the rotation of the Earth manifest itself as drifting (or some other irregularity?) in signals from celestial sources due the Doppler effect?
Yes, the two non-drifting signals show zero Doppler drift so they likely are terrestrial. Even so they are extremely interesting signals, as in they have some form of modulated content, and I will be further investigating them in the TBD Advanced Analysis section.
It would be great if an Astrophysics expert would comment on what sort of Doppler drift would be expected from this particular ATA orientation and time and date. Does anyone know of a Doppler drift calculation applet? Something like that could be very handy.
* Digging in the Noise
Lot's of stuff deep down in the noise. What is up with all the drifting-random-walks? All different shapes too.
This section is as much about the Exo 060 data file as it about exploring many advanced DSP techniques.