Discussion Forums

Community Meeting 2011-10-25

36 replies [Last post]
sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733

[10:59am] _Jens_: hello everybody
[10:59am] khrm: hello.
[11:00am] jrseti_: Hello! It is 11am California time
[11:01am] jrseti_: Jane (janebird) will not be here today, I assume. She is testing the SonATA observing
[11:01am] _Jens_: does that mean, that today indeed is the big day?
[11:01am] jrseti_: No, unfortunately! I'll explain that.
[11:02am] jrseti_: Agenda 1) ATA RE-LAUNCH. 2) GSoC Mentor Summit, 3) Ubuntu port
[11:02am] sigblips: 4) Empty pages on http://setiquest.org
[11:03am] jrseti_: Also: Michael is in China on vacation - won't be here
[11:03am] jrseti_: 1) ATA relaunch
[11:03am] jrseti_: Today was supposed to be the day!
[11:03am] jrseti_: But.....
[11:04am] jrseti_: We have been having trouble with the cooling of the signal processing room. We thought we were going to have to get more expensive air conditioning equipment.
[11:05am] jrseti_: But, someone suggested to have a tech come out and check out the existing air conditioner. Turned out that it had problems.
[11:05am] jrseti_: That was good news. It took several days to fix, but now the room is much cooler.
[11:06am] jrseti_: As a result, we had several days where we could not do any testing, and did not have time to get it done by today
[11:06am] _Jens_: so what's the new date?
[11:06am] jrseti_: So, we are going to hold off on announcing the official re-launch day, not rush it
[11:07am] jrseti_: I am assuming it will be 2 weeks from now, but that is just my guess.
[11:07am] jrseti_: It has to be coordinated with other people's availability. For instance, we want Seth to make presetations for the media
[11:07am] jrseti_: So, kinda out of my hands
[11:07am] sigblips: Two weeks from today is Election day.
[11:08am] jrseti_: Well, maybe not, then!
[11:08am] jrseti_: Maybe I'll know more next week
[11:09am] jrseti_: Jane is testing the system. It has been discovered that there are signal spikes at 51.2KHz intervals sometimes. They are trying to figure that out.
[11:10am] _Jens_: any plans to update setistars.org ? I suppose the average donator not visiting the setiquest-chats or forums would like to see some kind of progress.
[11:10am] sigblips: 25.6 kHz * 2 = 51.2 kHz. I blogged about those last year.
[11:10am] jrseti_: I'll email them about this. I'll probably write a blog and they will post it.Thanks for the reminder.
[11:11am] jrseti_: Sigblips: Can you give me the URL?
[11:11am] sigblips: I need to look at my notes but I think they are Walshing related. I'll send the link.
[11:13am] jrseti_: That is all I know.....
[11:13am] jrseti_: 2) GSoC Mentor Summit
[11:14am] jrseti_: Last weekend Gerry and I went to the Google Summer of Code Mentor Summit at Google.
[11:15am] jrseti_: It was a good experience, learned a lot from people that run and manage open source projects.
[11:15am] jrseti_: All food and drinks are free at Google, very nice
[11:16am] jrseti_: There were 350 attendees. Each was asked to bring chocolate from their part of the world. There was a LOT of chocolate
[11:17am] jrseti_: I talked a lot with people about their software development tools and processes. We really are behind as far as software project management is concerned (we really have none!)
[11:18am] sigblips: We have Redmine (issues.setiquest.org) but we don't use it! (:
[11:18am] jrseti_: One project was pretty cool. See http://astrometry.net/gallery.html
[11:19am] jrseti_: You can take a picture of anywhere in the sky, submit the image, and they will tell you the astronomical objects in the picture.
[11:19am] jrseti_: Amazing
[11:20am] jrseti_: Sigblips: Yes....We do not use it, but not much is going on to use it for.
[11:20am] jrseti_: One thing I learned is that other orgs have simple projects to involve more people that over time get interested in doing bigger things
[11:21am] vi joined the chat room.
[11:21am] _Jens_: is that possible in such a scientific matter as signal analysis as well?
[11:22am] jrseti_: _Jens_: good point. But there is actually a lot of other things we could have people do, like testing, documentation, that kind of thing.
[11:23am] jrseti_: For instance, I talked to a guy who works on the KDE project. The core programmers have to be user interface experts and really know their stuff, but they have a lot of opportunity for others to help on the less technical side.
[11:24am] sigblips: I found my 25.6 kHz blog references. Here they are:
[11:24am] sigblips: http://baudline.blogspot.com/2010/05/setiquest-exoplanet-060.html
[11:24am] sigblips: http://baudline.blogspot.com/2010/06/setiquest-pulsar-psr-b032954.html
[11:24am] sigblips: http://baudline.blogspot.com/2010/06/setiquest-crab-pulsar.html
[11:24am] jrseti_: But, for us to do that, we need to have a project management system in place, which we do not now.
[11:24am] sigblips: http://baudline.blogspot.com/2010/09/setiquest-deep-impact.html
[11:24am] sigblips: Search for 25600 or look for the polka dot elephant. That's it.
[11:25am] jrseti_: Sigblips: I'll pass it on to the tech gang. Thanks.
[11:26am] jrseti_: At the conference I got some ideas of open source project management tools to investigate. Ones that try to pull everything together into one manageable space.
[11:27am] khrm: Such as?
[11:28am] vi left the chat room. (Read error: Connection reset by peer)
[11:28am] jrseti_: KHRM: I don't have my notes with me! I don't remember. I'll maybe go over at next weeks meeting what I learn, and have others investigat also.
[11:28am] jrseti_: I am at a Starbucks in Sacramento right now
[11:29am] khrm: ok
[11:29am] vi joined the chat room.
[11:29am] jrseti_: Anyway - the GSoC mentor conference was educational.
[11:30am] jrseti_: This Thursday i am going to an HTML5 hackathon.
[11:30am] jrseti_: In Santa Clara, a city near the SETI offices
[11:32am] _Jens_: what's a hackathon?
[11:32am] jrseti_: On a related note: I need help getting the setiquest Explorer Android version built and working. I have requested an intern from USC (Univ of Southern California, in Los Angeles)
[11:32am] jrseti_: A hackathon is an event where a bunch of people get together and define and code up a project, all in a day
[11:33am] khrm: I tried learning ruby but I am quiet busy with college now.
[11:33am] jrseti_: http://en.wikipedia.org/wiki/Hackathon
[11:34am] jrseti_: I think the help I need is someone that has real experience with Flex, which setiquest Explorer is written in
[11:34am] jrseti_: We also need to migrate setiquest Explorer away from Flex, make it HTML5
[11:34am] sigblips: Is Flex the same as Flash?
[11:35am] jrseti_: Sort of - it is by Adobe, uses flash
[11:35am] jrseti_: Flex and Flash work closely together.
[11:36am] sigblips: Thought so. Thanks.
[11:36am] jrseti_: We need to not have any more projects that use non-open source as it's base. Unless we really need to, such as a really cool iPhone app
[11:37am] jrseti_: There is no reason that setiquest Explorer can not be HTML5
[11:37am] jrseti_: We had a couple of guys at NASA Goddard who wanted to do this for us, but I have not heard back from them
[11:37am] Turingi: would you think SETI would come to be known as a famous null result experiment?
[11:37am] jrseti_: Maybe! We don't know till we try
[11:38am] Turingi: a useful one nonetheless, to set a lower bound on the possibility of nearby civilizations
[11:38am] sigblips: Turing: is this a test? Hehehehe.
[11:38am] Turingi: s/lower/upper/
[11:39am] _Jens_: hum
[11:39am] jrseti_: 3) Ubuntu port
[11:40am] _Jens_: not that I'd expect a great response, but why isn't the need for people in the setiquest project communicated among TeamSETI members?
[11:40am] sigblips: Good question. Also why isn't setiQuest on the front page of http://seti.org ?
[11:41am] jrseti_: _Jens_ You are right - we should. But to be honest - we first need to define some simpler things for people to do.
[11:42am] jrseti_: Sigblips: It is, you have to scroll the small pictures. But, I'll request it to be one of the first ones
[11:43am] sigblips: IMO setiQuest should have precedence over TeamSETI.
[11:44am] _Jens_: not sure about that. But I suppose it's possible to promote them in an equally effective way
[11:47am] jrseti_: I'm back.... I just emailed the web admin about this, and CCd Jill
[11:48am] jrseti_: Silicon Wolves Computing Society has a fancy computer that runs Ubuntu, but has GPUs and FPGAs. They are going to try to get SonATA ported to their system
[11:48am] jrseti_: KHRM is going to help us get the existing SonATA running on Ubuntu. He already got SonATA working on Ubuntu for GSoC
[11:49am] khrm: Yes.
[11:49am] _Jens_: good work :)
[11:49am] jrseti_: Ys
[11:49am] jrseti_: Yes
[11:50am] jrseti_: They will get SonATA working as is on their Ubuntu system, then try to port the DX to their FPGAs. Then see if they can make it really fast
[11:50am] _Jens_: sorry, got to go. See you next week!
[11:50am] _Jens_ left the chat room.
[11:51am] jrseti_: Sigblips: How did your talk go at SETI several weeks ago?
[11:52am] sigblips: It went OK. Too much material, too little time. I tried to channel my inner Steve Jobs but it didn't help. (: I need to polish my presenting skills.
[11:53am] jrseti_: Was Jill there? Did she grill/drill you?
[11:54am] sigblips: Jill and Seth were there. They had many questions but they didn't grill me that much.
[11:54am] jrseti_: You got lucky!
[11:55am] jrseti_: I remember now - Gerrit was one of the open source project management tools used.
[11:56am] jrseti_: http://blog.assembla.com/assemblablog/tabid/12618/bid/40871/Introducing-...
[11:56am] jrseti_: http://code.google.com/p/gerrit/
[11:56am] jrseti_: It looked pretty comprehensive,
[11:56am] jrseti_: One guy showed me his management console
[11:57am] jrseti_: Anything else?
[11:57am] jrseti_: I talked too much
[11:57am] sigblips: 4) Empty pages on http://setiquest.org/
[11:59am] sigblips: Glossary, FAQ, and Legal. Also Site Map and Contact are out of date.
[11:59am] sigblips: What happened? And should these things be wiki pages?
[12:00pm] jrseti_: I'm sorry - which links?
[12:00pm] sigblips: http://setiquest.org/about/glossary
[12:00pm] sigblips: http://setiquest.org/about/faq
[12:00pm] sigblips: http://setiquest.org/legal
[12:00pm] khrm: http://setiquest.org/contact
[12:01pm] sigblips: http://setiquest.org/sitemap
[12:02pm] sigblips: http://setiquest.org/join-the-quest/events should also point to http://setiquest.org/wiki/index.php/setiquest_wiki:Current_events
[12:02pm] jrseti_: OK, I'll look into this. Strange
[12:03pm] jrseti_: t is after 12, I gotta go. Thanks all!
[12:03pm] khrm: Bye.
[12:03pm] sigblips: Bye yall. I'll post the transcript to the forum.
[12:04pm] jrseti_: thanks!
[12:04pm] jrseti_: bye

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
51.2 kHz spikes (plus rant!)

DFT 101 explains the spikes due to the Walsh system, as I outlined over a year ago in:

http://setiquest.org/forum/topic/walsh-system

I even provided an octave script to show how you could verify it.
I admit that I had to make an assumption (read: guess) what the actual cross-talk system did.
If there was more math and engineering diagrams available, people could actual contribute.

The rant: This is a perfect example why I hardly come to this site anymore.  A legitimate, interesting phenomenon was found and sound discussion starts, in this case the Walsh system / spikes.  I spent an afternoon writing a couple detailed, long posts, along with providing a simulation and detail mathematical description.  And then nothing. sigblips made a few comments, which I appreciated, since that meant my post was read by someone. BUT nothing was heard from the seti people, no comments, no corrections to the assumptions, no indication to whether the post was useful, nothing.  fast forward 1year+ and seti people are working on this spike problem.  Again, maybe my assumptions were wrong, and so my simulation is not applicable, but there was no feedback to say so, there are no block diagrams on the Walsh system in use so I have to go off the few bits of information to come down from high.

My day job involves very advanced research/work in signal detection, signal processing, and machine learning. I was under the impression that this type of  knowledge that I could contribute was what setiquest was hoping for.  I initially joined in the hopes to help out because this is an interesting application. I realize seti might not want to give out all it's secret sauce, but if they want people to help on the challenging problems more information and interaction is needed.  If you have a problem that you are spinning your wheels on, at least do a search on your web page to see if it's been discussed by us lowly posters.

Dave Robinson
Dave Robinson's picture
Offline
Joined: 2010-04-29
Posts: 196
With you all the way

Hi

I have to agree with you all the way. In fact I wrote an almost identical thread here

http://setiquest.org/forum/topic/i-begin-dispair

Followed by a detailed EMAIL I sent to JRSETI.

Whilst I have yet to abandon the Quest, I get really depressed when I find that the only contribution to the Forum is Sigblips log of the weekly meeting. I don't think we are alone in the way that we feel. Following Rob Ackerman's suggestion I went out and purchased a copy of Sky and Telescope, and compared how the Galaxyzoo project and the Setiquest project seem to work. Something is very - very wrong with Setiquest. I don't pretend to know what the solution is, but I really don't believe that Sigblips, Yourself and myself are the only signal processors (or in my case retired signal processor) are the only people on earth that are interested in independently analysing the data from the ATA. but I would have expected a really lively interchange of ideas, algorithms and hopefully discoveries.

I guess the Seti experts are just to busy to bother with a group of amateurs playing with their data.

Regards

Dave Robinson

matt79
Offline
Joined: 2011-06-24
Posts: 18
"Something is very - very

"Something is very - very wrong with Setiquest. I don't pretend to know what the solution is, but I really don't believe that Sigblips, Yourself and myself are the only signal processors (or in my case retired signal processor) are the only people on earth that are interested in independently analysing the data from the ATA...."

There's neutrino & gravitational wave data waiting to be analyzed...

Perhaps the general public would be interested in a new approach?

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
Hello matt79, I would very

Hello matt79,

I would very much like to analyze the data for the 1987 supernova event but unfortunately there is not enough high-quality data to analyze. It's just a handful of timestamps. I can't do anything with that.

matt79
Offline
Joined: 2011-06-24
Posts: 18
Any progress with the "raw

Any progress with the "raw data?"

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
what is your point?

especially tied with your follow-up response to sigblips ... this just sounds like a snippy person taking a jab because a couple people might have hurt your feelings?

No one is saying the approach to doing ET detection is necessarily wrong ... we're saying (well, I guess I can only speak for myself )  the level of interaction and discussion with the seti folks is wrong. 

There are many opportunities for improving the current algorithms. There's been of number of post's from Jill saying essentially that there is too much data to process in order to do real-time processing.  Just waiting for future computers to show up is for a lazy person.  It's probably not a stretch to say the algorithms are not 100% optimal yet.

matt79
Offline
Joined: 2011-06-24
Posts: 18
What do you think Max?

What do you think Max? Dave?

Worthwhile to check out the Rome, Maryland and Mont Blanc data?

matt79
Offline
Joined: 2011-06-24
Posts: 18
From another board: "So, we

From another board:

"So, we get some really anomalous neutrino emissions prior to a Supernovae event; the scientific community at large does not fully understand why this occurred because it does not fit our current model and understanding of the universe, so we generally shrug our shoulders and move on?

I don't get it"

matt79
Offline
Joined: 2011-06-24
Posts: 18
Jill?  Jon?  Anyone else from

Jill?  Jon?  Anyone else from SETI Inst?

Thoughts?

jctarter
Offline
Joined: 2010-08-27
Posts: 17
anomolous neutrinos

i'm not familiar with the data, and this is not my area of expertise.  if the researchers who reported the work are credible, they will have attempted to figure out what is going on and provide the data for others if they cannot.  if they aren't credible, the community of their colleagues in the field will make that abundantly clear.  there's always the potential for new physics and understanding with anomolous results, but only if they are statistically significant - the current case of the potentially superluminal neutrinos measured in Italy are a very good case in point.  nobody can bring themselves to believe it yet, and most folks think there will be an explanation, but everyone with relevant expertise is scrambling to try to understand what happened.  this is good.
jill

gerryharp
Offline
Joined: 2010-05-15
Posts: 365
paper summary related to superluminal neutrinos

http://physics.aps.org/synopsis-for/10.1103/PhysRevLett.107.181803
Interestingly, the paper does not discount the possibility that neutrinos can travel with superluminal velocity. But if they did...

matt79
Offline
Joined: 2011-06-24
Posts: 18
Thanks to all who

Thanks to all who responded.

Has anyone from the SETI community been in touch with the Mont Blanc scientists? Their infn addresses are active, and not hard to find.

Also, has anyone been spending any time on the Fermilab letter?  I'd love to see some fresh ideas..

Regards,

Matt

 

matt79
Offline
Joined: 2011-06-24
Posts: 18
No one has any thoughts on

No one has any thoughts on the Fermilab letter?  It's been well over a year since I mentioned it in the other thread.

It was published by the US Federal government (it's arguable whether they were aware of the content, of course).  It might be worthwhile to spend some time on it?
 

matt79
Offline
Joined: 2011-06-24
Posts: 18
Hi, In case anyone out there

Hi,

In case anyone out there cares, I'm moving on to a new hobby.  I've had enough "code cracking" for one life time.

I hope someone decides to do something with the 223 data at some point?  It's kind of long overdue...

Regards,

Matt

matt79
Offline
Joined: 2011-06-24
Posts: 18
Regarding statistical

Regarding statistical significance, here's two of the people who were involved in analyzing the Mont Blanc coincidences.

Prof. G.Pizzella guido.pizzella at lnf.infn.it

Dr. S.Vernetto vernetto at to.infn.it

I think they would have the most up to date information..

It would of course also be worthwhile to contact the IMB, KII and Baksan teams.

matt79
Offline
Joined: 2011-06-24
Posts: 18
I'd look into it myself, but

I'd look into it myself, but I'm:

-permantently disabled due to mental illness.

-innumerate (for all intents and purposes)

-in possession of a grade 10 education.

I really don't think I'm the best person for the job.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
I'm not interested

that's what I think.

I'm interested in improving the current algorithms in use.  I might be naive to think they need in improvement, but probably not. My expertise is in wireless communications algorithm design and implementation.  Other than a basic definition, I don't know the first thing about neutrinos and I'm fine with that.

a_feder
Offline
Joined: 2011-10-09
Posts: 34
There isn't really any will

There isn't really any will in SI to do what you describe. Probably the only thing that will really come out of this project is the very narrowly defined Galaxy Zoo app that is under development.

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
Hello Max and Dave,I agree

Hello Max and Dave,

I agree that setiQuest has been frustrating at times. I've expended a great deal of effort analyzing and measuring the ATA's distortion characteristics. You would think that the staff at the SETI Institute would be extremely interested in this but they're not. My first several blog posts received a great deal of attention but then interest declined as I focused more on flaws in the data than on signals. I felt bad about this so I stopped writing my baudline-setiQuest blog posts. I don't think my focusing on data flaws is what the SETI Institute bargained for when they made the setiQuest data public. I really can't blame their reaction either, honestly, who would like someone constantly pointing out their flaws?

That though is not what I think the problem with setiQuest is. IMO the problem with setiQuest has to do with culture and structure. Here are my observations:

The culture of the SETI Institute is more academic than that of a traditional business or an open community. They seem to thrive on an academic grad-student sort of approach as seen with the NSF REU and GSoC programs. With these programs someone else funds the student, $5K for the REU and GSoC examples, and then the student does exactly what they are told to do. This is great for getting free work done but it is horrible for community building. Here is why:

1) A big problem is that these programs are huge time sinks and I estimate that hundreds of hours of effort were expended on this year's REU and GSoC programs. Think of this in terms of inelastic collisions and conservation of energy. There is only so much time to go around and those hundreds of hours of internal effort could of been focused on the setiQuest community. Redirecting that energy elsewhere allowed the level of community forum traffic to wither.

2) Another problem with these programs is that the students are transitory and they don't attempt to be part of the setiQuest community. This is understandable since it would be a distraction and interfere with their paid work. [Note that leash was an exception but that was part of her REU job description.]

3) Yet another problem with these programs is that they tend to alienate the community. Two GSoC students were selected which is great for them but not for the 57 others who were not selected. Amazingly a large number of the non-selected students wanted to help and be free SETI Institute interns. It started strong but then unfortunately the free intern effort fizzled out. I guess the pain of working side-by-side with two of their peers who were getting paid was too great. I can't blame them.

4) An even bigger problem is that GSoC put a price tag on the cost of software contributions. This will have a negative impact on future contributions.

None of these are good ways to build a strong setiQuest community.

The structure of setiQuest has never been formally defined. After being a member for a year and a half I'm still not sure what it is. The structure of setiQuest appears to be more of a vertical top-down cathedral-like hierarchy than a horizontal open bazaar-like hierarchy. Yes, the source code is open but that does not make an open community. Some examples of top-down are the Cloudant, setiQuest Explorer, REU/GSoC, and recent GalaxyZoo initiatives. They appear out of nowhere and are handed down to the community. I understand that these people want to help the SETI Institute but they want nothing to do with the setiQuest community. What affect does this have on the setiQuest community? It says that non-community driven efforts are more important. While this may be true, it is not a good way to build a strong setiQuest community.

Can these culture and structure issues be resolved? It is possible but it won't be easy.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
max is my dog :)

I agree with what you said.
It is unfortunate that they do not want to take advantage of people willing to give them their time for FREE to help the cause.

Jens
Jens's picture
Offline
Joined: 2010-04-20
Posts: 19
Hello Dave and all the

Hello Dave and all the others,
unfortunately I'm not experienced in the field of signal processing - I barely know physics and after one semester of studying chemistry I decided to study law. So I'm merely a citizen scientist. But my problem is similar to yours - just as you think there are many signal processing geeks waiting to help out here, I believe there are many more laymen that might be interested in contributing to SETIQuest. It would be great to see this community strive, but before that happens, there has to be something here that draws people's attention.

If you enter the site and finally have found the SETIQuest Explorer (which for the absolute layman is the only real thing to do here besides entering the forum), you still read "The software was released in beta in Mid March. New users are being added to the beta program regularly. You can sign up here to be included in the beta", although the program has been active for quite some time now. And even I still haven't fully understood what the Status of the Explorer App is, what happens to data which have been marked by me and so on. So why would I bother play around too much with it let alone talk in the forums? Something has to happen here, which is visible to the people no matter if they're experts or laymen.

I used to participate in Stardust@home (http://stardustathome.ssl.berkeley.edu) which has really been fun (especially due to the statistics and rank systems, which made you feel some progress with what you're doing) and probably still is. I hope SETIQuest will provide a similarly pleasant environment in the future.

Best regards
Jens

gerryharp
Offline
Joined: 2010-05-15
Posts: 365
52 kHz is not a Walsh problem

Hi MPS
BTW, is the "pooper scooper" part an allusion to Piers Anthony's "Macroscope?" I really liked that novel in high school. Anyone know of it?
This communication comes kind of late so I apologise. There are a couple of points you may not know of (yet):
Jane can reproduce the problems with her data even when the telescope is turned OFF. Its been tracked to a network issue and dropped packets.
Although you guys were not present, at the moment Jane's problems (i.e. weird signals in SonATA data) appeared, we (Jane, Ken, Billy, myself) discussed Walsh misalignment/crosstalk as a potential culprit.
My point is that we're not ignoring the ideas that come up in forum postings, and we do understand the their content. A year ago last summer, Billy Barott spent the entire summer with an REU student and worked out all the ways Walsh functions can introduce artifacts into our signals and measured the relative contributions of the different walsh frequencies. They tracked down the phase shifts between the analog Walsh generator and the digital Walsh removal as well as the analog transition time which is much longer than 1 sample. Then, small tweaks were made to improve the system to be the best it can be with the current Walsh generators. Just this evening, I was speaking with Jack Welch (famous astronomer somehow associated with Jill) and he mentioned that if we ever had some money we should replace the Walsh generators with a more accurate phase switching system.
I'm sorry that this kind of info was not communicated sooner to you and the community. The forum content does make its way back to SETI staff. As for the other direction, the critique that  SI is not communicating enough with the community is undeniably true. Our staff here has been cut in half in the past year. Both Rob and Avinash who contributed to conversations much more than I are now gone. Not that we were so good at communicating before... but you see what I mean.
I've been, personally, quite interested in the OS movement and in setiQuest. I've spent what I consider to be substantial amounts of my time supporting the community through setiQuest data acquisition and publishing. It is true that I have not been a great communicator in the community, but without Rob and I, there would never have been any setiQuest data and this was about as much time as I could afford to commit. Unfortunately, even this level of committment is not sustainable in the current funding climate.
I would ask for forgiveness with the very slow pace at which I am learning about OS and how it may help me do my day job. Personally, I believe that the SETI observing team is on an upward trajectory. If the community doesn't die in the mean time, it has a future.

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
Hello Gerry,I agree that for

Hello Gerry,

I agree that for setiQuest to suceed that communication and time investment levels need to be increased. I didn't put it in those terms but that was basically what I was trying to say in my rambling culture and structure post.

Now this is really interesting:

"Jane can reproduce the problems with her data even when the telescope is turned OFF. Its been tracked to a network issue and dropped packets."

From http://baudline.blogspot.com/2010/06/setiquest-pulsar-psr-b032954.html :

"This number is interesting because 4315733.185 / 8738133 * 4096 = 2023.0000076 which is very close to the whole number 2023. Note that 2^12 = 4096 and it will also be derived below in the Quadrature Magnitude section in relation to 25600 Hz."

and

"... the frequency was carefully measured to be 25600.0 Hz with an error of less than ±0.1 Hz. The ATA's ADC sample rate 100 * 2^20 / 25600 Hz = 4096."

Do the number of data samples in the UDP packets happen to be 4096?

gerryharp
Offline
Joined: 2010-05-15
Posts: 365
yup

Hi SigBlips
Do the number of data samples in the UDP packets happen to be 4096?
Close. I believe the number of bytes in the UDP packets is 4096. The number of samples is 2048. That is coming out of the beamformer (8bit x 8bit).The data payload of the UPD packet (4096 bytes) is prepended with a 64 byte header. But the header does not change the packet rate, of course.
The the data go to the channelizers. This is a big Fourier Transform filter bank, which breaks 100 MHz into 256 different narrow band streams. Because of bit growth in the beamfomrer, we use a 16bitx16bit format for the comlex-valued signals. The payload of the packets out of the channelizer have the same size, so instead there are 1024 samples per packet.
 
Gerry
 

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
I just found this SonATA

I just found this SonATA document

https://github.com/setiQuest/SonATA/blob/master/doc/data-rates

that says:

"104.8576 MHz / 2048 samples/packets = 51,200 packets/sec"

The packet rate being double the Walshing rate will make it difficult to determine the source of the distortion by measurement alone. It is unfortunate that the rates are related by a multiple of two.

I'm confused. So the output from the beamformer is 16-bit quadrature samples at a 104.8576 Msample/sec rate? If so then this doesn't match the documentation.

janebird
Offline
Joined: 2010-04-22
Posts: 20
Beamfomer output

The output from the beamformer is indeed 8 bit x 8 bit .
I believe the statement in Gerry's post "Because of bit growth in the beamformer, we use 16bitx16bit format for the complex valued signals." should read
"Because of bit growth in the Channelizer, we use 16bit x 16bit format for the complex-valued data".

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
So the output of the

So the output of the beamformer is 8-bit quadrature samples at a 104.8576 Msample/sec rate?

And the output of the channelizers are 16-bit quadrature samples at a 819.2 or 409.6 Ksample/sec rate?

To combat a similar bit growth problem while decimating in baudline I use a feature called decimation gain which works in a similar way. http://baudline.com/manual/input.html#decimation_gain

a_feder
Offline
Joined: 2011-10-09
Posts: 34
Gerry, it is not just the

Gerry, it is not just the time you have, it is how you spend it. You are essentially building a bridge to nowhere here, and when anyone points that out, they are told: "If you don't agree with what we do, why are you here?" In which dimension is this sound project management?

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
That is a bit harsh Anders.

That is a bit harsh Anders. The SETI Institute is attempting to do the best they can with the limited resources that they have.

While I agree with your point about time expenditure, the rest of your comment is not constructive. Obnoxious comments like that are not how positive change is enacted. Glad to see that you are back with a vengeance.

a_feder
Offline
Joined: 2011-10-09
Posts: 34
I don't think that is harsh -

I don't think that is harsh - there can be many motivations for building a bridge to nowhere, i.e. fog, having turned the map upside down, etc. Stuff happens. But at the end of the day, it is still a bridge to nowhere if you don't alter the course, no matter how much time and effort you put into it.

gerryharp
Offline
Joined: 2010-05-15
Posts: 365
ok

Fair enough. The more effort you put into a project, the more likely it is to succeed.
I guess it is rather human to try reach beyond your own grasp.
Anway, I'm here today.

a_feder
Offline
Joined: 2011-10-09
Posts: 34
look

Here is an example of a successful open source interaction:

1. I report a bug.
2. The maintainer responds with detailed information on how to fix it.
3. I write a patch and submit it back to the maintainer.
4. The maintainer fixes up the loose ends and integrates the patch into the main source tree.
5. A new, improved version of the software is released for use by millions of users.

No abuse of the volunteer, no money under the table, no glossy websites. How did he pull it off? Here's the secret: I use the software every day. Literally every time I log into my netbook, I am presented with a naggy window prompting to me enter the PIN code for my GSM modem. I wanted this to go away. If I had been using some other operating system, the story would have ended there, but because the component was genuinely open source, I was able to take matters into my own hands and just fix it.

Hence the key to the success in this project is: how will you make SonATA into software that people use every day?

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
There isn't anything

There isn't anything preventing people from doing your first 3 steps with the SonATA codebase. Few have though.

SonATA is a very specialized and single-purpose collection of software. Turning it into something that people use on an everyday basis is a difficult problem. There are several ways to accomplish this. One is to transform SonATA into a generic tool. Unfortunately this would require enormous amounts of development resources that the SETI Institute doesn't have and it would distract SonATA from its primary mission which is to detect SETI.

SonATA will most likely find its greatest re-use with other competing SETI projects. SonATA is a great fit for University or Observatory SETI projects that operate on shoestring budgets. Then there is China. They are building a radio telescope that is several times larger than Arecibo and they have made numerous rumblings about an interest in SETI. The SonATA base makes a good starting point but a Chinese government funded SETI effort would eclipse everything else. The SKA may be a factor too if it ever gets built.

a_feder
Offline
Joined: 2011-10-09
Posts: 34
What's preventing people from

What's preventing people from doing the first 3 steps is lack of incentive.  Other projects  - or at least the successful ones - don't just assume that people will bring their own incentives. They make use cases. Maybe this is something setiQuest should focus on too? (You've already described one.)

Making software usable isn't necessarily easy, but saying that setiQuest can't do it because it is somehow qualitatively different from projects that succeed is in my view tantamount to giving up.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
unfortunately the name says it all

Max is my dog and I clean up his "business" in the yard.

gerryharp
Offline
Joined: 2010-05-15
Posts: 365
there you go

I guess there is a lot of implicit ambiguity in internet names.