Discussion Forums

open SonATA source code released

30 replies [Last post]
sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733

The open SonATA source code was released today (March 1st 2011) and you can get it here https://github.com/setiQuest/SonATA

For more information see: http://setiquest.org/content/details-open-source-project

stephf0716
Offline
Joined: 2010-06-10
Posts: 11
Who has successfully built it

Who has successfully built it so far? Please post your thoughts and experiences.

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
I'm building it on my free

I'm building it on my free Amazon AWS micro instance which is using the stock AMI with the gcc 4.1.2 version. I can't run SonATA on the micro instance but it is building fine. The steps are tricky and I will write up a short tutorial that documents the process. Doing this with the stock AMI on an AWS micro instance is challenging to say the least.

For an easier route I would recommend building SonATA with an openSUSE 11.3 install which is the SETI Institute's development platform.

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
Others might find this

Others might find this helpful so I thought I'd share.  Someone sent me a private message asking if the Amazon AWS micro instance is a virtual machine? Yes, the micro instance is a virtual machine that you connect to over the Internet.

I used the default Linux AMI that works with the free micro instance tier. The default AMI is a fork of RHEL that has been stripped of all excess packages. It is very bare bones and it requires a lot more work than openSuSE 11.3 does for building SonATA.  The micro instance is too small and underpowered to run SonATA though.

Some people may ask why am I doing something crazy like trying to build SonATA on a AWS micro instance?  Two reasons:

1) It is more of a challenge this way. (:
2) I want to start with a blank slate so I can more accurately document the build process.

khrm
Offline
Joined: 2011-03-20
Posts: 39
How to test our build if we

How to test our build if we have less memory? I have compiled and build the sonata successfully in a different distro but trouble is that I have just 2 gb ram. But I do have a two core ssse3 processor.

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
I talked to two of the SonATA

I talked to two of the SonATA developers today after Avinash's colloquium talk.  We just happened to discuss this exact topic since I have a similar hardware resource limitation.  They suggested that I use two or more networked machines (which have lesser amounts of RAM) to split up the seeker and the channelizers.  What a clever solution!

Here are some relevant documentation links:

http://setiquest.org/wiki/index.php/Seeker
http://setiquest.org/wiki/index.php/Detector_Chain

jrseti
jrseti's picture
Offline
Joined: 2010-07-22
Posts: 250
The suggestion SigBlips talks

The suggestion SigBlips talks about may be a good solution. The issue is with SonATA in that it it designed to have multiple computers running separate software components. It was designed to be highly scalable. It was not designed to run all as one peaceful set of programs on one computer.

If you have 3 computers available you can run a channelizer on one, the dx on another, and the seeker/packetread on the third. 

If anyone can get this running on their systems and document how you did it, it would be a great addition to our documentation.

-jrseti

janebird
Offline
Joined: 2010-04-22
Posts: 20
Running SonATA on multiple hosts

In order to run on multiple hosts, you need to edit the ~/SonATA/scripts/spacecraft-demo-xpol-env-vars.tcsh file which defines the physical configuration. Change CHANHOST1X and DXHOST1 to the names of the hosts where the processes will run. These must run on a host that has sse3 processors. The DXHOST will consume the most resources for the vger demo.
If your architechure is compatible, you may install SonATA on one host and copy ~/sonata_install to the other hosts.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
My attempts at building

I have a Mac, but I'm using VirtualBox to load a virtual Linux machine.

Today, I attempted (unsuccessfully) to build SonATA on Fedora 14.  I choose Fedora to start, because I've used that flavor of Linux the most.
My plan was to follow the instructions http://setiquest.org/content/sonata-build as close as possible with the obvious suitable changes ( using yum etc... ).

First deviation: I didn't do the ssh key and the openSUSE multimedia libs download.
Built tclreadline with no problems.
Built CppUnit with no problems.

My build fails on the sse-pkg.  When I do the make, it fails with cannot find ace/OS.h
After some investigating, I realized my virtual machine did not have the ACE framework and it appears deprecated in Fedora 14.

I noticed that the build instructions say to set sse-pkg/configure.in:
ACE_ROOT=$HOME/SonATA/packages/ACE_wrappers
In my download from git, there is not a 'packages' directory in SonATA.  Is there an error in the instructions? a missing step? or did I do something incorrect?

I spent some time trying to compile the ACE framework from source and load that. Unfortunately, I am brushing up against my Linux knowledge wall for this task.

Next, I am going to try two other virtual machines: openSUSE 11.4 and Fedora 12. I am hoping that the openSUSE will be easy.  Fedora 12, from what I can tell, is the last Fedora with ACE.

Since the build instructions make a statement about ACE_ROOT, I'm wondering if there is something missing from the SonATA download?

One a somewhat unrelated note, the build instructions tell us to give our username sudo priviledges without password.  The instructions said that the FAQ explains why.  I was not able to find the explanation on the FAQ http://setiquest.org/about/faq.

khrm
Offline
Joined: 2011-03-20
Posts: 39
I think you forgot this

I think you forgot this instruction?

cd ~/SonATA/scripts
./get_packages

I will hazard a guess that when we give vger-demo-xpol.tcl in seeker windows.
It uses sudo packagesend in back ground so how will you give password?

SonATA can be build on fedora 14 I have tested it after making some changes in makefile in sse-pkg. There might be some problem during runtime which are easily overcome.  Do reply if you are also able to build it.

Also ssh step is must. As you must be able to login to your ownself using ssh key not password.  That step is must otherwise you get error. Your dx and channelizer will not be able to run.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
SonATA/scripts/get_packages

Ah I did forget that step. 
I actually started with a 32-bit install of Fedora. I did remember to do the get_packages script that time, but forgot when I redid the install with the 64-bit Fedora.

I don't have much time to work on this install this weekend.  I'll definitely report on my progress.

By the way, you mentioned that you had to edit the sse-pkg makefile on your Fedora install. What were they? Do you think I'll need to make those changes as well? or will they be different for me?

khrm
Offline
Joined: 2011-03-20
Posts: 39
I didn't have fresh installed

I didn't have fresh installed but I think you would also need to do them as libmysqlclient_r will not be link but first try to build without any change. Line numbers have changed since when I build it I will post the updated change when I will get time.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
fedora-14 install progress

I was able to compile and build everything so far.  Testing the build does not work for me yet.

I repeated the build instructions with a few extra things.
1. I had to do: yum install java-1.6.0-openjdk-devel ( for the javac )
2. When I did a yum install mysql, it installed the shared objects into /usr/lib64/mysql.  The make files however were looking for the objects in /usr/local/mysql.  I made the appropriate symbolic links in order to make the compiler happy.

I get a lot of error messages when I did runsse.sh.  I think it deals with the ssh.  In my second pass through the build process, I did the sshd steps ( in my original post i did not ).  My virtual box has crashed and burned so I don't have the exact message.  Basically, there was a bunch of chan1 failures and something about the publickey.  That made me think it deals with the ssh.  I'll get the exact error message the next time I have a chance. But I didn't know if there is any way to figure out if I did the ssh/sshd correctly?

khrm
Offline
Joined: 2011-03-20
Posts: 39
Yes same was the case with me

Yes same was the case with me regarding libmysqlclient_r libraries. It was searching in /usr/local/mysql but it is in /usr/lib64/mysql. Thats what I change in the makefile.
To check ssh :
What is your hostname? Type: hostname

Then type: ssh <output of hostname command>

If you are able to login into your localhost it means you have done ssh step right.

BTW I thought openjdk is not recommeded but sunjdk.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
making progress

I had to do the following to get ssh issue resolved:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Now my error occurs when I do:
seeker>>  source vger-demo-xpol.tcl

I get something like: sudo: packetsend: unknown command

I don't know if packetsend is a command that is apart of SonATA or if it's a linux command. I tried 'yum install packetsend' and fedora couldn't find a package.

sigblips
sigblips's picture
Offline
Joined: 2010-04-20
Posts: 733
packetsend is a SonATA

packetsend is a SonATA utility program.

http://setiquest.org/wiki/index.php/Packet_Programs
https://github.com/setiQuest/SonATA/tree/master/sig-pkg/utilities/packet...

My guess is that either you didn't build packetsend or it is not in your path.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
should have done this before the previous post

I did a 'which packetsend' that results ~/sonata_install/bin/packetsend.
It appears to be installed and in my $PATH.

any other ideas?

jrseti
jrseti's picture
Offline
Joined: 2010-07-22
Posts: 250
When you started up seeker

When you started up seeker did you:

cd ~/sonata_install/scripts

tcsh

source spacecraft-demo-xpol-env-vars.tcsh

runsse.sh

Did you do the tcsh?

-jrseti

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
yep

I think janebird has the right thinking

janebird
Offline
Joined: 2010-04-22
Posts: 20
accessing packetsend

The problems you are having may be related to the way Fedora handles sudo.
Does it assume the root environment which doesn't have ~/sonata_install/bin in the path
rather than your own environment?
This is not a problem under Open Suse.
Try putting the full path name for packetsend in the runpacketsend-dx-vger-xpol.tcsh script.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
I will give this a try

Now that you mention it, sudo does use the root environment -- not mine.  I ran into a similar issue at work.  I will give this a try and report back.

khrm
Offline
Joined: 2011-03-20
Posts: 39
Oh Sorry I forgot to tell you

Oh Sorry I forgot to tell you about one more thing that you will have to change your /etc/sudoers. By default in fedora, it doesnot give your sudo home environment or environment you have set in bash.rc file.

Add this line above where I have :Defaults always_set_home
Here's my file:
<code>
# sudoers file.
#
# This file MUST be edited with the 'visudo' command as root.
# Failure to use 'visudo' may result in syntax or file permission errors
# that prevent sudo from running.
#
# See the sudoers man page for the details on how to write a sudoers file.
#

# Host alias specification

# User alias specification

# Cmnd alias specification

# Defaults specification

# Prevent environment variables from influencing programs in an
# unexpected or harmful way (CVE-2005-2959, CVE-2005-4158, CVE-2006-0151)
Defaults always_set_home
Defaults env_reset
# Change env_reset to !env_reset in previous line to keep all environment variables
# Following list will no longer be necessary after this change

#Defaults env_keep = "LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT #LC_MESSAGES LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_TIME LC_ALL #LANGUAGE LINGUAS XDG_SESSION_COOKIE"
# Comment out the preceding line and uncomment the following one if you need
# to use special input methods. This may allow users to compromise  the root
# account if they are allowed to run commands without authentication.
#Defaults env_keep = "LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT #LC_MESSAGES LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_TIME LC_ALL #LANGUAGE LINGUAS XDG_SESSION_COOKIE XMODIFIERS GTK_IM_MODULE QT_IM_MODULE #QT_IM_SWITCHER"

# In the default (unconfigured) configuration, sudo asks for the root password.
# This allows use of an ordinary user account for administration of a freshly
# installed system. When configuring sudo, delete the two
# following lines:
Defaults targetpw   # ask for the password of the target user i.e. root
ALL    ALL=(ALL) ALL   # WARNING! Only use this together with 'Defaults targetpw'!

# Runas alias specification

# User privilege specification
root    ALL=(ALL) ALL
khrm    ALL=(ALL) NOPASSWD: ALL

# Uncomment to allow people in group wheel to run all commands
# %wheel    ALL=(ALL) ALL

# Same thing without a password
# %wheel    ALL=(ALL) NOPASSWD: ALL

# Samples
# %users  ALL=/sbin/mount /cdrom,/sbin/umount /cdrom
# %users  localhost=/sbin/shutdown -h now
</code>

khrm
Offline
Joined: 2011-03-20
Posts: 39
This doesnot work on ubuntu.

This doesnot work on ubuntu. Other trick involve putting this line in bash.rc file:

alias sudo='sudo env PATH=$PATH'

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
pachetsend works now ... well executes

I made the following changes to my sudoers

Added:
Defaults always_set_home
Defaults !env_reset

commented out:
#Defaults env_reset
#Defaults secure_path ....

packetsend now executes. the issue was like janebird suspected in that sudo defaults to the root environment.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
current status -- assertation failure

I get the following assertation error:
packetsend: ../include/PsStruct.h:73 sonata-packetsend::ReadBuffer::ReadBuffer(int32_t,int32_t): Assertion 'buf' failed

I haven't looked at the code yet.  Probably running out of memory?  My Mac only has 2Gb of RAM and I only give my virtual machine 1 Gb of RAM.  That could be part of the issue.

My virtual machine is faking 64-bit, so seems like it could fake have 4Gb of RAM.  I'll look into that next.

janebird
Offline
Joined: 2010-04-22
Posts: 20
Hardware requirements are real

The assertion error means you are out of memory.
I have 4GB on my desktop. But I have to exit from the browser and mail before I can run the demo.
There needs to be 3 GB of memory free before you start SonATA.

maxs-pooper-scooper
Offline
Joined: 2010-07-28
Posts: 58
that's what I figured

I'll poke around the virtual machine settings.  Otherwise it might be time to upgrade my RAM.

I believe I have done the installation correctly ... just not able to run the demo due to my hardware.
I am actually more interested in augmenting the code base.  I should be able to make my own test programs for the different libraries. The 4GB RAM requirement is for running the whole system; GUI and all, correct?

janebird
Offline
Joined: 2010-04-22
Posts: 20
Hardware requirements

4GB RAM is the requirement for running the whole system. True.

khrm
Offline
Joined: 2011-03-20
Posts: 39
Yes you can run

Yes you can run separately.e.g. to run packetsend just type:

sudo packetsend -c -f ${HOME}/sonata_install/data/vger-xpol-2010-07-14-406.pktdata -J 229.1.1.1 -j 51100 -n 1 -i 1050 -b 1

Note it will consume lot of ram and if you want to know whether it is working or not just type iftop ( you might have to installed that first it might be in kbs).

BTW no hypervisor as far as I know can fake ram or maybe you imply something else.

I have been able to run SonATA on 2gb ram and 4 gb swap ( after closing nearly all useless background applications). Also I was not using any fancy stuff like compiz or kde at that time. Even then it runs  quite slowly. Used my 38-45 % swap and whole of the ram.

jrseti
jrseti's picture
Offline
Joined: 2010-07-22
Posts: 250
Last week I discovered that

Last week I discovered that we could not get ssh auto login to work on one of our computers. It was a new install of OpenSUSE 11.3. Anything I tried, it would not work, always asking for a password. After some debugging by starting the ssh daemon in debug mode I found the problem and added the solution to the SonATA build istructions page:

 

 

Important Note! If you can not configure the ssh keys properly to allow login without a password, there may be a problem with your sshd configuration. Please view the file /etc/ssh/sshd_config. Find the line defining the variable AuthorizedKeysFile. We have found instances where this value is defined incorrectly, even in fresh OpenSUSE installs.

To correct the problem, make sure the line defining the value of AuthorizedKeysFile is defined as follows:

 

AuthorizedKeysFile %h/.ssh/authorized_keys

 

We have encountered sshd_config files that do not contain the %h in the AuthorizedKeysFile definition, which causes automatic ssh login to fail.

That fixed the problem. It was a bug in the installed sshd_config file!

-jrseti

jrseti
jrseti's picture
Offline
Joined: 2010-07-22
Posts: 250
Last week I discovered that

Last week I discovered that we could not get ssh auto login to work on one of our computers. It was a new install of OpenSUSE 11.3. Anything I tried, it would not work, always asking for a password. After some debugging by starting the ssh daemon in debug mode I found the problem and added the solution to the SonATA build istructions page:

 

 

Important Note! If you can not configure the ssh keys properly to allow login without a password, there may be a problem with your sshd configuration. Please view the file /etc/ssh/sshd_config. Find the line defining the variable AuthorizedKeysFile. We have found instances where this value is defined incorrectly, even in fresh OpenSUSE installs.

To correct the problem, make sure the line defining the value of AuthorizedKeysFile is defined as follows:

 

AuthorizedKeysFile %h/.ssh/authorized_keys

 

We have encountered sshd_config files that do not contain the %h in the AuthorizedKeysFile definition, which causes automatic ssh login to fail.

That fixed the problem. It was a bug in the installed sshd_config file!

-jrseti