Data conduit protocol

From setiquest wiki

Jump to: navigation, search

The page specifies the protocol of the data flow from the ATA to the cloud.

The data format corresponds to the exact header of the packets sent out by the channelizer, with a little more information appended onto the end.

This will be useful for developers ingesting this data into SonATA software running on their systems. The data could be fed into running instances of a DX, for example, just by removing the extra fields appended to the header.

Metadata for each packet is included in the header. The overhead is very minimal, and with all the information in the header, all data received by a developer is immediately identifiable. This makes using the data easy for developers.

Feedback to ATA is outside the scope of this protocol.

Dataflow.png

ATA to Amazon Data Format

This is a first attempt to define the structure of the data packets sent from a computer at the ATA to the AWS server, and ultimately to any listeners to the real-time data stream.

This specification does not attempt to define the client program that sends this data, nor does it attempt to define the server program that receives this data.

Assumptions

Header and Data Specification

This header corresponds exactly to the header in the channelizer output, with some extra items added after the item "len". The items before "targetId" are defined in ATADataPacketHeader.h.

Name Size/Type Explanation
group 1 byte
Unsigned byte
The group of applications this type of packet belongs to. This is set to 0 at the ATA, the SETI data conduit will have a value of 1.
version 1 byte
Unsigned byte
The version of this header. Not defined yet. Currently in SonATA the version is 0x01.
bitsPerSample 1 byte
Unsigned byte
The number of bits per data sample.
binaryPoint 1 byte
Unsigned byte
The implied fixed point position in the data. Normally 0.
order 4 byte
Unsigned integer
An identifier showing the correct endian order. This should be 0xaabbccdd.
type 1 byte
Unsigned
The type of the data. This is a bitfield:
  • INTEGER = 0x00
  • FLOAT = 0x01
  • UNSIGNED = 0x00
  • SIGNED = 0x02
  • REAL = 0x00
  • COMPLEX = 0x04
  • DEFAULT_TYPE = INTEGER | SIGNED | COMPLEX
stream 1 byte
Unsigned
If the data is single pol and the receiving software is expecting dual pol, the sending software can clone the first stream and set this field to a value of 1 to signify this is a clone.
polCode 1 byte
signed
  • NONE = -1
  • LCIRC = 0
  • RCIRC = 1
  • XLINEAR = 2
  • YLINEAR = 3
  • BOTH X and Y = 4
hdrLen 1 byte
Unsigned
The number of bytes in the header.
src 4 bytes
Unsigned int
The source of the data:
  • BEAM_104MHZ = 0, from the beamformer.
  • CHAN_400KHZ = 1, from the channelizer.
chan 4 bytes
Unsigned int
The number of the channel this data came from in the channelizer.
seq 4 bytes
Unsigned int
The sequence number of this packet
freq 8 bytes
Double precision
The center frequency of the data. In MHz.
sampleRate 8 bytes
Double precision float
(beamformer_bandwidth / fraction) / num_channelizer_channels
usableFraction 4 bytes
single precision float
1.0 - beamformer_oversampling_pct
reserved 4 bytes
single precision float
Reserved for future use
abdTime 8 bytes
Unsigned integer
The time to the microsecond. UTC
flags 4 bytes
Unsigned int
Value of 1 specifies the data is valid.
len 4 bytes
Unsigned int
The number of bytes in the data.
targetId 4 bytes
unsigned integer
The ID of the target. Information on this target should be easily obtainable from a database we provide an interface to in the AWS server. if the value is 65535 this signifies the source is just an RA/DEC, see the ra/dec values in the header.
bandwidth 4 bytes
Single precision float
The bandwidth of the data in MHz.
az 4 bytes
Single precision float
The AZ in degrees.
el 4 bytes
Single precision float
The EL in degrees.
ra 4 bytes
Single precision float
The RA in degrees.
dec 4 bytes
Single precision float
The DEC in degrees.
data - unsigned char bytes.
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox