Subversion Repositories svnkaklik

For multiple antenna radioastronomy projects, system phase stability is a mandatory condition. It allows a precise, high resolution imaging of objects, increases signal to noise ratios in several observation methods and allows the use of advanced algorithms for signal processing.

The system phase stability is a mandatory condition for multi-antennas radioastronomy projects. It allows a precise, high resolution imaging of objects, increases signal to noise ratios in several observation methods and enables using of advanced algorithms for signal processing.

High phase stability in our scalable design is achieved through centralized frequency generation  and distribution with multi-output LVPECL \glos{LVPECL}{Low Voltage Emitter-coupled logic} hubs (CLKHUB02A), that have equiphased outputs for multiple devices. LVPECL logic is used on every system critical clock signal distribution hub. LVPECL logic has an advantage over LVDS \glos{LVDS}{Low-voltage differential signaling} in signal integrity robustness. LVPECL uses higher logical levels and higher signalling currents. Power consumption of LVPECL logic is nearly constant over the operating frequency range due to the use of bipolar transistors. This minimizes voltage glitches which are typical for CMOS \glos{CMOS}{Complementary metal–oxide–semiconductor } logic. One drawback of its parameters is a high power consumption of LVPECL logic which easily reaches tens of milliamperes per device.

The high phase stability is achieved in our scalable design through centralized frequency generation and distribution with multi-output Low Voltage Emitter-coupled logic (LVPECL\glos{LVPECL}{Low Voltage Emitter-coupled logic}) hubs (CLKHUB02A), which have equiphased outputs for multiple devices. LVPECL logic is used on every system critical clock signal distribution hub. LVPECL logic has the advantage over the Low-voltage differential signaling (LVDS\glos{LVDS}{Low-voltage differential signaling}) in the signal integrity robustness. LVPECL uses higher logical levels and higher signalling currents. The power consumption of LVPECL logic is nearly constant over the operating frequency range due to the use of bipolar transistors. This arrangement minimizes voltage glitches which are typical for CMOS\glos{CMOS}{Complementary metal–oxide–semiconductor} logic. One drawback of its parameters is a high power consumption of LVPECL logic, which reaches tens of milliamperes per device easily.

This design ensures that all system devices have access to the defined phase and known frequency.

This design ensures that all system devices have access to the defined phase and the known frequency.

This section deals with the description of the trial version based on Xilinx ML605 development board \ref[ML605-development-board]. The board had been used in a previous project and has not been used since then, but the FPGA parameters are more than sufficient of what we need for fast data acquisition system.

This section deals with the description of the trial version based on Xilinx ML605 development board, see Figure~\ref[ML605-development-board]. The board had been used in a previous project and has not been used since. However, the FPGA parameters are more than sufficient of what we need for fast data acquisition system.

We have used a centralized topology as a basis for frequency synthesis. One precise high-frequency and low-jitter digital oscillator has been used \cite[MLAB-GPSDO], while other working frequencies have been derived from it by the division of its signal. This central oscillator has a software defined GPS \glos{GPS}{Global Positioning System}  disciplined control loop for frequency stabilization.\fnote{SDGPSDO design has been developed in parallel to this diploma thesis as a related project, but it is not explicitly required by the thesis itself and thus it is described in a separate document}

We have used a centralized topology as a basis for frequency synthesis. One precise high-frequency and low-jitter digital oscillator has been used \cite[MLAB-GPSDO]. The other working frequencies have been derived from it by the division of its signal. This central oscillator has a software defined GPS\glos{GPS}{Global Positioning System}  disciplined control loop for frequency stabilization.\fnote{SDGPSDO design has been developed in parallel to this diploma project as a related project, but it is not explicitly required by the thesis itself and thus it is described in a separate document.}

We have used new methods of software frequency monitoring and compensation in order to meet modern requirements on radioastronomy equipment which need precise frequency and phase stability over a wide baseline scales for effective radioastronomy imaging.

We have used new methods of software frequency monitoring and compensation in order to meet modern requirements on the radioastronomy equipment, which needs the precise frequency and phase stability over a wide baseline scales for effective radioastronomy imaging.

The GPSDO device consists of Si570 chip with LVPECL output. Phase jitter of GPSDO \glos{GPSDO}{GPS disciplined oscillator} is determined mainly by Si570 phase noise. Parameters of the Si570 are summarized in the following table \ref[LO-noise] (source \cite[si570-chip] ).

The GPSDO device consists of Si570 chip with LVPECL output. The phase jitter of the GPS disciplined oscillator (GPSDO\glos{GPSDO}{GPS disciplined oscillator}) is determined mainly by Si570 phase noise. Parameters of the Si570 are summarized in Table~\ref[LO-noise] (source \cite[si570-chip] ).

The GPSDO design, that is included in data acquisition system, has special feature -- it generates time marks for a precise time-stamping of the received signal. Timestamps are created by disabling the local oscillator's outputs, connected to SDRX01B receivers, for 100 us.  As result, a rectangular click in the ADC input signal is created which appears as a horizontal line in spectrogram.

GPSDO design, which is included in the data acquisition system, has a special feature -- it generates time marks for a precise time-stamping of the received signal. Time-stamps are created by disabling the local oscillator outputs, connected to SDRX01B receivers, for 100 $\mu$s.  As the result, a rectangular click in the ADC input signal is created, which appears as a horizontal line in the spectrogram.

Timestamps should be seen in image \ref[meteor-reflection] (above and below the meteor reflection).

Time-stamps should be seen in the image in Figure~\ref[meteor-reflection] (above and below the meteor reflection).

Time-marking should be improved in future by digitization of GPS signal received by antenna on observational station. Following that, the GPS signal can be directly sampled by a dedicated receiver and one separate ADC module. Datafile then consists of samples from channels of radio-astronomy receivers along with the GPS signal containing precise time information.

Time-stamping should be improved in future by digitization of GPS signal received by the antenna on the observational station. Following that, the GPS signal can be directly sampled by a dedicated receiver and one separate ADC module. The datafile consists of samples from channels of radio-astronomy receivers along with the GPS signal containing the precise time information.

\caption/t Phase noise of the used Silicon Laboratories Si570 chip. Offset frequency is measured from carrier frequency. Values shown in the table are given for two different carrier frequencies.

\caption/t The phase noise of the used Silicon Laboratories Si570 chip. Offset frequency is measured from carrier frequency. Values shown in the table are given for two different carrier frequencies.

Several widely used and commercially easily accessible differential connectors were considered to be use in our design.

Several widely used and commercially easily accessible differential connectors were considered to be used in our design.

At the end, MiniSAS connector was chosen as the best option to be used in connecting together multiple ADC modules. A transition between SATA and miniSAS is achieved by SAS to SATA adapter cable which is commonly used in servers to connect SAS controller to multiple SATA hard disc in RAID systems and thus is commercially easily available. It is compatible with existing SATA cabling systems and aggregates multiple SATA cables to a single connector. It also has SPI configuration lines which can be seen in the following picture \ref[img-miniSAS-cable] as standard pinheader connector.

Finally, MiniSAS connector was chosen as the best option to be used in connecting multiple ADC modules together. A transition between SATA and miniSAS is achieved by SAS to SATA adapter cable, which is commonly used in servers to connect SAS controller to multiple SATA hard disc in RAID systems and thus is commercially easily available. It is compatible with existing SATA cabling systems and aggregates multiple SATA cables to a single connector. It also has SPI configuration lines which can be seen in Figure~\ref[img-miniSAS-cable] as the standard pinheader connector.

The main drawback of miniSAS PCB connectors lies in the fact, that they are manufactured in SMT versions only. SMT design may eventually decrease the durability of the connector even if outer metal housing of connector is designed to be mounted using a standard through-hole mounting method.

The main drawback of miniSAS PCB connectors lies in the fact, that they are manufactured in SMT versions only. SMT design may eventually decrease the durability of the connector even if the outer metal housing of the connector is designed to be mounted using a standard through-hole mounting method.

We use ADC devices that have DATA clock frequency eight times higher than sampling frequency in a single line output mode, implying a 40 MHz output bit rate. This implies a $ 1/4 \cdot 10^7 = 25\ $ns time length of data bit, which is equivalent to 7.5m light path in free space. If we use copper PCB with FR4 substrate layer or coaxial/twinax cable, we could obtain velocity factor of 0.66 at worst condition. Then the light path for the same bit rate $t_s$ will be 4.95 m. Although we do not have any cables in the system with comparable lengths, the worst data bit skew described by data sheets of the used components is $0.3 \cdot t_s$, which is 1.485 m. Therefore the length matching is not critical in our current design operating on lowest sampling speed. Length matching may become critical in future versions with higher sampling rates, where the cable length must be matched. However SATA cabling technology is already prepared for that case and matched SATA cables are a standard merchandise.

We use ADC devices that have DATA clock frequency eight times higher than sampling frequency in a single line output mode, implying a 40 MHz output bit rate. This implies a $ 1/4 \cdot 10^7 = 25\ $ns time length of data bit, which is equivalent to 7.5~m light path in a free space. If the copper PCB with FR4 substrate layer or the coaxial/twinax cable is used, we could obtain the velocity factor of 0.66 in the worst case. Consequently, the light path for the same bit rate $t_s$ will be 4.95~m. Although we do not have any cables in the system with comparable lengths, the worst data bit skew described by data sheets of the used components is $0.3 \cdot t_s$, which is 1.485~m. Therefore the length matching is not critical in our current design operating on the lowest sampling speed. The length matching may become critical in future versions with higher sampling rates, where the cable length must be matched. However SATA cabling technology is already prepared for that case and matched SATA cables are a standard merchandise.

As a result of our need to use the smallest number of cables possible, the choice fell on the serial LVDS format. Small number of differential pairs is an important parameter determining the construction complexity and reliability\cite[serial-lvds]. No many currently existing ADC devices have this kind of digital interface. An ultrasound AFE device chips seem to be ideal for this purpose -- the chip has integrated both front-end amplifiers and filters. It has a drawback though - it is incapable of handling differential input signal and has a relatively low dynamic range (as it consists only of 12bit ADC) and has many single ended ADC channels. Consequently, the scaling is possible only by a factor of 4 receivers (making 8 analogue single ended channels).

As a result of our need to use the smallest number of cables possible, the choice fell on the serial LVDS format. A small number of differential pairs is an important parameter determining the construction complexity and reliability\cite[serial-lvds]. No many currently existing ADC devices have this kind of digital interface. An ultrasound AFE device chips seem to be ideal for this purpose -- the chip has integrated both front-end amplifiers and filters. It has a drawback though. It is incapable of handling the differential input signal and has a relatively low dynamic range (as it consists only of 12bit ADC) and has many single ended ADC channels. Consequently, the scaling is possible only by a factor of 4 receivers (making 8 analog single ended channels).

If we add a requirement of separate output for every analogue channel and a 16bit depth we find that there are only a few 2-Channel simultaneous sampling ADCs currently existing which meet these criteria. We have summarized those ADCs in the following table \ref[ADC-types]

If we add a requirement of a separate output for every analog channel and a 16bit depth, we find that there are only a few 2-Channel simultaneous sampling ADCs currently existing which meet these criteria. We have summarized those ADCs in Table~\ref[ADC-types].

\typosize[9/11] \def\tabiteml{ }\let\tabitemr=\tabiteml

abiteml{ }\let\tabitemr=\tabiteml

All parts in this category are compatible with one board layout. The main differences lay in the sampling frequency and signal to noise ratio, with the slowest having a maximum sampling frequency of 20 MHz. However, all of them have a minimal sampling frequency of 5 MSPS and all are configurable over a serial interface (SPI). SPI seems to be a standard interface used in high-end ADC chips made by the largest manufacturers (Analog Devices, Linear technology, Texas instruments, Maxim integrated..).  We have selected two slowest types for our evaluation design. Following that, a PCB for this part have been designed.

All parts in this category are compatible with one board layout. The main differences lay in the sampling frequency and in the signal to noise ratio, with the slowest having a maximum sampling frequency of 20 MHz. However, all of them have a minimal sampling frequency of 5 MSPS and all are configurable over a serial interface (SPI). SPI seems to be a standard interface used in high-end ADC chips made by the largest manufacturers (Analog Devices, Linear technology, Texas instruments, Maxim integrated, etc.).  We have selected two slowest types for our evaluation design. Following that, a PCB for this part have been designed.

Data serial data outputs of ADC modules should be connected directly by LVDS signalling levels conducted by SATA cables to FPGAs for the basic primary signal processing. The ADC chips used in the modules have a selectable bit width of data output bus and thus the output SATA connectors have signals arranged to contain a single bit from every ADC channel.  This creates a signal concept enabling a selection of proper bus bit-width according to the sampling rate (higher bus bit-width downgrades signalling speed and vice versa.)

Data serial data outputs of ADC modules should be connected directly by LVDS signalling levels conducted by SATA cables to FPGAs for the basic primary signal processing. The ADC chips used in the modules have a selectable bit width of data output bus and thus the output SATA connectors have signals arranged to contain a single bit from every ADC channel.  This creates a signal concept enabling a selection of the proper bus bit-width according to the sampling rate (the higher bus bit-width downgrades signalling speed and vice versa.)

In order to connect the above mentioned signalling layout, miniSAS to multiple SATA cable should be used as described in section \ref[signal-cables].

In order to connect the above mentioned signalling layout, miniSAS to multiple SATA cable should be used as described in Section~\ref[signal-cables].

A KiCAD design suite had been chosen for PCB layout. However, the version, despite having integrated CERN Push \& Shove routing capability, is slightly unstable as it sometimes crushes due to an exception during routing. On the basis of these stability issues, the design had to be saved quite often. On the other hand, compared to commercially available solutions, such as MentorGraphics PADS or Cadence Orcad,  the Open-source KiCAD provides an acceptable option and it easily surpasses a widely used Eagle software.

KiCAD design suite had been chosen for PCB layout. However, the version, despite having integrated CERN Push \& Shove routing capability, is slightly unstable as it sometimes crushes due to an exception during routing. On the basis of these stability issues, the design had to be saved quite often. On the other hand, compared to commercially available solutions, such as MentorGraphics PADS or Cadence Orcad,  the Open-source KiCAD provides an acceptable option and it easily surpasses a widely used Eagle software.

All of the above-mentioned modes are supported by the module design. For the discussed data acquisition system, the 1-lane mode was selected. 1-lane mode allows a minimal number of differential pairs between ADCdual01A and FPGA. Digital signalling scheme used in 1-lane mode is shown in the following image \ref[1-line-out].

All of the above-mentioned modes are supported by the module design. For the discussed data acquisition system, the 1-lane mode was selected. 1-lane mode allows a minimal number of differential pairs between ADCdual01A and FPGA. Digital signalling scheme used in 1-lane mode is shown in Figure~\ref[1-line-out].

Both of the ADCdual01A modules were connected to FPGA ML605 board trough FMC2DIFF01A adapter board. The design of this adapter expects the presence of FMC LPC connector on host side and the board is, at the same time, not compatible with MLAB. It is, on the other hand, designed to meet the VITA 57 standard specifications for boards which support region 1 and region 3. VITA 57 regions are explained in the picture \ref[VITA57-regions].

Both of the ADCdual01A modules were connected to FPGA ML605 board trough FMC2DIFF01A adapter board. The design of this adapter expects the presence of FMC LPC connector on host side and the board is, at the same time, not compatible with MLAB. It is, on the other hand, designed to meet the VITA 57 standard specifications for boards which support region 1 and region 3. VITA 57 regions are explained in Figrue~\ref[VITA57-regions].

This industry standard guarantees the compatibility with other FPGA boards that have FMC LPC connectors for Mezzanine Card. Schematic diagram of designed adapter board is included in the appendix.

This industry standard guarantees the compatibility with other FPGA boards that have FMC LPC connectors for Mezzanine Card. Schematic diagram of designed adapter board is included in the Appendix.

The SY55857L is a fully differential, high-speed dual translator optimized to accept any logic standard from single-ended TTL/CMOS to differential LVDS, HSTL, or CML and translate it to LVPECL. Translation is guaranteed for speeds up to 2.5Gbps (2.5GHz toggle frequency). The SY55857L does not internally terminate its inputs, as different interfacing standards have different termination requirements\cite[SY55857L-chip].

The SY55857L is a fully differential, a high-speed dual translator optimized for accepting any logic standard from the single-ended TTL/CMOS to differential LVDS, HSTL, or CML and translate it to LVPECL. Translation is guaranteed for speeds up to 2.5Gbps (2.5GHz toggle frequency). The SY55857L does not internally terminate its inputs, as different interfacing standards have different termination requirements\cite[SY55857L-chip].

Inputs of both used chips are terminated accordingly to used logic. The LVDS input is terminated differentially by 100 $\Omega$ resistor between positive and negative inputs. PECL input is terminated by Thevenin resistor network. Thevenin termination method was selected as optimal one, due to the absence of a proper power voltage (1,3 V) for direct termination by 50 $\Omega$ resistors. Termination on FPGA side is realized directly by settings the proper digital logic type on input pins.

Inputs of both used chips are terminated accordingly to the used logic. The LVDS input is terminated differentially by 100~$\Omega$ resistor between the positive and the negative inputs. PECL input is terminated by Thevenin resistor network. Thevenin termination method was selected as optimal one, due to the absence of a proper power voltage (1.3 V) for direct termination by 50~$\Omega$ resistors. Termination on FPGA side is realized directly by settings the proper digital logic type on input pins.

Lengths of differential pairs routed on PCB of the module are not matched between the pairs. Length variation of differential pairs is not critical in our design according to facts discussed in paragraph \ref[diff-signaling]. Nevertheless, signals within differential pairs themselves are matched for length. Internal signal trace length matching of differential pairs is mandatory in order to minimize jitter and avoid a dynamic logic hazard conditions on digital signals, that represents the worst scenario. Thus the clocks' signals are routed in the most precise way on all designed boards.

Lengths of the differential pairs routed on PCB of the module are not matched between the pairs. The length variation of differential pairs is not critical in our design according to facts discussed in Section~\ref[diff-signaling]. Nevertheless, signals within differential pairs themselves are matched for length. Internal signal trace length matching of differential pairs is mandatory in order to minimize jitter and avoid a dynamic logic hazard conditions on digital signals, that represents the worst scenario. Thus the clocks signals are routed in the most precise way on all designed boards.

Signal configuration used in our trial design is described in the following tables \ref[minisas-interface], \ref[SPI-system] and \ref[clock-interconnections].

The signal configuration used in our trial design is described in Tables~\ref[minisas-interface], \ref[SPI-system] and \ref[clock-interconnections].

\caption/t SPI system interconnections

\caption/t SPI system interconnections.

\caption/t Clock system interconnections

\caption/t Clock system interconnections.

Several tasks in separate FPGA blocks are performed by FPGA. In the first block the FPGA prepares a sampling clock for ADCdual01A modules by dividing the signal from the main local oscillator. This task represents a separate block in FPGA and runs asynchronously to other logical circuits. Second block is a SPI configuration module, which sends configuration words to ADC modules and it is activated by opening of Xillybus interface file. Third block represents the main module, which resolves ADC - PC communication itself and it communicates via PCIe, collect data from ADC hardware and creates data packet \ref[xillybus-interface]. Last block is activated after the ADC is configurated via SPI.

Several tasks in the separate FPGA blocks are performed by FPGA. In the first block, the FPGA prepares a sampling clock for ADCdual01A modules by dividing the signal from the main local oscillator. This task represents a separate block in FPGA and runs asynchronously to other logical circuits. The second block is a SPI configuration module, which sends configuration words to ADC modules and it is activated by opening of Xillybus interface file. The third block represents the main module, which resolves ADC - PC communication itself and it communicates via PCIe, collect data from ADC hardware and creates data packet, Table~\ref[xillybus-interface]. The last block is activated after the ADC is configurated via SPI.

Communication over PCIe is managed by proprietary IP Core and Xillybus driver, which transfers data from FPGA registers to host PC. Data appear in a system device file named  "/dev/xillybus_data2_r" on the host computer. Binary data which appear in this file after its opening are shown in the table below \ref[xillybus-interface].

The communication over PCIe is managed by proprietary IP Core and Xillybus driver, which transfers data from FPGA registers to host PC. Data appear in a system device file named  "/dev/xillybus_data2_r" on the host computer. Binary data which appear in this file after its opening are shown in Table~\ref[xillybus-interface].

\caption/t System device "/dev/xillybus_data2_r" data format

\caption/t System device "/dev/xillybus_data2_r" data format.

Data packet block which is carried on PCI Express isa  described in the table \ref[xillybus-interface]. The data packet consist of several 32bit words. The first word contains FRAME number and it is filled with saw signal for now, with incremental step taking place every data packet transmission. The following data words contain samples from ADCs' first and second channel. Samples from every channel are transmitted in pairs of two samples. Number of ADC channels is expandable according to the number of physically connected channels. An CRC word may possibly be added in the future to the end of the transmission packet for data integrity validation.

The data packet block which is carried on PCI Express is described in Table~\ref[xillybus-interface]. The data packet consist of several 32bit words. The first word contains FRAME number and it is filled with saw signal for now, with incremental step taking place every data packet transmission. The following data words contain samples from ADCs' first and second channel. Samples from every channel are transmitted in pairs of two samples. Number of ADC channels is expandable according to the number of physically connected channels. An CRC word may possibly be added in the future to the end of the transmission packet for data integrity validation.

FRAME word at the beginning of data packet, now filled with incrementing and overflowing saw signal, is used to ensure that no data samples ale lost during the data transfers from FPGA. FRAME signal may be used in the future for pairing the ADC samples' data packet with another data packet. This new additional data packet should carry meta-data information about the sample time jitter, current accuracy of the local oscillator frequency etc.

FRAME word at the beginning of data packet, now filled with incrementing and overflowing saw signal, is used to ensure that no data samples ale lost during the data transfers from FPGA. FRAME signal may be used in the future for pairing the ADC samples data packet with another data packet. This new additional data packet should carry meta-data information about the sample time jitter, current accuracy of the local oscillator frequency etc.

Detailed description of the currently implemented FPGA functions can be found in a separate paper \cite[fpga-middleware]. HDL source codes for FPGA at a state in which it was used are included on the enclosed CD. More recent development versions are publicly available from MLAB sources repository.

Detailed description of the currently implemented FPGA functions can be found in a separate paper~\cite[fpga-middleware]. HDL source codes for FPGA at a state in which it was used are included on the enclosed CD. More recent development versions are publicly available from MLAB sources repository.

Detailed description of currently implemented FPGA functions can be found in separate paper \cite[fpga-middleware]. HDL \glos{HDL}{Hardware description language} source codes for FPGA at state which was used are included on enclosed CD. Future development versions are publicly available from MLAB sources repository \cite[mlab-sdrx].

Detailed description of currently implemented FPGA functions can be found in separate paper~\cite[fpga-middleware]. HDL\glos{HDL}{Hardware description language} source codes for FPGA at state which was used are included on enclosed CD. Future development versions are publicly available from MLAB sources repository~\cite[mlab-sdrx].

The interactive grabber-viewer user interface shows live oscilloscope-like time-value display for all data channels and live time-frequency scrolling display (a waterfall view) for displaying the frequency components of the grabbed signal. Signal is grabbed to file with exactly the same format, as described in the table \ref[xillybus-interface].

The interactive grabber-viewer user interface shows live oscilloscope-like time-value display for all data channels and live time-frequency scrolling display (a waterfall view) for displaying the frequency components of the grabbed signal. The signal is grabbed to the file with the exactly same format as described in Table \ref[xillybus-interface].