Subversion Repositories svnkaklik

\chap Trial version of the receiver, design and implementation

\chap Trial version of the digitizer

The whole design of the radioastronomic receiver digitization unit is meant to be used in a wide range of applications and tasks related to digitization of a signal. A good illustrating problem for its use is the signal digitization from multiple antenna arrays.

The whole design of the radioastronomic receiver digitization unit is meant to be used in a wide range of applications and tasks related to digitization of a signal. A good illustrating problem for its use is the signal digitization from multiple antenna arrays. Design and implementation of the system is presented in this chapter.

The sampling frequency has not been a limiting factor in the trial version. Generally, the sampling frequency is mostly limited by the sampling frequencies of the analog-to-digital conversion chips available on the market and by the interface bandwidth. The combination of required parameters -- dynamic range needing 16 bits  at least and a minimum sampling frequency of 1 Mega-Samples Per Second (MSPS\glos{MSPS}{Mega-Samples Per Second}) -- leads to the need of the high-end ADC chips. However, they do not support such low sampling frequencies at all. Their minimum sampling frequency is 5$\ $MSPS.

The sampling frequency has not been a limiting factor in the trial version. Generally, the sampling frequency is mostly limited by the sampling frequencies of the analog-to-digital conversion chips available on the market and by the interface bandwidth. The combination of required parameters -- dynamic range needing 16 bits  at least and a minimum sampling frequency of 1 Mega-Samples Per Second (MSPS\glos{MSPS}{Mega-Samples Per Second}) -- leads to the need of the high-end ADC chips. However, they support minimum sampling frequency 5$\ $MSPS.

We calculated the minimal data bandwidth data rate for eight receivers, 2~bytes per sample and 5$\ $MSPS as $8 \cdot 2 \cdot 5\cdot 10^6 = 80\ $MB/sec. Such a data rate is at the limit of the actual writing speed of a classical hard disk drive (HDD\glos{HDD}{Hard disk drive}) and it is almost a double the real bandwidth of USB~2.0\glos{USB 2.0}{Universal Serial Bus version 2.0} interface. As a result of these facts, we must use a faster interface. Such a faster interface is especially needed in cases in which we require faster sampling rates than ADC's minimal 5$\ $MSPS sample rate.

We calculated the minimal data bandwidth rate for eight receivers, 2~bytes per sample and 5$\ $MSPS as $8 \cdot 2 \cdot 5\cdot 10^6 = 80\ $MB/sec. Such a data rate is at the limit of the actual writing speed of a classical hard disk drive (HDD\glos{HDD}{Hard disk drive}) and it is almost a double the real bandwidth of USB~2.0\glos{USB 2.0}{Universal Serial Bus version 2.0} interface. As a result of these facts, we must use a faster interface. Such a faster interface is especially needed in cases in which we require faster sampling rates than ADC's minimal 5$\ $MSPS sample rate.

The above mentioned concept of the scalable design requires a relatively long circuit traces between ADC and the digital unit which captures the data and performs the computations. The long distance between the digital processing unit and the analog-to-digital conversion unit has the advantage in noise retention typically produced by digital circuits. Those digital circuits, such as FPGA\glos{FPGA}{Field-programmable gate array}, Ethernet or other flip-flops blocks and circuit traces, which work usually at high frequencies and emit the wide-band noise with relatively low power. In such cases, any increase in a distance between the noise source and the analog signal source increases S/N significantly. However, at the same time, a long distance introcuces problems with the digital signal transmission between ADC and the computational unit. However, this obstacle should be resolved more easily in a free-space than on board routing. The high-quality differential signalling shielded cables should be used, such as massively produced and cheap SATA\glos{SATA}{Serial ATA}\glos{ATA}{AT Attachment} cables. This technology has two advantages over PCB\glos{PCB}{printed circuit board} signal routing. First, it can use twisted pair of wires for leak inductance suppression in signal path. Second, the twisted pair may additionally be shielded by uninterrupted metal foil.

The above mentioned concept of the scalable design requires a relatively long circuit traces between ADC and the digital unit which captures the data and performs the computations. The long distance between the digital processing unit and the analog-to-digital conversion unit has the advantage in noise retention typically produced by digital circuits. Those digital circuits, such as FPGA\glos{FPGA}{Field-programmable gate array}, Ethernet or other flip-flops blocks and circuit traces work usually at high frequencies and emit the wide-band noise with relatively low power. In such cases, any increase in a distance between the noise source and the analog signal source increases S/N significantly. However, at the same time, a long distance introcuces problems with the digital signal transmission between ADC and the computational unit. This obstacle should be resolved more easily in a free-space than on board routing. The high-quality differential signalling shielded cables should be used, such as massively produced and cheap SATA\glos{SATA}{Serial ATA}\glos{ATA}{AT Attachment} cables. This technology has two advantages over PCB\glos{PCB}{printed circuit board} signal routing. First, it can use twisted pair of wires for leak inductance suppression in signal path. Second, the twisted pair may additionally be shielded by uninterrupted metal foil.

The high phase stability is achieved in our scalable design through centralized frequency generation and distribution with multi-output Low Voltage Emitter-coupled logic (LVPECL\glos{LVPECL}{Low Voltage Emitter-coupled logic}) hubs (CLKHUB02A), which have equiphased outputs for multiple devices. LVPECL logic is used on every system critical clock signal distribution hub. LVPECL logic has the advantage over the Low-voltage differential signaling (LVDS\glos{LVDS}{Low-voltage differential signaling}) in the signal integrity robustness. LVPECL uses higher logical levels and higher signalling currents. The power consumption of LVPECL logic is nearly constant over the operating frequency range due to the use of bipolar transistors. This arrangement minimizes voltage glitches which are typical for CMOS\glos{CMOS}{Complementary metal–oxide–semiconductor} logic. One drawback of its parameters is a high power consumption of LVPECL logic, which reaches tens of milliamperes per device easily.

The high phase stability is achieved in our scalable design through centralized frequency generation and distribution with multi-output Low Voltage Emitter-coupled logic (LVPECL\glos{LVPECL}{Low Voltage Emitter-coupled logic}) hubs (CLKHUB02A), which have equiphased outputs for multiple devices. The LVPECL logic is used on every system critical clock signal distribution hub. This logic has the advantage over the Low-voltage differential signaling (LVDS\glos{LVDS}{Low-voltage differential signaling}) in the signal integrity robustness. It uses higher logical levels and higher signalling currents. The power consumption of LVPECL logic is nearly constant over the operating frequency range due to the use of bipolar transistors. This arrangement minimizes voltage glitches which are typical for CMOS\glos{CMOS}{Complementary metal–oxide–semiconductor} logic. One drawback of its parameters is a high power consumption of LVPECL logic, which reaches tens of milliamperes per device easily.

This section deals with the description of the trial version based on Xilinx ML605 development board, see Figure~\ref[ML605-development-board]. The board had been used in a previous project and has not been used since. However, the FPGA parameters are more than sufficient of what we need for fast data acquisition system.

This section deals with the description of the trial version based on Xilinx ML605 development board, see Figure~\ref[ML605-development-board], available at the workplace. This FPGA parameters are more than sufficient of what we need for the fast data acquisition system being developed.

We have used a centralized topology as a basis for frequency synthesis. One precise high-frequency and low-jitter digital oscillator has been used \cite[MLAB-GPSDO]. The other working frequencies have been derived from it by the division of its signal. This central oscillator has a software defined GPS\glos{GPS}{Global Positioning System}  disciplined control loop for frequency stabilization.\fnote{SDGPSDO design has been developed in parallel to this diploma project as a related project, but it is not explicitly required by the thesis itself and thus it is described in a separate document.}

We have used a centralized topology as a basis for frequency synthesis. One precise high-frequency and low-jitter digital oscillator (GPSDO\glos{GPSDO}{GPS disciplined oscillator}) has been used \cite[MLAB-GPSDO]. The other working frequencies have been derived from it by the division of its signal. This central oscillator has a software defined GPS\glos{GPS}{Global Positioning System}  disciplined control loop for frequency stabilization.\fnote{SDGPSDO design has been developed in parallel to this diploma project as a related project, but it is not explicitly required by the thesis itself and thus it is described in a separate document.}

The GPSDO device consists of Si570 chip with LVPECL output. The phase jitter of the GPS disciplined oscillator (GPSDO\glos{GPSDO}{GPS disciplined oscillator}) is determined mainly by Si570 phase noise. Parameters of the Si570 are summarized in Table~\ref[LO-noise] (source \cite[si570-chip] ).

The GPSDO device consists of Si570 chip with LVPECL output. The phase jitter of the GPS disciplined oscillator is determined mainly by Si570 phase noise. Parameters of the Si570 are summarized in Table~\ref[LO-noise] (source \cite[si570-chip] ).

\caption/t The phase noise of the used Silicon Laboratories Si570 chip. Offset frequency is measured from carrier frequency. Values shown in the table are given for two different carrier frequencies.

\caption/t The phase noise of the used Silicon Laboratories Si570 chip. Offset frequency is measured from carrier frequency. Values shown in the table are given for two different carrier frequencies. Adopted from \cite[si570-chip].

Several widely used and commercially easily accessible differential connectors were considered to be used in our design.

Several widely used and commercially easily accessible differential connectors were considered to be used in our design:

* HDMI % [[http://en.wikipedia.org/wiki/Hdmi|HDMI]]</del>

* HDMI, % [[http://en.wikipedia.org/wiki/Hdmi|HDMI]]</del>

* SATA  		%{http://en.wikipedia.org/wiki/Serial_attached_SCSI#Connectors|SAS]]/[[http://en.wikipedia.org/wiki/Serial_ATA|SATA]]

* SATA,  		%{http://en.wikipedia.org/wiki/Serial_attached_SCSI#Connectors|SAS]]/[[http://en.wikipedia.org/wiki/Serial_ATA|SATA]]

* DisplayPort 		%[[http://en.wikipedia.org/wiki/Display_port|DisplayPort]]</del>

* DisplayPort, or 		%[[http://en.wikipedia.org/wiki/Display_port|DisplayPort]]</del>

* SAS/miniSAS

* SAS/miniSAS.

\caption/f An example of miniSAS cable similar to used.

\caption/f An example of a miniSAS cable.

We use ADC devices that have DATA clock frequency eight times higher than sampling frequency in a single line output mode, implying a 40 MHz output bit rate. This implies a $ 1/4 \cdot 10^7 = 25\ $ns time length of data bit, which is equivalent to 7.5~m light path in a free space. If the copper PCB with FR4 substrate layer or the coaxial/twinax cable is used, we could obtain the velocity factor of 0.66 in the worst case. Consequently, the light path for the same bit rate $t_s$ will be 4.95~m. Although we do not have any cables in the system with comparable lengths, the worst data bit skew described by data sheets of the used components is $0.3 \cdot t_s$, which is 1.485~m. Therefore the length matching is not critical in our current design operating on the lowest sampling speed. The length matching may become critical in future versions with higher sampling rates, where the cable length must be matched. However SATA cabling technology is already prepared for that case and matched SATA cables are a standard merchandise.

We use ADC devices that have DATA clock frequency eight times higher than sampling frequency in a single line output mode, implying a 40 MHz output bit rate. This implies a $t_s=25$~ns time length of data bit, which is equivalent to 7.5~m light path in a free space. If the copper PCB with FR4 substrate layer or the coaxial/twinax cable is used, we could obtain the velocity factor of 0.66 in the worst case. Consequently, the light path for the same bit rate $t_s$ will be 4.95~m. Although we do not have any cables in the system with comparable lengths, the worst data bit skew described by data sheets of the used components is $0.3 \cdot t_s$, which is 1.485~m. Therefore the length matching is not critical in our current design operating on the used sampling speed. The length matching may become critical in future versions with higher sampling rates, where the cable length must be matched. However SATA cabling technology is already prepared for that case and matched SATA cables are a standard merchandise.

There exist several standard ADC signalling formats currently used in communication with FPGA.

There exist several standard ADC signalling formats currently used in communication with FPGA:

  * DDR LVDS

  * DDR LVDS,

  * JEDEC 204B

  * JEDEC 204B,

  * JESD204A

  * JESD204A,

  * Paralel LVDS

  * Paralel LVDS,

  * Serdes

  * Serdes,

  * serial LVDS

  * serial LVDS.

As a result of our need to use the smallest number of cables possible, the choice fell on the serial LVDS format. A small number of differential pairs is an important parameter determining the construction complexity and reliability\cite[serial-lvds]. No many currently existing ADC devices have this kind of digital interface. An ultrasound AFE device chips seem to be ideal for this purpose -- the chip has integrated both front-end amplifiers and filters. It has a drawback though. It is incapable of handling the differential input signal and has a relatively low dynamic range (as it consists only of 12bit ADC) and has many single ended ADC channels. Consequently, the scaling is possible only by a factor of 4 receivers (making 8 analog single ended channels).

As a result of our need to use the smallest number of cables possible, the choice fell on the serial LVDS format. A small number of differential pairs is an important parameter determining the construction complexity and reliability~\cite[serial-lvds]. No many currently existing ADC devices have this kind of digital interface. An ultrasound AFE device chips seem to be ideal for this purpose -- the chip has integrated both front-end amplifiers and filters. It has a drawback though. It is incapable of handling the differential input signal and has a relatively low dynamic range (as it consists only of 12bit ADC) and has many single ended ADC channels. Consequently, the scaling is possible only by a factor of 4 receivers (making 8 analog single ended channels).

All parts in this category are compatible with one board layout. The main differences lay in the sampling frequency and in the signal to noise ratio, with the slowest having a maximum sampling frequency of 20 MHz. However, all of them have a minimal sampling frequency of 5 MSPS and all are configurable over a serial interface (SPI). SPI seems to be a standard interface used in high-end ADC chips made by the largest manufacturers (Analog Devices, Linear technology, Texas instruments, Maxim integrated, etc.).  We have selected two slowest types for our evaluation design. Following that, a PCB for this part have been designed.

All parts in this category are compatible with one board layout. The main differences lay in the sampling frequency and in the signal to noise ratio, with the slowest having a maximum sampling frequency of 20~MSPS. However, all of them have a minimal sampling frequency of 5~MSPS and all are configurable over a serial interface (SPI). SPI seems to be a standard interface used in high-end ADC chips made by the largest manufacturers (Analog Devices, Linear technology, Texas instruments, Maxim integrated, etc.).  For the first testing realisation, we have selected two slowest types for our evaluation design -- LTC2271 and LTC2190. Following that, a PCB for this part have been designed.

In order to connect the above mentioned signalling layout, miniSAS to multiple SATA cable should be used as described in Section~\ref[signal-cables].

In order to connect the above mentioned signalling layout, miniSAS to multiple SATA cable is used as described in Section~\ref[signal-cables].

As a part of work on the thesis, new PCB footprints for FMC, SATA, ADCs a and miniSAS connectors have been designed and were committed to KiCAD github library repository. They are now publicly available on the official KiCAD repository at GitHub.

KiCAD design suite had been chosen for PCB layout. As a part of work on the thesis, new PCB footprints for FMC, SATA, ADCs a and miniSAS connectors have been designed and were committed to official KiCAD GitHub library repository. Thus, they are now publicly available.

ADCdual01A module has several digital data output formats. Difference between these modes lays in the number of differential pairs used.

ADCdual01A module has several digital data output formats. Difference between these modes lays in the number of differential pairs used:

    * 1-lane mode

    * 1-lane mode,

    * 2-lane mode

    * 2-lane mode,

    * 4-lane mode

    * 4-lane mode.

All of the above-mentioned modes are supported by the module design. For the discussed data acquisition system, the 1-lane mode was selected. 1-lane mode allows a minimal number of differential pairs between ADCdual01A and FPGA. Digital signalling scheme used in 1-lane mode is shown in Figure~\ref[1-line-out].

All of the above-mentioned modes are supported by the module design. For the discussed data acquisition system, the 1-lane mode was selected. The 1-lane mode allows a minimal number of differential pairs between ADCdual01A and FPGA. Digital signalling scheme used in the 1-lane mode is shown in Figure~\ref[1-line-out].

Complete schematic diagram of ADCdual01A module board is included in the appendix.

Figure~\ref[adcdual-preview] shows realized ADCdual01A module. Complete schematic diagram of ADCdual01A module board is included in Appendix~\ref[adc-scheme].

\picw=10cm \cinspic ./img/FMC2DIFF_Top_Big.JPG

\picw=10cm \cinspic ./img/ADCdual01A_Top_Big.JPG

\picw=10cm \cinspic ./img/FMC2DIFF_Bottom_Big.JPG

\picw=10cm \cinspic ./img/ADCdual01A_Bottom_Big.JPG

\caption/f Realised PCB of FMC2DIFF01A module.

\caption/f Realised PCB of ADCdual01A module. Differential pairs routings are clearly visible.

Both of the ADCdual01A modules were connected to FPGA ML605 board trough FMC2DIFF01A adapter board. The design of this adapter expects the presence of FMC LPC connector on host side and the board is, at the same time, not compatible with MLAB. It is, on the other hand, designed to meet the VITA 57 standard specifications for boards which support region 1 and region 3. VITA 57 regions are explained in Figrue~\ref[VITA57-regions].

Both of the ADCdual01A modules were connected to FPGA ML605 board trough FMC2DIFF01A adapter board. The design of this adapter expects the presence of FMC LPC connector on host side. It is designed to meet the VITA 57 standard specifications for boards which support region 1 and region 3. VITA 57 regions are explained in Figure~\ref[VITA57-regions].

This industry standard guarantees the compatibility with other FPGA boards that have FMC LPC connectors for Mezzanine Card. Schematic diagram of designed adapter board is included in the Appendix.

This industry standard guarantees the compatibility with other FPGA boards that have FMC LPC connectors for Mezzanine Card. Schematic diagram of designed adapter board is included in Appendix~\ref[fmc-scheme].

\secc FPGA function

This section describes a specification of data concentrator built using FPGA board. The HDL implementation was created by my colleague Ond{\v r}ej Sychrovsk{\'y}. Detailed description of the currently implemented FPGA functions can be found in a separate paper~\cite[fpga-middleware].

Several tasks in the separate FPGA blocks are performed by FPGA. In the first block, the FPGA prepares a sampling clock for ADCdual01A modules by dividing the signal from the main local oscillator. This task represents a separate block in FPGA and runs asynchronously to other logical circuits. The second block is a SPI configuration module, which sends configuration words to ADC modules and it is activated by opening of Xillybus interface file. The third block represents the main module, which resolves ADC - PC communication itself and it communicates via PCIe, collect data from ADC hardware and creates data packet, Table~\ref[xillybus-interface]. The last block is activated after the ADC is configurated via SPI.

Several tasks in the separate IP blocks are performed by FPGA. In the first block, the FPGA prepares a sampling clock for ADCdual01A modules by dividing the signal from the main local oscillator. This task represents a separate block in FPGA and runs asynchronously to other logical circuits. The second block is a SPI configuration module, which sends configuration words to ADC modules and it is activated by opening of Xillybus interface file. The third block represents the main module, which resolves ADC -- PC communication itself and it communicates via PCIe, collect data from ADC hardware and creates data packet, Table~\ref[xillybus-interface]. The last block is activated after the ADC is configurated via SPI.

Detailed description of currently implemented FPGA functions can be found in separate paper~\cite[fpga-middleware]. HDL\glos{HDL}{Hardware description language} source codes for FPGA at state which was used are included on enclosed CD. Future development versions are publicly available from MLAB sources repository~\cite[mlab-sdrx].

HDL source codes for FPGA at a state in which it was used are included on the enclosed CD. Future development versions will be publicly available from MLAB sources repository~\cite[mlab-sdrx].

In order to read the data stream from the ADC drive, we use Gnuradio software. Gnuradio suite consists of gnuradio-companion package which is a graphical tool for creating signal-flow graphs and generating Python flow-graph source code. This tool has been used to create a basic RAW data grabber to record and interactively view waterfall plots using the data streams output from ADC modules.

In order to read the data stream from the ADC drive, we use Gnuradio software. Gnuradio suite consists of gnuradio-companion package which is a graphical tool for creating signal-flow graphs and generating Python flow-graph source code. This tool has been used to create a basic RAW data grabber to record and interactively view waterfall plots using the data streams output from ADC modules. The ADC recorder flow graph is shown in Figure~\ref[grabber-flow-graph].

The interactive grabber-viewer user interface shows live oscilloscope-like time-value display for all data channels and live time-frequency scrolling display (a waterfall view) for displaying the frequency components of the grabbed signal. The signal is grabbed to the file with the exactly same format as described in Table \ref[xillybus-interface].

The interactive grabber-viewer user interface shows live oscilloscope-like time-value display for all data channels and live time-frequency scrolling display (a waterfall view) for displaying the frequency components of the grabbed signal. The signal is grabbed to the file with the exactly same format as described in Table \ref[xillybus-interface]. An example of interactive grabber-viewer showing a part of the grabbed signal is in Figure~\ref[grabber-data].