| 1,49 → 1,51 |
|---|
| \chap Trial design |
| |
| The whole design of radioastronomy receiver digitalization unit is constructed to be used in a wide range of applications and tasks related to digitalization of signal from radioastronomy receivers. A good illustrating problem for its use is a signal digitalisation from multiple antenna arrays. This design will eventually become a part of MLAB Advanced Radio Astronomy System. |
| The whole design of radioastronomy receiver digitalization unit is constructed to be used in a wide range of applications and tasks related to digitalization of signal from radioastronomy receivers. A good illustrating problem for its use is a signal digitalisation from multiple antenna arrays. |
| |
| \sec Required parameters |
| |
| Wide dynamical range and high IP3 are desired. The receiver must accept wide dynamic signals because a typical radioastronomical signal has a form of a weak signal covered by a strong man-made noise. |
| We require following technical parameter, to supersede existing digitalization units solutions. |
| Primarily, we need wide dynamical range and high IP3. The receiver must accept wide dynamic signals because a typical radioastronomical signal has a form of a weak signal covered by a strong man-made noise or other undesired noises as lighting, Sun emissions etc. |
| |
| Summary of main required parameters follows |
| Summary of other additional required parameters follows |
| |
| \begitems |
| * Dynamical range better than 80 dB |
| * Dynamical range better than 80 dB see section \ref[dynamic-range-theory] for explanation |
| * Phase stability between channels |
| * Noise (all types) |
| * Low noise (all types) |
| * Sampling jitter better than 100 metres |
| * Support for any number of receivers in range 1 to 8 |
| \enditems |
| |
| Now we analyzes several parameters more precisely. |
| |
| \sec Sampling frequency |
| |
| Sampling frequency is limited by the technical constrains in the trial design. This parameter is especially limited by the sampling frequencies of analog-to-digital conversion chips available on the market and interface bandwidth. Combination of the required parameters -- dynamic range requiring at least 16bit and a minimum sampling frequency of 1$\ $MSPS leads to need of high end ADC chips which does not support such low sampling frequencies at all. Their minimum sampling frequency is 5$\ $MSPS. |
| Sampling frequency is not limited by the technical constrains in the trial design. This parameter is especially limited by the sampling frequencies of analog-to-digital conversion chips available on the market and interface bandwidth. Combination of the required parameters -- dynamic range requiring at least 16bit and a minimum sampling frequency of 1$\ $MSPS leads to need of high end ADC chips which does not support such low sampling frequencies at all. Their minimum sampling frequency is 5$\ $MSPS. |
| |
| We calculate minimum data bandwidth data rate for eight receivers, 2 bytes per sample and 5$\ $MSPS as $8 \cdot 2 \cdot 5\cdot 10^6 = 80\ $MB/s. Such data rate is at the limit of real writing speed o classical HDD and it is almost double of real bandwidth of USB 2.0 interface. |
| |
| We calculate minimum data bandwidth data rate for eight receivers, 2 bytes per sample and 5$\ $MSPS as $8 \cdot 2 \cdot 5\cdot 10^6 = 80\ $MB/s. Such data rate is at the limit of real writing speed o classical HDD and it is almost double of real bandwidth of USB 2.0 interface. As result of this facts we must use faster interface. Faster interface is especially needed in case where we need faster sampling rates than ADC minimal 5$\ $MSPS sample rate. |
| Most perspective interfaces for use in our type of application is USB 3.0 or PCI Express interface. Although USB 3.0 is new technology without availability of good development tools. We used PCI Express interface as simplest and most reliable solution. |
| |
| \sec System scalability |
| |
| For analogue channels scalability, special parameters of ADC modules are required. Ideally, there should be a separate output for each analogue channel in ADC module. ADC module must also have separate outputs for frames and data output clocks. These parameters allow for conduction at relatively low digital data rates. As a result, the digital signal can be conducted even through long wires. |
| For analogue channels scalability, special parameters of ADC modules are required. Ideally, there should be a separate output for each analogue channel in ADC module. ADC module must also have separate outputs for frames and data output clocks. These parameters allow for conduction at relatively low digital data rates. As a result, the digital signal can be conducted even through long wires. Modular concept allows separation from central logic which support optimization of number analogue channels. |
| |
| Clock signal will be handled distinctively in our scalable design. Selected ADC chip are guaranteed to have defined clock skew between sampling and data output clock. This allows taking data and frame clocks from the first ADC module only. The rest of the data and frame clocks from other ADC modules can be measured for diagnostic purposes (failure detection, jitter measurement etc.). If more robustness is required from designs DCO and FR signal may be collected from other modules and routed through an voting logic which will correct possible signal defects. |
| Clock and data signals will be then handled distinctively in our modular scalable design. Selected ADC chip are guaranteed to have defined clock skew between sampling and data output clock. This allows taking data and frame clocks from the first ADC module only. The rest of the data and frame clocks from other ADC modules can be measured for diagnostic purposes (failure detection, jitter measurement etc.) but these redundant signals are not used for data sampling. If more robustness is required in final application, DCO and FR signal may be collected from other modules and routed through an voting logic which will correct possible signal defects. |
| |
| This system concept allows for scalability, that is technically limited by a number of differential signals on host side and its computational power. There is another advantage of scalable data acquisition system -- an economic one. Observatories or end users can make a choice of how much money are they willing to spent on radioastronomy receiver system. This freedom of choice is especially useful for science sites without previous experience in radioastronomy observations. |
| |
| \secc Differential signaling |
| |
| The concept of scalable design requires relatively long circuit traces between ADC and digital unit which captures the data and performs the computations. The long distance between the digital processing unit and the analog-to-digital conversion unit has an advantage in noise retention typically produced by digital circuits. Those digital circuits, such as FPGA or other flip-flops block and circuit traces, usually work at high frequencies and emit wide-band noise with relatively low power. In such cases any increase in a distance between the noise source and analog signal source increase S/N significantly. However, at the same time a long distance brings problems with the digital signal transmission between ADC and computational unit. This obstacle should be resolved more easily in free-space than on board routing. The high-quality differential signalling shielded cables should be used. This technology has two advantages over PCB signal routing. First, it can use twisted pair of wires for leak inductance suppression in signal path and second, the twisted pair may additionally be shielded by uninterrupted metal foil. |
| The above mentioned concept of scalable design requires relatively long circuit traces between ADC and digital unit which captures the data and performs the computations. The long distance between the digital processing unit and the analog-to-digital conversion unit has an advantage in noise retention typically produced by digital circuits. Those digital circuits, such as FPGA, Ethernet or other flip-flops blocks and circuit traces, usually work at high frequencies and emit wide-band noise with relatively low power. In such cases any increase in a distance between the noise source and analog signal source increase S/N significantly. However, at the same time a long distance brings problems with the digital signal transmission between ADC and computational unit. But this obstacle should be resolved more easily in free-space than on board routing. The high-quality differential signalling shielded cables should be used such as massively produced and cheap SATA cables. This technology has two advantages over PCB signal routing. First, it can use twisted pair of wires for leak inductance suppression in signal path and second, the twisted pair may additionally be shielded by uninterrupted metal foil. |
| |
| \secc Phase matching |
| |
| For multiple antenna radioastronomy projects, system phase stability is a mandatory condition. It allows precise high resolution imaging of objects. |
| For multiple antenna radioastronomy projects, system phase stability is a mandatory condition. It allows precise high resolution imaging of objects, increases signal to noise ratios in several observation methods and allows use of advanced algorithms for signal processing. |
| |
| High phase stability in our scalable design is achieved through centralized frequency generation and distribution with multi-output LVPECL hubs, that have equiphased outputs for multiple devices. LVPECL logic is used on every system critical clock signal distribution hub. LVPECL logic has advantage over LVDS in signal integrity robustness. LVPECL uses higher logical levels and higher signalling currents. Consumption currents of LVPECL logic are near constant over operating frequency range due to use of bipolar transistor this minimises voltage glitches which are typical for CMOS logic. One drawbacks of that parameters is high power consumption of LVPECL logic. |
| High phase stability in our scalable design is achieved through centralized frequency generation and distribution with multi-output LVPECL hubs (CLKHUB02A), that have equiphased outputs for multiple devices. LVPECL logic is used on every system critical clock signal distribution hub. LVPECL logic has advantage over LVDS in signal integrity robustness. LVPECL uses higher logical levels and higher signalling currents. Power consumption of LVPECL logic are near constant over operating frequency range due to use of bipolar transistors this minimizes voltage glitches which are typical for CMOS logic. One drawbacks of that parameters is high power consumption of LVPECL logic which easily reach tens of milliamperes per device. |
| |
| This design ensures that all devices have access to the defined phase and known frequency. |
| This design ensures that all system devices have access to the defined phase and known frequency. |
| |
| |
| \sec System description |
| |
| In this section testing system will be described. |
| 50,8 → 52,8 |
|---|
| |
| \secc Frequency synthesis |
| |
| We have used a centralized topology as a basis for frequency synthesis. One precise high-frequency and low-jitter digital oscillator has been used \cite[MLAB-GPSDO], while other working frequencies have been derived from it by the division of its signal. This central oscillator has a software defined GPS disciplined control loop for frequency stabilization.\fnote{SDGPSDO design has been developed in parallel to this diploma thesis as a related project, but it is not explicitly required by the diploma thesis.} |
| We have used methods of frequency monitoring compensation in order to meet modern requirements on radioastronomy equipment which needs precise frequency and phase stability over a wide scale for effective radioastronomy imaging. |
| We have used a centralized topology as a basis for frequency synthesis. One precise high-frequency and low-jitter digital oscillator has been used \cite[MLAB-GPSDO], while other working frequencies have been derived from it by the division of its signal. This central oscillator has a software defined GPS disciplined control loop for frequency stabilization.\fnote{SDGPSDO design has been developed in parallel to this diploma thesis as a related project, but it is not explicitly required by the diploma thesis. Thus is described in separate document} |
| We have used new methods of software frequency monitoring and compensation in order to meet modern requirements on radioastronomy equipment which needs precise frequency and phase stability over a wide baseline scale for effective radioastronomy imaging. |
| |
| GPSDO device consists the Si570 chip with LVPECL output. Phase jitter of GPSDO is determined mainly by Si570 phase noise. Parameters of used Si570 from source \cite[si570-chip] are summarized in table \ref[LO-noise]. |
| |
| 68,7 → 70,7 |
|---|
| 10 [MHz] & –147 & –146 \cr |
| 100 [MHz] & n/a & –148 \cr |
| } |
| \caption/t The summary of available ADC types and theirs characteristics. |
| \caption/t Phase noise of used Silicon Laboratories Si570 chip. Offset frequency is measured from carrier frequency. Values are tabled for two district carrier frequencies. |
| \endinsert |
| |
| Every ADC module will be directly connected to CLKHUB02A module which takes sampling clock signal delivered by FPGA from main local oscillator. This signal should use high quality differential signaling cable -- we should use SATA cable for this purpose. FPGA may slightly affect clock signal quality by additive noise, but has negligible effect in application where developed system will be used. |
| 112,7 → 114,13 |
|---|
| \caption/f FPGA ML605 development board. |
| \endinsert |
| |
| <<<<<<< .mine |
| |
| |
| |
| ======= |
| |
| >>>>>>> .r1124 |
| \secc ADC selection |
| |
| There exist several ADC signaling formats currently used in communication with FPGA. |
| 126,7 → 134,7 |
|---|
| * serial LVDS |
| \enditems |
| |
| Because it uses the smallest number of differential pairs, the choice fell on the serial LVDS format. Small number of differential pairs is an important parameter determining the construction complexity and reliability\cite[serial-lvds]. |
| Because we need to use the smallest number of cables, the choice fell on the serial LVDS format. Small number of differential pairs is an important parameter determining the construction complexity and reliability\cite[serial-lvds]. |
| |
| An ultrasound AFE chip seems to be ideal for this purpose -- the chip has integrated both front-end amplifiers and filters. It has a drawback though - it is incapable of handling differential input signal and has a relatively low dynamic range (as it consists only of 12bit ADC). Because this IO has many ADC channels the scaling is possible only by a factor of 4 receivers (making 8 analogue channels). |
| |
| 201,10 → 209,10 |
|---|
| LVDS is intended to drive 50 $\Omega$ impedance transmission |
| line media such as PCB traces, backplanes, or cables. SY55855V inputs can be terminated with a single resistor between the true and the complement pins of a given input \cite[SY55855V-chip]. |
| |
| The SY55857L is a fully differential, high-speed dual translator optimized to accept any logic standard from single-ended TTL/CMOS to differential LVDS, HSTL, or CML and translate it to LVPECL. Translation is guaranteed for speeds up to 2.5Gbps (2.5GHz toggle frequency). The SY55857L does not internally terminate its inputs, as different interfacing standards have different termination requirements. |
| The SY55857L is a fully differential, high-speed dual translator optimized to accept any logic standard from single-ended TTL/CMOS to differential LVDS, HSTL, or CML and translate it to LVPECL. Translation is guaranteed for speeds up to 2.5Gbps (2.5GHz toggle frequency). The SY55857L does not internally terminate its inputs, as different interfacing standards have different termination requirements\cite[SY55857L-chip]. |
| |
| Inputs of both used chips are terminated accordingly to used logic. The LVDS input is terminated differentially by 100 $\Omega$ resistor between positive and negative inputs. PECL input is terminated by Thevenin resistor network. Thevenin termination method was selected as optimal due to absence of proper power voltage (1,3 V) for direct termination by 50 $\Omega$ resistors. Termination on FPGA side is realized directly by settings proper digital logic on input pins. |
| |
| |
| \midinsert |
| \picw=10cm \cinspic ./img/ML605-board.jpg |
| \caption/f FPGA ML605 development board. |
| 222,13 → 230,10 |
|---|
| Lengths of differential pairs routed on PCB of the module are not matched between the pairs. Length variation of differential pairs is not critical in our design according to facts discussed in paragraph \ref[diff-signaling]. Nevertheless, signals within differential pairs themselves are matched for length. Internal signal tracing of the length matchting of differential pairs is mandatory in order to avoid a dynamic logic hazard conditions on digital signals. Thus clocks' signals are routed in the most precise way on all designed boards. |
| |
| |
| Signal configuration used in our trial design is described in the following tables. |
| Signal configuration used in our trial design is described in the following tables \ref[minisas-interface], \ref[SPI-system] and \ref[clock-interconnections]. |
| |
| %% zapojeni SPI, FPGA zpatky necte konfiguraci, ale je tam na slepo nahravana. |
| |
| |
| |
| \midinsert \clabel[minisas-interface]{Grabber binary output format} |
| \midinsert \clabel[minisas-interface]{miniSAS differential pairs connections} |
| \ctable {cccc} |
| { |
| miniSAS & SATA pair & FMC signal & Used as \cr |
| 246,7 → 251,7 |
|---|
| |
| SPI interface is used in an unusual way in this design. SPI Data outputs from ADCs are not connected anywhere and read back is not possible, thus the configuration written to registers in ADC module cannot be validated. We have not observed any problems with this system, but it may be a possible source of failures. |
| |
| \midinsert \clabel[SPI-system]{Grabber binary output format} |
| \midinsert \clabel[SPI-system]{SPI configuration interface connections} |
| \ctable {ccc} |
| { |
| SPI connection J7 & FMC signal & Connected to \cr |
| 263,7 → 268,7 |
|---|
| \endinsert |
| |
| |
| \midinsert \clabel[clock-interconnections]{Grabber binary output format} |
| \midinsert \clabel[clock-interconnections]{System clock interconnections} |
| \ctable {lccc} |
| { |
| Signal & FMC signal & FMC2DIFF01A & ADCdual01A \cr |
| 279,7 → 284,11 |
|---|
| |
| \secc FPGA function |
| |
| <<<<<<< .mine |
| Several tasks in separate FPGA blocks are performed by FPGA. In first FPGA prepares sampling clock for ADCdual01A modules by division of main local oscillator. This task is separate block in FPGA and runs asynchronously to other logic. Second block is SPI configuration module, which sends configuration words to ADC modules it is activated by opening of Xillybus interface file. Third block is main module, which resolve ADC - PC communication itself it communicates via PCIe, collect data from ADC hardware and creates data packet \ref[xillybus-interface]. Last block is activated after ADC configuration. |
| ======= |
| Several tasks in our design are performed by FPGA. Firstly, FPGA prepares a sampling clock for ADCdual01A modules this task is separate block in FPGA and runs asynchronously compared to other logical circuits. The second block is a SPI configuration module, which sends the content of configuration registers to the ADC modules after opening of Xillybus interface file. The third block represents the main module which resolves ADC - PC communication itself. The last block is activated after ADC configuration. |
| >>>>>>> .r1124 |
| |
| Communication over PCIe is managed by proprietary IP Core and Xillybus driver, which transfers data from FPGA registers to host PC. Data appear in system device file named "/dev/xillybus_data2_r" on the host computer. Binary data which appear in this file after its opening are described in the table below \ref[xillybus-interface]. |
| |
| 293,9 → 302,12 |
|---|
| \caption/t System device "/dev/xillybus_data2_r" data format |
| \endinsert |
| |
| Detailed description of FPGA function can be found in \cite[fpga-middleware] |
| Data packet block which is carried on PCI Express is described by table \ref[xillybus-interface]. The data packet consist several 32bit words. First word contain FRAME number and it is filled by saw signal for now, which increments with every data packet transmission. Following data words contains samples from ADCs for first and second channel. Samples from every channel is transmitted in pairs of two samples. Number of ADC channels is expandable according to number of physically connected channels. An CRC word may be added in future at end of transmission packet for data integrity validation. |
| |
| FRAME word at beginning of data packet filled with incrementing and overflowing saw signal is used for ensure that no data samples ale lost during data transfers from FPGA. FRAME signal may be used in future for pairing the ADC samples data packet with another data packet in future. This new additional data packet should carry meta-data information about sample time jitter, current accuracy of local oscillator frequency etc. |
| |
| Detailed description of currently implemented FPGA functions can be found in separate paper \cite[fpga-middleware]. HDL source codes for FPGA at state which was used are included on enclosed CD. Future development versions are publicly available from MLAB sources repository. |
| |
| \secc Data reading and recording |
| |
| In order to read the data stream from the ADC drive, we use Gnuradio software. Gnuradio suite consists of gnuradio-companion which is a graphical tool for creating signal-flow graphs and generating flow-graph source code. This tool was used to create a basic RAW data grabber to record and interactively view the data stream output from ADC modules. |