Subversion Repositories svnkaklik

Rev

Rev 1149 | Rev 1151 | Go to most recent revision | Details | Compare with Previous | Last modification | View Log

Rev Author Line No. Line
1133 hlavac 1
\chap Trial version of the receiver, design and implementation
1073 kaklik 2
 
1149 hlavac 3
The whole design of the radioastronomic receiver digitization unit is meant to be used in a wide range of applications and tasks related to digitization of a signal. A good illustrating problem for its use is the signal digitization from multiple antenna arrays.
1083 kaklik 4
 
1132 kaklik 5
\midinsert
6
\clabel[expected-block-schematic]{Expected system block schematic}
7
\picw=\pdfpagewidth \setbox0=\hbox{\inspic ./img/Coherent_UHF_SDR_receiver.png }
8
\par\nobreak \vskip\wd0 \vskip-\ht0
9
\centerline {\kern\ht0 \pdfsave\pdfrotate{90}\rlap{\box0}\pdfrestore}
10
\caption/f Expected realisation of signal digitalisation unit.
11
\endinsert
12
 
1073 kaklik 13
\sec Required parameters
14
 
1149 hlavac 15
We require the following technical parameters in order to overcome the existing digitization units solutions. Primarily, we need a wide a dynamical range and a high third-order intercept point (IP3\glos{IP3}{Third-order intercept point}). The receiver must accept signals with the wide dynamics because a typical radioastronomical signal is a weak signal covered by a strong man-made noise or other undesired noises as lighting, Sun emissions, etc.
1073 kaklik 16
 
1149 hlavac 17
\medskip
18
\noindent
19
The summary of other additional required parameters:
20
%
1083 kaklik 21
\begitems
1149 hlavac 22
  * Dynamic range better than 80 dB, see section \ref[dynamic-range-theory] for the explanation.
23
  * Phase stability between channels.
24
  * Low noise (all types).
25
  * Sampling jitter better than 100 metres.
26
  * Support for any number of receivers in the range of 1 to 8.
1083 kaklik 27
\enditems
28
 
1149 hlavac 29
\noindent
30
We analyze several of the parameters more in detail in the sequel.
1125 kaklik 31
 
1098 kaklik 32
\sec Sampling frequency
1133 hlavac 33
 
1150 hlavac 34
The sampling frequency has not been a limiting factor in the trial version. Generally, the sampling frequency is mostly limited by the sampling frequencies of the analog-to-digital conversion chips available on the market and by the interface bandwidth. The combination of required parameters -- dynamic range needing 16 bits  at least and a minimum sampling frequency of 1 Mega-Samples Per Second (MSPS\glos{MSPS}{Mega-Samples Per Second}) -- leads to the need of the high-end ADC chips. However, they do not support such low sampling frequencies at all. Their minimum sampling frequency is 5$\ $MSPS.
1083 kaklik 35
 
1150 hlavac 36
We calculated the minimal data bandwidth data rate for eight receivers, 2~bytes per sample and 5$\ $MSPS as $8 \cdot 2 \cdot 5\cdot 10^6 = 80\ $MB/s. Such data rate is at the limit of the actual writing speed of a classical hard disk drive (HDD\glos{HDD}{Hard disk drive}) and it is almost a double the real bandwidth of USB~2.0\glos{USB 2.0}{Universal Serial Bus version 2.0} interface. As a result of these facts, we must use a faster interface. Such a faster interface is especially needed in cases in which we require faster sampling rates than ADC's minimal 5$\ $MSPS sample rate.
1135 kaklik 37
The most perspective interface for use in our type of application is USB 3.0 or PCI Express interface. However, USB 3.0 is a relatively new technology without good development tools currently available. We have used PCI Express \glos{PCI Express}{Peripheral Component Interconnect Express}  interface as the simplest and the most reliable solution.
1133 hlavac 38
 
1075 kaklik 39
\sec System scalability
1073 kaklik 40
 
1134 fluktuacia 41
For analogue channels' scalability, special parameters of ADC modules are required. Ideally, there should be a separate output for each analogue channel in ADC module. ADC module must also have separate outputs for frames and data output clocks. These parameters allow for conduction at relatively low digital data rates. As a result, the digital signal can be conducted even through long wires. Modular concept allows a separation from central logical unit which supports optimization of number analogue channels.
1073 kaklik 42
 
1135 kaklik 43
Clock and data signals will be then handled distinctively in our modular scalable design. Selected ADC chips are guaranteed to have defined clock skew between the sampling and data output clocks. This allows taking data and frame clocks from the first ADC module only. The rest of the data and frame clocks from other ADC modules can be measured for diagnostic purposes (failure detection, jitter measurement etc.), but these redundant signals are not used for data sampling. If more robustness is required in the final application, DCO \glos{DCO}{Data Clock Output} and FR signals may be collected from other modules and routed through an voting logic which will correct possible signal defects.
1075 kaklik 44
 
1133 hlavac 45
This system concept allows for scalability, that is technically limited by a number of differential signals on host side and its computational power.  There is another advantage of scalable data acquisition system -- an economic one. Observatories or end users can make a choice of how much money are they willing to spent on radioastronomy receiver system. This freedom of choice is especially useful for science sites without previous experience in radioastronomy observations.
1075 kaklik 46
 
1134 fluktuacia 47
\secc Differential signalling
1075 kaklik 48
 
1135 kaklik 49
The above mentioned concept of scalable design requires relatively long circuit traces between ADC and digital unit which captures the data and performs the computations. The long distance between the digital processing unit and the analog-to-digital conversion unit has an advantage in noise retention typically produced by digital circuits. Those digital circuits, such as FPGA \glos{FPGA}{Field-programmable gate array}, Ethernet or other flip-flops blocks and circuit traces, usually work at high frequencies and emit wide-band noise with relatively low power. In such cases any increase in a distance between the noise source and analog signal source increase S/N significantly. However, at the same time, a long distance brings problems with the digital signal transmission between ADC and computational unit. But this obstacle should be resolved more easily in free-space than on board routing. The high-quality differential signalling shielded cables should be used, such as massively produced and cheap SATA \glos{SATA}{Serial ATA} \glos{ATA}{AT Attachment} cables. This technology has two advantages over PCB \glos{PCB}{printed circuit board} signal routing. First, it can use twisted pair of wires for leak inductance suppression in signal path and second, the twisted pair may additionally be shielded by uninterrupted metal foil.
1075 kaklik 50
 
1076 kaklik 51
\secc Phase matching
52
 
1134 fluktuacia 53
For multiple antenna radioastronomy projects, system phase stability is a mandatory condition. It allows a precise, high resolution imaging of objects, increases signal to noise ratios in several observation methods and allows the use of advanced algorithms for signal processing.
1076 kaklik 54
 
1135 kaklik 55
High phase stability in our scalable design is achieved through centralized frequency generation  and distribution with multi-output LVPECL \glos{LVPECL}{Low Voltage Emitter-coupled logic} hubs (CLKHUB02A), that have equiphased outputs for multiple devices. LVPECL logic is used on every system critical clock signal distribution hub. LVPECL logic has an advantage over LVDS \glos{LVDS}{Low-voltage differential signaling} in signal integrity robustness. LVPECL uses higher logical levels and higher signalling currents. Power consumption of LVPECL logic is nearly constant over the operating frequency range due to the use of bipolar transistors. This minimizes voltage glitches which are typical for CMOS \glos{CMOS}{Complementary metal–oxide–semiconductor } logic. One drawback of its parameters is a high power consumption of LVPECL logic which easily reaches tens of milliamperes per device.
1076 kaklik 56
 
1133 hlavac 57
This design ensures that all system devices have access to the defined phase and known frequency.
1076 kaklik 58
 
1073 kaklik 59
\sec System description
60
 
1134 fluktuacia 61
This section deals with the description of the trial version based on Xilinx ML605 development board \ref[ML605-development-board]. The board had been used in a previous project and has not been used since then, but the FPGA parameters are more than sufficient of what we need for fast data acquisition system.
1073 kaklik 62
 
1133 hlavac 63
\secc Frequency synthesis
1075 kaklik 64
 
1135 kaklik 65
We have used a centralized topology as a basis for frequency synthesis. One precise high-frequency and low-jitter digital oscillator has been used \cite[MLAB-GPSDO], while other working frequencies have been derived from it by the division of its signal. This central oscillator has a software defined GPS \glos{GPS}{Global Positioning System}  disciplined control loop for frequency stabilization.\fnote{SDGPSDO design has been developed in parallel to this diploma thesis as a related project, but it is not explicitly required by the thesis itself and thus it is described in a separate document}
1134 fluktuacia 66
We have used new methods of software frequency monitoring and compensation in order to meet modern requirements on radioastronomy equipment which need precise frequency and phase stability over a wide baseline scales for effective radioastronomy imaging.
1133 hlavac 67
 
1135 kaklik 68
The GPSDO device consists of Si570 chip with LVPECL output. Phase jitter of GPSDO \glos{GPSDO}{GPS disciplined oscillator} is determined mainly by Si570 phase noise. Parameters of the Si570 are summarized in the following table \ref[LO-noise] (source \cite[si570-chip] ).
1112 kaklik 69
 
1134 fluktuacia 70
The GPSDO design, that is included in data acquisition system, has special feature -- it generates time marks for a precise time-stamping of the received signal. Timestamps are created by disabling the local oscillator's outputs, connected to SDRX01B receivers, for 100 us.  As result, a rectangular click in the ADC input signal is created which appears as a horizontal line in spectrogram.
71
Timestamps should be seen in image \ref[meteor-reflection] (above and below the meteor reflection).
1112 kaklik 72
 
1149 hlavac 73
Time-marking should be improved in future by digitization of GPS signal received by antenna on observational station. Following that, the GPS signal can be directly sampled by a dedicated receiver and one separate ADC module. Datafile then consists of samples from channels of radio-astronomy receivers along with the GPS signal containing precise time information.
1126 kaklik 74
 
75
 
1134 fluktuacia 76
\midinsert \clabel[LO-noise]{Phase noise of the local oscillator}
1113 kaklik 77
\ctable{lcc}{
1114 kaklik 78
	&	 \multispan2 \hfil Phase Noise [dBc/Hz] \hfil 		\cr
79
Offset Frequency	&	$F_{out}$ 156.25 MHz	& $F_{out}$ 622.08 MHz \cr
1113 kaklik 80
100 [Hz]	&	–105	&	–97 \cr
81
1 [kHz]	&	–122	&	–107 \cr
82
10 [kHz]	&	–128	&	–116 \cr
83
100 [kHz]	&	–135	&	–121 \cr
84
1 [MHz]	&	–144	&	–134 \cr
85
10 [MHz]	&	–147	&	–146 \cr
86
100 [MHz]	&	n/a	&	–148 \cr
87
}
1134 fluktuacia 88
\caption/t Phase noise of the used Silicon Laboratories Si570 chip. Offset frequency is measured from carrier frequency. Values shown in the table are given for two different carrier frequencies.
1113 kaklik 89
\endinsert
1112 kaklik 90
 
1134 fluktuacia 91
Every ADC module will be directly connected to CLKHUB02A module which takes sampling clock signal delivered by FPGA from the main local oscillator.  This signal should use high quality differential signalling cable -- we should use SATA cable for this purpose. FPGA may slightly affect the clock signal quality by adding a noise, but it has a negligible effect on the application where developed system will be used.
1112 kaklik 92
 
1098 kaklik 93
 
1145 kaklik 94
\label[signal-cables] \secc Signal cable connectors
1073 kaklik 95
 
1126 kaklik 96
 
1145 kaklik 97
 
1133 hlavac 98
Several widely used and commercially easily accessible differential connectors were considered to be use in our design.
1073 kaklik 99
 
100
\begitems
1077 kaklik 101
* HDMI % [[http://en.wikipedia.org/wiki/Hdmi|HDMI]]</del>
102
* SATA  		%{http://en.wikipedia.org/wiki/Serial_attached_SCSI#Connectors|SAS]]/[[http://en.wikipedia.org/wiki/Serial_ATA|SATA]]
103
* DisplayPort 		%[[http://en.wikipedia.org/wiki/Display_port|DisplayPort]]</del>
104
* SAS/miniSAS
1073 kaklik 105
\enditems
106
 
1134 fluktuacia 107
At the end, MiniSAS connector was chosen as the best option to be used in connecting together multiple ADC modules. A transition between SATA and miniSAS is achieved by SAS to SATA adapter cable which is commonly used in servers to connect SAS controller to multiple SATA hard disc in RAID systems and thus is commercially easily available. It is compatible with existing SATA cabling systems and aggregates multiple SATA cables to a single connector. It also has SPI configuration lines which can be seen in the following picture \ref[img-miniSAS-cable] as standard pinheader connector.
108
The main drawback of miniSAS PCB connectors lies in the fact, that they are manufactured in SMT versions only. SMT design may eventually decrease the durability of the connector even if outer metal housing of connector is designed to be mounted using a standard through-hole mounting method.
1073 kaklik 109
 
1085 kaklik 110
\midinsert
1086 kaklik 111
\clabel[img-miniSAS-cable]{Used miniSAS cable}
1118 kaklik 112
\picw=5cm \cinspic ./img/miniSAS_SATA_cable.jpg
113
\caption/f An example of miniSAS cable similar to used.
1085 kaklik 114
\endinsert
115
 
1086 kaklik 116
\secc Signal integrity requirements
1126 kaklik 117
 
1105 kaklik 118
\label[diff-signaling]
1085 kaklik 119
 
1134 fluktuacia 120
We use ADC devices that have DATA clock frequency eight times higher than sampling frequency in a single line output mode, implying a 40 MHz output bit rate. This implies a $ 1/4 \cdot 10^7 = 25\ $ns time length of data bit, which is equivalent to 7.5m light path in free space. If we use copper PCB with FR4 substrate layer or coaxial/twinax cable, we could obtain velocity factor of 0.66 at worst condition. Then the light path for the same bit rate $t_s$ will be 4.95 m. Although we do not have any cables in the system with comparable lengths, the worst data bit skew described by data sheets of the used components is $0.3 \cdot t_s$, which is 1.485 m. Therefore the length matching is not critical in our current design operating on lowest sampling speed. Length matching may become critical in future versions with higher sampling rates, where the cable length must be matched. However SATA cabling technology is already prepared for that case and matched SATA cables are a standard merchandise.
1086 kaklik 121
 
1093 fluktuacia 122
\secc ADC modules design
1073 kaklik 123
 
1124 kaklik 124
\midinsert
1126 kaklik 125
\clabel[adcdual-preview]{Preview of designed ADCdual PCB}
1132 kaklik 126
\picw=10cm \cinspic ./img/ADCdual01A_Top_Big.JPG
127
\picw=10cm \cinspic ./img/ADCdual01A_Bottom_Big.JPG
1134 fluktuacia 128
\caption/f Realised PCB of ADCdual01A modules. Differential pairs routings are clearly visible.
1124 kaklik 129
\endinsert
1083 kaklik 130
 
131
\secc ADC selection
132
 
1134 fluktuacia 133
There exist several standard ADC signalling formats currently used in communication with FPGA.
1083 kaklik 134
 
135
\begitems
136
  * DDR LVDS
137
  * JEDEC 204B
138
  * JESD204A
139
  * Paralel LVDS
140
  * Serdes
141
  * serial LVDS
142
\enditems
143
 
1134 fluktuacia 144
As a result of our need to use the smallest number of cables possible, the choice fell on the serial LVDS format. Small number of differential pairs is an important parameter determining the construction complexity and reliability\cite[serial-lvds]. No many currently existing ADC devices have this kind of digital interface. An ultrasound AFE device chips seem to be ideal for this purpose -- the chip has integrated both front-end amplifiers and filters. It has a drawback though - it is incapable of handling differential input signal and has a relatively low dynamic range (as it consists only of 12bit ADC) and has many single ended ADC channels. Consequently, the scaling is possible only by a factor of 4 receivers (making 8 analogue single ended channels).
1083 kaklik 145
 
1134 fluktuacia 146
If we add a requirement of separate output for every analogue channel and a 16bit depth we find that there are only a few 2-Channel simultaneous sampling ADCs currently existing which meet these criteria. We have summarized those ADCs in the following table \ref[ADC-types]
1083 kaklik 147
 
1133 hlavac 148
\midinsert
1126 kaklik 149
\typosize[9/11] \def\tabiteml{ }\let\tabitemr=\tabiteml
150
\clabel[ADC-types]{Available ADC types}
1116 kaklik 151
\ctable{lccccccc}{
1086 kaklik 152
\hfil ADC Type & LTC2271 & LTC2190 & LTC2191 & LTC2192 & LTC2193 & LTC2194 & LTC2195 \cr
1126 kaklik 153
SNR [dB] & 84.1 & 77 & 77 & 77 & 76.8 & 76.8 & 76.8  \cr
154
SFDR [dB] & 99 & 90 & 90 & 90 & 90 & 90 & 90  \cr
155
S/H Bandwidth [MHz] & 200 & \multispan6 550 \strut \cr
156
Sampling rate [MSPS] & 20 & 25 & 40 & 65 & 80 &  105 & 125  \cr
157
Configuration & \multispan7 SPI \strut \cr
158
Package & \multispan7 \hfil 52-Lead (7mm $\times$ 8mm) QFN \hfil \strut \cr
1086 kaklik 159
}
1134 fluktuacia 160
\caption/t The summary of the currently available ADC types and theirs characteristics.
1086 kaklik 161
\endinsert
1083 kaklik 162
 
1134 fluktuacia 163
All parts in this category are compatible with one board layout. The main differences lay in the sampling frequency and signal to noise ratio, with the slowest having a maximum sampling frequency of 20 MHz. However, all of them have a minimal sampling frequency of 5 MSPS and all are configurable over a serial interface (SPI). SPI seems to be a standard interface used in high-end ADC chips made by the largest manufacturers (Analog Devices, Linear technology, Texas instruments, Maxim integrated..).  We have selected two slowest types for our evaluation design. Following that, a PCB for this part have been designed.
164
We have decided that ADCdual01A modules will have a standard MLAB construction layout with four mounting holes in corners aligned in defined raster of 400 mils.
1083 kaklik 165
 
1134 fluktuacia 166
Data serial data outputs of ADC modules should be connected directly by LVDS signalling levels conducted by SATA cables to FPGAs for the basic primary signal processing. The ADC chips used in the modules have a selectable bit width of data output bus and thus the output SATA connectors have signals arranged to contain a single bit from every ADC channel.  This creates a signal concept enabling a selection of proper bus bit-width according to the sampling rate (higher bus bit-width downgrades signalling speed and vice versa.)
1105 kaklik 167
 
1126 kaklik 168
In order to connect the above mentioned signalling layout, miniSAS to multiple SATA cable should be used as described in section \ref[signal-cables].
1105 kaklik 169
 
1134 fluktuacia 170
A KiCAD design suite had been chosen for PCB layout. However, the version, despite having integrated CERN Push \& Shove routing capability, is slightly unstable as it sometimes crushes due to an exception during routing. On the basis of these stability issues, the design had to be saved quite often. On the other hand, compared to commercially available solutions, such as MentorGraphics PADS or Cadence Orcad,  the Open-source KiCAD provides an acceptable option and it easily surpasses a widely used Eagle software.
1105 kaklik 171
 
1133 hlavac 172
As a part of work on the thesis, new PCB footprints for FMC, SATA, ADCs a and miniSAS connectors have been designed and were committed to KiCAD github library repository. They are now publicly available on the official KiCAD repository at GitHub.
1105 kaklik 173
 
1126 kaklik 174
ADCdual01A module has several digital data output formats. Difference between these modes lays in the number of differential pairs used.
1105 kaklik 175
 
176
\begitems
177
    * 1-lane mode
1114 kaklik 178
    * 2-lane mode
179
    * 4-lane mode
1105 kaklik 180
\enditems
181
 
1133 hlavac 182
All of the above-mentioned modes are supported by the module design. For the discussed data acquisition system, the 1-lane mode was selected. 1-lane mode allows a minimal number of differential pairs between ADCdual01A and FPGA. Digital signalling scheme used in 1-lane mode is shown in the following image \ref[1-line-out].
1105 kaklik 183
 
184
\midinsert
185
\clabel[1-line-out]{Single line ADC output signals}
186
\picw=15cm \cinspic ./img/ADC_single_line_output.png
1122 fluktuacia 187
\caption/f Digital signalling schema for 1-line ADC digital output mode.
1105 kaklik 188
\endinsert
189
 
1134 fluktuacia 190
ADCdual01A parameters can be set either by jumper setup (referred to as a parallel programming  in the device's data sheet) or by SPI interface. SPI interface has been chosen for our system, because of the parallel programming's lack of options (test pattern output setup for example).
1105 kaklik 191
 
1133 hlavac 192
Complete schematic diagram of ADCdual01A module board is included in the appendix.
1105 kaklik 193
 
194
 
1073 kaklik 195
\secc ADC modules interface
196
 
1124 kaklik 197
\midinsert
1132 kaklik 198
\picw=10cm \cinspic ./img/FMC2DIFF_Top_Big.JPG
199
\picw=10cm \cinspic ./img/FMC2DIFF_Bottom_Big.JPG
200
\caption/f Realised PCB of FMC2DIFF01A module.
1124 kaklik 201
\endinsert
202
 
1126 kaklik 203
Both of the ADCdual01A modules were connected to FPGA ML605 board trough FMC2DIFF01A adapter board. The design of this adapter expects the presence of FMC LPC connector on host side and the board is, at the same time, not compatible with MLAB. It is, on the other hand, designed to meet the VITA 57 standard specifications for boards which support region 1 and region 3. VITA 57 regions are explained in the picture \ref[VITA57-regions].
1133 hlavac 204
This industry standard guarantees the compatibility with other FPGA boards that have FMC LPC connectors for Mezzanine Card. Schematic diagram of designed adapter board is included in the appendix.
1073 kaklik 205
 
1133 hlavac 206
The primary purpose of the PCB is to enable the connection of ADC modules located outside the PC case with ML605 development board. (In PC box analog circuits cannot be realized without the use of massive RFI mitigation techniques).
1134 fluktuacia 207
Differential signalling connectors should be used for conducting digital signal over relatively long cables. The signal integrity sensitive links (clocks) are equipped with output driver and translator to LVPECL logic for better signal transmission quality.
1083 kaklik 208
 
1134 fluktuacia 209
LVPECL level signal connectors on FMC2DIFF01A board are dedicated to transmit the clock signals. We have selected  the SY55855V and SY55857L dual translators. Dual configuration in useful due to fact, that SATA cable contains two differential pairs.
1119 kaklik 210
 
1134 fluktuacia 211
The SY55855V is a fully differential, CML/PECL/LVPECL-to-LVDS translator. It achieves LVDS signalling up to 1.5Gbps, depending on the distance and the characteristics of the media and noise coupling sources.
212
LVDS is intended to drive 50 $\Omega$ impedance transmission line media such as PCB traces, backplanes, or cables. SY55855V inputs can be terminated with a single resistor between the true and the complement pins of a given input \cite[SY55855V-chip].
1119 kaklik 213
 
1125 kaklik 214
The SY55857L is a fully differential, high-speed dual translator optimized to accept any logic standard from single-ended TTL/CMOS to differential LVDS, HSTL, or CML and translate it to LVPECL. Translation is guaranteed for speeds up to 2.5Gbps (2.5GHz toggle frequency). The SY55857L does not internally terminate its inputs, as different interfacing standards have different termination requirements\cite[SY55857L-chip].
1119 kaklik 215
 
1134 fluktuacia 216
Inputs of both used chips are terminated accordingly to used logic. The LVDS input is terminated differentially by 100 $\Omega$ resistor between positive and negative inputs. PECL input is terminated by Thevenin resistor network. Thevenin termination method was selected as optimal one, due to the absence of a proper power voltage (1,3 V) for direct termination by 50 $\Omega$ resistors. Termination on FPGA side is realized directly by settings the proper digital logic type on input pins.
1119 kaklik 217
 
1073 kaklik 218
\midinsert
1126 kaklik 219
\clabel[ML605-development-board]{ML605 development board}
1073 kaklik 220
\picw=10cm \cinspic ./img/ML605-board.jpg
1094 fluktuacia 221
\caption/f FPGA ML605 development board.
1073 kaklik 222
\endinsert
223
 
1096 kaklik 224
\midinsert
225
\clabel[VITA57-regions]{VITA57 board geometry}
226
\picw=10cm \cinspic ./img/VITA57_regions.png
227
\caption/f Definition of VITA57 regions.
228
\endinsert
229
 
1137 kaklik 230
Three differential logic input/output, one PECL input and one PECL output SATA connectors and two miniSAS connectors are populated on this board.  This set of connectors allows a connection of any number of ADC modules within the range of 1 to 8. ADC data outputs should be connected to the miniSAS connectors, while other supporting signals should be routed directly to SATA connectors on adapter.
1116 kaklik 231
 
1136 fluktuacia 232
Lengths of differential pairs routed on PCB of the module are not matched between the pairs. Length variation of differential pairs is not critical in our design according to facts discussed in paragraph \ref[diff-signaling]. Nevertheless, signals within differential pairs themselves are matched for length. Internal signal trace length matching of differential pairs is mandatory in order to minimize jitter and avoid a dynamic logic hazard conditions on digital signals, that represents the worst scenario. Thus the clocks' signals are routed in the most precise way on all designed boards.
1104 kaklik 233
 
1125 kaklik 234
Signal configuration used in our trial design is described in the following tables \ref[minisas-interface], \ref[SPI-system] and \ref[clock-interconnections].
1076 kaklik 235
 
1073 kaklik 236
 
1125 kaklik 237
\midinsert \clabel[minisas-interface]{miniSAS differential pairs connections}
1121 kaklik 238
\ctable {cccc}
239
{
240
miniSAS	&	SATA pair	&	FMC signal	&	Used as	\cr
241
P0	&	1	&	LA03	&	 not used 	\cr
242
P0	&	2	&	LA04	&	 not used 	\cr
243
P1	&	1	&	LA08	&	 not used 	\cr
244
P1	&	2	&	LA07	&	 not used 	\cr
245
P2	&	1	&	LA16	&	ADC1  CH1 (LTC2190)	\cr
246
P2	&	2	&	LA11	&	ADC1  CH2 (LTC2190) 	\cr
247
P3	&	1	&	LA17	&	ADC2 CH1 (LTC2271)	\cr
248
P3	&	2	&	LA15	&	ADC2 CH2 (LTC2271)	\cr
249
}
1133 hlavac 250
\caption/t miniSAS (FMC2DIFF01A J7) signal connections between modules.
1121 kaklik 251
\endinsert
252
 
1116 kaklik 253
 
1125 kaklik 254
\midinsert \clabel[SPI-system]{SPI configuration interface connections}
1121 kaklik 255
\ctable {ccc}
256
{
257
SPI connection J7	&	FMC signal	&	Connected to	\cr
258
SAS-AUX1	 &	LA14\_N	&	SPI DOUT	\cr
259
SAS-AUX2	 &	LA14\_P	&	SPI CLK	\cr
260
SAS-AUX3	 &	LA12\_N	&	CE ADC1	\cr
261
SAS-AUX4	 &	LA12\_P	&	CE ADC2	\cr
262
SAS-AUX5	 &	LA13\_N	&	soldered to GND	\cr
263
SAS-AUX6	 &	LA13\_P	&	not used	\cr
264
SAS-AUX7	 &	LA09\_N	&	not used	\cr
265
SAS-AUX8	 &	LA09\_P	&	soldered to GND	\cr
266
}
1133 hlavac 267
\caption/t SPI system interconnections
1121 kaklik 268
\endinsert
269
 
1133 hlavac 270
SPI interface is used in an unusual way in this design. SPI Data outputs from ADCs are not connected anywhere and read back is not possible, thus the configuration written to registers in ADC module cannot be validated. We have not observed any problems with this system, but it may be a possible source of failures.
1121 kaklik 271
 
1126 kaklik 272
 
1125 kaklik 273
\midinsert \clabel[clock-interconnections]{System clock interconnections}
1121 kaklik 274
\ctable {lccc}
275
{
276
Signal	&	FMC signal	&	FMC2DIFF01A	&	ADCdual01A	\cr
277
DCO	&	CLK1\_M2C	&	J5-1	&	J13-1	\cr
278
FR	&	LA18\_CC	&	J10-1	&	J12-1	\cr
279
ENC	&	LA01\_CC	&	J2-1(PECL OUT)	&	J3-1	\cr
280
SDGPSDO01A LO	&	CLK0\_M2C	&	J3-1 (PECL IN)	&	N/A	\cr
281
}
1133 hlavac 282
\caption/t Clock system interconnections
1121 kaklik 283
\endinsert
284
 
1133 hlavac 285
\secc FPGA function
1116 kaklik 286
 
1136 fluktuacia 287
Several tasks in separate FPGA blocks are performed by FPGA. In the first block the FPGA prepares a sampling clock for ADCdual01A modules by dividing the signal from the main local oscillator. This task represents a separate block in FPGA and runs asynchronously to other logical circuits. Second block is a SPI configuration module, which sends configuration words to ADC modules and it is activated by opening of Xillybus interface file. Third block represents the main module, which resolves ADC - PC communication itself and it communicates via PCIe, collect data from ADC hardware and creates data packet \ref[xillybus-interface]. Last block is activated after the ADC is configurated via SPI.
1116 kaklik 288
 
1136 fluktuacia 289
Communication over PCIe is managed by proprietary IP Core and Xillybus driver, which transfers data from FPGA registers to host PC. Data appear in a system device file named  "/dev/xillybus_data2_r" on the host computer. Binary data which appear in this file after its opening are shown in the table below \ref[xillybus-interface].
1116 kaklik 290
 
1133 hlavac 291
\midinsert
1127 kaklik 292
\def\tabiteml{ }\let\tabitemr=\tabiteml
1126 kaklik 293
\clabel[xillybus-interface]{Grabber binary output format}
294
\ctable {lccccccccc}{
1127 kaklik 295
\hfil & \multispan9 \hfil 160bit packet \hfil \strut \crl \tskip4pt
296
Data name &  FRAME  & \multispan2 \hfil ADC1 CH1 \hfil & \multispan2 \hfil ADC1 CH2 \hfil & \multispan2  \hfil ADC2 CH1 \hfil & \multispan2 \hfil ADC2 CH2 \hfil \strut  \cr
1073 kaklik 297
Data type & uint32 & int16 & int16 & int16 & int16 & int16 & int16 & int16 & int16 \cr
298
Content & saw signal & $t1$ &  $t_{1+1}$ &  $t1$ &  $t_{1+1}$ &  $t1$ &  $t_{1+1}$ &  $t1$ &  $t_{1+1}$ \cr
299
}
300
\caption/t System device "/dev/xillybus_data2_r" data format
301
\endinsert
302
 
1136 fluktuacia 303
Data packet block which is carried on PCI Express isa  described in the table \ref[xillybus-interface]. The data packet consist of several 32bit words. The first word contains FRAME number and it is filled with saw signal for now, with incremental step taking place every data packet transmission. The following data words contain samples from ADCs' first and second channel. Samples from every channel are transmitted in pairs of two samples. Number of ADC channels is expandable according to the number of physically connected channels. An CRC word may possibly be added in the future to the end of the transmission packet for data integrity validation.
1073 kaklik 304
 
1136 fluktuacia 305
FRAME word at the beginning of data packet, now filled with incrementing and overflowing saw signal, is used to ensure that no data samples ale lost during the data transfers from FPGA. FRAME signal may be used in the future for pairing the ADC samples' data packet with another data packet. This new additional data packet should carry meta-data information about the sample time jitter, current accuracy of the local oscillator frequency etc.
1116 kaklik 306
 
1136 fluktuacia 307
Detailed description of the currently implemented FPGA functions can be found in a separate paper \cite[fpga-middleware]. HDL source codes for FPGA at a state in which it was used are included on the enclosed CD. More recent development versions are publicly available from MLAB sources repository.
1137 kaklik 308
Detailed description of currently implemented FPGA functions can be found in separate paper \cite[fpga-middleware]. HDL \glos{HDL}{Hardware description language} source codes for FPGA at state which was used are included on enclosed CD. Future development versions are publicly available from MLAB sources repository \cite[mlab-sdrx].
1125 kaklik 309
 
1133 hlavac 310
\secc Data reading and recording
1073 kaklik 311
 
1136 fluktuacia 312
In order to read the data stream from the ADC drive, we use Gnuradio software. Gnuradio suite consists of gnuradio-companion package which is a graphical tool for creating signal-flow graphs and generating Python flow-graph source code. This tool has been used to create a basic RAW data grabber to record and interactively view waterfall plots using the data streams output from ADC modules.
1073 kaklik 313
 
314
\midinsert
1127 kaklik 315
\clabel[grabber-flow-graph]{Gnuradio flow graph for signal grabbing}
316
\picw=\pdfpagewidth \setbox0=\hbox{\inspic ./img/screenshots/Grabber.grc.png }
317
\par\nobreak \vskip\wd0 \vskip-\ht0
318
\centerline {\kern\ht0 \pdfsave\pdfrotate{90}\rlap{\box0}\pdfrestore}
319
\caption/f The ADC recorder flow graph created in gnuradio-companion.
1073 kaklik 320
\endinsert
321
 
322
\midinsert
323
\picw=15cm \cinspic ./img/screenshots/Grabber_running.png
1094 fluktuacia 324
\caption/f User interface window of a running ADC grabber.
1073 kaklik 325
\endinsert
326
 
1136 fluktuacia 327
The interactive grabber-viewer user interface shows live oscilloscope-like time-value display for all data channels and live time-frequency scrolling display (a waterfall view) for displaying the frequency components of the grabbed signal. Signal is grabbed to file with exactly the same format, as described in the table \ref[xillybus-interface].
1073 kaklik 328
 
1116 kaklik 329
 
330