I modified the analyzer so that it shows data from all four modules.

I find that there is evidence of corruption of the V1725 ADC data on a couple channels.   You can see an example of a waveform with corruption in the attachment.  You see that there seems to be a bunch of fluctuations of exactly 16 
ADC counts, which seems unphysical.  So far I see this problem in these channels:

module 0, ch 0
module 0, ch 1
module 0, ch 13
module 1, ch 4
module 3, ch 6

I saw corruption similar to this on the V1730 readout.  CAEN does have a scheme for ADC calibration (by poking a register), which worked well for the V1730.  I thought that I implemented this ADC calibration for the V1725, but I might 
have messed it.  I'll look at this again.

A little progress on a couple fronts:

1) We got the baseline DAC setting working correctly from both the CAEN command line tool and from the MIDAS
frontend.  Needed to add some extra sleep before the DAC settings were modified.

2) We figured out how to set the threshold for self-triggering on each channel. It is actually register 0x1n60
for this firmware.  The threshold is defined relative to the online calculated baseline. The threshold is
settable from the ODB.  However, the polarity of the triggering is currently hard-coded to be negative (bit 6 of
register 0x1n64).

3) We could confirm that the LVDS outputs from the V1725 appeared and disappeared as expected when the
self-trigger threshold was changed.

4) We added some basic documentation in Markdown here:

https://bitbucket.org/ttriumfdaq/dsproto_daq/src/master/

It turns out that the ZLE V1725 firmware we are using only supports reading out up to 4000 samples per channel.
 We need 80000 samples to readout 200us, which is requirement.

So we switch the V1725s to use the backup firmware on the board, which is the standard waveform firmware. 
Firmware version is 17200410.

Expected data size for event with 200us of data: 200000ns * 1/4 ns/sample * 16 ch * 4 boards * 2 bytes/sample =
 6.4MB per event

Measured max event rate of 60Hz with 385MB/s with 200us readout.

Needed to increase max event size, set buffer organization to 6 and set almost full to 32 in order to
accommodate the larger event size.

Changed some registers for different firmware:
- V1725_SELFTRIGGER_LOGIC

More tests needed

The V1725s have been changed to have board config 0x50 rather than 0x10. This means that they will now trigger on the leading edge of the pulse rather than the tailing edge. There is now much less jitter in the location of pulses in the digitized waveforms.

09:15:33.955 2019/11/12 [feov1725MTI00,ERROR] [feoV1725.cxx:663:link_thread,ERROR] Exiting thread 3 with error

It looks like there's a memory leak in the high voltage driver, and we were running out of memory (using a lot of swap). That *may* be related to the recent instability, but I'm not sure.

If things are more stable now, then the memory leak was probably the cause. If not, then we should try stopping the V1725 program, power-cycling the VME crate, then starting the V1725 program.


Yi: restarted the VME crate twice, now it seems like smooth.

Changed custom size to 500 (20 us), tested ok run 681