Pierre and I got the CERN proxy setup for the Darkside prototype.

Using your CERN single-sign-on identity, you should be able to login to this page

https://m-darkside.web.cern.ch/

and see our normal MIDAS webpage.

The CERN server is proxying the port 80 on ds-proto-daq.  You can also see all the other services through the
same page:

elog:
https://m-darkside.web.cern.ch/elog/DS+Prototype/

chronobox:
https://m-darkside.web.cern.ch/chronobox/

js-root:
https://m-darkside.web.cern.ch/rootana/


_________________________________

Technical details

1) We followed these instructions for creating a SSO-proxy:

https://cern.service-now.com/service-portal/article.do?n=KB0005442

We pointed the proxy to port 80 on ds-proto-daq

2) On ds-proto-daq, we needed to poke a hole through the firewall for port 80:


firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="188.184.28.139/32" port
protocol="tcp" port="80" accept"
firewall-cmd --reload

[root@ds-proto-daq ~]# firewall-cmd --list-all
public (active)
...
	rule family="ipv4" source address="188.184.28.139/32" port port="80" protocol="tcp" accept

This firewall rule is pointing to some particular IP that seems to be the proxy side of the server:

[root@ds-proto-daq ~]# host 188.184.28.139
139.28.184.188.in-addr.arpa domain name pointer oostandardprod-7b34bdf1f3.cern.ch.

It is not clear if this particular IP will be stable in long term.

3) We needed to modify mhttpd so it would serve content to hosts other than localhost.  So changed mhttpd
command from 

mhttpd -a localhost -D

to 

mhttpd -D

The last elog didn't go out cleanly.  Modified the elogd.cfg to point to the proxy.  Try again.

Notes on day:

1) Fixed the problem with the network interfaces.  Now the computer boots with the correct network configuration; outside world visible 
and private network on.

2) fan tray on VME crate seemed to be broken.  Got another VME crate from pool and installed it.  This VME crate seems to be working 
well. 

3) Recommissioned the DAQ setup.  Found a couple small bugs related to the V1725 self-trigger logic.  Fixed those and the V1725 self-
triggers seem to be working correctly.

4) Tried to install new CDM from TRIUMF (with ssh access), but clocks didn't stay synchronized.  Will bring module back to TRIUMF.

5) Added some code to V1725 frontend for clearing out the ZMQ buffers of extra events at the end of the run.  This is to protect 
against the case where the chronobox is triggering too fast for the V1725s.

Several points:

1) Gave a series of tutorial on DAQ to DS people yesterday and today.  Got a bunch of feedback, which I will pass on when I'm back at TRIUMF.

2) The computer ds-proto-daq was offline when I got in to lab this morning.  Hmm, not clear what is wrong with computer.  Didn't happen the 
first day.  Maybe another power blip?  Maybe we need a UPS for this DAQ machine, to protect it from power blips.

3) Using instructions from Luke, reconfigured the CDM to use the clock from the chronobox. 

4) Added scripts for putting the chronobox and the CDM into a sensible state.  Scripts are 

/home/dsproto/online/dsproto_daq/setup_chronobox.sh
/home/dsproto/online/dsproto_daq/setup_cdm.sh

The scripts need to be rerun whenever the chronobox or VME crate are power cycled.

5) Fix some bugs and added some new plots to online monitoring.  In particular, added a bunch of plots related to the chronobox data.

6) Found some problems with monitoring of chronobox trigger primitives, which I passed onto Bryerton.

I have done a couple longer tests of the DAQ setup.  The runs were done with a high trigger rate of ~60Hz, with
the V1725 asserting their busy to throttle the trigger.

I found that after a couple hours (~5hours) that we would no longer be getting ZMQ packets from the chronobox. 
 You can see this with error message like

15:29:04.092 2019/04/11 [feov1725MTI00,ERROR] [feoV1725.cxx:1033:read_trigger_event,ERROR] Error: did not
receive a ZMQ bank.  Stopping run.
10:26:37.158 2019/04/11 [mhttpd,INFO] Run #657 started

If I start a new run I am still missing ZMQ packets.  However, if I restart the frontend program (and hence
re-initialize the ZMQ link), then the chronobox does start sending triggers again.  So it may be more of a
problem with the ZMQ setup in the MIDAS frontend.  Needs more investigation.

Darkside people seem to be doing some tests at CERN next week.  It looks like they aren't going to use our DAQ (they will I think just use CAEN tools).  But we took 
opportunity to make TRIUMF DAQ work again.  There was a couple issues

1) I needed to inver the busy signal for input 0 to the chronobox in order to get any triggers out of the system.  Not sure why; I don't think I had to do that before.  But it is 
working now.

2) The CERN web proxy was not initially working.  Somehow I 'fixed' it by changing the proxy configuration variable of SERVICE_NAME from 'midas' to 'dummy'.  I don't 
understand why that fixed it.  But now you can see the DAQ here:

https://m-darkside.web.cern.ch/chronobox/

The DAQ seems to be ready for tests with proto-0 tomorrow.  

I had to invert the first busy input in order to get chronobox to produce triggers.  I modified the setup script

setup_chronobox.sh

so this is now the default.  Not sure why necessary.

Another interesting fact; it seems the chronobox only asks for a DHCP IP when it first boots.  I think that
chronobox and ds-proto-daq were rebooted at the same time; ds-proto-daq dhcp server was probably not running
when chronobox asked for IP.  chronobox got IP fine when it was power cycled.

Crew: Edgar Sanchez Garcia, Sam Hill, Simone Stracka, Tom Thorpe, Yi Wang

Morning: Tested photocurrents for MB2 on tabletop using SMU and Keithley switching matrix.  All measured between 1-10 microAmps.

Afternoon: Tested PDU mounting procedure with TPC assembled and with the mockup.  No showstoppers. This will be the procedure for installation
With the mockup on the table we double checked the resistances.  Confirmed all values are as before cooling indicating no additional damage to the wire-bondings.

Crew: Edgar Sanchez Garcia, Sam Hill, Simone Stracka, Tom Thorpe, Yi Wang

Morning: Set up equipment for the IV curve measurements. We enable the interlock system in the SMU in order to go higher than 42 V (handle with care).

Afternoon:  Measuring IV curves.  Note there is a voltage drop in the LV cable that needs to be compensated for.  

Chan 1 is set at +3.3V drawing 1.46A; sense line reads +2.39V
Chan 2 is set at -3.3V drawing 1.46A; sense line reads -2.39V
IV curves are scanning from 0-80V in 200mV steps w/ 100uA limit: elog:45/2 , elog:45/3
The two runs are visible ( elog:45/1 for color code and mapping )
Compare to elog:45/4 , obtained when probing individual tiles under more controlled light conditions

People: Edgar, Simone

Installed differential to single-ended converters in VME crate, and turned crate back on. elog:47/2

Channel count in the converters is 0 to 15 starting from the low end. Wired according to:
left board, channels 0..15 = PDM slot 1..16
right board, channels 0..8 = PDM slot 17..25

N.B. V1725 board #0 logic level is set to TTL (boards #1, #2, #3 to NIM) elog:47/3

In ODB / Equipment / V1725_Data00 / Settings /  set Channel Mask to 0x1555 for all V1725 boards (enables channels 0 2 4 6 8 10 12)
(board 0 receives 7 PDM inputs, boards 1,2,3 receive 6 PDM each)

In https://ds-proto-daq.cern.ch/chronobox/ , set Enable Channel [ch_enable] = 0x3F3F3F7F , Channel Assignment [ch_assign] = 0x00393340.
9 central PDM's assigned to "top" group, external PDM's assigned to "bottom" group elog:47/1

Wiki instructions with the script to get the CDM back into a sensible state seem outdated. The variables seem fine, though ... 
[dsproto@ds-proto-daq ~]$ esper-tool read 192.168.1.5 cdm ext_clk
[49999632]
[dsproto@ds-proto-daq ~]$ esper-tool read 192.168.1.5 lmk pll1_ld
[1]
[dsproto@ds-proto-daq ~]$ esper-tool read 192.168.1.5 lmk pll2_ld
[1]

We need python3 package tkinter installed on ds-proto-daq in order to run the steering module GUI. 

I don't have root privileges.

We have tried to connect the steering module to the DAQ pc via Luigi's USB-Ethernet adapter. 

That does not appear to work. We'll receive another USB-Ethernet adapter tomorrow, and it should be configured to have a static IP address : 192.168.121.1 and NetMask 255.255.255.0
For the time being, we are using Tom's laptop. 

The HV board from Naples did not turn on: Yury gave it to the CAEN guys to check and/or bring back for replacement.

Yi and Luigi rented a new HV module (A1519). The HV module (A1520P) we used for tests of the I-V script is also present in the Mainframe (see below).
The first 24 channels (slots 0 and 1) are therefore HV. If the new HV does not show up on time we'll try and adapt the cables to work with A1519.

The A2517A module is LV. This is currently operated from the DAQ pc using CAEN_HVPSS_ChannelsController.jnlp (located in the Desktop/SteeringModule folder).
The three low voltage channels (0,1,2) should be turned on at the same time by setting Pw = ON.

Settings: 

Channel 0 and 1: I0Set = 2.0 A , V0Set = 2.5 V ,  UNVThr = 0 V, OVVThr = 3.0 V, Intck = Disabled
IMon = 1.44 (this depends on the illumination) , VMon = 2.48 V , VCon = 2.79 V

Channel 2: I0Set = 1.0 A , V0Set = 5.0 V ,  UNVThr = 0 V, OVVThr = 5.5 V, Intck = Disabled
IMon = 0.07 , VMon = 4.998 V , VCon = 5.02 V

In case the channels trip they cannot be ramped back up unless the alarms are cleared.

People:  Simone, Sam, Tom, Edgar

Tested IV curve python script that runs using MIDAS. Measured resistance consistant with resistor specification. elog:46/1

The resistance is attached to ch11, and we called the script (channels go from 0 to 24):
chnum=11 ; python3 iv_curve.py --no_steering --caen_chan $chnum --stop_v 3 --step_v 0.001 --filename ivcurve_`date +%s`_$chnum.txt 

Issues:

No readout of trip condiotion therefore python script overides power supply trip and attemps to bring back voltage to next value. Potentially dangerous. Script only terminates with the final voltage value. See attached plot elog:46/1

Script terminated with error message:

Traceback (most recent call last):
  File "iv_curve.py", line 287, in <module>
    iv.run()
  File "iv_curve.py", line 269, in run
    self.set_caen_and_wait_for_readback(voltage)
  File "iv_curve.py", line 223, in set_caen_and_wait_for_readback
    raise RuntimeError("CAEN HV didn't respond to request to set voltage to %s; latest readback voltage is %s" % (voltage, rdb))
RuntimeError: CAEN HV didn't respond to request to set voltage to 2.976; latest readback voltage is 0.052000001072883606

Run 1016

Gas pocket: ON (thinkness 7mm)

Fields: Off

Threshold: 1000 ADC below baseline

Trace length: 200us total, 100 us pre-trigger

Threshold extend:  5x16 = 80ns

TPC Pressure: 201 mbarg

SiPM HV: 65 V

Number of Events: 10K

Run 1017

Gas pocket: ON (thinkness 7mm)

Fields:  drift 200 V/cm, extraction 2.8 kV/cm

Threshold: 1000 ADC below baseline

Trace length: 200us total, 100 us pre-trigger

Threshold extend:  5x16 = 80ns

TPC Pressure: 201 mbarg

SiPM HV: 65 V

Number of Events: 10K

Run 1018

Gas pocket: ON (thinkness 7mm)

Fields:  drift 200 V/cm, extraction 3.78 kV/cm

Threshold: 1000 ADC below baseline

Trace length: 200us total, 100 us pre-trigger

Threshold extend:  5x16 = 80ns

TPC Pressure: 217 mbarg

SiPM HV: 65 V

Number of Events: 10K

Run 1019

Gas pocket: ON (thinkness 7mm)

Fields:  drift 500 V/cm, extraction 3.78 kV/cm

Threshold: 1000 ADC below baseline

Trace length: 200us total, 100 us pre-trigger

Threshold extend:  5x16 = 80ns

TPC Pressure: 240

SiPM HV: 65 V

Number of Events: 10K

Run 1020 Laser Run

Chronobox: External trigger Enable

Chronobox: Enable Channel [ch_enable]: 0x00000000

Laser Intensity: MAX

Threshold: external clock (laser NIM), laser set to 1kHz

Trace length: 12us, 5us pre-trigger

SIPM HV: 65V

Comment: Recieved this error message but run seemed to continue ok and data appeared to be written: 19:56:04.839 2019/11/09 [feov1725MTI00,ERROR] [feoV1725.cxx:1048:read_trigger_event,ERROR] Error: did not receive a ZMQ bank after 100.000000 ms. Stopping run.   â•³

Run 1021 Laser Run

Chronobox: External trigger Enable

Chronobox: Enable Channel [ch_enable]: 0x00000000

Laser Intensity: MIN

Threshold: external clock (laser NIM), laser set to 1kHz

Trace length: 12us, 5us pre-trigger

SIPM HV: 65V

17:30

Took a set of IV curves using 0.5V steps from 30-50V and 0.05 steps above 50V.

Curves all look as expected, although reducing the step size makes the breakdown voltage harder to see on the plots.

Will use 0.02V steps in future.

Data can be found on dsproto machine at: /home/dsproto/online/dsproto_sy4527/ivdata_191111

Summary plot attached to this entry.

19:15

Refilled outer dewar for 10 mins

Attemted the following run, and it started but the rate was shoing 0 and it got stuck on 5 events

Was unable to connect to Chronobox webpage even after turning VME crate off and on.

Abandoned run and will attempt to fix in the morning.

Run 1073

Gas pocket: ON (thickness unknown)

Fields:  drift 200 V/cm, extraction 2.8 kV/cm

Threshold: 1000 ADC below baseline

Trace length: 200us total, 100 us pre-trigger

Threshold extend:  5x16 = 80ns

TPC Pressure:  252mbarg

SiPM HV: 65 V

Number of Events: 10K

Temp at start: 

A: 242.7K    B: 278.16K    C: 86.4K    D: 240.89K

Temp at End:

A: 248.87K    B: 278.24K    C: 86.37K    D: 247.75K

Took iv curves, data is on DAQ PC here: /home/dsproto/online/dsproto_sy4527/ivdata_191120

Laser Intensity: MAX

Threshold: external clock (laser NIM), laser set to 1kHz

Trace length: 12us, 5us pre-trigger

SIPM HV: 65V

Laser Intensity: MAX

Threshold: external clock (laser NIM), laser set to 1kHz

Trace length: 30us, 5us pre-trigger

SIPM HV: 65V