Firmware

DAQ cycle

Basic function of Kalliope is multi-hit, 1 ns timing resolution TDC with Pulse- or DC-mode data acquisition. The NIM-in trigger serves as the common start of the TDC. Kalliope also controls Amplifier-Shaper-Discriminator (ASD) characteristics of the front-end analog board by the analog to digital converter (DAC) parameters, which are stored in the FPGA memory region, called as RBCP registers.

Since Kalliope was originally developed for time recording of J-PARC MLF pulsed muon measurements, the original firmware was designed for the time structure of the pulsed muon beam and the corresponding decay positron hits: TDCstart = NIM-in trigger rate is low (e.g. 25Hz = 40 ms repetion), the hits may be high-rate but they seize in a short time (e.g. at most 64 µs), and there are plenty of time to transfer data to the DAQ-PC (40ms - 64µs ~ almost 40 ms). The design of DAQ cycle for Pulse-mode firmware:

  • TDCstart - multiple hit recording (max. 1000hits x 32ch) - transfer of data to PC after a fixed TDC window (DELAY=64µs default).

As the application of Kalliope extends to DC muon measurements at Osaka University RCNP MuSIC facility, and neutron time of flight (TOF) measurements at J-PARC MLF (at BL10, neutron imaging), DC-mode firmware has been developed. In DC muon measurements, the TDCstart happens at random, and the number of hits are only a few in ~10µs after the TDCstart. In neutron TOF measurements, TDCstart is in 25Hz, but the hits continues to 40ms. The design of DAQ cycle for DC-mode firmware is:

  • TDCstart - hit identification and transfer to PC - until next TDCstart comes and resets time.

At TRIUMF, the DC-mode firmware has been successfully tested up to ~100 kcps TDCstart rate, which is the practical maximum for the DC muon measurements.

SPI control patterns

Independent to the DAQ cycle, the synchronous serial interface (SPI) control of the analog board depends on the ASIC type, and the firmware becomes different accordingly. Also, NIM-TDC module uses the same SPI signal lines for NIM-IN/OUT channels. As of year 2021, there are three ASIC types (Volume2012, Volume2014, FGATI) and one NIM-TDC, yielding four possible branching types of firmware. Together with the DAQ cycle, the firmware types may branch to 2 (Pulse vs. DC) x 4 (Volume2012/2014/FGATI or NIM-TDC) = 8 types . For the Pulse-mode firmware, there is no support for FGATI (as of year 2021), and for DC-mode firmware, Volume2014 (edition=03) covers FGATI. The resulting firmware types are 6 in total.

Version history

The version numbers (YY.MM.DD-Ed) are written in the VER field of the (RBCP) register map in 4 byte data.

Version history of Pulse-mode firmware is listed below. The versions are specified by the date, and the edition number (Ed) is always 01. As of year 2021, FGATI may not be controled in any of the Pulse-mode firmware.

Pulse-mode Versions NIM-TDC 2012 2014 FGATI memo (Uchida->Kojima->BBT)
13.05.21- 01 Kojima final version for Volume2012
13.05.29(14.04.09)- 01 Kojima final version for NIM-TDC
16.01.14(16.02.08)- 01 Kojima final version for Volume2014
16.05.11- 01 BBT adjustments
16.05.16- 01 X final version for Volume2014
16.05.18- 01 X final version for Volume2012
16.08.22- 01 X final version for NIM-TDC

Version history of DC-mode firmware is listed below. A full discription may be found in Kalliope_DC manual [1]. The edition number (Ed) specifies the SPI control type (ASICs and NIM-TDC). FGATI (a new ASIC under development) may be controled by the Ed=03 (Volume2014) firmware.

DC-mode Versions NIM-TDC 2012 2014 FGATI memo (BBT designed)
16.06.24- 01 DC without time over threshold (TOT).
16.11.09- 01 02 03 X DC-TOT first version
16.11.14- 01 02* 03* X final version for NIM-TDC & Volume2012
18.12.21- 03 X Attempt to include FGATI in 03 (in vain)
19.02.18- 03 03 Ed=03 became compatible with FGATI
19.02.19- 03 03 final version for Volume2014 & FGATI

* WSL-Ubuntu returns an error on kc_ip command, but is working.

Behaviors of the firmware

The DAQ cycle of the firmware is described below more in detail.

Pulse-mode

  • NIM-in (TDCstart) negative leading edge reset and start time recording.
  • Negative leading edge timing of hits (LVTTL 32ch) are registered and stored in the memory.
    • If the number of hits exceeds the buffer (1000 hits / channel / cycle), Ch-full bit is set.
  • After the DELAYtime (see register map), TDC time window closes and TCP data-transfer starts.
    • If another TDCstart comes before the DELAYtime, the time is reset and data so far are discarded.
    • If TDCstart comes during the data transfer, Multi-start-error bit is set, implying a slow communication speed.
    • Hits arriving after the TDC time window are discarded.
    • If SiTCP buffer (32kB) becomes full, txBuffFull bit is set.
  • Overflow bits: Ch-full, Multi-start-error and txBuffFull (described in TCP data).

DC-mode

  • NIM-in (TDCstart) negative leading edge reset and start time recording.
    • GateNet time (5c event) is issued.
    • Event numer (01 event) is incremented and issued.
    • Every 2162^{16} ns = 65536 ns, TDC higher bytes (02 event) is issued.
  • All the hits are delayed by DELAYtime (see register map).
  • If an edge is identified in the input (LVTTL 32ch), the timing is registered.
    • If the edge is negative leading, 03 event is issued.
    • If the edge is positive leading, 04 event is issued.
    • The minimum separation of positive/negative edges is 8ns. If the digital pulse is shorter than that, it will not be registered.
    • If SiTCP buffer (32kB) becomes full, txBuffFull bit is set.
  • Overflow bits: txBuffFull.
  • Trailer is issued when the next TDCstart comes.

RBCP Register map

RBCP register map is listed in the table below. To see the contents in the RBCP register, run on a terminal
$./SlowControl/read-registers_ip 192.168.10.x.
The most important role of RBCP register in Kalliope is to store the DAC parameters for ASIC in DACData1 (or DACData2), and transfer them to ASIC in serialized protocol (SPI) to set the ASD characteristics. This procedure must be performed, and the other parameters (e.g. FPGA_CTRL, DELAY, GATENET_TIME and ASIC_POL) must be set prior to the data acquisition. Some of the parameters have different meanings between Pulse- and DC-mode firmware.

RBCP Address Length (byte) Name memo
0x000~0x003 4 VER firmware version in YY MM DD Ed
0x003~0x007 4 FPGA FPGA ID (=0x20020010)
0x008~0x00B 4 EVENT_NUM Trigger Count
0x00C 1 FPGA_CTRL FPGA Control bits. See below.
0x00D~0x00F 3 KEY_WORD Keyword (Pulse only). See below
0x010~0x013 4 DELAY In 8 ns unit. TDC Time Window (Pulse), or
Constant delay for all the hits (DC). See below.
0x014~0x017 4 GAP Transient time (pulse-legacy)
0x018~0x019 2 PARAM SPI and On-chip parameter
0x01A~0x01B 2 CMD SPI and On-chip command.
Triggers SPI data transfer when written.
0x01C~0x01D 2 DISPLAY1 On-chip display1 (pulse-legacy)
0x01E~0x01F 2 DISPLAY2 On-chip display2 (pulse-legacy)
0x020~0x07F 3 x 32 ch DACData1 ASIC control DAC Bank 1
0x080~0x0DF 3 x 32 ch DACData2 ASIC control DAC Bank 2
0x0E0 1 reserved
0x0E1~0x0E7 7 GATENET_TIME Gatenet time (DC). See below.
0x0E8 1 ASIC_POL ASIC SPI pattern and polarity (DC). See below.
0x0E9~0x0EF 7 reserved
0x0F0~0x0FF 8 reserved
0x100~0x13F 2 x 32 ch eCount Count rate for Bank 1 (pulse-legacy)
0x140~0x17F 2 x 32 ch nCount Count rate for Bank 2 (pulse-legacy)
0x180~0x1FF 128 reserved
0x200~0x27F 4 x 32 ch eInteg Integrate Count for Bank 1 (pulse-legacy)
0x280~0x2FF 4 x 32 ch nInteg Integrate Count for Bank 2 (pulse-legacy)

The important parameters are described below.

FPGA_CTRL (default=0x40)
bit7: reserved. 0
bit6: BYTE_SWAP_ENABLE 1=little endian / 0=big endian for TCP data.
bit5: reserved. 0
bit4: EVT04_Mask 1=suppress/0=enable 04 event in TCP data (DC).
bit3: COPHDD_MASK 1=suppress/0=enable Copper Header in TCP data (DC).
bit2: COPTRL_MASK 1=suppress/0=enable Copper Trailer in TCP data (DC).
bit1: GATNET_MASK 1=suppress/0=enable GateNet time in TCP data (DC).
bit0: TDC_ENABLE reserved. 0 (was effective in Pulse-mode, but obsolete.)
There has been no need to change this parameter, because all the DAQ-PC's nowadays are Intel and employ little endian. (Motorola used big endian.)
DELAY (Pulse: default=0x1F3F; DC: default=0x0000)
In Pulse-mode firmware, the TDC time window in 8ns unit. Default 0x1F3F x 8 = 63992 ns.
In DC-mode firmware, the delay time (in 8ns unit) added to all the hit signals. Default 0x0000 x 8 = 0 ns. Maximum is 0x007F x 8 = 1016 ns.
To set, run on a terminal $./SlowControl/dcdelay_ip 192.168.10.z <DELAY>.
GATENET_TIME (DC)
For DC-mode firmware, set the GATENET time in this field after Kalliope is powered on. Write from 0x0E1 to 0x0E7; the time is set when 0x0E7 is written. The time will increment automatically, and feed the GATENET time in 5c event of the TCP data. Read from 0x0E1 to 0x0E7; the time is defined when 0x0E1 is read. The format for GATENET time is described in TCP data structure section.
To set, run on a terminal $./sitcp_dump/kalliope-config 192.168.10.z.
ASIC_POL (DC) (default=0x00)
This register is defined only for DC-mode Volume2014 (Ed=03) firmware. This parameter enables the SPI pattern required for future ASICS, such as FGATI. For Volume201x, keep 0x00; for FGATI, set to 0x08.
This parameter is set in the DAC writing program, such as ./SlowControl/pcfgati_ip. For future a ASIC, understanding of its SPI protocol and the board design is necessary to select this parameter appropriately.
bit7: reserved. 0
bit6: reserved. 0
bit5: reserved. 0
bit4: reserved. 0
bit3-2: DAC_CLBn_SEL 00=Pre-clear / 01=Post-load / 1x=Data-shift enable (SSB)
bit1: reserved. DAC_SCLK_POL 0=positive-leading edge / 1=negatie-leading edge
bit0: reserved. DAC_CLBn_POL 0=negative logic / 1=positive logic.
Detailed SPI pattern is shown in the figure below
ASIC_POL_pattern
Figure 1: Pattern of SPI signals for ASIC_POL register values. Volume2014 requires 0x00 =(Pre-clear/positive-leading edge/negative logic), and FGATI requires 0x08=(SSB/positive-leading edge/negative logic)

SPI transfer to ASIC

When CMD field is written, the firmware interprets the command (CMD) and its parameter (PARAM) to take an action. The most important action is serializing the DAC parameter which have been stored in DACData1 (or DACData2), and transfer to ASICs in order to control its Amplifier-Shaper-Discriminator (ASD) characteristics. To access DACData1 (or DACData2) region, there are slow control programs prepared.

  • Writing DAC on DACDataN (N=1 or 2): run on a terminal
    $./SlowControl/pc2012_ip 192.168.10.z N <parameter file> (for Volume2012)
    $./SlowControl/pc2014_ip 192.168.10.z N <parameter file> (for Volume2014)
    $./SlowControl/pcfgati_ip 192.168.10.z N <parameter file> (for FGATI)
    where, N(=1 or 2) is the DACData bank number, and z is the IP address. The format of DAC parameter file is described in Software section.

  • Serializing DAC and transfering to ASIC: run on a terminal
    $./SlowControl/kc_ip 192.168.10.z 1 N
    which issues command CMD=1 (start SPI transfer) with parameter PARAM=N(1 or 2) to Kalliope device at IP=192.168.10.z. N specifies the DACData bank number where the DAC has been stored (usually 1 is used).

TCP data structure

The TCP data structure of Kalliope is based on Copper-Lite/Finesse data format with fixed headers (0x7FFF000A and 0xFFAA0000) and a trailer (0xFF550000). The data format is in 32 bits (4 bytes) unit, except the GATENET time, which is in 64 bits (8 bytes). Some fields are different between Pulse- and DC-mode firmware. A major difference is that the total length of data associated with one NIM-in trigger is know in Pulse-mode TDC data (Length field), but not in DC-mode data. This makes the DAQ programs having different structures to receive, decode and write the TCP data into the list-mode data.

Kalliope TCP data

DC-mode data. Issued as soon as event happens.         Pulse-mode data. Issued at the end of TDC time window. 
GATENET-time
MSB                                           LSB
 | 0x5c |          GATENET time (H) (24bit)    |          No GATENET time issued from Pulse-mode firmware
 |           GATENET time (L) (32bit)          |

Copper Header                                         
MSB                                           LSB      MSB                                           LSB       
 |                 0x7FFF000A                  |        |                 0x7FFF000A                  |        
 | 0x00 |          Keyword (24bit)             |        | 0x00 |          Keyword (24bit)             |        
 |                 0x00000000                  |        |   Length (32bit) # of bytes to Trailer      |  

01 event                                                           
 | 0x01 |        TriggerCount (24bit)          |        |               TriggerCount (32bit)          |        

Finesse Header                                                     
 |                 0xFFAA0000                  |        |                 0xFFAA0000                  |        
 |     TriggerCount (24bit)         |   0x00   |        |     TriggerCount (24bit)         |   0x00   |        

02 event                                               TDC stop-data                           
 | 0x02 |  IP(8bit)  |    TDC[31:16](16bit)    |        | 0x00  |"0"|ChFull|LastData|Ch[4:0]|TDC[15:0]|        

03/04 event                                           ...(repetition of TDC stop-data)...              
 | 0x03 |  Ch(8bit)  |    TDC[15:0](16bit)     |                                       
 | 0x04 |  Ch(8bit)  |    TDC[15:0](16bit)     |       TDC start-data                          
                                                        |MultiStartError| "001" |0x000 |  TDC[15:0]   |              
 ...(repetition of 02/03/04 events)...                                         

Copper Trailer                                                     
 |                 0xFF550000                  |        |                 0xFF550000                  |        
 | 0x00 | "00000" | txBuffFull | "11" | 0x0000 |        | 0x00 | "00000" | txBuffFull | "11" | 0x0000 |        

Parameter contents

  • GATENET time (epoch is January 1, 00:00.00 in 2008)
    • S[29:0] 30 bits in seconds
    • SS[14:0] 15 bits in 1/32768 seconds
    • US[10:0] 11 bits in 25 ns
  • Keyword
    • NIM-TDC: NIM-IN (3-bit) pattern, sampled at the end of TDC window (Pulse) or ~100ns (DC) from the TDCstart.
    • Volume2012/2014 (DC): 8ns free run counter (to check the pileup).
    • Volume2012/2014 (Pulse): 8ns free run counter ???
  • TriggerCount: increments in every NIM-in triggers.

  • 02 event (DC) is issued in every 65536 ns.

  • 03/04 event (DC) is issued whenever there is a hit.

  • Overflow / status bits

    • ChFull = "1" if the hit buffer is full. Data in later time is lost.
    • LastData = "1" if it is the last data of the channel.
    • TimeStampOverFlow = "1" if Niki Time stamp overflows.
    • MultiStartError = "1" if NIM-in trigger is received during SiTCP data transfer. Suggests that the data transfer time is too short. The trigger is ignored, and the event count will be discontinuous.
    • txBuffFull = "1" if SiTCP buffer is full. Suggests that the data trasfer rate is low.

SESoft has programed the VME crate controler for Niki A3N00 modules [2,3] to send COPPER-Lite/Finesse-like TCP data. The actual contents must be confirmed by the hardware.

Niki TCP data. 

Copper-lite Header (Niki version)
MSB                                           LSB
 |                 0x7FFF000A                  |
 |                 Module ID                   | IP addrss ?
 | 0x00 |            Keyword (24bit)           | 
 | 0x00 | Module[7:0]| MSCR[7:0]| GainMode[7:0]| Module[3:0]=Mod (N for A3N00), MSCR[3:0]=Reso (order unsure) 
 |                 Length[31:0]                | 

 |              TriggerCount (32bit)           | 

Finesse Header                                                     
 |                 0xFFAA0000                  | 
 |     TriggerCount (24bit)         |   0x00   | 

Niki hit-data                      
 |ID[7:0]|"0"|ChFull|LastData|Ch[4:0]|TDC[15:0]|

..(repetition of TDC stop-data)...         

Niki start-data                    
 |MultiStartError| "001" |0x000 |  TDC[15:0]   | (not sure about the contents)

 |                 0xFF550000                  |
 | 0x00 | "00000" | txBuffFull | "11" | 0x0000 |

Parameter contents

  • Niki hit-data (Pulse): ID[7:0] =
    • 0x00 reserved for Kalliope.
    • 0x1n for Niki TDC=TimeStamp[15:0] from n-th module.
    • 0x2n for Niki TDC=TimeStamp[31:16] from n-th module.
    • 0x3n for Niki TDC={NikiErrorByte[7:0],TimeStamp[39:32]} from n-th module.
      • NikiErrorByte[7:0]={"0000000",TimeStampOverFlow}
    • 0x4n for Niki TDC=ADC[15:0] from n-th module.

On-chip commands in Pulse-mode (legacy)

Since J-PARC MLF accelerator trigger is 25Hz, there are plenty of time available after the data transfer is done. Even if all the hit buffer of Kalliope may be filled (1000 hits x 32 channels), the time necessary for the data transfer from one board is 8 bit x 4 bytes x 1000 x 32 (bits) / 1Gbps = 1.024 M / 1 G = 1 ms. There is a possiblity for Kalliope to do something else while it is not taking data in Pulse-mode firmware. A NIM-in trigger driven alternation of the DAC parameters has been developed, together with count scalers on FPGA. This function has not been tested much, and there will be no immediate need to activate it, so the parameters related to this funciton is labeled (pulse-legacy) in the RBCP register map. For future reference, the function is described here. The timing chart of the trigger-driven DAC parameter transition is shown below.

TRGdrivenDAC
Figure 2: Timing chart to NIM-in trigger driven ASIC DAC transition.

On-chip command parameters

GAP(minimum 512 ns = 0x40)
The length of time in 8 ns unit where trigger-driven SPI is performed. This GAP window is set after the TDC time window (DELAY) and before the next NIM-in trigger (40ms repetition is assumed).
To set, run on a terminal $./SlowControl/tw 2 <GAP> which sets GAP to Kalliope device at IP=192.168.10.16.
DISPLAY1
???
DISPLAY2
???
eCount
Scaler for 32 channels in 0~DELAY time (DAC=bank1)
nCount
Scaler for 32 channels in DELAY+GAP ~ 40ms-GAP time (DAC=bank2)
eInteg
Integrated scaler for 32 channels in 0~DELAY time (DAC=bank1)
nInteg
Integrated scaler for 32 channels in DELAY+GAP ~ 40ms-GAP time (DAC=bank2)

On-chip command syntax

CMD PARAM memo
0x0001 bank number (=1 or 2) SPI transfer immediately
0x0002 NIM-in trigger driven SPI, start
0x0004 NIM-in trigger driven SPI, stop
0x0011 number of triggers Preset integration triggers for bank 1
0x0012 integration start (ends after preset reached)
0x0014 integration abort
0x0018 reset counters eInteg (0x200-27F)
0x0021 number of triggers Preset integration triggers for bank 2
0x0022 integration start (ends after preset reached)
0x0024 integration abort
0x0028 reset counters nInteg (0x280-2FF)

To run on-chip commands: run on a terminal $./SlowControl/kc_ip 192.168.10.z <CMD> <PARAM>
which issues command CMD with parameter PARAM to Kalliope device at IP=192.168.10.z.