Added some utility functions for, and documentation about, trigger types used in integtests by bieryAtFnal · Pull Request #161 · DUNE-DAQ/integrationtest

bieryAtFnal · 2026-06-22T18:40:31Z

Description

It is not always easy to determine which trigger type(s) are used in which integtests, nor how to specify the parameters for a given trigger type in a new integtest.

The changes in this PR attempt to help with that, and they provide several utility functions that can be used to enable Fake HSI triggers and/or set basic parameters associated with FakeHSI and/or RandomTriggerCandidateMaker triggers. This PR also include some preliminary documentation about which triggers are used in which integtests.

This PR is correlated with ones in several other repos:

To test these changes, the following steps can be used:

DATE_PREFIX=`date '+%d%b'`
TIME_SUFFIX=`date '+%H%M'`

source /cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
dbt-create -n NFD_DEV_260624_A9 ${DATE_PREFIX}FDDevIntegtestTriggerChoices_${TIME_SUFFIX}
cd ${DATE_PREFIX}FDDevIntegtestTriggerChoices_${TIME_SUFFIX}/sourcecode

git clone https://github.com/DUNE-DAQ/crtmodules.git -b kbiery/integtest_trigger_choices
git clone https://github.com/DUNE-DAQ/daqsystemtest.git -b kbiery/integtest_trigger_choices
git clone https://github.com/DUNE-DAQ/dfmodules.git -b kbiery/integtest_trigger_choices
git clone https://github.com/DUNE-DAQ/hsilibs.git -b develop
git clone https://github.com/DUNE-DAQ/listrev.git -b develop
git clone https://github.com/DUNE-DAQ/snbmodules.git -b develop
git clone https://github.com/DUNE-DAQ/trigger.git -b kbiery/integtest_trigger_choices
cd ..

cd pythoncode
git clone https://github.com/DUNE-DAQ/integrationtest.git -b kbiery/integtest_trigger_choices
cd ..

. ./env.sh
dbt-build -j 12
dbt-workarea-env

dunedaq_integtest_bundle.sh --verbosity 2 -r local --random 5

Type of change

Optimization (non-breaking change that improves code/performance)

Testing checklist

Full set of integration tests pass (dunedaq_integtest_bundle.sh)

Further checks

Code is commented where needed, particularly in hard-to-understand areas

Updated the survey document to include an introduction and a summary table for integration tests.

…formation in tables.

Updated the integtest documentation to include WIB counts and trigger rates for various tests.

…ndow parameters.

…rigger_choices

…/DUNE-DAQ/integrationtest into kbiery/integtest_trigger_choices

…rigger_choices

unneeded files.

Updated descriptions of trigger sources and their configurations in the integration tests documentation.

MRiganSUSX

Hi @bieryAtFnal,
Thanks for this updated, much cleaner, functionality.
I'm leaving several suggestions here and across the linked 5 PRs.

Don't think there are any glaring problems, some small style things and some places that could be tidied up.

MRiganSUSX · 2026-06-26T17:48:08Z

+#   exists in the input/template DUNE-DAQ configuration, so there is the possibility of some
+#   confusion if a kRandom TCReadoutMap entry exists.
+# ************************************************************
+def set_RTCM_trigger_params(integtest_params: data_classes.integtest_param_base_class, trigger_rate: float = None,


Should we be consistent here and keep the snake_case, ie set_rtcm_trigger_params ?
(It also breaks PEP 8 but I'm unsure whether we are adhering to that in out python repos)
If you agree, this will need to be addressed across the linked PRs as well.

lower-case "rtcm" seems reasonable to me. I've made that change.

MRiganSUSX · 2026-06-26T18:06:27Z

+def set_fake_hsi_trigger_params(integtest_params: data_classes.integtest_param_base_class, trigger_rate: float = None,
+                                readout_window_before_ticks: int = None, readout_window_after_ticks: int = None):
+    if trigger_rate is not None:
+        integtest_params.config_substitutions.append(
+            data_classes.attribute_substitution(
+                obj_class="FakeHSIEventGeneratorConf",
+                updates={"trigger_rate": trigger_rate},
+            )
+        )


This is not immediately problematic, but the old places this is replacing (tpreplay_test.py, trigger_bitwords_test.py) were more specific in targeting obj_id="fakehsi".
If we ever have a situation where we have multiple FakeHSIEventGeneratorConf (not sure it is a concern), all would be affected.
The suggestion would be to add an optional obj_id: str = "*" so one can be more specific if needed, with the default doing what it is doing now.

I have added the obj_id="fakehsi" argument, as suggested.

MRiganSUSX · 2026-06-26T18:09:59Z

+def enable_fake_hsi_trigger(integtest_params: data_classes.integtest_param_base_class, trigger_rate: float = None,
+                            readout_window_before_ticks: int = None, readout_window_after_ticks: int = None):
+    if not isinstance(integtest_params, data_classes.integtest_params_for_generated_dunedaq_config):
+        fail_msg = "The FakeHSI trigger can only be dynamically enabled in DUNE-DAQ configurations that are generated by the integration test infrastructure, not ones that are pre-defined."


189 chars is above most PEP8 string limits.
If you'd like, this could be split across multiple lines, ie:

fail_msg = ("The FakeHSI trigger can only be dynamically enabled in DUNE-DAQ " "configurations that are generated by the integration test infrastructure, " "not ones that are pre-defined.")

I have split up that long string, as suggested.

MRiganSUSX · 2026-06-26T18:12:34Z

    except IsADirectoryError:
      print(f"File {file} could not be deleted because it is actually a directory.")
+
+# ************************************************************


Most linters require 2 empty lines before any new def, as is the case earlier in this file.

MRiganSUSX · 2026-06-26T18:12:42Z

+    integtest_params.fake_hsi_enabled = True
+    set_fake_hsi_trigger_params(integtest_params, trigger_rate, readout_window_before_ticks, readout_window_after_ticks)
+
+# ************************************************************


Most linters require 2 empty lines before any new def, as is the case earlier in this file.

MRiganSUSX · 2026-06-26T18:14:32Z

+        integtest_params.config_substitutions.append(
+            data_classes.attribute_substitution(
+                obj_class="TCReadoutMap",
+                obj_id = "def-hsi-tc-map",


There should be no empty spaces around = in keyword arguments,
ie suggesting changing to obj_id="def-hsi-tc-map".

MRiganSUSX · 2026-06-26T18:15:09Z

+        integtest_params.config_substitutions.append(
+            data_classes.attribute_substitution(
+                obj_class="RandomTCMakerConf",
+                obj_id = "random-tc-generator",


Same as before, empty spaces around =:
obj_id="random-tc-generator",

I have made the suggested changes to spacing.

MRiganSUSX · 2026-06-26T18:17:54Z

@@ -0,0 +1,75 @@
+# Survey of trigger and readout window configurations that are used in existing integtests, Summer 2026


This documentation is GREAT.
My only concern is whether it makes sense to have the year in the filename instead (completely ok in the text), as updates later on can cause link breaks, or the file may look like it obsolete later on.

We can discuss.
I put the date in the filename to try to make it very clear that it is a snapshot and may not be up-to-date at some point in the future...

MRiganSUSX · 2026-06-26T18:21:05Z

+| long_window_readout_test.py | 0.05 Hz trigger rate, RWBT=-100000000, RWET=1000000, RWW=1.62 sec | - | - |
+| minimal_system_quick_test.py | 1 Hz trigger rate, RWBT=-2000, RWET=5, RWW=32.1 usec | - | - |
+| readout_type_scan_test.py | 1 Hz trigger rate, various readout window widths | - | various rates and readout window widths |
+| sample_ehn1_multihost_test.py | - | - | - |


For a person not reading the whole documentation, it may suggest this test has no triggers.
Maybe we should add a note or legend distinguishing "set by integtest" from "inherited from predefined config"; or add a brief entry such as "1 Hz RTCM + 3 Hz FakeHSI (from predefined config local-1x1-config)".

MRiganSUSX · 2026-06-26T18:26:40Z

+
+### Other integtests:
+
+| Integtest name | RTCM | FakeHSI | Triggers from TPs |


Please remember to open an issue for these parts that need updating.

I have filled in all of the entries in the tables, so I don't think that additional updates are needed...

…UNE-DAQ/integrationtest#161

…y doc.

Updated integration test details for trigger rates and configurations, including specific parameters for WIB and DAPHNE data. Clarified expected rates and added observed rates for various tests.

Kurt Biery and others added 23 commits May 23, 2026 11:51

Started a document to list which triggers are used in which integtests.

8d359a1

Revise Survey of Triggers in Integtests

c7f3e09

Updated the survey document to include an introduction and a summary table for integration tests.

added more introduction and organized integtest-based config param in…

aaed3a4

…formation in tables.

minor changes

d1df5f3

Updated 3ru_1df information

27c61e3

Enhance integtest documentation with WIB details

909c8bf

Updated the integtest documentation to include WIB counts and trigger rates for various tests.

Temporary file with functions that help to set trigger and readout wi…

324d151

…ndow parameters.

filled in MSQT and SFQT info

eac283e

Update trigger rates and parameters in documentation

0682880

Merge remote-tracking branch 'origin/develop' into kbiery/integtest_t…

8a09c46

…rigger_choices

Merge branch 'kbiery/integtest_trigger_choices' of https://github.com…

289eb5f

…/DUNE-DAQ/integrationtest into kbiery/integtest_trigger_choices

Merge remote-tracking branch 'origin/develop' into kbiery/integtest_t…

2d3328d

…rigger_choices

Merge remote-tracking branch 'origin/develop' into kbiery/integtest_t…

ad8ad1f

…rigger_choices

Merge remote-tracking branch 'origin/develop' into kbiery/integtest_t…

8d0adde

…rigger_choices

Merged develop into kbiery/integtest_trigger_choices and removed

d84e1ee

unneeded files.

merged develop into kbiery/integtest_trigger_choices

3181970

Updated survey document for June 2026

70a555e

Updated trigger rates for trigger_bitwords_test.py

78daf71

Updated SurveyOfTriggersUsedInIntegtestsMay2026.md

693fe0c

Refine trigger source descriptions in documentation

c39a2b0

Updated descriptions of trigger sources and their configurations in the integration tests documentation.

Updated SurveyOfTriggersUsedInIntegtestsMay2026.md

e49ea66

Renamed the trigger summary doc

c429b16

Renamed the trigger summary doc

24fce0f

bieryAtFnal changed the title ~~Added some utility functions for, and documentation, about trigger types used in integtests~~ Added some utility functions for, and documentation about, trigger types used in integtests Jun 24, 2026

bieryAtFnal requested a review from MRiganSUSX June 24, 2026 13:45

MRiganSUSX reviewed Jun 26, 2026

View reviewed changes

Kurt Biery and others added 2 commits June 30, 2026 11:11

Made the changes to utility_functions.py suggested by Michal in PR #161.

1d93173

Add trigger rate details for sample_ehn1_multihost_test

a26c58d

bieryAtFnal pushed a commit to DUNE-DAQ/trigger that referenced this pull request Jun 30, 2026

Modified the integtests to take into account Michal's suggestions in D…

00da783

…UNE-DAQ/integrationtest#161

Kurt Biery and others added 3 commits July 1, 2026 09:54

First attempt at filling in triggers for dfmodules tests in the Surve…

053608d

…y doc.

Revise integration test parameters and expected rates

bde350d

Updated integration test details for trigger rates and configurations, including specific parameters for WIB and DAPHNE data. Clarified expected rates and added observed rates for various tests.

Updated the trigger information for the remaining tests.

75d1071

bieryAtFnal requested a review from MRiganSUSX July 2, 2026 19:11

		@@ -0,0 +1,75 @@
		# Survey of trigger and readout window configurations that are used in existing integtests, Summer 2026


		### Other integtests:

		\| Integtest name \| RTCM \| FakeHSI \| Triggers from TPs \|

Uh oh!

Conversation

bieryAtFnal commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Testing checklist

Further checks

Uh oh!

MRiganSUSX left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bieryAtFnal commented Jun 22, 2026 •

edited

Loading