GPU coordnum by HanatoK · Pull Request #940 · Colvars/colvars

HanatoK · 2026-07-15T21:21:24Z

This PR should wait for #919 and #938. #919 is necessary for wrapping the distances using the internal PBC function on GPU. This PR is independent from #926.

Regarding the pairlist implementation, the GPU kernels in this PR try to match the CPU pairlist, which is slow for the time being. A better option is to implement at least a NAMD-style GPU pairlist, but since Colvars (i) does not allow atoms in an atom group to be reordered, and (ii) uses only a single cutoff for both the switching function and the pairlist, it is impossible or very diffcult to do so.

HanatoK · 2026-07-28T19:59:11Z

Initial performance benchmark of computing selfcoordnum of 718 atoms in NAMD:

CPU: 63.2372 ns/day
CPU without cvm::error_static in cvm::system_boundary_conditions::position_distance: 77.2318 ns/day
GPU: 130.812 ns/day

Nsys shows that the GPU kernel is still very computationally expensive as its execution time is multiple times of the NB kernel:

I would prefer optimizing the overall calculations in a new PR or #926.

Test Colvars configuration:

colvarsTrajFrequency 10
colvarsRestartFrequency 1000
smp gpu

colvar {
  name cv
  outputAppliedForce on
  selfCoordNum {
    group1 {
      atomNumbers {23 26 29 32 35 38 41 44 47 50 53 56 59 62 65 68 71 74 77 80 83 86 89 92 95 98 101 104 107 110 113 116 119 122 125 128 131 134 137 140 143 146 149 152 155 158 161 164 167 170 173 176 179 182 185 188 191 194 197 200 203 206 209 212 215 218 221 224 227 230 233 236 239 242 245 248 251 254 257 260 263 266 269 272 275 278 281 284 287 290 293 296 299 302 305 308 311 314 317 320 323 326 329 332 335 338 341 344 347 350 353 356 359 362 365 368 371 374 377 380 383 386 389 392 395 398 401 404 407 410 413 416 419 422 425 428 431 434 437 440 443 446 449 452 455 458 461 464 467 470 473 476 479 482 485 488 491 494 497 500 503 506 509 512 515 518 521 524 527 530 533 536 539 542 545 548 551 554 557 560 563 566 569 572 575 578 581 584 587 590 593 596 599 602 605 608 611 614 617 620 623 626 629 632 635 638 641 644 647 650 653 656 659 662 665 668 671 674 677 680 683 686 689 692 695 698 701 704 707 710 713 716 719 722 725 728 731 734 737 740 743 746 749 752 755 758 761 764 767 770 773 776 779 782 785 788 791 794 797 800 803 806 809 812 815 818 821 824 827 830 833 836 839 842 845 848 851 854 857 860 863 866 869 872 875 878 881 884 887 890 893 896 899 902 905 908 911 914 917 920 923 926 929 932 935 938 941 944 947 950 953 956 959 962 965 968 971 974 977 980 983 986 989 992 995 998 1001 1004 1007 1010 1013 1016 1019 1022 1025 1028 1031 1034 1037 1040 1043 1046 1049 1052 1055 1058 1061 1064 1067 1070 1073 1076 1079 1082 1085 1088 1091 1094 1097 1100 1103 1106 1109 1112 1115 1118 1121 1124 1127 1130 1133 1136 1139 1142 1145 1148 1151 1154 1157 1160 1163 1166 1169 1172 1175 1178 1181 1184 1187 1190 1193 1196 1199 1202 1205 1208 1211 1214 1217 1220 1223 1226 1229 1232 1235 1238 1241 1244 1247 1250 1253 1256 1259 1262 1265 1268 1271 1274 1277 1280 1283 1286 1289 1292 1295 1298 1301 1304 1307 1310 1313 1316 1319 1322 1325 1328 1331 1334 1337 1340 1343 1346 1349 1352 1355 1358 1361 1364 1367 1370 1373 1376 1379 1382 1385 1388 1391 1394 1397 1400 1403 1406 1409 1412 1415 1418 1421 1424 1427 1430 1433 1436 1439 1442 1445 1448 1451 1454 1457 1460 1463 1466 1469 1472 1475 1478 1481 1484 1487 1490 1493 1496 1499 1502 1505 1508 1511 1514 1517 1520 1523 1526 1529 1532 1535 1538 1541 1544 1547 1550 1553 1556 1559 1562 1565 1568 1571 1574 1577 1580 1583 1586 1589 1592 1595 1598 1601 1604 1607 1610 1613 1616 1619 1622 1625 1628 1631 1634 1637 1640 1643 1646 1649 1652 1655 1658 1661 1664 1667 1670 1673 1676 1679 1682 1685 1688 1691 1694 1697 1700 1703 1706 1709 1712 1715 1718 1721 1724 1727 1730 1733 1736 1739 1742 1745 1748 1751 1754 1757 1760 1763 1766 1769 1772 1775 1778 1781 1784 1787 1790 1793 1796 1799 1802 1805 1808 1811 1814 1817 1820 1823 1826 1829 1832 1835 1838 1841 1844 1847 1850 1853 1856 1859 1862 1865 1868 1871 1874 1877 1880 1883 1886 1889 1892 1895 1898 1901 1904 1907 1910 1913 1916 1919 1922 1925 1928 1931 1934 1937 1940 1943 1946 1949 1952 1955 1958 1961 1964 1967 1970 1973 1976 1979 1982 1985 1988 1991 1994 1997 2000 2003 2006 2009 2012 2015 2018 2021 2024 2027 2030 2033 2036 2039 2042 2045 2048 2051 2054 2057 2060 2063 2066 2069 2072 2075 2078 2081 2084 2087 2090 2093 2096 2099 2102 2105 2108 2111 2114 2117 2120 2123 2126 2129 2132 2135 2138 2141 2144 2147 2150 2153 2156 2159 2162 2165 2168 2171 2174}
    }
  }
}

harmonic {
  colvars cv
  centers 4.67348234758518e+03
  forceConstant 0.001
}

HanatoK · 2026-07-30T15:01:39Z

Performance test of coordnum with Apoa1

The configuration file is

indexFile index.ndx
# smp gpu

colvar {
  name cv
  coordNum {
    group1 {
      indexGroup backbone
    }
    group2 {
      indexGroup waters
    }
    cutoff 10.0
  }
}

harmonic {
  colvars cv
  centers 16774.0
  forceConstant 0.00001
}

smp gpu was turned on when testing with this PR and CUDAGM.

Equilibrium simulation without Colvars: 42.4303 ns/day
Colvars with CPU coordNum: 0.552597 ns/day
Colvars with GPU coordNum (this PR): 5.82301 ns/day

ef_use_internal_pbc is not used anymore.

Assisted-by: Claude Code v2.1.222 (Opus 5)

…berGroupToCenterKernel Assisted-by: Claude Code v2.1.222 (Opus 5)

…oupCUDAKernel1 Assisted-by: Claude Code v2.1.222 (Opus 5)

Assisted-by: Claude Code v2.1.222 (Opus 5)

1. Move the copy of lattice information before the copy of atoms 2. Lattice is not GPU-resident for the time being, so we just skip waiting update_lattice before entering the kernels Assisted-by: Claude Code v2.1.222 (Opus 5)

gpu_warp_size Assisted-by: Claude Code v2.1.222 (Opus 5)

Assisted-by: Claude Code v2.1.222 (Opus 5)

This commit also reverts the "pragma unroll" in 9ff3e96.

…ernel1 Assisted-by: Claude Code v2.1.222 (Opus 5)

…GroupsCUDAKernel1

…Kernel1

jhenin

Excellent! Just a small question about an API change.

Copilot

Pull request overview

Adds CUDA/HIP acceleration for coordination-number CVs, including pairlists, gradients, COM handling, PBC, and NAMD integration.

Changes:

Implements GPU kernels for coordination-number variants.
Adds GPU buffers, synchronization, device metadata, and dummy-group handling.
Integrates kernels into functional-test and NAMD builds.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/functional_gpu/CMakeLists.txt	Builds coordination-number GPU kernels.
src/cuda/colvarcomp_coordnums_kernel.h	Declares GPU coordination APIs.
src/cuda/colvarcomp_coordnums_kernel.cu	Implements coordination kernels and pairlists.
src/cuda/colvaratoms_kernel.h	Declares weighted-gradient support.
src/cuda/colvaratoms_kernel.cu	Implements weighted-gradient kernel.
src/colvarproxy.cpp	Logs GPU device details.
src/colvarproxy_system.h	Makes boundary retrieval overridable.
src/colvarproxy_gpu.h	Exposes GPU warp size.
src/colvarproxy_gpu.cpp	Queries GPU warp size.
src/colvarmodule.h	Enables integer powers on devices.
src/colvarmodule.cpp	Consolidates GPU logging.
src/colvarcomp.cpp	Supplies PBC data to GPU CVCs.
src/colvarcomp_distances.cpp	Reports GPU RMSD activation.
src/colvarcomp_coordnums.h	Adds coordination GPU interfaces.
src/colvarcomp_coordnums.cpp	Integrates GPU coordination execution.
src/colvaratoms.cpp	Synchronizes dummy positions to GPU.
src/colvaratoms_gpu.h	Adds dummy and weighted-gradient APIs.
src/colvaratoms_gpu.cpp	Manages new GPU atom buffers.
src/colvar_gpu_support.h	Adds HIP device-attribute mappings.
namd/cudaglobalmaster/colvarproxy_cudaglobalmaster.C	Synchronizes NAMD lattice updates.
namd/cudaglobalmaster/CMakeLists.txt	Builds coordination kernels for NAMD.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

giacomofiorin

Looks great to me! I have some minor comments below, but as far as I'm concerned they can be looked at in later PRs, or not at all.

Please merge as-is if you don't have any further commits.

giacomofiorin · 2026-08-24T19:06:34Z

+  // NOTE: Maybe it is better to build tile lists on GPU if we
+  // want to exclude non-interacting tiles periodically like NAMD.


giacomofiorin · 2026-08-24T19:08:09Z

+        }
+      }
+    }
+#if 0


Do you still need this?

Sorry. I missed this comment when merging. I have removed the commented code in cc0fc16.

giacomofiorin · 2026-09-01T19:12:31Z

+    //       with a cutoff. However, currently it is unclear to me
+      //       whether Colvars developers would accept the idea of
+    //       allowing re-sorting atoms dynamically in atom groups.


Well, this sounds like such a change would be up to someone's discretion: it really isn't.

in some variables, e.g. those that depend on reference coordinates, the order of atoms chosen by the user should be handled consistently. For most other variables, the order is immaterial and it would be okay to sort atoms based on any useful criteria!

Thanks for your comment!

giacomofiorin · 2026-09-01T20:29:08Z

@@ -497,10 +493,6 @@ int colvarmodule::parse_global_params(std::string const &conf)
      }
      case colvarproxy_smp::smp_mode_t::gpu: {
        this->log("EXPERIMENTAL GPU parallelism will be used. GPU information:\n");


Not sure I would rely on the current order of operation to ensure that the messages in colvarproxy::setup() would be printed right after this.

I will revise the code in #953.

See dc3a2b3

giacomofiorin · 2026-09-01T20:32:57Z

+    if (error_code == COLVARS_OK) {
+      cvmodule->log("This CV \"" + name + "\" of type \"" + function_type() + "\" will be calculated on GPU.\n");
+    }
+#endif


Nitpick: maybe this can be moved to the base class since you already have two duplicates of this for RMSD and coordNum

Currently not every CVC has GPU implementation, so I only add logging for those that are already have (like RMSD and coordnum).

HanatoK self-assigned this Jul 15, 2026

HanatoK force-pushed the coordnum-template-gpu-2 branch 2 times, most recently from 8eac9a6 to 6c72b36 Compare July 28, 2026 19:09

HanatoK marked this pull request as ready for review July 28, 2026 19:46

HanatoK requested review from giacomofiorin and jhenin July 28, 2026 19:48

HanatoK force-pushed the coordnum-template-gpu-2 branch 2 times, most recently from 4307cb5 to 749cf75 Compare August 11, 2026 15:36

HanatoK added 12 commits August 12, 2026 09:45

WIP: initial coordnum GPU kernels

0e0347e

refactor: refine the loop in buildTileLists

56e8129

fix: fix the compilation after rebase

012bdaf

ef_use_internal_pbc is not used anymore.

feat: implement internal boundary condition in GPU kernels

9b65c8f

refactor: allow the proxy to override how lattice is updated

23f567e

fix: update system boundaries in colvar::cvc::read_data_gpu

f41ed0a

feat: declare GPU support for coordnum

74669a2

fix: set the gradients in the groupCoord GPU kernel

cd0cb03

feat: show warp/wavefront size when initializing the colvarmodule

9b9cb2f

fix: make gpu_warp_size available without depending on init_gpu()

6286b12

fix: fix the compilation of gpu_warp_size

3fb52c8

chore: add log when GPU is used for RMSD and coordnum

ec87d1e

HanatoK force-pushed the coordnum-template-gpu-2 branch from 746569f to ec87d1e Compare August 12, 2026 14:45

HanatoK added 7 commits August 12, 2026 13:58

fix: use group->size() instead of group1->size()

38a0441

Assisted-by: Claude Code v2.1.222 (Opus 5)

fix: update d_com and d_cog for dummy groups

a104c6c

Assisted-by: Claude Code v2.1.222 (Opus 5)

fix: add guard for dummy groups in set_weighted_gradient_gpu

dbb0d2e

Assisted-by: Claude Code v2.1.222 (Opus 5)

fix: avoid stale value of h_coordnum_out when not in pairlist

53d8b69

Assisted-by: Claude Code v2.1.222 (Opus 5)

fix: avoid uninitialized read of com_grad_* in computeCoordinationNum…

8cb4f97

…berGroupToCenterKernel Assisted-by: Claude Code v2.1.222 (Opus 5)

fix: correctly check pairlist_elem in computeCoordinationNumberSelfGr…

b47f79d

…oupCUDAKernel1 Assisted-by: Claude Code v2.1.222 (Opus 5)

perf: reduce the shared memory usage by using a pairmask

5a925d6

Assisted-by: Claude Code v2.1.222 (Opus 5)

HanatoK added 8 commits August 12, 2026 16:02

fix: fix buildTileLists

4c594c8

Assisted-by: Claude Code v2.1.222 (Opus 5)

perf: unroll some loops in colvarcomp_coordnums_kernel.cu

9ff3e96

Assisted-by: Claude Code v2.1.222 (Opus 5)

chore: minor fixes

d676ad4

1. Move the copy of lattice information before the copy of atoms 2. Lattice is not GPU-resident for the time being, so we just skip waiting update_lattice before entering the kernels Assisted-by: Claude Code v2.1.222 (Opus 5)

chore: add comments about pairlist_transposed and refactor the

abde4b1

gpu_warp_size Assisted-by: Claude Code v2.1.222 (Opus 5)

chore: launch the COM-to-COM kernel in a single warp

5182956

Assisted-by: Claude Code v2.1.222 (Opus 5)

fix: clear the pairmask correctly

faa4a1f

fix: correct the checks of pairlist in self coordnum kernel

b594e25

This commit also reverts the "pragma unroll" in 9ff3e96.

perf: use #pragma unroll 4 in computeCoordinationNumberSelfGroupCUDAK…

7935453

…ernel1 Assisted-by: Claude Code v2.1.222 (Opus 5)

HanatoK mentioned this pull request Aug 13, 2026

[RFC] Refactor atom groups read data #953

Open

HanatoK added 5 commits August 13, 2026 18:11

fix: prevent potential integer overflow of GPU pairlist indexes

463d0ec

refactor: refactor the implementation of computeCoordinationNumberTwo…

be59990

…GroupsCUDAKernel1

fix: fix the atomicAdd of j-atom gradients and cleanups

9068e26

chore: minor cleanups

2256368

refactor: simplify the loop in computeCoordinationNumberSelfGroupCUDA…

16ec12c

…Kernel1

This was referenced Aug 18, 2026

Optimize the GPU pairlist of the selfcoordnum kernel #954

Merged

Optimize the GPU pairlist of the coordnum kernel #955

Merged

jhenin reviewed Aug 19, 2026

View reviewed changes

Comment thread src/colvarproxy_system.h

HanatoK mentioned this pull request Aug 20, 2026

Optimize the switching function of coordnum on GPU #956

Open

giacomofiorin requested a balanced review from Copilot August 27, 2026 14:33

Copilot started reviewing on behalf of giacomofiorin August 27, 2026 14:34 View session

Copilot AI reviewed Aug 27, 2026

View reviewed changes

Comment thread src/cuda/colvarcomp_coordnums_kernel.cu Outdated

chore: fix the typo in the comment

dfcd08c

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

jhenin approved these changes Aug 27, 2026

View reviewed changes

giacomofiorin approved these changes Sep 1, 2026

View reviewed changes

HanatoK merged commit f4b3b12 into Colvars:master Sep 1, 2026
2 checks passed

		// NOTE: Maybe it is better to build tile lists on GPU if we
		// want to exclude non-interacting tiles periodically like NAMD.

Uh oh!

Conversation

HanatoK commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HanatoK commented Jul 28, 2026

Uh oh!

HanatoK commented Jul 30, 2026

Uh oh!

jhenin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

giacomofiorin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

HanatoK commented Jul 15, 2026 •

edited

Loading