Affinity mapping

[Maff Glover (mo-mglover)]

Description

Rather than take on-trust that MPI ranks and threads are bound to cores as expected, perhaps according to launcher flags or OMP_ environment variables, it is useful to be able to test this directly. More specifically, test the affinity from a running application on the platform / in the environment that an application might be running.

Including this functionality in Vernier is useful because any client application will then have access to it. The new functionality is exposed via two new API calls (one C++, one Fortran). As with all Vernier calls, these can be built into an application, perhaps behind a switch.

Example:

• 7 ranks
  • One compute rank with 122 threads.
  • 6 I/O server ranks. (Single-threaded.)
• Each thread bound to one physical core only.
  • Virtual cores are empty.
  • This does need some knowledge of how cores are numbered on a node. lscpu will give that information.

--> AFFINITY MAP <--

Maximum number of (logical) cores: 256

Thread binding map, key:

    . = No threads running on this core.
    # = Multiple threads running on this core.
  0-9 = Threads 0-9.
  a-z = Threads 10-35.
  A-Z = Threads 36-61.
    $ = Threads 62 and greater.

MPI rank : ...THREADS..ON..CORES... : Num. cores available for migration.

           Cores ---->
00000000 : 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$...................................................................................................................................... :   1
00000001 : ..........................................................................................................................0..................................................................................................................................... :   1
00000002 : ...........................................................................................................................0.................................................................................................................................... :   1
00000003 : ............................................................................................................................0................................................................................................................................... :   1
00000004 : .............................................................................................................................0.................................................................................................................................. :   1
00000005 : ..............................................................................................................................0................................................................................................................................. :   1
00000006 : ...............................................................................................................................0................................................................................................................................ :   1

Linked issues

No associated issue.

Type of change

• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How has this been tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• New tests have been added
• Tests have been modified to accommodate this change

Checklist:

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes, for both debug and optimised builds