instrumentation: Collect custom content from CMake configure

Add a `CUSTOM_CONTENT` argument to `cmake_instrumentation()` for collecting
custom content from configure time.

Snippet files include a reference to a JSON file containing any `CUSTOM_CONTENT`
that was added by this command.

Fixes: #26703

Help/command/cmake_instrumentation.rst

@@ -21,6 +21,7 @@ This allows for configuring instrumentation at the project-level.
     [HOOKS <hooks>...]
     [OPTIONS <options>...]
     [CALLBACK <callback>]
+    [CUSTOM_CONTENT <name> <type> <content>]
   )
 
 The ``API_VERSION`` and ``DATA_VERSION`` must always be given.  Currently, the
@@ -36,6 +37,35 @@ Whenever ``cmake_instrumentation`` is invoked, a query file is generated in
 ``<build>/.cmake/instrumentation/v1/query/generated`` to enable instrumentation
 with the provided arguments.
 
+.. _`cmake_instrumentation Configure Content`:
+
+Custom Configure Content
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``CUSTOM_CONTENT`` argument specifies certain data from configure time to
+include in each :ref:`cmake-instrumentation v1 Snippet File` that
+corresponds to the configure step associated with the command. This may be used
+to associate instrumentation data with certain information about its
+configuration, such as the optimization level or whether it is part of a
+coverage build.
+
+``CUSTOM_CONTENT`` expects ``name``, ``type`` and ``content`` arguments.
+
+``name`` is a specifier to identify the content being reported.
+
+``type`` specifies how the content should be interpreted. Supported values are:
+  * ``STRING`` the content is a string.
+  * ``BOOL`` the content should be interpreted as a boolean. It will be ``true``
+    under the same conditions that ``if()`` would be true for the given value.
+  * ``LIST`` the content is a CMake ``;`` separated list that should be parsed.
+  * ``JSON`` the content should be parsed as a JSON string. This can be a
+    number such as ``1`` or ``5.0``, a quoted string such as ``\"string\"``,
+    a boolean value ``true``/``false``, or a JSON object such as
+    ``{ \"key\" : \"value\" }`` that may be constructed using
+    ``string(JSON ...)`` commands.
+
+``content`` is the actual content to report.
+
 Example
 ^^^^^^^
 
@@ -51,6 +81,9 @@ equivalent JSON query file.
     OPTIONS staticSystemInformation dynamicSystemInformation
     CALLBACK ${CMAKE_COMMAND} -P /path/to/handle_data.cmake
     CALLBACK ${CMAKE_COMMAND} -P /path/to/handle_data_2.cmake
+    CUSTOM_CONTENT myString STRING string
+    CUSTOM_CONTENT myList   LIST   "item1;item2"
+    CUSTOM_CONTENT myObject JSON   "{ \"key\" : \"value\" }"
   )
 
 .. code-block:: json
@@ -68,3 +101,18 @@ equivalent JSON query file.
       "/path/to/cmake -P /path/to/handle_data_2.cmake"
     ]
   }
+
+This will also result in a configure content JSON being reported in each
+:ref:`cmake-instrumentation v1 Snippet File` with the following contents:
+
+.. code-block:: json
+
+  {
+    "myString": "string",
+    "myList": [
+      "item1", "item2"
+    ],
+    "myObject": {
+      "key": "value"
+    }
+  }

Help/manual/cmake-instrumentation.7.rst

@@ -158,6 +158,10 @@ subdirectories:
   files, they should never be removed by other processes. Data collected here
   remains until after `Indexing`_ occurs and all `Callbacks`_ are executed.
 
+``data/content/``
+  A subset of the collected data, containing any
+  :ref:`cmake_instrumentation Configure Content` files.
+
 ``cdash/``
   Holds temporary files used internally to generate XML content to be submitted
   to CDash.
@@ -286,6 +290,8 @@ the `v1 Snippet File`_ and `v1 Index File`_. When using the `API v1`_, these
 files live in ``<build>/.cmake/instrumentation/v1/data/`` under the project
 build tree.
 
+.. _`cmake-instrumentation v1 Snippet File`:
+
 v1 Snippet File
 ---------------
 
@@ -394,6 +400,11 @@ and contain the following data:
     ``afterCPULoadAverage``
       The Average CPU Load at ``timeStop``.
 
+  ``configureContent``
+    The path to a :ref:`cmake_instrumentation Configure Content` file located under ``data``,
+    which may contain information about the CMake configure step corresponding
+    to this data.
+
 Example:
 
 .. code-block:: json
@@ -417,7 +428,8 @@ Example:
       "beforeHostMemoryUsed" : 6635832.0
     },
     "timeStart" : 1737053448177,
-    "duration" : 31
+    "duration" : 31,
+    "configureContent" : "content/configure-2025-07-11T12-46-32-0572.json"
   }
 
 v1 Index File

Source/cmInstrumentation.cxx

@@ -210,6 +210,39 @@ void cmInstrumentation::WriteJSONQuery(
     cmStrCat("query-", this->writtenJsonQueries++, ".json"));
 }
 
+void cmInstrumentation::AddCustomContent(std::string const& name,
+                                         Json::Value const& contents)
+{
+  this->customContent[name] = contents;
+}
+
+void cmInstrumentation::WriteCustomContent()
+{
+  if (!this->customContent.isNull()) {
+    this->WriteInstrumentationJson(
+      this->customContent, "data/content",
+      cmStrCat("configure-", this->ComputeSuffixTime(), ".json"));
+  }
+}
+
+std::string cmInstrumentation::GetLatestContentFile()
+{
+  std::string contentFile;
+  if (cmSystemTools::FileExists(
+        cmStrCat(this->timingDirv1, "/data/content"))) {
+    cmsys::Directory d;
+    if (d.Load(cmStrCat(this->timingDirv1, "/data/content"))) {
+      for (unsigned int i = 0; i < d.GetNumberOfFiles(); i++) {
+        std::string fname = d.GetFileName(i);
+        if (fname != "." && fname != ".." && fname > contentFile) {
+          contentFile = fname;
+        }
+      }
+    }
+  }
+  return contentFile;
+}
+
 void cmInstrumentation::ClearGeneratedQueries()
 {
   std::string dir = cmStrCat(this->timingDirv1, "/query/generated");
@@ -262,10 +295,8 @@ int cmInstrumentation::CollectTimingData(cmInstrumentationQuery::Hook hook)
     for (unsigned int i = 0; i < d.GetNumberOfFiles(); i++) {
       std::string fpath = d.GetFilePath(i);
       std::string fname = d.GetFile(i);
-      if (fname.rfind('.', 0) == 0) {
-        continue;
-      }
-      if (fname == file_name) {
+      if (fname.rfind('.', 0) == 0 || fname == file_name ||
+          d.FileIsDirectory(i)) {
         continue;
       }
       if (fname.rfind("index-", 0) == 0) {
@@ -325,6 +356,26 @@ int cmInstrumentation::CollectTimingData(cmInstrumentationQuery::Hook hook)
   }
   cmSystemTools::RemoveFile(index_path);
 
+  // Delete old content files
+  std::string const contentDir = cmStrCat(this->timingDirv1, "/data/content");
+  if (cmSystemTools::FileExists(contentDir)) {
+    std::string latestContent = this->GetLatestContentFile();
+    if (d.Load(contentDir)) {
+      for (unsigned int i = 0; i < d.GetNumberOfFiles(); i++) {
+        std::string fname = d.GetFileName(i);
+        std::string fpath = d.GetFilePath(i);
+        if (fname != "." && fname != ".." && fname != latestContent) {
+          int compare;
+          cmSystemTools::FileTimeCompare(
+            cmStrCat(contentDir, '/', latestContent), fpath, &compare);
+          if (compare == 1) {
+            cmSystemTools::RemoveFile(fpath);
+          }
+        }
+      }
+    }
+  }
+
   return 0;
 }
 
@@ -499,6 +550,11 @@ int cmInstrumentation::InstrumentCommand(
   int ret = callback();
   root["result"] = ret;
 
+  // Write configure content if command was configure
+  if (command_type == "configure") {
+    this->WriteCustomContent();
+  }
+
   // Exit early if configure didn't generate a query
   if (reloadQueriesAfterCommand == LoadQueriesAfter::Yes) {
     this->LoadQueries();
@@ -563,6 +619,12 @@ int cmInstrumentation::InstrumentCommand(
   root["role"] = command_type;
   root["workingDir"] = cmSystemTools::GetLogicalWorkingDirectory();
 
+  // Add custom configure content
+  std::string contentFile = this->GetLatestContentFile();
+  if (!contentFile.empty()) {
+    root["configureContent"] = cmStrCat("content/", contentFile);
+  }
+
   // Write Json
   cmsys::SystemInformation& info = this->GetSystemInformation();
   std::string const& file_name = cmStrCat(

Source/cmInstrumentation.h

@@ -60,6 +60,9 @@ public:
   void WriteJSONQuery(std::set<cmInstrumentationQuery::Option> const& options,
                       std::set<cmInstrumentationQuery::Hook> const& hooks,
                       std::vector<std::vector<std::string>> const& callback);
+  void AddCustomContent(std::string const& name, Json::Value const& contents);
+  void WriteCustomContent();
+  std::string GetLatestContentFile();
   void ClearGeneratedQueries();
   int CollectTimingData(cmInstrumentationQuery::Hook hook);
   int SpawnBuildDaemon();
@@ -101,6 +104,7 @@ private:
   bool hasQuery = false;
   bool ranSystemChecks = false;
   bool ranOSCheck = false;
+  Json::Value customContent;
 #ifndef CMAKE_BOOTSTRAP
   std::unique_ptr<cmsys::SystemInformation> systemInformation;
   cmsys::SystemInformation& GetSystemInformation();

Source/cmInstrumentationCommand.cxx

@@ -7,17 +7,23 @@ file LICENSE.rst or https://cmake.org/licensing for details.  */
 #include <cstdlib>
 #include <functional>
 #include <set>
+#include <sstream>
 
 #include <cmext/string_view>
 
+#include <cm3p/json/reader.h>
+#include <cm3p/json/value.h>
+
 #include "cmArgumentParser.h"
 #include "cmArgumentParserTypes.h"
 #include "cmExecutionStatus.h"
 #include "cmExperimental.h"
 #include "cmInstrumentation.h"
 #include "cmInstrumentationQuery.h"
+#include "cmList.h"
 #include "cmMakefile.h"
 #include "cmStringAlgorithms.h"
+#include "cmValue.h"
 #include "cmake.h"
 
 namespace {
@@ -82,14 +88,18 @@ bool cmInstrumentationCommand(std::vector<std::string> const& args,
     ArgumentParser::NonEmpty<std::vector<std::string>> Options;
     ArgumentParser::NonEmpty<std::vector<std::string>> Hooks;
     ArgumentParser::NonEmpty<std::vector<std::vector<std::string>>> Callbacks;
+    ArgumentParser::NonEmpty<std::vector<std::vector<std::string>>>
+      CustomContent;
   };
 
-  static auto const parser = cmArgumentParser<Arguments>{}
-                               .Bind("API_VERSION"_s, &Arguments::ApiVersion)
-                               .Bind("DATA_VERSION"_s, &Arguments::DataVersion)
-                               .Bind("OPTIONS"_s, &Arguments::Options)
-                               .Bind("HOOKS"_s, &Arguments::Hooks)
-                               .Bind("CALLBACK"_s, &Arguments::Callbacks);
+  static auto const parser =
+    cmArgumentParser<Arguments>{}
+      .Bind("API_VERSION"_s, &Arguments::ApiVersion)
+      .Bind("DATA_VERSION"_s, &Arguments::DataVersion)
+      .Bind("OPTIONS"_s, &Arguments::Options)
+      .Bind("HOOKS"_s, &Arguments::Hooks)
+      .Bind("CALLBACK"_s, &Arguments::Callbacks)
+      .Bind("CUSTOM_CONTENT"_s, &Arguments::CustomContent);
 
   std::vector<std::string> unparsedArguments;
   Arguments const arguments = parser.Parse(args, &unparsedArguments);
@@ -137,10 +147,45 @@ bool cmInstrumentationCommand(std::vector<std::string> const& args,
     hooks.insert(hook);
   }
 
-  status.GetMakefile()
-    .GetCMakeInstance()
-    ->GetInstrumentation()
-    ->WriteJSONQuery(options, hooks, arguments.Callbacks);
+  // Generate custom content
+  cmInstrumentation* instrumentation =
+    status.GetMakefile().GetCMakeInstance()->GetInstrumentation();
+  for (auto const& content : arguments.CustomContent) {
+    if (content.size() != 3) {
+      status.SetError("CUSTOM_CONTENT expected 3 arguments");
+      return false;
+    }
+    std::string const label = content[0];
+    std::string const type = content[1];
+    std::string const contentString = content[2];
+    Json::Value value;
+    if (type == "STRING") {
+      value = contentString;
+    } else if (type == "BOOL") {
+      value = !cmValue(contentString).IsOff();
+    } else if (type == "LIST") {
+      value = Json::arrayValue;
+      for (auto const& item : cmList(contentString)) {
+        value.append(item);
+      }
+    } else if (type == "JSON") {
+      Json::CharReaderBuilder builder;
+      std::istringstream iss(contentString);
+      if (!Json::parseFromStream(builder, iss, &value, nullptr)) {
+        status.SetError(
+          cmStrCat("failed to parse custom content as JSON: ", contentString));
+        return false;
+      }
+    } else {
+      status.SetError(
+        cmStrCat("got an invalid type for CUSTOM_CONTENT: ", type));
+      return false;
+    }
+    instrumentation->AddCustomContent(content.front(), value);
+  }
+
+  // Write query file
+  instrumentation->WriteJSONQuery(options, hooks, arguments.Callbacks);
 
   return true;
 }

Tests/RunCMake/Instrumentation/RunCMakeTest.cmake

@@ -6,7 +6,7 @@ function(instrument test)
   set(config "${CMAKE_CURRENT_LIST_DIR}/config")
   set(ENV{CMAKE_CONFIG_DIR} ${config})
   cmake_parse_arguments(ARGS
-    "BUILD;BUILD_MAKE_PROGRAM;INSTALL;TEST;COPY_QUERIES;COPY_QUERIES_GENERATED;NO_WARN;STATIC_QUERY;DYNAMIC_QUERY;INSTALL_PARALLEL;MANUAL_HOOK"
+    "BUILD;BUILD_MAKE_PROGRAM;INSTALL;TEST;COPY_QUERIES;COPY_QUERIES_GENERATED;NO_WARN;STATIC_QUERY;DYNAMIC_QUERY;INSTALL_PARALLEL;MANUAL_HOOK;PRESERVE_DATA;NO_CONFIGURE"
     "CHECK_SCRIPT;CONFIGURE_ARG" "" ${ARGN})
   set(RunCMake_TEST_BINARY_DIR ${RunCMake_BINARY_DIR}/${test})
   set(uuid "d16a3082-c4e1-489b-b90c-55750a334f27")
@@ -15,7 +15,11 @@ function(instrument test)
 
   # Clear previous instrumentation data
   # We can't use RunCMake_TEST_NO_CLEAN 0 because we preserve queries placed in the build tree after
-  file(REMOVE_RECURSE ${RunCMake_TEST_BINARY_DIR})
+  if (ARGS_PRESERVE_DATA)
+    file(REMOVE_RECURSE ${RunCMake_TEST_BINARY_DIR}/CMakeFiles)
+  else()
+    file(REMOVE_RECURSE ${RunCMake_TEST_BINARY_DIR})
+  endif()
 
   # Set hook command
   set(static_query_hook_arg 0)
@@ -59,7 +63,9 @@ function(instrument test)
   if(NOT RunCMake_GENERATOR_IS_MULTI_CONFIG)
     set(maybe_CMAKE_BUILD_TYPE -DCMAKE_BUILD_TYPE=Debug)
   endif()
-  run_cmake_with_options(${test} ${ARGS_CONFIGURE_ARG} ${maybe_CMAKE_BUILD_TYPE})
+  if (NOT ARGS_NO_CONFIGURE)
+    run_cmake_with_options(${test} ${ARGS_CONFIGURE_ARG} ${maybe_CMAKE_BUILD_TYPE})
+  endif()
 
   # Follow-up Commands
   if (ARGS_BUILD)
@@ -112,7 +118,7 @@ instrument(dynamic-query BUILD INSTALL TEST DYNAMIC_QUERY
 instrument(both-query BUILD INSTALL TEST DYNAMIC_QUERY
   CHECK_SCRIPT check-data-dir.cmake)
 
-# cmake_instrumentation command
+# Test cmake_instrumentation command
 instrument(cmake-command
   COPY_QUERIES NO_WARN DYNAMIC_QUERY
   CHECK_SCRIPT check-generated-queries.cmake)
@@ -135,6 +141,25 @@ instrument(cmake-command-cmake-build NO_WARN
   CHECK_SCRIPT check-no-make-program-hooks.cmake
 )
 
+# Test CUSTOM_CONTENT
+instrument(cmake-command-custom-content NO_WARN BUILD
+  CONFIGURE_ARG "-DN=1"
+)
+instrument(cmake-command-custom-content NO_WARN BUILD
+  CONFIGURE_ARG "-DN=2"
+  CHECK_SCRIPT check-custom-content.cmake
+  PRESERVE_DATA
+)
+instrument(cmake-command-custom-content NO_WARN NO_CONFIGURE
+  MANUAL_HOOK
+  PRESERVE_DATA
+  CHECK_SCRIPT check-custom-content-removed.cmake
+)
+
+instrument(cmake-command-custom-content-bad-type NO_WARN)
+instrument(cmake-command-custom-content-bad-content NO_WARN)
+
+# Test make/ninja hooks
 if(RunCMake_GENERATOR STREQUAL "MSYS Makefiles")
   # FIXME(#27079): This does not work for MSYS Makefiles.
   set(Skip_BUILD_MAKE_PROGRAM_Case 1)

Tests/RunCMake/Instrumentation/check-custom-content-removed.cmake

@@ -0,0 +1,11 @@
+include(${CMAKE_CURRENT_LIST_DIR}/verify-snippet.cmake)
+
+if (NOT IS_DIRECTORY "${v1}/data/content")
+  add_error("Custom content directory does not exist.")
+endif()
+
+file(GLOB content_files ${v1}/data/content/*)
+list(LENGTH content_files num)
+if (NOT ${num} EQUAL 1)
+  add_error("Found ${num} custom content files, expected 1.")
+endif()

Tests/RunCMake/Instrumentation/check-custom-content.cmake

@@ -0,0 +1,57 @@
+include(${CMAKE_CURRENT_LIST_DIR}/check-data-dir.cmake)
+
+if (NOT IS_DIRECTORY "${v1}/data/content")
+  add_error("Custom content directory does not exist.")
+endif()
+
+file(GLOB content_files ${v1}/data/content/*)
+list(LENGTH content_files num)
+if (NOT ${num} EQUAL 2)
+  add_error("Found ${num} custom content files, expected 2.")
+endif()
+
+macro(assert_key contents key expected)
+  string(JSON value ERROR_VARIABLE errors GET "${contents}" ${key})
+  if (errors)
+    add_error("Did not find expected key \"${key}\" in custom content.")
+  endif()
+  if (NOT ${value} MATCHES ${expected})
+    add_error("Unexpected data in custom content file:\nGot ${value}, Expected ${expected}.")
+  endif()
+endmacro()
+
+# Check contents of configureContent files
+set(firstFile "")
+foreach(content_file IN LISTS content_files)
+  read_json("${content_file}" contents)
+  assert_key("${contents}" myString "string")
+  assert_key("${contents}" myBool "OFF")
+  assert_key("${contents}" myInt "1")
+  assert_key("${contents}" myFloat "2.5")
+  assert_key("${contents}" myTrue "ON")
+  assert_key("${contents}" myList "[ \"a\", \"b\", \"c\" ]")
+  assert_key("${contents}" myObject "{.*\"key\".*:.*\"value\".*}")
+  if (NOT firstFile)
+    set(firstFile "${content_file}")
+  endif()
+  if ("${content_file}" STREQUAL "${firstFile}")
+    string(JSON firstN GET "${contents}" nConfigure)
+  else()
+    string(JSON secondN GET "${contents}" nConfigure)
+  endif()
+endforeach()
+
+# Ensure provided -DN=* arguments result in differing JSON contents
+math(EXPR expectedSecondN "3-${firstN}")
+if (NOT ${secondN} EQUAL ${expectedSecondN})
+  add_error("Configure content did not correspond to provided cache variables.\nGot: ${firstN} and ${secondN}")
+endif()
+
+# Ensure snippets reference valid files
+foreach(snippet IN LISTS snippets)
+  read_json("${snippet}" contents)
+  string(JSON filename GET "${contents}" configureContent)
+  if (NOT EXISTS "${v1}/data/${filename}")
+    add_error("Reference to content file that does not exist.")
+  endif()
+endforeach()

Tests/RunCMake/Instrumentation/check-data-dir.cmake

@@ -1,7 +1,7 @@
 include(${CMAKE_CURRENT_LIST_DIR}/verify-snippet.cmake)
 include(${CMAKE_CURRENT_LIST_DIR}/json.cmake)
 
-file(GLOB snippets ${v1}/data/*)
+file(GLOB snippets LIST_DIRECTORIES false ${v1}/data/*)
 if (NOT snippets)
   add_error("No snippet files generated")
 endif()

Tests/RunCMake/Instrumentation/cmake-command-custom-content-bad-content-result.txt

@@ -0,0 +1 @@
+1

Tests/RunCMake/Instrumentation/cmake-command-custom-content-bad-content-stderr.txt

@@ -0,0 +1,6 @@
+CMake Error at [^
+]*:1 \(cmake_instrumentation\):
+  cmake_instrumentation failed to parse custom content as JSON: Not valid
+  JSON content
+Call Stack \(most recent call first\):
+  CMakeLists.txt:[0-9]+ \(include\)

Tests/RunCMake/Instrumentation/cmake-command-custom-content-bad-type-result.txt

@@ -0,0 +1 @@
+1

Tests/RunCMake/Instrumentation/cmake-command-custom-content-bad-type-stderr.txt

@@ -0,0 +1,5 @@
+CMake Error at [^
+]*:1 \(cmake_instrumentation\):
+  cmake_instrumentation got an invalid type for CUSTOM_CONTENT: INVALID
+Call Stack \(most recent call first\):
+  CMakeLists.txt:[0-9]+ \(include\)

Tests/RunCMake/Instrumentation/query/cmake-command-custom-content-bad-content.cmake

@@ -0,0 +1,5 @@
+cmake_instrumentation(
+  API_VERSION 1
+  DATA_VERSION 1
+  CUSTOM_CONTENT myContent JSON "Not valid JSON content"
+)

Tests/RunCMake/Instrumentation/query/cmake-command-custom-content-bad-type.cmake

@@ -0,0 +1,5 @@
+cmake_instrumentation(
+  API_VERSION 1
+  DATA_VERSION 1
+  CUSTOM_CONTENT myContent INVALID "Not a valid type"
+)

Tests/RunCMake/Instrumentation/query/cmake-command-custom-content.cmake

@@ -0,0 +1,14 @@
+string(JSON object SET {} key \"value\")
+
+cmake_instrumentation(
+  API_VERSION 1
+  DATA_VERSION 1
+  CUSTOM_CONTENT nConfigure STRING "${N}"
+  CUSTOM_CONTENT myString STRING "string"
+  CUSTOM_CONTENT myList   LIST "a;b;c"
+  CUSTOM_CONTENT myBool   BOOL OFF
+  CUSTOM_CONTENT myObject JSON "${object}"
+  CUSTOM_CONTENT myInt    JSON 1
+  CUSTOM_CONTENT myFloat  JSON 2.5
+  CUSTOM_CONTENT myTrue   JSON true
+)