ConfigureLog: Version individual events instead of the whole log

In order to support multiple log versions without buffering the
entire log, move versioning to the level of individual events.
Multiple versions of an event may then be logged consecutively.

Issue: #23200

Help/manual/cmake-configure-log.7.rst

@@ -38,15 +38,12 @@ step finished normally, ends with a ``...`` document marker line:
 .. code-block:: yaml
 
   ---
-  version:
-    major: 1
-    minor: 0
   events:
     -
-      kind: "try_compile"
+      kind: "try_compile-v1"
       # (other fields omitted)
     -
-      kind: "try_compile"
+      kind: "try_compile-v1"
       # (other fields omitted)
   ...
 
@@ -55,15 +52,27 @@ the build tree and logs new events.
 
 The keys of the each document root mapping are:
 
-``version``
-  A YAML mapping that describes the schema version of the log document.
-  It has keys ``major`` and ``minor`` holding non-negative integer values.
-
 ``events``
   A YAML block sequence of nodes corresponding to events logged during
   one CMake "configure" step.  Each event is a YAML node containing one
   of the `Event Kinds`_ documented below.
 
+Log Versioning
+--------------
+
+Each of the `Event Kinds`_ is versioned independently.  The set of
+keys an event's log entry provides is specific to its major version.
+When an event is logged, the latest version of its event kind that is
+known to the running version of CMake is always written to the log.
+
+Tools reading the configure log must ignore event kinds and versions
+they do not understand:
+
+* A future version of CMake may introduce a new event kind or version.
+
+* If an existing build tree is re-configured with a different version of
+  CMake, the log may contain different versions of the same event kind.
+
 Text Block Encoding
 -------------------
 
@@ -84,7 +93,7 @@ Every event kind is represented by a YAML mapping of the form:
 
 .. code-block:: yaml
 
-  kind: "<kind>"
+  kind: "<kind>-v<major>"
   backtrace:
     - "<file>:<line> (<function>)"
   #...event-specific keys...
@@ -92,28 +101,33 @@ Every event kind is represented by a YAML mapping of the form:
 The keys common to all events are:
 
 ``kind``
-  A string identifying the event kind.
+  A string identifying the event kind and major version.
 
 ``backtrace``
   A YAML block sequence reporting the call stack of CMake source
   locations at which the event occurred.  Each node is a string
   specifying one location formatted as ``<file>:<line> (<function>)``.
 
-Additional mapping keys are specific to each event kind,
+Additional mapping keys are specific to each (versioned) event kind,
 described below.
 
-.. _`try_compile event`:
-
 Event Kind ``try_compile``
 --------------------------
 
 The :command:`try_compile` command logs ``try_compile`` events.
 
-A ``try_compile`` event is a YAML mapping:
+There is only one ``try_compile`` event major version, version 1.
+
+.. _`try_compile-v1 event`:
+
+``try_compile-v1`` Event
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+A ``try_compile-v1`` event is a YAML mapping:
 
 .. code-block:: yaml
 
-  kind: "try_compile"
+  kind: "try_compile-v1"
   backtrace:
     - "CMakeLists.txt:123 (try_compile)"
   directories:
@@ -126,7 +140,7 @@ A ``try_compile`` event is a YAML mapping:
       # ...
     exitCode: 0
 
-The keys specific to ``try_compile`` mappings are:
+The keys specific to ``try_compile-v1`` mappings are:
 
 ``directories``
   A mapping describing the directories associated with the
@@ -168,11 +182,18 @@ Event Kind ``try_run``
 
 The :command:`try_run` command logs ``try_run`` events.
 
-A ``try_run`` event is a YAML mapping:
+There is only one ``try_run`` event major version, version 1.
+
+.. _`try_run-v1 event`:
+
+``try_run-v1`` Event
+^^^^^^^^^^^^^^^^^^^^
+
+A ``try_run-v1`` event is a YAML mapping:
 
 .. code-block:: yaml
 
-  kind: "try_run"
+  kind: "try_run-v1"
   backtrace:
     - "CMakeLists.txt:456 (try_run)"
   directories:
@@ -193,8 +214,8 @@ A ``try_run`` event is a YAML mapping:
       # ...
     exitCode: 0
 
-The keys specific to ``try_run`` mappings include those
-documented by the `try_compile event`_, plus:
+The keys specific to ``try_run-v1`` mappings include those
+documented by the `try_compile-v1 event`_, plus:
 
 ``runResult``
   A mapping describing the result of running the test code.

Source/cmConfigureLog.cxx

@@ -8,6 +8,7 @@
 #include <sstream>
 #include <utility>
 
+#include <cmext/algorithm>
 #include <cmext/string_view>
 
 #include <cm3p/json/writer.h>
@@ -20,9 +21,17 @@
 #include "cmSystemTools.h"
 #include "cmake.h"
 
-cmConfigureLog::cmConfigureLog(std::string logDir)
+cmConfigureLog::cmConfigureLog(std::string logDir,
+                               std::vector<unsigned long> logVersions)
   : LogDir(std::move(logDir))
+  , LogVersions(std::move(logVersions))
 {
+  // Always emit events for the latest log version.
+  static const unsigned long LatestLogVersion = 1;
+  if (!cm::contains(this->LogVersions, LatestLogVersion)) {
+    this->LogVersions.emplace_back(LatestLogVersion);
+  }
+
   Json::StreamWriterBuilder builder;
   this->Encoder.reset(builder.newStreamWriter());
 }
@@ -35,6 +44,24 @@ cmConfigureLog::~cmConfigureLog()
   }
 }
 
+bool cmConfigureLog::IsAnyLogVersionEnabled(
+  std::vector<unsigned long> const& v) const
+{
+  // Both input lists are sorted.  Look for a matching element.
+  auto i1 = v.cbegin();
+  auto i2 = this->LogVersions.cbegin();
+  while (i1 != v.cend() && i2 != this->LogVersions.cend()) {
+    if (*i1 < *i2) {
+      ++i1;
+    } else if (*i2 < *i1) {
+      ++i2;
+    } else {
+      return true;
+    }
+  }
+  return false;
+}
+
 void cmConfigureLog::WriteBacktrace(cmMakefile const& mf)
 {
   std::vector<std::string> backtrace;
@@ -64,10 +91,6 @@ void cmConfigureLog::EnsureInit()
   this->Opened = true;
 
   this->Stream << "\n---\n";
-  this->BeginObject("version"_s);
-  this->WriteValue("major"_s, 1);
-  this->WriteValue("minor"_s, 0);
-  this->EndObject();
   this->BeginObject("events"_s);
 }
 

Source/cmConfigureLog.h

@@ -19,9 +19,15 @@ class cmMakefile;
 class cmConfigureLog
 {
 public:
-  cmConfigureLog(std::string logDir);
+  /** Construct with the log directory and a sorted list of enabled log
+      versions.  The latest log version will be enabled regardless.  */
+  cmConfigureLog(std::string logDir, std::vector<unsigned long> logVersions);
   ~cmConfigureLog();
 
+  /** Return true if at least one of the log versions in the given sorted
+      list is enabled.  */
+  bool IsAnyLogVersionEnabled(std::vector<unsigned long> const& v) const;
+
   void WriteBacktrace(cmMakefile const& mf);
 
   void EnsureInit();
@@ -49,6 +55,7 @@ public:
 
 private:
   std::string LogDir;
+  std::vector<unsigned long> LogVersions;
   cmsys::ofstream Stream;
   unsigned Indent = 0;
   bool Opened = false;

Source/cmTryCompileCommand.cxx

@@ -21,10 +21,14 @@ namespace {
 void WriteTryCompileEvent(cmConfigureLog& log, cmMakefile const& mf,
                           cmTryCompileResult const& compileResult)
 {
-  log.BeginEvent("try_compile");
-  log.WriteBacktrace(mf);
-  cmCoreTryCompile::WriteTryCompileEventFields(log, compileResult);
-  log.EndEvent();
+  static const std::vector<unsigned long> LogVersionsWithTryCompileV1{ 1 };
+
+  if (log.IsAnyLogVersionEnabled(LogVersionsWithTryCompileV1)) {
+    log.BeginEvent("try_compile-v1");
+    log.WriteBacktrace(mf);
+    cmCoreTryCompile::WriteTryCompileEventFields(log, compileResult);
+    log.EndEvent();
+  }
 }
 #endif
 }

Source/cmTryRunCommand.cxx

@@ -40,28 +40,32 @@ void WriteTryRunEvent(cmConfigureLog& log, cmMakefile const& mf,
                       cmTryCompileResult const& compileResult,
                       cmTryRunResult const& runResult)
 {
-  log.BeginEvent("try_run");
-  log.WriteBacktrace(mf);
-  cmCoreTryCompile::WriteTryCompileEventFields(log, compileResult);
-
-  log.BeginObject("runResult"_s);
-  log.WriteValue("variable"_s, runResult.Variable);
-  log.WriteValue("cached"_s, runResult.VariableCached);
-  if (runResult.Stdout) {
-    log.WriteLiteralTextBlock("stdout"_s, *runResult.Stdout);
-  }
-  if (runResult.Stderr) {
-    log.WriteLiteralTextBlock("stderr"_s, *runResult.Stderr);
-  }
-  if (runResult.ExitCode) {
-    try {
-      log.WriteValue("exitCode"_s, std::stoi(*runResult.ExitCode));
-    } catch (std::invalid_argument const&) {
-      log.WriteValue("exitCode"_s, *runResult.ExitCode);
+  static const std::vector<unsigned long> LogVersionsWithTryRunV1{ 1 };
+
+  if (log.IsAnyLogVersionEnabled(LogVersionsWithTryRunV1)) {
+    log.BeginEvent("try_run-v1");
+    log.WriteBacktrace(mf);
+    cmCoreTryCompile::WriteTryCompileEventFields(log, compileResult);
+
+    log.BeginObject("runResult"_s);
+    log.WriteValue("variable"_s, runResult.Variable);
+    log.WriteValue("cached"_s, runResult.VariableCached);
+    if (runResult.Stdout) {
+      log.WriteLiteralTextBlock("stdout"_s, *runResult.Stdout);
+    }
+    if (runResult.Stderr) {
+      log.WriteLiteralTextBlock("stderr"_s, *runResult.Stderr);
+    }
+    if (runResult.ExitCode) {
+      try {
+        log.WriteValue("exitCode"_s, std::stoi(*runResult.ExitCode));
+      } catch (std::invalid_argument const&) {
+        log.WriteValue("exitCode"_s, *runResult.ExitCode);
+      }
     }
+    log.EndObject();
+    log.EndEvent();
   }
-  log.EndObject();
-  log.EndEvent();
 }
 #endif
 

Source/cmake.cxx

@@ -2428,7 +2428,8 @@ int cmake::ActualConfigure()
   if (!this->GetIsInTryCompile()) {
     this->TruncateOutputLog("CMakeConfigureLog.yaml");
     this->ConfigureLog = cm::make_unique<cmConfigureLog>(
-      cmStrCat(this->GetHomeOutputDirectory(), "/CMakeFiles"_s));
+      cmStrCat(this->GetHomeOutputDirectory(), "/CMakeFiles"_s),
+      std::vector<unsigned long>());
   }
 #endif
 

Tests/RunCMake/try_compile/Inspect-config.txt

@@ -1,11 +1,8 @@
 ^
 ---
-version:
-  major: 1
-  minor: 0
 events:
   -
-    kind: "try_compile"
+    kind: "try_compile-v1"
     backtrace:
       - "[^"]*/Modules/CMakeDetermineCompilerABI.cmake:[0-9]+ \(try_compile\)"
       - "[^"]*/Modules/CMakeTestCCompiler.cmake:[0-9]+ \(CMAKE_DETERMINE_COMPILER_ABI\)"
@@ -20,7 +17,7 @@ events:
       stdout: \|.*
       exitCode: 0
   -
-    kind: "try_compile"
+    kind: "try_compile-v1"
     backtrace:
       - "[^"]*/Modules/CMakeDetermineCompilerABI.cmake:[0-9]+ \(try_compile\)"
       - "[^"]*/Modules/CMakeTestCXXCompiler.cmake:[0-9]+ \(CMAKE_DETERMINE_COMPILER_ABI\)"

Tests/RunCMake/try_run/ConfigureLog-config.txt

@@ -1,11 +1,8 @@
 ^
 ---
-version:
-  major: 1
-  minor: 0
 events:
   -
-    kind: "try_compile"
+    kind: "try_compile-v1"
     backtrace:
       - "[^"]*/Modules/CMakeDetermineCompilerABI.cmake:[0-9]+ \(try_compile\)"
       - "[^"]*/Modules/CMakeTestCCompiler.cmake:[0-9]+ \(CMAKE_DETERMINE_COMPILER_ABI\)"
@@ -19,7 +16,7 @@ events:
       stdout: \|.*
       exitCode: 0
   -
-    kind: "try_run"
+    kind: "try_run-v1"
     backtrace:
       - "ConfigureLog.cmake:[0-9]+ \(try_run\)"
       - "CMakeLists.txt:[0-9]+ \(include\)"
@@ -35,7 +32,7 @@ events:
       variable: "RUN_RESULT"
       cached: true
   -
-    kind: "try_run"
+    kind: "try_run-v1"
     backtrace:
       - "ConfigureLog.cmake:[0-9]+ \(try_run\)"
       - "CMakeLists.txt:[0-9]+ \(include\)"
@@ -56,7 +53,7 @@ events:
         Output, with backslash '\\\\', on stderr!
       exitCode: 12
   -
-    kind: "try_run"
+    kind: "try_run-v1"
     backtrace:
       - "ConfigureLog.cmake:[0-9]+ \(try_run\)"
       - "CMakeLists.txt:[0-9]+ \(include\)"
@@ -76,7 +73,7 @@ events:
         Output on stdout!
       exitCode: 12
   -
-    kind: "try_run"
+    kind: "try_run-v1"
     backtrace:
       - "ConfigureLog.cmake:[0-9]+ \(try_run\)"
       - "CMakeLists.txt:[0-9]+ \(include\)"