server-mode: Report information relevant for a codemodel

Add "codemodel" command to report information relevant to feed a code
model.

Help/manual/cmake-server.7.rst

@@ -374,3 +374,187 @@ CMake will reply (after reporting progress information)::
   [== CMake Server ==[
   {"cookie":"","inReplyTo":"compute","type":"reply"}
   ]== CMake Server ==]
+
+
+Type "codemodel"
+^^^^^^^^^^^^^^^^
+
+The "codemodel" request can be used after a project was "compute"d successfully.
+
+It will list the complete project structure as it is known to cmake.
+
+The reply will contain a key "projects", which will contain a list of
+project objects, one for each (sub-)project defined in the cmake build system.
+
+Each project object can have the following keys:
+
+"name"
+  contains the (sub-)projects name.
+"sourceDirectory"
+  contains the current source directory
+"buildDirectory"
+  contains the current build directory.
+"configurations"
+  contains a list of configuration objects.
+
+Configuration objects are used to destinquish between different
+configurations the build directory might have enabled. While most generators
+only support one configuration, others support several.
+
+Each configuration object can have the following keys:
+
+"name"
+  contains the name of the configuration. The name may be empty.
+"targets"
+  contains a list of target objects, one for each build target.
+
+Target objects define individual build targets for a certain configuration.
+
+Each target object can have the following keys:
+
+"name"
+  contains the name of the target.
+"type"
+  defines the type of build of the target. Possible values are
+  "STATIC_LIBRARY", "MODULE_LIBRARY", "SHARED_LIBRARY", "OBJECT_LIBRARY",
+  "EXECUTABLE", "UTILITY" and "INTERFACE_LIBRARY".
+"fullName"
+  contains the full name of the build result (incl. extensions, etc.).
+"sourceDirectory"
+  contains the current source directory.
+"buildDirectory"
+  contains the current build directory.
+"artifacts"
+  with a list of build artifacts. The list is sorted with the most
+  important artifacts first (e.g. a .DLL file is listed before a
+  .PDB file on windows).
+"linkerLanguage"
+  contains the language of the linker used to produce the artifact.
+"linkLibraries"
+  with a list of libraries to link to. This value is encoded in the
+  system's native shell format.
+"linkFlags"
+  with a list of flags to pass to the linker. This value is encoded in
+  the system's native shell format.
+"linkLanguageFlags"
+  with the flags for a compiler using the linkerLanguage. This value is
+  encoded in the system's native shell format.
+"frameworkPath"
+  with the framework path (on Apple computers). This value is encoded
+  in the system's native shell format.
+"linkPath"
+  with the link path. This value is encoded in the system's native shell
+  format.
+"sysroot"
+  with the sysroot path.
+"fileGroups"
+  contains the source files making up the target.
+
+FileGroups are used to group sources using similar settings together.
+
+Each fileGroup object may contain the following keys:
+
+"language"
+  contains the programming language used by all files in the group.
+"compileFlags"
+  with a string containing all the flags passed to the compiler
+  when building any of the files in this group. This value is encoded in
+  the system's native shell format.
+"includePath"
+  with a list of include paths. Each include path is an object
+  containing a "path" with the actual include path and "isSystem" with a bool
+  value informing whether this is a normal include or a system include. This
+  value is encoded in the system's native shell format.
+"defines"
+  with a list of defines in the form "SOMEVALUE" or "SOMEVALUE=42". This
+  value is encoded in the system's native shell format.
+"sources"
+  with a list of source files.
+
+All file paths in the fileGroup are either absolute or relative to the
+sourceDirectory of the target.
+
+Example::
+
+  [== CMake Server ==[
+  {"type":"project"}
+  ]== CMake Server ==]
+
+CMake will reply::
+
+  [== CMake Server ==[
+  {
+    "cookie":"",
+    "type":"reply",
+    "inReplyTo":"project",
+
+    "projects":
+    [
+      {
+        "name":"CMAKE_FORM",
+        "sourceDirectory":"/home/code/src/cmake/Source/CursesDialog/form"
+        "buildDirectory":"/tmp/cmake-build-test/Source/CursesDialog/form",
+        "configurations":
+        [
+          {
+            "name":"",
+            "targets":
+            [
+              {
+                "artifactDirectory":"/tmp/cmake/Source/CursesDialog/form",
+                "fileGroups":
+                [
+                  {
+                    "compileFlags":"  -std=gnu11",
+                    "defines":
+                    [
+                      "SOMETHING=1",
+                      "LIBARCHIVE_STATIC"
+                    ],
+                    "includePath":
+                    [
+                      { "path":"/tmp/cmake-build-test/Utilities" },
+                      { "isSystem": true, "path":"/usr/include/something" },
+                      ...
+                    ]
+                    "language":"C",
+                    "sources":
+                    [
+                      "fld_arg.c",
+                      ...
+                      "fty_regex.c"
+                    ]
+                  }
+                ],
+                "fullName":"libcmForm.a",
+                "linkerLanguage":"C",
+                "name":"cmForm",
+                "type":"STATIC_LIBRARY"
+              }
+            ]
+          }
+        ],
+      },
+      ...
+    ]
+  }
+  ]== CMake Server ==]
+
+The output can be tailored to the specific needs via parameter passed when
+requesting "project" information.
+
+You can have a "depth" key, which accepts "project", "configuration" and
+"target" as string values. These cause the output to be trimmed at the
+appropriate depth of the output tree.
+
+You can also set "configurations" to an array of strings with configuration
+names to list. This will cause any configuration that is not listed to be
+trimmed from the output.
+
+Generated files can be included in the listing by setting "includeGeneratedFiles"
+to "true". This setting defaults to "false", so generated files are not
+listed by default.
+
+Finally you can limit the target types that are going to be listed. This is
+done by providing a list of target types as an array of strings to the
+"targetTypes" key.

Source/cmServerDictionary.h

@@ -6,6 +6,7 @@
 
 // Vocabulary:
 
+static const std::string kCODE_MODEL_TYPE = "codemodel";
 static const std::string kCOMPUTE_TYPE = "compute";
 static const std::string kCONFIGURE_TYPE = "configure";
 static const std::string kERROR_TYPE = "error";
@@ -17,29 +18,50 @@ static const std::string kREPLY_TYPE = "reply";
 static const std::string kSET_GLOBAL_SETTINGS_TYPE = "setGlobalSettings";
 static const std::string kSIGNAL_TYPE = "signal";
 
+static const std::string kARTIFACTS_KEY = "artifacts";
 static const std::string kBUILD_DIRECTORY_KEY = "buildDirectory";
 static const std::string kCACHE_ARGUMENTS_KEY = "cacheArguments";
 static const std::string kCAPABILITIES_KEY = "capabilities";
 static const std::string kCHECK_SYSTEM_VARS_KEY = "checkSystemVars";
+static const std::string kCOMPILE_FLAGS_KEY = "compileFlags";
+static const std::string kCONFIGURATIONS_KEY = "configurations";
 static const std::string kCOOKIE_KEY = "cookie";
 static const std::string kDEBUG_OUTPUT_KEY = "debugOutput";
+static const std::string kDEFINES_KEY = "defines";
 static const std::string kERROR_MESSAGE_KEY = "errorMessage";
 static const std::string kEXTRA_GENERATOR_KEY = "extraGenerator";
+static const std::string kFILE_GROUPS_KEY = "fileGroups";
+static const std::string kFRAMEWORK_PATH_KEY = "frameworkPath";
+static const std::string kFULL_NAME_KEY = "fullName";
 static const std::string kGENERATOR_KEY = "generator";
+static const std::string kINCLUDE_PATH_KEY = "includePath";
 static const std::string kIS_EXPERIMENTAL_KEY = "isExperimental";
+static const std::string kIS_GENERATED_KEY = "isGenerated";
+static const std::string kIS_SYSTEM_KEY = "isSystem";
+static const std::string kLANGUAGE_KEY = "language";
+static const std::string kLINKER_LANGUAGE_KEY = "linkerLanguage";
+static const std::string kLINK_FLAGS_KEY = "linkFlags";
+static const std::string kLINK_LANGUAGE_FLAGS_KEY = "linkLanguageFlags";
+static const std::string kLINK_LIBRARIES_KEY = "linkLibraries";
+static const std::string kLINK_PATH_KEY = "linkPath";
 static const std::string kMAJOR_KEY = "major";
 static const std::string kMESSAGE_KEY = "message";
 static const std::string kMINOR_KEY = "minor";
 static const std::string kNAME_KEY = "name";
+static const std::string kPATH_KEY = "path";
 static const std::string kPROGRESS_CURRENT_KEY = "progressCurrent";
 static const std::string kPROGRESS_MAXIMUM_KEY = "progressMaximum";
 static const std::string kPROGRESS_MESSAGE_KEY = "progressMessage";
 static const std::string kPROGRESS_MINIMUM_KEY = "progressMinimum";
+static const std::string kPROJECTS_KEY = "projects";
 static const std::string kPROTOCOL_VERSION_KEY = "protocolVersion";
 static const std::string kREPLY_TO_KEY = "inReplyTo";
 static const std::string kSOURCE_DIRECTORY_KEY = "sourceDirectory";
+static const std::string kSOURCES_KEY = "sources";
 static const std::string kSUPPORTED_PROTOCOL_VERSIONS =
   "supportedProtocolVersions";
+static const std::string kSYSROOT_KEY = "sysroot";
+static const std::string kTARGETS_KEY = "targets";
 static const std::string kTITLE_KEY = "title";
 static const std::string kTRACE_EXPAND_KEY = "traceExpand";
 static const std::string kTRACE_KEY = "trace";

Source/cmServerProtocol.cxx

@@ -3,9 +3,13 @@
 #include "cmServerProtocol.h"
 
 #include "cmExternalMakefileProjectGenerator.h"
+#include "cmGeneratorTarget.h"
 #include "cmGlobalGenerator.h"
+#include "cmLocalGenerator.h"
+#include "cmMakefile.h"
 #include "cmServer.h"
 #include "cmServerDictionary.h"
+#include "cmSourceFile.h"
 #include "cmSystemTools.h"
 #include "cmake.h"
 
@@ -16,6 +20,49 @@
 #include "cm_jsoncpp_value.h"
 #endif
 
+#include <algorithm>
+#include <string>
+#include <vector>
+
+// Get rid of some windows macros:
+#undef max
+
+namespace {
+
+static std::vector<std::string> getConfigurations(const cmake* cm)
+{
+  std::vector<std::string> configurations;
+  auto makefiles = cm->GetGlobalGenerator()->GetMakefiles();
+  if (makefiles.empty()) {
+    return configurations;
+  }
+
+  makefiles[0]->GetConfigurations(configurations);
+  if (configurations.empty())
+    configurations.push_back("");
+  return configurations;
+}
+
+static bool hasString(const Json::Value& v, const std::string& s)
+{
+  return !v.isNull() &&
+    std::find_if(v.begin(), v.end(), [s](const Json::Value& i) {
+      return i.asString() == s;
+    }) != v.end();
+}
+
+template <class T>
+static Json::Value fromStringList(const T& in)
+{
+  Json::Value result = Json::arrayValue;
+  for (const std::string& i : in) {
+    result.append(i);
+  }
+  return result;
+}
+
+} // namespace
+
 cmServerRequest::cmServerRequest(cmServer* server, const std::string& t,
                                  const std::string& c, const Json::Value& d)
   : Type(t)
@@ -270,6 +317,9 @@ const cmServerResponse cmServerProtocol1_0::Process(
 {
   assert(this->m_State >= STATE_ACTIVE);
 
+  if (request.Type == kCODE_MODEL_TYPE) {
+    return this->ProcessCodeModel(request);
+  }
   if (request.Type == kCOMPUTE_TYPE) {
     return this->ProcessCompute(request);
   }
@@ -291,6 +341,342 @@ bool cmServerProtocol1_0::IsExperimental() const
   return true;
 }
 
+class LanguageData
+{
+public:
+  bool operator==(const LanguageData& other) const;
+
+  void SetDefines(const std::set<std::string>& defines);
+
+  bool IsGenerated = false;
+  std::string Language;
+  std::string Flags;
+  std::vector<std::string> Defines;
+  std::vector<std::pair<std::string, bool> > IncludePathList;
+};
+
+bool LanguageData::operator==(const LanguageData& other) const
+{
+  return Language == other.Language && Defines == other.Defines &&
+    Flags == other.Flags && IncludePathList == other.IncludePathList &&
+    IsGenerated == other.IsGenerated;
+}
+
+void LanguageData::SetDefines(const std::set<std::string>& defines)
+{
+  std::vector<std::string> result;
+  for (auto i : defines) {
+    result.push_back(i);
+  }
+  std::sort(result.begin(), result.end());
+  Defines = result;
+}
+
+namespace std {
+
+template <>
+struct hash<LanguageData>
+{
+  std::size_t operator()(const LanguageData& in) const
+  {
+    using std::hash;
+    size_t result =
+      hash<std::string>()(in.Language) ^ hash<std::string>()(in.Flags);
+    for (auto i : in.IncludePathList) {
+      result = result ^ (hash<std::string>()(i.first) ^
+                         (i.second ? std::numeric_limits<size_t>::max() : 0));
+    }
+    for (auto i : in.Defines) {
+      result = result ^ hash<std::string>()(i);
+    }
+    result =
+      result ^ (in.IsGenerated ? std::numeric_limits<size_t>::max() : 0);
+    return result;
+  }
+};
+
+} // namespace std
+
+static Json::Value DumpSourceFileGroup(const LanguageData& data,
+                                       const std::vector<std::string>& files,
+                                       const std::string& baseDir)
+{
+  Json::Value result = Json::objectValue;
+
+  if (!data.Language.empty()) {
+    result[kLANGUAGE_KEY] = data.Language;
+    if (!data.Flags.empty()) {
+      result[kCOMPILE_FLAGS_KEY] = data.Flags;
+    }
+    if (!data.IncludePathList.empty()) {
+      Json::Value includes = Json::arrayValue;
+      for (auto i : data.IncludePathList) {
+        Json::Value tmp = Json::objectValue;
+        tmp[kPATH_KEY] = i.first;
+        if (i.second) {
+          tmp[kIS_SYSTEM_KEY] = i.second;
+        }
+        includes.append(tmp);
+      }
+      result[kINCLUDE_PATH_KEY] = includes;
+    }
+    if (!data.Defines.empty()) {
+      result[kDEFINES_KEY] = fromStringList(data.Defines);
+    }
+  }
+
+  result[kIS_GENERATED_KEY] = data.IsGenerated;
+
+  Json::Value sourcesValue = Json::arrayValue;
+  for (auto i : files) {
+    const std::string relPath =
+      cmSystemTools::RelativePath(baseDir.c_str(), i.c_str());
+    sourcesValue.append(relPath.size() < i.size() ? relPath : i);
+  }
+
+  result[kSOURCES_KEY] = sourcesValue;
+  return result;
+}
+
+static Json::Value DumpSourceFilesList(
+  cmGeneratorTarget* target, const std::string& config,
+  const std::map<std::string, LanguageData>& languageDataMap)
+{
+  // Collect sourcefile groups:
+
+  std::vector<cmSourceFile*> files;
+  target->GetSourceFiles(files, config);
+
+  std::unordered_map<LanguageData, std::vector<std::string> > fileGroups;
+  for (cmSourceFile* file : files) {
+    LanguageData fileData;
+    fileData.Language = file->GetLanguage();
+    if (!fileData.Language.empty()) {
+      const LanguageData& ld = languageDataMap.at(fileData.Language);
+      cmLocalGenerator* lg = target->GetLocalGenerator();
+
+      std::string compileFlags = ld.Flags;
+      lg->AppendFlags(compileFlags, file->GetProperty("COMPILE_FLAGS"));
+      fileData.Flags = compileFlags;
+
+      fileData.IncludePathList = ld.IncludePathList;
+
+      std::set<std::string> defines;
+      lg->AppendDefines(defines, file->GetProperty("COMPILE_DEFINITIONS"));
+      const std::string defPropName =
+        "COMPILE_DEFINITIONS_" + cmSystemTools::UpperCase(config);
+      lg->AppendDefines(defines, file->GetProperty(defPropName));
+      defines.insert(ld.Defines.begin(), ld.Defines.end());
+
+      fileData.SetDefines(defines);
+    }
+
+    fileData.IsGenerated = file->GetPropertyAsBool("GENERATED");
+    std::vector<std::string>& groupFileList = fileGroups[fileData];
+    groupFileList.push_back(file->GetFullPath());
+  }
+
+  const std::string baseDir = target->Makefile->GetCurrentSourceDirectory();
+  Json::Value result = Json::arrayValue;
+  for (auto it = fileGroups.begin(); it != fileGroups.end(); ++it) {
+    Json::Value group = DumpSourceFileGroup(it->first, it->second, baseDir);
+    if (!group.isNull())
+      result.append(group);
+  }
+
+  return result;
+}
+
+static Json::Value DumpTarget(cmGeneratorTarget* target,
+                              const std::string& config)
+{
+  cmLocalGenerator* lg = target->GetLocalGenerator();
+  const cmState* state = lg->GetState();
+
+  const cmState::TargetType type = target->GetType();
+  const std::string typeName = state->GetTargetTypeName(type);
+
+  Json::Value ttl = Json::arrayValue;
+  ttl.append("EXECUTABLE");
+  ttl.append("STATIC_LIBRARY");
+  ttl.append("SHARED_LIBRARY");
+  ttl.append("MODULE_LIBRARY");
+  ttl.append("OBJECT_LIBRARY");
+  ttl.append("UTILITY");
+  ttl.append("INTERFACE_LIBRARY");
+
+  if (!hasString(ttl, typeName) || target->IsImported()) {
+    return Json::Value();
+  }
+
+  Json::Value result = Json::objectValue;
+  result[kNAME_KEY] = target->GetName();
+
+  result[kTYPE_KEY] = typeName;
+  result[kFULL_NAME_KEY] = target->GetFullName(config);
+  result[kSOURCE_DIRECTORY_KEY] = lg->GetCurrentSourceDirectory();
+  result[kBUILD_DIRECTORY_KEY] = lg->GetCurrentBinaryDirectory();
+
+  if (target->HaveWellDefinedOutputFiles()) {
+    Json::Value artifacts = Json::arrayValue;
+    artifacts.append(target->GetFullPath(config, false));
+    if (target->IsDLLPlatform()) {
+      artifacts.append(target->GetFullPath(config, true));
+      const cmGeneratorTarget::OutputInfo* output =
+        target->GetOutputInfo(config);
+      if (output && !output->PdbDir.empty()) {
+        artifacts.append(output->PdbDir + '/' + target->GetPDBName(config));
+      }
+    }
+    result[kARTIFACTS_KEY] = artifacts;
+
+    result[kLINKER_LANGUAGE_KEY] = target->GetLinkerLanguage(config);
+
+    std::string linkLibs;
+    std::string linkFlags;
+    std::string linkLanguageFlags;
+    std::string frameworkPath;
+    std::string linkPath;
+    lg->GetTargetFlags(config, linkLibs, linkLanguageFlags, linkFlags,
+                       frameworkPath, linkPath, target, false);
+
+    linkLibs = cmSystemTools::TrimWhitespace(linkLibs);
+    linkFlags = cmSystemTools::TrimWhitespace(linkFlags);
+    linkLanguageFlags = cmSystemTools::TrimWhitespace(linkLanguageFlags);
+    frameworkPath = cmSystemTools::TrimWhitespace(frameworkPath);
+    linkPath = cmSystemTools::TrimWhitespace(linkPath);
+
+    if (!cmSystemTools::TrimWhitespace(linkLibs).empty()) {
+      result[kLINK_LIBRARIES_KEY] = linkLibs;
+    }
+    if (!cmSystemTools::TrimWhitespace(linkFlags).empty()) {
+      result[kLINK_FLAGS_KEY] = linkFlags;
+    }
+    if (!cmSystemTools::TrimWhitespace(linkLanguageFlags).empty()) {
+      result[kLINK_LANGUAGE_FLAGS_KEY] = linkLanguageFlags;
+    }
+    if (!frameworkPath.empty()) {
+      result[kFRAMEWORK_PATH_KEY] = frameworkPath;
+    }
+    if (!linkPath.empty()) {
+      result[kLINK_PATH_KEY] = linkPath;
+    }
+    const std::string sysroot =
+      lg->GetMakefile()->GetSafeDefinition("CMAKE_SYSROOT");
+    if (!sysroot.empty()) {
+      result[kSYSROOT_KEY] = sysroot;
+    }
+  }
+
+  std::set<std::string> languages;
+  target->GetLanguages(languages, config);
+  std::map<std::string, LanguageData> languageDataMap;
+  for (auto lang : languages) {
+    LanguageData& ld = languageDataMap[lang];
+    ld.Language = lang;
+    lg->GetTargetCompileFlags(target, config, lang, ld.Flags);
+    std::set<std::string> defines;
+    lg->GetTargetDefines(target, config, lang, defines);
+    ld.SetDefines(defines);
+    std::vector<std::string> includePathList;
+    lg->GetIncludeDirectories(includePathList, target, lang, config, true);
+    for (auto i : includePathList) {
+      ld.IncludePathList.push_back(
+        std::make_pair(i, target->IsSystemIncludeDirectory(i, config)));
+    }
+  }
+
+  Json::Value sourceGroupsValue =
+    DumpSourceFilesList(target, config, languageDataMap);
+  if (!sourceGroupsValue.empty()) {
+    result[kFILE_GROUPS_KEY] = sourceGroupsValue;
+  }
+
+  return result;
+}
+
+static Json::Value DumpTargetsList(
+  const std::vector<cmLocalGenerator*>& generators, const std::string& config)
+{
+  Json::Value result = Json::arrayValue;
+
+  std::vector<cmGeneratorTarget*> targetList;
+  for (const auto& lgIt : generators) {
+    auto list = lgIt->GetGeneratorTargets();
+    targetList.insert(targetList.end(), list.begin(), list.end());
+  }
+  std::sort(targetList.begin(), targetList.end());
+
+  for (cmGeneratorTarget* target : targetList) {
+    Json::Value tmp = DumpTarget(target, config);
+    if (!tmp.isNull()) {
+      result.append(tmp);
+    }
+  }
+
+  return result;
+}
+
+static Json::Value DumpProjectList(const cmake* cm, const std::string config)
+{
+  Json::Value result = Json::arrayValue;
+
+  auto globalGen = cm->GetGlobalGenerator();
+
+  for (const auto& projectIt : globalGen->GetProjectMap()) {
+    Json::Value pObj = Json::objectValue;
+    pObj[kNAME_KEY] = projectIt.first;
+
+    assert(projectIt.second.size() >
+           0); // All Projects must have at least one local generator
+    const cmLocalGenerator* lg = projectIt.second.at(0);
+
+    // Project structure information:
+    const cmMakefile* mf = lg->GetMakefile();
+    pObj[kSOURCE_DIRECTORY_KEY] = mf->GetCurrentSourceDirectory();
+    pObj[kBUILD_DIRECTORY_KEY] = mf->GetCurrentBinaryDirectory();
+    pObj[kTARGETS_KEY] = DumpTargetsList(projectIt.second, config);
+
+    result.append(pObj);
+  }
+
+  return result;
+}
+
+static Json::Value DumpConfiguration(const cmake* cm,
+                                     const std::string& config)
+{
+  Json::Value result = Json::objectValue;
+  result[kNAME_KEY] = config;
+
+  result[kPROJECTS_KEY] = DumpProjectList(cm, config);
+
+  return result;
+}
+
+static Json::Value DumpConfigurationsList(const cmake* cm)
+{
+  Json::Value result = Json::arrayValue;
+
+  for (const std::string& c : getConfigurations(cm)) {
+    result.append(DumpConfiguration(cm, c));
+  }
+
+  return result;
+}
+
+cmServerResponse cmServerProtocol1_0::ProcessCodeModel(
+  const cmServerRequest& request)
+{
+  if (this->m_State != STATE_COMPUTED) {
+    return request.ReportError("No build system was generated yet.");
+  }
+
+  Json::Value result = Json::objectValue;
+  result[kCONFIGURATIONS_KEY] = DumpConfigurationsList(this->CMakeInstance());
+  return request.Reply(result);
+}
+
 cmServerResponse cmServerProtocol1_0::ProcessCompute(
   const cmServerRequest& request)
 {

Source/cmServerProtocol.h

@@ -108,6 +108,7 @@ private:
                   std::string* errorMessage) override;
 
   // Handle requests:
+  cmServerResponse ProcessCodeModel(const cmServerRequest& request);
   cmServerResponse ProcessCompute(const cmServerRequest& request);
   cmServerResponse ProcessConfigure(const cmServerRequest& request);
   cmServerResponse ProcessGlobalSettings(const cmServerRequest& request);