Merge topic 'submit-url'

2915a75615 CTest: Add documentation and release notes for SubmitURL
d6475daa79 Modules/CTest: Set SubmitURL
938f06fda6 ctest_submit: Add parameter SUBMIT_URL
65e725c957 CTest: Add option SubmitURL
65f1fc9d63 CTest: Add function GetSubmitURL
2bedd5fb7c ctest_submit: Remove submit method from log output

Acked-by: Kitware Robot <[email protected]>
Merge-request: !2719

Help/command/ctest_submit.rst

@@ -6,6 +6,7 @@ Perform the :ref:`CTest Submit Step` as a :ref:`Dashboard Client`.
 ::
 
   ctest_submit([PARTS <part>...] [FILES <file>...]
+               [SUBMIT_URL <url>]
                [HTTPHEADER <header>]
                [RETRY_COUNT <count>]
                [RETRY_DELAY <delay>]
@@ -39,6 +40,10 @@ The options are:
   Specify an explicit list of specific files to be submitted.
   Each individual file must exist at the time of the call.
 
+``SUBMIT_URL <url>``
+  The ``http`` or ``https`` URL of the dashboard server to send the submission
+  to.  If not given, the :variable:`CTEST_SUBMIT_URL` variable is used.
+
 ``HTTPHEADER <HTTP-header>``
   Specify HTTP header to be included in the request to CDash during submission.
   This suboption can be repeated several times.
@@ -68,6 +73,7 @@ Submit to CDash Upload API
 ::
 
   ctest_submit(CDASH_UPLOAD <file> [CDASH_UPLOAD_TYPE <type>]
+               [SUBMIT_URL <url>]
                [HTTPHEADER <header>]
                [RETRY_COUNT <count>]
                [RETRY_DELAY <delay>]
@@ -80,5 +86,5 @@ with a content hash of the file. If CDash does not already have the file,
 then it is uploaded. Along with the file, a CDash type string is specified
 to tell CDash which handler to use to process the data.
 
-This signature accepts the ``HTTPHEADER``, ``RETRY_COUNT``, ``RETRY_DELAY``,
-``RETURN_VALUE`` and ``QUIET`` options as described above.
+This signature accepts the ``SUBMIT_URL``, ``HTTPHEADER``, ``RETRY_COUNT``,
+``RETRY_DELAY``, ``RETURN_VALUE`` and ``QUIET`` options as described above.

Help/manual/cmake-variables.7.rst

@@ -569,6 +569,7 @@ Variables for CTest
    /variable/CTEST_RUN_CURRENT_SCRIPT
    /variable/CTEST_SCP_COMMAND
    /variable/CTEST_SITE
+   /variable/CTEST_SUBMIT_URL
    /variable/CTEST_SOURCE_DIRECTORY
    /variable/CTEST_SVN_COMMAND
    /variable/CTEST_SVN_OPTIONS

Help/manual/ctest.1.rst

@@ -1109,38 +1109,45 @@ Configuration settings include:
   * :module:`CTest` module variable: ``CTEST_CURL_OPTIONS``
 
 ``DropLocation``
-  The path on the dashboard server to send the submission.
+  Legacy option.  When ``SubmitURL`` is not set, it is constructed from
+  ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+  ``DropLocation``.
 
   * `CTest Script`_ variable: :variable:`CTEST_DROP_LOCATION`
   * :module:`CTest` module variable: ``DROP_LOCATION`` if set,
     else ``CTEST_DROP_LOCATION``
 
 ``DropMethod``
-  Specify the method by which results should be submitted to the
-  dashboard server.  The value may be ``http`` or ``https``.
+  Legacy option.  When ``SubmitURL`` is not set, it is constructed from
+  ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+  ``DropLocation``.
 
   * `CTest Script`_ variable: :variable:`CTEST_DROP_METHOD`
   * :module:`CTest` module variable: ``DROP_METHOD`` if set,
     else ``CTEST_DROP_METHOD``
 
 ``DropSite``
-  The dashboard server name.
+  Legacy option.  When ``SubmitURL`` is not set, it is constructed from
+  ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+  ``DropLocation``.
 
   * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE`
   * :module:`CTest` module variable: ``DROP_SITE`` if set,
     else ``CTEST_DROP_SITE``
 
 ``DropSitePassword``
-  The dashboard server login password, if any
-  (for ``ftp``, ``http``, and ``https``).
+  Legacy option.  When ``SubmitURL`` is not set, it is constructed from
+  ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+  ``DropLocation``.
 
   * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE_PASSWORD`
   * :module:`CTest` module variable: ``DROP_SITE_PASSWORD`` if set,
     else ``CTEST_DROP_SITE_PASWORD``
 
 ``DropSiteUser``
-  The dashboard server login user name, if any
-  (for ``ftp``, ``http``, and ``https``).
+  Legacy option.  When ``SubmitURL`` is not set, it is constructed from
+  ``DropMethod``, ``DropSiteUser``, ``DropSitePassword``, ``DropSite``, and
+  ``DropLocation``.
 
   * `CTest Script`_ variable: :variable:`CTEST_DROP_SITE_USER`
   * :module:`CTest` module variable: ``DROP_SITE_USER`` if set,
@@ -1166,6 +1173,14 @@ Configuration settings include:
   * :module:`CTest` module variable: ``SITE``,
     initialized by the :command:`site_name` command
 
+``SubmitURL``
+  The ``http`` or ``https`` URL of the dashboard server to send the submission
+  to.
+
+  * `CTest Script`_ variable: :variable:`CTEST_SUBMIT_URL`
+  * :module:`CTest` module variable: ``SUBMIT_URL`` if set,
+    else ``CTEST_SUBMIT_URL``
+
 ``TriggerSite``
   Legacy option.  Not used.
 

Help/release/dev/ctest-submit-url.rst

@@ -0,0 +1,7 @@
+ctest-submit-url
+----------------
+
+* CTest learned to accept the dashboard server submission URL from a single
+  variable.  See the ``SubmitURL`` setting in :manual:`ctest(1)`,
+  the :variable:`CTEST_SUBMIT_URL` variable, and the ``SUBMIT_URL``
+  argument of the :command:`ctest_submit` command.

Help/variable/CTEST_SUBMIT_URL.rst

@@ -0,0 +1,5 @@
+CTEST_SUBMIT_URL
+----------------
+
+Specify the CTest ``SubmitURL`` setting
+in a :manual:`ctest(1)` dashboard client script.

Modules/CTest.cmake

@@ -28,10 +28,7 @@ To enable submissions to a CDash server, create a ``CTestConfig.cmake``
 file at the top of the project with content such as::
 
   set(CTEST_NIGHTLY_START_TIME "01:00:00 UTC")
-  set(CTEST_DROP_METHOD "http")
-  set(CTEST_DROP_SITE "my.cdash.org")
-  set(CTEST_DROP_LOCATION "/submit.php?project=MyProject")
-  set(CTEST_DROP_SITE_CDASH TRUE)
+  set(CTEST_SUBMIT_URL "http://my.cdash.org/submit.php?project=MyProject")
 
 (the CDash server can provide the file to a project administrator who
 configures ``MyProject``).  Settings in the config file are shared by
@@ -89,6 +86,7 @@ if(BUILD_TESTING)
   if(EXISTS "${PROJECT_SOURCE_DIR}/CTestConfig.cmake")
     include("${PROJECT_SOURCE_DIR}/CTestConfig.cmake")
     SET_IF_SET_AND_NOT_SET(NIGHTLY_START_TIME "${CTEST_NIGHTLY_START_TIME}")
+    SET_IF_SET_AND_NOT_SET(SUBMIT_URL "${CTEST_SUBMIT_URL}")
     SET_IF_SET_AND_NOT_SET(DROP_METHOD "${CTEST_DROP_METHOD}")
     SET_IF_SET_AND_NOT_SET(DROP_SITE "${CTEST_DROP_SITE}")
     SET_IF_SET_AND_NOT_SET(DROP_SITE_USER "${CTEST_DROP_SITE_USER}")
@@ -111,6 +109,18 @@ if(BUILD_TESTING)
   endif()
   SET_IF_NOT_SET (NIGHTLY_START_TIME "00:00:00 EDT")
 
+  if(NOT SUBMIT_URL)
+    set(SUBMIT_URL "${DROP_METHOD}://")
+    if(DROP_SITE_USER)
+      string(APPEND SUBMIT_URL "${DROP_SITE_USER}")
+      if(DROP_SITE_PASSWORD)
+        string(APPEND SUBMIT_URL ":${DROP_SITE_PASSWORD}")
+      endif()
+      string(APPEND SUBMIT_URL "@")
+    endif()
+    string(APPEND SUBMIT_URL "${DROP_SITE}${DROP_SITE_LOCATION}")
+  endif()
+
   find_program(CVSCOMMAND cvs )
   set(CVS_UPDATE_OPTIONS "-d -A -P" CACHE STRING
     "Options passed to the cvs update command.")

Modules/DartConfiguration.tcl.in

@@ -20,12 +20,7 @@ BuildName: @BUILDNAME@
 LabelsForSubprojects: @CTEST_LABELS_FOR_SUBPROJECTS@
 
 # Submission information
-DropSite: @DROP_SITE@
-DropLocation: @DROP_LOCATION@
-DropSiteUser: @DROP_SITE_USER@
-DropSitePassword: @DROP_SITE_PASSWORD@
-DropSiteMode: @DROP_SITE_MODE@
-DropMethod: @DROP_METHOD@
+SubmitURL: @SUBMIT_URL@
 
 # Dashboard start time
 NightlyStartTime: @NIGHTLY_START_TIME@

Source/CTest/cmCTestSubmitCommand.cxx

@@ -15,36 +15,28 @@ class cmExecutionStatus;
 
 cmCTestGenericHandler* cmCTestSubmitCommand::InitializeHandler()
 {
-  const char* ctestDropMethod =
-    this->Makefile->GetDefinition("CTEST_DROP_METHOD");
-  const char* ctestDropSite = this->Makefile->GetDefinition("CTEST_DROP_SITE");
-  const char* ctestDropLocation =
-    this->Makefile->GetDefinition("CTEST_DROP_LOCATION");
-  if (!ctestDropMethod) {
-    ctestDropMethod = "http";
-  }
+  const char* submitURL = !this->SubmitURL.empty()
+    ? this->SubmitURL.c_str()
+    : this->Makefile->GetDefinition("CTEST_SUBMIT_URL");
 
-  if (!ctestDropSite) {
-    // error: CDash requires CTEST_DROP_SITE definition
-    // in CTestConfig.cmake
-  }
-  if (!ctestDropLocation) {
-    // error: CDash requires CTEST_DROP_LOCATION definition
-    // in CTestConfig.cmake
+  if (submitURL) {
+    this->CTest->SetCTestConfiguration("SubmitURL", submitURL, this->Quiet);
+  } else {
+    this->CTest->SetCTestConfigurationFromCMakeVariable(
+      this->Makefile, "DropMethod", "CTEST_DROP_METHOD", this->Quiet);
+    this->CTest->SetCTestConfigurationFromCMakeVariable(
+      this->Makefile, "DropSiteUser", "CTEST_DROP_SITE_USER", this->Quiet);
+    this->CTest->SetCTestConfigurationFromCMakeVariable(
+      this->Makefile, "DropSitePassword", "CTEST_DROP_SITE_PASSWORD",
+      this->Quiet);
+    this->CTest->SetCTestConfigurationFromCMakeVariable(
+      this->Makefile, "DropSite", "CTEST_DROP_SITE", this->Quiet);
+    this->CTest->SetCTestConfigurationFromCMakeVariable(
+      this->Makefile, "DropLocation", "CTEST_DROP_LOCATION", this->Quiet);
   }
-  this->CTest->SetCTestConfiguration("DropMethod", ctestDropMethod,
-                                     this->Quiet);
-  this->CTest->SetCTestConfiguration("DropSite", ctestDropSite, this->Quiet);
-  this->CTest->SetCTestConfiguration("DropLocation", ctestDropLocation,
-                                     this->Quiet);
 
   this->CTest->SetCTestConfigurationFromCMakeVariable(
     this->Makefile, "CurlOptions", "CTEST_CURL_OPTIONS", this->Quiet);
-  this->CTest->SetCTestConfigurationFromCMakeVariable(
-    this->Makefile, "DropSiteUser", "CTEST_DROP_SITE_USER", this->Quiet);
-  this->CTest->SetCTestConfigurationFromCMakeVariable(
-    this->Makefile, "DropSitePassword", "CTEST_DROP_SITE_PASSWORD",
-    this->Quiet);
 
   const char* notesFilesVariable =
     this->Makefile->GetDefinition("CTEST_NOTES_FILES");
@@ -184,6 +176,11 @@ bool cmCTestSubmitCommand::CheckArgumentKeyword(std::string const& arg)
     return true;
   }
 
+  if (arg == "SUBMIT_URL") {
+    this->ArgumentDoing = ArgumentDoingSubmitURL;
+    return true;
+  }
+
   if (arg == "INTERNAL_TEST_CHECKSUM") {
     this->InternalTest = true;
     return true;
@@ -249,6 +246,12 @@ bool cmCTestSubmitCommand::CheckArgumentValue(std::string const& arg)
     return true;
   }
 
+  if (this->ArgumentDoing == ArgumentDoingSubmitURL) {
+    this->ArgumentDoing = ArgumentDoingNone;
+    this->SubmitURL = arg;
+    return true;
+  }
+
   // Look for other arguments.
   return this->Superclass::CheckArgumentValue(arg);
 }

Source/CTest/cmCTestSubmitCommand.h

@@ -71,6 +71,7 @@ protected:
     ArgumentDoingCDashUpload,
     ArgumentDoingCDashUploadType,
     ArgumentDoingHttpHeader,
+    ArgumentDoingSubmitURL,
     ArgumentDoingLast2
   };
 
@@ -85,6 +86,7 @@ protected:
   std::string CDashUploadFile;
   std::string CDashUploadType;
   std::vector<std::string> HttpHeaders;
+  std::string SubmitURL;
 };
 
 #endif

Source/CTest/cmCTestSubmitHandler.cxx

@@ -162,7 +162,6 @@ bool cmCTestSubmitHandler::SubmitUsingHTTP(
 
   /* In windows, this will init the winsock stuff */
   ::curl_global_init(CURL_GLOBAL_ALL);
-  std::string dropMethod(this->CTest->GetCTestConfiguration("DropMethod"));
   std::string curlopt(this->CTest->GetCTestConfiguration("CurlOptions"));
   std::vector<std::string> args;
   cmSystemTools::ExpandListArgument(curlopt, args);
@@ -495,27 +494,6 @@ void cmCTestSubmitHandler::ParseResponse(
   }
 }
 
-void cmCTestSubmitHandler::ConstructCDashURL(std::string& dropMethod,
-                                             std::string& url)
-{
-  dropMethod = this->CTest->GetCTestConfiguration("DropMethod");
-  url = dropMethod;
-  url += "://";
-  if (!this->CTest->GetCTestConfiguration("DropSiteUser").empty()) {
-    url += this->CTest->GetCTestConfiguration("DropSiteUser");
-    cmCTestOptionalLog(
-      this->CTest, HANDLER_OUTPUT,
-      this->CTest->GetCTestConfiguration("DropSiteUser").c_str(), this->Quiet);
-    if (!this->CTest->GetCTestConfiguration("DropSitePassword").empty()) {
-      url += ":" + this->CTest->GetCTestConfiguration("DropSitePassword");
-      cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT, ":******", this->Quiet);
-    }
-    url += "@";
-  }
-  url += this->CTest->GetCTestConfiguration("DropSite") +
-    this->CTest->GetCTestConfiguration("DropLocation");
-}
-
 int cmCTestSubmitHandler::HandleCDashUploadFile(std::string const& file,
                                                 std::string const& typeString)
 {
@@ -536,16 +514,15 @@ int cmCTestSubmitHandler::HandleCDashUploadFile(std::string const& file,
   curl.SetCurlOptions(args);
   curl.SetTimeOutSeconds(SUBMIT_TIMEOUT_IN_SECONDS_DEFAULT);
   curl.SetHttpHeaders(this->HttpHeaders);
-  std::string dropMethod;
-  std::string url;
-  this->ConstructCDashURL(dropMethod, url);
+  std::string url = this->CTest->GetSubmitURL();
   std::string fields;
   std::string::size_type pos = url.find('?');
   if (pos != std::string::npos) {
     fields = url.substr(pos + 1);
     url = url.substr(0, pos);
   }
-  if (!(dropMethod == "http" || dropMethod == "https")) {
+  if (!cmHasLiteralPrefix(url, "http://") &&
+      !cmHasLiteralPrefix(url, "https://")) {
     cmCTestLog(this->CTest, ERROR_MESSAGE,
                "Only http and https are supported for CDASH_UPLOAD\n");
     return -1;
@@ -872,10 +849,7 @@ int cmCTestSubmitHandler::ProcessHandler()
       cnt++;
     }
   }
-  cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT,
-                     "Submit files (using "
-                       << this->CTest->GetCTestConfiguration("DropMethod")
-                       << ")" << std::endl,
+  cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT, "Submit files\n",
                      this->Quiet);
   const char* specificTrack = this->CTest->GetSpecificTrack();
   if (specificTrack) {
@@ -885,72 +859,32 @@ int cmCTestSubmitHandler::ProcessHandler()
   }
   this->SetLogFile(&ofs);
 
-  std::string dropMethod(this->CTest->GetCTestConfiguration("DropMethod"));
-
-  if (dropMethod.empty()) {
-    dropMethod = "http";
+  std::string url = this->CTest->GetSubmitURL();
+  cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT,
+                     "   SubmitURL: " << url << '\n', this->Quiet);
+  if (!this->SubmitUsingHTTP(buildDirectory + "/Testing/" +
+                               this->CTest->GetCurrentTag(),
+                             files, prefix, url)) {
+    cmCTestLog(this->CTest, ERROR_MESSAGE,
+               "   Problems when submitting via HTTP\n");
+    ofs << "   Problems when submitting via HTTP\n";
+    return -1;
   }
-
-  if (dropMethod == "http" || dropMethod == "https") {
-    std::string url = dropMethod;
-    url += "://";
-    ofs << "Using drop method: " << dropMethod << std::endl;
-    cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT,
-                       "   Using HTTP submit method" << std::endl
-                                                     << "   Drop site:" << url,
-                       this->Quiet);
-    if (!this->CTest->GetCTestConfiguration("DropSiteUser").empty()) {
-      url += this->CTest->GetCTestConfiguration("DropSiteUser");
-      cmCTestOptionalLog(
-        this->CTest, HANDLER_OUTPUT,
-        this->CTest->GetCTestConfiguration("DropSiteUser").c_str(),
-        this->Quiet);
-      if (!this->CTest->GetCTestConfiguration("DropSitePassword").empty()) {
-        url += ":" + this->CTest->GetCTestConfiguration("DropSitePassword");
-        cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT, ":******",
-                           this->Quiet);
-      }
-      url += "@";
-      cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT, "@", this->Quiet);
-    }
-    url += this->CTest->GetCTestConfiguration("DropSite") +
-      this->CTest->GetCTestConfiguration("DropLocation");
+  if (this->HasErrors) {
+    cmCTestLog(this->CTest, HANDLER_OUTPUT,
+               "   Errors occurred during submission.\n");
+    ofs << "   Errors occurred during submission.\n";
+  } else {
     cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT,
-                       this->CTest->GetCTestConfiguration("DropSite")
-                         << this->CTest->GetCTestConfiguration("DropLocation")
+                       "   Submission successful"
+                         << (this->HasWarnings ? ", with warnings." : "")
                          << std::endl,
                        this->Quiet);
-    if (!this->SubmitUsingHTTP(buildDirectory + "/Testing/" +
-                                 this->CTest->GetCurrentTag(),
-                               files, prefix, url)) {
-      cmCTestLog(this->CTest, ERROR_MESSAGE,
-                 "   Problems when submitting via HTTP" << std::endl);
-      ofs << "   Problems when submitting via HTTP" << std::endl;
-      return -1;
-    }
-    if (this->HasErrors) {
-      cmCTestLog(this->CTest, HANDLER_OUTPUT,
-                 "   Errors occurred during "
-                 "submission."
-                   << std::endl);
-      ofs << "   Errors occurred during submission. " << std::endl;
-    } else {
-      cmCTestOptionalLog(this->CTest, HANDLER_OUTPUT,
-                         "   Submission successful"
-                           << (this->HasWarnings ? ", with warnings." : "")
-                           << std::endl,
-                         this->Quiet);
-      ofs << "   Submission successful"
-          << (this->HasWarnings ? ", with warnings." : "") << std::endl;
-    }
-
-    return 0;
+    ofs << "   Submission successful"
+        << (this->HasWarnings ? ", with warnings." : "") << std::endl;
   }
 
-  cmCTestLog(this->CTest, ERROR_MESSAGE,
-             "   Unknown submission method: \"" << dropMethod << "\""
-                                                << std::endl);
-  return -1;
+  return 0;
 }
 
 std::string cmCTestSubmitHandler::GetSubmitResultsPrefix()

Source/CTest/cmCTestSubmitHandler.h

@@ -48,8 +48,6 @@ public:
     this->HttpHeaders = v;
   }
 
-  void ConstructCDashURL(std::string& dropMethod, std::string& url);
-
 private:
   void SetLogFile(std::ostream* ost) { this->LogFile = ost; }
 

Source/cmCTest.cxx

@@ -2626,6 +2626,32 @@ void cmCTest::SetCTestConfiguration(const char* name, const char* value,
   this->CTestConfiguration[name] = value;
 }
 
+std::string cmCTest::GetSubmitURL()
+{
+  std::string url = this->GetCTestConfiguration("SubmitURL");
+  if (url.empty()) {
+    std::string method = this->GetCTestConfiguration("DropMethod");
+    std::string user = this->GetCTestConfiguration("DropSiteUser");
+    std::string password = this->GetCTestConfiguration("DropSitePassword");
+    std::string site = this->GetCTestConfiguration("DropSite");
+    std::string location = this->GetCTestConfiguration("DropLocation");
+
+    url = method.empty() ? "http" : method;
+    url += "://";
+    if (!user.empty()) {
+      url += user;
+      if (!password.empty()) {
+        url += ':';
+        url += password;
+      }
+      url += '@';
+    }
+    url += site;
+    url += location;
+  }
+  return url;
+}
+
 std::string cmCTest::GetCurrentTag()
 {
   return this->CurrentTag;

Source/cmCTest.h

@@ -176,6 +176,8 @@ public:
                              bool suppress = false);
   void EmptyCTestConfiguration();
 
+  std::string GetSubmitURL();
+
   /**
    * constructor and destructor
    */

Tests/RunCMake/ctest_submit/FailDrop-http-stdout.txt

@@ -1,3 +1,2 @@
-Submit files \(using http\)
-   Using HTTP submit method
-   Drop site:http://
+Submit files
+   SubmitURL: http://-no-site-

Tests/RunCMake/ctest_submit/FailDrop-https-stdout.txt

@@ -1,3 +1,2 @@
-Submit files \(using https\)
-   Using HTTP submit method
-   Drop site:https://
+Submit files
+   SubmitURL: https://-no-site-