Auto merge of #62564 - Mark-Simulacrum:azure-line-endings, r=pietroalbini

Ensure that checkout is with \n line endings

During installation of mingw, at least, the git directories change, so
we need to reset the core.autocrlf config to false.

Once we finish checking out submodules, check that the line endings are
\n and not \r\n.

Artifacts were built via the last try on #62545; I've manually confirmed that `install.sh` appears to no longer have `\r\n` line endings.

Fixes #62276.

2 files changed, 34 insertions, 0 deletions

diff --git a/.azure-pipelines/steps/install-windows-build-deps.yml b/.azure-pipelines/steps/install-windows-build-deps.yml
index 04662a69c87..ed06679464c 100644
--- a/.azure-pipelines/steps/install-windows-build-deps.yml
+++ b/.azure-pipelines/steps/install-windows-build-deps.yml
@@ -9,6 +9,9 @@ steps:
   displayName: "Ensure build happens on C:/ instead of D:/"
   condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
 
+- bash: git config --replace-all --global core.autocrlf false
+  displayName: "Disable git automatic line ending conversion (on C:/)"
+
 # Download and install MSYS2, needed primarily for the test suite (run-make) but
 # also used by the MinGW toolchain for assembling things.
 #
diff --git a/.azure-pipelines/steps/run.yml b/.azure-pipelines/steps/run.yml
index 9e6fb6c8a8c..c73753a5857 100644
--- a/.azure-pipelines/steps/run.yml
+++ b/.azure-pipelines/steps/run.yml
@@ -12,6 +12,9 @@ steps:
 # Azure's Windows image. Having the conversion enabled caused regressions both
 # in our test suite (it broke miri tests) and in the ecosystem, since we
 # started shipping install scripts with CRLF endings instead of the old LF.
+#
+# Note that we do this a couple times during the build as the PATH and current
+# user/directory change, e.g. when mingw is enabled.
 - bash: git config --global core.autocrlf false
   displayName: "Disable git automatic line ending conversion"
 
@@ -70,6 +73,14 @@ steps:
   displayName: Enable IPv6
   condition: and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
 
+# Disable automatic line ending conversion (again). On Windows, when we're
+# installing dependencies, something switches the git configuration directory or
+# re-enables autocrlf. We've not tracked down the exact cause -- and there may
+# be multiple -- but this should ensure submodules are checked out with the
+# appropriate line endings.
+- bash: git config --replace-all --global core.autocrlf false
+  displayName: "Disable git automatic line ending conversion"
+
 # Check out all our submodules, but more quickly than using git by using one of
 # our custom scripts
 - bash: |
@@ -84,6 +95,26 @@ steps:
   condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
   displayName: Check out submodules (Windows)
 
+# See also the disable for autocrlf above, this just checks that it worked
+#
+# We check both in rust-lang/rust and in a submodule to make sure both are
+# accurate. Submodules are checked out significantly later than the main
+# repository in this script, so settings can (and do!) change between then.
+#
+# Linux (and maybe macOS) builders don't currently have dos2unix so just only
+# run this step on Windows.
+- bash: |
+    set -x
+    # print out the git configuration so we can better investigate failures in
+    # the following
+    git config --list --show-origin
+    dos2unix -ih Cargo.lock src/tools/rust-installer/install-template.sh
+    endings=$(dos2unix -ic Cargo.lock src/tools/rust-installer/install-template.sh)
+    # if endings has non-zero length, error out
+    if [ -n "$endings" ]; then exit 1 ; fi
+  condition: and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
+  displayName: Verify line endings are LF
+
 # Ensure the `aws` CLI is installed so we can deploy later on, cache docker
 # images, etc.
 - bash: |