Refactoring: Introduce distinct host and target rpath var setters.

Two line summary: Distinguish HOST_RPATH and TARGET_RPATH; added
RPATH_LINK_SEARCH; skip tests broken in stage1; general cleanup.

`HOST_RPATH_VAR$(1)_T_$(2)_H_$(3)` and `TARGET_RPATH_VAR$(1)_T_$(2)_H_$(3)`
both match the format of the old `RPATH_VAR$(1)_T_$(2)_H_$(3)` (which
is still being set the same way that it was before, to one of either
HOST/TARGET depending on what stage we are building).  Namely, the format
is <XXX>_RPATH_VAR = "<LD_LIB_PATH_ENVVAR>=<COLON_SEP_PATH_ENTRIES>"

What this commit does:

* Pass both of the (newly introduced) HOST and TARGET rpath setup vars
  to `maketest.py`

* Update `maketest.py` to no longer update the LD_LIBRARY_PATH itself
  Instead, it passes along the HOST and TARGET rpath setup vars in
  environment variables `HOST_RPATH_ENV` and `TARGET_RPATH_ENV`

* Also, pass the current stage number to maketest.py; it in turn
  passes it (via an env var) to run-make tests.

  This allows the run-make tests to selectively change behavior
  (e.g. turn themselves off) to deal with incompatibilities with
  e.g. stage1.

* Cleanup: Distinguish in tools.mk between the command to run (`RUN`)
  and the file to generate to drive that command (`RUN_BINFILE`).  The
  main thing this enables is that `RUN` can now setup the
  `TARGET_RPATH_ENV` without having to dirty up the runner code in
  each of the `run-make` Makefiles.

* Cleanup: Factored out commands to delete dylib/rlib into
  REMOVE_DYLIBS/REMOVE_RLIBS.

  There were places where we were only calling `rm $(call DYLIB,foo)`
  even though we really needed to get rid of the whole glob (at least
  based on alex's findings on #13753 that removing the symlink does not
  suffice).

  Therefore rather than peppering the code with the awkward
  `rm $(TMPDIR)/$(call DYLIB_GLOB,foo)`, I instead introduced a common
  `REMOVE_DYLIBS` user function that expands into that when called.
  After I adding an analogous `REMOVE_RLIBS`, I changed all of the
  existing calls that rm dylibs or rlibs to use these routines
  instead.

  Note that the latter is not a true refactoring since I may have
  changed cases where it was our intent to only remove the sym-link.
  (But if that is the case, then we need to more deeply investigate
  alex's findings on #13753 where the system was still dynamically
  loading up the non-symlinked libraries that it finds on the load
  path.)

* Added RPATH_LINK_SEARCH command and use it on Linux.

  On some platforms, namely Linux, when you have libboot.so that has
  its internal rpath set (to e.g. $(ORIGIN)/path/to/HOSTDIR), the
  linker still complains when you do the link step and it does not
  know where to find libraries that libboot.so depends upon that live
  in HOSTDIR (think e.g. librustuv.so).

  As far as I can tell, the GNU linker will consult the
  LD_LIBRARY_PATH as part of the linking process to find such
  libraries.  But if you want to be more careful and not override
  LD_LIBRARY_PATH for the `gcc` invocation, then you need some other
  way to tell the linker where it can find the libraries that
  libboot.so needs.  The solution to this on Linux is the
  `-Wl,-rpath-link` command line option.

  However, this command line option does not exist on Mac OS X, (which
  appears to be figuring out how to resolve the libboot.dylib
  dependency by some other means, perhaps by consulting the rpath
  setting within libboot.dylib).

  So, in order to abstract over this distinction, I added the
  RPATH_LINK_SEARCH macro to the run-make infrastructure and added
  calls to it where necessary to get Linux working.  On architectures
  other than Linux, the macro expands to nothing.

* Disable miscellaneous tests atop stage1.

* An especially interesting instance of the previous bullet point:
  Excuse regex from doing rustdoc tests atop stage1.

  This was a (nearly-) final step to get `make check-stage1` working
  again.

  The use of a special-case check for regex here is ugly but is
  analogous other similar checks for regex such as the one that landed
  in PR #13844.

  The way this is written, the user will get a reminder that
  doc-crate-regex is being skipped whenever their rules attempt to do
  the crate documentation tests.  This is deliberate: I want people
  running `make check-stage1` to be reminded about which cases are
  being skipped.  (But if such echo noise is considered offensive, it
  can obviously be removed.)

* Got windows working with the above changes.

  This portion of the commit is a cleanup revision of the (previously
  mentioned on try builds) re-architecting of how the LD_LIBRARY_PATH
  setup and extension is handled in order to accommodate Windows' (1.)
  use of `$PATH` for that purpose and (2.) use of spaces in `$PATH`
  entries (problematic for make and for interoperation with tools at
  the shell).

* In addition, since the code has been rearchitected to pass the
  HOST_RPATH_DIR/TARGET_RPATH_DIR rather than a whole sh
  environment-variable setting command, there is no need to for the
  convert_path_spec calls in maketest.py, which in fact were put in
  place to placate Windows but were now causing the Windows builds to
  fail.  Instead we just convert the paths to absolute paths just like
  all of the other path arguments.

Also, note for makefile hackers: apparently you cannot quote operands
to `ifeq` in Makefile (or at least, you need to be careful about
adding them, e.g. to only one side).

22 files changed, 158 insertions, 55 deletions

diff --git a/mk/main.mk b/mk/main.mk
index 5c0bffda7be..d652fac12f8 100644
--- a/mk/main.mk
+++ b/mk/main.mk
@@ -349,18 +349,45 @@ CFGFLAG$(1)_T_$(2)_H_$(3) = stage$(1)
 endef
 
 # Same macro/variables as above, but defined in a separate loop so it can use
-# all the varibles above for all archs. The RPATH_VAR setup sometimes needs to
+# all the variables above for all archs. The RPATH_VAR setup sometimes needs to
 # reach across triples to get things in order.
+#
+# Defines (with the standard $(1)_T_$(2)_H_$(3) suffix):
+# * `LD_LIBRARY_PATH_ENV_NAME`: the name for the key to use in the OS
+#   environment to access or extend the lookup path for dynamic
+#   libraries.  Note on Windows, that key is `$PATH`, and thus not
+#   only conflates programs with dynamic libraries, but also often
+#   contains spaces which confuse make.
+# * `LD_LIBRARY_PATH_ENV_HOSTDIR`: the entry to add to lookup path for the host
+# * `LD_LIBRARY_PATH_ENV_TARGETDIR`: the entry to add to lookup path for target
+# 
+# Below that, HOST_RPATH_VAR and TARGET_RPATH_VAR are defined in terms of the
+# above settings.
+# 
 define SREQ_CMDS
 
 ifeq ($$(OSTYPE_$(3)),apple-darwin)
-  RPATH_VAR$(1)_T_$(2)_H_$(3) := \
-      DYLD_LIBRARY_PATH="$$$$DYLD_LIBRARY_PATH:$$(CURDIR)/$$(HLIB$(1)_H_$(3))"
+  LD_LIBRARY_PATH_ENV_NAME$(1)_T_$(2)_H_$(3) := DYLD_LIBRARY_PATH
+else
+ifeq ($$(CFG_WINDOWSY_$(2)),1)
+  LD_LIBRARY_PATH_ENV_NAME$(1)_T_$(2)_H_$(3) := PATH
 else
-  RPATH_VAR$(1)_T_$(2)_H_$(3) := \
-      LD_LIBRARY_PATH="$$$$LD_LIBRARY_PATH:$$(CURDIR)/$$(HLIB$(1)_H_$(3))"
+  LD_LIBRARY_PATH_ENV_NAME$(1)_T_$(2)_H_$(3) := LD_LIBRARY_PATH
+endif
 endif
 
+LD_LIBRARY_PATH_ENV_HOSTDIR$(1)_T_$(2)_H_$(3) := \
+    $$(CURDIR)/$$(HLIB$(1)_H_$(3))
+LD_LIBRARY_PATH_ENV_TARGETDIR$(1)_T_$(2)_H_$(3) := \
+    $$(CURDIR)/$$(TLIB1_T_$(2)_H_$(CFG_BUILD))
+
+HOST_RPATH_VAR$(1)_T_$(2)_H_$(3) := \
+  $$(LD_LIBRARY_PATH_ENV_NAME$(1)_T_$(2)_H_$(3))=$$$$$$(LD_LIBRARY_PATH_ENV_NAME$(1)_T_$(2)_H_$(3)):$$(LD_LIBRARY_PATH_ENV_HOSTDIR$(1)_T_$(2)_H_$(3))
+TARGET_RPATH_VAR$(1)_T_$(2)_H_$(3) := \
+  $$(LD_LIBRARY_PATH_ENV_NAME$(1)_T_$(2)_H_$(3))=$$$$$$(LD_LIBRARY_PATH_ENV_NAME$(1)_T_$(2)_H_$(3)):$$(LD_LIBRARY_PATH_ENV_TARGETDIR$(1)_T_$(2)_H_$(3))
+
+RPATH_VAR$(1)_T_$(2)_H_$(3) := $$(HOST_RPATH_VAR$(1)_T_$(2)_H_$(3))
+
 # Pass --cfg stage0 only for the build->host part of stage0;
 # if you're building a cross config, the host->* parts are
 # effectively stage1, since it uses the just-built stage0.
@@ -376,13 +403,7 @@ ifeq ($(1),0)
 ifneq ($(strip $(CFG_BUILD)),$(strip $(3)))
 CFGFLAG$(1)_T_$(2)_H_$(3) = stage1
 
-ifeq ($$(OSTYPE_$(3)),apple-darwin)
-  RPATH_VAR$(1)_T_$(2)_H_$(3) := \
-      DYLD_LIBRARY_PATH="$$$$DYLD_LIBRARY_PATH:$$(CURDIR)/$$(TLIB1_T_$(2)_H_$(CFG_BUILD))"
-else
-  RPATH_VAR$(1)_T_$(2)_H_$(3) := \
-      LD_LIBRARY_PATH="$$$$LD_LIBRARY_PATH:$$(CURDIR)/$$(TLIB1_T_$(2)_H_$(CFG_BUILD))"
-endif
+RPATH_VAR$(1)_T_$(2)_H_$(3) := $$(TARGET_RPATH_VAR$(1)_T_$(2)_H_$(3))
 endif
 endif
 
diff --git a/mk/tests.mk b/mk/tests.mk
index 8f20d55e385..25f03751074 100644
--- a/mk/tests.mk
+++ b/mk/tests.mk
@@ -793,8 +793,27 @@ else
 CRATEDOCTESTDEP_$(1)_$(2)_$(3)_$(4) = $$(RSINPUTS_$(4))
 endif
 
+# (Issues #13732, #13983, #14000) The doc for the regex crate includes
+# uses of the `regex!` macro from the regex_macros crate.  There is
+# normally a dependence injected that makes the target's regex depend
+# upon the host's regex_macros (see #13845), but that dependency
+# injection is currently skipped for stage1 as a special case.
+#
+# Therefore, as a further special case, this conditional skips
+# attempting to run the doc tests for the regex crate atop stage1,
+# (since there is no regex_macros crate for the stage1 rustc to load).
+#
+# (Another approach for solving this would be to inject the desired
+# dependence for stage1 as well, by setting things up to generate a
+# regex_macros crate that was compatible with the stage1 rustc and
+# thus re-enable our ability to run this test.)
+ifeq (stage$(1)-crate-$(4),stage1-crate-regex)
+check-stage$(1)-T-$(2)-H-$(3)-doc-crate-$(4)-exec:
+	@$$(call E, skipping doc-crate-$(4) as it uses macros and cannot run at stage$(1))
+else
 check-stage$(1)-T-$(2)-H-$(3)-doc-crate-$(4)-exec: \
 	$$(call TEST_OK_FILE,$(1),$(2),$(3),doc-crate-$(4))
+endif
 
 ifeq ($(2),$$(CFG_BUILD))
 $$(call TEST_OK_FILE,$(1),$(2),$(3),doc-crate-$(4)): $$(CRATEDOCTESTDEP_$(1)_$(2)_$(3)_$(4))
@@ -951,7 +970,10 @@ $(3)/test/run-make/%-$(1)-T-$(2)-H-$(3).ok: \
 	    "$$(CC_$(3)) $$(CFG_GCCISH_CFLAGS_$(3))" \
 	    $$(HBIN$(1)_H_$(3))/rustdoc$$(X_$(3)) \
 	    "$$(TESTNAME)" \
-	    "$$(RPATH_VAR$(1)_T_$(2)_H_$(3))"
+	    $$(LD_LIBRARY_PATH_ENV_NAME$(1)_T_$(2)_H_$(3)) \
+	    "$$(LD_LIBRARY_PATH_ENV_HOSTDIR$(1)_T_$(2)_H_$(3))" \
+	    "$$(LD_LIBRARY_PATH_ENV_TARGETDIR$(1)_T_$(2)_H_$(3))" \
+	    $(1)
 	@touch $$@
 else
 # FIXME #11094 - The above rule doesn't work right for multiple targets
diff --git a/src/etc/maketest.py b/src/etc/maketest.py
index 7646113560b..0e2c1e77ab4 100644
--- a/src/etc/maketest.py
+++ b/src/etc/maketest.py
@@ -30,6 +30,10 @@ def putenv(name, value):
         value = normalize_path(value)
     os.putenv(name, value)
 
+def convert_path_spec(name, value):
+    if os.name == 'nt' and name != 'PATH':
+        value = ":".join(normalize_path(v) for v in value.split(";"))
+    return value
 
 make = sys.argv[2]
 putenv('RUSTC', os.path.abspath(sys.argv[3]))
@@ -37,13 +41,10 @@ putenv('TMPDIR', os.path.abspath(sys.argv[4]))
 putenv('CC', sys.argv[5])
 putenv('RUSTDOC', os.path.abspath(sys.argv[6]))
 filt = sys.argv[7]
-ldpath = sys.argv[8]
-if ldpath != '':
-    name = ldpath.split('=')[0]
-    value = ldpath.split('=')[1]
-    if os.name == 'nt' and name != 'PATH':
-        value = ":".join(normalize_path(v) for v in value.split(";"))
-    os.putenv(name, value)
+putenv('LD_LIB_PATH_ENVVAR', sys.argv[8]);
+putenv('HOST_RPATH_DIR', os.path.abspath(sys.argv[9]));
+putenv('TARGET_RPATH_DIR', os.path.abspath(sys.argv[10]));
+putenv('RUST_BUILD_STAGE', sys.argv[11])
 
 if not filt in sys.argv[1]:
     sys.exit(0)
diff --git a/src/test/run-make/bootstrap-from-c-with-green/Makefile b/src/test/run-make/bootstrap-from-c-with-green/Makefile
index 43388fd9a6f..3b28af9b0e8 100644
--- a/src/test/run-make/bootstrap-from-c-with-green/Makefile
+++ b/src/test/run-make/bootstrap-from-c-with-green/Makefile
@@ -1,9 +1,13 @@
 -include ../tools.mk
 
+HOST_LIB_DIR=$(TMPDIR)/../../../stage$(RUST_BUILD_STAGE)/lib
+# This overrides the LD_LIBRARY_PATH for RUN
+TARGET_RPATH_DIR:=$(TARGET_RPATH_DIR):$(TMPDIR)
+
 all:
 	$(RUSTC) lib.rs
 	ln -nsf $(call DYLIB,boot-*) $(call DYLIB,boot)
-	$(CC) main.c -o $(call RUN,main) -lboot
+	$(CC) main.c -o $(call RUN_BINFILE,main) $(call RPATH_LINK_SEARCH,$(HOST_LIB_DIR)) -lboot
 	$(call RUN,main)
-	rm $(call DYLIB,boot)
+	$(call REMOVE_DYLIBS,boot)
 	$(call FAIL,main)
diff --git a/src/test/run-make/bootstrap-from-c-with-native/Makefile b/src/test/run-make/bootstrap-from-c-with-native/Makefile
index 43388fd9a6f..3b28af9b0e8 100644
--- a/src/test/run-make/bootstrap-from-c-with-native/Makefile
+++ b/src/test/run-make/bootstrap-from-c-with-native/Makefile
@@ -1,9 +1,13 @@
 -include ../tools.mk
 
+HOST_LIB_DIR=$(TMPDIR)/../../../stage$(RUST_BUILD_STAGE)/lib
+# This overrides the LD_LIBRARY_PATH for RUN
+TARGET_RPATH_DIR:=$(TARGET_RPATH_DIR):$(TMPDIR)
+
 all:
 	$(RUSTC) lib.rs
 	ln -nsf $(call DYLIB,boot-*) $(call DYLIB,boot)
-	$(CC) main.c -o $(call RUN,main) -lboot
+	$(CC) main.c -o $(call RUN_BINFILE,main) $(call RPATH_LINK_SEARCH,$(HOST_LIB_DIR)) -lboot
 	$(call RUN,main)
-	rm $(call DYLIB,boot)
+	$(call REMOVE_DYLIBS,boot)
 	$(call FAIL,main)
diff --git a/src/test/run-make/c-dynamic-dylib/Makefile b/src/test/run-make/c-dynamic-dylib/Makefile
index 2b2e5d56e92..c4720c418a7 100644
--- a/src/test/run-make/c-dynamic-dylib/Makefile
+++ b/src/test/run-make/c-dynamic-dylib/Makefile
@@ -9,6 +9,6 @@ all: $(call DYLIB,cfoo)
 	$(RUSTC) foo.rs
 	$(RUSTC) bar.rs
 	$(call RUN,bar)
-	rm $(TMPDIR)/$(call DYLIB_GLOB,cfoo)
+	$(call REMOVE_DYLIBS,cfoo)
 	$(call FAIL,bar)
 endif
diff --git a/src/test/run-make/c-dynamic-rlib/Makefile b/src/test/run-make/c-dynamic-rlib/Makefile
index 18992703b2c..e15cfd34d6c 100644
--- a/src/test/run-make/c-dynamic-rlib/Makefile
+++ b/src/test/run-make/c-dynamic-rlib/Makefile
@@ -1,5 +1,8 @@
 -include ../tools.mk
 
+# This overrides the LD_LIBRARY_PATH for RUN
+TARGET_RPATH_DIR:=$(TARGET_RPATH_DIR):$(TMPDIR)
+
 # This hits an assertion in the linker on older versions of osx apparently
 ifeq ($(shell uname),Darwin)
 all:
@@ -8,7 +11,7 @@ else
 all: $(call DYLIB,cfoo)
 	$(RUSTC) foo.rs
 	$(RUSTC) bar.rs
-	LD_LIBRARY_PATH=$(TMPDIR) $(call RUN,bar)
-	rm $(TMPDIR)/$(call DYLIB_GLOB,cfoo)
+	$(call RUN,bar)
+	$(call REMOVE_DYLIBS,cfoo)
 	$(call FAIL,bar)
 endif
diff --git a/src/test/run-make/c-link-to-rust-dylib/Makefile b/src/test/run-make/c-link-to-rust-dylib/Makefile
index fb57a08a826..e743004a9cb 100644
--- a/src/test/run-make/c-link-to-rust-dylib/Makefile
+++ b/src/test/run-make/c-link-to-rust-dylib/Makefile
@@ -1,9 +1,11 @@
 -include ../tools.mk
 
+HOST_LIB_DIR=$(TMPDIR)/../../../stage$(RUST_BUILD_STAGE)/lib
+
 all:
 	$(RUSTC) foo.rs
 	ln -s $(call DYLIB,foo-*) $(call DYLIB,foo)
-	$(CC) bar.c -lfoo -o $(call RUN,bar) -Wl,-rpath,$(TMPDIR)
+	$(CC) bar.c -lfoo -o $(call RUN_BINFILE,bar) $(call RPATH_LINK_SEARCH,$(HOST_LIB_DIR)) -Wl,-rpath,$(TMPDIR)
 	$(call RUN,bar)
-	rm $(call DYLIB,foo)
+	$(call REMOVE_DYLIBS,foo)
 	$(call FAIL,bar)
diff --git a/src/test/run-make/c-link-to-rust-staticlib/Makefile b/src/test/run-make/c-link-to-rust-staticlib/Makefile
index 7312a65c812..40b6feac678 100644
--- a/src/test/run-make/c-link-to-rust-staticlib/Makefile
+++ b/src/test/run-make/c-link-to-rust-staticlib/Makefile
@@ -11,7 +11,7 @@ ifneq ($(shell uname),FreeBSD)
 all:
 	$(RUSTC) foo.rs
 	ln -s $(call STATICLIB,foo-*) $(call STATICLIB,foo)
-	$(CC) bar.c -lfoo -o $(call RUN,bar) $(EXTRAFLAGS) -lstdc++
+	$(CC) bar.c -lfoo -o $(call RUN_BINFILE,bar) $(EXTRAFLAGS) -lstdc++
 	$(call RUN,bar)
 	rm $(call STATICLIB,foo*)
 	$(call RUN,bar)
diff --git a/src/test/run-make/c-static-dylib/Makefile b/src/test/run-make/c-static-dylib/Makefile
index 62d9c8e90f2..6b047846cfd 100644
--- a/src/test/run-make/c-static-dylib/Makefile
+++ b/src/test/run-make/c-static-dylib/Makefile
@@ -5,5 +5,5 @@ all: $(call STATICLIB,cfoo)
 	$(RUSTC) bar.rs
 	rm $(TMPDIR)/$(call STATICLIB_GLOB,cfoo)
 	$(call RUN,bar)
-	rm $(TMPDIR)/$(call DYLIB_GLOB,foo)
+	$(call REMOVE_DYLIBS,foo)
 	$(call FAIL,bar)
diff --git a/src/test/run-make/c-static-rlib/Makefile b/src/test/run-make/c-static-rlib/Makefile
index 09eb4b1249e..02b24ef9846 100644
--- a/src/test/run-make/c-static-rlib/Makefile
+++ b/src/test/run-make/c-static-rlib/Makefile
@@ -3,6 +3,6 @@
 all: $(call STATICLIB,cfoo)
 	$(RUSTC) foo.rs
 	$(RUSTC) bar.rs
-	rm $(TMPDIR)/$(call RLIB_GLOB,foo)
+	$(call REMOVE_RLIBS,foo)
 	rm $(TMPDIR)/$(call STATICLIB_GLOB,cfoo)
 	$(call RUN,bar)
diff --git a/src/test/run-make/dylib-chain/Makefile b/src/test/run-make/dylib-chain/Makefile
index e60e904240d..2149f245147 100644
--- a/src/test/run-make/dylib-chain/Makefile
+++ b/src/test/run-make/dylib-chain/Makefile
@@ -6,7 +6,7 @@ all:
 	$(RUSTC) m3.rs
 	$(RUSTC) m4.rs
 	$(call RUN,m4)
-	rm $(TMPDIR)/$(call DYLIB_GLOB,m1)
-	rm $(TMPDIR)/$(call DYLIB_GLOB,m2)
-	rm $(TMPDIR)/$(call DYLIB_GLOB,m3)
+	$(call REMOVE_DYLIBS,m1)
+	$(call REMOVE_DYLIBS,m2)
+	$(call REMOVE_DYLIBS,m3)
 	$(call FAIL,m4)
diff --git a/src/test/run-make/extern-fn-reachable/Makefile b/src/test/run-make/extern-fn-reachable/Makefile
index 0560626c999..56748b1eb9b 100644
--- a/src/test/run-make/extern-fn-reachable/Makefile
+++ b/src/test/run-make/extern-fn-reachable/Makefile
@@ -1,5 +1,8 @@
 -include ../tools.mk
 
+# This overrides the LD_LIBRARY_PATH for RUN
+TARGET_RPATH_DIR:=$(TARGET_RPATH_DIR):$(TMPDIR)
+
 all:
 	$(RUSTC) dylib.rs -o $(TMPDIR)/libdylib.so
 	$(RUSTC) main.rs
diff --git a/src/test/run-make/lto-smoke-c/Makefile b/src/test/run-make/lto-smoke-c/Makefile
index 85b8d0e2dd8..49a04ce42a0 100644
--- a/src/test/run-make/lto-smoke-c/Makefile
+++ b/src/test/run-make/lto-smoke-c/Makefile
@@ -19,5 +19,5 @@ CC := $(CC:-g=)
 all:
 	$(RUSTC) foo.rs -Z lto
 	ln -s $(call STATICLIB,foo-*) $(call STATICLIB,foo)
-	$(CC) bar.c -lfoo -o $(call RUN,bar) $(EXTRAFLAGS) -lstdc++
+	$(CC) bar.c -lfoo -o $(call RUN_BINFILE,bar) $(EXTRAFLAGS) -lstdc++
 	$(call RUN,bar)
diff --git a/src/test/run-make/lto-syntax-extension/Makefile b/src/test/run-make/lto-syntax-extension/Makefile
index 4652556d344..c522f30e2df 100644
--- a/src/test/run-make/lto-syntax-extension/Makefile
+++ b/src/test/run-make/lto-syntax-extension/Makefile
@@ -1,6 +1,18 @@
 -include ../tools.mk
 
-all:
+# This test attempts to use syntax extensions, which are known to be
+# incompatible with stage1 at the moment.
+
+ifeq ($(RUST_BUILD_STAGE),1)
+DOTEST=
+else
+DOTEST=dotest
+endif
+
+all: $(DOTEST)
+
+dotest:
+	env
 	$(RUSTC) lib.rs
 	$(RUSTC) main.rs -Z lto
 	$(call RUN,main)
diff --git a/src/test/run-make/missing-crate-dependency/Makefile b/src/test/run-make/missing-crate-dependency/Makefile
index a470ee0a7c1..3f8b97f2566 100644
--- a/src/test/run-make/missing-crate-dependency/Makefile
+++ b/src/test/run-make/missing-crate-dependency/Makefile
@@ -3,7 +3,7 @@
 all: 
 	$(RUSTC) --crate-type=rlib crateA.rs
 	$(RUSTC) --crate-type=rlib crateB.rs
-	rm $(TMPDIR)/$(call RLIB_GLOB,crateA)
+	$(call REMOVE_RLIBS,crateA)
 	# Ensure crateC fails to compile since dependency crateA is missing
 	$(RUSTC) crateC.rs 2>&1 | \
 		grep "error: can't find crate for \`crateA\` which \`crateB\` depends on"
diff --git a/src/test/run-make/mixing-libs/Makefile b/src/test/run-make/mixing-libs/Makefile
index 4de0cb32762..babeeef164d 100644
--- a/src/test/run-make/mixing-libs/Makefile
+++ b/src/test/run-make/mixing-libs/Makefile
@@ -5,5 +5,5 @@ all:
 	$(RUSTC) dylib.rs
 	$(RUSTC) rlib.rs --crate-type=dylib
 	$(RUSTC) dylib.rs
-	rm $(call DYLIB,rlib-*)
+	$(call REMOVE_DYLIBS,rlib)
 	$(RUSTC) prog.rs && exit 1 || exit 0
diff --git a/src/test/run-make/obey-crate-type-flag/Makefile b/src/test/run-make/obey-crate-type-flag/Makefile
index a3c58a6bf17..903349152df 100644
--- a/src/test/run-make/obey-crate-type-flag/Makefile
+++ b/src/test/run-make/obey-crate-type-flag/Makefile
@@ -7,7 +7,7 @@
 # fail if an rlib was built
 all:
 	$(RUSTC) test.rs
-	rm $(TMPDIR)/$(call RLIB_GLOB,test)
-	rm $(TMPDIR)/$(call DYLIB_GLOB,test)
+	$(call REMOVE_RLIBS,test)
+	$(call REMOVE_DYLIBS,test)
 	$(RUSTC) --crate-type dylib test.rs
-	rm $(TMPDIR)/$(call RLIB_GLOB,test) && exit 1 || exit 0
+	$(call REMOVE_RLIBS,test) && exit 1 || exit 0
diff --git a/src/test/run-make/output-type-permutations/Makefile b/src/test/run-make/output-type-permutations/Makefile
index 5a309cddeb1..c163a5bec08 100644
--- a/src/test/run-make/output-type-permutations/Makefile
+++ b/src/test/run-make/output-type-permutations/Makefile
@@ -2,8 +2,8 @@
 
 all:
 	$(RUSTC) foo.rs --crate-type=rlib,dylib,staticlib
-	rm $(TMPDIR)/$(call RLIB_GLOB,bar)
-	rm $(TMPDIR)/$(call DYLIB_GLOB,bar)
+	$(call REMOVE_RLIBS,bar)
+	$(call REMOVE_DYLIBS,bar)
 	rm $(TMPDIR)/$(call STATICLIB_GLOB,bar)
 	$(RUSTC) foo.rs --crate-type=bin
 	rm $(TMPDIR)/$(call BIN,bar)
@@ -41,4 +41,4 @@ all:
 	cmp $(TMPDIR)/foo.bc $(TMPDIR)/bar.bc
 	rm $(TMPDIR)/bar.bc
 	rm $(TMPDIR)/foo.bc
-	rm $(TMPDIR)/$(call RLIB_GLOB,bar)
+	$(call REMOVE_RLIBS,bar)
diff --git a/src/test/run-make/prefer-dylib/Makefile b/src/test/run-make/prefer-dylib/Makefile
index 38759b2f524..fe9bbb95095 100644
--- a/src/test/run-make/prefer-dylib/Makefile
+++ b/src/test/run-make/prefer-dylib/Makefile
@@ -5,4 +5,4 @@ all:
 	$(RUSTC) foo.rs -C prefer-dynamic
 	$(call RUN,foo)
 	rm $(TMPDIR)/*bar*
-	$(call FAILS,foo)
+	$(call FAIL,foo)
diff --git a/src/test/run-make/tools.mk b/src/test/run-make/tools.mk
index 90274cdb3e7..dedd739052c 100644
--- a/src/test/run-make/tools.mk
+++ b/src/test/run-make/tools.mk
@@ -4,8 +4,20 @@ export DYLD_LIBRARY_PATH:=$(TMPDIR):$(DYLD_LIBRARY_PATH)
 RUSTC := $(RUSTC) --out-dir $(TMPDIR) -L $(TMPDIR)
 CC := $(CC) -L $(TMPDIR)
 
-RUN = $(TMPDIR)/$(1)
-FAILS = $(TMPDIR)/$(1) && exit 1 || exit 0
+# These deliberately use `=` and not `:=` so that client makefiles can
+# augment HOST_RPATH_DIR / TARGET_RPATH_DIR.
+HOST_RPATH_ENV = \
+    $(LD_LIB_PATH_ENVVAR)=$$$(LD_LIB_PATH_ENVVAR):$(HOST_RPATH_DIR)
+TARGET_RPATH_ENV = \
+    $(LD_LIB_PATH_ENVVAR)=$$$(LD_LIB_PATH_ENVVAR):$(TARGET_RPATH_DIR)
+
+# This is the name of the binary we will generate and run; use this
+# e.g. for `$(CC) -o $(RUN_BINFILE)`.
+RUN_BINFILE = $(TMPDIR)/$(1)
+
+# RUN and FAIL are basic way we will invoke the generated binary.  On
+# non-windows platforms, they set the LD_LIBRARY_PATH environment
+# variable before running the binary.
 
 RLIB_GLOB = lib$(1)*.rlib
 STATICLIB = $(TMPDIR)/lib$(1).a
@@ -18,20 +30,32 @@ IS_WINDOWS=1
 endif
 
 ifeq ($(UNAME),Darwin)
+RUN = $(TARGET_RPATH_ENV) $(RUN_BINFILE)
+FAIL = $(TARGET_RPATH_ENV) $(RUN_BINFILE) && exit 1 || exit 0
 DYLIB_GLOB = lib$(1)*.dylib
 DYLIB = $(TMPDIR)/lib$(1).dylib
+RPATH_LINK_SEARCH =
 else
 ifdef IS_WINDOWS
+RUN = PATH="$(PATH):$(TARGET_RPATH_DIR)" $(RUN_BINFILE)
+FAIL = PATH="$(PATH):$(TARGET_RPATH_DIR)" $(RUN_BINFILE) && exit 1 || exit 0
 DYLIB_GLOB = $(1)*.dll
 DYLIB = $(TMPDIR)/$(1).dll
 BIN = $(1).exe
-export PATH := $(PATH):$(LD_LIBRARY_PATH)
+RPATH_LINK_SEARCH =
+RUSTC := PATH="$(PATH):$(LD_LIBRARY_PATH)" $(RUSTC)
 else
+RUN = $(TARGET_RPATH_ENV) $(RUN_BINFILE)
+FAIL = $(TARGET_RPATH_ENV) $(RUN_BINFILE) && exit 1 || exit 0
 DYLIB_GLOB = lib$(1)*.so
 DYLIB = $(TMPDIR)/lib$(1).so
+RPATH_LINK_SEARCH = -Wl,-rpath-link=$(1)
 endif
 endif
 
+REMOVE_DYLIBS     = rm $(TMPDIR)/$(call DYLIB_GLOB,$(1))
+REMOVE_RLIBS      = rm $(TMPDIR)/$(call RLIB_GLOB,$(1))
+
 %.a: %.o
 	ar crus $@ $<
 %.dylib: %.o
diff --git a/src/test/run-make/unicode-input/Makefile b/src/test/run-make/unicode-input/Makefile
index 529bfc24a40..f834a85cdcc 100644
--- a/src/test/run-make/unicode-input/Makefile
+++ b/src/test/run-make/unicode-input/Makefile
@@ -1,9 +1,21 @@
 -include ../tools.mk
 
+# This test attempts to run rustc itself from the compiled binary; but
+# that means that you need to set the LD_LIBRARY_PATH for rustc itself
+# while running multiple_files, and that won't work for stage1.
+
 # FIXME ignore windows
 ifndef IS_WINDOWS
+ifeq ($(RUST_BUILD_STAGE),1)
+DOTEST=
+else
+DOTEST=dotest
+endif
+endif
+
+all: $(DOTEST)
 
-all:
+dotest:
 	# check that we don't ICE on unicode input, issue #11178
 	$(RUSTC) multiple_files.rs
 	$(call RUN,multiple_files)  "$(RUSTC)" "$(TMPDIR)"
@@ -12,8 +24,3 @@ all:
 	# correct length. issue #8706
 	$(RUSTC) span_length.rs
 	$(call RUN,span_length) "$(RUSTC)" "$(TMPDIR)"
-
-else
-all:
-
-endif