Sync with Tramp 2.5.3.2. Don't merge with master

* doc/misc/tramp.texi (Android shell setup): Rework.
(Frequently Asked Questions): Improve recommendations for speeding up.

* doc/misc/trampver.texi:
* lisp/net/trampver.el: Change version to "2.5.4-pre".

* lisp/net/tramp-adb.el (tramp-methods): Use "%d".
(tramp-adb-handle-directory-files-and-attributes): Fix "." and
".." in listing.
(tramp-adb-handle-file-attributes)
(tramp-adb-handle-directory-files-and-attributes)
(tramp-adb-handle-file-name-all-completions): Pipe "ls" output
through "cat", in order to avoid quoting special characters.
(tramp-adb-maybe-open-connection): Compute args from `tramp-login-args'.

* lisp/net/tramp-compat.el (tramp-compat-replace-regexp-in-region):
New defalias.

* lisp/net/tramp-gvfs.el (tramp-gvfs-do-copy-or-rename-file):
Adapt check for proper remote command.

* lisp/net/tramp.el (tramp-methods): Adapt docstring.
(tramp-handle-make-process): Check for adb device if indicated.
(tramp-get-remote-tmpdir): Cache result in temporary connection
property.

* test/lisp/net/tramp-tests.el  (tramp-test17-insert-directory)
(tramp-test22-file-times, tramp--test-utf8): Adapt tests.
(tramp--test-shell-file-name): Do not depend on `tramp--test-adb-p'.
(tramp-test46-unload): Ignore autoload functions in
`tramp-file-name' structure tests, since `tramp-file-name-handler'
is also autoloaded in Emacs 29.

8 files changed, 132 insertions, 54 deletions

diff --git a/doc/misc/tramp.texi b/doc/misc/tramp.texi
index 3dc6da6e7d0..8ab678a1e10 100644
--- a/doc/misc/tramp.texi
+++ b/doc/misc/tramp.texi
@@ -2704,14 +2704,31 @@ where @samp{192.168.0.26} is the Android device's IP address.
 (@pxref{Predefined connection information}).
 
 @item
-@value{tramp} requires preserving @env{PATH} environment variable from
-user settings.  Android devices prefer @file{/system/xbin} path over
-@file{/system/bin}.  Both of these are set as follows:
+On the Android device the directory names are prefixed with an
+application specific prefix, which is
+@file{/data/data/com.termux/files/usr/bin} instead of @file{/usr/bin}
+in the @code{Termux} case.  You must adapt the file names in
+@code{tramp-remote-path}, for example via connection-local
+@ifinfo
+settings (@pxref{Connection Variables, , , emacs}):
+@end ifinfo
+@ifnotinfo
+settings:
+@end ifnotinfo
 
 @lisp
 @group
-(add-to-list 'tramp-remote-path 'tramp-own-remote-path)
-(add-to-list 'tramp-remote-path "/system/xbin")
+(connection-local-set-profile-variables
+ 'tramp-connection-local-termux-profile
+ `((tramp-remote-path
+    . ,(mapcar
+        (lambda (x)
+          (if (stringp x) (concat "/data/data/com.termux/files" x) x))
+        (copy-tree tramp-remote-path)))))
+
+(connection-local-set-profiles
+ '(:application tramp :machine "192.168.0.26")
+ 'tramp-connection-local-termux-profile)
 @end group
 @end lisp
 
@@ -2720,7 +2737,9 @@ When the Android device is not @samp{rooted}, specify a writable
 directory for temporary files:
 
 @lisp
-(add-to-list 'tramp-remote-process-environment "TMPDIR=$HOME")
+(add-to-list 'tramp-connection-properties
+             (list (regexp-quote "192.168.0.26")
+                   "tmpdir" "/data/data/com.termux/files/home/tmp"))
 @end lisp
 
 @item
@@ -2748,11 +2767,17 @@ the previous example, fix the connection properties as follows:
 @group
 (add-to-list 'tramp-connection-properties
              (list (regexp-quote "android") "remote-shell" "sh"))
+(add-to-list 'tramp-connection-properties
+             (list (regexp-quote "android")
+                   "tmpdir" "/data/data/com.termux/files/home/tmp"))
+(connection-local-set-profiles
+ '(:application tramp :machine "android")
+ 'tramp-connection-local-termux-profile)
 @end group
 @end lisp
 
 @noindent
-Open a remote connection with a more concise command @kbd{C-x C-f
+Open a remote connection with the more concise command @kbd{C-x C-f
 @trampfn{ssh,android,} @key{RET}}.
 @end itemize
 
@@ -4385,9 +4410,12 @@ authentication delays.  During these operations, @value{tramp}'s
 responsiveness slows down.  Some suggestions within the scope of
 @value{tramp}'s settings include:
 
+@itemize @minus
+@item
 Use an external method, such as @option{scp}, which are faster than
-internal methods.
+internal methods for large files.
 
+@item
 Keep the file @code{tramp-persistency-file-name}, which is where
 @value{tramp} caches remote information about hosts and files.  Caching
 is enabled by default.  Don't disable it.
@@ -4398,6 +4426,7 @@ files are not independently updated outside @value{tramp}'s control.
 That cache cleanup will be necessary if the remote directories or
 files are updated independent of @value{tramp}.
 
+@item
 Disable version control to avoid delays:
 
 @lisp
@@ -4417,9 +4446,17 @@ about, for example:
 (setq vc-handled-backends '(SVN Git))
 @end lisp
 
+@item
+@vindex remote-file-name-inhibit-locks
+Disable file locks.  Set @code{remote-file-name-inhibit-locks} to
+@code{t} if you know that different Emacs sessions are not modifying
+the same remote file.
+
+@item
 Disable excessive traces.  Set @code{tramp-verbose} to 3 or lower,
 default being 3.  Increase trace levels temporarily when hunting for
 bugs.
+@end itemize
 
 
 @item
diff --git a/doc/misc/trampver.texi b/doc/misc/trampver.texi
index 8352fed1b70..6aa5ffe1b3a 100644
--- a/doc/misc/trampver.texi
+++ b/doc/misc/trampver.texi
@@ -8,7 +8,7 @@
 @c In the Tramp GIT, the version numbers are auto-frobbed from
 @c tramp.el, and the bug report address is auto-frobbed from
 @c configure.ac.
-@set trampver 2.5.3.28.2
+@set trampver 2.5.4-pre
 @set trampurl https://www.gnu.org/software/tramp/
 @set tramp-bug-report-address tramp-devel@@gnu.org
 @set emacsver 25.1
diff --git a/lisp/net/tramp-adb.el b/lisp/net/tramp-adb.el
index 1fe10a560b1..53796503bcd 100644
--- a/lisp/net/tramp-adb.el
+++ b/lisp/net/tramp-adb.el
@@ -95,7 +95,7 @@ It is used for TCP/IP devices."
  (add-to-list 'tramp-methods
               `(,tramp-adb-method
                 (tramp-login-program ,tramp-adb-program)
-                (tramp-login-args    (("shell")))
+                (tramp-login-args    (("-s" "%d") ("shell")))
                 (tramp-direct-async  t)
                 (tramp-tmpdir        "/data/local/tmp")
                 (tramp-default-port  5555)))
@@ -256,7 +256,7 @@ arguments to pass to the OPERATION."
           v localname (format "file-attributes-%s" id-format)
         (and
          (tramp-adb-send-command-and-check
-          v (format "%s -d -l %s"
+          v (format "%s -d -l %s | cat"
                     (tramp-adb-get-ls-command v)
                     (tramp-shell-quote-argument localname)))
          (with-current-buffer (tramp-get-buffer v)
@@ -315,20 +315,25 @@ arguments to pass to the OPERATION."
                                full match id-format nosort count)
          (with-current-buffer (tramp-get-buffer v)
            (when (tramp-adb-send-command-and-check
-                  v (format "%s -a -l %s"
+                  v (format "%s -a -l %s | cat"
                             (tramp-adb-get-ls-command v)
                             (tramp-shell-quote-argument localname)))
-             ;; We insert also filename/. and filename/.., because "ls" doesn't.
-             ;; Looks like it does include them in toybox, since Android 6.
+             ;; We insert also filename/. and filename/.., because
+             ;; "ls" doesn't on some file systems, like "sdcard".
              (unless (re-search-backward "\\.$" nil t)
                (narrow-to-region (point-max) (point-max))
                (tramp-adb-send-command
-                v (format "%s -d -a -l %s %s"
+                v (format "%s -d -a -l %s %s | cat"
                           (tramp-adb-get-ls-command v)
                           (tramp-shell-quote-argument
                            (tramp-compat-file-name-concat localname "."))
                           (tramp-shell-quote-argument
                            (tramp-compat-file-name-concat localname ".."))))
+               (tramp-compat-replace-regexp-in-region
+                (regexp-quote
+                 (tramp-compat-file-name-unquote
+                  (file-name-as-directory localname)))
+                "" (point-min))
                (widen)))
            (tramp-adb-sh-fix-ls-output)
            (let ((result (tramp-do-parse-file-attributes-with-ls
@@ -474,7 +479,7 @@ Emacs dired can't find files."
    (with-parsed-tramp-file-name (expand-file-name directory) nil
      (with-tramp-file-property v localname "file-name-all-completions"
        (tramp-adb-send-command
-        v (format "%s -a %s"
+        v (format "%s -a %s | cat"
                   (tramp-adb-get-ls-command v)
                   (tramp-shell-quote-argument localname)))
        (mapcar
@@ -485,9 +490,8 @@ Emacs dired can't find files."
         (with-current-buffer (tramp-get-buffer v)
           (delete-dups
            (append
-            ;; In older Android versions, "." and ".." are not
-            ;; included.  In newer versions (toybox, since Android 6)
-            ;; they are.  We fix this by `delete-dups'.
+            ;; On some file systems like "sdcard", "." and ".." are
+            ;; not included.  We fix this by `delete-dups'.
             '("." "..")
             (delq
              nil
@@ -1270,9 +1274,8 @@ connection if a previous connection has died for some reason."
         (with-tramp-progress-reporter vec 3 "Opening adb shell connection"
           (let* ((coding-system-for-read 'utf-8-dos) ; Is this correct?
                  (process-connection-type tramp-process-connection-type)
-                 (args (if (> (length host) 0)
-                           (list "-s" device "shell")
-                         (list "shell")))
+                 (args (tramp-expand-args
+                        vec 'tramp-login-args ?d (or device "")))
                  (p (let ((default-directory
                             tramp-compat-temporary-file-directory))
                       (apply #'start-process (tramp-get-connection-name vec) buf
diff --git a/lisp/net/tramp-compat.el b/lisp/net/tramp-compat.el
index aead1dedd24..91c450eee4e 100644
--- a/lisp/net/tramp-compat.el
+++ b/lisp/net/tramp-compat.el
@@ -398,6 +398,29 @@ CONDITION can also be a list of error conditions."
                          (car components))
                  (cdr components)))))))
 
+;; Function `replace-regexp-in-region' is new in Emacs 28.1.
+(defalias 'tramp-compat-replace-regexp-in-region
+  (if (fboundp 'replace-regexp-in-region)
+      #'replace-regexp-in-region
+    (lambda (regexp replacement &optional start end)
+      (if start
+          (when (< start (point-min))
+            (error "Start before start of buffer"))
+        (setq start (point)))
+      (if end
+          (when (> end (point-max))
+            (error "End after end of buffer"))
+        (setq end (point-max)))
+      (save-excursion
+        (let ((matches 0)
+              (case-fold-search nil))
+          (goto-char start)
+          (while (re-search-forward regexp end t)
+            (replace-match replacement t)
+            (setq matches (1+ matches)))
+          (and (not (zerop matches))
+               matches))))))
+
 (dolist (elt (all-completions "tramp-compat-" obarray 'functionp))
   (put (intern elt) 'tramp-suppress-trace t))
 
diff --git a/lisp/net/tramp-gvfs.el b/lisp/net/tramp-gvfs.el
index e0b786f015a..207e588f347 100644
--- a/lisp/net/tramp-gvfs.el
+++ b/lisp/net/tramp-gvfs.el
@@ -1053,12 +1053,14 @@ file names."
                        ;; code in case of direct copy/move.  Apply
                        ;; sanity checks.
                        (or (not equal-remote)
-                           (tramp-gvfs-send-command
-                            v "gvfs-info" (tramp-gvfs-url-file-name newname))
-                           (eq op 'copy)
-                           (not (tramp-gvfs-send-command
-                                 v "gvfs-info"
-                                 (tramp-gvfs-url-file-name filename)))))
+                           (and
+                            (tramp-gvfs-send-command
+                             v "gvfs-info"
+                             (tramp-gvfs-url-file-name newname))
+                            (or (eq op 'copy)
+                                (not (tramp-gvfs-send-command
+                                      v "gvfs-info"
+                                      (tramp-gvfs-url-file-name filename)))))))
 
                 (if (or (not equal-remote)
                         (and equal-remote
diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el
index 95d674911f3..f7243625c7c 100644
--- a/lisp/net/tramp.el
+++ b/lisp/net/tramp.el
@@ -257,6 +257,7 @@ pair of the form (KEY VALUE).  The following KEYs are defined:
       argument if it is supported.
     - \"%y\" is replaced by the `tramp-scp-force-scp-protocol'
       argument if it is supported.
+    - \"%d\" is replaced by the device detected by `tramp-adb-get-device'.
 
     The existence of `tramp-login-args', combined with the
     absence of `tramp-copy-args', is an indication that the
@@ -4244,6 +4245,8 @@ substitution.  SPEC-LIST is a list of char/value pairs used for
           ;; is different between tramp-sh.el, and tramp-adb.el or
           ;; tramp-sshfs.el.
           (let* ((sh-file-name-handler-p (tramp-sh-file-name-handler-p v))
+                 (adb-file-name-handler-p
+                  (tramp-adb-file-name-p (tramp-make-tramp-file-name v)))
                  (login-program
                   (tramp-get-method-parameter v 'tramp-login-program))
                  ;; We don't create the temporary file.  In fact, it
@@ -4263,6 +4266,10 @@ substitution.  SPEC-LIST is a list of char/value pairs used for
                   (when sh-file-name-handler-p
                     (tramp-compat-funcall
                      'tramp-ssh-controlmaster-options v)))
+                 (device
+                  (when adb-file-name-handler-p
+                    (tramp-compat-funcall
+                     'tramp-adb-get-device v)))
                  login-args p)
 
             ;; Replace `login-args' place holders.  Split
@@ -4279,7 +4286,7 @@ substitution.  SPEC-LIST is a list of char/value pairs used for
                  v 'tramp-login-args
                  ?h (or host "") ?u (or user "") ?p (or port "")
                  ?c (format-spec (or options "") (format-spec-make ?t tmpfile))
-                 ?l ""))))
+                 ?d (or device "") ?l ""))))
              p (make-process
                 :name name :buffer buffer
                 :command (append `(,login-program) login-args command)
@@ -5451,7 +5458,7 @@ This handles also chrooted environments, which are not regarded as local."
 
 (defun tramp-get-remote-tmpdir (vec)
   "Return directory for temporary files on the remote host identified by VEC."
-  (with-tramp-connection-property vec "tmpdir"
+  (with-tramp-connection-property (tramp-get-process vec) "remote-tmpdir"
     (let ((dir
            (tramp-make-tramp-file-name
             vec (or (tramp-get-method-parameter vec 'tramp-tmpdir) "/tmp"))))
diff --git a/lisp/net/trampver.el b/lisp/net/trampver.el
index e14c26d8999..80a533cb992 100644
--- a/lisp/net/trampver.el
+++ b/lisp/net/trampver.el
@@ -7,7 +7,7 @@
 ;; Maintainer: Michael Albinus <michael.albinus@gmx.de>
 ;; Keywords: comm, processes
 ;; Package: tramp
-;; Version: 2.5.3.28.2
+;; Version: 2.5.4-pre
 ;; Package-Requires: ((emacs "25.1"))
 ;; Package-Type: multi
 ;; URL: https://www.gnu.org/software/tramp/
@@ -40,7 +40,7 @@
 ;; ./configure" to change them.
 
 ;;;###tramp-autoload
-(defconst tramp-version "2.5.3.28.2"
+(defconst tramp-version "2.5.4-pre"
   "This version of Tramp.")
 
 ;;;###tramp-autoload
@@ -78,7 +78,7 @@
 ;; Check for Emacs version.
 (let ((x   (if (not (string-lessp emacs-version "25.1"))
       "ok"
-    (format "Tramp 2.5.3.28.2 is not fit for %s"
+    (format "Tramp 2.5.4-pre is not fit for %s"
             (replace-regexp-in-string "\n" "" (emacs-version))))))
   (unless (string-equal "ok" x) (error "%s" x)))
 
diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index cee3651f213..06920ad5343 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -3138,20 +3138,21 @@ This tests also `file-directory-p' and `file-accessible-directory-p'."
               (goto-char (point-min))
               (should
                (looking-at-p (format "^.+ %s/$" (regexp-quote tmp-name1)))))
-            (with-temp-buffer
-              (insert-directory
-               (file-name-as-directory tmp-name1) "-al" nil 'full-directory-p)
-              (goto-char (point-min))
-              (should
-               (looking-at-p
-                (concat
-                 ;; There might be a summary line.
-                 "\\(total.+[[:digit:]]+ ?[kKMGTPEZY]?i?B?\n\\)?"
-                 ;; We don't know in which order ".", ".." and "foo" appear.
-                 (format
-                  "\\(.+ %s\\( ->.+\\)?\n\\)\\{%d\\}"
-                  (regexp-opt (directory-files tmp-name1))
-                  (length (directory-files tmp-name1)))))))
+            (let ((directory-files (directory-files tmp-name1)))
+              (with-temp-buffer
+                (insert-directory
+                 (file-name-as-directory tmp-name1) "-al" nil 'full-directory-p)
+                (goto-char (point-min))
+                (should
+                 (looking-at-p
+                  (concat
+                   ;; There might be a summary line.
+                   "\\(total.+[[:digit:]]+ ?[kKMGTPEZY]?i?B?\n\\)?"
+                   ;; We don't know in which order ".", ".." and "foo" appear.
+                   (format
+                    "\\(.+ %s\\( ->.+\\)?\n\\)\\{%d\\}"
+                    (regexp-opt directory-files)
+                    (length directory-files)))))))
 
             ;; Check error cases.
             (when (and (tramp--test-supports-set-file-modes-p)
@@ -4005,7 +4006,8 @@ This tests also `make-symbolic-link', `file-truename' and `add-name-to-file'."
                             (file-attributes tmp-name1))))
             ;; Skip the test, if the remote handler is not able to set
             ;; the correct time.
-            (skip-unless (set-file-times tmp-name1 (seconds-to-time 1)))
+            ;; Some remote machines cannot resolve seconds.  So we use a minute.
+            (skip-unless (set-file-times tmp-name1 (seconds-to-time 60)))
             ;; Dumb remote shells without perl(1) or stat(1) are not
             ;; able to return the date correctly.  They say "don't know".
             (unless (tramp-compat-time-equal-p
@@ -4016,7 +4018,7 @@ This tests also `make-symbolic-link', `file-truename' and `add-name-to-file'."
                (tramp-compat-time-equal-p
                 (tramp-compat-file-attribute-modification-time
                  (file-attributes tmp-name1))
-                (seconds-to-time 1)))
+                (seconds-to-time 60)))
               (write-region "bla" nil tmp-name2)
               (should (file-exists-p tmp-name2))
               (should (file-newer-than-file-p tmp-name2 tmp-name1))
@@ -4027,12 +4029,12 @@ This tests also `make-symbolic-link', `file-truename' and `add-name-to-file'."
               ;; regular files, there shouldn't be a difference.
               (when (tramp--test-emacs28-p)
                 (with-no-warnings
-                  (set-file-times tmp-name1 (seconds-to-time 1) 'nofollow)
+                  (set-file-times tmp-name1 (seconds-to-time 60) 'nofollow)
                   (should
                    (tramp-compat-time-equal-p
                     (tramp-compat-file-attribute-modification-time
                      (file-attributes tmp-name1))
-                    (seconds-to-time 1)))))))
+                    (seconds-to-time 60)))))))
 
         ;; Cleanup.
         (ignore-errors
@@ -4439,7 +4441,10 @@ This tests also `make-symbolic-link', `file-truename' and `add-name-to-file'."
 
 (defun tramp--test-shell-file-name ()
   "Return default remote shell."
-  (if (tramp--test-adb-p) "/system/bin/sh" "/bin/sh"))
+  (if (file-exists-p
+       (concat
+        (file-remote-p tramp-test-temporary-file-directory) "/system/bin/sh"))
+      "/system/bin/sh" "/bin/sh"))
 
 (ert-deftest tramp-test28-process-file ()
   "Check `process-file'."
@@ -6815,7 +6820,8 @@ Use the \"ls\" command."
        "银河系漫游指南系列"
        "Автостопом по гала́ктике"
        ;; Use codepoints without a name.  See Bug#31272.
-       "™›šbung"
+       ;; Works on some Android systems only.
+       (unless (tramp--test-adb-p) "™›šbung")
        ;; Use codepoints from Supplementary Multilingual Plane (U+10000
        ;; to U+1FFFF).
        "🌈🍒👋")
@@ -7354,7 +7360,7 @@ Since it unloads Tramp, it shall be the last test to run."
   (should-not (cl--find-class 'tramp-file-name))
   (mapatoms
    (lambda (x)
-     (and (functionp x)
+     (and (functionp x) (null (autoloadp (symbol-function x)))
           (string-match-p "tramp-file-name" (symbol-name x))
           (ert-fail (format "Structure function `%s' still exists" x)))))