Avoid recursive process filters in lisp/jsonrpc.el (bug#60088)

jsonrpc.el may lose JSON-RPC messages because of recursive process
filters.  The problem happens in jsonrpc.el's jsonrpc--process-filter.

The code of the process filter didn't expect to be called recursively
and fails in that case.

But that can happen if the three conditions are verified.

1. the client code invoked by its jsonrpc--connection-receive inside
   the process filter callee immediately sends follow-up input to
   process within the same Lisp stack.  This is a common scenario,
   especially during LSP initialiation sequence used by Eglot, a
   jsonrpc.el client.

2. that follow-up message is large enough for process-send-string to
   send the input in bunches (output from processes can arrive in
   between bunches).

3. the process happens to have already some more output ready

The fix in this commit detects recursive invocations and immediately
re-schedules them as non-recursive calls to the
jsonrpc--process-filter (but started from timers).

* lisp/jsonrpc.el (jsonrpc--process-filter): Rework.
(Version): Bump to 1.0.16.

1 files changed, 17 insertions, 2 deletions

diff --git a/lisp/jsonrpc.el b/lisp/jsonrpc.el
index 90833e1c1d7..2d562610b3f 100644
--- a/lisp/jsonrpc.el
+++ b/lisp/jsonrpc.el
@@ -4,7 +4,7 @@
 
 ;; Author: João Távora <joaotavora@gmail.com>
 ;; Keywords: processes, languages, extensions
-;; Version: 1.0.15
+;; Version: 1.0.16
 ;; Package-Requires: ((emacs "25.2"))
 
 ;; This is a GNU ELPA :core package.  Avoid functionality that is not
@@ -548,11 +548,26 @@ With optional CLEANUP, kill any associated buffers."
         (delete-process proc)
         (funcall (jsonrpc--on-shutdown connection) connection)))))
 
-(defun jsonrpc--process-filter (proc string)
+(defvar jsonrpc--in-process-filter nil
+  "Non-nil if inside `jsonrpc--process-filter'.")
+
+(cl-defun jsonrpc--process-filter (proc string)
   "Called when new data STRING has arrived for PROC."
+  (when jsonrpc--in-process-filter
+    ;; Problematic recursive process filters may happen if
+    ;; `jsonrpc--connection-receive', called by us, eventually calls
+    ;; client code which calls `process-send-string' (which see) to,
+    ;; say send a follow-up message.  If that happens to writes enough
+    ;; bytes for pending output to be received, we will lose JSONRPC
+    ;; messages.  In that case, remove recursiveness by re-scheduling
+    ;; ourselves to run from within a timer as soon as possible
+    ;; (bug#60088)
+    (run-at-time 0 nil #'jsonrpc--process-filter proc string)
+    (cl-return-from jsonrpc--process-filter))
   (when (buffer-live-p (process-buffer proc))
     (with-current-buffer (process-buffer proc)
       (let* ((inhibit-read-only t)
+             (jsonrpc--in-process-filter t)
              (connection (process-get proc 'jsonrpc-connection))
              (expected-bytes (jsonrpc--expected-bytes connection)))
         ;; Insert the text, advancing the process marker.