Avoid memory corruption with specpdl overflow + edebug (Bug#30481)

If grow_specpdl fails due to outgrowing max_specpdl_size, it will
signal an error *before* growing the specpdl array.  Therefore, when
handling the signal, specpdl_ptr points past the end of the specpdl
array and any further use of of specpdl before unwinding (e.g., if
edebug binds signal-hook-function) will cause memory corruption.
* src/eval.c (signal_or_quit): Don't call `signal-hook-function' if
the specpdl_ptr is already past the end of the specpdl array.
* test/src/eval-tests.el (eval-tests--exceed-specbind-limit)
(eval-exceed-specbind-with-signal-hook): New test & helper function.

1 files changed, 4 insertions, 1 deletions

diff --git a/src/eval.c b/src/eval.c
index e05a17f7b4b..ca1eb84ff3f 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -1553,7 +1553,10 @@ signal_or_quit (Lisp_Object error_symbol, Lisp_Object data, bool keyboard_quit)
 
   /* This hook is used by edebug.  */
   if (! NILP (Vsignal_hook_function)
-      && ! NILP (error_symbol))
+      && ! NILP (error_symbol)
+      /* Don't try to call a lisp function if we've already overflowed
+         the specpdl stack.  */
+      && specpdl_ptr < specpdl + specpdl_size)
     {
       /* Edebug takes care of restoring these variables when it exits.  */
       if (lisp_eval_depth + 20 > max_lisp_eval_depth)