Use a dedicated type to represent interpreted-function valuesscratch/interpreted-function

Change `function` so that when evaluating #'(lambda ...)
we return an object of type `interpreted-function` rather than
a list starting with one of `lambda` or `closure`.
The new type reuses the existing PVEC_CLOSURE (nee PVEC_COMPILED)
tag and tries to align the corresponding elements:

- the arglist, the docstring, and the interactive-form go in the
  same slots as for byte-code functions.
- the body of the function goes in the slot used for the bytecode string.
- the lexical context goes in the slot used for the constants of
  bytecoded functions.

The first point above means that `help-function-arglist`,
`documentation`, and `interactive-form`s don't need to
distinguish interpreted and bytecode functions any more.

Main benefits of the change:

- We can now reliably distinguish a list from a function value.
- `cl-defmethod` can dispatch on `interactive-function` and `closure`.
  Dispatch on `function` also works now for interpreted functions but still
  won't work for functions represented as lists or as symbols, of course.
- Function values are now self-evaluating.  That was alrready the case
  when byte-compiled, but not when interpreted since
  (eval '(closure ...)) signals a void-function error.
  That also avoids false-positive warnings about "don't quote your lambdas"
  when doing things like `(mapcar ',func ...)`.

* src/eval.c (Fmake_interpreted_closure): New function.
(Ffunction): Use it and change calling convention of
`Vinternal_make_interpreted_closure_function`.
(FUNCTIONP, Fcommandp, eval_sub, funcall_general, funcall_lambda)
(Ffunc_arity, lambda_arity): Simplify.
(funcall_lambda): Adjust to new representation.
(syms_of_eval): `defsubr` the new function.  Remove definition of `Qclosure`.

* lisp/emacs-lisp/cconv.el (cconv-make-interpreted-closure):
Change calling convention and use `make-interpreted-closure`.

* src/data.c (Fcl_type_of): Distinguish `byte-code-function`s from
`interpreted-function`s.
(Fclosurep, finterpreted_function_p): New functions.
(Fbyte_code_function_p): Don't be confused by `interpreted-function`s.
(Finteractive_form, Fcommand_modes): Simplify.
(syms_of_data): Define new type symbols and `defsubr` the two
new functions.

* lisp/emacs-lisp/cl-print.el (cl-print-object) <interpreted-function>:
New method.

* lisp/emacs-lisp/oclosure.el (oclosure): Refine the parent
to be `closure`.
(oclosure--fix-type, oclosure-type): Simplify.
(oclosure--copy, oclosure--get, oclosure--set): Adjust to
new representation.

* src/callint.c (Fcall_interactively): Adjust to new representation.

* src/lread.c (bytecode_from_rev_list):

* lisp/simple.el (function-documentation):
* lisp/help.el (help-function-arglist): Remove the old `closure` case
and adjust the byte-code case so it handles `interpreted-function`s.

* lisp/emacs-lisp/cl-preloaded.el (closure): New type.
(byte-code-function): Add it as a parent.
(interpreted-function): Adjust parent (the type itself was already
added earlier by accident).

* lisp/emacs-lisp/bytecomp.el (byte-compile--reify-function): Adjust to
new representation.
(byte-compile): Use `interpreted-function-p`.

* lisp/emacs-lisp/byte-opt.el (byte-compile-inline-expand): Adjust to
new representation.
(side-effect-free-fns): Add `interpreted-function-p` and `closurep`.

* src/profiler.c (trace_hash, ffunction_equal): Simplify.
* lisp/profiler.el (profiler-function-equal): Simplify.

* lisp/emacs-lisp/nadvice.el (advice--interactive-form-1):
Use `interpreted-function-p`; adjust to new representation; and take
advantage of the fact that function values are now self-evaluating.

* lisp/emacs-lisp/lisp-mode.el (closure):
Remove `lisp-indent-function` property.

* lisp/emacs-lisp/disass.el (disassemble-internal): Adjust to
new representation.
* lisp/emacs-lisp/edebug.el (edebug--strip-instrumentation):
Use `interpreted-function-p`.
* lisp/emacs-lisp/comp-common.el (comp-known-type-specifiers):
Add `closurep` and `interpreted-function-p`.

* test/lisp/help-fns-tests.el (help-fns-test-lisp-defun): Adjust to
more precise type info in `describe-function`.
* test/lisp/erc/resources/erc-d/erc-d-tests.el (erc-d--render-entries):
Use `interpreted-function-p`.
* test/lisp/emacs-lisp/macroexp-resources/vk.el (vk-f4, vk-f5):
Don't hardcode function values.

* doc/lispref/functions.texi (Anonymous Functions): Don't suggest that
function values are lists.  Reword "self-quoting" to reflect the
fact that #' doesn't return the exact same object.  Update examples
with the new shape of the return value.

* doc/lispref/variables.texi (Lexical Binding):
* doc/lispref/lists.texi (Rearrangement):
* doc/lispref/control.texi (Handling Errors): Update examples to reflect
new representation of function values.

8 files changed, 97 insertions, 52 deletions

diff --git a/doc/lispref/compile.texi b/doc/lispref/compile.texi
index 00602198da5..3fbf39b349d 100644
--- a/doc/lispref/compile.texi
+++ b/doc/lispref/compile.texi
@@ -37,7 +37,7 @@ variable binding for @code{no-byte-compile} into it, like this:
 * Docs and Compilation::        Dynamic loading of documentation strings.
 * Eval During Compile::         Code to be evaluated when you compile.
 * Compiler Errors::             Handling compiler error messages.
-* Byte-Code Objects::           The data type used for byte-compiled functions.
+* Closure Objects::             The data type used for byte-compiled functions.
 * Disassembly::                 Disassembling byte-code; how to read byte-code.
 @end menu
 
@@ -120,7 +120,7 @@ replacing the previous definition with the compiled one.  The function
 definition of @var{symbol} must be the actual code for the function;
 @code{byte-compile} does not handle function indirection.  The return
 value is the byte-code function object which is the compiled
-definition of @var{symbol} (@pxref{Byte-Code Objects}).
+definition of @var{symbol} (@pxref{Closure Objects}).
 
 @example
 @group
@@ -487,21 +487,22 @@ string for details.
 using @code{error}.  If so, set @code{byte-compile-error-on-warn} to a
 non-@code{nil} value.
 
-@node Byte-Code Objects
-@section Byte-Code Function Objects
+@node Closure Objects
+@section Closure Function Objects
 @cindex compiled function
 @cindex byte-code function
 @cindex byte-code object
 
-  Byte-compiled functions have a special data type: they are
-@dfn{byte-code function objects}.  Whenever such an object appears as
-a function to be called, Emacs uses the byte-code interpreter to
-execute the byte-code.
+  Byte-compiled functions use a special data type: they are closures.
+Closures are used both for byte-compiled Lisp functions as well as for
+interpreted Lisp functions.  Whenever such an object appears as
+a function to be called, Emacs uses the appropriate interpreter to
+execute either the byte-code or the non-compiled Lisp code.
 
-  Internally, a byte-code function object is much like a vector; its
+  Internally, a closure is much like a vector; its
 elements can be accessed using @code{aref}.  Its printed
 representation is like that for a vector, with an additional @samp{#}
-before the opening @samp{[}.  It must have at least four elements;
+before the opening @samp{[}.  It must have at least three elements;
 there is no maximum number, but only the first six elements have any
 normal use.  They are:
 
@@ -515,20 +516,28 @@ zero to 6, and the maximum number of arguments in bits 8 to 14.  If
 the argument list uses @code{&rest}, then bit 7 is set; otherwise it's
 cleared.
 
-If @var{argdesc} is a list, the arguments will be dynamically bound
+When the closure is a byte-code function,
+if @var{argdesc} is a list, the arguments will be dynamically bound
 before executing the byte code.  If @var{argdesc} is an integer, the
 arguments will be instead pushed onto the stack of the byte-code
 interpreter, before executing the code.
 
-@item byte-code
-The string containing the byte-code instructions.
+@item code
+For interpreted functions, this element is the (non-empty) list of Lisp
+forms that make up the function's body.  For byte-compiled functions, it
+is the string containing the byte-code instructions.
 
 @item constants
-The vector of Lisp objects referenced by the byte code.  These include
-symbols used as function names and variable names.
+For byte-compiled functions, this holds the vector of Lisp objects
+referenced by the byte code.  These include symbols used as function
+names and variable names.
+For interpreted functions, this is @code{nil} if the function is using the old
+dynamically scoped dialect of Emacs Lisp, and otherwise it holds the
+function's lexical environment.
 
 @item stacksize
-The maximum stack size this function needs.
+The maximum stack size this function needs.  This element is left unused
+for interpreted functions.
 
 @item docstring
 The documentation string (if any); otherwise, @code{nil}.  The value may
@@ -558,8 +567,8 @@ representation.  It is the definition of the command
 @code{make-byte-code}:
 
 @defun make-byte-code &rest elements
-This function constructs and returns a byte-code function object
-with @var{elements} as its elements.
+This function constructs and returns a closure which represents the
+byte-code function object with @var{elements} as its elements.
 @end defun
 
   You should not try to come up with the elements for a byte-code
@@ -567,6 +576,20 @@ function yourself, because if they are inconsistent, Emacs may crash
 when you call the function.  Always leave it to the byte compiler to
 create these objects; it makes the elements consistent (we hope).
 
+The primitive way to create an interpreted function is with
+@code{make-interpreted-closure}:
+
+@defun make-interpreted-closure args body env &optional docstring iform
+This function constructs and returns a closure representing the
+interpreted function with arguments @var{args} and whose body is made of
+@var{body} which must be a non-@code{nil} list of Lisp forms.  @var{env} is the
+lexical environment in the same form as used with @code{eval}
+(@pxref{Eval}).  The documentation @var{docstring} if non-@code{nil} should be
+a string, and the interactive form @var{iform} if non-@code{nil} should be of
+the form @w{@code{(interactive @var{arg-descriptor})}} (@pxref{Using
+Interactive}).
+@end defun
+
 @node Disassembly
 @section Disassembled Byte-Code
 @cindex disassembled byte-code
@@ -595,7 +618,7 @@ name of an existing buffer.  Then the output goes there, at point, and
 point is left before the output.
 
 The argument @var{object} can be a function name, a lambda expression
-(@pxref{Lambda Expressions}), or a byte-code object (@pxref{Byte-Code
+(@pxref{Lambda Expressions}), or a byte-code object (@pxref{Closure
 Objects}).  If it is a lambda expression, @code{disassemble} compiles
 it and disassembles the resulting compiled code.
 @end deffn
diff --git a/doc/lispref/control.texi b/doc/lispref/control.texi
index f9f3389c398..46024e2fdee 100644
--- a/doc/lispref/control.texi
+++ b/doc/lispref/control.texi
@@ -2411,7 +2411,7 @@ point where we signaled the original error:
 @group
 Debugger entered--Lisp error: (error "Oops")
   signal(error ("Oops"))
-  (closure (t) (err) (signal 'error (cdr err)))((user-error "Oops"))
+  #f(lambda (err) [t] (signal 'error (cdr err)))((user-error "Oops"))
   user-error("Oops")
   @dots{}
   eval((handler-bind ((user-error (lambda (err) @dots{}
diff --git a/doc/lispref/elisp.texi b/doc/lispref/elisp.texi
index ec93a0b9c8a..339272d1f05 100644
--- a/doc/lispref/elisp.texi
+++ b/doc/lispref/elisp.texi
@@ -323,7 +323,7 @@ Programming Types
 * Macro Type::          A method of expanding an expression into another
                           expression, more fundamental but less pretty.
 * Primitive Function Type::     A function written in C, callable from Lisp.
-* Byte-Code Type::      A function written in Lisp, then compiled.
+* Closure Type::        A function written in Lisp, then compiled.
 * Record Type::         Compound objects with programmer-defined types.
 * Type Descriptors::    Objects holding information about types.
 * Autoload Type::       A type used for automatically loading seldom-used
@@ -657,7 +657,7 @@ Byte Compilation
 * Docs and Compilation::    Dynamic loading of documentation strings.
 * Eval During Compile::     Code to be evaluated when you compile.
 * Compiler Errors::         Handling compiler error messages.
-* Byte-Code Objects::       The data type used for byte-compiled functions.
+* Closure Objects::         The data type used for byte-compiled functions.
 * Disassembly::             Disassembling byte-code; how to read byte-code.
 
 Native Compilation
diff --git a/doc/lispref/functions.texi b/doc/lispref/functions.texi
index ff635fc54b2..c57de08460f 100644
--- a/doc/lispref/functions.texi
+++ b/doc/lispref/functions.texi
@@ -130,7 +130,7 @@ it also encloses an environment of lexical variable bindings.
 
 @item byte-code function
 A function that has been compiled by the byte compiler.
-@xref{Byte-Code Type}.
+@xref{Closure Type}.
 
 @item autoload object
 @cindex autoload object
@@ -227,6 +227,16 @@ Compilation}), or natively-compiled (@pxref{Native Compilation}), or
 a function loaded from a dynamic module (@pxref{Dynamic Modules}).
 @end defun
 
+@defun interpreted-function-p object
+This function returns @code{t} if @var{object} is an interpreted function.
+@end defun
+
+@defun closurep object
+This function returns @code{t} if @var{object} is a closure, which is
+a particular kind of function object.  Currently closures are used
+for all byte-code functions and all interpreted functions.
+@end defun
+
 @defun subr-arity subr
 This works like @code{func-arity}, but only for built-in functions and
 without symbol indirection.  It signals an error for non-built-in
@@ -1136,8 +1146,7 @@ Functions}).  @xref{describe-symbols example}, for a realistic example
 of this.
 
   When defining a lambda expression that is to be used as an anonymous
-function, you can in principle use any method to construct the list.
-But typically you should use the @code{lambda} macro, or the
+function, you should use the @code{lambda} macro, or the
 @code{function} special form, or the @code{#'} read syntax:
 
 @defmac lambda args [doc] [interactive] body@dots{}
@@ -1145,17 +1154,18 @@ This macro returns an anonymous function with argument list
 @var{args}, documentation string @var{doc} (if any), interactive spec
 @var{interactive} (if any), and body forms given by @var{body}.
 
-Under dynamic binding, this macro effectively makes @code{lambda}
-forms self-quoting: evaluating a form whose @sc{car} is @code{lambda}
-yields the form itself:
+For example, this macro makes @code{lambda} forms almost self-quoting:
+evaluating a form whose @sc{car} is @code{lambda} yields a value that is
+almost like the form itself:
 
 @example
 (lambda (x) (* x x))
-     @result{} (lambda (x) (* x x))
+     @result{} #f(lambda (x) :dynbind (* x x))
 @end example
 
-Note that when evaluating under lexical binding the result is a
-closure object (@pxref{Closures}).
+When evaluating under lexical binding the result is a similar
+closure object, where the @code{:dynbind} marker is replaced by the
+captured variables (@pxref{Closures}).
 
 The @code{lambda} form has one other effect: it tells the Emacs
 evaluator and byte-compiler that its argument is a function, by using
@@ -1164,8 +1174,8 @@ evaluator and byte-compiler that its argument is a function, by using
 
 @defspec function function-object
 @cindex function quoting
-This special form returns @var{function-object} without evaluating it.
-In this, it is similar to @code{quote} (@pxref{Quoting}).  But unlike
+This special form returns the function value of the @var{function-object}.
+In many ways, it is similar to @code{quote} (@pxref{Quoting}).  But unlike
 @code{quote}, it also serves as a note to the Emacs evaluator and
 byte-compiler that @var{function-object} is intended to be used as a
 function.  Assuming @var{function-object} is a valid lambda
@@ -1495,7 +1505,7 @@ distinguish between a function cell that is void and one set to
 @group
 (defun bar (n) (+ n 2))
 (symbol-function 'bar)
-     @result{} (lambda (n) (+ n 2))
+     @result{} #f(lambda (n) [t] (+ n 2))
 @end group
 @group
 (fset 'baz 'bar)
@@ -1608,7 +1618,7 @@ argument list and body forms as the remaining elements:
 @example
 ;; @r{lexical binding is enabled.}
 (lambda (x) (* x x))
-     @result{} (closure (t) (x) (* x x))
+     @result{} #f(lambda (x) [t] (* x x))
 @end example
 
 @noindent
diff --git a/doc/lispref/lists.texi b/doc/lispref/lists.texi
index 1409e51c0d4..06472539744 100644
--- a/doc/lispref/lists.texi
+++ b/doc/lispref/lists.texi
@@ -1249,7 +1249,7 @@ this is not guaranteed to happen):
 
 @group
 (symbol-function 'add-foo)
-     @result{} (lambda (x) (nconc '(foo) x))
+     @result{} #f(lambda (x) [t] (nconc '(foo) x))
 @end group
 
 @group
@@ -1267,7 +1267,7 @@ this is not guaranteed to happen):
 
 @group
 (symbol-function 'add-foo)
-     @result{} (lambda (x) (nconc '(foo 1 2 3 4) x))
+     @result{} #f(lambda (x) [t] (nconc '(foo 1 2 3 4) x))
 @end group
 @end smallexample
 @end defun
diff --git a/doc/lispref/objects.texi b/doc/lispref/objects.texi
index aa1e073042f..cf703aba9c8 100644
--- a/doc/lispref/objects.texi
+++ b/doc/lispref/objects.texi
@@ -244,7 +244,7 @@ latter are unique to Emacs Lisp.
 * Macro Type::          A method of expanding an expression into another
                           expression, more fundamental but less pretty.
 * Primitive Function Type::     A function written in C, callable from Lisp.
-* Byte-Code Type::      A function written in Lisp, then compiled.
+* Closure Type::        A function written in Lisp.
 * Record Type::         Compound objects with programmer-defined types.
 * Type Descriptors::    Objects holding information about types.
 * Autoload Type::       A type used for automatically loading seldom-used
@@ -1458,18 +1458,24 @@ with the name of the subroutine.
 @end group
 @end example
 
-@node Byte-Code Type
-@subsection Byte-Code Function Type
+@node Closure Type
+@subsection Closure Function Type
 
-@dfn{Byte-code function objects} are produced by byte-compiling Lisp
-code (@pxref{Byte Compilation}).  Internally, a byte-code function
-object is much like a vector; however, the evaluator handles this data
-type specially when it appears in a function call.  @xref{Byte-Code
-Objects}.
+@dfn{Closures} are function objects produced when turning a function
+definition into a function value.  Closures are used both for
+byte-compiled Lisp functions as well as for interpreted Lisp functions.
+Closures can be produced by byte-compiling Lisp code (@pxref{Byte
+Compilation}) or simply by evaluating a lambda expression without
+compiling it, resulting in an interpreted function.  Internally,
+a closure is much like a vector; however, the evaluator
+handles this data type specially when it appears in a function call.
+@xref{Closure Objects}.
 
 The printed representation and read syntax for a byte-code function
 object is like that for a vector, with an additional @samp{#} before the
-opening @samp{[}.
+opening @samp{[}.  When printed for human consumption, it is printed as
+a special kind of list with an additional @samp{#f} before the opening
+@samp{(}.
 
 @node Record Type
 @subsection Record Type
@@ -2042,10 +2048,7 @@ with references to further information.
 @xref{Buffer Basics, bufferp}.
 
 @item byte-code-function-p
-@xref{Byte-Code Type, byte-code-function-p}.
-
-@item compiled-function-p
-@xref{Byte-Code Type, compiled-function-p}.
+@xref{Closure Type, byte-code-function-p}.
 
 @item case-table-p
 @xref{Case Tables, case-table-p}.
@@ -2056,9 +2059,15 @@ with references to further information.
 @item char-table-p
 @xref{Char-Tables, char-table-p}.
 
+@item closurep
+@xref{What Is a Function, closurep}.
+
 @item commandp
 @xref{Interactive Call, commandp}.
 
+@item compiled-function-p
+@xref{Closure Type, compiled-function-p}.
+
 @item condition-variable-p
 @xref{Condition Variables, condition-variable-p}.
 
@@ -2098,6 +2107,9 @@ with references to further information.
 @item integerp
 @xref{Predicates on Numbers, integerp}.
 
+@item interpreted-function-p
+@xref{What Is a Function, interpreted-function-p}.
+
 @item keymapp
 @xref{Creating Keymaps, keymapp}.
 
diff --git a/doc/lispref/sequences.texi b/doc/lispref/sequences.texi
index c9e47624878..4c5525f10c5 100644
--- a/doc/lispref/sequences.texi
+++ b/doc/lispref/sequences.texi
@@ -1583,7 +1583,7 @@ nonempty vector that is not @code{eq} to any existing vector.
 
 The @code{vconcat} function also allows byte-code function objects as
 arguments.  This is a special feature to make it easy to access the entire
-contents of a byte-code function object.  @xref{Byte-Code Objects}.
+contents of a byte-code function object.  @xref{Closure Objects}.
 
 For other concatenation functions, see @code{mapconcat} in @ref{Mapping
 Functions}, @code{concat} in @ref{Creating Strings}, and @code{append}
diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index 4d61d461deb..16b6b52e5f1 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -1079,7 +1079,7 @@ Here is an example:
 (let ((x 0))             ; @r{@code{x} is lexically bound.}
   (setq my-ticker (lambda ()
                     (setq x (1+ x)))))
-    @result{} (closure ((x . 0)) ()
+    @result{} #f(lambda () [(x 0)]
           (setq x (1+ x)))
 
 (funcall my-ticker)