Merge from lexical-binding branch.

* doc/lispref/eval.texi (Eval): Discourage the use of `eval'.
Document its new `lexical' argument.

* doc/lispref/variables.texi (Defining Variables): Mention the new meaning of `defvar'.
(Lexical Binding): New sub-section.

* lisp/Makefile.in (BIG_STACK_DEPTH, BIG_STACK_OPTS, BYTE_COMPILE_FLAGS):
New variables.
(compile-onefile, .el.elc, compile-calc, recompile): Use them.
(COMPILE_FIRST): Add macroexp and cconv.
* lisp/makefile.w32-in: Mirror changes in Makefile.in.

* lisp/vc/cvs-status.el:
* lisp/vc/diff-mode.el:
* lisp/vc/log-edit.el:
* lisp/vc/log-view.el:
* lisp/vc/smerge-mode.el:
* lisp/textmodes/bibtex-style.el:
* textmodes/css.el:
* lisp/startup.el:
* lisp/uniquify.el:
* lisp/minibuffer.el: 
* lisp/newcomment.el: 
* lisp/reveal.el: 
* lisp/server.el: 
* lisp/mpc.el: 
* lisp/emacs-lisp/smie.el: 
* lisp/doc-view.el: 
* lisp/dired.el: 
* lisp/abbrev.el: Use lexical binding.

* lisp/custom.el (custom-initialize-default, custom-declare-variable):
Use `defvar'.

* lisp/files.el (lexical-binding): Declare safe.

* lisp/help-fns.el (help-split-fundoc): Return nil if there's nothing else
than the arglist.
(help-add-fundoc-usage): Don't add `Not documented'.
(help-function-arglist): Handle closures, subroutines, and new
byte-code-functions.
(help-make-usage): Remove leading underscores.
(describe-function-1): Handle closures.
(describe-variable): Use special-variable-p for completion.

* lisp/simple.el (with-wrapper-hook, apply-partially): Move to subr.el.

* lisp/subr.el (apply-partially): Use new closures rather than CL.
(--dolist-tail--, --dotimes-limit--): Don't declare dynamic.
(dolist, dotimes): Use slightly different expansion for lexical code.
(functionp): Move to C.
(letrec): New macro.
(with-wrapper-hook): Use it and apply-partially instead of CL.
(eval-after-load): Preserve lexical-binding.
(save-window-excursion, with-output-to-temp-buffer): Turn them
into macros.

* lisp/emacs-lisp/advice.el (ad-arglist): Use help-function-arglist.

* lisp/emacs-lisp/autoload.el (make-autoload): Don't burp on trivial macros.

* lisp/emacs-lisp/byte-opt.el: Use lexical binding.
(byte-inline-lapcode): Remove (to bytecomp).
(byte-compile-inline-expand): Pay attention to inlining to/from
lexically bound code.
(byte-compile-unfold-lambda): Don't handle byte-code-functions
any more.
(byte-optimize-form-code-walker): Don't handle save-window-excursion
any more and don't call compiler-macros.
(byte-compile-splice-in-already-compiled-code): Remove.
(byte-code): Don't inline any more.
(disassemble-offset): Receive `bytes' as argument rather than via
dynamic scoping.
(byte-compile-tag-number): Declare before first use.
(byte-decompile-bytecode-1): Handle new byte-codes, don't change
`return' even if make-spliceable.
(byte-compile-side-effect-and-error-free-ops): Add stack-ref, remove
obsolete interactive-p.
(byte-optimize-lapcode): Optimize new lap-codes.
Don't trip up on new form of `byte-constant' lap code.

* lisp/emacs-lisp/byte-run.el (make-obsolete): Don't set the `byte-compile'
handler any more.

* lisp/emacs-lisp/bytecomp.el: Use lexical binding instead of
a "bytecomp-" prefix.  Macroexpand everything as a separate phase.
(byte-compile-initial-macro-environment):
Handle declare-function here.
(byte-compile--lexical-environment): New var.
(byte-stack-ref, byte-stack-set, byte-discardN)
(byte-discardN-preserve-tos): New lap codes.
(byte-interactive-p): Don't use any more.
(byte-compile-push-bytecodes, byte-compile-push-bytecode-const2):
New macros.
(byte-compile-lapcode): Use them and handle new lap codes.
(byte-compile-obsolete): Remove.
(byte-compile-arglist-signature): Handle new byte-code arg"lists".
(byte-compile-arglist-warn): Check late def of inlinable funs.
(byte-compile-cl-warn): Don't silence warnings for compiler-macros
since they should have been expanded by now.
(byte-compile--outbuffer): Rename from bytecomp-outbuffer.
(byte-compile-from-buffer): Remove unused second arg.
(byte-compile-preprocess): New function.
(byte-compile-toplevel-file-form): New function to distinguish
file-form calls from outside from file-form calls from hunk-handlers.
(byte-compile-file-form): Simplify.
(byte-compile-file-form-defsubst): Remove.
(byte-compile-file-form-defmumble): Simplify now that
byte-compile-lambda always returns a byte-code-function.
(byte-compile): Preprocess.
(byte-compile-byte-code-maker, byte-compile-byte-code-unmake):
Remove, not used any more.
(byte-compile-arglist-vars, byte-compile-make-lambda-lexenv)
(byte-compile-make-args-desc): New funs.
(byte-compile-lambda): Handle lexical functions.  Always return
a byte-code-function.
(byte-compile-reserved-constants): New var, to make up room for
closed-over variables.
(byte-compile-constants-vector): Obey it.
(byte-compile-top-level): New args `lexenv' and `reserved-csts'.
(byte-compile-macroexpand-declare-function): New function.
(byte-compile-form): Call byte-compile-unfold-bcf to inline immediate
byte-code-functions.
(byte-compile-form): Check obsolescence here.
(byte-compile-inline-lapcode, byte-compile-unfold-bcf): New functions.
(byte-compile-variable-ref): Remove.
(byte-compile-dynamic-variable-op): New fun.
(byte-compile-dynamic-variable-bind, byte-compile-variable-ref)
(byte-compile-variable-set): New funs.
(byte-compile-discard): Add 2 args.
(byte-compile-stack-ref, byte-compile-stack-set)
(byte-compile-make-closure, byte-compile-get-closed-var): New funs.
(byte-compile-funarg, byte-compile-funarg-2): Remove, handled in
macroexpand-all instead.
(byte-compile-quote-form): Remove.
(byte-compile-push-binding-init, byte-compile-not-lexical-var-p)
(byte-compile-bind, byte-compile-unbind): New funs.
(byte-compile-let): Handle let* and lexical binding.
(byte-compile-let*): Remove.
(byte-compile-catch, byte-compile-unwind-protect)
(byte-compile-track-mouse, byte-compile-condition-case):
Handle a new :fun-body form, used for lexical scoping.
(byte-compile-save-window-excursion)
(byte-compile-with-output-to-temp-buffer): Remove.
(byte-compile-defun): Simplify.
(byte-compile-stack-adjustment): New fun.
(byte-compile-out): Use it.
(byte-compile-refresh-preloaded): Don't reload byte-compiler files.

* lisp/emacs-lisp/cconv.el: New file.

* lisp/emacs-lisp/cl-extra.el (cl-macroexpand-all): Properly quote CL
closures.

* lisp/emacs-lisp/cl-macs.el (cl-byte-compile-block)
(cl-byte-compile-throw): Remove.
(cl-block-wrapper, cl-block-throw): Use compiler-macros instead.

* lisp/emacs-lisp/cl.el (pushnew): Silence warning.

* lisp/emacs-lisp/disass.el (disassemble-internal): Handle new
`closure' objects.
(disassemble-1): Handle new byte codes.

* lisp/emacs-lisp/edebug.el (edebug-eval-defun)
(edebug-eval-top-level-form): Use eval-sexp-add-defvars.
(edebug-toggle): Avoid `eval'.

* lisp/emacs-lisp/eieio-comp.el: Remove.

* lisp/emacs-lisp/eieio.el (byte-compile-file-form-defmethod):
Don't autoload.
(eieio-defgeneric-form-primary-only-one): Use `byte-compile' rather
than the internal `byte-compile-lambda'.
(defmethod): Don't hide code under quotes.
(eieio-defmethod): New `code' argument.

* lisp/emacs-lisp/float-sup.el (pi): Don't declare as dynamically bound.

* lisp/emacs-lisp/lisp-mode.el (eval-last-sexp-1):
Use eval-sexp-add-defvars.
(eval-sexp-add-defvars): New fun.

* lisp/emacs-lisp/macroexp.el: Use lexical binding.
(macroexpand-all-1): Check obsolete macros.  Expand compiler-macros.
Don't convert ' to #' without checking that it's indeed quoting
a lambda.

* lisp/emacs-lisp/pcase.el: Don't use destructuring-bind.
(pcase--memoize): Rename from pcase-memoize.  Change weakness.
(pcase): Add `let' pattern.
Change memoization so it actually works.
(pcase-mutually-exclusive-predicates): Add byte-code-function-p.
(pcase--u1) <guard, pred>: Fix possible shadowing problem.
<let>: New case.

* src/alloc.c (Fmake_symbol): Init new `declared_special' field.

* src/buffer.c (defvar_per_buffer): Set new `declared_special' field.

* src/bytecode.c (Bstack_ref, Bstack_set, Bstack_set2, BdiscardN):
New byte-codes.
(exec_byte_code): New function extracted from Fbyte_code to handle new
calling convention for byte-code-functions.  Add new byte-codes.

* src/callint.c (Fcall_interactively): Preserve lexical-binding mode for
interactive spec.

* src/doc.c (Fdocumentation, store_function_docstring):
* src/data.c (Finteractive_form): Handle closures.

* src/eval.c (Fsetq): Handle lexical vars.
(Fdefun, Fdefmacro, Ffunction): Make closures when needed.
(Fdefconst, Fdefvaralias, Fdefvar): Mark as dynamic.
(FletX, Flet): Obey lexical binding.
(Fcommandp): Handle closures.
(Feval): New `lexical' arg.
(eval_sub): New function extracted from Feval.  Use it almost
everywhere where Feval was used.  Look up vars in lexical env.
Handle closures.
(Ffunctionp): Move from subr.el.
(Ffuncall): Handle closures.
(apply_lambda): Remove `eval_flags'.
(funcall_lambda): Handle closures and new byte-code-functions.
(Fspecial_variable_p): New function.
(syms_of_eval): Initialize the Vinternal_interpreter_environment var,
but without exporting it to Lisp.

* src/fns.c (concat, mapcar1): Accept byte-code-functions.

* src/image.c (parse_image_spec): Use Ffunctionp.

* src/keyboard.c (eval_dyn): New fun.
(menu_item_eval_property): Use it.

* src/lisp.h (struct Lisp_Symbol): New field `declared_special'.

* src/lread.c (lisp_file_lexically_bound_p): New function.
(Fload): Bind Qlexical_binding.
(readevalloop): Remove `evalfun' arg.
Bind Qinternal_interpreter_environment.
(Feval_buffer): Bind Qlexical_binding.
(defvar_int, defvar_bool, defvar_lisp_nopro, defvar_kboard):
Mark as dynamic.
(syms_of_lread): Declare `lexical-binding'.

* src/window.c (Ftemp_output_buffer_show): New fun.
(Fsave_window_excursion):
* src/print.c (Fwith_output_to_temp_buffer): Move to subr.el.

2 files changed, 79 insertions, 0 deletions

diff --git a/test/ChangeLog b/test/ChangeLog
index b247b88bc94..dc9b87adfac 100644
--- a/test/ChangeLog
+++ b/test/ChangeLog
@@ -1,3 +1,7 @@
+2011-03-10  Stefan Monnier  <monnier@iro.umontreal.ca>
+
+        * automated/lexbind-tests.el: New file.
+
 2011-03-05  Glenn Morris  <rgm@gnu.org>
 
         * eshell.el: Move here from lisp/eshell/esh-test.el.
diff --git a/test/automated/lexbind-tests.el b/test/automated/lexbind-tests.el
new file mode 100644
index 00000000000..95b8bbe8858
--- /dev/null
+++ b/test/automated/lexbind-tests.el
@@ -0,0 +1,75 @@
+;;; lexbind-tests.el --- Testing the lexbind byte-compiler
+
+;; Copyright (C) 2011  Free Software Foundation, Inc.
+
+;; Author: Stefan Monnier <monnier@iro.umontreal.ca>
+;; Keywords:
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;
+
+;;; Code:
+
+(require 'ert)
+
+(defconst lexbind-tests
+  `(
+    (let ((f #'car))
+      (let ((f (lambda (x) (cons (funcall f x) (cdr x)))))
+        (funcall f '(1 . 2))))
+    )
+  "List of expression for test.
+Each element will be executed by interpreter and with
+bytecompiled code, and their results compared.")
+
+
+
+(defun lexbind-check-1 (pat)
+  "Return non-nil if PAT is the same whether directly evalled or compiled."
+  (let ((warning-minimum-log-level :emergency)
+        (byte-compile-warnings nil)
+        (v0 (condition-case nil
+                (eval pat t)
+              (error nil)))
+        (v1 (condition-case nil
+                (funcall (let ((lexical-binding t))
+                           (byte-compile `(lambda nil ,pat))))
+              (error nil))))
+    (equal v0 v1)))
+
+(put 'lexbind-check-1 'ert-explainer 'lexbind-explain-1)
+
+(defun lexbind-explain-1 (pat)
+  (let ((v0 (condition-case nil
+                (eval pat t)
+              (error nil)))
+        (v1 (condition-case nil
+                (funcall (let ((lexical-binding t))
+                           (byte-compile (list 'lambda nil pat))))
+              (error nil))))
+    (format "Expression `%s' gives `%s' if directly evalled, `%s' if compiled."
+            pat v0 v1)))
+
+(ert-deftest lexbind-tests ()
+  "Test the Emacs byte compiler lexbind handling."
+  (dolist (pat lexbind-tests)
+    (should (lexbind-check-1 pat))))
+
+
+
+(provide 'lexbind-tests)
+;;; lexbind-tests.el ends here