Revision: miles@gnu.org--gnu-2004/emacs--unicode--0--patch-15

Merge from emacs--cvs-trunk--0

Patches applied:

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-218
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-220
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-221
   Restore deleted tagline in etc/TUTORIAL.ru

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-222
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-228
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-229
   Remove TeX output files from the archive

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-230
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-247
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-248
   src/lisp.h (CYCLE_CHECK): Macro moved from xfaces.c

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-249
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-256
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-258
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-263
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-264
   Update from CVS: lispref/display.texi: emacs -> Emacs.

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-265
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-274
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-275
   Update from CVS: man/makefile.w32-in: Revert last change

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-276
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-295
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-296
   Allow restarting an existing debugger session that's exited

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-297
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-299
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-300
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-327
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-328
   Update from CVS: src/.gdbinit (xsymbol): Fix last change.

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-329
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-344
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-345
   Tweak source regexps so that building in place won't cause problems

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-346
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-351
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-352
   Update from CVS: lisp/flymake.el: New file.

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-353
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-361
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-362
   Support " [...]" style defaults in minibuffer-electric-default-mode

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-363
   (read-number): Use canonical format for default in prompt.

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-364
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-367
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-368
   Improve display-supports-face-attributes-p on non-ttys

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-369
   Rewrite face-differs-from-default-p

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-370
   Move `display-supports-face-attributes-p' entirely into C code

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-371
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-372
   Simplify face-differs-from-default-p; don't consider :stipple.

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-373
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-374
   (tty_supports_face_attributes_p): Ensure attributes differ from default

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-375
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-376
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-377
   (Fdisplay_supports_face_attributes_p): Work around bootstrapping problem

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-378
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-380
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-381
   Face merging cleanups

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-382
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-384
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-385
   src/xfaces.c (push_named_merge_point): Return 0 if a cycle is detected

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-386
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-395
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-396
   Tweak arch tagging to make build/install-in-place less annoying

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-397
   Work around vc-arch problems when building eshell

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-398
   Tweak permissions

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-399
   Tweak directory permissions

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-400
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-401
   More build-in-place tweaking of arch tagging

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-402
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-403
   Yet more build-in-place tweaking of arch tagging

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-404
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-409
   Update from CVS

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-410
   Make sure image types are initialized for lookup too

 * miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-411
 - miles@gnu.org--gnu-2004/emacs--cvs-trunk--0--patch-416
   Update from CVS

1 files changed, 359 insertions, 210 deletions

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index bfa507b851a..274480a36de 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3,6 +3,7 @@
 ;; Copyright (C) 2003, 04  Free Software Foundation, Inc.
 
 ;; Author: Dave Love <fx@gnu.org>
+;; Maintainer: FSF
 ;; Created: Nov 2003
 ;; Keywords: languages
 
@@ -45,19 +46,18 @@
 ;; I've installed a minor mode to do the job properly in Emacs 22.
 ;; Other things seem more natural or canonical here, e.g. the
 ;; {beginning,end}-of-defun implementation dealing with nested
-;; definitions, and the inferior mode following `cmuscheme'.  (The
-;; inferior mode should be able to find the source of errors from
-;; `python-send-region' & al via `compilation-minor-mode', but I can't
-;; make that work with the current (March '04) compile.el.)
-;; Successive TABs cycle between possible indentations for the line.
+;; definitions, and the inferior mode following `cmuscheme'.  The
+;; inferior mode can find the source of errors from
+;; `python-send-region' & al via `compilation-minor-mode'.  Successive
+;; TABs cycle between possible indentations for the line.  There is
+;; symbol completion using lookup in Python.
 
 ;; Even where it has similar facilities, this is incompatible with
 ;; python-mode.el in various respects.  For instance, various key
 ;; bindings are changed to obey Emacs conventions, and things like
 ;; marking blocks and `beginning-of-defun' behave differently.
 
-;; TODO: See various Fixmes below.  It should be possible to arrange
-;; some sort of completion using the inferior interpreter.
+;; TODO: See various Fixmes below.
 
 ;;; Code:
 
@@ -66,10 +66,8 @@
 (require 'comint)
 (eval-when-compile
   (require 'compile)
-  (autoload 'Info-last "info")
-  (autoload 'Info-exit "info")
   (autoload 'info-lookup-maybe-add-help "info-look"))
-(autoload 'compilation-start "compile") ; spurious compiler warning anyway
+(autoload 'compilation-start "compile")
 
 (defgroup python nil
   "Silly walks in the Python language"
@@ -204,6 +202,8 @@ Used for syntactic keywords.  N is the match number (1, 2 or 3)."
     (define-key map "\C-c\C-z" 'python-switch-to-python)
     (define-key map "\C-c\C-m" 'python-load-file)
     (define-key map "\C-c\C-l" 'python-load-file) ; a la cmuscheme
+    (substitute-key-definition 'complete-symbol 'python-complete-symbol
+                               map global-map)
     ;; Fixme: Add :help to menu.
     (easy-menu-define python-menu map "Python Mode menu"
       '("Python"
@@ -262,9 +262,7 @@ Used for syntactic keywords.  N is the match number (1, 2 or 3)."
 ;;;; Utility stuff
 
 (defsubst python-in-string/comment ()
-  "Return non-nil if point is in a Python literal (a comment or string).
-Optional argument LIM indicates the beginning of the containing form,
-i.e. the limit on how far back to scan."
+  "Return non-nil if point is in a Python literal (a comment or string)."
   (syntax-ppss-context (syntax-ppss)))
 
 (defconst python-space-backslash-table
@@ -299,16 +297,18 @@ comments and strings, or that the bracket/paren nesting depth is nonzero."
                (syntax-ppss (line-beginning-position)))))))
 
 (defun python-comment-line-p ()
-  "Return non-nil if current line has only a comment or is blank."
+  "Return non-nil iff current line has only a comment."
   (save-excursion
-    (back-to-indentation)
-    (looking-at (rx (or (syntax comment-start) line-end)))))
+    (end-of-line)
+    (when (eq 'comment (syntax-ppss-context (syntax-ppss)))
+      (back-to-indentation)
+      (looking-at (rx (or (syntax comment-start) line-end))))))
 
 (defun python-beginning-of-string ()
   "Go to beginning of string around point.
 Do nothing if not in string."
   (let ((state (syntax-ppss)))
-    (when (nth 3 state)
+    (when (eq 'string (syntax-ppss-context state))
       (goto-char (nth 8 state)))))
 
 (defun python-open-block-statement-p (&optional bos)
@@ -323,7 +323,8 @@ BOS non-nil means point is known to be at beginning of statement."
                                      line-end))
                             (save-excursion (python-end-of-statement))
                             t)
-         (not (python-in-string/comment)))))
+         (not (progn (goto-char (match-beginning 0))
+                     (python-in-string/comment))))))
 
 (defun python-close-block-statement-p (&optional bos)
   "Return non-nil if current line is a statement closing a block.
@@ -384,7 +385,8 @@ Otherwise indent them to column zero."
 (defcustom python-honour-comment-indentation nil
   "Non-nil means indent relative to preceding comment line.
 Only do this for comments where the leading comment character is followed
-by space."
+by space.  This doesn't apply to comment lines, which are always indented
+in lines with preceding comments."
   :type 'boolean
   :group 'python)
 
@@ -514,6 +516,16 @@ Set `python-indent' locally to the value guessed."
                                      (- python-indent)))
                           0)))))))))
 
+(defun python-comment-indent ()
+  "`comment-indent-function' for Python."
+  ;; If previous non-blank line was a comment, use its indentation.
+  ;; FIXME: This seems unnecessary since the default code delegates to
+  ;; indent-according-to-mode.  --Stef
+  (unless (bobp)
+    (save-excursion
+      (forward-comment -1)
+      (if (eq ?# (char-after)) (current-column)))))
+
 ;;;; Cycling through the possible indentations with successive TABs.
 
 ;; These don't need to be buffer-local since they're only relevant
@@ -538,11 +550,17 @@ Set `python-indent' locally to the value guessed."
                       (point))))
 
 (defun python-indentation-levels ()
-  "Return a list of possible indentations for this statement.
+  "Return a list of possible indentations for this line.
 Includes the default indentation and those which would close all
-enclosing blocks."
+enclosing blocks.  Assumes the line has already been indented per
+`python-indent-line'.  Elements of the list are actually pairs:
+\(INDENTATION . TEXT), where TEXT is the initial text of the
+corresponding block opening (or nil)."
   (save-excursion
-    (let ((levels (list (cons (current-indentation) nil))))
+    (let ((levels (list (cons (current-indentation)
+                              (save-excursion
+                                (if (python-beginning-of-block)
+                                    (python-initial-text)))))))
       ;; Only one possibility if we immediately follow a block open or
       ;; are in a continuation line.
       (unless (or (python-continuation-line-p)
@@ -568,8 +586,7 @@ enclosing blocks."
       (if (> (- (point-max) pos) (point))
           (goto-char (- (point-max) pos))))))
 
-;; Fixme: Is the arg necessary?
-(defun python-indent-line (&optional arg)
+(defun python-indent-line ()
   "Indent current line as Python code.
 When invoked via `indent-for-tab-command', cycle through possible
 indentations for current line.  The cycle is broken by a command different
@@ -586,13 +603,30 @@ from `indent-for-tab-command', i.e. successive TABs do the cycling."
                  (beginning-of-line)
                  (delete-horizontal-space)
                  (indent-to (car (nth python-indent-index python-indent-list)))
-                 (let ((text (cdr (nth python-indent-index
-                                       python-indent-list))))
-                   (if text (message "Closes: %s" text)))))
+                 (if (python-block-end-p)
+                     (let ((text (cdr (nth python-indent-index
+                                           python-indent-list))))
+                       (if text
+                           (message "Closes: %s" text))))))
       (python-indent-line-1)
       (setq python-indent-list (python-indentation-levels)
             python-indent-list-length (length python-indent-list)
             python-indent-index (1- python-indent-list-length)))))
+
+(defun python-block-end-p ()
+  "Non-nil if this is a line in a statement closing a block,
+or a blank line indented to where it would close a block."
+  (and (not (python-comment-line-p))
+       (or (python-close-block-statement-p t)
+           (< (current-indentation)
+              (save-excursion
+                (python-previous-statement)
+                (current-indentation))))))
+
+;; Fixme: Define an indent-region-function.  It should probably leave
+;; lines alone if the indentation is already at one of the allowed
+;; levels.  Otherwise, M-C-\ typically keeps indenting more deeply
+;; down a function.
 
 ;;;; Movement.
 
@@ -629,8 +663,7 @@ start of buffer."
   "`end-of-defun-function' for Python.
 Finds end of innermost nested class or method definition."
   (let ((orig (point))
-        (pattern (rx (and line-start (0+ space)
-                          (or "def" "class") space))))
+        (pattern (rx (and line-start (0+ space) (or "def" "class") space))))
     ;; Go to start of current block and check whether it's at top
     ;; level.  If it is, and not a block start, look forward for
     ;; definition statement.
@@ -829,7 +862,8 @@ move and return nil.  Otherwise return t."
 Makes nested Imenu menus from nested `class' and `def' statements.
 The nested menus are headed by an item referencing the outer
 definition; it has a space prepended to the name so that it sorts
-first with `imenu--sort-by-name'."
+first with `imenu--sort-by-name' (though, unfortunately, sub-menus
+precede it)."
   (unless (boundp 'python-recursing)            ; dynamically bound below
     (goto-char (point-min)))            ; normal call from Imenu
   (let (index-alist                     ; accumulated value to return
@@ -914,13 +948,20 @@ See `python-check-command' for the default."
                                         (file-name-nondirectory name))))))))
   (setq python-saved-check-command command)
   (save-some-buffers (not compilation-ask-about-save) nil)
-  (compilation-start command))
+  (let ((compilation-error-regexp-alist
+         (cons '("(\\([^,]+\\), line \\([0-9]+\\))" 1 2)
+               compilation-error-regexp-alist)))
+    (compilation-start command)))
 
 ;;;; Inferior mode stuff (following cmuscheme).
 
+;; Fixme: Make sure we can work with IPython.
+
 (defcustom python-python-command "python"
   "*Shell command to run Python interpreter.
-Any arguments can't contain whitespace."
+Any arguments can't contain whitespace.
+Note that IPython may not work properly; it must at least be used with the
+`-cl' flag, i.e. use `ipython -cl'."
   :group 'python
   :type 'string)
 
@@ -937,40 +978,66 @@ Additional arguments are added when the command is used by `run-python'
 et al.")
 
 (defvar python-buffer nil
-  "*The current python process buffer.
-To run multiple Python processes, start the first with \\[run-python].
-It will be in a buffer named *Python*.  Rename that with
-\\[rename-buffer].  Now start a new process with \\[run-python].  It
-will be in a new buffer, named *Python*.  Switch between the different
-process buffers with \\[switch-to-buffer].
-
-Commands that send text from source buffers to Python processes have
-to choose a process to send to.  This is determined by global variable
-`python-buffer'.  Suppose you have three inferior Pythons running:
-    Buffer      Process
-    foo         python
-    bar         python<2>
-    *Python*    python<3>
-If you do a \\[python-send-region-and-go] command on some Python source
-code, what process does it go to?
-
-- In a process buffer (foo, bar, or *Python*), send it to that process.
-- In some other buffer (e.g. a source file), send it to the process
-  attached to `python-buffer'.
-Process selection is done by function `python-proc'.
-
-Whenever \\[run-python] starts a new process, it resets `python-buffer'
-to be the new process's buffer.  If you only run one process, this will
-do the right thing.  If you run multiple processes, you can change
-`python-buffer' to another process buffer with \\[set-variable].")
+  "The current python process buffer."
+  ;; Fixme: a single process is currently assumed, so that this doc
+  ;; is misleading.
+
+;;   "*The current python process buffer.
+;; To run multiple Python processes, start the first with \\[run-python].
+;; It will be in a buffer named *Python*.  Rename that with
+;; \\[rename-buffer].  Now start a new process with \\[run-python].  It
+;; will be in a new buffer, named *Python*.  Switch between the different
+;; process buffers with \\[switch-to-buffer].
+
+;; Commands that send text from source buffers to Python processes have
+;; to choose a process to send to.  This is determined by global variable
+;; `python-buffer'.  Suppose you have three inferior Pythons running:
+;;     Buffer   Process
+;;     foo              python
+;;     bar              python<2>
+;;     *Python*    python<3>
+;; If you do a \\[python-send-region-and-go] command on some Python source
+;; code, what process does it go to?
+
+;; - In a process buffer (foo, bar, or *Python*), send it to that process.
+;; - In some other buffer (e.g. a source file), send it to the process
+;;   attached to `python-buffer'.
+;; Process selection is done by function `python-proc'.
+
+;; Whenever \\[run-python] starts a new process, it resets `python-buffer'
+;; to be the new process's buffer.  If you only run one process, this will
+;; do the right thing.  If you run multiple processes, you can change
+;; `python-buffer' to another process buffer with \\[set-variable]."
+  )
 
 (defconst python-compilation-regexp-alist
+  ;; FIXME: maybe these should move to compilation-error-regexp-alist-alist.
   `((,(rx (and line-start (1+ (any " \t")) "File \""
                (group (1+ (not (any "\"<")))) ; avoid `<stdin>' &c
                "\", line " (group (1+ digit))))
-     1 python-compilation-line-number))
+     1 2)
+    (,(rx (and " in file " (group (1+ not-newline)) " on line "
+               (group (1+ digit))))
+     1 2))
   "`compilation-error-regexp-alist' for inferior Python.")
 
+(defvar inferior-python-mode-map
+  (let ((map (make-sparse-keymap)))
+    ;; This will inherit from comint-mode-map.
+    (define-key map "\C-c\C-l" 'python-load-file)
+    (define-key map "\C-c\C-v" 'python-check)
+    ;; Note that we _can_ still use these commands which send to the
+    ;; Python process even at the prompt iff we have a normal prompt,
+    ;; i.e. '>>> ' and not '... '.  See the comment before
+    ;; python-send-region.  Fixme: uncomment these if we address that.
+
+    ;; (define-key map [(meta ?\t)] 'python-complete-symbol)
+    ;; (define-key map "\C-c\C-f" 'python-describe-symbol)
+    map))
+
+;; Fixme: This should inherit some stuff from python-mode, but I'm not
+;; sure how much: at least some keybindings, like C-c C-f; syntax?;
+;; font-locking, e.g. for triple-quoted strings?
 (define-derived-mode inferior-python-mode comint-mode "Inferior Python"
   "Major mode for interacting with an inferior Python process.
 A Python process can be started with \\[run-python].
@@ -991,14 +1058,13 @@ For running multiple processes in multiple buffers, see `python-buffer'.
   :group 'python
   (set-syntax-table python-mode-syntax-table)
   (setq mode-line-process '(":%s"))
-  ;; Fixme: Maybe install some python-mode bindings too.
-  (define-key inferior-python-mode-map "\C-c\C-l" 'python-load-file)
-  (define-key inferior-python-mode-map "\C-c\C-z" 'python-switch-to-python)
-  (add-hook 'comint-input-filter-functions 'python-input-filter nil t)
+  (set (make-local-variable 'comint-input-filter) 'python-input-filter)
   (add-hook 'comint-preoutput-filter-functions #'python-preoutput-filter
             nil t)
-  ;; Still required by `comint-redirect-send-command', for instance:
-  (set (make-local-variable 'comint-prompt-regexp) "^\\([>.]\\{3\\} \\)+")
+  ;; Still required by `comint-redirect-send-command', for instance
+  ;; (and we need to match things like `>>> ... >>> '):
+  (set (make-local-variable 'comint-prompt-regexp)
+       (rx (and line-start (1+ (and (repeat 3 (any ">.")) ?\ )))))
   (set (make-local-variable 'compilation-error-regexp-alist)
        python-compilation-regexp-alist)
   (compilation-shell-minor-mode 1))
@@ -1009,15 +1075,9 @@ Default ignores all inputs of 0, 1, or 2 non-blank characters."
   :type 'regexp
   :group 'python)
 
-(defvar python-orig-start nil
-  "Marker to the start of the region passed to the inferior Python.
-It can also be a filename.")
-
 (defun python-input-filter (str)
   "`comint-input-filter' function for inferior Python.
-Don't save anything for STR matching `inferior-python-filter-regexp'.
-Also resets variables for adjusting error messages."
-  (setq python-orig-start nil)
+Don't save anything for STR matching `inferior-python-filter-regexp'."
   (not (string-match inferior-python-filter-regexp str)))
 
 ;; Fixme: Loses with quoted whitespace.
@@ -1030,21 +1090,8 @@ Also resets variables for adjusting error messages."
           (t (let ((pos (string-match "[^ \t]" string)))
                (if pos (python-args-to-list (substring string pos))))))))
 
-(defun python-compilation-line-number (file col)
-  "Return error descriptor of error found for FILE, column COL.
-Used as line-number hook function in `python-compilation-regexp-alist'."
-  (let ((line (string-to-number (match-string 2))))
-    (cons (point-marker)
-          (if (and (markerp python-orig-start)
-                   (marker-buffer python-orig-start))
-              (with-current-buffer (marker-buffer python-orig-start)
-                (goto-char python-orig-start)
-                (forward-line (1- line)))
-            (list (if (stringp python-orig-start) python-orig-start file)
-                  line nil)))))
-
 (defvar python-preoutput-result nil
-  "Data from output line last `_emacs_out' line seen by the preoutput filter.")
+  "Data from last `_emacs_out' line seen by the preoutput filter.")
 
 (defvar python-preoutput-continuation nil
   "If non-nil, funcall this when `python-preoutput-filter' sees `_emacs_ok'.")
@@ -1055,7 +1102,9 @@ Used as line-number hook function in `python-compilation-regexp-alist'."
 ;; `python-preoutput-continuation' if we get it.
 (defun python-preoutput-filter (s)
   "`comint-preoutput-filter-functions' function: ignore prompts not at bol."
-  (cond ((and (string-match "\\`[.>]\\{3\\} \\'" s)
+  (cond ((and (string-match (rx (and string-start (repeat 3 (any ".>"))
+                                     " " string-end))
+                            s)
               (/= (let ((inhibit-field-text-motion t))
                     (line-beginning-position))
                   (point)))
@@ -1076,10 +1125,10 @@ Used as line-number hook function in `python-compilation-regexp-alist'."
 CMD is the Python command to run.  NOSHOW non-nil means don't show the
 buffer automatically.
 If there is a process already running in `*Python*', switch to
-that buffer.  Interactively a prefix arg, allows you to edit the initial
-command line (default is the value of `python-command'); `-i' etc. args
-will be added to this as appropriate.  Runs the hooks
-`inferior-python-mode-hook' (after the `comint-mode-hook' is run).
+that buffer.  Interactively, a prefix arg allows you to edit the initial
+command line (default is `python-command'); `-i' etc. args will be added
+to this as appropriate.  Runs the hook `inferior-python-mode-hook'
+\(after the `comint-mode-hook' is run).
 \(Type \\[describe-mode] in the process buffer for a list of commands.)"
   (interactive (list (if current-prefix-arg
                          (read-string "Run Python: " python-command)
@@ -1089,82 +1138,78 @@ will be added to this as appropriate.  Runs the hooks
   ;; Fixme: Consider making `python-buffer' buffer-local as a buffer
   ;; (not a name) in Python buffers from which `run-python' &c is
   ;; invoked.  Would support multiple processes better.
-  (unless (comint-check-proc "*Python*")
-    (let ((cmdlist (append (python-args-to-list cmd) '("-i"))))
+  (unless (comint-check-proc python-buffer)
+    (let* ((cmdlist (append (python-args-to-list cmd) '("-i")))
+           (path (getenv "PYTHONPATH"))
+           (process-environment         ; to import emacs.py
+            (push (concat "PYTHONPATH=" data-directory
+                          (if path (concat ":" path)))
+                  process-environment)))
       (set-buffer (apply 'make-comint "Python" (car cmdlist) nil
-                         (cdr cmdlist))))
+                         (cdr cmdlist)))
+      (setq python-buffer "*Python*"))
     (inferior-python-mode)
     ;; Load function defintions we need.
     ;; Before the preoutput function was used, this was done via -c in
     ;; cmdlist, but that loses the banner and doesn't run the startup
-    ;; file.
-    (python-send-string "\
-def _emacs_execfile (file):  # execute file and remove it
-    from os import remove
-    try: execfile (file, globals (), globals ())
-    finally: remove (file)
-
-def _emacs_args (name):  # get arglist of name for eldoc &c
-    import inspect
-    parts = name.split ('.')
-    if len (parts) > 1:
-        try: exec 'import ' + parts[0]
-        except: return None
-    try: exec 'func='+name # lose if name is keyword or undefined
-    except: return None
-    if inspect.isbuiltin (func):
-        doc = func.__doc__
-        if doc.find (' ->') != -1:
-            print '_emacs_out', doc.split (' ->')[0]
-        elif doc.find ('\\n') != -1:
-            print '_emacs_out', doc.split ('\\n')[0]
-        return None
-    if inspect.ismethod (func): func = func.im_func
-    if not inspect.isfunction (func):
-        return None
-    (args, varargs, varkw, defaults) = inspect.getargspec (func)
-    print '_emacs_out', func.__name__+inspect.formatargspec (args, varargs, varkw, defaults)
-
-print '_emacs_ok'"))
-  (unless noshow (pop-to-buffer (setq python-buffer "*Python*"))))
+    ;; file.  The code might be inline here, but there's enough that it
+    ;; seems worth putting in a separate file, and it's probably cleaner
+    ;; to put it in a module.
+    (python-send-string "import emacs"))
+  (unless noshow (pop-to-buffer python-buffer)))
+
+;; Fixme: We typically lose if the inferior isn't in the normal REPL,
+;; e.g. prompt is `help> '.  Probably raise an error if the form of
+;; the prompt is unexpected; actually, it needs to be `>>> ', not
+;; `... ', i.e. we're not inputting a block &c.  However, this may not
+;; be the place to do it, e.g. we might actually want to send commands
+;; having set up such a state.
+
+(defun python-send-command (command)
+  "Like `python-send-string' but resets `compilation-minor-mode'."
+  (goto-char (point-max))
+  (let ((end (marker-position (process-mark (python-proc)))))
+    (compilation-forget-errors)
+    (python-send-string command)
+    (set-marker compilation-parsing-end end)
+    (setq compilation-last-buffer (current-buffer))))
 
 (defun python-send-region (start end)
   "Send the region to the inferior Python process."
   ;; The region is evaluated from a temporary file.  This avoids
   ;; problems with blank lines, which have different semantics
   ;; interactively and in files.  It also saves the inferior process
-  ;; buffer filling up with interpreter prompts.  We need a function
-  ;; to remove the temporary file when it has been evaluated, which
-  ;; unfortunately means using a not-quite pristine interpreter
-  ;; initially.  Unfortunately we also get tracebacks which look like:
-  ;; 
-  ;; >>> Traceback (most recent call last):
-  ;;   File "<stdin>", line 1, in ?
-  ;;   File "<string>", line 4, in _emacs_execfile
-  ;;   File "/tmp/py7734RSB", line 11
+  ;; buffer filling up with interpreter prompts.  We need a Python
+  ;; function to remove the temporary file when it has been evaluated
+  ;; (though we could probably do it in Lisp with a Comint output
+  ;; filter).  This function also catches exceptions and truncates
+  ;; tracebacks not to mention the frame of the function itself.
   ;;
   ;; The compilation-minor-mode parsing takes care of relating the
-  ;; reference to the temporary file to the source.  Fixme:
-  ;; comint-filter the first two lines of the traceback?
+  ;; reference to the temporary file to the source.
+  ;;
+  ;; Fixme: Write a `coding' header to the temp file if the region is
+  ;; non-ASCII.
   (interactive "r")
   (let* ((f (make-temp-file "py"))
-         (command (format "_emacs_execfile(%S)" f))
+         (command (format "emacs.eexecfile(%S)" f))
          (orig-start (copy-marker start)))
-    (if (save-excursion
-          (goto-char start)
-          (/= 0 (current-indentation))) ; need dummy block
-        (write-region "if True:\n" nil f nil 'nomsg))
+    (when (save-excursion
+            (goto-char start)
+            (/= 0 (current-indentation))) ; need dummy block
+      (save-excursion
+        (goto-char orig-start)
+        ;; Wrong if we had indented code at buffer start.
+        (set-marker orig-start (line-beginning-position 0)))
+      (write-region "if True:\n" nil f nil 'nomsg))
     (write-region start end f t 'nomsg)
-    (when python-buffer
+    (let ((proc (python-proc)))         ;Make sure we're running a process.
       (with-current-buffer python-buffer
-        (let ((end (marker-position (process-mark (python-proc)))))
-          (set (make-local-variable 'python-orig-start) orig-start)
-          (set (make-local-variable 'compilation-error-list) nil)
-          (let ((comint-input-filter-functions
-                 (delete 'python-input-filter comint-input-filter-functions)))
-            (python-send-string command))
-          (set-marker compilation-parsing-end end)
-          (setq compilation-last-buffer (current-buffer)))))))
+        (python-send-command command)
+        ;; Tell compile.el to redirect error locations in file `f' to
+        ;; positions past marker `orig-start'.  It has to be done *after*
+        ;; python-send-command's call to compilation-forget-errors.
+        (compilation-fake-loc orig-start f)))))
 
 (defun python-send-string (string)
   "Evaluate STRING in inferior Python process."
@@ -1177,6 +1222,8 @@ print '_emacs_ok'"))
   (interactive)
   (python-send-region (point-min) (point-max)))
 
+;; Fixme: Try to define the function or class within the relevant
+;; module, not just at top level.
 (defun python-send-defun ()
   "Send the current defun (class or method) to the inferior Python process."
   (interactive)
@@ -1223,39 +1270,33 @@ function location information for debugging, and supports users of
 module-qualified names."
   (interactive (comint-get-source "Load Python file: " python-prev-dir/file
                                   python-source-modes
-                                  t)) ; because execfile needs exact name
-  (comint-check-source file-name) ; Check to see if buffer needs saved.
+                                  t))   ; because execfile needs exact name
+  (comint-check-source file-name)     ; Check to see if buffer needs saving.
   (setq python-prev-dir/file (cons (file-name-directory file-name)
                                    (file-name-nondirectory file-name)))
-  (when python-buffer
+  (let ((proc (python-proc)))           ;Make sure we have a process.
     (with-current-buffer python-buffer
-      (let ((end (marker-position (process-mark (python-proc)))))
-        (set (make-local-variable 'compilation-error-list) nil)
-        ;; (set (make-local-variable 'compilation-old-error-list) nil)
-        (let ((comint-input-filter-functions
-               (delete 'python-input-filter comint-input-filter-functions)))
-          (python-send-string
-           (if (string-match "\\.py\\'" file-name)
-               ;; Fixme: make sure the directory is in the path list
-               (let ((module (file-name-sans-extension
-                              (file-name-nondirectory file-name))))
-                 (set (make-local-variable 'python-orig-start) nil)
-                 (format "\
-if globals().has_key(%S): reload(%s)
-else: import %s
-" module module module))
-             (set (make-local-variable 'python-orig-start) file-name)
-             (format "execfile('%s')" file-name))))
-        (set-marker compilation-parsing-end end)
-        (setq compilation-last-buffer (current-buffer))))))
-
-;; Fixme: Should this start a process if there isn't one?  (Unlike cmuscheme.)
+      ;; Fixme: I'm not convinced by this logic from python-mode.el.
+      (python-send-command
+       (if (string-match "\\.py\\'" file-name)
+           (let ((module (file-name-sans-extension
+                          (file-name-nondirectory file-name))))
+             (format "emacs.eimport(%S,%S)"
+                     module (file-name-directory file-name)))
+         (format "execfile(%S)" file-name)))
+      (message "%s loaded" file-name))))
+
+;; Fixme: If we need to start the process, wait until we've got the OK
+;; from the startup.
 (defun python-proc ()
-  "Return the current Python process.  See variable `python-buffer'."
-  (let ((proc (get-buffer-process (if (eq major-mode 'inferior-python-mode)
-                                      (current-buffer)
-                                    python-buffer))))
-    (or proc (error "No current process.  See variable `python-buffer'"))))
+  "Return the current Python process.
+See variable `python-buffer'.  Starts a new process if necessary."
+  (or (if python-buffer
+          (get-buffer-process (if (eq major-mode 'inferior-python-mode)
+                                  (current-buffer)
+                                python-buffer)))
+      (progn (run-python nil t)
+             (python-proc))))
 
 ;;;; Context-sensitive help.
 
@@ -1267,33 +1308,47 @@ else: import %s
   "Syntax table giving `.' symbol syntax.
 Otherwise inherits from `python-mode-syntax-table'.")
 
+(defvar view-return-to-alist)
+(eval-when-compile (autoload 'help-buffer "help-fns"))
+
 ;; Fixme: Should this actually be used instead of info-look, i.e. be
-;; bound to C-h S?
+;; bound to C-h S?  Can we use other pydoc stuff before python 2.2?
 (defun python-describe-symbol (symbol)
-  "Get help on SYMBOL using `pydoc'.
-Interactively, prompt for symbol."
-  ;; Note that we do this in the inferior process, not a separate one to
+  "Get help on SYMBOL using `help'.
+Interactively, prompt for symbol.
+
+Symbol may be anything recognized by the interpreter's `help' command --
+e.g. `CALLS' -- not just variables in scope.
+This only works for Python version 2.2 or newer since earlier interpreters
+don't support `help'."
+  ;; Note that we do this in the inferior process, not a separate one, to
   ;; ensure the environment is appropriate.
   (interactive
    (let ((symbol (with-syntax-table python-dotty-syntax-table
                    (current-word)))
-         (enable-recursive-minibuffers t)
-         val)
-     (setq val (read-string (if symbol
-                                (format "Describe symbol (default %s): "
-                                      symbol)
-                              "Describe symbol: ")
-                            nil nil symbol))
-     (list (or val symbol))))
+         (enable-recursive-minibuffers t))
+     (list (read-string (if symbol
+                            (format "Describe symbol (default %s): " symbol)
+                          "Describe symbol: ")
+                        nil nil symbol))))
   (if (equal symbol "") (error "No symbol"))
   (let* ((func `(lambda ()
-                  (comint-redirect-send-command (format "help(%S)\n" ,symbol)
+                  (comint-redirect-send-command (format "emacs.ehelp(%S)\n"
+                                                        ,symbol)
                                                 "*Help*" nil))))
     ;; Ensure we have a suitable help buffer.
-    (let (temp-buffer-show-hook)        ; avoid xref stuff
-      (with-output-to-temp-buffer "*Help*"
+    ;; Fixme: Maybe process `Related help topics' a la help xrefs and
+    ;; allow C-c C-f in help buffer.
+    (let ((temp-buffer-show-hook        ; avoid xref stuff
+           (lambda ()
+             (toggle-read-only 1)
+             (setq view-return-to-alist
+                   (list (cons (selected-window) help-return-method))))))
+      (help-setup-xref (list 'python-describe-symbol symbol) (interactive-p))
+      (with-output-to-temp-buffer (help-buffer)
         (with-current-buffer standard-output
-          (set (make-local-variable 'comint-redirect-subvert-readonly) t))))
+          (set (make-local-variable 'comint-redirect-subvert-readonly) t)
+          (print-help-return-message))))
     (if (and python-buffer (get-buffer python-buffer))
         (with-current-buffer python-buffer
           (funcall func))
@@ -1302,6 +1357,15 @@ Interactively, prompt for symbol."
 
 (add-to-list 'debug-ignored-errors "^No symbol")
 
+(defun python-send-receive (string)
+  "Send STRING to inferior Python (if any) and return result.
+The result is what follows `_emacs_out' in the output (or nil)."
+  (let ((proc (python-proc)))
+    (python-send-string string)
+    (setq python-preoutput-result nil)
+    (accept-process-output proc 5)
+    python-preoutput-result))
+
 ;; Fixme: try to make it work with point in the arglist.  Also, is
 ;; there anything reasonable we can do with random methods?
 ;; (Currently only works with functions.)
@@ -1310,14 +1374,9 @@ Interactively, prompt for symbol."
 Only works when point is in a function name, not its arglist, for instance.
 Assumes an inferior Python is running."
   (let ((symbol (with-syntax-table python-dotty-syntax-table
-                  (current-word)))
-        (proc (and python-buffer (python-proc))))
-    (when (and proc symbol)
-      (python-send-string
-       (format "_emacs_args(%S)" symbol))
-      (setq python-preoutput-result nil)
-      (accept-process-output proc 1)
-      python-preoutput-result)))
+                  (current-word))))
+    (when symbol
+      (python-send-receive (format "emacs.eargs(%S)" symbol)))))
 
 ;;;; Info-look functionality.
 
@@ -1331,11 +1390,13 @@ Used with `eval-after-load'."
          ;; Whether info files have a Python version suffix, e.g. in Debian.
          (versioned
           (with-temp-buffer
-            (Info-mode)
+            (with-no-warnings (Info-mode))
             (condition-case ()
                 ;; Don't use `info' because it would pop-up a *info* buffer.
-                (Info-goto-node (format "(python%s-lib)Miscellaneous Index"
-                                        version))
+                (with-no-warnings
+                 (Info-goto-node (format "(python%s-lib)Miscellaneous Index"
+                                         version))
+                 t)
               (error nil)))))
     (info-lookup-maybe-add-help
      :mode 'python-mode
@@ -1401,7 +1462,7 @@ The criterion is either a match for `jython-mode' via
                 (while (re-search-forward
                         (rx (and line-start (or "import" "from") (1+ space)
                                  (group (1+ (not (any " \t\n."))))))
-                        10000        ; Probably not worth customizing.
+                        (+ (point-min) 10000) ; Probably not worth customizing.
                         t)
                   (if (member (match-string 1) python-jython-packages)
                       (throw 'done t))))
@@ -1519,11 +1580,97 @@ Uses `python-beginning-of-block', `python-end-of-block'."
   (python-end-of-block)
   (exchange-point-and-mark))
 
+;;;; Completion.
+
+(defun python-symbol-completions (symbol)
+  "Return a list of completions of the string SYMBOL from Python process.
+The list is sorted."
+  (when symbol
+    (let ((completions
+           (condition-case ()
+               (car (read-from-string (python-send-receive
+                                       (format "emacs.complete(%S)" symbol))))
+             (error nil))))
+      (sort
+       ;; We can get duplicates from the above -- don't know why.
+       (delete-dups completions)
+       #'string<))))
+
+(defun python-partial-symbol ()
+  "Return the partial symbol before point (for completion)."
+  (let ((end (point))
+        (start (save-excursion
+                 (and (re-search-backward
+                       (rx (and (or buffer-start (regexp "[^[:alnum:]._]"))
+                                (group (1+ (regexp "[[:alnum:]._]")))
+                                point))
+                       nil t)
+                      (match-beginning 1)))))
+    (if start (buffer-substring-no-properties start end))))
+
+;; Fixme: We should have an abstraction of this sort of thing in the
+;; core.
+(defun python-complete-symbol ()
+  "Perform completion on the Python symbol preceding point.
+Repeating the command scrolls the completion window."
+  (interactive)
+  (let ((window (get-buffer-window "*Completions*")))
+    (if (and (eq last-command this-command)
+             window (window-live-p window) (window-buffer window)
+             (buffer-name (window-buffer window)))
+        (with-current-buffer (window-buffer window)
+          (if (pos-visible-in-window-p (point-max) window)
+              (set-window-start window (point-min))
+            (save-selected-window
+              (select-window window)
+              (scroll-up))))
+      ;; Do completion.
+      (let* ((end (point))
+             (symbol (python-partial-symbol))
+             (completions (python-symbol-completions symbol))
+             (completion (if completions
+                             (try-completion symbol completions))))
+        (when symbol
+          (cond ((eq completion t))
+                ((null completion)
+                 (message "Can't find completion for \"%s\"" symbol)
+                 (ding))
+                ((not (string= symbol completion))
+                 (delete-region (- end (length symbol)) end)
+                 (insert completion))
+                (t
+                 (message "Making completion list...")
+                 (with-output-to-temp-buffer "*Completions*"
+                   (display-completion-list completions))
+                 (message "Making completion list...%s" "done"))))))))
+
+(eval-when-compile (require 'hippie-exp))
+
+(defun python-try-complete (old)
+  "Completion function for Python for use with `hippie-expand'."
+  (when (eq major-mode 'python-mode)    ; though we only add it locally
+    (unless old
+      (let ((symbol (python-partial-symbol)))
+        (he-init-string (- (point) (length symbol)) (point))
+        (if (not (he-string-member he-search-string he-tried-table))
+            (push he-search-string he-tried-table))
+        (setq he-expand-list
+              (and symbol (python-symbol-completions symbol)))))
+    (while (and he-expand-list
+                (he-string-member (car he-expand-list) he-tried-table))
+      (pop he-expand-list))
+    (if he-expand-list
+        (progn
+          (he-substitute-string (pop he-expand-list))
+          t)
+      (if old (he-reset-string))
+      nil)))
+
 ;;;; Modes.
 
 (defvar outline-heading-end-regexp)
 (defvar eldoc-print-current-symbol-info-function)
-(defvar python-mode-running)
+
 ;;;###autoload
 (define-derived-mode python-mode fundamental-mode "Python"
   "Major mode for editing Python files.
@@ -1565,11 +1712,10 @@ lines count as headers.
                                    ))
   (set (make-local-variable 'parse-sexp-lookup-properties) t)
   (set (make-local-variable 'comment-start) "# ")
-  ;; Fixme: define a comment-indent-function?
+  (set (make-local-variable 'comment-indent-function) #'python-comment-indent)
   (set (make-local-variable 'indent-line-function) #'python-indent-line)
   (set (make-local-variable 'paragraph-start) "\\s-*$")
-  (set (make-local-variable 'fill-paragraph-function)
-       'python-fill-paragraph)
+  (set (make-local-variable 'fill-paragraph-function) 'python-fill-paragraph)
   (set (make-local-variable 'require-final-newline) t)
   (set (make-local-variable 'add-log-current-defun-function)
        #'python-current-defun)
@@ -1587,6 +1733,9 @@ lines count as headers.
        #'python-eldoc-function)
   (add-hook 'eldoc-mode-hook
             '(lambda () (run-python 0 t)) nil t) ; need it running
+  (if (featurep 'hippie-exp)
+      (set (make-local-variable 'hippie-expand-try-functions-list)
+           (cons 'python-try-complete hippie-expand-try-functions-list)))
   (unless font-lock-mode (font-lock-mode 1))
   (when python-guess-indent (python-guess-indent))
   (set (make-local-variable 'python-command) python-python-command)