New syntax-propertize functionality.

* lisp/font-lock.el (font-lock-syntactic-keywords): Make obsolete.
(font-lock-fontify-syntactic-keywords-region): Move handling of
font-lock-syntactically-fontified to...
(font-lock-default-fontify-region): ...here.
Let syntax-propertize-function take precedence.
(font-lock-fontify-syntactically-region): Cal syntax-propertize.

* lisp/emacs-lisp/regexp-opt.el (regexp-opt-depth): Skip named groups.

* lisp/emacs-lisp/syntax.el (syntax-propertize-function)
(syntax-propertize-chunk-size, syntax-propertize--done)
(syntax-propertize-extend-region-functions): New vars.
(syntax-propertize-wholelines, syntax-propertize-multiline)
(syntax-propertize--shift-groups, syntax-propertize-via-font-lock)
(syntax-propertize): New functions.
(syntax-propertize-rules): New macro.
(syntax-ppss-flush-cache): Set syntax-propertize--done.
(syntax-ppss): Call syntax-propertize.

* lisp/progmodes/ada-mode.el (ada-set-syntax-table-properties)
(ada-after-change-function, ada-initialize-syntax-table-properties)
(ada-handle-syntax-table-properties): Only define when
syntax-propertize is not available.
(ada-mode): Use syntax-propertize-function.

* lisp/progmodes/autoconf.el (autoconf-mode):
Use syntax-propertize-function.
(autoconf-font-lock-syntactic-keywords): Remove.

* lisp/progmodes/cfengine.el (cfengine-mode):
Use syntax-propertize-function.
(cfengine-font-lock-syntactic-keywords): Remove.

* lisp/progmodes/cperl-mode.el (cperl-mode): Use syntax-propertize-function.

* lisp/progmodes/fortran.el (fortran-mode): Use syntax-propertize-function.
(fortran--font-lock-syntactic-keywords): New var.
(fortran-line-length): Update syntax-propertize-function and
fortran--font-lock-syntactic-keywords.

* lisp/progmodes/gud.el (gdb-script-syntax-propertize-function): New var;
replaces gdb-script-font-lock-syntactic-keywords.
(gdb-script-mode): Use it.

* lisp/progmodes/js.el (js--regexp-literal): Define while compiling.
(js-syntax-propertize-function): New var; replaces
js-font-lock-syntactic-keywords.
(js-mode): Use it.

* lisp/progmodes/make-mode.el (makefile-syntax-propertize-function):
New var; replaces makefile-font-lock-syntactic-keywords.
(makefile-mode): Use it.
(makefile-imake-mode): Adjust.

* lisp/progmodes/mixal-mode.el (mixal-syntax-propertize-function): New var;
replaces mixal-font-lock-syntactic-keywords.
(mixal-mode): Use it.

* lisp/progmodes/octave-mod.el (octave-syntax-propertize-sqs): New function
to replace octave-font-lock-close-quotes.
(octave-syntax-propertize-function): New function to replace
octave-font-lock-syntactic-keywords.
(octave-mode): Use it.

* lisp/progmodes/perl-mode.el (perl-syntax-propertize-function): New fun to
replace perl-font-lock-syntactic-keywords.
(perl-syntax-propertize-special-constructs): New fun to replace
perl-font-lock-special-syntactic-constructs.
(perl-font-lock-syntactic-face-function): New fun.
(perl-mode): Use it.

* lisp/progmodes/python.el (python-syntax-propertize-function): New var to
replace python-font-lock-syntactic-keywords.
(python-mode): Use it.
(python-quote-syntax): Simplify and adjust to new use.

* lisp/progmodes/ruby-mode.el (ruby-here-doc-beg-re):
Define while compiling.
(ruby-here-doc-end-re, ruby-here-doc-beg-match)
(ruby-font-lock-syntactic-keywords, ruby-comment-beg-syntax)
(syntax-ppss, ruby-in-ppss-context-p, ruby-in-here-doc-p)
(ruby-here-doc-find-end, ruby-here-doc-beg-syntax)
(ruby-here-doc-end-syntax): Only define when
syntax-propertize is not available.
(ruby-syntax-propertize-function, ruby-syntax-propertize-heredoc):
New functions.
(ruby-in-ppss-context-p): Update to new syntax of heredocs.
(electric-indent-chars): Silence bytecompiler.
(ruby-mode): Use prog-mode, syntax-propertize-function, and
electric-indent-chars.

* lisp/progmodes/sh-script.el (sh-st-symbol): Remove.
(sh-font-lock-close-heredoc, sh-font-lock-open-heredoc): Add eol arg.
(sh-font-lock-flush-syntax-ppss-cache, sh-font-lock-here-doc): Remove.
(sh-font-lock-quoted-subshell): Assume we've already matched $(.
(sh-font-lock-paren): Set syntax-multiline.
(sh-font-lock-syntactic-keywords): Remove.
(sh-syntax-propertize-function): New function to replace it.
(sh-mode): Use it.

* lisp/progmodes/simula.el (simula-syntax-propertize-function): New var to
replace simula-font-lock-syntactic-keywords.
(simula-mode): Use it.

* lisp/progmodes/tcl.el (tcl-syntax-propertize-function): New var to
replace tcl-font-lock-syntactic-keywords.
(tcl-mode): Use it.

* lisp/progmodes/vhdl-mode.el (vhdl-mode): Use syntax-propertize-function
if available.
(vhdl-fontify-buffer): Adjust.

* lisp/textmodes/bibtex.el (bibtex-mode): Use syntax-propertize-function.

* lisp/textmodes/reftex.el (font-lock-syntactic-keywords): Don't declare
since we don't use it.

* lisp/textmodes/sgml-mode.el (sgml-syntax-propertize-function): New var to
replace sgml-font-lock-syntactic-keywords.
(sgml-mode): Use it.

* lisp/textmodes/tex-mode.el (tex-common-initialization, doctex-mode):
Use syntax-propertize-function.

* lisp/textmodes/texinfo.el (texinfo-syntax-propertize-function): New fun
to replace texinfo-font-lock-syntactic-keywords.
(texinfo-mode): Use it.

* test/indent/octave.m: Remove some `fixindent' not needed any more.

1 files changed, 48 insertions, 48 deletions

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 2f65ffa1e17..10e852223ce 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -166,29 +166,32 @@
           symbol-end)
      . font-lock-builtin-face)))
 
-(defconst python-font-lock-syntactic-keywords
+(defconst python-syntax-propertize-function
   ;; Make outer chars of matching triple-quote sequences into generic
   ;; string delimiters.  Fixme: Is there a better way?
   ;; First avoid a sequence preceded by an odd number of backslashes.
-  `((,(rx (not (any ?\\))
-          ?\\ (* (and ?\\ ?\\))
-          (group (syntax string-quote))
-          (backref 1)
-          (group (backref 1)))
-     (2 ,(string-to-syntax "\"")))      ; dummy
-    (,(rx (group (optional (any "uUrR"))) ; prefix gets syntax property
-          (optional (any "rR"))           ; possible second prefix
-          (group (syntax string-quote))   ; maybe gets property
-          (backref 2)                     ; per first quote
-          (group (backref 2)))            ; maybe gets property
-     (1 (python-quote-syntax 1))
-     (2 (python-quote-syntax 2))
-     (3 (python-quote-syntax 3)))
-    ;; This doesn't really help.
-;;;     (,(rx (and ?\\ (group ?\n))) (1 " "))
-    ))
-
-(defun python-quote-syntax (n)
+  (syntax-propertize-rules
+   (;; (rx (not (any ?\\))
+    ;;     ?\\ (* (and ?\\ ?\\))
+    ;;     (group (syntax string-quote))
+    ;;     (backref 1)
+    ;;     (group (backref 1)))
+    ;; ĦBackrefs don't work in syntax-propertize-rules!
+    "[^\\]\\\\\\(\\\\\\\\\\)*\\(?:''\\('\\)\\|\"\"\\(?2:\"\\)\\)"
+    (2 "\""))                           ; dummy
+   (;; (rx (optional (group (any "uUrR"))) ; prefix gets syntax property
+    ;;     (optional (any "rR"))           ; possible second prefix
+    ;;     (group (syntax string-quote))   ; maybe gets property
+    ;;     (backref 2)                     ; per first quote
+    ;;     (group (backref 2)))            ; maybe gets property
+    ;; ĦBackrefs don't work in syntax-propertize-rules!
+    "\\([RUru]\\)?[Rr]?\\(?:\\('\\)'\\('\\)\\|\\(?2:\"\\)\"\\(?3:\"\\)\\)"
+    (3 (ignore (python-quote-syntax))))
+   ;; This doesn't really help.
+   ;;((rx (and ?\\ (group ?\n))) (1 " "))
+   ))
+
+(defun python-quote-syntax ()
   "Put `syntax-table' property correctly on triple quote.
 Used for syntactic keywords.  N is the match number (1, 2 or 3)."
   ;; Given a triple quote, we have to check the context to know
@@ -206,28 +209,25 @@ Used for syntactic keywords.  N is the match number (1, 2 or 3)."
   ;; x '"""' x """ \"""" x
   (save-excursion
     (goto-char (match-beginning 0))
-    (cond
-     ;; Consider property for the last char if in a fenced string.
-     ((= n 3)
-      (let* ((font-lock-syntactic-keywords nil)
-             (syntax (syntax-ppss)))
-        (when (eq t (nth 3 syntax))     ; after unclosed fence
-          (goto-char (nth 8 syntax))    ; fence position
-          (skip-chars-forward "uUrR")   ; skip any prefix
-          ;; Is it a matching sequence?
-          (if (eq (char-after) (char-after (match-beginning 2)))
-              (eval-when-compile (string-to-syntax "|"))))))
-     ;; Consider property for initial char, accounting for prefixes.
-     ((or (and (= n 2)                  ; leading quote (not prefix)
-               (= (match-beginning 1) (match-end 1))) ; prefix is null
-          (and (= n 1)                  ; prefix
-               (/= (match-beginning 1) (match-end 1)))) ; non-empty
-      (let ((font-lock-syntactic-keywords nil))
-        (unless (eq 'string (syntax-ppss-context (syntax-ppss)))
-          (eval-when-compile (string-to-syntax "|")))))
-     ;; Otherwise (we're in a non-matching string) the property is
-     ;; nil, which is OK.
-     )))
+    (let ((syntax (save-match-data (syntax-ppss))))
+      (cond
+       ((eq t (nth 3 syntax))           ; after unclosed fence
+        ;; Consider property for the last char if in a fenced string.
+        (goto-char (nth 8 syntax))      ; fence position
+        (skip-chars-forward "uUrR")     ; skip any prefix
+        ;; Is it a matching sequence?
+        (if (eq (char-after) (char-after (match-beginning 2)))
+            (put-text-property (match-beginning 3) (match-end 3)
+                               'syntax-table (string-to-syntax "|"))))
+       ((match-end 1)
+        ;; Consider property for initial char, accounting for prefixes.
+        (put-text-property (match-beginning 1) (match-end 1)
+                           'syntax-table (string-to-syntax "|")))
+       (t
+        ;; Consider property for initial char, accounting for prefixes.
+        (put-text-property (match-beginning 2) (match-end 2)
+                           'syntax-table (string-to-syntax "|"))))
+      )))
 
 ;; This isn't currently in `font-lock-defaults' as probably not worth
 ;; it -- we basically only mess with a few normally-symbol characters.
@@ -2495,12 +2495,12 @@ with skeleton expansions for compound statement templates.
   :group 'python
   (set (make-local-variable 'font-lock-defaults)
        '(python-font-lock-keywords nil nil nil nil
-                                   (font-lock-syntactic-keywords
-                                    . python-font-lock-syntactic-keywords)
-                                   ;; This probably isn't worth it.
-                                   ;; (font-lock-syntactic-face-function
-                                   ;;  . python-font-lock-syntactic-face-function)
-                                   ))
+         ;; This probably isn't worth it.
+         ;; (font-lock-syntactic-face-function
+         ;;  . python-font-lock-syntactic-face-function)
+         ))
+  (set (make-local-variable 'syntax-propertize-function)
+       python-syntax-propertize-function)
   (set (make-local-variable 'parse-sexp-lookup-properties) t)
   (set (make-local-variable 'parse-sexp-ignore-comments) t)
   (set (make-local-variable 'comment-start) "# ")