Fix problems caused by new implementation of sub-word mode

* lisp/subr.el (forward-word-strictly, backward-word-strictly):
New functions.
(word-move-empty-char-table): New variable.

* etc/NEWS: Mention 'forward-word-strictly' and
'backward-word-strictly'.

* doc/lispref/positions.texi (Word Motion): Document
'find-word-boundary-function-table', 'forward-word-strictly', and
'backward-word-strictly'.  (Bug#22560)

* src/syntax.c (syms_of_syntax)
<find-word-boundary-function-table>: Doc fix.

* lisp/wdired.el (wdired-xcase-word):
* lisp/textmodes/texnfo-upd.el (texinfo-copy-node-name)
(texinfo-copy-section-title, texinfo-start-menu-description)
(texinfo-copy-menu-title, texinfo-specific-section-type)
(texinfo-insert-node-lines, texinfo-copy-next-section-title):
* lisp/textmodes/texinfo.el (texinfo-clone-environment)
(texinfo-insert-@end):
* lisp/textmodes/texinfmt.el (texinfo-format-scan)
(texinfo-anchor, texinfo-multitable-widths)
(texinfo-multitable-item):
* lisp/textmodes/tex-mode.el (latex-env-before-change):
* lisp/textmodes/flyspell.el (texinfo-mode-flyspell-verify):
* lisp/skeleton.el (skeleton-insert):
* lisp/simple.el (count-words):
* lisp/progmodes/vhdl-mode.el (vhdl-beginning-of-libunit)
(vhdl-beginning-of-defun, vhdl-beginning-of-statement-1)
(vhdl-update-sensitivity-list, vhdl-template-block)
(vhdl-template-break, vhdl-template-case, vhdl-template-default)
(vhdl-template-default-indent, vhdl-template-for-loop)
(vhdl-template-if-then-use, vhdl-template-bare-loop)
(vhdl-template-nature, vhdl-template-procedural)
(vhdl-template-process, vhdl-template-selected-signal-asst)
(vhdl-template-type, vhdl-template-variable)
(vhdl-template-while-loop, vhdl-beginning-of-block)
(vhdl-hooked-abbrev, vhdl-port-copy, vhdl-hs-forward-sexp-func):
* lisp/progmodes/verilog-mode.el (verilog-backward-sexp)
(verilog-forward-sexp, verilog-beg-of-statement)
(verilog-set-auto-endcomments, verilog-backward-token)
(verilog-do-indent):
* lisp/progmodes/vera-mode.el (vera-guess-basic-syntax)
(vera-indent-block-closing):
* lisp/progmodes/simula.el (simula-context)
(simula-backward-up-level, simula-forward-down-level)
(simula-previous-statement, simula-next-statement)
(simula-skip-comment-backward, simula-calculate-indent)
(simula-find-if, simula-electric-keyword):
* lisp/progmodes/sh-script.el (sh-smie--rc-newline-semi-p):
* lisp/progmodes/ruby-mode.el (ruby-smie--redundant-do-p)
(ruby-smie--forward-token, ruby-smie--backward-token)
(ruby-singleton-class-p, ruby-calculate-indent)
(ruby-forward-sexp, ruby-backward-sexp):
* lisp/progmodes/ps-mode.el (ps-run-goto-error):
* lisp/progmodes/perl-mode.el (perl-syntax-propertize-function)
(perl-syntax-propertize-special-constructs)
(perl-backward-to-start-of-continued-exp):
* lisp/progmodes/pascal.el (pascal-indent-declaration):
* lisp/progmodes/octave.el (octave-function-file-p):
* lisp/progmodes/mantemp.el (mantemp-insert-cxx-syntax):
* lisp/progmodes/js.el (js--forward-function-decl):
* lisp/progmodes/idlwave.el (idlwave-show-begin-check)
(idlwave-beginning-of-block, idlwave-end-of-block)
(idlwave-block-jump-out, idlwave-determine-class):
* lisp/progmodes/icon.el (icon-is-continuation-line)
(icon-backward-to-start-of-continued-exp, end-of-icon-defun):
* lisp/progmodes/hideif.el (hide-ifdef-define):
* lisp/progmodes/f90.el (f90-change-keywords):
* lisp/progmodes/cperl-mode.el (cperl-electric-pod)
(cperl-linefeed, cperl-electric-terminator)
(cperl-find-pods-heres, cperl-fix-line-spacing)
(cperl-invert-if-unless):
* lisp/progmodes/cc-engine.el (c-forward-<>-arglist-recur):
* lisp/progmodes/cc-align.el (c-lineup-java-inher):
* lisp/progmodes/ada-mode.el (ada-compile-goto-error)
(ada-adjust-case-skeleton, ada-create-case-exception)
(ada-create-case-exception-substring)
(ada-case-read-exceptions-from-file, ada-after-keyword-p)
(ada-scan-paramlist, ada-get-current-indent, ada-get-indent-end)
(ada-get-indent-if, ada-get-indent-block-start)
(ada-get-indent-loop, ada-get-indent-type)
(ada-search-prev-end-stmt, ada-check-defun-name)
(ada-goto-decl-start, ada-goto-matching-start)
(ada-goto-matching-end, ada-looking-at-semi-or)
(ada-looking-at-semi-private, ada-in-paramlist-p)
(ada-search-ignore-complex-boolean, ada-move-to-start)
(ada-move-to-end, ada-which-function, ada-gen-treat-proc):
* lisp/net/quickurl.el (quickurl-grab-url):
* lisp/mail/sendmail.el (mail-do-fcc):
* lisp/mail/rmail.el (rmail-resend):
* lisp/mail/mailabbrev.el (mail-abbrev-complete-alias):
* lisp/mail/mail-extr.el (mail-extract-address-components):
* lisp/json.el (json-read-keyword):
* lisp/files.el (insert-directory):
* lisp/emacs-lisp/checkdoc.el (checkdoc-this-string-valid-engine):
* lisp/completion.el (symbol-under-point, symbol-before-point)
(symbol-before-point-for-complete, next-cdabbrev)
(add-completions-from-c-buffer):
* lisp/cedet/semantic/texi.el (semantic-up-context)
(semantic-beginning-of-context):
* lisp/cedet/semantic/bovine/el.el (semantic-get-local-variables):
use 'forward-word-strictly' and 'backward-word-strictly' instead
of 'forward-word' and 'backward-word'.

1 files changed, 48 insertions, 12 deletions

diff --git a/doc/lispref/positions.texi b/doc/lispref/positions.texi
index b88e76e8230..1d748b87529 100644
--- a/doc/lispref/positions.texi
+++ b/doc/lispref/positions.texi
@@ -191,18 +191,25 @@ in the opposite direction.
 @node Word Motion
 @subsection Motion by Words
 
-  These functions for parsing words use the syntax table to decide
-whether a given character is part of a word.  @xref{Syntax Tables}.
+  The functions for parsing words described below use the syntax table
+to decide whether a given character is part of a word.  @xref{Syntax
+Tables}.
 
 @deffn Command forward-word &optional count
 This function moves point forward @var{count} words (or backward if
 @var{count} is negative).  If @var{count} is omitted or @code{nil}, it
-defaults to 1.
+defaults to 1.  In an interactive call, @var{count} is specified by
+the numeric prefix argument.
 
 ``Moving one word'' means moving until point crosses a
-word-constituent character and then encounters a word-separator
-character.  However, this function cannot move point past the boundary
-of the accessible portion of the buffer, or across a field boundary
+word-constituent character, which indicates the beginning of a word,
+and then continue moving until the word ends.  By default, characters
+that begin and end words, known as @dfn{word boundaries}, are defined
+by the current buffer's syntax table (@pxref{Syntax Class Table}), but
+modes can override that by setting up a suitable
+@code{find-word-boundary-function-table}, described below.  In any
+case, this function cannot move point past the boundary of the
+accessible portion of the buffer, or across a field boundary
 (@pxref{Fields}).  The most common case of a field boundary is the end
 of the prompt in the minibuffer.
 
@@ -214,8 +221,6 @@ the buffer boundary or field boundary.
 If @code{inhibit-field-text-motion} is non-@code{nil},
 this function ignores field boundaries.
 
-In an interactive call, @var{count} is specified by the numeric prefix
-argument.
 @end deffn
 
 @deffn Command backward-word &optional count
@@ -225,10 +230,10 @@ backward until encountering the front of a word, rather than forward.
 
 @defopt words-include-escapes
 @c Emacs 19 feature
-This variable affects the behavior of @code{forward-word} and everything
-that uses it.  If it is non-@code{nil}, then characters in the
-escape and character-quote syntax classes count as part of
-words.  Otherwise, they do not.
+This variable affects the behavior of @code{forward-word} and
+@code{backward-word}, and everything that uses them.  If it is
+non-@code{nil}, then characters in the escape and character-quote
+syntax classes count as part of words.  Otherwise, they do not.
 @end defopt
 
 @defvar inhibit-field-text-motion
@@ -237,6 +242,37 @@ If this variable is non-@code{nil}, certain motion functions including
 @code{forward-paragraph} ignore field boundaries.
 @end defvar
 
+@defvar find-word-boundary-function-table
+This variable affects the behavior of @code{forward-word} and
+@code{backward-word}, and everything that uses them.  Its value is a
+char-table (@pxref{Char-Tables}) of functions to search for word
+boundaries.  If a character has a non-@code{nil} entry in this table,
+then when a word starts or ends with that character, the corresponding
+function will be called with 2 arguments: @var{pos} and @var{limit}.
+The function should return the position of the other word boundary.
+Specifically, if @var{pos} is smaller than @var{limit}, then @var{pos}
+is at the beginning of a word, and the function should return the
+position after the last character of the word; otherwise, @var{pos} is
+at the last character of a word, and the function should return the
+position of that word's first character.
+@end defvar
+
+@defun forward-word-strictly &optional count
+This function is like @code{forward-word}, but it is not affected by
+@code{find-word-boundary-function-table}.  Lisp programs that should
+not change behavior when word movement is modified by modes which set
+that table, such as @code{subword-mode}, should use this function
+instead of @code{forward-word}.
+@end defun
+
+@defun backward-word-strictly &optional count
+This function is like @code{backward-word}, but it is not affected by
+@code{find-word-boundary-function-table}.  Like with
+@code{forward-word-strictly}, use this function instead of
+@code{backward-word} when movement by words should only consider
+syntax tables.
+@end defun
+
 @node Buffer End Motion
 @subsection Motion to an End of the Buffer
 @cindex move to beginning or end of buffer