Emacs 'movemail' is now a configure-time option

The new configure option --with-mailutils lets the builder say
that Emacs should assume that GNU Mailutils is installed, instead
of continuing to build and install its own limited and insecure
substitute for 'movemail'.
* INSTALL, etc/NEWS, etc/PROBLEMS: Mention --with-mailutils.
* configure.ac: Add --with-mailutils option.
(with_mailutils): New variable.
Do not bother configuring 'movemail' when not building it.
Warn about issues relating to --with-mailutils.
* doc/emacs/rmail.texi (Movemail): Mention --with-mailutils.
(Movemail, Remote Mailboxes): Document port numbers in
POP and IMAP URLs.
* lib-src/Makefile.in (with_mailutils): New macro.
(UTILITIES): Use it.

6 files changed, 175 insertions, 119 deletions

diff --git a/INSTALL b/INSTALL
index 10b17560031..a99f1078044 100644
--- a/INSTALL
+++ b/INSTALL
@@ -259,12 +259,23 @@ inconvenient for Emacs when running as daemon or running via a remote
 ssh connection.  In order to completely prevent the use of D-Bus, configure
 Emacs with the options '--without-dbus --without-gconf --without-gsettings'.
 
+To read email via a network protocol like IMAP or POP, you can
+configure Emacs with the option '--with-mailutils', so that it always
+uses the GNU Mailutils 'movemail' program to retrieve mail.  Otherwise
+the Emacs build procedure builds and installs an auxiliary 'movemail'
+program, a limited and insecure substitute that Emacs can use when
+Mailutils is not installed; when this happens, there are several
+configure options such as --without-pop that provide fine-grained
+control over Emacs 'movemail' construction.
+
 The Emacs mail reader RMAIL is configured to be able to read mail from
 a POP3 server by default.  Versions of the POP protocol older than
-POP3 are not supported.  For Kerberos-authenticated POP add
-'--with-kerberos', for Hesiod support add '--with-hesiod'.  While POP3
-is always enabled, whether Emacs actually uses POP is controlled by
-individual users--see the Rmail chapter of the Emacs manual.
+POP3 are not supported.  While POP3 support is typically enabled,
+whether Emacs actually uses POP3 is controlled by individual users;
+see the Rmail chapter of the Emacs manual.  Unless you configure
+--with-mailutils, it is a good idea to configure --without-pop so that
+users are less likely to inadvertently read email via insecure
+channels.
 
 For image support you may have to download, build, and install the
 appropriate image support libraries for image types other than XBM and
@@ -538,7 +549,8 @@ information on this.
 Emacs info files.
 
 8) If your system uses lock files to interlock access to mailer inbox files,
-then you might need to make the movemail program setuid or setgid
+and if you did not configure --with-mailutils, then you might need to
+make the Emacs-specific 'movemail' program setuid or setgid in order
 to enable it to write the lock files.  We believe this is safe.
 
 9) You are done!  You can remove executables and object files from
diff --git a/configure.ac b/configure.ac
index 4d9ba9636d5..47d0e5cd19a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -240,6 +240,18 @@ AC_DEFUN([OPTION_DEFAULT_ON], [dnl
    m4_bpatsubst([with_$1], [[^0-9a-z]], [_])=$with_features])dnl
 ])dnl
 
+# FIXME: The default options '--without-mailutils --with-pop' result
+# in a movemail implementation that supports only unencrypted POP3
+# connections.  Encrypted connections should be the default.
+
+OPTION_DEFAULT_OFF([mailutils],
+  [rely on GNU Mailutils being installed; this makes the options
+   --without-pop through --with-mailhost irrelevant)])
+if test "$with_mailutils" = no; then
+  with_mailutils=
+fi
+AC_SUBST([with_mailutils])
+
 OPTION_DEFAULT_ON([pop],[don't support POP mail retrieval with movemail])
 if test "$with_pop" = yes; then
    AC_DEFINE(MAIL_USE_POP)
@@ -3789,87 +3801,91 @@ fi
 AC_SUBST(LIBXML2_LIBS)
 AC_SUBST(LIBXML2_CFLAGS)
 
-# Check for mail-locking functions in a "mail" library.  Probably this should
-# have the same check as for liblockfile below.
-AC_CHECK_LIB(mail, maillock, have_mail=yes, have_mail=no)
-if test $have_mail = yes; then
-  LIBS_MAIL=-lmail
-  AC_DEFINE(HAVE_LIBMAIL, 1, [Define to 1 if you have the 'mail' library (-lmail).])
-
-  OLD_LIBS=$LIBS
-  LIBS="$LIBS_MAIL $LIBS"
-  AC_CHECK_FUNCS(touchlock)
-  LIBS=$OLD_LIBS
-else
-  LIBS_MAIL=
-fi
-dnl Debian, at least:
-AC_CHECK_LIB(lockfile, maillock, have_lockfile=yes, have_lockfile=no)
-if test $have_lockfile = yes; then
-   LIBS_MAIL=-llockfile
-   AC_DEFINE(HAVE_LIBLOCKFILE, 1, [Define to 1 if you have the 'lockfile' library (-llockfile).])
-else
-# If we have the shared liblockfile, assume we must use it for mail
-# locking (e.g. Debian).  If we couldn't link against liblockfile
-# (no liblockfile.a installed), ensure that we don't need to.
-  dnl This works for files generally, not just executables.
-  dnl Should we look elsewhere for it?  Maybe examine /etc/ld.so.conf?
-  AC_CHECK_PROG(liblockfile, liblockfile.so, yes, no,
-                /usr/lib:/lib:/usr/local/lib:$LD_LIBRARY_PATH)
-  if test $ac_cv_prog_liblockfile = yes; then
-    AC_MSG_ERROR([Shared liblockfile found but can't link against it.
+BLESSMAIL_TARGET=
+LIBS_MAIL=
+if test ! "$with_mailutils"; then
+  # Check for mail-locking functions in a "mail" library.  Probably this should
+  # have the same check as for liblockfile below.
+  AC_CHECK_LIB([mail], [maillock], [have_mail=yes], [have_mail=no])
+  if test $have_mail = yes; then
+    LIBS_MAIL=-lmail
+    AC_DEFINE([HAVE_LIBMAIL], [1],
+      [Define to 1 if you have the 'mail' library (-lmail).])
+
+    OLD_LIBS=$LIBS
+    LIBS="$LIBS_MAIL $LIBS"
+    AC_CHECK_FUNCS([touchlock])
+    LIBS=$OLD_LIBS
+  fi
+  dnl Debian, at least:
+  AC_CHECK_LIB([lockfile], [maillock], [have_lockfile=yes], [have_lockfile=no])
+  if test $have_lockfile = yes; then
+     LIBS_MAIL=-llockfile
+     AC_DEFINE([HAVE_LIBLOCKFILE], [1],
+       [Define to 1 if you have the 'lockfile' library (-llockfile).])
+  else
+    # If we have the shared liblockfile, assume we must use it for mail
+    # locking (e.g. Debian).  If we couldn't link against liblockfile
+    # (no liblockfile.a installed), ensure that we don't need to.
+    dnl This works for files generally, not just executables.
+    dnl Should we look elsewhere for it?  Maybe examine /etc/ld.so.conf?
+    AC_CHECK_PROG([liblockfile], [liblockfile.so], [yes], [no],
+                  [/usr/lib:/lib:/usr/local/lib:$LD_LIBRARY_PATH])
+    if test "$ac_cv_prog_liblockfile" = yes; then
+      AC_MSG_ERROR([Shared liblockfile found but can't link against it.
 This probably means that movemail could lose mail.
 There may be a 'development' package to install containing liblockfile.])
+    fi
   fi
-fi
-AC_CHECK_HEADERS_ONCE(maillock.h)
-AC_SUBST(LIBS_MAIL)
+  AC_CHECK_HEADERS([maillock.h])
 
-## Define MAIL_USE_FLOCK (or LOCKF) if the mailer uses flock (or lockf) to
-## interlock access to the mail spool.  The alternative is a lock file named
-## /usr/spool/mail/$USER.lock.
-mail_lock=no
-case "$opsys" in
-  aix4-2) mail_lock="lockf" ;;
-
-  gnu|freebsd|dragonfly|netbsd|openbsd|darwin) mail_lock="flock" ;;
-
-  ## On GNU/Linux systems, both methods are used by various mail programs.
-  ## I assume most people are using newer mailers that have heard of flock.
-  ## Change this if you need to.
-  ## Debian contains a patch which says: "On Debian/GNU/Linux systems,
-  ## configure gets the right answers, and that means *NOT* using flock.
-  ## Using flock is guaranteed to be the wrong thing. See Debian Policy
-  ## for details." and then uses '#ifdef DEBIAN'.  Unfortunately the
-  ## Debian maintainer hasn't provided a clean fix for Emacs.
-  ## movemail.c will use 'maillock' when MAILDIR, HAVE_LIBMAIL and
-  ## HAVE_MAILLOCK_H are defined, so the following appears to be the
-  ## correct logic.  -- fx
-  ## We must check for HAVE_LIBLOCKFILE too, as movemail does.
-  ## liblockfile is a Free Software replacement for libmail, used on
-  ## Debian systems and elsewhere. -rfr.
-  gnu-*)
-    mail_lock="flock"
-    if test $have_mail = yes || test $have_lockfile = yes; then
-      test $ac_cv_header_maillock_h = yes && mail_lock=no
-    fi
-    ;;
+  ## Define MAIL_USE_FLOCK (or LOCKF) if the mailer uses flock (or lockf) to
+  ## interlock access to the mail spool.  The alternative is a lock file named
+  ## /usr/spool/mail/$USER.lock.
+  mail_lock=no
+  case $opsys in
+    aix4-2) mail_lock="lockf" ;;
+
+    gnu|freebsd|dragonfly|netbsd|openbsd|darwin) mail_lock="flock" ;;
+
+    ## On GNU/Linux systems, both methods are used by various mail programs.
+    ## I assume most people are using newer mailers that have heard of flock.
+    ## Change this if you need to.
+    ## Debian contains a patch which says: "On Debian/GNU/Linux systems,
+    ## configure gets the right answers, and that means *NOT* using flock.
+    ## Using flock is guaranteed to be the wrong thing. See Debian Policy
+    ## for details." and then uses '#ifdef DEBIAN'.  Unfortunately the
+    ## Debian maintainer hasn't provided a clean fix for Emacs.
+    ## movemail.c will use 'maillock' when MAILDIR, HAVE_LIBMAIL and
+    ## HAVE_MAILLOCK_H are defined, so the following appears to be the
+    ## correct logic.  -- fx
+    ## We must check for HAVE_LIBLOCKFILE too, as movemail does.
+    ## liblockfile is a Free Software replacement for libmail, used on
+    ## Debian systems and elsewhere. -rfr.
+    gnu-*)
+      mail_lock="flock"
+      if test $have_mail = yes || test $have_lockfile = yes; then
+        test $ac_cv_header_maillock_h = yes && mail_lock=no
+      fi ;;
 
-  mingw32)
-    mail_lock="none-needed" ;;
-esac
+    mingw32)
+      mail_lock="none-needed" ;;
+  esac
 
-BLESSMAIL_TARGET=
-case "$mail_lock" in
-  flock) AC_DEFINE(MAIL_USE_FLOCK, 1, [Define if the mailer uses flock to interlock the mail spool.]) ;;
+  case $mail_lock in
+    flock) AC_DEFINE([MAIL_USE_FLOCK], [1],
+             [Define if the mailer uses flock to interlock the mail spool.]);;
 
-  lockf) AC_DEFINE(MAIL_USE_LOCKF, 1, [Define if the mailer uses lockf to interlock the mail spool.]) ;;
+    lockf) AC_DEFINE([MAIL_USE_LOCKF], [1],
+             [Define if the mailer uses lockf to interlock the mail spool.]);;
 
-  none-needed) ;;
+    none-needed) ;;
 
-  *) BLESSMAIL_TARGET="need-blessmail" ;;
-esac
-AC_SUBST(BLESSMAIL_TARGET)
+    *) BLESSMAIL_TARGET="need-blessmail" ;;
+  esac
+fi
+AC_SUBST([BLESSMAIL_TARGET])
+AC_SUBST([LIBS_MAIL])
 
 OLD_LIBS=$LIBS
 LIBS="$LIB_PTHREAD $LIB_MATH $LIBS"
@@ -5469,4 +5485,21 @@ AC_MSG_ERROR(['etc/refcards/emacsver.tex' could not be made.])
 
 AC_OUTPUT
 
+if test ! "$with_mailutils"; then
+  emacs_use_mailutils="use '$0 --with-mailutils'"
+  case `(movemail --version) 2>/dev/null` in
+    *Mailutils*) ;;
+    *) emacs_use_mailutils="install GNU Mailutils
+<http://mailutils.org> and $emacs_use_mailutils";;
+  esac
+  if test "$with_pop" = yes; then
+    AC_MSG_WARN([This configuration installs a 'movemail' program
+that retrieves POP3 email via only insecure channels.
+To fix this you can $emacs_use_mailutils.])
+  else
+    AC_MSG_NOTICE([This configuration installs a 'movemail' program that cannot
+retrieve POP3 email.  You might want to $emacs_use_mailutils.])
+  fi
+fi
+
 test "$MAKE" = make || AC_MSG_NOTICE([Now you can run '$MAKE'.])
diff --git a/doc/emacs/rmail.texi b/doc/emacs/rmail.texi
index d46398281fb..b64ec223be7 100644
--- a/doc/emacs/rmail.texi
+++ b/doc/emacs/rmail.texi
@@ -1380,15 +1380,17 @@ which applies the code when displaying the text.
 your Rmail file (@pxref{Rmail Inbox}).  When loaded for the first time,
 Rmail attempts to locate the @code{movemail} program and determine its
 version.  There are two versions of the @code{movemail} program: the
-native one, shipped with GNU Emacs (the Emacs version) and the one
-included in GNU mailutils (the mailutils version,
-@pxref{movemail,,,mailutils,GNU mailutils}).  They support the same
+GNU Mailutils version (@pxref{movemail,,,mailutils,GNU mailutils}),
+and an Emacs-specific version that is built and installed unless Emacs
+was configured using the @option{--with-mailutils} option.
+The two @command{mailtool} versions support the same
 command line syntax and the same basic subset of options.  However, the
 Mailutils version offers additional features.
 
-  The Emacs version of @code{movemail} is able to retrieve mail from
-the usual Unix mailbox formats and from remote mailboxes using the
-POP3 protocol.
+  The Emacs version of @command{movemail} can retrieve mail from the
+usual Unix mailbox formats.  Although it can also use the POP3
+protocol, this is not recommended because it does not support POP3 via
+encrypted TLS channels.
 
   The Mailutils version is able to handle a wide set of mailbox
 formats, such as plain Unix mailboxes, @code{maildir} and @code{MH}
@@ -1401,7 +1403,7 @@ in @ref{URL,,,mailutils,Mailbox URL Formats}.  In short, a
 @acronym{URL} is:
 
 @smallexample
-@var{proto}://[@var{user}[:@var{password}]@@]@var{host-or-file-name}
+@var{proto}://[@var{user}[:@var{password}]@@]@var{host-or-file-name}[:@var{port}]
 @end smallexample
 
 @noindent
@@ -1422,6 +1424,9 @@ User password to access the remote mailbox.
 @item host-or-file-name
 Hostname of the remote server for remote mailboxes or file name of a
 local mailbox.
+
+@item port
+Optional port number, if not the default for that protocal.
 @end table
 
 @noindent
@@ -1429,19 +1434,20 @@ local mailbox.
 
 @table @code
 @item mbox
-Usual Unix mailbox format.  In this case, neither @var{user} nor
-@var{pass} are used, and @var{host-or-file-name} denotes the file name
-of the mailbox file, e.g., @code{mbox://var/spool/mail/smith}.
+Usual Unix mailbox format.  In this case, @var{user}, @var{pass} and
+@var{port} are not used, and @var{host-or-file-name} denotes the file
+name of the mailbox file, e.g., @code{mbox://var/spool/mail/smith}.
 
 @item mh
-A local mailbox in the @acronym{MH} format.  @var{user} and
-@var{pass} are not used.  @var{host-or-file-name} denotes the name of
-@acronym{MH} folder, e.g., @code{mh://Mail/inbox}.
+A local mailbox in the @acronym{MH} format.  @var{user}, @var{pass}
+and @var{port} are not used.  @var{host-or-file-name} denotes the name
+of @acronym{MH} folder, e.g., @code{mh://Mail/inbox}.
 
 @item maildir
-A local mailbox in the @acronym{maildir} format.  @var{user} and
-@var{pass} are not used, and @var{host-or-file-name} denotes the name of
-@code{maildir} mailbox, e.g., @code{maildir://mail/inbox}.
+A local mailbox in the @acronym{maildir} format.  @var{user},
+@var{pass} and @var{port} are not used, and @var{host-or-file-name}
+denotes the name of @code{maildir} mailbox, e.g.,
+@code{maildir://mail/inbox}.
 
 @item file
 Any local mailbox format.  Its actual format is detected automatically
@@ -1451,15 +1457,15 @@ by @code{movemail}.
 A remote mailbox to be accessed via POP3 protocol.  @var{user}
 specifies the remote user name to use, @var{pass} may be used to
 specify the user password, @var{host-or-file-name} is the name or IP
-address of the remote mail server to connect to; e.g.,
-@code{pop://smith:guessme@@remote.server.net}.
+address of the remote mail server to connect to, and @var{port} is the
+port number; e.g., @code{pop://smith:guessme@@remote.server.net:995}.
 
 @item imap
 A remote mailbox to be accessed via IMAP4 protocol.  @var{user}
 specifies the remote user name to use, @var{pass} may be used to
 specify the user password, @var{host-or-file-name} is the name or IP
-address of the remote mail server to connect to;
-e.g., @code{imap://smith:guessme@@remote.server.net}.
+address of the remote mail server to connect to, and @var{port} is the
+port number; e.g., @code{imap://smith:guessme@@remote.server.net:993}.
 @end table
 
   Alternatively, you can specify the file name of the mailbox to use.
@@ -1483,44 +1489,40 @@ listed in @code{rmail-movemail-search-path}, then in @code{exec-path}
 @pindex movemail
 
   Some sites use a method called POP for accessing users' inbox data
-instead of storing the data in inbox files.  By default, the @code{Emacs
-movemail} can work with POP (unless the Emacs @code{configure} script
-was run with the option @samp{--without-pop}).
-
-Similarly, the Mailutils @code{movemail} by default supports POP, unless
-it was configured with the @samp{--disable-pop} option.
-
-Both versions of @code{movemail} only work with POP3, not with older
-versions of POP.
+instead of storing the data in inbox files.  The Mailutils
+@command{movemail} by default supports POP with TLS encryption.
+Although the @command{Emacs movemail} supports unencrypted POP, this
+is not recommended due to the lack of encryption.  Both versions of
+@command{movemail} work only with POP3, not with older versions of
+POP.
 
 @cindex @env{MAILHOST} environment variable
 @cindex POP mailboxes
-  No matter which flavor of @code{movemail} you use, you can specify
+  You can specify
 a POP inbox by using a POP @dfn{URL} (@pxref{Movemail}).  A POP
 @acronym{URL} is of the form
-@samp{pop://@var{username}@@@var{hostname}}, where
-@var{hostname} is the host name or IP address of the remote mail
+@samp{pop://@var{username}@@@var{hostname}:@var{port}}, where
+@var{hostname} and @var{port} are the host name (or IP address)
+and port number of the remote mail
 server and @var{username} is the user name on that server.
 Additionally, you may specify the password in the mailbox @acronym{URL}:
-@samp{pop://@var{username}:@var{password}@@@var{hostname}}.  In this
+@samp{pop://@var{username}:@var{password}@@@var{hostname}:@var{port}}.  In this
 case, @var{password} takes preference over the one set by
 @code{rmail-remote-password} (see below).  This is especially useful
 if you have several remote mailboxes with different passwords.
 
   For backward compatibility, Rmail also supports an alternative way of
 specifying remote POP mailboxes.  Specifying an inbox name in the form
-@samp{po:@var{username}:@var{hostname}} is equivalent to
-@samp{pop://@var{username}@@@var{hostname}}.  If you omit the
+@samp{po:@var{username}:@var{hostname}:@var{port}} is equivalent to
+@samp{pop://@var{username}@@@var{hostname}:@var{port}}.  If you omit the
 @var{:hostname} part, the @env{MAILHOST} environment variable specifies
 the machine on which to look for the POP server.
 
-@c FIXME mention --with-hesiod "support Hesiod to get the POP server host"?
-
 @cindex IMAP mailboxes
   Another method for accessing remote mailboxes is IMAP@.  This method is
 supported only by the Mailutils @code{movemail}.  To specify an IMAP
 mailbox in the inbox list, use the following mailbox @acronym{URL}:
-@samp{imap://@var{username}[:@var{password}]@@@var{hostname}}.  The
+@samp{imap://@var{username}[:@var{password}]@@@var{hostname}:@var{port}}.  The
 @var{password} part is optional, as described above.
 
 @vindex rmail-remote-password
@@ -1550,9 +1552,7 @@ preserve your inbox contents; use @code{rmail-preserve-inbox} instead.
 
 @cindex Kerberos POP authentication
   The @code{movemail} program installed at your site may support
-Kerberos authentication (the Emacs @code{movemail} does so if Emacs was
-configured with the option @code{--with-kerberos} or
-@code{--with-kerberos5}).  If it is supported, it is used by default
+Kerberos authentication.  If it is supported, it is used by default
 whenever you attempt to retrieve POP mail when
 @code{rmail-remote-password} and @code{rmail-remote-password-required}
 are unset.
diff --git a/etc/NEWS b/etc/NEWS
index 88ae7cf1f52..f047aa86f9f 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -37,6 +37,12 @@ Use 'configure --with-gnutls=no' to build even when GnuTLS is missing.
 ** GnuTLS version 2.12.2 or later is now required, instead of merely
 version 2.6.6 or later.
 
+** The new option 'configure --with-mailutils' causes Emacs to rely on
+GNU Mailutils 'movemail' to retrieve email.  By default, the Emacs
+build procedure continues to build and install a limited and insecure
+'movemail' substitute.  Although --with-mailutils is recommended, it
+is not yet the default due to backward-compatibility concerns.
+
 ** The new option 'configure --enable-gcc-warnings=warn-only' causes
 GCC to issue warnings without stopping the build.  This behavior is
 now the default in developer builds.  As before, use
diff --git a/etc/PROBLEMS b/etc/PROBLEMS
index b8923201014..eddae12a70f 100644
--- a/etc/PROBLEMS
+++ b/etc/PROBLEMS
@@ -323,6 +323,7 @@ or set the variable 'cannot-suspend' to a non-nil value.
 
 *** movemail compiled with POP support can't connect to the POP server.
 
+This problem can occur if you do not configure --with-mailutils.
 Make sure that the 'pop' entry in /etc/services, or in the services
 NIS map if your machine uses NIS, has the same port number as the
 entry on the POP server.  A common error is for the POP server to be
diff --git a/lib-src/Makefile.in b/lib-src/Makefile.in
index d0aa9cc7c27..b2cd66c0498 100644
--- a/lib-src/Makefile.in
+++ b/lib-src/Makefile.in
@@ -100,6 +100,9 @@ bindir=@bindir@
 # --libexecdir option to '../configure'.
 libexecdir=@libexecdir@
 
+# Nonempty if Emacs can assume Mailutils is installed.
+with_mailutils=@with_mailutils@
+
 # Directory for local state files for all programs.
 localstatedir=@localstatedir@
 
@@ -160,7 +163,8 @@ INSTALLABLES = etags${EXEEXT} ctags${EXEEXT} emacsclient${EXEEXT} $(CLIENTW) \
 
 # Things that Emacs runs internally, or during the build process,
 #  which should not be installed in bindir.
-UTILITIES = profile${EXEEXT} movemail${EXEEXT} hexl${EXEEXT} \
+UTILITIES = profile${EXEEXT} hexl${EXEEXT} \
+            $(if $(with_mailutils), , movemail${EXEEXT}) \
             $(and $(use_gamedir), update-game-score${EXEEXT})
 
 DONT_INSTALL= make-docfile${EXEEXT}