Mention AddressSanitizer etc. in etc/DEBUG

* etc/DEBUG: Modernize for AddressSanitizer etc.
* etc/NEWS: Defer to etc/DEBUG for this.

2 files changed, 82 insertions, 47 deletions

diff --git a/etc/DEBUG b/etc/DEBUG
index d401d0be901..717553871a7 100644
--- a/etc/DEBUG
+++ b/etc/DEBUG
@@ -12,24 +12,21 @@ debugging techniques.
 *** Configuring Emacs for debugging
 
 It is best to configure and build Emacs with special options that will
-make the debugging easier.  Here's the configure-time options we
+make the debugging easier.  Here are the configure-time options we
 recommend (they are in addition to any other options you might need,
 such as --prefix):
 
   ./configure --enable-checking='yes,glyphs' --enable-check-lisp-object-type \
     CFLAGS='-O0 -g3'
 
-The CFLAGS value is important: debugging optimized code can be very
-hard.  (If the problem only happens with optimized code, you may need
-to enable optimizations.  If that happens, try using -Og first,
-instead of -O2, as the former will disable some optimizations that
-make debugging some code exceptionally hard.)
+The -O0 flag is important, as debugging optimized code can be hard.
+If the problem happens only with optimized code, you may need to
+enable optimizations.  If that happens, try using -Og first instead of
+-O2, as -Og disables some optimizations that make debugging some code
+exceptionally hard.
 
-Modern versions of GCC support more elaborate debug info that is
-available by just using the -g3 compiler switch.  Try using -gdwarf-4
-in addition to -g3, and if that fails, try -gdwarf-3.  This is
-especially important if you have to debug optimized code.  More info
-about this is available below; search for "analyze failed assertions".
+Older versions of GCC may need more than just the -g3 flag.  For more,
+search for "analyze failed assertions" below.
 
 The 2 --enable-* switches are optional.  They don't have any effect on
 debugging with GDB, but will compile additional code that might catch
@@ -184,20 +181,15 @@ Good luck!
 
 ** When you are trying to analyze failed assertions or backtraces, it
 is essential to compile Emacs with flags suitable for debugging.
-With GCC 4.8 or later, you can invoke 'make' with CFLAGS="-Og -g3".
-With older GCC or non-GCC compilers, you can use CFLAGS="-O0 -g3".
+With GCC 4.8 or later, you can invoke 'make' with CFLAGS="-O0 -g3".
+With older GCC, you can use CFLAGS="-O0 -g3 -gdwarf-4", replacing "4"
+by the highest version of DWARF that your compiler supports;
+with non-GCC compilers, "-O0 -g3" may be the best you can do.
 With GCC and higher optimization levels such as -O2, the
 -fno-omit-frame-pointer and -fno-crossjumping options are often
 essential.  The latter prevents GCC from using the same abort call for
 all assertions in a given function, rendering the stack backtrace
 useless for identifying the specific failed assertion.
-Some versions of GCC support recent versions of the DWARF standard for
-debugging info, but default to older versions; for example, they could
-support -gdwarf-4 compiler option (for DWARF v4), but default to
-version 2 of the DWARF standard.  For best results in debugging
-abilities, find out the highest version of DWARF your GCC can support,
-and use the corresponding -gdwarf-N switch instead of just -g (you
-will still need -g3, as in "-gdwarf-4 -g3").
 
 ** It is a good idea to run Emacs under GDB (or some other suitable
 debugger) *all the time*.  Then, when Emacs crashes, you will be able
@@ -923,41 +915,83 @@ setting the new-console option before running Emacs under GDB:
   (gdb) set new-console 1
   (gdb) run
 
-** Running Emacs built with malloc debugging packages
+** Running Emacs with undefined-behavior sanitization
 
-If Emacs exhibits bugs that seem to be related to use of memory
-allocated off the heap, it might be useful to link Emacs with a
-special debugging library, such as Electric Fence (a.k.a. efence) or
-GNU Checker, which helps find such problems.
+Building Emacs with undefined-behavior sanitization can help debug
+integer overflow and other undefined behavior in C code.  To use
+UndefinedBehaviorSanitizer with GCC and similar compilers, append
+'-fsanitize=undefined' to CFLAGS, either when running 'configure' or
+running 'make'.  For example:
 
-Emacs compiled with such packages might not run without some hacking,
-because Emacs replaces the system's memory allocation functions with
-its own versions, and because the dumping process might be
-incompatible with the way these packages use to track allocated
-memory.  Here are some of the changes you might find necessary:
+  ./configure CFLAGS='-O0 -g3 -fsanitize=undefined'
 
-  - Make sure unexec is disabled, e.g., './configure --without-unexec'.
+You may need to append '-static-libubsan' to CFLAGS if your version of
+GCC is installed in an unusual location.
 
-  - Configure with a different --prefix= option.  If you use GCC,
-    version 2.7.2 is preferred, as some malloc debugging packages
-    work a lot better with it than with 2.95 or later versions.
+When using GDB to debug an executable with undefined-behavior
+sanitization, the GDB command:
 
-  - Type "make" then "make -k install".
+  (gdb) rbreak ^__ubsan_handle_
 
-  - If required, invoke the package-specific command to prepare
-    src/temacs for execution.
+will let you gain control when an error is detected and before
+UndefinedBehaviorSanitizer outputs to stderr or terminates the
+program.
 
-  - cd ..; src/temacs
+** Running Emacs with address sanitization
 
-(Note that this runs 'temacs' instead of the usual 'emacs' executable.
-This avoids problems with dumping Emacs mentioned above.)
+Building Emacs with address sanitization can help debug memory-use
+problems.  To use AddressSanitizer with GCC and similar compilers,
+append '-fsanitize=address' to CFLAGS, either when running 'configure'
+or running 'make'.  Configure, build and run Emacs with
+ASAN_OPTIONS='detect_leaks=0' in the environment to suppress
+diagnostics of minor memory leaks in Emacs.  For example:
 
-Some malloc debugging libraries might print lots of false alarms for
-bitfields used by Emacs in some data structures.  If you want to get
-rid of the false alarms, you will have to hack the definitions of
-these data structures on the respective headers to remove the ':N'
-bitfield definitions (which will cause each such field to use a full
-int).
+  export ASAN_OPTIONS='detect_leaks=0'
+  ./configure CFLAGS='-O0 -g3 -fsanitize=address'
+  make
+  src/emacs
+
+You may need to append '-static-libasan' to CFLAGS if your version of
+GCC is installed in an unusual location.
+
+When using GDB to debug an executable with address sanitization, the
+GDB command:
+
+  (gdb) rbreak ^__asan_report_
+
+will let you gain control when an error is detected and before
+AddressSanitizer outputs to stderr or terminates the program.
+
+Address sanitization is incompatible with undefined-behavior
+sanitization, unfortunately.  Address sanitization is also
+incompatible with the --with-dumping=unexec option of 'configure'.
+
+** Running Emacs under Valgrind
+
+Valgrind <http://valgrind.org/> is free software that can be useful
+when debugging low-level Emacs problems.  Unlike GCC sanitizers,
+Valgrind does not need you to compile Emacs with special debugging
+flags, so it can be helpful in investigating problems that vanish when
+Emacs is recompiled with debugging enabled.  However, by default
+Valgrind generates many false alarms with Emacs, and you will need to
+maintain a suppressions file to suppress these false alarms and use
+Valgrind effectively.  For example, you might invoke Valgrind this
+way:
+
+   valgrind --suppressions=valgrind.supp ./emacs
+
+where valgrind.supp contains groups of lines like the following, which
+suppresses some Valgrind false alarms during Emacs garbage collection:
+
+   {
+     Fgarbage_collect Cond - conservative garbage collection
+     Memcheck:Cond
+     ...
+     fun:Fgarbage_collect
+   }
+
+Unfortunately Valgrind suppression files tend to be system-dependent,
+so you will need to keep one around that matches your system.
 
 ** How to recover buffer contents from an Emacs core dump file
 
diff --git a/etc/NEWS b/etc/NEWS
index 051063171e0..b444fc59677 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -93,7 +93,8 @@ change to one of the data structures that it relies on.
 ** The configure options '--enable-checking=conslist' and
 '--enable-checking=xmallocoverrun' have been withdrawn.  The former
 made Emacs irredeemably slow, and the latter made it crash.  Neither
-option was useful with modern debugging tools.
+option was useful with modern debugging tools such as AddressSanitizer
+(see etc/DEBUG).
 
 ---
 ** Emacs now requires GTK 2.24 and GTK 3.10 for the GTK 2 and GTK 3