Compute C decls for DEFSYMs automatically

Fixes Bug#15880.
This patch also makes Q constants (e.g., Qnil) constant addresses
from the C point of view.
* make-docfile.c: Revamp to generate table of symbols, too.
Include <stdbool.h>.
(xstrdup): New function.
(main): Don't process the same file twice.
(SYMBOL): New constant in enum global_type.
(struct symbol): Turn 'value' member into a union, either v.value
for int or v.svalue for string.  All uses changed.
(add_global): New arg svalue, which overrides value, so that globals
can have a string value.
(close_emacs_global): New arg num_symbols; all uses changed.
Output lispsym decl.
(write_globals): Output symbol globals too.  Output more
ATTRIBUTE_CONST, now that Qnil etc. are C constants.
Output defsym_name table.
(scan_c_file): Move most of guts into ...
(scan_c_stream): ... new function.  Scan for DEFSYMs and
record symbols found.  Don't read past EOF if file doesn't
end in newline.
* alloc.c, bidi.c, buffer.c, bytecode.c, callint.c, casefiddle:
* casetab.c, category.c, ccl.c, charset.c, chartab.c, cmds.c, coding.c:
* composite.c, data.c, dbusbind.c, decompress.c, dired.c, dispnew.c:
* doc.c, editfns.c, emacs.c, eval.c, fileio.c, fns.c, font.c, fontset.c:
* frame.c, fringe.c, ftfont.c, ftxfont.c, gfilenotify.c, gnutls.c:
* image.c, inotify.c, insdel.c, keyboard.c, keymap.c, lread.c:
* macfont.m, macros.c, minibuf.c, nsfns.m, nsfont.m, nsimage.m:
* nsmenu.m, nsselect.m, nsterm.m, print.c, process.c, profiler.c:
* search.c, sound.c, syntax.c, term.c, terminal.c, textprop.c, undo.c:
* window.c, xdisp.c, xfaces.c, xfns.c, xftfont.c, xmenu.c, xml.c:
* xselect.c, xsettings.c, xterm.c:
Remove Q vars that represent symbols (e.g., Qnil, Qt, Qemacs).
These names are now defined automatically by make-docfile.
* alloc.c (init_symbol): New function.
(Fmake_symbol): Use it.
(c_symbol_p): New function.
(valid_lisp_object_p, purecopy): Use it.
* alloc.c (marked_pinned_symbols):
Use make_lisp_symbol instead of make_lisp_ptr.
(garbage_collect_1): Mark lispsym symbols.
(CHECK_ALLOCATED_AND_LIVE_SYMBOL): New macro.
(mark_object): Use it.
(sweep_symbols): Sweep lispsym symbols.
(symbol_uses_obj): New function.
(which_symbols): Use it.  Work for lispsym symbols, too.
(init_alloc_once): Initialize Vpurify_flag here; no need to wait,
since Qt's address is already known now.
(syms_of_alloc): Add lispsym count to symbols_consed.
* buffer.c (init_buffer_once): Compare to Qnil, not to make_number (0),
when testing whether storage is all bits zero.
* dispextern (struct image_type):
* font.c (font_property_table):
* frame.c (struct frame_parm_table, frame_parms):
* keyboard.c (scroll_bar_parts, struct event_head):
* xdisp.c (struct props):
Use XSYMBOL_INIT (Qfoo) and struct Lisp_Symbol * rather than &Qfoo and
Lisp_Object *, since Qfoo is no longer an object whose address can be
taken.  All uses changed.
* eval.c (run_hook): New function.  Most uses of Frun_hooks changed to
use it, so that they no longer need to take the address of a Lisp sym.
(syms_of_eval): Don't use DEFSYM on Vrun_hooks, as it's a variable.
* frame.c (syms_of_frame): Add defsyms for the frame_parms table.
* keyboard.c (syms_of_keyboard): Don't DEFSYM Qmenu_bar here.
DEFSYM Qdeactivate_mark before the corresponding var.
* keymap.c (syms_of_keymap): Use DEFSYM for Qmenu_bar and Qmode_line
instead of interning their symbols; this avoids duplicates.
(LISP_INITIALLY, TAG_PTR)
(DEFINE_LISP_SYMBOL_BEGIN, DEFINE_LISP_SYMBOL_END, XSYMBOL_INIT):
New macros.
(LISP_INITIALLY_ZERO): Use it.
(enum symbol_interned, enum symbol_redirect, struct Lisp_Symbol)
(EXFUN, DEFUN_ARGS_MANY, DEFUN_ARGS_UNEVALLED, DEFUN_ARGS_*):
Move decls up, to avoid forward uses.  Include globals.h earlier, too.
(make_lisp_symbol): New function.
(XSETSYMBOL): Use it.
(DEFSYM): Now just a placeholder for make-docfile.
* lread.c (DEFINE_SYMBOLS): Define, for globals.h.
(intern_sym): New function, with body taken from old intern_driver.
(intern_driver): Use it.  Last arg is now Lisp integer, not ptrdiff_t.
All uses changed.
(define_symbol): New function.
(init_obarray): Define the C symbols taken from lispsym.
Use plain DEFSYM for Qt and Qnil.
* syntax.c (init_syntax_once): No need to worry about
Qchar_table_extra_slots.

2 files changed, 226 insertions, 76 deletions

diff --git a/lib-src/ChangeLog b/lib-src/ChangeLog
index acbbd3a02df..8bdf7d1fb16 100644
--- a/lib-src/ChangeLog
+++ b/lib-src/ChangeLog
@@ -1,3 +1,26 @@
+2015-01-05  Paul Eggert  <eggert@cs.ucla.edu>
+
+        Compute C decls for DEFSYMs automatically
+        Fixes Bug#15880.
+        * make-docfile.c: Revamp to generate table of symbols, too.
+        Include <stdbool.h>.
+        (xstrdup): New function.
+        (main): Don't process the same file twice.
+        (SYMBOL): New constant in enum global_type.
+        (struct symbol): Turn 'value' member into a union, either v.value
+        for int or v.svalue for string.  All uses changed.
+        (add_global): New arg svalue, which overrides value, so that globals
+        can have a string value.
+        (close_emacs_global): New arg num_symbols; all uses changed.
+        Output lispsym decl.
+        (write_globals): Output symbol globals too.  Output more
+        ATTRIBUTE_CONST, now that Qnil etc. are C constants.
+        Output defsym_name table.
+        (scan_c_file): Move most of guts into ...
+        (scan_c_stream): ... new function.  Scan for DEFSYMs and
+        record symbols found.  Don't read past EOF if file doesn't
+        end in newline.
+
 2015-01-04  Paul Eggert  <eggert@cs.ucla.edu>
 
         'temacs -nw' should not call missing functions
diff --git a/lib-src/make-docfile.c b/lib-src/make-docfile.c
index f74b3d516d1..b05a634eb75 100644
--- a/lib-src/make-docfile.c
+++ b/lib-src/make-docfile.c
@@ -36,6 +36,7 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
 
 #include <config.h>
 
+#include <stdbool.h>
 #include <stdio.h>
 #include <stdlib.h>   /* config.h unconditionally includes this anyway */
 
@@ -63,6 +64,7 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
 static int scan_file (char *filename);
 static int scan_lisp_file (const char *filename, const char *mode);
 static int scan_c_file (char *filename, const char *mode);
+static int scan_c_stream (FILE *infile);
 static void start_globals (void);
 static void write_globals (void);
 
@@ -106,6 +108,17 @@ xmalloc (unsigned int size)
   return result;
 }
 
+/* Like strdup, but get fatal error if memory is exhausted.  */
+
+static char *
+xstrdup (char *s)
+{
+  char *result = strdup (s);
+  if (! result)
+    fatal ("virtual memory exhausted", 0);
+  return result;
+}
+
 /* Like realloc but get fatal error if memory is exhausted.  */
 
 static void *
@@ -123,7 +136,6 @@ main (int argc, char **argv)
 {
   int i;
   int err_count = 0;
-  int first_infile;
 
   progname = argv[0];
 
@@ -167,16 +179,21 @@ main (int argc, char **argv)
   if (generate_globals)
     start_globals ();
 
-  first_infile = i;
-  for (; i < argc; i++)
+  if (argc <= i)
+    scan_c_stream (stdin);
+  else
     {
-      int j;
-      /* Don't process one file twice.  */
-      for (j = first_infile; j < i; j++)
-        if (! strcmp (argv[i], argv[j]))
-          break;
-      if (j == i)
-        err_count += scan_file (argv[i]);
+      int first_infile = i;
+      for (; i < argc; i++)
+        {
+          int j;
+          /* Don't process one file twice.  */
+          for (j = first_infile; j < i; j++)
+            if (strcmp (argv[i], argv[j]) == 0)
+              break;
+          if (j == i)
+            err_count += scan_file (argv[i]);
+        }
     }
 
   if (err_count == 0 && generate_globals)
@@ -528,13 +545,15 @@ write_c_args (char *func, char *buf, int minargs, int maxargs)
 }
 
 /* The types of globals.  These are sorted roughly in decreasing alignment
-   order to avoid allocation gaps, except that functions are last.  */
+   order to avoid allocation gaps, except that symbols and functions
+   are last.  */
 enum global_type
 {
   INVALID,
   LISP_OBJECT,
   EMACS_INTEGER,
   BOOLEAN,
+  SYMBOL,
   FUNCTION
 };
 
@@ -543,7 +562,11 @@ struct global
 {
   enum global_type type;
   char *name;
-  int value;
+  union
+  {
+    int value;
+    char const *svalue;
+  } v;
 };
 
 /* All the variable names we saw while scanning C sources in `-g'
@@ -553,7 +576,7 @@ int num_globals_allocated;
 struct global *globals;
 
 static void
-add_global (enum global_type type, char *name, int value)
+add_global (enum global_type type, char *name, int value, char const *svalue)
 {
   /* Ignore the one non-symbol that can occur.  */
   if (strcmp (name, "..."))
@@ -574,7 +597,10 @@ add_global (enum global_type type, char *name, int value)
 
       globals[num_globals - 1].type = type;
       globals[num_globals - 1].name = name;
-      globals[num_globals - 1].value = value;
+      if (svalue)
+        globals[num_globals - 1].v.svalue = svalue;
+      else
+        globals[num_globals - 1].v.value = value;
     }
 }
 
@@ -591,17 +617,44 @@ compare_globals (const void *a, const void *b)
 }
 
 static void
-close_emacs_globals (void)
+close_emacs_globals (int num_symbols)
 {
-  puts ("};");
-  puts ("extern struct emacs_globals globals;");
+  printf (("};\n"
+           "extern struct emacs_globals globals;\n"
+           "\n"
+           "#ifndef DEFINE_SYMBOLS\n"
+           "extern\n"
+           "#endif\n"
+           "struct Lisp_Symbol lispsym[%d];\n"),
+          num_symbols);
 }
 
 static void
 write_globals (void)
 {
-  int i, seen_defun = 0;
+  int i, j;
+  bool seen_defun = false;
+  int symnum = 0;
+  int num_symbols = 0;
   qsort (globals, num_globals, sizeof (struct global), compare_globals);
+
+  j = 0;
+  for (i = 0; i < num_globals; i++)
+    {
+      while (i + 1 < num_globals
+             && strcmp (globals[i].name, globals[i + 1].name) == 0)
+        {
+          if (globals[i].type == FUNCTION
+              && globals[i].v.value != globals[i + 1].v.value)
+            error ("function '%s' defined twice with differing signatures",
+                   globals[i].name);
+          i++;
+        }
+      num_symbols += globals[i].type == SYMBOL;
+      globals[j++] = globals[i];
+    }
+  num_globals = j;
+
   for (i = 0; i < num_globals; ++i)
     {
       char const *type = 0;
@@ -617,12 +670,13 @@ write_globals (void)
         case LISP_OBJECT:
           type = "Lisp_Object";
           break;
+        case SYMBOL:
         case FUNCTION:
           if (!seen_defun)
             {
-              close_emacs_globals ();
+              close_emacs_globals (num_symbols);
               putchar ('\n');
-              seen_defun = 1;
+              seen_defun = true;
             }
           break;
         default:
@@ -635,6 +689,13 @@ write_globals (void)
           printf ("#define %s globals.f_%s\n",
                   globals[i].name, globals[i].name);
         }
+      else if (globals[i].type == SYMBOL)
+        printf (("DEFINE_LISP_SYMBOL_BEGIN (%s)\n"
+                 "#define a%s (&lispsym[%d])\n"
+                 "#define %s make_lisp_symbol (a%s)\n"
+                 "DEFINE_LISP_SYMBOL_END (a%s)\n\n"),
+                globals[i].name, globals[i].name, symnum++,
+                globals[i].name, globals[i].name, globals[i].name);
       else
         {
           /* It would be nice to have a cleaner way to deal with these
@@ -647,39 +708,65 @@ write_globals (void)
             fputs ("_Noreturn ", stdout);
 
           printf ("EXFUN (%s, ", globals[i].name);
-          if (globals[i].value == -1)
+          if (globals[i].v.value == -1)
             fputs ("MANY", stdout);
-          else if (globals[i].value == -2)
+          else if (globals[i].v.value == -2)
             fputs ("UNEVALLED", stdout);
           else
-            printf ("%d", globals[i].value);
+            printf ("%d", globals[i].v.value);
           putchar (')');
 
           /* It would be nice to have a cleaner way to deal with these
              special hacks, too.  */
-          if (strcmp (globals[i].name, "Fbyteorder") == 0
+          if (strcmp (globals[i].name, "Fatom") == 0
+              || strcmp (globals[i].name, "Fbyteorder") == 0
+              || strcmp (globals[i].name, "Fcharacterp") == 0
+              || strcmp (globals[i].name, "Fchar_or_string_p") == 0
+              || strcmp (globals[i].name, "Fconsp") == 0
+              || strcmp (globals[i].name, "Feq") == 0
+              || strcmp (globals[i].name, "Fface_attribute_relative_p") == 0
               || strcmp (globals[i].name, "Fframe_windows_min_size") == 0
+              || strcmp (globals[i].name, "Fgnutls_errorp") == 0
               || strcmp (globals[i].name, "Fidentity") == 0
+              || strcmp (globals[i].name, "Fintegerp") == 0
+              || strcmp (globals[i].name, "Finteractive") == 0
+              || strcmp (globals[i].name, "Ffloatp") == 0
+              || strcmp (globals[i].name, "Flistp") == 0
               || strcmp (globals[i].name, "Fmax_char") == 0
-              || strcmp (globals[i].name, "Ftool_bar_height") == 0)
+              || strcmp (globals[i].name, "Fnatnump") == 0
+              || strcmp (globals[i].name, "Fnlistp") == 0
+              || strcmp (globals[i].name, "Fnull") == 0
+              || strcmp (globals[i].name, "Fnumberp") == 0
+              || strcmp (globals[i].name, "Fstringp") == 0
+              || strcmp (globals[i].name, "Fsymbolp") == 0
+              || strcmp (globals[i].name, "Ftool_bar_height") == 0
+              || strcmp (globals[i].name, "Fwindow__sanitize_window_sizes") == 0
+#ifndef WINDOWSNT
+              || strcmp (globals[i].name, "Fgnutls_available_p") == 0
+              || strcmp (globals[i].name, "Fzlib_available_p") == 0
+#endif
+              || 0)
             fputs (" ATTRIBUTE_CONST", stdout);
 
           puts (";");
         }
-
-      while (i + 1 < num_globals
-             && !strcmp (globals[i].name, globals[i + 1].name))
-        {
-          if (globals[i].type == FUNCTION
-              && globals[i].value != globals[i + 1].value)
-            error ("function '%s' defined twice with differing signatures",
-                   globals[i].name);
-          ++i;
-        }
     }
 
   if (!seen_defun)
-    close_emacs_globals ();
+    close_emacs_globals (num_symbols);
+
+  puts ("#ifdef DEFINE_SYMBOLS");
+  puts ("static char const *const defsym_name[] = {");
+  for (int i = 0; i < num_globals; i++)
+    {
+      if (globals[i].type == SYMBOL)
+        printf ("\t\"%s\",\n", globals[i].v.svalue);
+      while (i + 1 < num_globals
+             && strcmp (globals[i].name, globals[i + 1].name) == 0)
+        i++;
+    }
+  puts ("};");
+  puts ("#endif");
 }
 
 
@@ -692,9 +779,6 @@ static int
 scan_c_file (char *filename, const char *mode)
 {
   FILE *infile;
-  register int c;
-  register int commas;
-  int minargs, maxargs;
   int extension = filename[strlen (filename) - 1];
 
   if (extension == 'o')
@@ -720,8 +804,15 @@ scan_c_file (char *filename, const char *mode)
 
   /* Reset extension to be able to detect duplicate files.  */
   filename[strlen (filename) - 1] = extension;
+  return scan_c_stream (infile);
+}
+
+static int
+scan_c_stream (FILE *infile)
+{
+  int commas, minargs, maxargs;
+  int c = '\n';
 
-  c = '\n';
   while (!feof (infile))
     {
       int doc_keyword = 0;
@@ -750,37 +841,53 @@ scan_c_file (char *filename, const char *mode)
           if (c != 'F')
             continue;
           c = getc (infile);
-          if (c != 'V')
-            continue;
-          c = getc (infile);
-          if (c != 'A')
-            continue;
-          c = getc (infile);
-          if (c != 'R')
-            continue;
-          c = getc (infile);
-          if (c != '_')
-            continue;
-
-          defvarflag = 1;
-
-          c = getc (infile);
-          defvarperbufferflag = (c == 'P');
-          if (generate_globals)
+          if (c == 'S')
             {
-              if (c == 'I')
-                type = EMACS_INTEGER;
-              else if (c == 'L')
-                type = LISP_OBJECT;
-              else if (c == 'B')
-                type = BOOLEAN;
+              c = getc (infile);
+              if (c != 'Y')
+                continue;
+              c = getc (infile);
+              if (c != 'M')
+                continue;
+              c = getc (infile);
+              if (c != ' ' && c != '\t' && c != '(')
+                continue;
+              type = SYMBOL;
             }
+          else if (c == 'V')
+            {
+              c = getc (infile);
+              if (c != 'A')
+                continue;
+              c = getc (infile);
+              if (c != 'R')
+                continue;
+              c = getc (infile);
+              if (c != '_')
+                continue;
 
-          c = getc (infile);
-          /* We need to distinguish between DEFVAR_BOOL and
-             DEFVAR_BUFFER_DEFAULTS.  */
-          if (generate_globals && type == BOOLEAN && c != 'O')
-            type = INVALID;
+              defvarflag = 1;
+
+              c = getc (infile);
+              defvarperbufferflag = (c == 'P');
+              if (generate_globals)
+                {
+                  if (c == 'I')
+                    type = EMACS_INTEGER;
+                  else if (c == 'L')
+                    type = LISP_OBJECT;
+                  else if (c == 'B')
+                    type = BOOLEAN;
+                }
+
+              c = getc (infile);
+              /* We need to distinguish between DEFVAR_BOOL and
+                 DEFVAR_BUFFER_DEFAULTS.  */
+              if (generate_globals && type == BOOLEAN && c != 'O')
+                type = INVALID;
+            }
+          else
+            continue;
         }
       else if (c == 'D')
         {
@@ -797,7 +904,7 @@ scan_c_file (char *filename, const char *mode)
 
       if (generate_globals
           && (!defvarflag || defvarperbufferflag || type == INVALID)
-          && !defunflag)
+          && !defunflag && type != SYMBOL)
         continue;
 
       while (c != '(')
@@ -807,15 +914,19 @@ scan_c_file (char *filename, const char *mode)
           c = getc (infile);
         }
 
-      /* Lisp variable or function name.  */
-      c = getc (infile);
-      if (c != '"')
-        continue;
-      c = read_c_string_or_comment (infile, -1, 0, 0);
+      if (type != SYMBOL)
+        {
+          /* Lisp variable or function name.  */
+          c = getc (infile);
+          if (c != '"')
+            continue;
+          c = read_c_string_or_comment (infile, -1, 0, 0);
+        }
 
       if (generate_globals)
         {
           int i = 0;
+          char const *svalue = 0;
 
           /* Skip "," and whitespace.  */
           do
@@ -827,6 +938,8 @@ scan_c_file (char *filename, const char *mode)
           /* Read in the identifier.  */
           do
             {
+              if (c < 0)
+                goto eof;
               input_buffer[i++] = c;
               c = getc (infile);
             }
@@ -837,13 +950,27 @@ scan_c_file (char *filename, const char *mode)
           name = xmalloc (i + 1);
           memcpy (name, input_buffer, i + 1);
 
+          if (type == SYMBOL)
+            {
+              do
+                c = getc (infile);
+              while (c == ' ' || c == '\t' || c == '\n' || c == '\r');
+              if (c != '"')
+                continue;
+              c = read_c_string_or_comment (infile, -1, 0, 0);
+              svalue = xstrdup (input_buffer);
+            }
+
           if (!defunflag)
             {
-              add_global (type, name, 0);
+              add_global (type, name, 0, svalue);
               continue;
             }
         }
 
+      if (type == SYMBOL)
+        continue;
+
       /* DEFVAR_LISP ("name", addr, "doc")
          DEFVAR_LISP ("name", addr /\* doc *\/)
          DEFVAR_LISP ("name", addr, doc: /\* doc *\/)  */
@@ -896,7 +1023,7 @@ scan_c_file (char *filename, const char *mode)
 
       if (generate_globals)
         {
-          add_global (FUNCTION, name, maxargs);
+          add_global (FUNCTION, name, maxargs, 0);
           continue;
         }