1 files changed, 94 insertions, 36 deletions

diff --git a/admin/notes/unicode b/admin/notes/unicode
index bc7279150a9..014bfb9b0d5 100644
--- a/admin/notes/unicode
+++ b/admin/notes/unicode
@@ -1,6 +1,6 @@
                                             -*-mode: text; coding: utf-8;-*-
 
-Copyright (C) 2002-2017 Free Software Foundation, Inc.
+Copyright (C) 2002-2022 Free Software Foundation, Inc.
 See the end of the file for license conditions.
 
 Importing a new Unicode Standard version into Emacs
@@ -11,15 +11,38 @@ Emacs uses the following files from the Unicode Character Database
 
   . UnicodeData.txt
   . Blocks.txt
-  . BidiMirroring.txt
   . BidiBrackets.txt
+  . BidiMirroring.txt
   . IVD_Sequences.txt
   . NormalizationTest.txt
+  . PropertyValueAliases.txt
+  . ScriptExtensions.txt
+  . Scripts.txt
   . SpecialCasing.txt
+  . confusables.txt
+  . emoji-data.txt
+  . emoji-zwj-sequences.txt
+  . emoji-sequences.txt
   . BidiCharacterTest.txt
 
-First, the first 7 files need to be copied into admin/unidata/, and
-then Emacs should be rebuilt for them to take effect.  Rebuilding
+Emacs also uses the file emoji-test.txt which should be imported from
+the Unicode's Public/emoji/ directory, and IdnaMappingTable.txt from
+the Public/idna/ directory.
+
+First, the first 14 files, emoji-test.txt and IdnaMappingTable.txt
+need to be copied into admin/unidata/, and the file
+https://www.unicode.org/copyright.html should be copied over
+copyright.html in admin/unidata (some of them might need trailing
+whitespace removed before they can be committed to the Emacs
+repository).
+
+Next, review the assignment of default values of the Bidi Class
+property to blocks in the file extracted/DerivedBidiClass.txt from the
+UCD (search for "unassigned" in that file).  Any changes should be
+reflected in the unidata-gen.el file, where it sets up the default
+values around line 210.
+
+Then Emacs should be rebuilt for them to take effect.  Rebuilding
 Emacs updates several derived files elsewhere in the Emacs source
 tree, mainly in lisp/international/.
 
@@ -28,7 +51,10 @@ files, pay attention to any warning or error messages.  In particular,
 admin/unidata/unidata-gen.el will complain if UnicodeData.txt defines
 new bidirectional attributes of characters, because unidata-gen.el,
 bidi.c and dispextern.h need to be updated in that case; failure to do
-so will cause aborts in redisplay.
+so will cause aborts in redisplay.  unidata-gen.el will also complain
+if the format of the Unicode Copyright notice in copyright.html
+changed in significant ways; in that case, update the regular
+expression in unidata-gen-file used to extract the copyright string.
 
 Next, review the changes in UnicodeData.txt vs the previous version
 used by Emacs.  Any changes, be it introduction of new scripts or
@@ -40,15 +66,23 @@ and see if any changes in admin/unidata/blocks.awk are required.
 
 The setting of char-width-table around line 1200 of characters.el
 should be checked against the latest version of the Unicode file
-EastAsianWidth.txt, and any discrepancies fixed.
+EastAsianWidth.txt, and any discrepancies fixed: double-width
+characters are those marked with W or F in that file.  Zero-width
+characters are not taken from EastAsianWidth.txt, they are those whose
+Unicode General Category property is one of Mn, Me, or Cf, and also
+Hangul jungseong and jongseong characters (a.k.a. "Jamo medial vowels"
+and "Jamo final consonants").
 
 Any new scripts added by UnicodeData.txt will also need updates to
 script-representative-chars defined in fontset.el, and also the list
 of OTF script tags in otf-script-alist, whose source is on this page:
 
-  https://www.microsoft.com/typography/otspec/scripttags.htm
+  https://docs.microsoft.com/en-us/typography/opentype/spec/scripttags
 
 Other databases in fontset.el might also need to be updated as needed.
+One notable place to check is the function setup-default-fontset,
+where new scripts will generally need some addition, most probably to
+the list of "simple" scripts (search for "Simple").
 
 The function 'ucs-names', defined in lisp/international/mule-cmds.el,
 might need to be updated because it knows about used and unused ranges
@@ -65,7 +99,51 @@ regarding failing lines.
 
 The file BidiCharacterTest.txt should be copied to the test suite, and
 if its format has changed, the file biditest.el there should be
-modified to follow suit.
+modified to follow suit.  If there's trailing whitespace in
+BidiCharacterTest.txt, it should be removed before committing the new
+version.
+
+src/macuvs.h is a generated file, but if it has changed as a result
+of the updates, please commit it as well (see
+admin/unidata/Makefile.in for an explanation).
+
+Visit "emoji-data.txt" with the rebuilt Emacs, and check that an
+appropriate font is being used for the emoji (by default Emacs uses
+"Noto Color Emoji").  Running the following command in that buffer
+will give you an idea of which codepoints are not supported by
+whichever font Emacs is using.
+
+(defun check-emoji-coverage (font-name-regexp)
+"Display a buffer containing emoji codepoints for which FONT-NAME is not used.
+This must be run from a buffer in the format of emoji-data.txt.
+FONT-NAME-REGEXP is checked using `string-match'."
+(interactive "MFont Name: ")
+(save-excursion
+(goto-char (point-min))
+(let (res char name ifont)
+  (while (re-search-forward "; Emoji_Presentation [^(]+(\\(.\\)[).]" nil t)
+    (setq char (aref (match-string 1) 0))
+    (setq ifont (car (internal-char-font nil char)))
+    (when ifont
+      (setq name (font-xlfd-name ifont)))
+    (if (or (not ifont) (not (string-match font-name-regexp name)))
+        (setq res (concat (string char) res))))
+  (when res
+    (with-output-to-temp-buffer "*Check-Emoji-Coverage*"
+      (princ (format "Font not matching '%s' was used for the following characters:\n%s"
+                     font-name-regexp (reverse res))))))))
+
+Visit "emoji-zwj-sequences.txt" and "emoji-sequences.txt" with the
+rebuilt Emacs, and check that the sample sequences are composed
+properly.  Also check the Unicode style chart file available at
+https://unicode.org/emoji/charts/emoji-style.txt for any issues
+involving VS-15 and VS-16, if so you may need to update the value
+generated for auto-composition-emoji-eligible-codepoints by
+admin/unidata/emoji-zwj.awk.  Note that your emoji font might not have
+glyphs for the newest codepoints yet.
+
+Finally, etc/NEWS should be updated to announce the support for the
+new Unicode version.
 
 Problems, fixmes and other unicode-related issues
 -------------------------------------------------------------
@@ -85,7 +163,7 @@ regard to completeness.
         code (keymap.c and print.c).
 
  * Rationalize character syntax and its relationship to the Unicode
-   database.  (Applies mainly to symbol an punctuation syntax.)
+   database.  (Applies mainly to symbol and punctuation syntax.)
 
  * Fontset handling and customization needs work.  We want to relate
    fonts to scripts, probably based on the Unicode blocks.  The
@@ -230,36 +308,15 @@ nontrivial changes to the build process.
 
         admin/charsets/mapfiles/cns2ucsdkw.txt
 
- * iso-2022-7bit
-
-     This file switches between CJK charsets, which is not encoded in UTF-8.
+ * iso-2022-jp
 
-        etc/HELLO
-
-     Each of these files contains just one CJK charset, but Emacs
-     currently has no easy way to specify set-charset-priority on a
-     per-file basis, so converting any of these files to UTF-8 might
-     change the file's appearance when viewed by an Emacs that is
-     operating in some other language environment.
+     This contains just one CJK charset, but Emacs currently has no
+     easy way to specify set-charset-priority on a per-file basis, so
+     converting this file to UTF-8 might change the file's appearance
+     when viewed by an Emacs that is operating in some other language
+     environment.
 
         etc/tutorials/TUTORIAL.ja
-        lisp/international/ja-dic-cnv.el
-        lisp/international/ja-dic-utl.el
-        lisp/international/kinsoku.el
-        lisp/international/kkc.el
-        lisp/international/titdic-cnv.el
-        lisp/language/japan-util.el
-        lisp/language/japanese.el
-        lisp/leim/quail/cyril-jis.el
-        lisp/leim/quail/hanja-jis.el
-        lisp/leim/quail/japanese.el
-        lisp/leim/quail/py-punct.el
-        lisp/leim/quail/pypunct-b5.el
-
-     This file contains just Chinese characters, and has same problem.
-     Also, it contains characters that cannot be encoded in UTF-8.
-
-        lisp/international/titdic-cnv.el
 
  * utf-8-emacs
 
@@ -272,6 +329,7 @@ nontrivial changes to the build process.
         lisp/language/tibetan.el
         lisp/leim/quail/ethiopic.el
         lisp/leim/quail/tibetan.el
+        lisp/international/titdic-cnv.el
 
  * binary files