aboutsummaryrefslogtreecommitdiffstats
path: root/src/coding.c
diff options
context:
space:
mode:
Diffstat (limited to 'src/coding.c')
-rw-r--r--src/coding.c10391
1 files changed, 5576 insertions, 4815 deletions
diff --git a/src/coding.c b/src/coding.c
index 37a5e886547..a5104c5909c 100644
--- a/src/coding.c
+++ b/src/coding.c
@@ -2,6 +2,9 @@
2 Copyright (C) 1995, 1997, 1998 Electrotechnical Laboratory, JAPAN. 2 Copyright (C) 1995, 1997, 1998 Electrotechnical Laboratory, JAPAN.
3 Licensed to the Free Software Foundation. 3 Licensed to the Free Software Foundation.
4 Copyright (C) 2001 Free Software Foundation, Inc. 4 Copyright (C) 2001 Free Software Foundation, Inc.
5 Copyright (C) 2001, 2002
6 National Institute of Advanced Industrial Science and Technology (AIST)
7 Registration Number H13PRO009
5 8
6This file is part of GNU Emacs. 9This file is part of GNU Emacs.
7 10
@@ -24,340 +27,292 @@ Boston, MA 02111-1307, USA. */
24 27
25 0. General comments 28 0. General comments
26 1. Preamble 29 1. Preamble
27 2. Emacs' internal format (emacs-mule) handlers 30 2. Emacs' internal format (emacs-utf-8) handlers
28 3. ISO2022 handlers 31 3. UTF-8 handlers
29 4. Shift-JIS and BIG5 handlers 32 4. UTF-16 handlers
30 5. CCL handlers 33 5. Charset-base coding systems handlers
31 6. End-of-line handlers 34 6. emacs-mule (old Emacs' internal format) handlers
32 7. C library functions 35 7. ISO2022 handlers
33 8. Emacs Lisp library functions 36 8. Shift-JIS and BIG5 handlers
34 9. Post-amble 37 9. CCL handlers
38 10. C library functions
39 11. Emacs Lisp library functions
40 12. Postamble
35 41
36*/ 42*/
37 43
38/*** 0. General comments ***/ 44/*** 0. General comments ***
39 45
40 46
41/*** GENERAL NOTE on CODING SYSTEMS *** 47CODING SYSTEM
42 48
43 A coding system is an encoding mechanism for one or more character 49 Coding system is an encoding mechanism of one or more character
44 sets. Here's a list of coding systems which Emacs can handle. When 50 sets. Here's a list of coding system types supported by Emacs.
45 we say "decode", it means converting some other coding system to 51 When we say "decode", it means converting a text encoded by some
46 Emacs' internal format (emacs-mule), and when we say "encode", 52 coding system into Emacs' internal format (emacs-utf-8), and when we
47 it means converting the coding system emacs-mule to some other 53 say "encode", it means converting a text of emacs-utf-8 to some
48 coding system. 54 other coding system.
49 55
50 0. Emacs' internal format (emacs-mule) 56 Emacs represents a coding system by a Lisp symbol. Each symbol is a
57 key to the hash table Vcharset_hash_table. This hash table
58 associates the symbol to the corresponding detailed specifications.
51 59
52 Emacs itself holds a multi-lingual character in buffers and strings 60 Before using a coding system for decoding and encoding, we setup a
53 in a special format. Details are described in section 2. 61 structure of type `struct coding_system'. This structure keeps
62 various information about a specific code conversion (e.g. the
63 location of source and destination data).
54 64
55 1. ISO2022 65 Coding systems are classified into the following types by how to
66 represent a character in a byte sequence. Here's a brief descrition
67 about type.
56 68
57 The most famous coding system for multiple character sets. X's 69 o Emacs' internal format (emacs-utf-8)
58 Compound Text, various EUCs (Extended Unix Code), and coding 70
59 systems used in Internet communication such as ISO-2022-JP are 71 The extended UTF-8 which allows eight-bit raw bytes mixed with
60 all variants of ISO2022. Details are described in section 3. 72 character codes. Emacs holds characters in buffers and strings by
73 this format.
74
75 o UTF-8
76
77 o UTF-16
78
79 o Charset-base coding system
80
81 A coding system defined by one or more (coded) character sets.
82 Decoding and encoding are done by code converter defined for each
83 character set.
61 84
62 2. SJIS (or Shift-JIS or MS-Kanji-Code) 85 o Old Emacs' internal format (emacs-mule)
63 86
87 The coding system adopted by an old versions of Emacs (20 and 21).
88
89 o ISO2022-base coding system
90
91 The most famous coding system for multiple character sets. X's
92 Compound Text, various EUCs (Extended Unix Code), and coding systems
93 used in the Internet communication such as ISO-2022-JP are all
94 variants of ISO2022.
95
96 o SJIS (or Shift-JIS or MS-Kanji-Code)
97
64 A coding system to encode character sets: ASCII, JISX0201, and 98 A coding system to encode character sets: ASCII, JISX0201, and
65 JISX0208. Widely used for PC's in Japan. Details are described in 99 JISX0208. Widely used for PC's in Japan. Details are described in
66 section 4. 100 section 8.
67 101
68 3. BIG5 102 o BIG5
69 103
70 A coding system to encode the character sets ASCII and Big5. Widely 104 A coding system to encode character sets: ASCII and Big5. Widely
71 used for Chinese (mainly in Taiwan and Hong Kong). Details are 105 used by Chinese (mainly in Taiwan and Hong Kong). Details are
72 described in section 4. In this file, when we write "BIG5" 106 described in section 8. In this file, when we write "big5" (all
73 (all uppercase), we mean the coding system, and when we write 107 lowercase), we mean the coding system, and when we write "Big5"
74 "Big5" (capitalized), we mean the character set. 108 (capitalized), we mean the character set.
75 109
76 4. Raw text 110 o CCL
77 111
78 A coding system for text containing random 8-bit code. Emacs does 112 If a user wants to decode/encode a text encoded in a coding system
79 no code conversion on such text except for end-of-line format. 113 not listed above, he can supply a decoder and an encoder for it in
114 CCL (Code Conversion Language) programs. Emacs executes the CCL
115 program while decoding/encoding.
80 116
81 5. Other 117 o Raw-text
82 118
83 If a user wants to read/write text encoded in a coding system not 119 A coding system for a text containing raw eight-bit data. Emacs
84 listed above, he can supply a decoder and an encoder for it as CCL 120 treat each byte of source text as a character (except for
85 (Code Conversion Language) programs. Emacs executes the CCL program 121 end-of-line conversion).
86 while reading/writing.
87 122
88 Emacs represents a coding system by a Lisp symbol that has a property 123 o No-conversion
89 `coding-system'. But, before actually using the coding system, the
90 information about it is set in a structure of type `struct
91 coding_system' for rapid processing. See section 6 for more details.
92 124
93*/ 125 Like raw text, but don't do end-of-line conversion.
94 126
95/*** GENERAL NOTES on END-OF-LINE FORMAT ***
96 127
97 How end-of-line of text is encoded depends on the operating system. 128END-OF-LINE FORMAT
98 For instance, Unix's format is just one byte of `line-feed' code, 129
130 How end-of-line of a text is encoded depends on a system. For
131 instance, Unix's format is just one byte of LF (line-feed) code,
99 whereas DOS's format is two-byte sequence of `carriage-return' and 132 whereas DOS's format is two-byte sequence of `carriage-return' and
100 `line-feed' codes. MacOS's format is usually one byte of 133 `line-feed' codes. MacOS's format is usually one byte of
101 `carriage-return'. 134 `carriage-return'.
102 135
103 Since text character encoding and end-of-line encoding are 136 Since text characters encoding and end-of-line encoding are
104 independent, any coding system described above can have any 137 independent, any coding system described above can take any format
105 end-of-line format. So Emacs has information about end-of-line 138 of end-of-line (except for no-conversion).
106 format in each coding-system. See section 6 for more details.
107 139
108*/ 140*/
109 141
142/* COMMON MACROS */
143
144
110/*** GENERAL NOTES on `detect_coding_XXX ()' functions *** 145/*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
111 146
112 These functions check if a text between SRC and SRC_END is encoded 147 These functions check if a byte sequence specified as a source in
113 in the coding system category XXX. Each returns an integer value in 148 CODING conforms to the format of XXX. Return 1 if the data contains
114 which appropriate flag bits for the category XXX are set. The flag 149 a byte sequence which can be decoded into non-ASCII characters by
115 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the 150 the coding system. Otherwize (i.e. the data contains only ASCII
116 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes 151 characters or invalid sequence) return 0.
117 of the range 0x80..0x9F are in multibyte form. */ 152
153 It also resets some bits of an integer pointed by MASK. The macros
154 CATEGORY_MASK_XXX specifies each bit of this integer.
155
156 Below is the template of these functions. */
157
118#if 0 158#if 0
119int 159static int
120detect_coding_emacs_mule (src, src_end, multibytep) 160detect_coding_XXX (coding, mask)
121 unsigned char *src, *src_end; 161 struct coding_system *coding;
122 int multibytep; 162 int *mask;
123{ 163{
124 ... 164 unsigned char *src = coding->source;
165 unsigned char *src_end = coding->source + coding->src_bytes;
166 int multibytep = coding->src_multibyte;
167 int c;
168 int found = 0;
169 ...;
170
171 while (1)
172 {
173 /* Get one byte from the source. If the souce is exausted, jump
174 to no_more_source:. */
175 ONE_MORE_BYTE (c);
176 /* Check if it conforms to XXX. If not, break the loop. */
177 }
178 /* As the data is invalid for XXX, reset a proper bits. */
179 *mask &= ~CODING_CATEGORY_XXX;
180 return 0;
181 no_more_source:
182 /* The source exausted. */
183 if (!found)
184 /* ASCII characters only. */
185 return 0;
186 /* Some data should be decoded into non-ASCII characters. */
187 *mask &= CODING_CATEGORY_XXX;
188 return 1;
125} 189}
126#endif 190#endif
127 191
128/*** GENERAL NOTES on `decode_coding_XXX ()' functions *** 192/*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
129 193
130 These functions decode SRC_BYTES length of unibyte text at SOURCE 194 These functions decode a byte sequence specified as a source by
131 encoded in CODING to Emacs' internal format. The resulting 195 CODING. The resulting multibyte text goes to a place pointed to by
132 multibyte text goes to a place pointed to by DESTINATION, the length 196 CODING->charbuf, the length of which should not exceed
133 of which should not exceed DST_BYTES. 197 CODING->charbuf_size;
134 198
135 These functions set the information about original and decoded texts 199 These functions set the information of original and decoded texts in
136 in the members `produced', `produced_char', `consumed', and 200 CODING->consumed, CODING->consumed_char, and CODING->charbuf_used.
137 `consumed_char' of the structure *CODING. They also set the member 201 They also set CODING->result to one of CODING_RESULT_XXX indicating
138 `result' to one of CODING_FINISH_XXX indicating how the decoding 202 how the decoding is finished.
139 finished.
140 203
141 DST_BYTES zero means that the source area and destination area are 204 Below is the template of these functions. */
142 overlapped, which means that we can produce a decoded text until it
143 reaches the head of the not-yet-decoded source text.
144 205
145 Below is a template for these functions. */
146#if 0 206#if 0
147static void 207static void
148decode_coding_XXX (coding, source, destination, src_bytes, dst_bytes) 208decode_coding_XXXX (coding)
149 struct coding_system *coding; 209 struct coding_system *coding;
150 unsigned char *source, *destination;
151 int src_bytes, dst_bytes;
152{ 210{
153 ... 211 unsigned char *src = coding->source + coding->consumed;
212 unsigned char *src_end = coding->source + coding->src_bytes;
213 /* SRC_BASE remembers the start position in source in each loop.
214 The loop will be exited when there's not enough source code, or
215 when there's no room in CHARBUF for a decoded character. */
216 unsigned char *src_base;
217 /* A buffer to produce decoded characters. */
218 int *charbuf = coding->charbuf;
219 int *charbuf_end = charbuf + coding->charbuf_size;
220 int multibytep = coding->src_multibyte;
221
222 while (1)
223 {
224 src_base = src;
225 if (charbuf < charbuf_end)
226 /* No more room to produce a decoded character. */
227 break;
228 ONE_MORE_BYTE (c);
229 /* Decode it. */
230 }
231
232 no_more_source:
233 if (src_base < src_end
234 && coding->mode & CODING_MODE_LAST_BLOCK)
235 /* If the source ends by partial bytes to construct a character,
236 treat them as eight-bit raw data. */
237 while (src_base < src_end && charbuf < charbuf_end)
238 *charbuf++ = *src_base++;
239 /* Remember how many bytes and characters we consumed. If the
240 source is multibyte, the bytes and chars are not identical. */
241 coding->consumed = coding->consumed_char = src_base - coding->source;
242 /* Remember how many characters we produced. */
243 coding->charbuf_used = charbuf - coding->charbuf;
154} 244}
155#endif 245#endif
156 246
157/*** GENERAL NOTES on `encode_coding_XXX ()' functions *** 247/*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
158 248
159 These functions encode SRC_BYTES length text at SOURCE from Emacs' 249 These functions encode SRC_BYTES length text at SOURCE of Emacs'
160 internal multibyte format to CODING. The resulting unibyte text 250 internal multibyte format by CODING. The resulting byte sequence
161 goes to a place pointed to by DESTINATION, the length of which 251 goes to a place pointed to by DESTINATION, the length of which
162 should not exceed DST_BYTES. 252 should not exceed DST_BYTES.
163 253
164 These functions set the information about original and encoded texts 254 These functions set the information of original and encoded texts in
165 in the members `produced', `produced_char', `consumed', and 255 the members produced, produced_char, consumed, and consumed_char of
166 `consumed_char' of the structure *CODING. They also set the member 256 the structure *CODING. They also set the member result to one of
167 `result' to one of CODING_FINISH_XXX indicating how the encoding 257 CODING_RESULT_XXX indicating how the encoding finished.
168 finished.
169 258
170 DST_BYTES zero means that the source area and destination area are 259 DST_BYTES zero means that source area and destination area are
171 overlapped, which means that we can produce encoded text until it 260 overlapped, which means that we can produce a encoded text until it
172 reaches at the head of the not-yet-encoded source text. 261 reaches at the head of not-yet-encoded source text.
173 262
174 Below is a template for these functions. */ 263 Below is a template of these functions. */
175#if 0 264#if 0
176static void 265static void
177encode_coding_XXX (coding, source, destination, src_bytes, dst_bytes) 266encode_coding_XXX (coding)
178 struct coding_system *coding; 267 struct coding_system *coding;
179 unsigned char *source, *destination;
180 int src_bytes, dst_bytes;
181{ 268{
182 ... 269 int multibytep = coding->dst_multibyte;
270 int *charbuf = coding->charbuf;
271 int *charbuf_end = charbuf->charbuf + coding->charbuf_used;
272 unsigned char *dst = coding->destination + coding->produced;
273 unsigned char *dst_end = coding->destination + coding->dst_bytes;
274 unsigned char *adjusted_dst_end = dst_end - _MAX_BYTES_PRODUCED_IN_LOOP_;
275 int produced_chars = 0;
276
277 for (; charbuf < charbuf_end && dst < adjusted_dst_end; charbuf++)
278 {
279 int c = *charbuf;
280 /* Encode C into DST, and increment DST. */
281 }
282 label_no_more_destination:
283 /* How many chars and bytes we produced. */
284 coding->produced_char += produced_chars;
285 coding->produced = dst - coding->destination;
183} 286}
184#endif 287#endif
185 288
186/*** COMMONLY USED MACROS ***/
187
188/* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
189 get one, two, and three bytes from the source text respectively.
190 If there are not enough bytes in the source, they jump to
191 `label_end_of_loop'. The caller should set variables `coding',
192 `src' and `src_end' to appropriate pointer in advance. These
193 macros are called from decoding routines `decode_coding_XXX', thus
194 it is assumed that the source text is unibyte. */
195
196#define ONE_MORE_BYTE(c1) \
197 do { \
198 if (src >= src_end) \
199 { \
200 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
201 goto label_end_of_loop; \
202 } \
203 c1 = *src++; \
204 } while (0)
205
206#define TWO_MORE_BYTES(c1, c2) \
207 do { \
208 if (src + 1 >= src_end) \
209 { \
210 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
211 goto label_end_of_loop; \
212 } \
213 c1 = *src++; \
214 c2 = *src++; \
215 } while (0)
216
217
218/* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
219 form if MULTIBYTEP is nonzero. */
220
221#define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep) \
222 do { \
223 if (src >= src_end) \
224 { \
225 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
226 goto label_end_of_loop; \
227 } \
228 c1 = *src++; \
229 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
230 c1 = *src++ - 0x20; \
231 } while (0)
232
233/* Set C to the next character at the source text pointed by `src'.
234 If there are not enough characters in the source, jump to
235 `label_end_of_loop'. The caller should set variables `coding'
236 `src', `src_end', and `translation_table' to appropriate pointers
237 in advance. This macro is used in encoding routines
238 `encode_coding_XXX', thus it assumes that the source text is in
239 multibyte form except for 8-bit characters. 8-bit characters are
240 in multibyte form if coding->src_multibyte is nonzero, else they
241 are represented by a single byte. */
242
243#define ONE_MORE_CHAR(c) \
244 do { \
245 int len = src_end - src; \
246 int bytes; \
247 if (len <= 0) \
248 { \
249 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
250 goto label_end_of_loop; \
251 } \
252 if (coding->src_multibyte \
253 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
254 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
255 else \
256 c = *src, bytes = 1; \
257 if (!NILP (translation_table)) \
258 c = translate_char (translation_table, c, -1, 0, 0); \
259 src += bytes; \
260 } while (0)
261
262
263/* Produce a multibyte form of character C to `dst'. Jump to
264 `label_end_of_loop' if there's not enough space at `dst'.
265
266 If we are now in the middle of a composition sequence, the decoded
267 character may be ALTCHAR (for the current composition). In that
268 case, the character goes to coding->cmp_data->data instead of
269 `dst'.
270
271 This macro is used in decoding routines. */
272
273#define EMIT_CHAR(c) \
274 do { \
275 if (! COMPOSING_P (coding) \
276 || coding->composing == COMPOSITION_RELATIVE \
277 || coding->composing == COMPOSITION_WITH_RULE) \
278 { \
279 int bytes = CHAR_BYTES (c); \
280 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
281 { \
282 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
283 goto label_end_of_loop; \
284 } \
285 dst += CHAR_STRING (c, dst); \
286 coding->produced_char++; \
287 } \
288 \
289 if (COMPOSING_P (coding) \
290 && coding->composing != COMPOSITION_RELATIVE) \
291 { \
292 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
293 coding->composition_rule_follows \
294 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
295 } \
296 } while (0)
297
298
299#define EMIT_ONE_BYTE(c) \
300 do { \
301 if (dst >= (dst_bytes ? dst_end : src)) \
302 { \
303 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
304 goto label_end_of_loop; \
305 } \
306 *dst++ = c; \
307 } while (0)
308
309#define EMIT_TWO_BYTES(c1, c2) \
310 do { \
311 if (dst + 2 > (dst_bytes ? dst_end : src)) \
312 { \
313 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
314 goto label_end_of_loop; \
315 } \
316 *dst++ = c1, *dst++ = c2; \
317 } while (0)
318
319#define EMIT_BYTES(from, to) \
320 do { \
321 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
322 { \
323 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
324 goto label_end_of_loop; \
325 } \
326 while (from < to) \
327 *dst++ = *from++; \
328 } while (0)
329
330 289
331/*** 1. Preamble ***/ 290/*** 1. Preamble ***/
332 291
333#ifdef emacs
334#include <config.h> 292#include <config.h>
335#endif
336
337#include <stdio.h> 293#include <stdio.h>
338 294
339#ifdef emacs
340
341#include "lisp.h" 295#include "lisp.h"
342#include "buffer.h" 296#include "buffer.h"
297#include "character.h"
343#include "charset.h" 298#include "charset.h"
344#include "composite.h"
345#include "ccl.h" 299#include "ccl.h"
300#include "composite.h"
346#include "coding.h" 301#include "coding.h"
347#include "window.h" 302#include "window.h"
348 303
349#else /* not emacs */ 304Lisp_Object Vcoding_system_hash_table;
350
351#include "mulelib.h"
352
353#endif /* not emacs */
354 305
355Lisp_Object Qcoding_system, Qeol_type; 306Lisp_Object Qcoding_system, Qcoding_aliases, Qeol_type;
307Lisp_Object Qunix, Qdos, Qmac;
356Lisp_Object Qbuffer_file_coding_system; 308Lisp_Object Qbuffer_file_coding_system;
357Lisp_Object Qpost_read_conversion, Qpre_write_conversion; 309Lisp_Object Qpost_read_conversion, Qpre_write_conversion;
310Lisp_Object Qdefault_char;
358Lisp_Object Qno_conversion, Qundecided; 311Lisp_Object Qno_conversion, Qundecided;
312Lisp_Object Qcharset, Qiso_2022, Qutf_8, Qutf_16, Qshift_jis, Qbig5;
313Lisp_Object Qutf_16_be_nosig, Qutf_16_be, Qutf_16_le_nosig, Qutf_16_le;
314Lisp_Object Qsignature, Qendian, Qbig, Qlittle;
359Lisp_Object Qcoding_system_history; 315Lisp_Object Qcoding_system_history;
360Lisp_Object Qsafe_chars;
361Lisp_Object Qvalid_codes; 316Lisp_Object Qvalid_codes;
362 317
363extern Lisp_Object Qinsert_file_contents, Qwrite_region; 318extern Lisp_Object Qinsert_file_contents, Qwrite_region;
@@ -373,10 +328,6 @@ Lisp_Object eol_mnemonic_unix, eol_mnemonic_dos, eol_mnemonic_mac;
373 decided. */ 328 decided. */
374Lisp_Object eol_mnemonic_undecided; 329Lisp_Object eol_mnemonic_undecided;
375 330
376/* Format of end-of-line decided by system. This is CODING_EOL_LF on
377 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac. */
378int system_eol_type;
379
380#ifdef emacs 331#ifdef emacs
381 332
382Lisp_Object Vcoding_system_list, Vcoding_system_alist; 333Lisp_Object Vcoding_system_list, Vcoding_system_alist;
@@ -419,9 +370,6 @@ struct coding_system safe_terminal_coding;
419/* Coding system of what is sent from terminal keyboard. */ 370/* Coding system of what is sent from terminal keyboard. */
420struct coding_system keyboard_coding; 371struct coding_system keyboard_coding;
421 372
422/* Default coding system to be used to write a file. */
423struct coding_system default_buffer_file_coding;
424
425Lisp_Object Vfile_coding_system_alist; 373Lisp_Object Vfile_coding_system_alist;
426Lisp_Object Vprocess_coding_system_alist; 374Lisp_Object Vprocess_coding_system_alist;
427Lisp_Object Vnetwork_coding_system_alist; 375Lisp_Object Vnetwork_coding_system_alist;
@@ -430,42 +378,6 @@ Lisp_Object Vlocale_coding_system;
430 378
431#endif /* emacs */ 379#endif /* emacs */
432 380
433Lisp_Object Qcoding_category, Qcoding_category_index;
434
435/* List of symbols `coding-category-xxx' ordered by priority. */
436Lisp_Object Vcoding_category_list;
437
438/* Table of coding categories (Lisp symbols). */
439Lisp_Object Vcoding_category_table;
440
441/* Table of names of symbol for each coding-category. */
442char *coding_category_name[CODING_CATEGORY_IDX_MAX] = {
443 "coding-category-emacs-mule",
444 "coding-category-sjis",
445 "coding-category-iso-7",
446 "coding-category-iso-7-tight",
447 "coding-category-iso-8-1",
448 "coding-category-iso-8-2",
449 "coding-category-iso-7-else",
450 "coding-category-iso-8-else",
451 "coding-category-ccl",
452 "coding-category-big5",
453 "coding-category-utf-8",
454 "coding-category-utf-16-be",
455 "coding-category-utf-16-le",
456 "coding-category-raw-text",
457 "coding-category-binary"
458};
459
460/* Table of pointers to coding systems corresponding to each coding
461 categories. */
462struct coding_system *coding_system_table[CODING_CATEGORY_IDX_MAX];
463
464/* Table of coding category masks. Nth element is a mask for a coding
465 category of which priority is Nth. */
466static
467int coding_priorities[CODING_CATEGORY_IDX_MAX];
468
469/* Flag to tell if we look up translation table on character code 381/* Flag to tell if we look up translation table on character code
470 conversion. */ 382 conversion. */
471Lisp_Object Venable_character_translation; 383Lisp_Object Venable_character_translation;
@@ -480,7 +392,7 @@ Lisp_Object Qtranslation_table_for_decode;
480Lisp_Object Qtranslation_table_for_encode; 392Lisp_Object Qtranslation_table_for_encode;
481 393
482/* Alist of charsets vs revision number. */ 394/* Alist of charsets vs revision number. */
483Lisp_Object Vcharset_revision_alist; 395static Lisp_Object Vcharset_revision_table;
484 396
485/* Default coding systems used for process I/O. */ 397/* Default coding systems used for process I/O. */
486Lisp_Object Vdefault_process_coding_system; 398Lisp_Object Vdefault_process_coding_system;
@@ -495,26 +407,1034 @@ static int inhibit_pre_post_conversion;
495Lisp_Object Vchar_coding_system_table; 407Lisp_Object Vchar_coding_system_table;
496Lisp_Object Qchar_coding_system; 408Lisp_Object Qchar_coding_system;
497 409
498/* Return `safe-chars' property of coding system CODING. Don't check 410/* Two special coding systems. */
499 validity of CODING. */ 411Lisp_Object Vsjis_coding_system;
412Lisp_Object Vbig5_coding_system;
413
414
415static int detect_coding_utf_8 P_ ((struct coding_system *, int *));
416static void decode_coding_utf_8 P_ ((struct coding_system *));
417static int encode_coding_utf_8 P_ ((struct coding_system *));
418
419static int detect_coding_utf_16 P_ ((struct coding_system *, int *));
420static void decode_coding_utf_16 P_ ((struct coding_system *));
421static int encode_coding_utf_16 P_ ((struct coding_system *));
422
423static int detect_coding_iso_2022 P_ ((struct coding_system *, int *));
424static void decode_coding_iso_2022 P_ ((struct coding_system *));
425static int encode_coding_iso_2022 P_ ((struct coding_system *));
426
427static int detect_coding_emacs_mule P_ ((struct coding_system *, int *));
428static void decode_coding_emacs_mule P_ ((struct coding_system *));
429static int encode_coding_emacs_mule P_ ((struct coding_system *));
430
431static int detect_coding_sjis P_ ((struct coding_system *, int *));
432static void decode_coding_sjis P_ ((struct coding_system *));
433static int encode_coding_sjis P_ ((struct coding_system *));
434
435static int detect_coding_big5 P_ ((struct coding_system *, int *));
436static void decode_coding_big5 P_ ((struct coding_system *));
437static int encode_coding_big5 P_ ((struct coding_system *));
438
439static int detect_coding_ccl P_ ((struct coding_system *, int *));
440static void decode_coding_ccl P_ ((struct coding_system *));
441static int encode_coding_ccl P_ ((struct coding_system *));
442
443static void decode_coding_raw_text P_ ((struct coding_system *));
444static int encode_coding_raw_text P_ ((struct coding_system *));
445
446
447/* ISO2022 section */
448
449#define CODING_ISO_INITIAL(coding, reg) \
450 (XINT (AREF (AREF (CODING_ID_ATTRS ((coding)->id), \
451 coding_attr_iso_initial), \
452 reg)))
453
454
455#define CODING_ISO_REQUEST(coding, charset_id) \
456 ((charset_id <= (coding)->max_charset_id \
457 ? (coding)->safe_charsets[charset_id] \
458 : -1))
459
460
461#define CODING_ISO_FLAGS(coding) \
462 ((coding)->spec.iso_2022.flags)
463#define CODING_ISO_DESIGNATION(coding, reg) \
464 ((coding)->spec.iso_2022.current_designation[reg])
465#define CODING_ISO_INVOCATION(coding, plane) \
466 ((coding)->spec.iso_2022.current_invocation[plane])
467#define CODING_ISO_SINGLE_SHIFTING(coding) \
468 ((coding)->spec.iso_2022.single_shifting)
469#define CODING_ISO_BOL(coding) \
470 ((coding)->spec.iso_2022.bol)
471#define CODING_ISO_INVOKED_CHARSET(coding, plane) \
472 CODING_ISO_DESIGNATION ((coding), CODING_ISO_INVOCATION ((coding), (plane)))
473
474/* Control characters of ISO2022. */
475 /* code */ /* function */
476#define ISO_CODE_LF 0x0A /* line-feed */
477#define ISO_CODE_CR 0x0D /* carriage-return */
478#define ISO_CODE_SO 0x0E /* shift-out */
479#define ISO_CODE_SI 0x0F /* shift-in */
480#define ISO_CODE_SS2_7 0x19 /* single-shift-2 for 7-bit code */
481#define ISO_CODE_ESC 0x1B /* escape */
482#define ISO_CODE_SS2 0x8E /* single-shift-2 */
483#define ISO_CODE_SS3 0x8F /* single-shift-3 */
484#define ISO_CODE_CSI 0x9B /* control-sequence-introducer */
485
486/* All code (1-byte) of ISO2022 is classified into one of the
487 followings. */
488enum iso_code_class_type
489 {
490 ISO_control_0, /* Control codes in the range
491 0x00..0x1F and 0x7F, except for the
492 following 5 codes. */
493 ISO_carriage_return, /* ISO_CODE_CR (0x0D) */
494 ISO_shift_out, /* ISO_CODE_SO (0x0E) */
495 ISO_shift_in, /* ISO_CODE_SI (0x0F) */
496 ISO_single_shift_2_7, /* ISO_CODE_SS2_7 (0x19) */
497 ISO_escape, /* ISO_CODE_SO (0x1B) */
498 ISO_control_1, /* Control codes in the range
499 0x80..0x9F, except for the
500 following 3 codes. */
501 ISO_single_shift_2, /* ISO_CODE_SS2 (0x8E) */
502 ISO_single_shift_3, /* ISO_CODE_SS3 (0x8F) */
503 ISO_control_sequence_introducer, /* ISO_CODE_CSI (0x9B) */
504 ISO_0x20_or_0x7F, /* Codes of the values 0x20 or 0x7F. */
505 ISO_graphic_plane_0, /* Graphic codes in the range 0x21..0x7E. */
506 ISO_0xA0_or_0xFF, /* Codes of the values 0xA0 or 0xFF. */
507 ISO_graphic_plane_1 /* Graphic codes in the range 0xA1..0xFE. */
508 };
500 509
501Lisp_Object 510/** The macros CODING_ISO_FLAG_XXX defines a flag bit of the
502coding_safe_chars (coding) 511 `iso-flags' attribute of an iso2022 coding system. */
512
513/* If set, produce long-form designation sequence (e.g. ESC $ ( A)
514 instead of the correct short-form sequence (e.g. ESC $ A). */
515#define CODING_ISO_FLAG_LONG_FORM 0x0001
516
517/* If set, reset graphic planes and registers at end-of-line to the
518 initial state. */
519#define CODING_ISO_FLAG_RESET_AT_EOL 0x0002
520
521/* If set, reset graphic planes and registers before any control
522 characters to the initial state. */
523#define CODING_ISO_FLAG_RESET_AT_CNTL 0x0004
524
525/* If set, encode by 7-bit environment. */
526#define CODING_ISO_FLAG_SEVEN_BITS 0x0008
527
528/* If set, use locking-shift function. */
529#define CODING_ISO_FLAG_LOCKING_SHIFT 0x0010
530
531/* If set, use single-shift function. Overwrite
532 CODING_ISO_FLAG_LOCKING_SHIFT. */
533#define CODING_ISO_FLAG_SINGLE_SHIFT 0x0020
534
535/* If set, use designation escape sequence. */
536#define CODING_ISO_FLAG_DESIGNATION 0x0040
537
538/* If set, produce revision number sequence. */
539#define CODING_ISO_FLAG_REVISION 0x0080
540
541/* If set, produce ISO6429's direction specifying sequence. */
542#define CODING_ISO_FLAG_DIRECTION 0x0100
543
544/* If set, assume designation states are reset at beginning of line on
545 output. */
546#define CODING_ISO_FLAG_INIT_AT_BOL 0x0200
547
548/* If set, designation sequence should be placed at beginning of line
549 on output. */
550#define CODING_ISO_FLAG_DESIGNATE_AT_BOL 0x0400
551
552/* If set, do not encode unsafe charactes on output. */
553#define CODING_ISO_FLAG_SAFE 0x0800
554
555/* If set, extra latin codes (128..159) are accepted as a valid code
556 on input. */
557#define CODING_ISO_FLAG_LATIN_EXTRA 0x1000
558
559#define CODING_ISO_FLAG_COMPOSITION 0x2000
560
561#define CODING_ISO_FLAG_EUC_TW_SHIFT 0x4000
562
563#define CODING_ISO_FLAG_FULL_SUPPORT 0x8000
564
565/* A character to be produced on output if encoding of the original
566 character is prohibited by CODING_ISO_FLAG_SAFE. */
567#define CODING_INHIBIT_CHARACTER_SUBSTITUTION '?'
568
569
570/* UTF-16 section */
571#define CODING_UTF_16_BOM(coding) \
572 ((coding)->spec.utf_16.bom)
573
574#define CODING_UTF_16_ENDIAN(coding) \
575 ((coding)->spec.utf_16.endian)
576
577#define CODING_UTF_16_SURROGATE(coding) \
578 ((coding)->spec.utf_16.surrogate)
579
580
581/* CCL section */
582#define CODING_CCL_DECODER(coding) \
583 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_decoder)
584#define CODING_CCL_ENCODER(coding) \
585 AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_encoder)
586#define CODING_CCL_VALIDS(coding) \
587 (XSTRING (AREF (CODING_ID_ATTRS ((coding)->id), coding_attr_ccl_valids)) \
588 ->data)
589
590/* Index for each coding category in `coding_category_table' */
591
592enum coding_category
593 {
594 coding_category_iso_7,
595 coding_category_iso_7_tight,
596 coding_category_iso_8_1,
597 coding_category_iso_8_2,
598 coding_category_iso_7_else,
599 coding_category_iso_8_else,
600 coding_category_utf_8,
601 coding_category_utf_16_auto,
602 coding_category_utf_16_be,
603 coding_category_utf_16_le,
604 coding_category_utf_16_be_nosig,
605 coding_category_utf_16_le_nosig,
606 coding_category_charset,
607 coding_category_sjis,
608 coding_category_big5,
609 coding_category_ccl,
610 coding_category_emacs_mule,
611 /* All above are targets of code detection. */
612 coding_category_raw_text,
613 coding_category_undecided,
614 coding_category_max
615 };
616
617/* Definitions of flag bits used in detect_coding_XXXX. */
618#define CATEGORY_MASK_ISO_7 (1 << coding_category_iso_7)
619#define CATEGORY_MASK_ISO_7_TIGHT (1 << coding_category_iso_7_tight)
620#define CATEGORY_MASK_ISO_8_1 (1 << coding_category_iso_8_1)
621#define CATEGORY_MASK_ISO_8_2 (1 << coding_category_iso_8_2)
622#define CATEGORY_MASK_ISO_7_ELSE (1 << coding_category_iso_7_else)
623#define CATEGORY_MASK_ISO_8_ELSE (1 << coding_category_iso_8_else)
624#define CATEGORY_MASK_UTF_8 (1 << coding_category_utf_8)
625#define CATEGORY_MASK_UTF_16_BE (1 << coding_category_utf_16_be)
626#define CATEGORY_MASK_UTF_16_LE (1 << coding_category_utf_16_le)
627#define CATEGORY_MASK_UTF_16_BE_NOSIG (1 << coding_category_utf_16_be_nosig)
628#define CATEGORY_MASK_UTF_16_LE_NOSIG (1 << coding_category_utf_16_le_nosig)
629#define CATEGORY_MASK_CHARSET (1 << coding_category_charset)
630#define CATEGORY_MASK_SJIS (1 << coding_category_sjis)
631#define CATEGORY_MASK_BIG5 (1 << coding_category_big5)
632#define CATEGORY_MASK_CCL (1 << coding_category_ccl)
633#define CATEGORY_MASK_EMACS_MULE (1 << coding_category_emacs_mule)
634
635/* This value is returned if detect_coding_mask () find nothing other
636 than ASCII characters. */
637#define CATEGORY_MASK_ANY \
638 (CATEGORY_MASK_ISO_7 \
639 | CATEGORY_MASK_ISO_7_TIGHT \
640 | CATEGORY_MASK_ISO_8_1 \
641 | CATEGORY_MASK_ISO_8_2 \
642 | CATEGORY_MASK_ISO_7_ELSE \
643 | CATEGORY_MASK_ISO_8_ELSE \
644 | CATEGORY_MASK_UTF_8 \
645 | CATEGORY_MASK_UTF_16_BE \
646 | CATEGORY_MASK_UTF_16_LE \
647 | CATEGORY_MASK_UTF_16_BE_NOSIG \
648 | CATEGORY_MASK_UTF_16_LE_NOSIG \
649 | CATEGORY_MASK_CHARSET \
650 | CATEGORY_MASK_SJIS \
651 | CATEGORY_MASK_BIG5 \
652 | CATEGORY_MASK_CCL \
653 | CATEGORY_MASK_EMACS_MULE)
654
655
656#define CATEGORY_MASK_ISO_7BIT \
657 (CATEGORY_MASK_ISO_7 | CATEGORY_MASK_ISO_7_TIGHT)
658
659#define CATEGORY_MASK_ISO_8BIT \
660 (CATEGORY_MASK_ISO_8_1 | CATEGORY_MASK_ISO_8_2)
661
662#define CATEGORY_MASK_ISO_ELSE \
663 (CATEGORY_MASK_ISO_7_ELSE | CATEGORY_MASK_ISO_8_ELSE)
664
665#define CATEGORY_MASK_ISO_ESCAPE \
666 (CATEGORY_MASK_ISO_7 \
667 | CATEGORY_MASK_ISO_7_TIGHT \
668 | CATEGORY_MASK_ISO_7_ELSE \
669 | CATEGORY_MASK_ISO_8_ELSE)
670
671#define CATEGORY_MASK_ISO \
672 ( CATEGORY_MASK_ISO_7BIT \
673 | CATEGORY_MASK_ISO_8BIT \
674 | CATEGORY_MASK_ISO_ELSE)
675
676#define CATEGORY_MASK_UTF_16 \
677 (CATEGORY_MASK_UTF_16_BE \
678 | CATEGORY_MASK_UTF_16_LE \
679 | CATEGORY_MASK_UTF_16_BE_NOSIG \
680 | CATEGORY_MASK_UTF_16_LE_NOSIG)
681
682
683/* List of symbols `coding-category-xxx' ordered by priority. This
684 variable is exposed to Emacs Lisp. */
685static Lisp_Object Vcoding_category_list;
686
687/* Table of coding categories (Lisp symbols). This variable is for
688 internal use oly. */
689static Lisp_Object Vcoding_category_table;
690
691/* Table of coding-categories ordered by priority. */
692static enum coding_category coding_priorities[coding_category_max];
693
694/* Nth element is a coding context for the coding system bound to the
695 Nth coding category. */
696static struct coding_system coding_categories[coding_category_max];
697
698static int detected_mask[coding_category_raw_text] =
699 { CATEGORY_MASK_ISO,
700 CATEGORY_MASK_ISO,
701 CATEGORY_MASK_ISO,
702 CATEGORY_MASK_ISO,
703 CATEGORY_MASK_ISO,
704 CATEGORY_MASK_ISO,
705 CATEGORY_MASK_UTF_8,
706 CATEGORY_MASK_UTF_16,
707 CATEGORY_MASK_UTF_16,
708 CATEGORY_MASK_UTF_16,
709 CATEGORY_MASK_UTF_16,
710 CATEGORY_MASK_UTF_16,
711 CATEGORY_MASK_CHARSET,
712 CATEGORY_MASK_SJIS,
713 CATEGORY_MASK_BIG5,
714 CATEGORY_MASK_CCL,
715 CATEGORY_MASK_EMACS_MULE
716 };
717
718/*** Commonly used macros and functions ***/
719
720#ifndef min
721#define min(a, b) ((a) < (b) ? (a) : (b))
722#endif
723#ifndef max
724#define max(a, b) ((a) > (b) ? (a) : (b))
725#endif
726
727#define CODING_GET_INFO(coding, attrs, eol_type, charset_list) \
728 do { \
729 attrs = CODING_ID_ATTRS (coding->id); \
730 eol_type = CODING_ID_EOL_TYPE (coding->id); \
731 if (VECTORP (eol_type)) \
732 eol_type = Qunix; \
733 charset_list = CODING_ATTR_CHARSET_LIST (attrs); \
734 } while (0)
735
736
737/* Safely get one byte from the source text pointed by SRC which ends
738 at SRC_END, and set C to that byte. If there are not enough bytes
739 in the source, it jumps to `no_more_source'. The caller
740 should declare and set these variables appropriately in advance:
741 src, src_end, multibytep
742*/
743
744#define ONE_MORE_BYTE(c) \
745 do { \
746 if (src == src_end) \
747 { \
748 if (src_base < src) \
749 coding->result = CODING_RESULT_INSUFFICIENT_SRC; \
750 goto no_more_source; \
751 } \
752 c = *src++; \
753 if (multibytep && (c & 0x80)) \
754 { \
755 if ((c & 0xFE) != 0xC0) \
756 error ("Undecodable char found"); \
757 c = ((c & 1) << 6) | *src++; \
758 } \
759 consumed_chars++; \
760 } while (0)
761
762
763#define ONE_MORE_BYTE_NO_CHECK(c) \
764 do { \
765 c = *src++; \
766 if (multibytep && (c & 0x80)) \
767 { \
768 if ((c & 0xFE) != 0xC0) \
769 error ("Undecodable char found"); \
770 c = ((c & 1) << 6) | *src++; \
771 } \
772 } while (0)
773
774
775/* Store a byte C in the place pointed by DST and increment DST to the
776 next free point, and increment PRODUCED_CHARS. The caller should
777 assure that C is 0..127, and declare and set the variable `dst'
778 appropriately in advance.
779*/
780
781
782#define EMIT_ONE_ASCII_BYTE(c) \
783 do { \
784 produced_chars++; \
785 *dst++ = (c); \
786 } while (0)
787
788
789/* Like EMIT_ONE_ASCII_BYTE byt store two bytes; C1 and C2. */
790
791#define EMIT_TWO_ASCII_BYTES(c1, c2) \
792 do { \
793 produced_chars += 2; \
794 *dst++ = (c1), *dst++ = (c2); \
795 } while (0)
796
797
798/* Store a byte C in the place pointed by DST and increment DST to the
799 next free point, and increment PRODUCED_CHARS. If MULTIBYTEP is
800 nonzero, store in an appropriate multibyte from. The caller should
801 declare and set the variables `dst' and `multibytep' appropriately
802 in advance. */
803
804#define EMIT_ONE_BYTE(c) \
805 do { \
806 produced_chars++; \
807 if (multibytep) \
808 { \
809 int ch = (c); \
810 if (ch >= 0x80) \
811 ch = BYTE8_TO_CHAR (ch); \
812 CHAR_STRING_ADVANCE (ch, dst); \
813 } \
814 else \
815 *dst++ = (c); \
816 } while (0)
817
818
819/* Like EMIT_ONE_BYTE, but emit two bytes; C1 and C2. */
820
821#define EMIT_TWO_BYTES(c1, c2) \
822 do { \
823 produced_chars += 2; \
824 if (multibytep) \
825 { \
826 CHAR_STRING_ADVANCE ((int) (c1), dst); \
827 CHAR_STRING_ADVANCE ((int) (c2), dst); \
828 } \
829 else \
830 { \
831 *dst++ = (c1); \
832 *dst++ = (c2); \
833 } \
834 } while (0)
835
836
837#define EMIT_THREE_BYTES(c1, c2, c3) \
838 do { \
839 EMIT_ONE_BYTE (c1); \
840 EMIT_TWO_BYTES (c2, c3); \
841 } while (0)
842
843
844#define EMIT_FOUR_BYTES(c1, c2, c3, c4) \
845 do { \
846 EMIT_TWO_BYTES (c1, c2); \
847 EMIT_TWO_BYTES (c3, c4); \
848 } while (0)
849
850
851#define CODING_DECODE_CHAR(coding, src, src_base, src_end, charset, code, c) \
852 do { \
853 charset_map_loaded = 0; \
854 c = DECODE_CHAR (charset, code); \
855 if (charset_map_loaded) \
856 { \
857 unsigned char *orig = coding->source; \
858 EMACS_INT offset; \
859 \
860 coding_set_source (coding); \
861 offset = coding->source - orig; \
862 src += offset; \
863 src_base += offset; \
864 src_end += offset; \
865 } \
866 } while (0)
867
868
869#define ASSURE_DESTINATION(bytes) \
870 do { \
871 if (dst + (bytes) >= dst_end) \
872 { \
873 int more_bytes = charbuf_end - charbuf + (bytes); \
874 \
875 dst = alloc_destination (coding, more_bytes, dst); \
876 dst_end = coding->destination + coding->dst_bytes; \
877 } \
878 } while (0)
879
880
881
882static void
883coding_set_source (coding)
884 struct coding_system *coding;
885{
886 if (BUFFERP (coding->src_object))
887 {
888 if (coding->src_pos < 0)
889 coding->source = GAP_END_ADDR + coding->src_pos_byte;
890 else
891 {
892 if (coding->src_pos < GPT
893 && coding->src_pos + coding->src_chars >= GPT)
894 move_gap_both (coding->src_pos, coding->src_pos_byte);
895 coding->source = BYTE_POS_ADDR (coding->src_pos_byte);
896 }
897 }
898 else if (STRINGP (coding->src_object))
899 {
900 coding->source = (XSTRING (coding->src_object)->data
901 + coding->src_pos_byte);
902 }
903 else
904 /* Otherwise, the source is C string and is never relocated
905 automatically. Thus we don't have to update anything. */
906 ;
907}
908
909static void
910coding_set_destination (coding)
911 struct coding_system *coding;
912{
913 if (BUFFERP (coding->dst_object))
914 {
915 /* We are sure that coding->dst_pos_byte is before the gap of the
916 buffer. */
917 coding->destination = (BUF_BEG_ADDR (XBUFFER (coding->dst_object))
918 + coding->dst_pos_byte - 1);
919 if (coding->src_pos < 0)
920 /* The source and destination is in the same buffer. */
921 coding->dst_bytes = (GAP_END_ADDR
922 - (coding->src_bytes - coding->consumed)
923 - coding->destination);
924 else
925 coding->dst_bytes = (BUF_GAP_END_ADDR (XBUFFER (coding->dst_object))
926 - coding->destination);
927 }
928 else
929 /* Otherwise, the destination is C string and is never relocated
930 automatically. Thus we don't have to update anything. */
931 ;
932}
933
934
935static void
936coding_alloc_by_realloc (coding, bytes)
937 struct coding_system *coding;
938 EMACS_INT bytes;
939{
940 coding->destination = (unsigned char *) xrealloc (coding->destination,
941 coding->dst_bytes + bytes);
942 coding->dst_bytes += bytes;
943}
944
945static void
946coding_alloc_by_making_gap (coding, bytes)
947 struct coding_system *coding;
948 EMACS_INT bytes;
949{
950 Lisp_Object this_buffer;
951
952 this_buffer = Fcurrent_buffer ();
953 if (EQ (this_buffer, coding->dst_object))
954 {
955 EMACS_INT add = coding->src_bytes - coding->consumed;
956
957 GAP_SIZE -= add; ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
958 make_gap (bytes);
959 GAP_SIZE += add; ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
960 }
961 else
962 {
963 set_buffer_internal (XBUFFER (coding->dst_object));
964 make_gap (bytes);
965 set_buffer_internal (XBUFFER (this_buffer));
966 }
967}
968
969
970static unsigned char *
971alloc_destination (coding, nbytes, dst)
503 struct coding_system *coding; 972 struct coding_system *coding;
973 int nbytes;
974 unsigned char *dst;
504{ 975{
505 Lisp_Object coding_spec, plist, safe_chars; 976 EMACS_INT offset = dst - coding->destination;
506 977
507 coding_spec = Fget (coding->symbol, Qcoding_system); 978 if (BUFFERP (coding->dst_object))
508 plist = XVECTOR (coding_spec)->contents[3]; 979 coding_alloc_by_making_gap (coding, nbytes);
509 safe_chars = Fplist_get (XVECTOR (coding_spec)->contents[3], Qsafe_chars); 980 else
510 return (CHAR_TABLE_P (safe_chars) ? safe_chars : Qt); 981 coding_alloc_by_realloc (coding, nbytes);
982 coding->result = CODING_RESULT_SUCCESS;
983 coding_set_destination (coding);
984 dst = coding->destination + offset;
985 return dst;
511} 986}
512 987
513#define CODING_SAFE_CHAR_P(safe_chars, c) \ 988
514 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c))) 989/*** 2. Emacs' internal format (emacs-utf-8) ***/
990
991
515 992
516 993
517/*** 2. Emacs internal format (emacs-mule) handlers ***/ 994/*** 3. UTF-8 ***/
995
996/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
997 Check if a text is encoded in UTF-8. If it is, return
998 CATEGORY_MASK_UTF_8, else return 0. */
999
1000#define UTF_8_1_OCTET_P(c) ((c) < 0x80)
1001#define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
1002#define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
1003#define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
1004#define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
1005#define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
1006
1007static int
1008detect_coding_utf_8 (coding, mask)
1009 struct coding_system *coding;
1010 int *mask;
1011{
1012 unsigned char *src = coding->source, *src_base = src;
1013 unsigned char *src_end = coding->source + coding->src_bytes;
1014 int multibytep = coding->src_multibyte;
1015 int consumed_chars = 0;
1016 int found = 0;
1017
1018 /* A coding system of this category is always ASCII compatible. */
1019 src += coding->head_ascii;
1020
1021 while (1)
1022 {
1023 int c, c1, c2, c3, c4;
1024
1025 ONE_MORE_BYTE (c);
1026 if (UTF_8_1_OCTET_P (c))
1027 continue;
1028 ONE_MORE_BYTE (c1);
1029 if (! UTF_8_EXTRA_OCTET_P (c1))
1030 break;
1031 if (UTF_8_2_OCTET_LEADING_P (c))
1032 {
1033 found++;
1034 continue;
1035 }
1036 ONE_MORE_BYTE (c2);
1037 if (! UTF_8_EXTRA_OCTET_P (c2))
1038 break;
1039 if (UTF_8_3_OCTET_LEADING_P (c))
1040 {
1041 found++;
1042 continue;
1043 }
1044 ONE_MORE_BYTE (c3);
1045 if (! UTF_8_EXTRA_OCTET_P (c3))
1046 break;
1047 if (UTF_8_4_OCTET_LEADING_P (c))
1048 {
1049 found++;
1050 continue;
1051 }
1052 ONE_MORE_BYTE (c4);
1053 if (! UTF_8_EXTRA_OCTET_P (c4))
1054 break;
1055 if (UTF_8_5_OCTET_LEADING_P (c))
1056 {
1057 found++;
1058 continue;
1059 }
1060 break;
1061 }
1062 *mask &= ~CATEGORY_MASK_UTF_8;
1063 return 0;
1064
1065 no_more_source:
1066 if (! found)
1067 return 0;
1068 *mask &= CATEGORY_MASK_UTF_8;
1069 return 1;
1070}
1071
1072
1073static void
1074decode_coding_utf_8 (coding)
1075 struct coding_system *coding;
1076{
1077 unsigned char *src = coding->source + coding->consumed;
1078 unsigned char *src_end = coding->source + coding->src_bytes;
1079 unsigned char *src_base;
1080 int *charbuf = coding->charbuf;
1081 int *charbuf_end = charbuf + coding->charbuf_size;
1082 int consumed_chars = 0, consumed_chars_base;
1083 int multibytep = coding->src_multibyte;
1084 Lisp_Object attr, eol_type, charset_list;
1085
1086 CODING_GET_INFO (coding, attr, eol_type, charset_list);
1087
1088 while (1)
1089 {
1090 int c, c1, c2, c3, c4, c5;
1091
1092 src_base = src;
1093 consumed_chars_base = consumed_chars;
1094
1095 if (charbuf >= charbuf_end)
1096 break;
1097
1098 ONE_MORE_BYTE (c1);
1099 if (UTF_8_1_OCTET_P(c1))
1100 {
1101 c = c1;
1102 if (c == '\r')
1103 {
1104 if (EQ (eol_type, Qdos))
1105 {
1106 if (src == src_end)
1107 goto no_more_source;
1108 if (*src == '\n')
1109 ONE_MORE_BYTE (c);
1110 }
1111 else if (EQ (eol_type, Qmac))
1112 c = '\n';
1113 }
1114 }
1115 else
1116 {
1117 ONE_MORE_BYTE (c2);
1118 if (! UTF_8_EXTRA_OCTET_P (c2))
1119 goto invalid_code;
1120 if (UTF_8_2_OCTET_LEADING_P (c1))
1121 c = ((c1 & 0x1F) << 6) | (c2 & 0x3F);
1122 else
1123 {
1124 ONE_MORE_BYTE (c3);
1125 if (! UTF_8_EXTRA_OCTET_P (c3))
1126 goto invalid_code;
1127 if (UTF_8_3_OCTET_LEADING_P (c1))
1128 c = (((c1 & 0xF) << 12)
1129 | ((c2 & 0x3F) << 6) | (c3 & 0x3F));
1130 else
1131 {
1132 ONE_MORE_BYTE (c4);
1133 if (! UTF_8_EXTRA_OCTET_P (c4))
1134 goto invalid_code;
1135 if (UTF_8_4_OCTET_LEADING_P (c1))
1136 c = (((c1 & 0x7) << 18) | ((c2 & 0x3F) << 12)
1137 | ((c3 & 0x3F) << 6) | (c4 & 0x3F));
1138 else
1139 {
1140 ONE_MORE_BYTE (c5);
1141 if (! UTF_8_EXTRA_OCTET_P (c5))
1142 goto invalid_code;
1143 if (UTF_8_5_OCTET_LEADING_P (c1))
1144 {
1145 c = (((c1 & 0x3) << 24) | ((c2 & 0x3F) << 18)
1146 | ((c3 & 0x3F) << 12) | ((c4 & 0x3F) << 6)
1147 | (c5 & 0x3F));
1148 if (c > MAX_CHAR)
1149 goto invalid_code;
1150 }
1151 else
1152 goto invalid_code;
1153 }
1154 }
1155 }
1156 }
1157
1158 *charbuf++ = c;
1159 continue;
1160
1161 invalid_code:
1162 src = src_base;
1163 consumed_chars = consumed_chars_base;
1164 ONE_MORE_BYTE (c);
1165 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
1166 coding->errors++;
1167 }
1168
1169 no_more_source:
1170 coding->consumed_char += consumed_chars_base;
1171 coding->consumed = src_base - coding->source;
1172 coding->charbuf_used = charbuf - coding->charbuf;
1173}
1174
1175
1176static int
1177encode_coding_utf_8 (coding)
1178 struct coding_system *coding;
1179{
1180 int multibytep = coding->dst_multibyte;
1181 int *charbuf = coding->charbuf;
1182 int *charbuf_end = charbuf + coding->charbuf_used;
1183 unsigned char *dst = coding->destination + coding->produced;
1184 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1185 int produced_chars;
1186 int c;
1187
1188 if (multibytep)
1189 {
1190 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
1191
1192 while (charbuf < charbuf_end)
1193 {
1194 unsigned char str[MAX_MULTIBYTE_LENGTH], *p, *pend = str;
1195
1196 ASSURE_DESTINATION (safe_room);
1197 c = *charbuf++;
1198 CHAR_STRING_ADVANCE (c, pend);
1199 for (p = str; p < pend; p++)
1200 EMIT_ONE_BYTE (*p);
1201 }
1202 }
1203 else
1204 {
1205 int safe_room = MAX_MULTIBYTE_LENGTH;
1206
1207 while (charbuf < charbuf_end)
1208 {
1209 ASSURE_DESTINATION (safe_room);
1210 c = *charbuf++;
1211 dst += CHAR_STRING (c, dst);
1212 produced_chars++;
1213 }
1214 }
1215 coding->result = CODING_RESULT_SUCCESS;
1216 coding->produced_char += produced_chars;
1217 coding->produced = dst - coding->destination;
1218 return 0;
1219}
1220
1221
1222/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1223 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
1224 Little Endian (otherwise). If it is, return
1225 CATEGORY_MASK_UTF_16_BE or CATEGORY_MASK_UTF_16_LE,
1226 else return 0. */
1227
1228#define UTF_16_HIGH_SURROGATE_P(val) \
1229 (((val) & 0xFC00) == 0xD800)
1230
1231#define UTF_16_LOW_SURROGATE_P(val) \
1232 (((val) & 0xFC00) == 0xDC00)
1233
1234#define UTF_16_INVALID_P(val) \
1235 (((val) == 0xFFFE) \
1236 || ((val) == 0xFFFF) \
1237 || UTF_16_LOW_SURROGATE_P (val))
1238
1239
1240static int
1241detect_coding_utf_16 (coding, mask)
1242 struct coding_system *coding;
1243 int *mask;
1244{
1245 unsigned char *src = coding->source, *src_base = src;
1246 unsigned char *src_end = coding->source + coding->src_bytes;
1247 int multibytep = coding->src_multibyte;
1248 int consumed_chars = 0;
1249 int c1, c2;
1250
1251 ONE_MORE_BYTE (c1);
1252 ONE_MORE_BYTE (c2);
1253
1254 if ((c1 == 0xFF) && (c2 == 0xFE))
1255 {
1256 *mask &= CATEGORY_MASK_UTF_16_LE;
1257 return 1;
1258 }
1259 else if ((c1 == 0xFE) && (c2 == 0xFF))
1260 {
1261 *mask &= CATEGORY_MASK_UTF_16_BE;
1262 return 1;
1263 }
1264 no_more_source:
1265 return 0;
1266}
1267
1268static void
1269decode_coding_utf_16 (coding)
1270 struct coding_system *coding;
1271{
1272 unsigned char *src = coding->source + coding->consumed;
1273 unsigned char *src_end = coding->source + coding->src_bytes;
1274 unsigned char *src_base, *surrogate_high_base;
1275 int *charbuf = coding->charbuf;
1276 int *charbuf_end = charbuf + coding->charbuf_size;
1277 int consumed_chars = 0, consumed_chars_base;
1278 int multibytep = coding->src_multibyte;
1279 enum utf_16_bom_type bom = CODING_UTF_16_BOM (coding);
1280 enum utf_16_endian_type endian = CODING_UTF_16_ENDIAN (coding);
1281 int surrogate = CODING_UTF_16_SURROGATE (coding);
1282 Lisp_Object attr, eol_type, charset_list;
1283
1284 CODING_GET_INFO (coding, attr, eol_type, charset_list);
1285
1286 if (bom != utf_16_without_bom)
1287 {
1288 int c, c1, c2;
1289
1290 src_base = src;
1291 ONE_MORE_BYTE (c1);
1292 ONE_MORE_BYTE (c2);
1293 c = (c1 << 16) | c2;
1294 if (bom == utf_16_with_bom)
1295 {
1296 if (endian == utf_16_big_endian
1297 ? c != 0xFFFE : c != 0xFEFF)
1298 {
1299 /* We are sure that there's enouph room at CHARBUF. */
1300 *charbuf++ = c1;
1301 *charbuf++ = c2;
1302 coding->errors++;
1303 }
1304 }
1305 else
1306 {
1307 if (c == 0xFFFE)
1308 CODING_UTF_16_ENDIAN (coding)
1309 = endian = utf_16_big_endian;
1310 else if (c == 0xFEFF)
1311 CODING_UTF_16_ENDIAN (coding)
1312 = endian = utf_16_little_endian;
1313 else
1314 {
1315 CODING_UTF_16_ENDIAN (coding)
1316 = endian = utf_16_big_endian;
1317 src = src_base;
1318 }
1319 }
1320 CODING_UTF_16_BOM (coding) = utf_16_with_bom;
1321 }
1322
1323 while (1)
1324 {
1325 int c, c1, c2;
1326
1327 src_base = src;
1328 consumed_chars_base = consumed_chars;
1329
1330 if (charbuf + 2 >= charbuf_end)
1331 break;
1332
1333 ONE_MORE_BYTE (c1);
1334 ONE_MORE_BYTE (c2);
1335 c = (endian == utf_16_big_endian
1336 ? ((c1 << 16) | c2) : ((c2 << 16) | c1));
1337 if (surrogate)
1338 {
1339 if (! UTF_16_LOW_SURROGATE_P (c))
1340 {
1341 if (endian == utf_16_big_endian)
1342 c1 = surrogate >> 8, c2 = surrogate & 0xFF;
1343 else
1344 c1 = surrogate & 0xFF, c2 = surrogate >> 8;
1345 *charbuf++ = c1;
1346 *charbuf++ = c2;
1347 coding->errors++;
1348 if (UTF_16_HIGH_SURROGATE_P (c))
1349 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
1350 else
1351 *charbuf++ = c;
1352 }
1353 else
1354 {
1355 c = ((surrogate - 0xD800) << 10) | (c - 0xDC00);
1356 CODING_UTF_16_SURROGATE (coding) = surrogate = 0;
1357 *charbuf++ = c;
1358 }
1359 }
1360 else
1361 {
1362 if (UTF_16_HIGH_SURROGATE_P (c))
1363 CODING_UTF_16_SURROGATE (coding) = surrogate = c;
1364 else
1365 *charbuf++ = c;
1366 }
1367 }
1368
1369 no_more_source:
1370 coding->consumed_char += consumed_chars_base;
1371 coding->consumed = src_base - coding->source;
1372 coding->charbuf_used = charbuf - coding->charbuf;
1373}
1374
1375static int
1376encode_coding_utf_16 (coding)
1377 struct coding_system *coding;
1378{
1379 int multibytep = coding->dst_multibyte;
1380 int *charbuf = coding->charbuf;
1381 int *charbuf_end = charbuf + coding->charbuf_used;
1382 unsigned char *dst = coding->destination + coding->produced;
1383 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1384 int safe_room = 8;
1385 enum utf_16_bom_type bom = CODING_UTF_16_BOM (coding);
1386 int big_endian = CODING_UTF_16_ENDIAN (coding) == utf_16_big_endian;
1387 int produced_chars = 0;
1388 Lisp_Object attrs, eol_type, charset_list;
1389 int c;
1390
1391 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
1392
1393 if (bom == utf_16_with_bom)
1394 {
1395 ASSURE_DESTINATION (safe_room);
1396 if (big_endian)
1397 EMIT_TWO_BYTES (0xFF, 0xFE);
1398 else
1399 EMIT_TWO_BYTES (0xFE, 0xFF);
1400 CODING_UTF_16_BOM (coding) = utf_16_without_bom;
1401 }
1402
1403 while (charbuf < charbuf_end)
1404 {
1405 ASSURE_DESTINATION (safe_room);
1406 c = *charbuf++;
1407 if (c >= 0x110000)
1408 c = 0xFFFF;
1409
1410 if (c < 0x10000)
1411 {
1412 if (big_endian)
1413 EMIT_TWO_BYTES (c >> 8, c & 0xFF);
1414 else
1415 EMIT_TWO_BYTES (c & 0xFF, c >> 8);
1416 }
1417 else
1418 {
1419 int c1, c2;
1420
1421 c -= 0x10000;
1422 c1 = (c >> 10) + 0xD800;
1423 c2 = (c & 0x3FF) + 0xDC00;
1424 if (big_endian)
1425 EMIT_FOUR_BYTES (c1 >> 8, c1 & 0xFF, c2 >> 8, c2 & 0xFF);
1426 else
1427 EMIT_FOUR_BYTES (c1 & 0xFF, c1 >> 8, c2 & 0xFF, c2 >> 8);
1428 }
1429 }
1430 coding->result = CODING_RESULT_SUCCESS;
1431 coding->produced = dst - coding->destination;
1432 coding->produced_char += produced_chars;
1433 return 0;
1434}
1435
1436
1437/*** 6. Old Emacs' internal format (emacs-mule) ***/
518 1438
519/* Emacs' internal format for representation of multiple character 1439/* Emacs' internal format for representation of multiple character
520 sets is a kind of multi-byte encoding, i.e. characters are 1440 sets is a kind of multi-byte encoding, i.e. characters are
@@ -556,7 +1476,7 @@ coding_safe_chars (coding)
556 In that case, a sequence of one-byte codes has a slightly different 1476 In that case, a sequence of one-byte codes has a slightly different
557 form. 1477 form.
558 1478
559 Firstly, all characters in eight-bit-control are represented by 1479 At first, all characters in eight-bit-control are represented by
560 one-byte sequences which are their 8-bit code. 1480 one-byte sequences which are their 8-bit code.
561 1481
562 Next, character composition data are represented by the byte 1482 Next, character composition data are represented by the byte
@@ -565,12 +1485,12 @@ coding_safe_chars (coding)
565 METHOD is 0xF0 plus one of composition method (enum 1485 METHOD is 0xF0 plus one of composition method (enum
566 composition_method), 1486 composition_method),
567 1487
568 BYTES is 0xA0 plus the byte length of these composition data, 1488 BYTES is 0xA0 plus a byte length of this composition data,
569 1489
570 CHARS is 0xA0 plus the number of characters composed by these 1490 CHARS is 0x20 plus a number of characters composed by this
571 data, 1491 data,
572 1492
573 COMPONENTs are characters of multibyte form or composition 1493 COMPONENTs are characters of multibye form or composition
574 rules encoded by two-byte of ASCII codes. 1494 rules encoded by two-byte of ASCII codes.
575 1495
576 In addition, for backward compatibility, the following formats are 1496 In addition, for backward compatibility, the following formats are
@@ -587,568 +1507,515 @@ coding_safe_chars (coding)
587 represents a composition rule. 1507 represents a composition rule.
588 */ 1508 */
589 1509
590enum emacs_code_class_type emacs_code_class[256]; 1510char emacs_mule_bytes[256];
591 1511
592/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions". 1512/* Leading-code followed by extended leading-code. */
593 Check if a text is encoded in Emacs' internal format. If it is, 1513#define LEADING_CODE_PRIVATE_11 0x9A /* for private DIMENSION1 of 1-column */
594 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */ 1514#define LEADING_CODE_PRIVATE_12 0x9B /* for private DIMENSION1 of 2-column */
1515#define LEADING_CODE_PRIVATE_21 0x9C /* for private DIMENSION2 of 1-column */
1516#define LEADING_CODE_PRIVATE_22 0x9D /* for private DIMENSION2 of 2-column */
595 1517
596static int 1518
597detect_coding_emacs_mule (src, src_end, multibytep) 1519int
598 unsigned char *src, *src_end; 1520emacs_mule_char (coding, composition, nbytes, nchars)
599 int multibytep; 1521 struct coding_system *coding;
1522 int composition;
1523 int *nbytes, *nchars;
600{ 1524{
601 unsigned char c; 1525 unsigned char *src = coding->source + coding->consumed;
602 int composing = 0; 1526 unsigned char *src_end = coding->source + coding->src_bytes;
603 /* Dummy for ONE_MORE_BYTE. */ 1527 int multibytep = coding->src_multibyte;
604 struct coding_system dummy_coding; 1528 unsigned char *src_base = src;
605 struct coding_system *coding = &dummy_coding; 1529 struct charset *charset;
1530 unsigned code;
1531 int c;
1532 int consumed_chars = 0;
606 1533
607 while (1) 1534 ONE_MORE_BYTE (c);
1535 if (composition)
608 { 1536 {
609 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 1537 c -= 0x20;
610 1538 if (c == 0x80)
611 if (composing)
612 { 1539 {
1540 ONE_MORE_BYTE (c);
613 if (c < 0xA0) 1541 if (c < 0xA0)
614 composing = 0; 1542 goto invalid_code;
615 else if (c == 0xA0) 1543 *nbytes = src - src_base;
616 { 1544 *nchars = consumed_chars;
617 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 1545 return (c - 0x80);
618 c &= 0x7F;
619 }
620 else
621 c -= 0x20;
622 } 1546 }
1547 }
623 1548
624 if (c < 0x20) 1549 switch (emacs_mule_bytes[c])
1550 {
1551 case 2:
1552 if (! (charset = emacs_mule_charset[c]))
1553 goto invalid_code;
1554 ONE_MORE_BYTE (c);
1555 code = c & 0x7F;
1556 break;
1557
1558 case 3:
1559 if (c == LEADING_CODE_PRIVATE_11
1560 || c == LEADING_CODE_PRIVATE_12)
625 { 1561 {
626 if (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) 1562 ONE_MORE_BYTE (c);
627 return 0; 1563 if (! (charset = emacs_mule_charset[c]))
1564 goto invalid_code;
1565 ONE_MORE_BYTE (c);
1566 code = c & 0x7F;
628 } 1567 }
629 else if (c >= 0x80 && c < 0xA0) 1568 else
630 { 1569 {
631 if (c == 0x80) 1570 if (! (charset = emacs_mule_charset[c]))
632 /* Old leading code for a composite character. */ 1571 goto invalid_code;
633 composing = 1; 1572 ONE_MORE_BYTE (c);
634 else 1573 code = (c & 0x7F) << 7;
635 { 1574 ONE_MORE_BYTE (c);
636 unsigned char *src_base = src - 1; 1575 code |= c & 0x7F;
637 int bytes;
638
639 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base, src_end - src_base,
640 bytes))
641 return 0;
642 src = src_base + bytes;
643 }
644 } 1576 }
1577 break;
1578
1579 case 4:
1580 if (! (charset = emacs_mule_charset[c]))
1581 goto invalid_code;
1582 ONE_MORE_BYTE (c);
1583 code = (c & 0x7F) << 7;
1584 ONE_MORE_BYTE (c);
1585 code |= c & 0x7F;
1586 break;
1587
1588 case 1:
1589 code = c;
1590 charset = CHARSET_FROM_ID (ASCII_BYTE_P (code) ? charset_ascii
1591 : code < 0xA0 ? charset_8_bit_control
1592 : charset_8_bit_graphic);
1593 break;
1594
1595 default:
1596 abort ();
645 } 1597 }
646 label_end_of_loop: 1598 c = DECODE_CHAR (charset, code);
647 return CODING_CATEGORY_MASK_EMACS_MULE; 1599 if (c < 0)
1600 goto invalid_code;
1601 *nbytes = src - src_base;
1602 *nchars = consumed_chars;
1603 return c;
1604
1605 no_more_source:
1606 return -2;
1607
1608 invalid_code:
1609 return -1;
648} 1610}
649 1611
650 1612
651/* Record the starting position START and METHOD of one composition. */ 1613/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1614 Check if a text is encoded in `emacs-mule'. */
652 1615
653#define CODING_ADD_COMPOSITION_START(coding, start, method) \ 1616static int
654 do { \ 1617detect_coding_emacs_mule (coding, mask)
655 struct composition_data *cmp_data = coding->cmp_data; \ 1618 struct coding_system *coding;
656 int *data = cmp_data->data + cmp_data->used; \ 1619 int *mask;
657 coding->cmp_data_start = cmp_data->used; \ 1620{
658 data[0] = -1; \ 1621 unsigned char *src = coding->source, *src_base = src;
659 data[1] = cmp_data->char_offset + start; \ 1622 unsigned char *src_end = coding->source + coding->src_bytes;
660 data[3] = (int) method; \ 1623 int multibytep = coding->src_multibyte;
661 cmp_data->used += 4; \ 1624 int consumed_chars = 0;
662 } while (0) 1625 int c;
1626 int found = 0;
663 1627
664/* Record the ending position END of the current composition. */ 1628 /* A coding system of this category is always ASCII compatible. */
1629 src += coding->head_ascii;
665 1630
666#define CODING_ADD_COMPOSITION_END(coding, end) \ 1631 while (1)
667 do { \ 1632 {
668 struct composition_data *cmp_data = coding->cmp_data; \ 1633 ONE_MORE_BYTE (c);
669 int *data = cmp_data->data + coding->cmp_data_start; \ 1634
670 data[0] = cmp_data->used - coding->cmp_data_start; \ 1635 if (c == 0x80)
671 data[2] = cmp_data->char_offset + end; \ 1636 {
672 } while (0) 1637 /* Perhaps the start of composite character. We simple skip
1638 it because analyzing it is too heavy for detecting. But,
1639 at least, we check that the composite character
1640 constitues of more than 4 bytes. */
1641 unsigned char *src_base;
1642
1643 repeat:
1644 src_base = src;
1645 do
1646 {
1647 ONE_MORE_BYTE (c);
1648 }
1649 while (c >= 0xA0);
673 1650
674/* Record one COMPONENT (alternate character or composition rule). */ 1651 if (src - src_base <= 4)
1652 break;
1653 found = 1;
1654 if (c == 0x80)
1655 goto repeat;
1656 }
675 1657
676#define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \ 1658 if (c < 0x80)
677 (coding->cmp_data->data[coding->cmp_data->used++] = component) 1659 {
1660 if (c < 0x20
1661 && (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO))
1662 break;
1663 }
1664 else
1665 {
1666 unsigned char *src_base = src - 1;
678 1667
1668 do
1669 {
1670 ONE_MORE_BYTE (c);
1671 }
1672 while (c >= 0xA0);
1673 if (src - src_base != emacs_mule_bytes[*src_base])
1674 break;
1675 found = 1;
1676 }
1677 }
1678 *mask &= ~CATEGORY_MASK_EMACS_MULE;
1679 return 0;
679 1680
680/* Get one byte from a data pointed by SRC and increment SRC. If SRC 1681 no_more_source:
681 is not less than SRC_END, return -1 without incrementing Src. */ 1682 if (!found)
1683 return 0;
1684 *mask &= CATEGORY_MASK_EMACS_MULE;
1685 return 1;
1686}
682 1687
683#define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
684 1688
1689/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
685 1690
686/* Decode a character represented as a component of composition 1691/* Decode a character represented as a component of composition
687 sequence of Emacs 20 style at SRC. Set C to that character, store 1692 sequence of Emacs 20/21 style at SRC. Set C to that character and
688 its multibyte form sequence at P, and set P to the end of that 1693 update SRC to the head of next character (or an encoded composition
689 sequence. If no valid character is found, set C to -1. */ 1694 rule). If SRC doesn't points a composition component, set C to -1.
690 1695 If SRC points an invalid byte sequence, global exit by a return
691#define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \ 1696 value 0. */
692 do { \ 1697
693 int bytes; \ 1698#define DECODE_EMACS_MULE_COMPOSITION_CHAR(buf) \
694 \ 1699 if (1) \
695 c = SAFE_ONE_MORE_BYTE (); \ 1700 { \
696 if (c < 0) \ 1701 int c; \
697 break; \ 1702 int nbytes, nchars; \
698 if (CHAR_HEAD_P (c)) \
699 c = -1; \
700 else if (c == 0xA0) \
701 { \
702 c = SAFE_ONE_MORE_BYTE (); \
703 if (c < 0xA0) \
704 c = -1; \
705 else \
706 { \
707 c -= 0xA0; \
708 *p++ = c; \
709 } \
710 } \
711 else if (BASE_LEADING_CODE_P (c - 0x20)) \
712 { \
713 unsigned char *p0 = p; \
714 \ 1703 \
715 c -= 0x20; \ 1704 if (src == src_end) \
716 *p++ = c; \ 1705 break; \
717 bytes = BYTES_BY_CHAR_HEAD (c); \ 1706 c = emacs_mule_char (coding, 1, &nbytes, &nchars); \
718 while (--bytes) \ 1707 if (c < 0) \
719 { \ 1708 { \
720 c = SAFE_ONE_MORE_BYTE (); \ 1709 if (c == -2) \
721 if (c < 0) \ 1710 break; \
722 break; \ 1711 goto invalid_code; \
723 *p++ = c; \ 1712 } \
724 } \ 1713 *buf++ = c; \
725 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes)) \ 1714 src += nbytes; \
726 c = STRING_CHAR (p0, bytes); \ 1715 consumed_chars += nchars; \
727 else \ 1716 } \
728 c = -1; \ 1717 else
729 } \
730 else \
731 c = -1; \
732 } while (0)
733 1718
734 1719
735/* Decode a composition rule represented as a component of composition 1720/* Decode a composition rule represented as a component of composition
736 sequence of Emacs 20 style at SRC. Set C to the rule. If not 1721 sequence of Emacs 20 style at SRC. Set C to the rule. If SRC
737 valid rule is found, set C to -1. */ 1722 points an invalid byte sequence, set C to -1. */
738 1723
739#define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \ 1724#define DECODE_EMACS_MULE_COMPOSITION_RULE(buf) \
740 do { \ 1725 do { \
741 c = SAFE_ONE_MORE_BYTE (); \ 1726 int c, gref, nref; \
1727 \
1728 if (src < src_end) \
1729 goto invalid_code; \
1730 ONE_MORE_BYTE_NO_CHECK (c); \
742 c -= 0xA0; \ 1731 c -= 0xA0; \
743 if (c < 0 || c >= 81) \ 1732 if (c < 0 || c >= 81) \
744 c = -1; \ 1733 goto invalid_code; \
745 else \ 1734 \
746 { \ 1735 gref = c / 9, nref = c % 9; \
747 gref = c / 9, nref = c % 9; \ 1736 *buf++ = COMPOSITION_ENCODE_RULE (gref, nref); \
748 c = COMPOSITION_ENCODE_RULE (gref, nref); \
749 } \
750 } while (0) 1737 } while (0)
751 1738
752 1739
753/* Decode composition sequence encoded by `emacs-mule' at the source 1740#define ADD_COMPOSITION_DATA(buf, method, nchars) \
754 pointed by SRC. SRC_END is the end of source. Store information 1741 do { \
755 of the composition in CODING->cmp_data. 1742 *buf++ = -5; \
756 1743 *buf++ = coding->produced_char + char_offset; \
757 For backward compatibility, decode also a composition sequence of 1744 *buf++ = CODING_ANNOTATE_COMPOSITION_MASK; \
758 Emacs 20 style. In that case, the composition sequence contains 1745 *buf++ = method; \
759 characters that should be extracted into a buffer or string. Store 1746 *buf++ = nchars; \
760 those characters at *DESTINATION in multibyte form. 1747 } while (0)
761
762 If we encounter an invalid byte sequence, return 0.
763 If we encounter an insufficient source or destination, or
764 insufficient space in CODING->cmp_data, return 1.
765 Otherwise, return consumed bytes in the source.
766
767*/
768static INLINE int
769decode_composition_emacs_mule (coding, src, src_end,
770 destination, dst_end, dst_bytes)
771 struct coding_system *coding;
772 unsigned char *src, *src_end, **destination, *dst_end;
773 int dst_bytes;
774{
775 unsigned char *dst = *destination;
776 int method, data_len, nchars;
777 unsigned char *src_base = src++;
778 /* Store components of composition. */
779 int component[COMPOSITION_DATA_MAX_BUNCH_LENGTH];
780 int ncomponent;
781 /* Store multibyte form of characters to be composed. This is for
782 Emacs 20 style composition sequence. */
783 unsigned char buf[MAX_COMPOSITION_COMPONENTS * MAX_MULTIBYTE_LENGTH];
784 unsigned char *bufp = buf;
785 int c, i, gref, nref;
786 1748
787 if (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH
788 >= COMPOSITION_DATA_SIZE)
789 {
790 coding->result = CODING_FINISH_INSUFFICIENT_CMP;
791 return -1;
792 }
793 1749
794 ONE_MORE_BYTE (c); 1750#define DECODE_EMACS_MULE_21_COMPOSITION(c) \
795 if (c - 0xF0 >= COMPOSITION_RELATIVE 1751 do { \
796 && c - 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS) 1752 /* Emacs 21 style format. The first three bytes at SRC are \
797 { 1753 (METHOD - 0xF0), (BYTES - 0xA0), (CHARS - 0xA0), where BYTES is \
798 int with_rule; 1754 the byte length of this composition information, CHARS is the \
1755 number of characters composed by this composition. */ \
1756 enum composition_method method = c - 0xF0; \
1757 int consumed_chars_limit; \
1758 int nbytes, nchars; \
1759 \
1760 ONE_MORE_BYTE (c); \
1761 nbytes = c - 0xA0; \
1762 if (nbytes < 3) \
1763 goto invalid_code; \
1764 ONE_MORE_BYTE (c); \
1765 nchars = c - 0xA0; \
1766 ADD_COMPOSITION_DATA (charbuf, method, nchars); \
1767 consumed_chars_limit = consumed_chars_base + nbytes; \
1768 if (method != COMPOSITION_RELATIVE) \
1769 { \
1770 int i = 0; \
1771 while (consumed_chars < consumed_chars_limit) \
1772 { \
1773 if (i % 2 && method != COMPOSITION_WITH_ALTCHARS) \
1774 DECODE_EMACS_MULE_COMPOSITION_RULE (charbuf); \
1775 else \
1776 DECODE_EMACS_MULE_COMPOSITION_CHAR (charbuf); \
1777 } \
1778 if (consumed_chars < consumed_chars_limit) \
1779 goto invalid_code; \
1780 } \
1781 } while (0)
799 1782
800 method = c - 0xF0;
801 with_rule = (method == COMPOSITION_WITH_RULE
802 || method == COMPOSITION_WITH_RULE_ALTCHARS);
803 ONE_MORE_BYTE (c);
804 data_len = c - 0xA0;
805 if (data_len < 4
806 || src_base + data_len > src_end)
807 return 0;
808 ONE_MORE_BYTE (c);
809 nchars = c - 0xA0;
810 if (c < 1)
811 return 0;
812 for (ncomponent = 0; src < src_base + data_len; ncomponent++)
813 {
814 /* If it is longer than this, it can't be valid. */
815 if (ncomponent >= COMPOSITION_DATA_MAX_BUNCH_LENGTH)
816 return 0;
817 1783
818 if (ncomponent % 2 && with_rule) 1784#define DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION(c) \
819 { 1785 do { \
820 ONE_MORE_BYTE (gref); 1786 /* Emacs 20 style format for relative composition. */ \
821 gref -= 32; 1787 /* Store multibyte form of characters to be composed. */ \
822 ONE_MORE_BYTE (nref); 1788 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
823 nref -= 32; 1789 int *buf = components; \
824 c = COMPOSITION_ENCODE_RULE (gref, nref); 1790 int i, j; \
825 } 1791 \
826 else 1792 src = src_base; \
827 { 1793 ONE_MORE_BYTE (c); /* skip 0x80 */ \
828 int bytes; 1794 for (i = 0; i < MAX_COMPOSITION_COMPONENTS; i++) \
829 if (UNIBYTE_STR_AS_MULTIBYTE_P (src, src_end - src, bytes)) 1795 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
830 c = STRING_CHAR (src, bytes); 1796 if (i < 2) \
831 else 1797 goto invalid_code; \
832 c = *src, bytes = 1; 1798 ADD_COMPOSITION_DATA (charbuf, COMPOSITION_RELATIVE, i); \
833 src += bytes; 1799 for (j = 0; j < i; j++) \
834 } 1800 *charbuf++ = components[j]; \
835 component[ncomponent] = c; 1801 } while (0)
836 }
837 }
838 else
839 {
840 /* This may be an old Emacs 20 style format. See the comment at
841 the section 2 of this file. */
842 while (src < src_end && !CHAR_HEAD_P (*src)) src++;
843 if (src == src_end
844 && !(coding->mode & CODING_MODE_LAST_BLOCK))
845 goto label_end_of_loop;
846 1802
847 src_end = src;
848 src = src_base + 1;
849 if (c < 0xC0)
850 {
851 method = COMPOSITION_RELATIVE;
852 for (ncomponent = 0; ncomponent < MAX_COMPOSITION_COMPONENTS;)
853 {
854 DECODE_EMACS_MULE_COMPOSITION_CHAR (c, bufp);
855 if (c < 0)
856 break;
857 component[ncomponent++] = c;
858 }
859 if (ncomponent < 2)
860 return 0;
861 nchars = ncomponent;
862 }
863 else if (c == 0xFF)
864 {
865 method = COMPOSITION_WITH_RULE;
866 src++;
867 DECODE_EMACS_MULE_COMPOSITION_CHAR (c, bufp);
868 if (c < 0)
869 return 0;
870 component[0] = c;
871 for (ncomponent = 1;
872 ncomponent < MAX_COMPOSITION_COMPONENTS * 2 - 1;)
873 {
874 DECODE_EMACS_MULE_COMPOSITION_RULE (c);
875 if (c < 0)
876 break;
877 component[ncomponent++] = c;
878 DECODE_EMACS_MULE_COMPOSITION_CHAR (c, bufp);
879 if (c < 0)
880 break;
881 component[ncomponent++] = c;
882 }
883 if (ncomponent < 3)
884 return 0;
885 nchars = (ncomponent + 1) / 2;
886 }
887 else
888 return 0;
889 }
890 1803
891 if (buf == bufp || dst + (bufp - buf) <= (dst_bytes ? dst_end : src)) 1804#define DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION(c) \
892 { 1805 do { \
893 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, method); 1806 /* Emacs 20 style format for rule-base composition. */ \
894 for (i = 0; i < ncomponent; i++) 1807 /* Store multibyte form of characters to be composed. */ \
895 CODING_ADD_COMPOSITION_COMPONENT (coding, component[i]); 1808 int components[MAX_COMPOSITION_COMPONENTS * 2 - 1]; \
896 CODING_ADD_COMPOSITION_END (coding, coding->produced_char + nchars); 1809 int *buf = components; \
897 if (buf < bufp) 1810 int i, j; \
898 { 1811 \
899 unsigned char *p = buf; 1812 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
900 EMIT_BYTES (p, bufp); 1813 for (i = 0; i < MAX_COMPOSITION_COMPONENTS; i++) \
901 *destination += bufp - buf; 1814 { \
902 coding->produced_char += nchars; 1815 DECODE_EMACS_MULE_COMPOSITION_RULE (buf); \
903 } 1816 DECODE_EMACS_MULE_COMPOSITION_CHAR (buf); \
904 return (src - src_base); 1817 } \
905 } 1818 if (i < 1 || (buf - components) % 2 == 0) \
906 label_end_of_loop: 1819 goto invalid_code; \
907 return -1; 1820 if (charbuf + i + (i / 2) + 1 < charbuf_end) \
908} 1821 goto no_more_source; \
1822 ADD_COMPOSITION_DATA (buf, COMPOSITION_WITH_RULE, i); \
1823 for (j = 0; j < i; j++) \
1824 *charbuf++ = components[j]; \
1825 for (j = 0; j < i; j += 2) \
1826 *charbuf++ = components[j]; \
1827 } while (0)
909 1828
910/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
911 1829
912static void 1830static void
913decode_coding_emacs_mule (coding, source, destination, src_bytes, dst_bytes) 1831decode_coding_emacs_mule (coding)
914 struct coding_system *coding; 1832 struct coding_system *coding;
915 unsigned char *source, *destination;
916 int src_bytes, dst_bytes;
917{ 1833{
918 unsigned char *src = source; 1834 unsigned char *src = coding->source + coding->consumed;
919 unsigned char *src_end = source + src_bytes; 1835 unsigned char *src_end = coding->source + coding->src_bytes;
920 unsigned char *dst = destination;
921 unsigned char *dst_end = destination + dst_bytes;
922 /* SRC_BASE remembers the start position in source in each loop.
923 The loop will be exited when there's not enough source code, or
924 when there's not enough destination area to produce a
925 character. */
926 unsigned char *src_base; 1836 unsigned char *src_base;
1837 int *charbuf = coding->charbuf;
1838 int *charbuf_end = charbuf + coding->charbuf_size;
1839 int consumed_chars = 0, consumed_chars_base;
1840 int char_offset = 0;
1841 int multibytep = coding->src_multibyte;
1842 Lisp_Object attrs, eol_type, charset_list;
927 1843
928 coding->produced_char = 0; 1844 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
929 while ((src_base = src) < src_end) 1845
1846 while (1)
930 { 1847 {
931 unsigned char tmp[MAX_MULTIBYTE_LENGTH], *p; 1848 int c;
932 int bytes;
933 1849
934 if (*src == '\r') 1850 src_base = src;
935 { 1851 consumed_chars_base = consumed_chars;
936 int c = *src++;
937 1852
938 if (coding->eol_type == CODING_EOL_CR) 1853 if (charbuf >= charbuf_end)
939 c = '\n'; 1854 break;
940 else if (coding->eol_type == CODING_EOL_CRLF) 1855
1856 ONE_MORE_BYTE (c);
1857
1858 if (c < 0x80)
1859 {
1860 if (c == '\r')
941 { 1861 {
942 ONE_MORE_BYTE (c); 1862 if (EQ (eol_type, Qdos))
943 if (c != '\n')
944 { 1863 {
945 if (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL) 1864 if (src == src_end)
946 { 1865 goto no_more_source;
947 coding->result = CODING_FINISH_INCONSISTENT_EOL; 1866 if (*src == '\n')
948 goto label_end_of_loop; 1867 ONE_MORE_BYTE (c);
949 }
950 src--;
951 c = '\r';
952 } 1868 }
1869 else if (EQ (eol_type, Qmac))
1870 c = '\n';
953 } 1871 }
954 *dst++ = c; 1872 *charbuf++ = c;
955 coding->produced_char++; 1873 char_offset++;
956 continue;
957 } 1874 }
958 else if (*src == '\n') 1875 else if (c == 0x80)
959 { 1876 {
960 if ((coding->eol_type == CODING_EOL_CR 1877 if (charbuf + 5 + (MAX_COMPOSITION_COMPONENTS * 2) - 1 > charbuf_end)
961 || coding->eol_type == CODING_EOL_CRLF) 1878 break;
962 && coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL) 1879 ONE_MORE_BYTE (c);
963 { 1880 if (c - 0xF0 >= COMPOSITION_RELATIVE
964 coding->result = CODING_FINISH_INCONSISTENT_EOL; 1881 && c - 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS)
965 goto label_end_of_loop; 1882 DECODE_EMACS_MULE_21_COMPOSITION (c);
966 } 1883 else if (c < 0xC0)
967 *dst++ = *src++; 1884 DECODE_EMACS_MULE_20_RELATIVE_COMPOSITION (c);
968 coding->produced_char++; 1885 else if (c == 0xFF)
969 continue; 1886 DECODE_EMACS_MULE_20_RULEBASE_COMPOSITION (c);
1887 else
1888 goto invalid_code;
970 } 1889 }
971 else if (*src == 0x80) 1890 else if (c < 0xA0 && emacs_mule_bytes[c] > 1)
972 { 1891 {
973 /* Start of composition data. */ 1892 int nbytes, nchars;
974 int consumed = decode_composition_emacs_mule (coding, src, src_end, 1893 src--;
975 &dst, dst_end, 1894 c = emacs_mule_char (coding, 0, &nbytes, &nchars);
976 dst_bytes); 1895 if (c < 0)
977 if (consumed < 0)
978 goto label_end_of_loop;
979 else if (consumed > 0)
980 { 1896 {
981 src += consumed; 1897 if (c == -2)
982 continue; 1898 break;
1899 goto invalid_code;
983 } 1900 }
984 bytes = CHAR_STRING (*src, tmp); 1901 *charbuf++ = c;
985 p = tmp; 1902 char_offset++;
986 src++;
987 }
988 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src, src_end - src, bytes))
989 {
990 p = src;
991 src += bytes;
992 }
993 else
994 {
995 bytes = CHAR_STRING (*src, tmp);
996 p = tmp;
997 src++;
998 }
999 if (dst + bytes >= (dst_bytes ? dst_end : src))
1000 {
1001 coding->result = CODING_FINISH_INSUFFICIENT_DST;
1002 break;
1003 } 1903 }
1004 while (bytes--) *dst++ = *p++; 1904 continue;
1005 coding->produced_char++; 1905
1906 invalid_code:
1907 src = src_base;
1908 consumed_chars = consumed_chars_base;
1909 ONE_MORE_BYTE (c);
1910 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
1911 coding->errors++;
1006 } 1912 }
1007 label_end_of_loop:
1008 coding->consumed = coding->consumed_char = src_base - source;
1009 coding->produced = dst - destination;
1010}
1011 1913
1914 no_more_source:
1915 coding->consumed_char += consumed_chars_base;
1916 coding->consumed = src_base - coding->source;
1917 coding->charbuf_used = charbuf - coding->charbuf;
1918}
1012 1919
1013/* Encode composition data stored at DATA into a special byte sequence
1014 starting by 0x80. Update CODING->cmp_data_start and maybe
1015 CODING->cmp_data for the next call. */
1016
1017#define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1018 do { \
1019 unsigned char buf[1024], *p0 = buf, *p; \
1020 int len = data[0]; \
1021 int i; \
1022 \
1023 buf[0] = 0x80; \
1024 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1025 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1026 p = buf + 4; \
1027 if (data[3] == COMPOSITION_WITH_RULE \
1028 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1029 { \
1030 p += CHAR_STRING (data[4], p); \
1031 for (i = 5; i < len; i += 2) \
1032 { \
1033 int gref, nref; \
1034 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1035 *p++ = 0x20 + gref; \
1036 *p++ = 0x20 + nref; \
1037 p += CHAR_STRING (data[i + 1], p); \
1038 } \
1039 } \
1040 else \
1041 { \
1042 for (i = 4; i < len; i++) \
1043 p += CHAR_STRING (data[i], p); \
1044 } \
1045 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1046 \
1047 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1048 { \
1049 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1050 goto label_end_of_loop; \
1051 } \
1052 while (p0 < p) \
1053 *dst++ = *p0++; \
1054 coding->cmp_data_start += data[0]; \
1055 if (coding->cmp_data_start == coding->cmp_data->used \
1056 && coding->cmp_data->next) \
1057 { \
1058 coding->cmp_data = coding->cmp_data->next; \
1059 coding->cmp_data_start = 0; \
1060 } \
1061 } while (0)
1062 1920
1921#define EMACS_MULE_LEADING_CODES(id, codes) \
1922 do { \
1923 if (id < 0xA0) \
1924 codes[0] = id, codes[1] = 0; \
1925 else if (id < 0xE0) \
1926 codes[0] = 0x9A, codes[1] = id; \
1927 else if (id < 0xF0) \
1928 codes[0] = 0x9B, codes[1] = id; \
1929 else if (id < 0xF5) \
1930 codes[0] = 0x9C, codes[1] = id; \
1931 else \
1932 codes[0] = 0x9D, codes[1] = id; \
1933 } while (0);
1063 1934
1064static void encode_eol P_ ((struct coding_system *, unsigned char *,
1065 unsigned char *, int, int));
1066 1935
1067static void 1936static int
1068encode_coding_emacs_mule (coding, source, destination, src_bytes, dst_bytes) 1937encode_coding_emacs_mule (coding)
1069 struct coding_system *coding; 1938 struct coding_system *coding;
1070 unsigned char *source, *destination;
1071 int src_bytes, dst_bytes;
1072{ 1939{
1073 unsigned char *src = source; 1940 int multibytep = coding->dst_multibyte;
1074 unsigned char *src_end = source + src_bytes; 1941 int *charbuf = coding->charbuf;
1075 unsigned char *dst = destination; 1942 int *charbuf_end = charbuf + coding->charbuf_used;
1076 unsigned char *dst_end = destination + dst_bytes; 1943 unsigned char *dst = coding->destination + coding->produced;
1077 unsigned char *src_base; 1944 unsigned char *dst_end = coding->destination + coding->dst_bytes;
1945 int safe_room = 8;
1946 unsigned char *adjusted_dst_end =dst_end - 8;
1947 int produced_chars = 0;
1948 Lisp_Object attrs, eol_type, charset_list;
1078 int c; 1949 int c;
1079 int char_offset;
1080 int *data;
1081 1950
1082 Lisp_Object translation_table; 1951 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
1083 1952
1084 translation_table = Qnil; 1953 while (charbuf < charbuf_end)
1085
1086 /* Optimization for the case that there's no composition. */
1087 if (!coding->cmp_data || coding->cmp_data->used == 0)
1088 { 1954 {
1089 encode_eol (coding, source, destination, src_bytes, dst_bytes); 1955 ASSURE_DESTINATION (safe_room);
1090 return; 1956 c = *charbuf++;
1091 } 1957 if (ASCII_CHAR_P (c))
1092 1958 EMIT_ONE_ASCII_BYTE (c);
1093 char_offset = coding->cmp_data->char_offset; 1959 else
1094 data = coding->cmp_data->data + coding->cmp_data_start;
1095 while (1)
1096 {
1097 src_base = src;
1098
1099 /* If SRC starts a composition, encode the information about the
1100 composition in advance. */
1101 if (coding->cmp_data_start < coding->cmp_data->used
1102 && char_offset + coding->consumed_char == data[1])
1103 {
1104 ENCODE_COMPOSITION_EMACS_MULE (coding, data);
1105 char_offset = coding->cmp_data->char_offset;
1106 data = coding->cmp_data->data + coding->cmp_data_start;
1107 }
1108
1109 ONE_MORE_CHAR (c);
1110 if (c == '\n' && (coding->eol_type == CODING_EOL_CRLF
1111 || coding->eol_type == CODING_EOL_CR))
1112 { 1960 {
1113 if (coding->eol_type == CODING_EOL_CRLF) 1961 struct charset *charset;
1114 EMIT_TWO_BYTES ('\r', c); 1962 unsigned code;
1963 int dimension;
1964 int emacs_mule_id;
1965 unsigned char leading_codes[2];
1966
1967 charset = char_charset (c, charset_list, &code);
1968 if (! charset)
1969 {
1970 c = coding->default_char;
1971 if (ASCII_CHAR_P (c))
1972 {
1973 EMIT_ONE_ASCII_BYTE (c);
1974 continue;
1975 }
1976 charset = char_charset (c, charset_list, &code);
1977 }
1978 dimension = CHARSET_DIMENSION (charset);
1979 emacs_mule_id = CHARSET_EMACS_MULE_ID (charset);
1980 EMACS_MULE_LEADING_CODES (emacs_mule_id, leading_codes);
1981 EMIT_ONE_BYTE (leading_codes[0]);
1982 if (leading_codes[1])
1983 EMIT_ONE_BYTE (leading_codes[1]);
1984 if (dimension == 1)
1985 EMIT_ONE_BYTE (code);
1115 else 1986 else
1116 EMIT_ONE_BYTE ('\r'); 1987 {
1988 EMIT_ONE_BYTE (code >> 8);
1989 EMIT_ONE_BYTE (code & 0xFF);
1990 }
1117 } 1991 }
1118 else if (SINGLE_BYTE_CHAR_P (c))
1119 EMIT_ONE_BYTE (c);
1120 else
1121 EMIT_BYTES (src_base, src);
1122 coding->consumed_char++;
1123 } 1992 }
1124 label_end_of_loop: 1993 coding->result = CODING_RESULT_SUCCESS;
1125 coding->consumed = src_base - source; 1994 coding->produced_char += produced_chars;
1126 coding->produced = coding->produced_char = dst - destination; 1995 coding->produced = dst - coding->destination;
1127 return; 1996 return 0;
1128} 1997}
1129 1998
1130 1999
1131/*** 3. ISO2022 handlers ***/ 2000/*** 7. ISO2022 handlers ***/
1132 2001
1133/* The following note describes the coding system ISO2022 briefly. 2002/* The following note describes the coding system ISO2022 briefly.
1134 Since the intention of this note is to help understand the 2003 Since the intention of this note is to help understand the
1135 functions in this file, some parts are NOT ACCURATE or are OVERLY 2004 functions in this file, some parts are NOT ACCURATE or OVERLY
1136 SIMPLIFIED. For thorough understanding, please refer to the 2005 SIMPLIFIED. For thorough understanding, please refer to the
1137 original document of ISO2022. This is equivalent to the standard 2006 original document of ISO2022.
1138 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
1139 2007
1140 ISO2022 provides many mechanisms to encode several character sets 2008 ISO2022 provides many mechanisms to encode several character sets
1141 in 7-bit and 8-bit environments. For 7-bit environments, all text 2009 in 7-bit and 8-bit environments. For 7-bite environments, all text
1142 is encoded using bytes less than 128. This may make the encoded 2010 is encoded using bytes less than 128. This may make the encoded
1143 text a little bit longer, but the text passes more easily through 2011 text a little bit longer, but the text passes more easily through
1144 several types of gateway, some of which strip off the MSB (Most 2012 several gateways, some of which strip off MSB (Most Signigant Bit).
1145 Significant Bit).
1146 2013
1147 There are two kinds of character sets: control character sets and 2014 There are two kinds of character sets: control character set and
1148 graphic character sets. The former contain control characters such 2015 graphic character set. The former contains control characters such
1149 as `newline' and `escape' to provide control functions (control 2016 as `newline' and `escape' to provide control functions (control
1150 functions are also provided by escape sequences). The latter 2017 functions are also provided by escape sequences). The latter
1151 contain graphic characters such as 'A' and '-'. Emacs recognizes 2018 contains graphic characters such as 'A' and '-'. Emacs recognizes
1152 two control character sets and many graphic character sets. 2019 two control character sets and many graphic character sets.
1153 2020
1154 Graphic character sets are classified into one of the following 2021 Graphic character sets are classified into one of the following
@@ -1160,14 +2027,14 @@ encode_coding_emacs_mule (coding, source, destination, src_bytes, dst_bytes)
1160 - DIMENSION2_CHARS96 2027 - DIMENSION2_CHARS96
1161 2028
1162 In addition, each character set is assigned an identification tag, 2029 In addition, each character set is assigned an identification tag,
1163 unique for each set, called the "final character" (denoted as <F> 2030 unique for each set, called "final character" (denoted as <F>
1164 hereafter). The <F> of each character set is decided by ECMA(*) 2031 hereafter). The <F> of each character set is decided by ECMA(*)
1165 when it is registered in ISO. The code range of <F> is 0x30..0x7F 2032 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1166 (0x30..0x3F are for private use only). 2033 (0x30..0x3F are for private use only).
1167 2034
1168 Note (*): ECMA = European Computer Manufacturers Association 2035 Note (*): ECMA = European Computer Manufacturers Association
1169 2036
1170 Here are examples of graphic character sets [NAME(<F>)]: 2037 Here are examples of graphic character set [NAME(<F>)]:
1171 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ... 2038 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1172 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ... 2039 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1173 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ... 2040 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
@@ -1258,13 +2125,13 @@ encode_coding_emacs_mule (coding, source, destination, src_bytes, dst_bytes)
1258 7-bit environment, non-locking-shift, and non-single-shift. 2125 7-bit environment, non-locking-shift, and non-single-shift.
1259 2126
1260 Note (**): If <F> is '@', 'A', or 'B', the intermediate character 2127 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
1261 '(' can be omitted. We refer to this as "short-form" hereafter. 2128 '(' must be omitted. We refer to this as "short-form" hereafter.
1262 2129
1263 Now you may notice that there are a lot of ways of encoding the 2130 Now you may notice that there are a lot of ways for encoding the
1264 same multilingual text in ISO2022. Actually, there exist many 2131 same multilingual text in ISO2022. Actually, there exist many
1265 coding systems such as Compound Text (used in X11's inter client 2132 coding systems such as Compound Text (used in X11's inter client
1266 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR 2133 communication, ISO-2022-JP (used in Japanese internet), ISO-2022-KR
1267 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian 2134 (used in Korean internet), EUC (Extended UNIX Code, used in Asian
1268 localized platforms), and all of these are variants of ISO2022. 2135 localized platforms), and all of these are variants of ISO2022.
1269 2136
1270 In addition to the above, Emacs handles two more kinds of escape 2137 In addition to the above, Emacs handles two more kinds of escape
@@ -1286,19 +2153,19 @@ encode_coding_emacs_mule (coding, source, destination, src_bytes, dst_bytes)
1286 o ESC '3' -- start relative composition with alternate chars (**) 2153 o ESC '3' -- start relative composition with alternate chars (**)
1287 o ESC '4' -- start rule-base composition with alternate chars (**) 2154 o ESC '4' -- start rule-base composition with alternate chars (**)
1288 Since these are not standard escape sequences of any ISO standard, 2155 Since these are not standard escape sequences of any ISO standard,
1289 the use of them with these meanings is restricted to Emacs only. 2156 the use of them for these meaning is restricted to Emacs only.
1290 2157
1291 (*) This form is used only in Emacs 20.5 and older versions, 2158 (*) This form is used only in Emacs 20.5 and the older versions,
1292 but the newer versions can safely decode it. 2159 but the newer versions can safely decode it.
1293 (**) This form is used only in Emacs 21.1 and newer versions, 2160 (**) This form is used only in Emacs 21.1 and the newer versions,
1294 and the older versions can't decode it. 2161 and the older versions can't decode it.
1295 2162
1296 Here's a list of example usages of these composition escape 2163 Here's a list of examples usages of these composition escape
1297 sequences (categorized by `enum composition_method'). 2164 sequences (categorized by `enum composition_method').
1298 2165
1299 COMPOSITION_RELATIVE: 2166 COMPOSITION_RELATIVE:
1300 ESC 0 CHAR [ CHAR ] ESC 1 2167 ESC 0 CHAR [ CHAR ] ESC 1
1301 COMPOSITION_WITH_RULE: 2168 COMPOSITOIN_WITH_RULE:
1302 ESC 2 CHAR [ RULE CHAR ] ESC 1 2169 ESC 2 CHAR [ RULE CHAR ] ESC 1
1303 COMPOSITION_WITH_ALTCHARS: 2170 COMPOSITION_WITH_ALTCHARS:
1304 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 2171 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
@@ -1307,80 +2174,159 @@ encode_coding_emacs_mule (coding, source, destination, src_bytes, dst_bytes)
1307 2174
1308enum iso_code_class_type iso_code_class[256]; 2175enum iso_code_class_type iso_code_class[256];
1309 2176
1310#define CHARSET_OK(idx, charset, c) \ 2177#define SAFE_CHARSET_P(coding, id) \
1311 (coding_system_table[idx] \ 2178 ((id) <= (coding)->max_charset_id \
1312 && (charset == CHARSET_ASCII \ 2179 && (coding)->safe_charsets[id] >= 0)
1313 || (safe_chars = coding_safe_chars (coding_system_table[idx]), \ 2180
1314 CODING_SAFE_CHAR_P (safe_chars, c))) \ 2181
1315 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \ 2182#define SHIFT_OUT_OK(category) \
1316 charset) \ 2183 (CODING_ISO_INITIAL (&coding_categories[category], 1) >= 0)
1317 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION)) 2184
2185static void
2186setup_iso_safe_charsets (Lisp_Object attrs)
2187{
2188 Lisp_Object charset_list, safe_charsets;
2189 Lisp_Object request;
2190 Lisp_Object reg_usage;
2191 Lisp_Object tail;
2192 int reg94, reg96;
2193 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
2194 int max_charset_id;
2195
2196 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
2197 if ((flags & CODING_ISO_FLAG_FULL_SUPPORT)
2198 && ! EQ (charset_list, Viso_2022_charset_list))
2199 {
2200 CODING_ATTR_CHARSET_LIST (attrs)
2201 = charset_list = Viso_2022_charset_list;
2202 ASET (attrs, coding_attr_safe_charsets, Qnil);
2203 }
2204
2205 if (STRINGP (AREF (attrs, coding_attr_safe_charsets)))
2206 return;
2207
2208 max_charset_id = 0;
2209 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2210 {
2211 int id = XINT (XCAR (tail));
2212 if (max_charset_id < id)
2213 max_charset_id = id;
2214 }
2215
2216 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
2217 make_number (255));
2218 request = AREF (attrs, coding_attr_iso_request);
2219 reg_usage = AREF (attrs, coding_attr_iso_usage);
2220 reg94 = XINT (XCAR (reg_usage));
2221 reg96 = XINT (XCDR (reg_usage));
2222
2223 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
2224 {
2225 Lisp_Object id;
2226 Lisp_Object reg;
2227 struct charset *charset;
2228
2229 id = XCAR (tail);
2230 charset = CHARSET_FROM_ID (XINT (id));
2231 reg = Fcdr (Fassq (request, id));
2232 if (! NILP (reg))
2233 XSTRING (safe_charsets)->data[XINT (id)] = XINT (reg);
2234 else if (charset->iso_chars_96)
2235 {
2236 if (reg96 < 4)
2237 XSTRING (safe_charsets)->data[XINT (id)] = reg96;
2238 }
2239 else
2240 {
2241 if (reg94 < 4)
2242 XSTRING (safe_charsets)->data[XINT (id)] = reg94;
2243 }
2244 }
2245 ASET (attrs, coding_attr_safe_charsets, safe_charsets);
2246}
1318 2247
1319#define SHIFT_OUT_OK(idx) \
1320 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1321 2248
1322/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions". 2249/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1323 Check if a text is encoded in ISO2022. If it is, return an 2250 Check if a text is encoded in ISO2022. If it is, returns an
1324 integer in which appropriate flag bits any of: 2251 integer in which appropriate flag bits any of:
1325 CODING_CATEGORY_MASK_ISO_7 2252 CATEGORY_MASK_ISO_7
1326 CODING_CATEGORY_MASK_ISO_7_TIGHT 2253 CATEGORY_MASK_ISO_7_TIGHT
1327 CODING_CATEGORY_MASK_ISO_8_1 2254 CATEGORY_MASK_ISO_8_1
1328 CODING_CATEGORY_MASK_ISO_8_2 2255 CATEGORY_MASK_ISO_8_2
1329 CODING_CATEGORY_MASK_ISO_7_ELSE 2256 CATEGORY_MASK_ISO_7_ELSE
1330 CODING_CATEGORY_MASK_ISO_8_ELSE 2257 CATEGORY_MASK_ISO_8_ELSE
1331 are set. If a code which should never appear in ISO2022 is found, 2258 are set. If a code which should never appear in ISO2022 is found,
1332 returns 0. */ 2259 returns 0. */
1333 2260
1334static int 2261static int
1335detect_coding_iso2022 (src, src_end, multibytep) 2262detect_coding_iso_2022 (coding, mask)
1336 unsigned char *src, *src_end; 2263 struct coding_system *coding;
1337 int multibytep; 2264 int *mask;
1338{ 2265{
1339 int mask = CODING_CATEGORY_MASK_ISO; 2266 unsigned char *src = coding->source, *src_base = src;
1340 int mask_found = 0; 2267 unsigned char *src_end = coding->source + coding->src_bytes;
2268 int multibytep = coding->src_multibyte;
2269 int mask_iso = CATEGORY_MASK_ISO;
2270 int mask_found = 0, mask_8bit_found = 0;
1341 int reg[4], shift_out = 0, single_shifting = 0; 2271 int reg[4], shift_out = 0, single_shifting = 0;
1342 int c, c1, charset; 2272 int id;
1343 /* Dummy for ONE_MORE_BYTE. */ 2273 int c, c1;
1344 struct coding_system dummy_coding; 2274 int consumed_chars = 0;
1345 struct coding_system *coding = &dummy_coding; 2275 int i;
1346 Lisp_Object safe_chars; 2276
2277 for (i = coding_category_iso_7; i <= coding_category_iso_8_else; i++)
2278 {
2279 struct coding_system *this = &(coding_categories[i]);
2280 Lisp_Object attrs, val;
2281
2282 attrs = CODING_ID_ATTRS (this->id);
2283 if (CODING_ISO_FLAGS (this) & CODING_ISO_FLAG_FULL_SUPPORT
2284 && ! EQ (CODING_ATTR_SAFE_CHARSETS (attrs), Viso_2022_charset_list))
2285 setup_iso_safe_charsets (attrs);
2286 val = CODING_ATTR_SAFE_CHARSETS (attrs);
2287 this->max_charset_id = XSTRING (val)->size - 1;
2288 this->safe_charsets = (char *) XSTRING (val)->data;
2289 }
2290
2291 /* A coding system of this category is always ASCII compatible. */
2292 src += coding->head_ascii;
1347 2293
1348 reg[0] = CHARSET_ASCII, reg[1] = reg[2] = reg[3] = -1; 2294 reg[0] = charset_ascii, reg[1] = reg[2] = reg[3] = -1;
1349 while (mask && src < src_end) 2295 while (mask_iso && src < src_end)
1350 { 2296 {
1351 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 2297 ONE_MORE_BYTE (c);
1352 switch (c) 2298 switch (c)
1353 { 2299 {
1354 case ISO_CODE_ESC: 2300 case ISO_CODE_ESC:
1355 if (inhibit_iso_escape_detection) 2301 if (inhibit_iso_escape_detection)
1356 break; 2302 break;
1357 single_shifting = 0; 2303 single_shifting = 0;
1358 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 2304 ONE_MORE_BYTE (c);
1359 if (c >= '(' && c <= '/') 2305 if (c >= '(' && c <= '/')
1360 { 2306 {
1361 /* Designation sequence for a charset of dimension 1. */ 2307 /* Designation sequence for a charset of dimension 1. */
1362 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1, multibytep); 2308 ONE_MORE_BYTE (c1);
1363 if (c1 < ' ' || c1 >= 0x80 2309 if (c1 < ' ' || c1 >= 0x80
1364 || (charset = iso_charset_table[0][c >= ','][c1]) < 0) 2310 || (id = iso_charset_table[0][c >= ','][c1]) < 0)
1365 /* Invalid designation sequence. Just ignore. */ 2311 /* Invalid designation sequence. Just ignore. */
1366 break; 2312 break;
1367 reg[(c - '(') % 4] = charset; 2313 reg[(c - '(') % 4] = id;
1368 } 2314 }
1369 else if (c == '$') 2315 else if (c == '$')
1370 { 2316 {
1371 /* Designation sequence for a charset of dimension 2. */ 2317 /* Designation sequence for a charset of dimension 2. */
1372 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 2318 ONE_MORE_BYTE (c);
1373 if (c >= '@' && c <= 'B') 2319 if (c >= '@' && c <= 'B')
1374 /* Designation for JISX0208.1978, GB2312, or JISX0208. */ 2320 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
1375 reg[0] = charset = iso_charset_table[1][0][c]; 2321 reg[0] = id = iso_charset_table[1][0][c];
1376 else if (c >= '(' && c <= '/') 2322 else if (c >= '(' && c <= '/')
1377 { 2323 {
1378 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1, multibytep); 2324 ONE_MORE_BYTE (c1);
1379 if (c1 < ' ' || c1 >= 0x80 2325 if (c1 < ' ' || c1 >= 0x80
1380 || (charset = iso_charset_table[1][c >= ','][c1]) < 0) 2326 || (id = iso_charset_table[1][c >= ','][c1]) < 0)
1381 /* Invalid designation sequence. Just ignore. */ 2327 /* Invalid designation sequence. Just ignore. */
1382 break; 2328 break;
1383 reg[(c - '(') % 4] = charset; 2329 reg[(c - '(') % 4] = id;
1384 } 2330 }
1385 else 2331 else
1386 /* Invalid designation sequence. Just ignore. */ 2332 /* Invalid designation sequence. Just ignore. */
@@ -1389,38 +2335,44 @@ detect_coding_iso2022 (src, src_end, multibytep)
1389 else if (c == 'N' || c == 'O') 2335 else if (c == 'N' || c == 'O')
1390 { 2336 {
1391 /* ESC <Fe> for SS2 or SS3. */ 2337 /* ESC <Fe> for SS2 or SS3. */
1392 mask &= CODING_CATEGORY_MASK_ISO_7_ELSE; 2338 mask_iso &= CATEGORY_MASK_ISO_7_ELSE;
1393 break; 2339 break;
1394 } 2340 }
1395 else if (c >= '0' && c <= '4') 2341 else if (c >= '0' && c <= '4')
1396 { 2342 {
1397 /* ESC <Fp> for start/end composition. */ 2343 /* ESC <Fp> for start/end composition. */
1398 mask_found |= CODING_CATEGORY_MASK_ISO; 2344 mask_found |= CATEGORY_MASK_ISO;
1399 break; 2345 break;
1400 } 2346 }
1401 else 2347 else
1402 /* Invalid escape sequence. Just ignore. */ 2348 {
1403 break; 2349 /* Invalid escape sequence. */
2350 mask_iso &= ~CATEGORY_MASK_ISO_ESCAPE;
2351 break;
2352 }
1404 2353
1405 /* We found a valid designation sequence for CHARSET. */ 2354 /* We found a valid designation sequence for CHARSET. */
1406 mask &= ~CODING_CATEGORY_MASK_ISO_8BIT; 2355 mask_iso &= ~CATEGORY_MASK_ISO_8BIT;
1407 c = MAKE_CHAR (charset, 0, 0); 2356 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7],
1408 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7, charset, c)) 2357 id))
1409 mask_found |= CODING_CATEGORY_MASK_ISO_7; 2358 mask_found |= CATEGORY_MASK_ISO_7;
1410 else 2359 else
1411 mask &= ~CODING_CATEGORY_MASK_ISO_7; 2360 mask_iso &= ~CATEGORY_MASK_ISO_7;
1412 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT, charset, c)) 2361 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_tight],
1413 mask_found |= CODING_CATEGORY_MASK_ISO_7_TIGHT; 2362 id))
2363 mask_found |= CATEGORY_MASK_ISO_7_TIGHT;
1414 else 2364 else
1415 mask &= ~CODING_CATEGORY_MASK_ISO_7_TIGHT; 2365 mask_iso &= ~CATEGORY_MASK_ISO_7_TIGHT;
1416 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE, charset, c)) 2366 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_7_else],
1417 mask_found |= CODING_CATEGORY_MASK_ISO_7_ELSE; 2367 id))
2368 mask_found |= CATEGORY_MASK_ISO_7_ELSE;
1418 else 2369 else
1419 mask &= ~CODING_CATEGORY_MASK_ISO_7_ELSE; 2370 mask_iso &= ~CATEGORY_MASK_ISO_7_ELSE;
1420 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE, charset, c)) 2371 if (SAFE_CHARSET_P (&coding_categories[coding_category_iso_8_else],
1421 mask_found |= CODING_CATEGORY_MASK_ISO_8_ELSE; 2372 id))
2373 mask_found |= CATEGORY_MASK_ISO_8_ELSE;
1422 else 2374 else
1423 mask &= ~CODING_CATEGORY_MASK_ISO_8_ELSE; 2375 mask_iso &= ~CATEGORY_MASK_ISO_8_ELSE;
1424 break; 2376 break;
1425 2377
1426 case ISO_CODE_SO: 2378 case ISO_CODE_SO:
@@ -1429,15 +2381,15 @@ detect_coding_iso2022 (src, src_end, multibytep)
1429 single_shifting = 0; 2381 single_shifting = 0;
1430 if (shift_out == 0 2382 if (shift_out == 0
1431 && (reg[1] >= 0 2383 && (reg[1] >= 0
1432 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE) 2384 || SHIFT_OUT_OK (coding_category_iso_7_else)
1433 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE))) 2385 || SHIFT_OUT_OK (coding_category_iso_8_else)))
1434 { 2386 {
1435 /* Locking shift out. */ 2387 /* Locking shift out. */
1436 mask &= ~CODING_CATEGORY_MASK_ISO_7BIT; 2388 mask_iso &= ~CATEGORY_MASK_ISO_7BIT;
1437 mask_found |= CODING_CATEGORY_MASK_ISO_SHIFT; 2389 mask_found |= CATEGORY_MASK_ISO_ELSE;
1438 } 2390 }
1439 break; 2391 break;
1440 2392
1441 case ISO_CODE_SI: 2393 case ISO_CODE_SI:
1442 if (inhibit_iso_escape_detection) 2394 if (inhibit_iso_escape_detection)
1443 break; 2395 break;
@@ -1445,8 +2397,8 @@ detect_coding_iso2022 (src, src_end, multibytep)
1445 if (shift_out == 1) 2397 if (shift_out == 1)
1446 { 2398 {
1447 /* Locking shift in. */ 2399 /* Locking shift in. */
1448 mask &= ~CODING_CATEGORY_MASK_ISO_7BIT; 2400 mask_iso &= ~CATEGORY_MASK_ISO_7BIT;
1449 mask_found |= CODING_CATEGORY_MASK_ISO_SHIFT; 2401 mask_found |= CATEGORY_MASK_ISO_ELSE;
1450 } 2402 }
1451 break; 2403 break;
1452 2404
@@ -1455,31 +2407,31 @@ detect_coding_iso2022 (src, src_end, multibytep)
1455 case ISO_CODE_SS2: 2407 case ISO_CODE_SS2:
1456 case ISO_CODE_SS3: 2408 case ISO_CODE_SS3:
1457 { 2409 {
1458 int newmask = CODING_CATEGORY_MASK_ISO_8_ELSE; 2410 int newmask = CATEGORY_MASK_ISO_8_ELSE;
1459 2411
1460 if (inhibit_iso_escape_detection) 2412 if (inhibit_iso_escape_detection)
1461 break; 2413 break;
1462 if (c != ISO_CODE_CSI) 2414 if (c != ISO_CODE_CSI)
1463 { 2415 {
1464 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_1]->flags 2416 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
1465 & CODING_FLAG_ISO_SINGLE_SHIFT) 2417 & CODING_ISO_FLAG_SINGLE_SHIFT)
1466 newmask |= CODING_CATEGORY_MASK_ISO_8_1; 2418 newmask |= CATEGORY_MASK_ISO_8_1;
1467 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_2]->flags 2419 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_2])
1468 & CODING_FLAG_ISO_SINGLE_SHIFT) 2420 & CODING_ISO_FLAG_SINGLE_SHIFT)
1469 newmask |= CODING_CATEGORY_MASK_ISO_8_2; 2421 newmask |= CATEGORY_MASK_ISO_8_2;
1470 single_shifting = 1; 2422 single_shifting = 1;
1471 } 2423 }
1472 if (VECTORP (Vlatin_extra_code_table) 2424 if (VECTORP (Vlatin_extra_code_table)
1473 && !NILP (XVECTOR (Vlatin_extra_code_table)->contents[c])) 2425 && !NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
1474 { 2426 {
1475 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_1]->flags 2427 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
1476 & CODING_FLAG_ISO_LATIN_EXTRA) 2428 & CODING_ISO_FLAG_LATIN_EXTRA)
1477 newmask |= CODING_CATEGORY_MASK_ISO_8_1; 2429 newmask |= CATEGORY_MASK_ISO_8_1;
1478 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_2]->flags 2430 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_2])
1479 & CODING_FLAG_ISO_LATIN_EXTRA) 2431 & CODING_ISO_FLAG_LATIN_EXTRA)
1480 newmask |= CODING_CATEGORY_MASK_ISO_8_2; 2432 newmask |= CATEGORY_MASK_ISO_8_2;
1481 } 2433 }
1482 mask &= newmask; 2434 mask_iso &= newmask;
1483 mask_found |= newmask; 2435 mask_found |= newmask;
1484 } 2436 }
1485 break; 2437 break;
@@ -1493,18 +2445,19 @@ detect_coding_iso2022 (src, src_end, multibytep)
1493 else if (c < 0xA0) 2445 else if (c < 0xA0)
1494 { 2446 {
1495 single_shifting = 0; 2447 single_shifting = 0;
2448 mask_8bit_found = 1;
1496 if (VECTORP (Vlatin_extra_code_table) 2449 if (VECTORP (Vlatin_extra_code_table)
1497 && !NILP (XVECTOR (Vlatin_extra_code_table)->contents[c])) 2450 && !NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
1498 { 2451 {
1499 int newmask = 0; 2452 int newmask = 0;
1500 2453
1501 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_1]->flags 2454 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_1])
1502 & CODING_FLAG_ISO_LATIN_EXTRA) 2455 & CODING_ISO_FLAG_LATIN_EXTRA)
1503 newmask |= CODING_CATEGORY_MASK_ISO_8_1; 2456 newmask |= CATEGORY_MASK_ISO_8_1;
1504 if (coding_system_table[CODING_CATEGORY_IDX_ISO_8_2]->flags 2457 if (CODING_ISO_FLAGS (&coding_categories[coding_category_iso_8_2])
1505 & CODING_FLAG_ISO_LATIN_EXTRA) 2458 & CODING_ISO_FLAG_LATIN_EXTRA)
1506 newmask |= CODING_CATEGORY_MASK_ISO_8_2; 2459 newmask |= CATEGORY_MASK_ISO_8_2;
1507 mask &= newmask; 2460 mask_iso &= newmask;
1508 mask_found |= newmask; 2461 mask_found |= newmask;
1509 } 2462 }
1510 else 2463 else
@@ -1512,183 +2465,180 @@ detect_coding_iso2022 (src, src_end, multibytep)
1512 } 2465 }
1513 else 2466 else
1514 { 2467 {
1515 mask &= ~(CODING_CATEGORY_MASK_ISO_7BIT 2468 mask_iso &= ~(CATEGORY_MASK_ISO_7BIT
1516 | CODING_CATEGORY_MASK_ISO_7_ELSE); 2469 | CATEGORY_MASK_ISO_7_ELSE);
1517 mask_found |= CODING_CATEGORY_MASK_ISO_8_1; 2470 mask_found |= CATEGORY_MASK_ISO_8_1;
2471 mask_8bit_found = 1;
1518 /* Check the length of succeeding codes of the range 2472 /* Check the length of succeeding codes of the range
1519 0xA0..0FF. If the byte length is odd, we exclude 2473 0xA0..0FF. If the byte length is odd, we exclude
1520 CODING_CATEGORY_MASK_ISO_8_2. We can check this only 2474 CATEGORY_MASK_ISO_8_2. We can check this only
1521 when we are not single shifting. */ 2475 when we are not single shifting. */
1522 if (!single_shifting 2476 if (!single_shifting
1523 && mask & CODING_CATEGORY_MASK_ISO_8_2) 2477 && mask_iso & CATEGORY_MASK_ISO_8_2)
1524 { 2478 {
1525 int i = 1; 2479 int i = 1;
1526 while (src < src_end) 2480 while (src < src_end)
1527 { 2481 {
1528 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 2482 ONE_MORE_BYTE (c);
1529 if (c < 0xA0) 2483 if (c < 0xA0)
1530 break; 2484 break;
1531 i++; 2485 i++;
1532 } 2486 }
1533 2487
1534 if (i & 1 && src < src_end) 2488 if (i & 1 && src < src_end)
1535 mask &= ~CODING_CATEGORY_MASK_ISO_8_2; 2489 mask_iso &= ~CATEGORY_MASK_ISO_8_2;
1536 else 2490 else
1537 mask_found |= CODING_CATEGORY_MASK_ISO_8_2; 2491 mask_found |= CATEGORY_MASK_ISO_8_2;
1538 } 2492 }
1539 } 2493 }
1540 break; 2494 break;
1541 } 2495 }
1542 } 2496 }
1543 label_end_of_loop: 2497 no_more_source:
1544 return (mask & mask_found); 2498 if (!mask_iso)
2499 {
2500 *mask &= ~CATEGORY_MASK_ISO;
2501 return 0;
2502 }
2503 if (!mask_found)
2504 return 0;
2505 *mask &= mask_iso & mask_found;
2506 if (! mask_8bit_found)
2507 *mask &= ~(CATEGORY_MASK_ISO_8BIT | CATEGORY_MASK_ISO_8_ELSE);
2508 return 1;
1545} 2509}
1546 2510
1547/* Decode a character of which charset is CHARSET, the 1st position
1548 code is C1, the 2nd position code is C2, and return the decoded
1549 character code. If the variable `translation_table' is non-nil,
1550 returned the translated code. */
1551
1552#define DECODE_ISO_CHARACTER(charset, c1, c2) \
1553 (NILP (translation_table) \
1554 ? MAKE_CHAR (charset, c1, c2) \
1555 : translate_char (translation_table, -1, charset, c1, c2))
1556 2511
1557/* Set designation state into CODING. */ 2512/* Set designation state into CODING. */
1558#define DECODE_DESIGNATION(reg, dimension, chars, final_char) \ 2513#define DECODE_DESIGNATION(reg, dim, chars_96, final) \
1559 do { \ 2514 do { \
1560 int charset, c; \ 2515 int id, prev; \
1561 \ 2516 \
1562 if (final_char < '0' || final_char >= 128) \ 2517 if (final < '0' || final >= 128 \
1563 goto label_invalid_code; \ 2518 || ((id = ISO_CHARSET_TABLE (dim, chars_96, final)) < 0) \
1564 charset = ISO_CHARSET_TABLE (make_number (dimension), \ 2519 || !SAFE_CHARSET_P (coding, id)) \
1565 make_number (chars), \ 2520 { \
1566 make_number (final_char)); \ 2521 CODING_ISO_DESIGNATION (coding, reg) = -2; \
1567 c = MAKE_CHAR (charset, 0, 0); \ 2522 goto invalid_code; \
1568 if (charset >= 0 \ 2523 } \
1569 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \ 2524 prev = CODING_ISO_DESIGNATION (coding, reg); \
1570 || CODING_SAFE_CHAR_P (safe_chars, c))) \ 2525 CODING_ISO_DESIGNATION (coding, reg) = id; \
1571 { \ 2526 /* If there was an invalid designation to REG previously, and this \
1572 if (coding->spec.iso2022.last_invalid_designation_register == 0 \ 2527 designation is ASCII to REG, we should keep this designation \
1573 && reg == 0 \ 2528 sequence. */ \
1574 && charset == CHARSET_ASCII) \ 2529 if (prev == -2 && id == charset_ascii) \
1575 { \ 2530 goto invalid_code; \
1576 /* We should insert this designation sequence as is so \
1577 that it is surely written back to a file. */ \
1578 coding->spec.iso2022.last_invalid_designation_register = -1; \
1579 goto label_invalid_code; \
1580 } \
1581 coding->spec.iso2022.last_invalid_designation_register = -1; \
1582 if ((coding->mode & CODING_MODE_DIRECTION) \
1583 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1584 charset = CHARSET_REVERSE_CHARSET (charset); \
1585 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1586 } \
1587 else \
1588 { \
1589 coding->spec.iso2022.last_invalid_designation_register = reg; \
1590 goto label_invalid_code; \
1591 } \
1592 } while (0) 2531 } while (0)
1593 2532
1594/* Allocate a memory block for storing information about compositions.
1595 The block is chained to the already allocated blocks. */
1596 2533
1597void 2534#define MAYBE_FINISH_COMPOSITION() \
1598coding_allocate_composition_data (coding, char_offset) 2535 do { \
1599 struct coding_system *coding; 2536 int i; \
1600 int char_offset; 2537 if (composition_state == COMPOSING_NO) \
1601{ 2538 break; \
1602 struct composition_data *cmp_data 2539 /* It is assured that we have enough room for producing \
1603 = (struct composition_data *) xmalloc (sizeof *cmp_data); 2540 characters stored in the table `components'. */ \
1604 2541 if (charbuf + component_idx > charbuf_end) \
1605 cmp_data->char_offset = char_offset; 2542 goto no_more_source; \
1606 cmp_data->used = 0; 2543 composition_state = COMPOSING_NO; \
1607 cmp_data->prev = coding->cmp_data; 2544 if (method == COMPOSITION_RELATIVE \
1608 cmp_data->next = NULL; 2545 || method == COMPOSITION_WITH_ALTCHARS) \
1609 if (coding->cmp_data) 2546 { \
1610 coding->cmp_data->next = cmp_data; 2547 for (i = 0; i < component_idx; i++) \
1611 coding->cmp_data = cmp_data; 2548 *charbuf++ = components[i]; \
1612 coding->cmp_data_start = 0; 2549 char_offset += component_idx; \
1613} 2550 } \
2551 else \
2552 { \
2553 for (i = 0; i < component_idx; i += 2) \
2554 *charbuf++ = components[i]; \
2555 char_offset += (component_idx / 2) + 1; \
2556 } \
2557 } while (0)
2558
1614 2559
1615/* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4. 2560/* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1616 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1 2561 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1617 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1 2562 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1618 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1 2563 ESC 3 : altchar composition : ESC 3 CHAR ... ESC 0 CHAR ... ESC 1
1619 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1 2564 ESC 4 : alt&rule composition : ESC 4 CHAR RULE ... CHAR ESC 0 CHAR ... ESC 1
1620 */ 2565 */
1621 2566
1622#define DECODE_COMPOSITION_START(c1) \ 2567#define DECODE_COMPOSITION_START(c1) \
1623 do { \ 2568 do { \
1624 if (coding->composing == COMPOSITION_DISABLED) \ 2569 if (c1 == '0' \
1625 { \ 2570 && composition_state == COMPOSING_COMPONENT_CHAR) \
1626 *dst++ = ISO_CODE_ESC; \ 2571 { \
1627 *dst++ = c1 & 0x7f; \ 2572 component_len = component_idx; \
1628 coding->produced_char += 2; \ 2573 composition_state = COMPOSING_CHAR; \
1629 } \ 2574 } \
1630 else if (!COMPOSING_P (coding)) \ 2575 else \
1631 { \ 2576 { \
1632 /* This is surely the start of a composition. We must be sure \ 2577 unsigned char *p; \
1633 that coding->cmp_data has enough space to store the \ 2578 \
1634 information about the composition. If not, terminate the \ 2579 MAYBE_FINISH_COMPOSITION (); \
1635 current decoding loop, allocate one more memory block for \ 2580 if (charbuf + MAX_COMPOSITION_COMPONENTS > charbuf_end) \
1636 coding->cmp_data in the caller, then start the decoding \ 2581 goto no_more_source; \
1637 loop again. We can't allocate memory here directly because \ 2582 for (p = src; p < src_end - 1; p++) \
1638 it may cause buffer/string relocation. */ \ 2583 if (*p == ISO_CODE_ESC && p[1] == '1') \
1639 if (!coding->cmp_data \ 2584 break; \
1640 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \ 2585 if (p == src_end - 1) \
1641 >= COMPOSITION_DATA_SIZE)) \ 2586 { \
1642 { \ 2587 if (coding->mode & CODING_MODE_LAST_BLOCK) \
1643 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \ 2588 goto invalid_code; \
1644 goto label_end_of_loop; \ 2589 goto no_more_source; \
1645 } \ 2590 } \
1646 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \ 2591 \
1647 : c1 == '2' ? COMPOSITION_WITH_RULE \ 2592 /* This is surely the start of a composition. */ \
1648 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \ 2593 method = (c1 == '0' ? COMPOSITION_RELATIVE \
1649 : COMPOSITION_WITH_RULE_ALTCHARS); \ 2594 : c1 == '2' ? COMPOSITION_WITH_RULE \
1650 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \ 2595 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1651 coding->composing); \ 2596 : COMPOSITION_WITH_RULE_ALTCHARS); \
1652 coding->composition_rule_follows = 0; \ 2597 composition_state = (c1 <= '2' ? COMPOSING_CHAR \
1653 } \ 2598 : COMPOSING_COMPONENT_CHAR); \
1654 else \ 2599 component_idx = component_len = 0; \
1655 { \ 2600 } \
1656 /* We are already handling a composition. If the method is \
1657 the following two, the codes following the current escape \
1658 sequence are actual characters stored in a buffer. */ \
1659 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1660 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1661 { \
1662 coding->composing = COMPOSITION_RELATIVE; \
1663 coding->composition_rule_follows = 0; \
1664 } \
1665 } \
1666 } while (0) 2601 } while (0)
1667 2602
1668/* Handle composition end sequence ESC 1. */
1669 2603
1670#define DECODE_COMPOSITION_END(c1) \ 2604/* Handle compositoin end sequence ESC 1. */
2605
2606#define DECODE_COMPOSITION_END() \
1671 do { \ 2607 do { \
1672 if (! COMPOSING_P (coding)) \ 2608 int nchars = (component_len > 0 ? component_idx - component_len \
2609 : method == COMPOSITION_RELATIVE ? component_idx \
2610 : (component_idx + 1) / 2); \
2611 int i; \
2612 int *saved_charbuf = charbuf; \
2613 \
2614 ADD_COMPOSITION_DATA (charbuf, method, nchars); \
2615 if (method != COMPOSITION_RELATIVE) \
1673 { \ 2616 { \
1674 *dst++ = ISO_CODE_ESC; \ 2617 if (component_len == 0) \
1675 *dst++ = c1; \ 2618 for (i = 0; i < component_idx; i++) \
1676 coding->produced_char += 2; \ 2619 *charbuf++ = components[i]; \
2620 else \
2621 for (i = 0; i < component_len; i++) \
2622 *charbuf++ = components[i]; \
2623 *saved_charbuf = saved_charbuf - charbuf; \
1677 } \ 2624 } \
2625 if (method == COMPOSITION_WITH_RULE) \
2626 for (i = 0; i < component_idx; i += 2, char_offset++) \
2627 *charbuf++ = components[i]; \
1678 else \ 2628 else \
1679 { \ 2629 for (i = component_len; i < component_idx; i++, char_offset++) \
1680 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \ 2630 *charbuf++ = components[i]; \
1681 coding->composing = COMPOSITION_NO; \ 2631 coding->annotated = 1; \
1682 } \ 2632 composition_state = COMPOSING_NO; \
1683 } while (0) 2633 } while (0)
1684 2634
2635
1685/* Decode a composition rule from the byte C1 (and maybe one more byte 2636/* Decode a composition rule from the byte C1 (and maybe one more byte
1686 from SRC) and store one encoded composition rule in 2637 from SRC) and store one encoded composition rule in
1687 coding->cmp_data. */ 2638 coding->cmp_data. */
1688 2639
1689#define DECODE_COMPOSITION_RULE(c1) \ 2640#define DECODE_COMPOSITION_RULE(c1) \
1690 do { \ 2641 do { \
1691 int rule = 0; \
1692 (c1) -= 32; \ 2642 (c1) -= 32; \
1693 if (c1 < 81) /* old format (before ver.21) */ \ 2643 if (c1 < 81) /* old format (before ver.21) */ \
1694 { \ 2644 { \
@@ -1696,172 +2646,168 @@ coding_allocate_composition_data (coding, char_offset)
1696 int nref = (c1) % 9; \ 2646 int nref = (c1) % 9; \
1697 if (gref == 4) gref = 10; \ 2647 if (gref == 4) gref = 10; \
1698 if (nref == 4) nref = 10; \ 2648 if (nref == 4) nref = 10; \
1699 rule = COMPOSITION_ENCODE_RULE (gref, nref); \ 2649 c1 = COMPOSITION_ENCODE_RULE (gref, nref); \
1700 } \ 2650 } \
1701 else if (c1 < 93) /* new format (after ver.21) */ \ 2651 else if (c1 < 93) /* new format (after ver.21) */ \
1702 { \ 2652 { \
1703 ONE_MORE_BYTE (c2); \ 2653 ONE_MORE_BYTE (c2); \
1704 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \ 2654 c1 = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1705 } \ 2655 } \
1706 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \ 2656 else \
1707 coding->composition_rule_follows = 0; \ 2657 c1 = 0; \
1708 } while (0) 2658 } while (0)
1709 2659
1710 2660
1711/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */ 2661/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1712 2662
1713static void 2663static void
1714decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes) 2664decode_coding_iso_2022 (coding)
1715 struct coding_system *coding; 2665 struct coding_system *coding;
1716 unsigned char *source, *destination;
1717 int src_bytes, dst_bytes;
1718{ 2666{
1719 unsigned char *src = source; 2667 unsigned char *src = coding->source + coding->consumed;
1720 unsigned char *src_end = source + src_bytes; 2668 unsigned char *src_end = coding->source + coding->src_bytes;
1721 unsigned char *dst = destination;
1722 unsigned char *dst_end = destination + dst_bytes;
1723 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1724 int charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0);
1725 int charset1 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 1);
1726 /* SRC_BASE remembers the start position in source in each loop.
1727 The loop will be exited when there's not enough source code
1728 (within macro ONE_MORE_BYTE), or when there's not enough
1729 destination area to produce a character (within macro
1730 EMIT_CHAR). */
1731 unsigned char *src_base; 2669 unsigned char *src_base;
1732 int c, charset; 2670 int *charbuf = coding->charbuf;
1733 Lisp_Object translation_table; 2671 int *charbuf_end = charbuf + coding->charbuf_size - 4;
1734 Lisp_Object safe_chars; 2672 int consumed_chars = 0, consumed_chars_base;
1735 2673 int char_offset = 0;
1736 safe_chars = coding_safe_chars (coding); 2674 int multibytep = coding->src_multibyte;
1737 2675 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1738 if (NILP (Venable_character_translation)) 2676 int charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
1739 translation_table = Qnil; 2677 int charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
1740 else 2678 struct charset *charset;
1741 { 2679 int c;
1742 translation_table = coding->translation_table_for_decode; 2680 /* For handling composition sequence. */
1743 if (NILP (translation_table)) 2681#define COMPOSING_NO 0
1744 translation_table = Vstandard_translation_table_for_decode; 2682#define COMPOSING_CHAR 1
1745 } 2683#define COMPOSING_RULE 2
1746 2684#define COMPOSING_COMPONENT_CHAR 3
1747 coding->result = CODING_FINISH_NORMAL; 2685#define COMPOSING_COMPONENT_RULE 4
2686
2687 int composition_state = COMPOSING_NO;
2688 enum composition_method method;
2689 int components[MAX_COMPOSITION_COMPONENTS * 2 + 1];
2690 int component_idx;
2691 int component_len;
2692 Lisp_Object attrs, eol_type, charset_list;
2693
2694 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
2695 setup_iso_safe_charsets (attrs);
1748 2696
1749 while (1) 2697 while (1)
1750 { 2698 {
1751 int c1, c2; 2699 int c1, c2;
1752 2700
1753 src_base = src; 2701 src_base = src;
2702 consumed_chars_base = consumed_chars;
2703
2704 if (charbuf >= charbuf_end)
2705 break;
2706
1754 ONE_MORE_BYTE (c1); 2707 ONE_MORE_BYTE (c1);
1755 2708
1756 /* We produce no character or one character. */ 2709 /* We produce no character or one character. */
1757 switch (iso_code_class [c1]) 2710 switch (iso_code_class [c1])
1758 { 2711 {
1759 case ISO_0x20_or_0x7F: 2712 case ISO_0x20_or_0x7F:
1760 if (COMPOSING_P (coding) && coding->composition_rule_follows) 2713 if (composition_state != COMPOSING_NO)
1761 { 2714 {
1762 DECODE_COMPOSITION_RULE (c1); 2715 if (composition_state == COMPOSING_RULE
1763 continue; 2716 || composition_state == COMPOSING_COMPONENT_RULE)
2717 {
2718 DECODE_COMPOSITION_RULE (c1);
2719 components[component_idx++] = c1;
2720 composition_state--;
2721 continue;
2722 }
2723 else if (method == COMPOSITION_WITH_RULE)
2724 composition_state = COMPOSING_RULE;
2725 else if (method == COMPOSITION_WITH_RULE_ALTCHARS
2726 && composition_state == COMPOSING_COMPONENT_CHAR)
2727 composition_state = COMPOSING_COMPONENT_CHAR;
1764 } 2728 }
1765 if (charset0 < 0 || CHARSET_CHARS (charset0) == 94) 2729 if (charset_id_0 < 0
2730 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_0)))
1766 { 2731 {
1767 /* This is SPACE or DEL. */ 2732 /* This is SPACE or DEL. */
1768 charset = CHARSET_ASCII; 2733 charset = CHARSET_FROM_ID (charset_ascii);
1769 break; 2734 break;
1770 } 2735 }
1771 /* This is a graphic character, we fall down ... */ 2736 /* This is a graphic character, we fall down ... */
1772 2737
1773 case ISO_graphic_plane_0: 2738 case ISO_graphic_plane_0:
1774 if (COMPOSING_P (coding) && coding->composition_rule_follows) 2739 if (composition_state == COMPOSING_RULE)
1775 { 2740 {
1776 DECODE_COMPOSITION_RULE (c1); 2741 DECODE_COMPOSITION_RULE (c1);
1777 continue; 2742 components[component_idx++] = c1;
2743 composition_state = COMPOSING_CHAR;
1778 } 2744 }
1779 charset = charset0; 2745 charset = CHARSET_FROM_ID (charset_id_0);
1780 break; 2746 break;
1781 2747
1782 case ISO_0xA0_or_0xFF: 2748 case ISO_0xA0_or_0xFF:
1783 if (charset1 < 0 || CHARSET_CHARS (charset1) == 94 2749 if (charset_id_1 < 0
1784 || coding->flags & CODING_FLAG_ISO_SEVEN_BITS) 2750 || ! CHARSET_ISO_CHARS_96 (CHARSET_FROM_ID (charset_id_1))
1785 goto label_invalid_code; 2751 || CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS)
2752 goto invalid_code;
1786 /* This is a graphic character, we fall down ... */ 2753 /* This is a graphic character, we fall down ... */
1787 2754
1788 case ISO_graphic_plane_1: 2755 case ISO_graphic_plane_1:
1789 if (charset1 < 0) 2756 if (charset_id_1 < 0)
1790 goto label_invalid_code; 2757 goto invalid_code;
1791 charset = charset1; 2758 charset = CHARSET_FROM_ID (charset_id_1);
1792 break; 2759 break;
1793 2760
1794 case ISO_control_0:
1795 if (COMPOSING_P (coding))
1796 DECODE_COMPOSITION_END ('1');
1797
1798 /* All ISO2022 control characters in this class have the
1799 same representation in Emacs internal format. */
1800 if (c1 == '\n'
1801 && (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
1802 && (coding->eol_type == CODING_EOL_CR
1803 || coding->eol_type == CODING_EOL_CRLF))
1804 {
1805 coding->result = CODING_FINISH_INCONSISTENT_EOL;
1806 goto label_end_of_loop;
1807 }
1808 charset = CHARSET_ASCII;
1809 break;
1810
1811 case ISO_control_1:
1812 if (COMPOSING_P (coding))
1813 DECODE_COMPOSITION_END ('1');
1814 goto label_invalid_code;
1815
1816 case ISO_carriage_return: 2761 case ISO_carriage_return:
1817 if (COMPOSING_P (coding)) 2762 if (c1 == '\r')
1818 DECODE_COMPOSITION_END ('1');
1819
1820 if (coding->eol_type == CODING_EOL_CR)
1821 c1 = '\n';
1822 else if (coding->eol_type == CODING_EOL_CRLF)
1823 { 2763 {
1824 ONE_MORE_BYTE (c1); 2764 if (EQ (eol_type, Qdos))
1825 if (c1 != ISO_CODE_LF)
1826 { 2765 {
1827 if (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL) 2766 if (src == src_end)
1828 { 2767 goto no_more_source;
1829 coding->result = CODING_FINISH_INCONSISTENT_EOL; 2768 if (*src == '\n')
1830 goto label_end_of_loop; 2769 ONE_MORE_BYTE (c1);
1831 }
1832 src--;
1833 c1 = '\r';
1834 } 2770 }
2771 else if (EQ (eol_type, Qmac))
2772 c1 = '\n';
1835 } 2773 }
1836 charset = CHARSET_ASCII; 2774 /* fall through */
2775
2776 case ISO_control_0:
2777 MAYBE_FINISH_COMPOSITION ();
2778 charset = CHARSET_FROM_ID (charset_ascii);
1837 break; 2779 break;
1838 2780
2781 case ISO_control_1:
2782 MAYBE_FINISH_COMPOSITION ();
2783 goto invalid_code;
2784
1839 case ISO_shift_out: 2785 case ISO_shift_out:
1840 if (! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT) 2786 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
1841 || CODING_SPEC_ISO_DESIGNATION (coding, 1) < 0) 2787 || CODING_ISO_DESIGNATION (coding, 1) < 0)
1842 goto label_invalid_code; 2788 goto invalid_code;
1843 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; 2789 CODING_ISO_INVOCATION (coding, 0) = 1;
1844 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0); 2790 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
1845 continue; 2791 continue;
1846 2792
1847 case ISO_shift_in: 2793 case ISO_shift_in:
1848 if (! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT)) 2794 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT))
1849 goto label_invalid_code; 2795 goto invalid_code;
1850 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; 2796 CODING_ISO_INVOCATION (coding, 0) = 0;
1851 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0); 2797 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
1852 continue; 2798 continue;
1853 2799
1854 case ISO_single_shift_2_7: 2800 case ISO_single_shift_2_7:
1855 case ISO_single_shift_2: 2801 case ISO_single_shift_2:
1856 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT)) 2802 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
1857 goto label_invalid_code; 2803 goto invalid_code;
1858 /* SS2 is handled as an escape sequence of ESC 'N' */ 2804 /* SS2 is handled as an escape sequence of ESC 'N' */
1859 c1 = 'N'; 2805 c1 = 'N';
1860 goto label_escape_sequence; 2806 goto label_escape_sequence;
1861 2807
1862 case ISO_single_shift_3: 2808 case ISO_single_shift_3:
1863 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT)) 2809 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT))
1864 goto label_invalid_code; 2810 goto invalid_code;
1865 /* SS2 is handled as an escape sequence of ESC 'O' */ 2811 /* SS2 is handled as an escape sequence of ESC 'O' */
1866 c1 = 'O'; 2812 c1 = 'O';
1867 goto label_escape_sequence; 2813 goto label_escape_sequence;
@@ -1874,7 +2820,7 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
1874 case ISO_escape: 2820 case ISO_escape:
1875 ONE_MORE_BYTE (c1); 2821 ONE_MORE_BYTE (c1);
1876 label_escape_sequence: 2822 label_escape_sequence:
1877 /* Escape sequences handled by Emacs are invocation, 2823 /* Escape sequences handled here are invocation,
1878 designation, direction specification, and character 2824 designation, direction specification, and character
1879 composition specification. */ 2825 composition specification. */
1880 switch (c1) 2826 switch (c1)
@@ -1882,89 +2828,93 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
1882 case '&': /* revision of following character set */ 2828 case '&': /* revision of following character set */
1883 ONE_MORE_BYTE (c1); 2829 ONE_MORE_BYTE (c1);
1884 if (!(c1 >= '@' && c1 <= '~')) 2830 if (!(c1 >= '@' && c1 <= '~'))
1885 goto label_invalid_code; 2831 goto invalid_code;
1886 ONE_MORE_BYTE (c1); 2832 ONE_MORE_BYTE (c1);
1887 if (c1 != ISO_CODE_ESC) 2833 if (c1 != ISO_CODE_ESC)
1888 goto label_invalid_code; 2834 goto invalid_code;
1889 ONE_MORE_BYTE (c1); 2835 ONE_MORE_BYTE (c1);
1890 goto label_escape_sequence; 2836 goto label_escape_sequence;
1891 2837
1892 case '$': /* designation of 2-byte character set */ 2838 case '$': /* designation of 2-byte character set */
1893 if (! (coding->flags & CODING_FLAG_ISO_DESIGNATION)) 2839 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
1894 goto label_invalid_code; 2840 goto invalid_code;
1895 ONE_MORE_BYTE (c1); 2841 ONE_MORE_BYTE (c1);
1896 if (c1 >= '@' && c1 <= 'B') 2842 if (c1 >= '@' && c1 <= 'B')
1897 { /* designation of JISX0208.1978, GB2312.1980, 2843 { /* designation of JISX0208.1978, GB2312.1980,
1898 or JISX0208.1980 */ 2844 or JISX0208.1980 */
1899 DECODE_DESIGNATION (0, 2, 94, c1); 2845 DECODE_DESIGNATION (0, 2, 0, c1);
1900 } 2846 }
1901 else if (c1 >= 0x28 && c1 <= 0x2B) 2847 else if (c1 >= 0x28 && c1 <= 0x2B)
1902 { /* designation of DIMENSION2_CHARS94 character set */ 2848 { /* designation of DIMENSION2_CHARS94 character set */
1903 ONE_MORE_BYTE (c2); 2849 ONE_MORE_BYTE (c2);
1904 DECODE_DESIGNATION (c1 - 0x28, 2, 94, c2); 2850 DECODE_DESIGNATION (c1 - 0x28, 2, 0, c2);
1905 } 2851 }
1906 else if (c1 >= 0x2C && c1 <= 0x2F) 2852 else if (c1 >= 0x2C && c1 <= 0x2F)
1907 { /* designation of DIMENSION2_CHARS96 character set */ 2853 { /* designation of DIMENSION2_CHARS96 character set */
1908 ONE_MORE_BYTE (c2); 2854 ONE_MORE_BYTE (c2);
1909 DECODE_DESIGNATION (c1 - 0x2C, 2, 96, c2); 2855 DECODE_DESIGNATION (c1 - 0x2C, 2, 1, c2);
1910 } 2856 }
1911 else 2857 else
1912 goto label_invalid_code; 2858 goto invalid_code;
1913 /* We must update these variables now. */ 2859 /* We must update these variables now. */
1914 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0); 2860 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
1915 charset1 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 1); 2861 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
1916 continue; 2862 continue;
1917 2863
1918 case 'n': /* invocation of locking-shift-2 */ 2864 case 'n': /* invocation of locking-shift-2 */
1919 if (! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT) 2865 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
1920 || CODING_SPEC_ISO_DESIGNATION (coding, 2) < 0) 2866 || CODING_ISO_DESIGNATION (coding, 2) < 0)
1921 goto label_invalid_code; 2867 goto invalid_code;
1922 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; 2868 CODING_ISO_INVOCATION (coding, 0) = 2;
1923 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0); 2869 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
1924 continue; 2870 continue;
1925 2871
1926 case 'o': /* invocation of locking-shift-3 */ 2872 case 'o': /* invocation of locking-shift-3 */
1927 if (! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT) 2873 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LOCKING_SHIFT)
1928 || CODING_SPEC_ISO_DESIGNATION (coding, 3) < 0) 2874 || CODING_ISO_DESIGNATION (coding, 3) < 0)
1929 goto label_invalid_code; 2875 goto invalid_code;
1930 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; 2876 CODING_ISO_INVOCATION (coding, 0) = 3;
1931 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0); 2877 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
1932 continue; 2878 continue;
1933 2879
1934 case 'N': /* invocation of single-shift-2 */ 2880 case 'N': /* invocation of single-shift-2 */
1935 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT) 2881 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
1936 || CODING_SPEC_ISO_DESIGNATION (coding, 2) < 0) 2882 || CODING_ISO_DESIGNATION (coding, 2) < 0)
1937 goto label_invalid_code; 2883 goto invalid_code;
1938 charset = CODING_SPEC_ISO_DESIGNATION (coding, 2); 2884 charset = CHARSET_FROM_ID (CODING_ISO_DESIGNATION (coding, 2));
1939 ONE_MORE_BYTE (c1); 2885 ONE_MORE_BYTE (c1);
1940 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0)) 2886 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
1941 goto label_invalid_code; 2887 goto invalid_code;
1942 break; 2888 break;
1943 2889
1944 case 'O': /* invocation of single-shift-3 */ 2890 case 'O': /* invocation of single-shift-3 */
1945 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT) 2891 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
1946 || CODING_SPEC_ISO_DESIGNATION (coding, 3) < 0) 2892 || CODING_ISO_DESIGNATION (coding, 3) < 0)
1947 goto label_invalid_code; 2893 goto invalid_code;
1948 charset = CODING_SPEC_ISO_DESIGNATION (coding, 3); 2894 charset = CHARSET_FROM_ID (CODING_ISO_DESIGNATION (coding, 3));
1949 ONE_MORE_BYTE (c1); 2895 ONE_MORE_BYTE (c1);
1950 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0)) 2896 if (c1 < 0x20 || (c1 >= 0x80 && c1 < 0xA0))
1951 goto label_invalid_code; 2897 goto invalid_code;
1952 break; 2898 break;
1953 2899
1954 case '0': case '2': case '3': case '4': /* start composition */ 2900 case '0': case '2': case '3': case '4': /* start composition */
2901 if (! (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK))
2902 goto invalid_code;
1955 DECODE_COMPOSITION_START (c1); 2903 DECODE_COMPOSITION_START (c1);
1956 continue; 2904 continue;
1957 2905
1958 case '1': /* end composition */ 2906 case '1': /* end composition */
1959 DECODE_COMPOSITION_END (c1); 2907 if (composition_state == COMPOSING_NO)
2908 goto invalid_code;
2909 DECODE_COMPOSITION_END ();
1960 continue; 2910 continue;
1961 2911
1962 case '[': /* specification of direction */ 2912 case '[': /* specification of direction */
1963 if (coding->flags & CODING_FLAG_ISO_NO_DIRECTION) 2913 if (! CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DIRECTION)
1964 goto label_invalid_code; 2914 goto invalid_code;
1965 /* For the moment, nested direction is not supported. 2915 /* For the moment, nested direction is not supported.
1966 So, `coding->mode & CODING_MODE_DIRECTION' zero means 2916 So, `coding->mode & CODING_MODE_DIRECTION' zero means
1967 left-to-right, and nonzero means right-to-left. */ 2917 left-to-right, and nozero means right-to-left. */
1968 ONE_MORE_BYTE (c1); 2918 ONE_MORE_BYTE (c1);
1969 switch (c1) 2919 switch (c1)
1970 { 2920 {
@@ -1977,7 +2927,7 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
1977 if (c1 == ']') 2927 if (c1 == ']')
1978 coding->mode &= ~CODING_MODE_DIRECTION; 2928 coding->mode &= ~CODING_MODE_DIRECTION;
1979 else 2929 else
1980 goto label_invalid_code; 2930 goto invalid_code;
1981 break; 2931 break;
1982 2932
1983 case '2': /* start of right-to-left direction */ 2933 case '2': /* start of right-to-left direction */
@@ -1985,63 +2935,91 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
1985 if (c1 == ']') 2935 if (c1 == ']')
1986 coding->mode |= CODING_MODE_DIRECTION; 2936 coding->mode |= CODING_MODE_DIRECTION;
1987 else 2937 else
1988 goto label_invalid_code; 2938 goto invalid_code;
1989 break; 2939 break;
1990 2940
1991 default: 2941 default:
1992 goto label_invalid_code; 2942 goto invalid_code;
1993 } 2943 }
1994 continue; 2944 continue;
1995 2945
1996 default: 2946 default:
1997 if (! (coding->flags & CODING_FLAG_ISO_DESIGNATION)) 2947 if (! (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATION))
1998 goto label_invalid_code; 2948 goto invalid_code;
1999 if (c1 >= 0x28 && c1 <= 0x2B) 2949 if (c1 >= 0x28 && c1 <= 0x2B)
2000 { /* designation of DIMENSION1_CHARS94 character set */ 2950 { /* designation of DIMENSION1_CHARS94 character set */
2001 ONE_MORE_BYTE (c2); 2951 ONE_MORE_BYTE (c2);
2002 DECODE_DESIGNATION (c1 - 0x28, 1, 94, c2); 2952 DECODE_DESIGNATION (c1 - 0x28, 1, 0, c2);
2003 } 2953 }
2004 else if (c1 >= 0x2C && c1 <= 0x2F) 2954 else if (c1 >= 0x2C && c1 <= 0x2F)
2005 { /* designation of DIMENSION1_CHARS96 character set */ 2955 { /* designation of DIMENSION1_CHARS96 character set */
2006 ONE_MORE_BYTE (c2); 2956 ONE_MORE_BYTE (c2);
2007 DECODE_DESIGNATION (c1 - 0x2C, 1, 96, c2); 2957 DECODE_DESIGNATION (c1 - 0x2C, 1, 1, c2);
2008 } 2958 }
2009 else 2959 else
2010 goto label_invalid_code; 2960 goto invalid_code;
2011 /* We must update these variables now. */ 2961 /* We must update these variables now. */
2012 charset0 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 0); 2962 charset_id_0 = CODING_ISO_INVOKED_CHARSET (coding, 0);
2013 charset1 = CODING_SPEC_ISO_PLANE_CHARSET (coding, 1); 2963 charset_id_1 = CODING_ISO_INVOKED_CHARSET (coding, 1);
2014 continue; 2964 continue;
2015 } 2965 }
2016 } 2966 }
2017 2967
2018 /* Now we know CHARSET and 1st position code C1 of a character. 2968 /* Now we know CHARSET and 1st position code C1 of a character.
2019 Produce a multibyte sequence for that character while getting 2969 Produce a decoded character while getting 2nd position code
2020 2nd position code C2 if necessary. */ 2970 C2 if necessary. */
2021 if (CHARSET_DIMENSION (charset) == 2) 2971 c1 &= 0x7F;
2972 if (CHARSET_DIMENSION (charset) > 1)
2022 { 2973 {
2023 ONE_MORE_BYTE (c2); 2974 ONE_MORE_BYTE (c2);
2024 if (c1 < 0x80 ? c2 < 0x20 || c2 >= 0x80 : c2 < 0xA0) 2975 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
2025 /* C2 is not in a valid range. */ 2976 /* C2 is not in a valid range. */
2026 goto label_invalid_code; 2977 goto invalid_code;
2978 c1 = (c1 << 8) | (c2 & 0x7F);
2979 if (CHARSET_DIMENSION (charset) > 2)
2980 {
2981 ONE_MORE_BYTE (c2);
2982 if (c2 < 0x20 || (c2 >= 0x80 && c2 < 0xA0))
2983 /* C2 is not in a valid range. */
2984 goto invalid_code;
2985 c1 = (c1 << 8) | (c2 & 0x7F);
2986 }
2987 }
2988
2989 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c1, c);
2990 if (c < 0)
2991 {
2992 MAYBE_FINISH_COMPOSITION ();
2993 for (; src_base < src; src_base++, char_offset++)
2994 {
2995 if (ASCII_BYTE_P (*src_base))
2996 *charbuf++ = *src_base;
2997 else
2998 *charbuf++ = BYTE8_TO_CHAR (*src_base);
2999 }
3000 }
3001 else if (composition_state == COMPOSING_NO)
3002 {
3003 *charbuf++ = c;
3004 char_offset++;
2027 } 3005 }
2028 c = DECODE_ISO_CHARACTER (charset, c1, c2); 3006 else
2029 EMIT_CHAR (c); 3007 components[component_idx++] = c;
2030 continue; 3008 continue;
2031 3009
2032 label_invalid_code: 3010 invalid_code:
2033 coding->errors++; 3011 MAYBE_FINISH_COMPOSITION ();
2034 if (COMPOSING_P (coding))
2035 DECODE_COMPOSITION_END ('1');
2036 src = src_base; 3012 src = src_base;
2037 c = *src++; 3013 consumed_chars = consumed_chars_base;
2038 EMIT_CHAR (c); 3014 ONE_MORE_BYTE (c);
3015 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
3016 coding->errors++;
2039 } 3017 }
2040 3018
2041 label_end_of_loop: 3019 no_more_source:
2042 coding->consumed = coding->consumed_char = src_base - source; 3020 coding->consumed_char += consumed_chars_base;
2043 coding->produced = dst - destination; 3021 coding->consumed = src_base - coding->source;
2044 return; 3022 coding->charbuf_used = charbuf - coding->charbuf;
2045} 3023}
2046 3024
2047 3025
@@ -2049,9 +3027,9 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
2049 3027
2050/* 3028/*
2051 It is not enough to say just "ISO2022" on encoding, we have to 3029 It is not enough to say just "ISO2022" on encoding, we have to
2052 specify more details. In Emacs, each ISO2022 coding system 3030 specify more details. In Emacs, each coding system of ISO2022
2053 variant has the following specifications: 3031 variant has the following specifications:
2054 1. Initial designation to G0 through G3. 3032 1. Initial designation to G0 thru G3.
2055 2. Allows short-form designation? 3033 2. Allows short-form designation?
2056 3. ASCII should be designated to G0 before control characters? 3034 3. ASCII should be designated to G0 before control characters?
2057 4. ASCII should be designated to G0 at end of line? 3035 4. ASCII should be designated to G0 at end of line?
@@ -2061,8 +3039,8 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
2061 And the following two are only for Japanese: 3039 And the following two are only for Japanese:
2062 8. Use ASCII in place of JIS0201-1976-Roman? 3040 8. Use ASCII in place of JIS0201-1976-Roman?
2063 9. Use JISX0208-1983 in place of JISX0208-1978? 3041 9. Use JISX0208-1983 in place of JISX0208-1978?
2064 These specifications are encoded in `coding->flags' as flag bits 3042 These specifications are encoded in CODING_ISO_FLAGS (coding) as flag bits
2065 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more 3043 defined by macros CODING_ISO_FLAG_XXX. See `coding.h' for more
2066 details. 3044 details.
2067*/ 3045*/
2068 3046
@@ -2073,115 +3051,128 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
2073 3051
2074#define ENCODE_DESIGNATION(charset, reg, coding) \ 3052#define ENCODE_DESIGNATION(charset, reg, coding) \
2075 do { \ 3053 do { \
2076 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \ 3054 unsigned char final_char = CHARSET_ISO_FINAL (charset); \
2077 char *intermediate_char_94 = "()*+"; \ 3055 char *intermediate_char_94 = "()*+"; \
2078 char *intermediate_char_96 = ",-./"; \ 3056 char *intermediate_char_96 = ",-./"; \
2079 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \ 3057 int revision = -1; \
2080 \ 3058 int c; \
2081 if (revision < 255) \ 3059 \
3060 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_REVISION) \
3061 revision = XINT (CHARSET_ISO_REVISION (charset)); \
3062 \
3063 if (revision >= 0) \
2082 { \ 3064 { \
2083 *dst++ = ISO_CODE_ESC; \ 3065 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '&'); \
2084 *dst++ = '&'; \ 3066 EMIT_ONE_BYTE ('@' + revision); \
2085 *dst++ = '@' + revision; \
2086 } \ 3067 } \
2087 *dst++ = ISO_CODE_ESC; \ 3068 EMIT_ONE_ASCII_BYTE (ISO_CODE_ESC); \
2088 if (CHARSET_DIMENSION (charset) == 1) \ 3069 if (CHARSET_DIMENSION (charset) == 1) \
2089 { \ 3070 { \
2090 if (CHARSET_CHARS (charset) == 94) \ 3071 if (! CHARSET_ISO_CHARS_96 (charset)) \
2091 *dst++ = (unsigned char) (intermediate_char_94[reg]); \ 3072 c = intermediate_char_94[reg]; \
2092 else \ 3073 else \
2093 *dst++ = (unsigned char) (intermediate_char_96[reg]); \ 3074 c = intermediate_char_96[reg]; \
3075 EMIT_ONE_ASCII_BYTE (c); \
2094 } \ 3076 } \
2095 else \ 3077 else \
2096 { \ 3078 { \
2097 *dst++ = '$'; \ 3079 EMIT_ONE_ASCII_BYTE ('$'); \
2098 if (CHARSET_CHARS (charset) == 94) \ 3080 if (! CHARSET_ISO_CHARS_96 (charset)) \
2099 { \ 3081 { \
2100 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \ 3082 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_LONG_FORM \
2101 || reg != 0 \ 3083 || reg != 0 \
2102 || final_char < '@' || final_char > 'B') \ 3084 || final_char < '@' || final_char > 'B') \
2103 *dst++ = (unsigned char) (intermediate_char_94[reg]); \ 3085 EMIT_ONE_ASCII_BYTE (intermediate_char_94[reg]); \
2104 } \ 3086 } \
2105 else \ 3087 else \
2106 *dst++ = (unsigned char) (intermediate_char_96[reg]); \ 3088 EMIT_ONE_ASCII_BYTE (intermediate_char_96[reg]); \
2107 } \ 3089 } \
2108 *dst++ = final_char; \ 3090 EMIT_ONE_ASCII_BYTE (final_char); \
2109 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \ 3091 \
3092 CODING_ISO_DESIGNATION (coding, reg) = CHARSET_ID (charset); \
2110 } while (0) 3093 } while (0)
2111 3094
3095
2112/* The following two macros produce codes (control character or escape 3096/* The following two macros produce codes (control character or escape
2113 sequence) for ISO2022 single-shift functions (single-shift-2 and 3097 sequence) for ISO2022 single-shift functions (single-shift-2 and
2114 single-shift-3). */ 3098 single-shift-3). */
2115 3099
2116#define ENCODE_SINGLE_SHIFT_2 \ 3100#define ENCODE_SINGLE_SHIFT_2 \
2117 do { \ 3101 do { \
2118 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \ 3102 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
2119 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \ 3103 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'N'); \
2120 else \ 3104 else \
2121 *dst++ = ISO_CODE_SS2; \ 3105 EMIT_ONE_BYTE (ISO_CODE_SS2); \
2122 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \ 3106 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
2123 } while (0) 3107 } while (0)
2124 3108
2125#define ENCODE_SINGLE_SHIFT_3 \ 3109
2126 do { \ 3110#define ENCODE_SINGLE_SHIFT_3 \
2127 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \ 3111 do { \
2128 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \ 3112 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
2129 else \ 3113 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'O'); \
2130 *dst++ = ISO_CODE_SS3; \ 3114 else \
2131 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \ 3115 EMIT_ONE_BYTE (ISO_CODE_SS3); \
3116 CODING_ISO_SINGLE_SHIFTING (coding) = 1; \
2132 } while (0) 3117 } while (0)
2133 3118
3119
2134/* The following four macros produce codes (control character or 3120/* The following four macros produce codes (control character or
2135 escape sequence) for ISO2022 locking-shift functions (shift-in, 3121 escape sequence) for ISO2022 locking-shift functions (shift-in,
2136 shift-out, locking-shift-2, and locking-shift-3). */ 3122 shift-out, locking-shift-2, and locking-shift-3). */
2137 3123
2138#define ENCODE_SHIFT_IN \ 3124#define ENCODE_SHIFT_IN \
2139 do { \ 3125 do { \
2140 *dst++ = ISO_CODE_SI; \ 3126 EMIT_ONE_ASCII_BYTE (ISO_CODE_SI); \
2141 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \ 3127 CODING_ISO_INVOCATION (coding, 0) = 0; \
2142 } while (0) 3128 } while (0)
2143 3129
2144#define ENCODE_SHIFT_OUT \ 3130
2145 do { \ 3131#define ENCODE_SHIFT_OUT \
2146 *dst++ = ISO_CODE_SO; \ 3132 do { \
2147 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \ 3133 EMIT_ONE_ASCII_BYTE (ISO_CODE_SO); \
3134 CODING_ISO_INVOCATION (coding, 0) = 1; \
2148 } while (0) 3135 } while (0)
2149 3136
2150#define ENCODE_LOCKING_SHIFT_2 \ 3137
2151 do { \ 3138#define ENCODE_LOCKING_SHIFT_2 \
2152 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \ 3139 do { \
2153 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \ 3140 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3141 CODING_ISO_INVOCATION (coding, 0) = 2; \
2154 } while (0) 3142 } while (0)
2155 3143
2156#define ENCODE_LOCKING_SHIFT_3 \ 3144
2157 do { \ 3145#define ENCODE_LOCKING_SHIFT_3 \
2158 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \ 3146 do { \
2159 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \ 3147 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, 'n'); \
3148 CODING_ISO_INVOCATION (coding, 0) = 3; \
2160 } while (0) 3149 } while (0)
2161 3150
3151
2162/* Produce codes for a DIMENSION1 character whose character set is 3152/* Produce codes for a DIMENSION1 character whose character set is
2163 CHARSET and whose position-code is C1. Designation and invocation 3153 CHARSET and whose position-code is C1. Designation and invocation
2164 sequences are also produced in advance if necessary. */ 3154 sequences are also produced in advance if necessary. */
2165 3155
2166#define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \ 3156#define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2167 do { \ 3157 do { \
2168 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \ 3158 int id = CHARSET_ID (charset); \
3159 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
2169 { \ 3160 { \
2170 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \ 3161 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
2171 *dst++ = c1 & 0x7F; \ 3162 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
2172 else \ 3163 else \
2173 *dst++ = c1 | 0x80; \ 3164 EMIT_ONE_BYTE (c1 | 0x80); \
2174 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \ 3165 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
2175 break; \ 3166 break; \
2176 } \ 3167 } \
2177 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \ 3168 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
2178 { \ 3169 { \
2179 *dst++ = c1 & 0x7F; \ 3170 EMIT_ONE_ASCII_BYTE (c1 & 0x7F); \
2180 break; \ 3171 break; \
2181 } \ 3172 } \
2182 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \ 3173 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
2183 { \ 3174 { \
2184 *dst++ = c1 | 0x80; \ 3175 EMIT_ONE_BYTE (c1 | 0x80); \
2185 break; \ 3176 break; \
2186 } \ 3177 } \
2187 else \ 3178 else \
@@ -2189,32 +3180,35 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
2189 must invoke it, or, at first, designate it to some graphic \ 3180 must invoke it, or, at first, designate it to some graphic \
2190 register. Then repeat the loop to actually produce the \ 3181 register. Then repeat the loop to actually produce the \
2191 character. */ \ 3182 character. */ \
2192 dst = encode_invocation_designation (charset, coding, dst); \ 3183 dst = encode_invocation_designation (charset, coding, dst, \
3184 &produced_chars); \
2193 } while (1) 3185 } while (1)
2194 3186
3187
2195/* Produce codes for a DIMENSION2 character whose character set is 3188/* Produce codes for a DIMENSION2 character whose character set is
2196 CHARSET and whose position-codes are C1 and C2. Designation and 3189 CHARSET and whose position-codes are C1 and C2. Designation and
2197 invocation codes are also produced in advance if necessary. */ 3190 invocation codes are also produced in advance if necessary. */
2198 3191
2199#define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \ 3192#define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2200 do { \ 3193 do { \
2201 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \ 3194 int id = CHARSET_ID (charset); \
3195 if (CODING_ISO_SINGLE_SHIFTING (coding)) \
2202 { \ 3196 { \
2203 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \ 3197 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SEVEN_BITS) \
2204 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \ 3198 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
2205 else \ 3199 else \
2206 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \ 3200 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
2207 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \ 3201 CODING_ISO_SINGLE_SHIFTING (coding) = 0; \
2208 break; \ 3202 break; \
2209 } \ 3203 } \
2210 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \ 3204 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 0)) \
2211 { \ 3205 { \
2212 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \ 3206 EMIT_TWO_ASCII_BYTES ((c1) & 0x7F, (c2) & 0x7F); \
2213 break; \ 3207 break; \
2214 } \ 3208 } \
2215 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \ 3209 else if (id == CODING_ISO_INVOKED_CHARSET (coding, 1)) \
2216 { \ 3210 { \
2217 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \ 3211 EMIT_TWO_BYTES ((c1) | 0x80, (c2) | 0x80); \
2218 break; \ 3212 break; \
2219 } \ 3213 } \
2220 else \ 3214 else \
@@ -2222,73 +3216,49 @@ decode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
2222 must invoke it, or, at first, designate it to some graphic \ 3216 must invoke it, or, at first, designate it to some graphic \
2223 register. Then repeat the loop to actually produce the \ 3217 register. Then repeat the loop to actually produce the \
2224 character. */ \ 3218 character. */ \
2225 dst = encode_invocation_designation (charset, coding, dst); \ 3219 dst = encode_invocation_designation (charset, coding, dst, \
3220 &produced_chars); \
2226 } while (1) 3221 } while (1)
2227 3222
2228#define ENCODE_ISO_CHARACTER(c) \
2229 do { \
2230 int charset, c1, c2; \
2231 \
2232 SPLIT_CHAR (c, charset, c1, c2); \
2233 if (CHARSET_DEFINED_P (charset)) \
2234 { \
2235 if (CHARSET_DIMENSION (charset) == 1) \
2236 { \
2237 if (charset == CHARSET_ASCII \
2238 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2239 charset = charset_latin_jisx0201; \
2240 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2241 } \
2242 else \
2243 { \
2244 if (charset == charset_jisx0208 \
2245 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2246 charset = charset_jisx0208_1978; \
2247 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2248 } \
2249 } \
2250 else \
2251 { \
2252 *dst++ = c1; \
2253 if (c2 >= 0) \
2254 *dst++ = c2; \
2255 } \
2256 } while (0)
2257
2258
2259/* Instead of encoding character C, produce one or two `?'s. */
2260 3223
2261#define ENCODE_UNSAFE_CHARACTER(c) \ 3224#define ENCODE_ISO_CHARACTER(charset, c) \
2262 do { \ 3225 do { \
2263 ENCODE_ISO_CHARACTER (CODING_INHIBIT_CHARACTER_SUBSTITUTION); \ 3226 int code = ENCODE_CHAR ((charset),(c)); \
2264 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \ 3227 \
2265 ENCODE_ISO_CHARACTER (CODING_INHIBIT_CHARACTER_SUBSTITUTION); \ 3228 if (CHARSET_DIMENSION (charset) == 1) \
3229 ENCODE_ISO_CHARACTER_DIMENSION1 ((charset), code); \
3230 else \
3231 ENCODE_ISO_CHARACTER_DIMENSION2 ((charset), code >> 8, code & 0xFF); \
2266 } while (0) 3232 } while (0)
2267 3233
2268 3234
2269/* Produce designation and invocation codes at a place pointed by DST 3235/* Produce designation and invocation codes at a place pointed by DST
2270 to use CHARSET. The element `spec.iso2022' of *CODING is updated. 3236 to use CHARSET. The element `spec.iso_2022' of *CODING is updated.
2271 Return new DST. */ 3237 Return new DST. */
2272 3238
2273unsigned char * 3239unsigned char *
2274encode_invocation_designation (charset, coding, dst) 3240encode_invocation_designation (charset, coding, dst, p_nchars)
2275 int charset; 3241 struct charset *charset;
2276 struct coding_system *coding; 3242 struct coding_system *coding;
2277 unsigned char *dst; 3243 unsigned char *dst;
3244 int *p_nchars;
2278{ 3245{
3246 int multibytep = coding->dst_multibyte;
3247 int produced_chars = *p_nchars;
2279 int reg; /* graphic register number */ 3248 int reg; /* graphic register number */
3249 int id = CHARSET_ID (charset);
2280 3250
2281 /* At first, check designations. */ 3251 /* At first, check designations. */
2282 for (reg = 0; reg < 4; reg++) 3252 for (reg = 0; reg < 4; reg++)
2283 if (charset == CODING_SPEC_ISO_DESIGNATION (coding, reg)) 3253 if (id == CODING_ISO_DESIGNATION (coding, reg))
2284 break; 3254 break;
2285 3255
2286 if (reg >= 4) 3256 if (reg >= 4)
2287 { 3257 {
2288 /* CHARSET is not yet designated to any graphic registers. */ 3258 /* CHARSET is not yet designated to any graphic registers. */
2289 /* At first check the requested designation. */ 3259 /* At first check the requested designation. */
2290 reg = CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset); 3260 reg = CODING_ISO_REQUEST (coding, id);
2291 if (reg == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION) 3261 if (reg < 0)
2292 /* Since CHARSET requests no special designation, designate it 3262 /* Since CHARSET requests no special designation, designate it
2293 to graphic register 0. */ 3263 to graphic register 0. */
2294 reg = 0; 3264 reg = 0;
@@ -2296,8 +3266,8 @@ encode_invocation_designation (charset, coding, dst)
2296 ENCODE_DESIGNATION (charset, reg, coding); 3266 ENCODE_DESIGNATION (charset, reg, coding);
2297 } 3267 }
2298 3268
2299 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != reg 3269 if (CODING_ISO_INVOCATION (coding, 0) != reg
2300 && CODING_SPEC_ISO_INVOCATION (coding, 1) != reg) 3270 && CODING_ISO_INVOCATION (coding, 1) != reg)
2301 { 3271 {
2302 /* Since the graphic register REG is not invoked to any graphic 3272 /* Since the graphic register REG is not invoked to any graphic
2303 planes, invoke it to graphic plane 0. */ 3273 planes, invoke it to graphic plane 0. */
@@ -2312,14 +3282,14 @@ encode_invocation_designation (charset, coding, dst)
2312 break; 3282 break;
2313 3283
2314 case 2: /* graphic register 2 */ 3284 case 2: /* graphic register 2 */
2315 if (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT) 3285 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
2316 ENCODE_SINGLE_SHIFT_2; 3286 ENCODE_SINGLE_SHIFT_2;
2317 else 3287 else
2318 ENCODE_LOCKING_SHIFT_2; 3288 ENCODE_LOCKING_SHIFT_2;
2319 break; 3289 break;
2320 3290
2321 case 3: /* graphic register 3 */ 3291 case 3: /* graphic register 3 */
2322 if (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT) 3292 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_SINGLE_SHIFT)
2323 ENCODE_SINGLE_SHIFT_3; 3293 ENCODE_SINGLE_SHIFT_3;
2324 else 3294 else
2325 ENCODE_LOCKING_SHIFT_3; 3295 ENCODE_LOCKING_SHIFT_3;
@@ -2327,98 +3297,55 @@ encode_invocation_designation (charset, coding, dst)
2327 } 3297 }
2328 } 3298 }
2329 3299
3300 *p_nchars = produced_chars;
2330 return dst; 3301 return dst;
2331} 3302}
2332 3303
2333/* Produce 2-byte codes for encoded composition rule RULE. */ 3304/* The following three macros produce codes for indicating direction
2334 3305 of text. */
2335#define ENCODE_COMPOSITION_RULE(rule) \ 3306#define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2336 do { \
2337 int gref, nref; \
2338 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2339 *dst++ = 32 + 81 + gref; \
2340 *dst++ = 32 + nref; \
2341 } while (0)
2342
2343/* Produce codes for indicating the start of a composition sequence
2344 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2345 which specify information about the composition. See the comment
2346 in coding.h for the format of DATA. */
2347
2348#define ENCODE_COMPOSITION_START(coding, data) \
2349 do { \ 3307 do { \
2350 coding->composing = data[3]; \ 3308 if (CODING_ISO_FLAGS (coding) == CODING_ISO_FLAG_SEVEN_BITS) \
2351 *dst++ = ISO_CODE_ESC; \ 3309 EMIT_TWO_ASCII_BYTES (ISO_CODE_ESC, '['); \
2352 if (coding->composing == COMPOSITION_RELATIVE) \
2353 *dst++ = '0'; \
2354 else \ 3310 else \
2355 { \ 3311 EMIT_ONE_BYTE (ISO_CODE_CSI); \
2356 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2357 ? '3' : '4'); \
2358 coding->cmp_data_index = coding->cmp_data_start + 4; \
2359 coding->composition_rule_follows = 0; \
2360 } \
2361 } while (0) 3312 } while (0)
2362 3313
2363/* Produce codes for indicating the end of the current composition. */
2364 3314
2365#define ENCODE_COMPOSITION_END(coding, data) \ 3315#define ENCODE_DIRECTION_R2L() \
2366 do { \ 3316 do { \
2367 *dst++ = ISO_CODE_ESC; \ 3317 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
2368 *dst++ = '1'; \ 3318 EMIT_TWO_ASCII_BYTES ('2', ']'); \
2369 coding->cmp_data_start += data[0]; \
2370 coding->composing = COMPOSITION_NO; \
2371 if (coding->cmp_data_start == coding->cmp_data->used \
2372 && coding->cmp_data->next) \
2373 { \
2374 coding->cmp_data = coding->cmp_data->next; \
2375 coding->cmp_data_start = 0; \
2376 } \
2377 } while (0) 3319 } while (0)
2378 3320
2379/* Produce composition start sequence ESC 0. Here, this sequence
2380 doesn't mean the start of a new composition but means that we have
2381 just produced components (alternate chars and composition rules) of
2382 the composition and the actual text follows in SRC. */
2383 3321
2384#define ENCODE_COMPOSITION_FAKE_START(coding) \ 3322#define ENCODE_DIRECTION_L2R() \
2385 do { \ 3323 do { \
2386 *dst++ = ISO_CODE_ESC; \ 3324 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst); \
2387 *dst++ = '0'; \ 3325 EMIT_TWO_ASCII_BYTES ('0', ']'); \
2388 coding->composing = COMPOSITION_RELATIVE; \
2389 } while (0)
2390
2391/* The following three macros produce codes for indicating direction
2392 of text. */
2393#define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2394 do { \
2395 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
2396 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2397 else \
2398 *dst++ = ISO_CODE_CSI; \
2399 } while (0) 3326 } while (0)
2400 3327
2401#define ENCODE_DIRECTION_R2L \
2402 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
2403
2404#define ENCODE_DIRECTION_L2R \
2405 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
2406 3328
2407/* Produce codes for designation and invocation to reset the graphic 3329/* Produce codes for designation and invocation to reset the graphic
2408 planes and registers to initial state. */ 3330 planes and registers to initial state. */
2409#define ENCODE_RESET_PLANE_AND_REGISTER \ 3331#define ENCODE_RESET_PLANE_AND_REGISTER() \
2410 do { \ 3332 do { \
2411 int reg; \ 3333 int reg; \
2412 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \ 3334 struct charset *charset; \
2413 ENCODE_SHIFT_IN; \ 3335 \
2414 for (reg = 0; reg < 4; reg++) \ 3336 if (CODING_ISO_INVOCATION (coding, 0) != 0) \
2415 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \ 3337 ENCODE_SHIFT_IN; \
2416 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \ 3338 for (reg = 0; reg < 4; reg++) \
2417 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \ 3339 if (CODING_ISO_INITIAL (coding, reg) >= 0 \
2418 ENCODE_DESIGNATION \ 3340 && (CODING_ISO_DESIGNATION (coding, reg) \
2419 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \ 3341 != CODING_ISO_INITIAL (coding, reg))) \
3342 { \
3343 charset = CHARSET_FROM_ID (CODING_ISO_INITIAL (coding, reg)); \
3344 ENCODE_DESIGNATION (charset, reg, coding); \
3345 } \
2420 } while (0) 3346 } while (0)
2421 3347
3348
2422/* Produce designation sequences of charsets in the line started from 3349/* Produce designation sequences of charsets in the line started from
2423 SRC to a place pointed by DST, and return updated DST. 3350 SRC to a place pointed by DST, and return updated DST.
2424 3351
@@ -2426,40 +3353,51 @@ encode_invocation_designation (charset, coding, dst)
2426 find all the necessary designations. */ 3353 find all the necessary designations. */
2427 3354
2428static unsigned char * 3355static unsigned char *
2429encode_designation_at_bol (coding, translation_table, src, src_end, dst) 3356encode_designation_at_bol (coding, charbuf, charbuf_end, dst)
2430 struct coding_system *coding; 3357 struct coding_system *coding;
2431 Lisp_Object translation_table; 3358 int *charbuf, *charbuf_end;
2432 unsigned char *src, *src_end, *dst; 3359 unsigned char *dst;
2433{ 3360{
2434 int charset, c, found = 0, reg; 3361 struct charset *charset;
2435 /* Table of charsets to be designated to each graphic register. */ 3362 /* Table of charsets to be designated to each graphic register. */
2436 int r[4]; 3363 int r[4];
3364 int c, found = 0, reg;
3365 int produced_chars = 0;
3366 int multibytep = coding->dst_multibyte;
3367 Lisp_Object attrs;
3368 Lisp_Object charset_list;
3369
3370 attrs = CODING_ID_ATTRS (coding->id);
3371 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
3372 if (EQ (charset_list, Qiso_2022))
3373 charset_list = Viso_2022_charset_list;
2437 3374
2438 for (reg = 0; reg < 4; reg++) 3375 for (reg = 0; reg < 4; reg++)
2439 r[reg] = -1; 3376 r[reg] = -1;
2440 3377
2441 while (found < 4) 3378 while (found < 4)
2442 { 3379 {
2443 ONE_MORE_CHAR (c); 3380 int id;
3381
3382 c = *charbuf++;
2444 if (c == '\n') 3383 if (c == '\n')
2445 break; 3384 break;
2446 3385 charset = char_charset (c, charset_list, NULL);
2447 charset = CHAR_CHARSET (c); 3386 id = CHARSET_ID (charset);
2448 reg = CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset); 3387 reg = CODING_ISO_REQUEST (coding, id);
2449 if (reg != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION && r[reg] < 0) 3388 if (reg >= 0 && r[reg] < 0)
2450 { 3389 {
2451 found++; 3390 found++;
2452 r[reg] = charset; 3391 r[reg] = id;
2453 } 3392 }
2454 } 3393 }
2455 3394
2456 label_end_of_loop:
2457 if (found) 3395 if (found)
2458 { 3396 {
2459 for (reg = 0; reg < 4; reg++) 3397 for (reg = 0; reg < 4; reg++)
2460 if (r[reg] >= 0 3398 if (r[reg] >= 0
2461 && CODING_SPEC_ISO_DESIGNATION (coding, reg) != r[reg]) 3399 && CODING_ISO_DESIGNATION (coding, reg) != r[reg])
2462 ENCODE_DESIGNATION (r[reg], reg, coding); 3400 ENCODE_DESIGNATION (CHARSET_FROM_ID (r[reg]), reg, coding);
2463 } 3401 }
2464 3402
2465 return dst; 3403 return dst;
@@ -2467,184 +3405,105 @@ encode_designation_at_bol (coding, translation_table, src, src_end, dst)
2467 3405
2468/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */ 3406/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2469 3407
2470static void 3408static int
2471encode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes) 3409encode_coding_iso_2022 (coding)
2472 struct coding_system *coding; 3410 struct coding_system *coding;
2473 unsigned char *source, *destination;
2474 int src_bytes, dst_bytes;
2475{ 3411{
2476 unsigned char *src = source; 3412 int multibytep = coding->dst_multibyte;
2477 unsigned char *src_end = source + src_bytes; 3413 int *charbuf = coding->charbuf;
2478 unsigned char *dst = destination; 3414 int *charbuf_end = charbuf + coding->charbuf_used;
2479 unsigned char *dst_end = destination + dst_bytes; 3415 unsigned char *dst = coding->destination + coding->produced;
2480 /* Since the maximum bytes produced by each loop is 20, we subtract 19 3416 unsigned char *dst_end = coding->destination + coding->dst_bytes;
2481 from DST_END to assure overflow checking is necessary only at the 3417 int safe_room = 16;
2482 head of loop. */ 3418 int bol_designation
2483 unsigned char *adjusted_dst_end = dst_end - 19; 3419 = (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL
2484 /* SRC_BASE remembers the start position in source in each loop. 3420 && CODING_ISO_BOL (coding));
2485 The loop will be exited when there's not enough source text to 3421 int produced_chars = 0;
2486 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when 3422 Lisp_Object attrs, eol_type, charset_list;
2487 there's not enough destination area to produce encoded codes 3423 int ascii_compatible;
2488 (within macro EMIT_BYTES). */
2489 unsigned char *src_base;
2490 int c; 3424 int c;
2491 Lisp_Object translation_table;
2492 Lisp_Object safe_chars;
2493 3425
2494 safe_chars = coding_safe_chars (coding); 3426 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
2495 3427
2496 if (NILP (Venable_character_translation)) 3428 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
2497 translation_table = Qnil;
2498 else
2499 {
2500 translation_table = coding->translation_table_for_encode;
2501 if (NILP (translation_table))
2502 translation_table = Vstandard_translation_table_for_encode;
2503 }
2504 3429
2505 coding->consumed_char = 0; 3430 while (charbuf < charbuf_end)
2506 coding->errors = 0;
2507 while (1)
2508 { 3431 {
2509 src_base = src; 3432 ASSURE_DESTINATION (safe_room);
2510 3433
2511 if (dst >= (dst_bytes ? adjusted_dst_end : (src - 19))) 3434 if (bol_designation)
2512 { 3435 {
2513 coding->result = CODING_FINISH_INSUFFICIENT_DST; 3436 unsigned char *dst_prev = dst;
2514 break;
2515 }
2516 3437
2517 if (coding->flags & CODING_FLAG_ISO_DESIGNATE_AT_BOL
2518 && CODING_SPEC_ISO_BOL (coding))
2519 {
2520 /* We have to produce designation sequences if any now. */ 3438 /* We have to produce designation sequences if any now. */
2521 dst = encode_designation_at_bol (coding, translation_table, 3439 dst = encode_designation_at_bol (coding, charbuf, charbuf_end, dst);
2522 src, src_end, dst); 3440 bol_designation = 0;
2523 CODING_SPEC_ISO_BOL (coding) = 0; 3441 /* We are sure that designation sequences are all ASCII bytes. */
2524 } 3442 produced_chars += dst - dst_prev;
2525
2526 /* Check composition start and end. */
2527 if (coding->composing != COMPOSITION_DISABLED
2528 && coding->cmp_data_start < coding->cmp_data->used)
2529 {
2530 struct composition_data *cmp_data = coding->cmp_data;
2531 int *data = cmp_data->data + coding->cmp_data_start;
2532 int this_pos = cmp_data->char_offset + coding->consumed_char;
2533
2534 if (coding->composing == COMPOSITION_RELATIVE)
2535 {
2536 if (this_pos == data[2])
2537 {
2538 ENCODE_COMPOSITION_END (coding, data);
2539 cmp_data = coding->cmp_data;
2540 data = cmp_data->data + coding->cmp_data_start;
2541 }
2542 }
2543 else if (COMPOSING_P (coding))
2544 {
2545 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2546 if (coding->cmp_data_index == coding->cmp_data_start + data[0])
2547 /* We have consumed components of the composition.
2548 What follows in SRC is the composition's base
2549 text. */
2550 ENCODE_COMPOSITION_FAKE_START (coding);
2551 else
2552 {
2553 int c = cmp_data->data[coding->cmp_data_index++];
2554 if (coding->composition_rule_follows)
2555 {
2556 ENCODE_COMPOSITION_RULE (c);
2557 coding->composition_rule_follows = 0;
2558 }
2559 else
2560 {
2561 if (coding->flags & CODING_FLAG_ISO_SAFE
2562 && ! CODING_SAFE_CHAR_P (safe_chars, c))
2563 ENCODE_UNSAFE_CHARACTER (c);
2564 else
2565 ENCODE_ISO_CHARACTER (c);
2566 if (coding->composing == COMPOSITION_WITH_RULE_ALTCHARS)
2567 coding->composition_rule_follows = 1;
2568 }
2569 continue;
2570 }
2571 }
2572 if (!COMPOSING_P (coding))
2573 {
2574 if (this_pos == data[1])
2575 {
2576 ENCODE_COMPOSITION_START (coding, data);
2577 continue;
2578 }
2579 }
2580 } 3443 }
2581 3444
2582 ONE_MORE_CHAR (c); 3445 c = *charbuf++;
2583 3446
2584 /* Now encode the character C. */ 3447 /* Now encode the character C. */
2585 if (c < 0x20 || c == 0x7F) 3448 if (c < 0x20 || c == 0x7F)
2586 { 3449 {
2587 if (c == '\r') 3450 if (c == '\n'
3451 || (c == '\r' && EQ (eol_type, Qmac)))
2588 { 3452 {
2589 if (! (coding->mode & CODING_MODE_SELECTIVE_DISPLAY)) 3453 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
3454 ENCODE_RESET_PLANE_AND_REGISTER ();
3455 if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_INIT_AT_BOL)
2590 { 3456 {
2591 if (coding->flags & CODING_FLAG_ISO_RESET_AT_CNTL) 3457 int i;
2592 ENCODE_RESET_PLANE_AND_REGISTER; 3458
2593 *dst++ = c; 3459 for (i = 0; i < 4; i++)
2594 continue; 3460 CODING_ISO_DESIGNATION (coding, i)
3461 = CODING_ISO_INITIAL (coding, i);
2595 } 3462 }
2596 /* fall down to treat '\r' as '\n' ... */ 3463 bol_designation
2597 c = '\n'; 3464 = CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_DESIGNATE_AT_BOL;
2598 }
2599 if (c == '\n')
2600 {
2601 if (coding->flags & CODING_FLAG_ISO_RESET_AT_EOL)
2602 ENCODE_RESET_PLANE_AND_REGISTER;
2603 if (coding->flags & CODING_FLAG_ISO_INIT_AT_BOL)
2604 bcopy (coding->spec.iso2022.initial_designation,
2605 coding->spec.iso2022.current_designation,
2606 sizeof coding->spec.iso2022.initial_designation);
2607 if (coding->eol_type == CODING_EOL_LF
2608 || coding->eol_type == CODING_EOL_UNDECIDED)
2609 *dst++ = ISO_CODE_LF;
2610 else if (coding->eol_type == CODING_EOL_CRLF)
2611 *dst++ = ISO_CODE_CR, *dst++ = ISO_CODE_LF;
2612 else
2613 *dst++ = ISO_CODE_CR;
2614 CODING_SPEC_ISO_BOL (coding) = 1;
2615 }
2616 else
2617 {
2618 if (coding->flags & CODING_FLAG_ISO_RESET_AT_CNTL)
2619 ENCODE_RESET_PLANE_AND_REGISTER;
2620 *dst++ = c;
2621 } 3465 }
3466 else if (CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_CNTL)
3467 ENCODE_RESET_PLANE_AND_REGISTER ();
3468 EMIT_ONE_ASCII_BYTE (c);
2622 } 3469 }
2623 else if (ASCII_BYTE_P (c)) 3470 else if (ASCII_CHAR_P (c))
2624 ENCODE_ISO_CHARACTER (c);
2625 else if (SINGLE_BYTE_CHAR_P (c))
2626 { 3471 {
2627 *dst++ = c; 3472 if (ascii_compatible)
2628 coding->errors++; 3473 EMIT_ONE_ASCII_BYTE (c);
3474 else
3475 ENCODE_ISO_CHARACTER (CHARSET_FROM_ID (charset_ascii), c);
2629 } 3476 }
2630 else if (coding->flags & CODING_FLAG_ISO_SAFE
2631 && ! CODING_SAFE_CHAR_P (safe_chars, c))
2632 ENCODE_UNSAFE_CHARACTER (c);
2633 else 3477 else
2634 ENCODE_ISO_CHARACTER (c); 3478 {
3479 struct charset *charset = char_charset (c, charset_list, NULL);
2635 3480
2636 coding->consumed_char++; 3481 if (!charset)
3482 {
3483 c = coding->default_char;
3484 charset = char_charset (c, charset_list, NULL);
3485 }
3486 ENCODE_ISO_CHARACTER (charset, c);
3487 }
2637 } 3488 }
2638 3489
2639 label_end_of_loop: 3490 if (coding->mode & CODING_MODE_LAST_BLOCK
2640 coding->consumed = src_base - source; 3491 && CODING_ISO_FLAGS (coding) & CODING_ISO_FLAG_RESET_AT_EOL)
2641 coding->produced = coding->produced_char = dst - destination; 3492 {
3493 ASSURE_DESTINATION (safe_room);
3494 ENCODE_RESET_PLANE_AND_REGISTER ();
3495 }
3496 coding->result = CODING_RESULT_SUCCESS;
3497 CODING_ISO_BOL (coding) = bol_designation;
3498 coding->produced_char += produced_chars;
3499 coding->produced = dst - coding->destination;
3500 return 0;
2642} 3501}
2643 3502
2644 3503
2645/*** 4. SJIS and BIG5 handlers ***/ 3504/*** 8,9. SJIS and BIG5 handlers ***/
2646 3505
2647/* Although SJIS and BIG5 are not ISO coding systems, they are used 3506/* Although SJIS and BIG5 are not ISO's coding system, they are used
2648 quite widely. So, for the moment, Emacs supports them in the bare 3507 quite widely. So, for the moment, Emacs supports them in the bare
2649 C code. But, in the future, they may be supported only by CCL. */ 3508 C code. But, in the future, they may be supported only by CCL. */
2650 3509
@@ -2653,12 +3512,12 @@ encode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
2653 as is. A character of charset katakana-jisx0201 is encoded by 3512 as is. A character of charset katakana-jisx0201 is encoded by
2654 "position-code + 0x80". A character of charset japanese-jisx0208 3513 "position-code + 0x80". A character of charset japanese-jisx0208
2655 is encoded in 2-byte but two position-codes are divided and shifted 3514 is encoded in 2-byte but two position-codes are divided and shifted
2656 so that it fits in the range below. 3515 so that it fit in the range below.
2657 3516
2658 --- CODE RANGE of SJIS --- 3517 --- CODE RANGE of SJIS ---
2659 (character set) (range) 3518 (character set) (range)
2660 ASCII 0x00 .. 0x7F 3519 ASCII 0x00 .. 0x7F
2661 KATAKANA-JISX0201 0xA1 .. 0xDF 3520 KATAKANA-JISX0201 0xA0 .. 0xDF
2662 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF 3521 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2663 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC 3522 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2664 ------------------------------- 3523 -------------------------------
@@ -2667,7 +3526,7 @@ encode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
2667 3526
2668/* BIG5 is a coding system encoding two character sets: ASCII and 3527/* BIG5 is a coding system encoding two character sets: ASCII and
2669 Big5. An ASCII character is encoded as is. Big5 is a two-byte 3528 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2670 character set and is encoded in two bytes. 3529 character set and is encoded in two-byte.
2671 3530
2672 --- CODE RANGE of BIG5 --- 3531 --- CODE RANGE of BIG5 ---
2673 (character set) (range) 3532 (character set) (range)
@@ -2676,321 +3535,266 @@ encode_coding_iso2022 (coding, source, destination, src_bytes, dst_bytes)
2676 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE 3535 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2677 -------------------------- 3536 --------------------------
2678 3537
2679 Since the number of characters in Big5 is larger than maximum 3538 */
2680 characters in Emacs' charset (96x96), it can't be handled as one
2681 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2682 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2683 contains frequently used characters and the latter contains less
2684 frequently used characters. */
2685
2686/* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2687 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2688 C1 and C2 are the 1st and 2nd position-codes of of Emacs' internal
2689 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2690
2691/* Number of Big5 characters which have the same code in 1st byte. */
2692#define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2693
2694#define DECODE_BIG5(b1, b2, charset, c1, c2) \
2695 do { \
2696 unsigned int temp \
2697 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2698 if (b1 < 0xC9) \
2699 charset = charset_big5_1; \
2700 else \
2701 { \
2702 charset = charset_big5_2; \
2703 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2704 } \
2705 c1 = temp / (0xFF - 0xA1) + 0x21; \
2706 c2 = temp % (0xFF - 0xA1) + 0x21; \
2707 } while (0)
2708
2709#define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2710 do { \
2711 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2712 if (charset == charset_big5_2) \
2713 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2714 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2715 b2 = temp % BIG5_SAME_ROW; \
2716 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2717 } while (0)
2718 3539
2719/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions". 3540/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2720 Check if a text is encoded in SJIS. If it is, return 3541 Check if a text is encoded in SJIS. If it is, return
2721 CODING_CATEGORY_MASK_SJIS, else return 0. */ 3542 CATEGORY_MASK_SJIS, else return 0. */
2722 3543
2723static int 3544static int
2724detect_coding_sjis (src, src_end, multibytep) 3545detect_coding_sjis (coding, mask)
2725 unsigned char *src, *src_end; 3546 struct coding_system *coding;
2726 int multibytep; 3547 int *mask;
2727{ 3548{
3549 unsigned char *src = coding->source, *src_base = src;
3550 unsigned char *src_end = coding->source + coding->src_bytes;
3551 int multibytep = coding->src_multibyte;
3552 int consumed_chars = 0;
3553 int found = 0;
2728 int c; 3554 int c;
2729 /* Dummy for ONE_MORE_BYTE. */ 3555
2730 struct coding_system dummy_coding; 3556 /* A coding system of this category is always ASCII compatible. */
2731 struct coding_system *coding = &dummy_coding; 3557 src += coding->head_ascii;
2732 3558
2733 while (1) 3559 while (1)
2734 { 3560 {
2735 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 3561 ONE_MORE_BYTE (c);
2736 if (c < 0x80) 3562 if (c < 0x80)
2737 continue; 3563 continue;
2738 if (c == 0x80 || c == 0xA0 || c > 0xEF) 3564 if ((c >= 0x81 && c <= 0x9F) || (c >= 0xE0 && c <= 0xEF))
2739 return 0;
2740 if (c <= 0x9F || c >= 0xE0)
2741 { 3565 {
2742 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 3566 ONE_MORE_BYTE (c);
2743 if (c < 0x40 || c == 0x7F || c > 0xFC) 3567 if (c < 0x40 || c == 0x7F || c > 0xFC)
2744 return 0; 3568 break;
3569 found = 1;
2745 } 3570 }
3571 else if (c >= 0xA0 && c < 0xE0)
3572 found = 1;
3573 else
3574 break;
2746 } 3575 }
2747 label_end_of_loop: 3576 *mask &= ~CATEGORY_MASK_SJIS;
2748 return CODING_CATEGORY_MASK_SJIS; 3577 return 0;
3578
3579 no_more_source:
3580 if (!found)
3581 return 0;
3582 *mask &= CATEGORY_MASK_SJIS;
3583 return 1;
2749} 3584}
2750 3585
2751/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions". 3586/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2752 Check if a text is encoded in BIG5. If it is, return 3587 Check if a text is encoded in BIG5. If it is, return
2753 CODING_CATEGORY_MASK_BIG5, else return 0. */ 3588 CATEGORY_MASK_BIG5, else return 0. */
2754 3589
2755static int 3590static int
2756detect_coding_big5 (src, src_end, multibytep) 3591detect_coding_big5 (coding, mask)
2757 unsigned char *src, *src_end; 3592 struct coding_system *coding;
2758 int multibytep; 3593 int *mask;
2759{ 3594{
3595 unsigned char *src = coding->source, *src_base = src;
3596 unsigned char *src_end = coding->source + coding->src_bytes;
3597 int multibytep = coding->src_multibyte;
3598 int consumed_chars = 0;
3599 int found = 0;
2760 int c; 3600 int c;
2761 /* Dummy for ONE_MORE_BYTE. */
2762 struct coding_system dummy_coding;
2763 struct coding_system *coding = &dummy_coding;
2764
2765 while (1)
2766 {
2767 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
2768 if (c < 0x80)
2769 continue;
2770 if (c < 0xA1 || c > 0xFE)
2771 return 0;
2772 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep);
2773 if (c < 0x40 || (c > 0x7F && c < 0xA1) || c > 0xFE)
2774 return 0;
2775 }
2776 label_end_of_loop:
2777 return CODING_CATEGORY_MASK_BIG5;
2778}
2779
2780/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2781 Check if a text is encoded in UTF-8. If it is, return
2782 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2783
2784#define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2785#define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2786#define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2787#define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2788#define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2789#define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2790#define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2791 3601
2792static int 3602 /* A coding system of this category is always ASCII compatible. */
2793detect_coding_utf_8 (src, src_end, multibytep) 3603 src += coding->head_ascii;
2794 unsigned char *src, *src_end;
2795 int multibytep;
2796{
2797 unsigned char c;
2798 int seq_maybe_bytes;
2799 /* Dummy for ONE_MORE_BYTE. */
2800 struct coding_system dummy_coding;
2801 struct coding_system *coding = &dummy_coding;
2802 3604
2803 while (1) 3605 while (1)
2804 { 3606 {
2805 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 3607 ONE_MORE_BYTE (c);
2806 if (UTF_8_1_OCTET_P (c)) 3608 if (c < 0x80)
2807 continue; 3609 continue;
2808 else if (UTF_8_2_OCTET_LEADING_P (c)) 3610 if (c >= 0xA1)
2809 seq_maybe_bytes = 1;
2810 else if (UTF_8_3_OCTET_LEADING_P (c))
2811 seq_maybe_bytes = 2;
2812 else if (UTF_8_4_OCTET_LEADING_P (c))
2813 seq_maybe_bytes = 3;
2814 else if (UTF_8_5_OCTET_LEADING_P (c))
2815 seq_maybe_bytes = 4;
2816 else if (UTF_8_6_OCTET_LEADING_P (c))
2817 seq_maybe_bytes = 5;
2818 else
2819 return 0;
2820
2821 do
2822 { 3611 {
2823 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 3612 ONE_MORE_BYTE (c);
2824 if (!UTF_8_EXTRA_OCTET_P (c)) 3613 if (c < 0x40 || (c >= 0x7F && c <= 0xA0))
2825 return 0; 3614 return 0;
2826 seq_maybe_bytes--; 3615 found = 1;
2827 } 3616 }
2828 while (seq_maybe_bytes > 0); 3617 else
3618 break;
2829 } 3619 }
2830 3620 *mask &= ~CATEGORY_MASK_BIG5;
2831 label_end_of_loop:
2832 return CODING_CATEGORY_MASK_UTF_8;
2833}
2834
2835/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2836 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
2837 Little Endian (otherwise). If it is, return
2838 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
2839 else return 0. */
2840
2841#define UTF_16_INVALID_P(val) \
2842 (((val) == 0xFFFE) \
2843 || ((val) == 0xFFFF))
2844
2845#define UTF_16_HIGH_SURROGATE_P(val) \
2846 (((val) & 0xD800) == 0xD800)
2847
2848#define UTF_16_LOW_SURROGATE_P(val) \
2849 (((val) & 0xDC00) == 0xDC00)
2850
2851static int
2852detect_coding_utf_16 (src, src_end, multibytep)
2853 unsigned char *src, *src_end;
2854 int multibytep;
2855{
2856 unsigned char c1, c2;
2857 /* Dummy for TWO_MORE_BYTES. */
2858 struct coding_system dummy_coding;
2859 struct coding_system *coding = &dummy_coding;
2860
2861 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1, multibytep);
2862 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2, multibytep);
2863
2864 if ((c1 == 0xFF) && (c2 == 0xFE))
2865 return CODING_CATEGORY_MASK_UTF_16_LE;
2866 else if ((c1 == 0xFE) && (c2 == 0xFF))
2867 return CODING_CATEGORY_MASK_UTF_16_BE;
2868
2869 label_end_of_loop:
2870 return 0; 3621 return 0;
3622
3623 no_more_source:
3624 if (!found)
3625 return 0;
3626 *mask &= CATEGORY_MASK_BIG5;
3627 return 1;
2871} 3628}
2872 3629
2873/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". 3630/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
2874 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */ 3631 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
2875 3632
2876static void 3633static void
2877decode_coding_sjis_big5 (coding, source, destination, 3634decode_coding_sjis (coding)
2878 src_bytes, dst_bytes, sjis_p)
2879 struct coding_system *coding; 3635 struct coding_system *coding;
2880 unsigned char *source, *destination;
2881 int src_bytes, dst_bytes;
2882 int sjis_p;
2883{ 3636{
2884 unsigned char *src = source; 3637 unsigned char *src = coding->source + coding->consumed;
2885 unsigned char *src_end = source + src_bytes; 3638 unsigned char *src_end = coding->source + coding->src_bytes;
2886 unsigned char *dst = destination;
2887 unsigned char *dst_end = destination + dst_bytes;
2888 /* SRC_BASE remembers the start position in source in each loop.
2889 The loop will be exited when there's not enough source code
2890 (within macro ONE_MORE_BYTE), or when there's not enough
2891 destination area to produce a character (within macro
2892 EMIT_CHAR). */
2893 unsigned char *src_base; 3639 unsigned char *src_base;
2894 Lisp_Object translation_table; 3640 int *charbuf = coding->charbuf;
3641 int *charbuf_end = charbuf + coding->charbuf_size;
3642 int consumed_chars = 0, consumed_chars_base;
3643 int multibytep = coding->src_multibyte;
3644 struct charset *charset_roman, *charset_kanji, *charset_kana;
3645 Lisp_Object attrs, eol_type, charset_list, val;
2895 3646
2896 if (NILP (Venable_character_translation)) 3647 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
2897 translation_table = Qnil; 3648
2898 else 3649 val = charset_list;
2899 { 3650 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
2900 translation_table = coding->translation_table_for_decode; 3651 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
2901 if (NILP (translation_table)) 3652 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val)));
2902 translation_table = Vstandard_translation_table_for_decode;
2903 }
2904 3653
2905 coding->produced_char = 0;
2906 while (1) 3654 while (1)
2907 { 3655 {
2908 int c, charset, c1, c2; 3656 int c, c1;
2909 3657
2910 src_base = src; 3658 src_base = src;
2911 ONE_MORE_BYTE (c1); 3659 consumed_chars_base = consumed_chars;
2912 3660
2913 if (c1 < 0x80) 3661 if (charbuf >= charbuf_end)
3662 break;
3663
3664 ONE_MORE_BYTE (c);
3665
3666 if (c == '\r')
2914 { 3667 {
2915 charset = CHARSET_ASCII; 3668 if (EQ (eol_type, Qdos))
2916 if (c1 < 0x20)
2917 { 3669 {
2918 if (c1 == '\r') 3670 if (src == src_end)
2919 { 3671 goto no_more_source;
2920 if (coding->eol_type == CODING_EOL_CRLF) 3672 if (*src == '\n')
2921 { 3673 ONE_MORE_BYTE (c);
2922 ONE_MORE_BYTE (c2);
2923 if (c2 == '\n')
2924 c1 = c2;
2925 else if (coding->mode
2926 & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
2927 {
2928 coding->result = CODING_FINISH_INCONSISTENT_EOL;
2929 goto label_end_of_loop;
2930 }
2931 else
2932 /* To process C2 again, SRC is subtracted by 1. */
2933 src--;
2934 }
2935 else if (coding->eol_type == CODING_EOL_CR)
2936 c1 = '\n';
2937 }
2938 else if (c1 == '\n'
2939 && (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
2940 && (coding->eol_type == CODING_EOL_CR
2941 || coding->eol_type == CODING_EOL_CRLF))
2942 {
2943 coding->result = CODING_FINISH_INCONSISTENT_EOL;
2944 goto label_end_of_loop;
2945 }
2946 } 3674 }
3675 else if (EQ (eol_type, Qmac))
3676 c = '\n';
2947 } 3677 }
2948 else 3678 else
2949 { 3679 {
2950 if (sjis_p) 3680 struct charset *charset;
3681
3682 if (c < 0x80)
3683 charset = charset_roman;
3684 else
2951 { 3685 {
2952 if (c1 == 0x80 || c1 == 0xA0 || c1 > 0xEF) 3686 if (c >= 0xF0)
2953 goto label_invalid_code; 3687 goto invalid_code;
2954 if (c1 <= 0x9F || c1 >= 0xE0) 3688 if (c < 0xA0 || c >= 0xE0)
2955 { 3689 {
2956 /* SJIS -> JISX0208 */ 3690 /* SJIS -> JISX0208 */
2957 ONE_MORE_BYTE (c2); 3691 ONE_MORE_BYTE (c1);
2958 if (c2 < 0x40 || c2 == 0x7F || c2 > 0xFC) 3692 if (c1 < 0x40 || c1 == 0x7F || c1 > 0xFC)
2959 goto label_invalid_code; 3693 goto invalid_code;
2960 DECODE_SJIS (c1, c2, c1, c2); 3694 c = (c << 8) | c1;
2961 charset = charset_jisx0208; 3695 SJIS_TO_JIS (c);
3696 charset = charset_kanji;
2962 } 3697 }
2963 else 3698 else
2964 /* SJIS -> JISX0201-Kana */ 3699 /* SJIS -> JISX0201-Kana */
2965 charset = charset_katakana_jisx0201; 3700 charset = charset_kana;
3701 }
3702 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
3703 }
3704 *charbuf++ = c;
3705 continue;
3706
3707 invalid_code:
3708 src = src_base;
3709 consumed_chars = consumed_chars_base;
3710 ONE_MORE_BYTE (c);
3711 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
3712 coding->errors++;
3713 }
3714
3715 no_more_source:
3716 coding->consumed_char += consumed_chars_base;
3717 coding->consumed = src_base - coding->source;
3718 coding->charbuf_used = charbuf - coding->charbuf;
3719}
3720
3721static void
3722decode_coding_big5 (coding)
3723 struct coding_system *coding;
3724{
3725 unsigned char *src = coding->source + coding->consumed;
3726 unsigned char *src_end = coding->source + coding->src_bytes;
3727 unsigned char *src_base;
3728 int *charbuf = coding->charbuf;
3729 int *charbuf_end = charbuf + coding->charbuf_size;
3730 int consumed_chars = 0, consumed_chars_base;
3731 int multibytep = coding->src_multibyte;
3732 struct charset *charset_roman, *charset_big5;
3733 Lisp_Object attrs, eol_type, charset_list, val;
3734
3735 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
3736 val = charset_list;
3737 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
3738 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
3739
3740 while (1)
3741 {
3742 int c, c1;
3743
3744 src_base = src;
3745 consumed_chars_base = consumed_chars;
3746
3747 if (charbuf >= charbuf_end)
3748 break;
3749
3750 ONE_MORE_BYTE (c);
3751
3752 if (c == '\r')
3753 {
3754 if (EQ (eol_type, Qdos))
3755 {
3756 if (src == src_end)
3757 goto no_more_source;
3758 if (*src == '\n')
3759 ONE_MORE_BYTE (c);
2966 } 3760 }
3761 else if (EQ (eol_type, Qmac))
3762 c = '\n';
3763 }
3764 else
3765 {
3766 struct charset *charset;
3767 if (c < 0x80)
3768 charset = charset_roman;
2967 else 3769 else
2968 { 3770 {
2969 /* BIG5 -> Big5 */ 3771 /* BIG5 -> Big5 */
2970 if (c1 < 0xA0 || c1 > 0xFE) 3772 if (c < 0xA1 || c > 0xFE)
2971 goto label_invalid_code; 3773 goto invalid_code;
2972 ONE_MORE_BYTE (c2); 3774 ONE_MORE_BYTE (c1);
2973 if (c2 < 0x40 || (c2 > 0x7E && c2 < 0xA1) || c2 > 0xFE) 3775 if (c1 < 0x40 || (c1 > 0x7E && c1 < 0xA1) || c1 > 0xFE)
2974 goto label_invalid_code; 3776 goto invalid_code;
2975 DECODE_BIG5 (c1, c2, charset, c1, c2); 3777 c = c << 8 | c1;
3778 charset = charset_big5;
2976 } 3779 }
3780 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c, c);
2977 } 3781 }
2978 3782
2979 c = DECODE_ISO_CHARACTER (charset, c1, c2); 3783 *charbuf++ = c;
2980 EMIT_CHAR (c);
2981 continue; 3784 continue;
2982 3785
2983 label_invalid_code: 3786 invalid_code:
2984 coding->errors++;
2985 src = src_base; 3787 src = src_base;
2986 c = *src++; 3788 consumed_chars = consumed_chars_base;
2987 EMIT_CHAR (c); 3789 ONE_MORE_BYTE (c);
3790 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
3791 coding->errors++;
2988 } 3792 }
2989 3793
2990 label_end_of_loop: 3794 no_more_source:
2991 coding->consumed = coding->consumed_char = src_base - source; 3795 coding->consumed_char += consumed_chars_base;
2992 coding->produced = dst - destination; 3796 coding->consumed = src_base - coding->source;
2993 return; 3797 coding->charbuf_used = charbuf - coding->charbuf;
2994} 3798}
2995 3799
2996/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". 3800/* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
@@ -3001,333 +3805,554 @@ decode_coding_sjis_big5 (coding, source, destination,
3001 charsets are produced without any encoding. If SJIS_P is 1, encode 3805 charsets are produced without any encoding. If SJIS_P is 1, encode
3002 SJIS text, else encode BIG5 text. */ 3806 SJIS text, else encode BIG5 text. */
3003 3807
3004static void 3808static int
3005encode_coding_sjis_big5 (coding, source, destination, 3809encode_coding_sjis (coding)
3006 src_bytes, dst_bytes, sjis_p)
3007 struct coding_system *coding; 3810 struct coding_system *coding;
3008 unsigned char *source, *destination;
3009 int src_bytes, dst_bytes;
3010 int sjis_p;
3011{ 3811{
3012 unsigned char *src = source; 3812 int multibytep = coding->dst_multibyte;
3013 unsigned char *src_end = source + src_bytes; 3813 int *charbuf = coding->charbuf;
3014 unsigned char *dst = destination; 3814 int *charbuf_end = charbuf + coding->charbuf_used;
3015 unsigned char *dst_end = destination + dst_bytes; 3815 unsigned char *dst = coding->destination + coding->produced;
3016 /* SRC_BASE remembers the start position in source in each loop. 3816 unsigned char *dst_end = coding->destination + coding->dst_bytes;
3017 The loop will be exited when there's not enough source text to 3817 int safe_room = 4;
3018 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when 3818 int produced_chars = 0;
3019 there's not enough destination area to produce encoded codes 3819 Lisp_Object attrs, eol_type, charset_list, val;
3020 (within macro EMIT_BYTES). */ 3820 int ascii_compatible;
3021 unsigned char *src_base; 3821 struct charset *charset_roman, *charset_kanji, *charset_kana;
3022 Lisp_Object translation_table; 3822 int c;
3023
3024 if (NILP (Venable_character_translation))
3025 translation_table = Qnil;
3026 else
3027 {
3028 translation_table = coding->translation_table_for_encode;
3029 if (NILP (translation_table))
3030 translation_table = Vstandard_translation_table_for_encode;
3031 }
3032 3823
3033 while (1) 3824 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
3034 { 3825 val = charset_list;
3035 int c, charset, c1, c2; 3826 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
3827 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
3828 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val)));
3036 3829
3037 src_base = src; 3830 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
3038 ONE_MORE_CHAR (c);
3039 3831
3832 while (charbuf < charbuf_end)
3833 {
3834 ASSURE_DESTINATION (safe_room);
3835 c = *charbuf++;
3040 /* Now encode the character C. */ 3836 /* Now encode the character C. */
3041 if (SINGLE_BYTE_CHAR_P (c)) 3837 if (ASCII_CHAR_P (c) && ascii_compatible)
3838 EMIT_ONE_ASCII_BYTE (c);
3839 else
3042 { 3840 {
3043 switch (c) 3841 unsigned code;
3842 struct charset *charset = char_charset (c, charset_list, &code);
3843
3844 if (!charset)
3044 { 3845 {
3045 case '\r': 3846 c = coding->default_char;
3046 if (!coding->mode & CODING_MODE_SELECTIVE_DISPLAY) 3847 charset = char_charset (c, charset_list, &code);
3047 { 3848 }
3048 EMIT_ONE_BYTE (c); 3849 if (code == CHARSET_INVALID_CODE (charset))
3049 break; 3850 abort ();
3050 } 3851 if (charset == charset_kanji)
3051 c = '\n'; 3852 {
3052 case '\n': 3853 int c1, c2;
3053 if (coding->eol_type == CODING_EOL_CRLF) 3854 JIS_TO_SJIS (code);
3054 { 3855 c1 = code >> 8, c2 = code & 0xFF;
3055 EMIT_TWO_BYTES ('\r', c); 3856 EMIT_TWO_BYTES (c1, c2);
3056 break;
3057 }
3058 else if (coding->eol_type == CODING_EOL_CR)
3059 c = '\r';
3060 default:
3061 EMIT_ONE_BYTE (c);
3062 } 3857 }
3858 else if (charset == charset_kana)
3859 EMIT_ONE_BYTE (code | 0x80);
3860 else
3861 EMIT_ONE_ASCII_BYTE (code & 0x7F);
3063 } 3862 }
3863 }
3864 coding->result = CODING_RESULT_SUCCESS;
3865 coding->produced_char += produced_chars;
3866 coding->produced = dst - coding->destination;
3867 return 0;
3868}
3869
3870static int
3871encode_coding_big5 (coding)
3872 struct coding_system *coding;
3873{
3874 int multibytep = coding->dst_multibyte;
3875 int *charbuf = coding->charbuf;
3876 int *charbuf_end = charbuf + coding->charbuf_used;
3877 unsigned char *dst = coding->destination + coding->produced;
3878 unsigned char *dst_end = coding->destination + coding->dst_bytes;
3879 int safe_room = 4;
3880 int produced_chars = 0;
3881 Lisp_Object attrs, eol_type, charset_list, val;
3882 int ascii_compatible;
3883 struct charset *charset_roman, *charset_big5;
3884 int c;
3885
3886 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
3887 val = charset_list;
3888 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
3889 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
3890 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
3891
3892 while (charbuf < charbuf_end)
3893 {
3894 ASSURE_DESTINATION (safe_room);
3895 c = *charbuf++;
3896 /* Now encode the character C. */
3897 if (ASCII_CHAR_P (c) && ascii_compatible)
3898 EMIT_ONE_ASCII_BYTE (c);
3064 else 3899 else
3065 { 3900 {
3066 SPLIT_CHAR (c, charset, c1, c2); 3901 unsigned code;
3067 if (sjis_p) 3902 struct charset *charset = char_charset (c, charset_list, &code);
3903
3904 if (! charset)
3068 { 3905 {
3069 if (charset == charset_jisx0208 3906 c = coding->default_char;
3070 || charset == charset_jisx0208_1978) 3907 charset = char_charset (c, charset_list, &code);
3071 {
3072 ENCODE_SJIS (c1, c2, c1, c2);
3073 EMIT_TWO_BYTES (c1, c2);
3074 }
3075 else if (charset == charset_katakana_jisx0201)
3076 EMIT_ONE_BYTE (c1 | 0x80);
3077 else if (charset == charset_latin_jisx0201)
3078 EMIT_ONE_BYTE (c1);
3079 else
3080 /* There's no way other than producing the internal
3081 codes as is. */
3082 EMIT_BYTES (src_base, src);
3083 } 3908 }
3084 else 3909 if (code == CHARSET_INVALID_CODE (charset))
3910 abort ();
3911 if (charset == charset_big5)
3085 { 3912 {
3086 if (charset == charset_big5_1 || charset == charset_big5_2) 3913 int c1, c2;
3087 { 3914
3088 ENCODE_BIG5 (charset, c1, c2, c1, c2); 3915 c1 = code >> 8, c2 = code & 0xFF;
3089 EMIT_TWO_BYTES (c1, c2); 3916 EMIT_TWO_BYTES (c1, c2);
3090 }
3091 else
3092 /* There's no way other than producing the internal
3093 codes as is. */
3094 EMIT_BYTES (src_base, src);
3095 } 3917 }
3918 else
3919 EMIT_ONE_ASCII_BYTE (code & 0x7F);
3096 } 3920 }
3097 coding->consumed_char++;
3098 } 3921 }
3099 3922 coding->result = CODING_RESULT_SUCCESS;
3100 label_end_of_loop: 3923 coding->produced_char += produced_chars;
3101 coding->consumed = src_base - source; 3924 coding->produced = dst - coding->destination;
3102 coding->produced = coding->produced_char = dst - destination; 3925 return 0;
3103} 3926}
3104 3927
3105 3928
3106/*** 5. CCL handlers ***/ 3929/*** 10. CCL handlers ***/
3107 3930
3108/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions". 3931/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3109 Check if a text is encoded in a coding system of which 3932 Check if a text is encoded in a coding system of which
3110 encoder/decoder are written in CCL program. If it is, return 3933 encoder/decoder are written in CCL program. If it is, return
3111 CODING_CATEGORY_MASK_CCL, else return 0. */ 3934 CATEGORY_MASK_CCL, else return 0. */
3112 3935
3113static int 3936static int
3114detect_coding_ccl (src, src_end, multibytep) 3937detect_coding_ccl (coding, mask)
3115 unsigned char *src, *src_end; 3938 struct coding_system *coding;
3116 int multibytep; 3939 int *mask;
3117{ 3940{
3118 unsigned char *valid; 3941 unsigned char *src = coding->source, *src_base = src;
3119 int c; 3942 unsigned char *src_end = coding->source + coding->src_bytes;
3120 /* Dummy for ONE_MORE_BYTE. */ 3943 int multibytep = coding->src_multibyte;
3121 struct coding_system dummy_coding; 3944 int consumed_chars = 0;
3122 struct coding_system *coding = &dummy_coding; 3945 int found = 0;
3123 3946 unsigned char *valids = CODING_CCL_VALIDS (coding);
3124 /* No coding system is assigned to coding-category-ccl. */ 3947 int head_ascii = coding->head_ascii;
3125 if (!coding_system_table[CODING_CATEGORY_IDX_CCL]) 3948 Lisp_Object attrs;
3126 return 0; 3949
3950 coding = &coding_categories[coding_category_ccl];
3951 attrs = CODING_ID_ATTRS (coding->id);
3952 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
3953 src += head_ascii;
3127 3954
3128 valid = coding_system_table[CODING_CATEGORY_IDX_CCL]->spec.ccl.valid_codes;
3129 while (1) 3955 while (1)
3130 { 3956 {
3131 ONE_MORE_BYTE_CHECK_MULTIBYTE (c, multibytep); 3957 int c;
3132 if (! valid[c]) 3958 ONE_MORE_BYTE (c);
3133 return 0; 3959 if (! valids[c])
3960 break;
3961 if (!found && valids[c] > 1)
3962 found = 1;
3134 } 3963 }
3135 label_end_of_loop: 3964 *mask &= ~CATEGORY_MASK_CCL;
3136 return CODING_CATEGORY_MASK_CCL; 3965 return 0;
3137}
3138
3139
3140/*** 6. End-of-line handlers ***/
3141 3966
3142/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */ 3967 no_more_source:
3968 if (!found)
3969 return 0;
3970 *mask &= CATEGORY_MASK_CCL;
3971 return 1;
3972}
3143 3973
3144static void 3974static void
3145decode_eol (coding, source, destination, src_bytes, dst_bytes) 3975decode_coding_ccl (coding)
3146 struct coding_system *coding; 3976 struct coding_system *coding;
3147 unsigned char *source, *destination;
3148 int src_bytes, dst_bytes;
3149{ 3977{
3150 unsigned char *src = source; 3978 unsigned char *src = coding->source + coding->consumed;
3151 unsigned char *dst = destination; 3979 unsigned char *src_end = coding->source + coding->src_bytes;
3152 unsigned char *src_end = src + src_bytes; 3980 int *charbuf = coding->charbuf;
3153 unsigned char *dst_end = dst + dst_bytes; 3981 int *charbuf_end = charbuf + coding->charbuf_size;
3154 Lisp_Object translation_table; 3982 int consumed_chars = 0;
3155 /* SRC_BASE remembers the start position in source in each loop. 3983 int multibytep = coding->src_multibyte;
3156 The loop will be exited when there's not enough source code 3984 struct ccl_program ccl;
3157 (within macro ONE_MORE_BYTE), or when there's not enough 3985 int source_charbuf[1024];
3158 destination area to produce a character (within macro 3986 int source_byteidx[1024];
3159 EMIT_CHAR). */ 3987
3160 unsigned char *src_base; 3988 setup_ccl_program (&ccl, CODING_CCL_DECODER (coding));
3161 int c; 3989
3162 3990 while (src < src_end)
3163 translation_table = Qnil;
3164 switch (coding->eol_type)
3165 { 3991 {
3166 case CODING_EOL_CRLF: 3992 unsigned char *p = src;
3167 while (1) 3993 int *source, *source_end;
3168 { 3994 int i = 0;
3169 src_base = src;
3170 ONE_MORE_BYTE (c);
3171 if (c == '\r')
3172 {
3173 ONE_MORE_BYTE (c);
3174 if (c != '\n')
3175 {
3176 if (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
3177 {
3178 coding->result = CODING_FINISH_INCONSISTENT_EOL;
3179 goto label_end_of_loop;
3180 }
3181 src--;
3182 c = '\r';
3183 }
3184 }
3185 else if (c == '\n'
3186 && (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL))
3187 {
3188 coding->result = CODING_FINISH_INCONSISTENT_EOL;
3189 goto label_end_of_loop;
3190 }
3191 EMIT_CHAR (c);
3192 }
3193 break;
3194 3995
3195 case CODING_EOL_CR: 3996 if (multibytep)
3196 while (1) 3997 while (i < 1024 && p < src_end)
3998 {
3999 source_byteidx[i] = p - src;
4000 source_charbuf[i++] = STRING_CHAR_ADVANCE (p);
4001 }
4002 else
4003 while (i < 1024 && p < src_end)
4004 source_charbuf[i++] = *p++;
4005
4006 if (p == src_end && coding->mode & CODING_MODE_LAST_BLOCK)
4007 ccl.last_block = 1;
4008
4009 source = source_charbuf;
4010 source_end = source + i;
4011 while (source < source_end)
3197 { 4012 {
3198 src_base = src; 4013 ccl_driver (&ccl, source, charbuf,
3199 ONE_MORE_BYTE (c); 4014 source_end - source, charbuf_end - charbuf);
3200 if (c == '\n') 4015 source += ccl.consumed;
3201 { 4016 charbuf += ccl.produced;
3202 if (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL) 4017 if (ccl.status != CCL_STAT_SUSPEND_BY_DST)
3203 { 4018 break;
3204 coding->result = CODING_FINISH_INCONSISTENT_EOL;
3205 goto label_end_of_loop;
3206 }
3207 }
3208 else if (c == '\r')
3209 c = '\n';
3210 EMIT_CHAR (c);
3211 } 4019 }
4020 if (source < source_end)
4021 src += source_byteidx[source - source_charbuf];
4022 else
4023 src = p;
4024 consumed_chars += source - source_charbuf;
4025
4026 if (ccl.status != CCL_STAT_SUSPEND_BY_SRC
4027 && ccl.status != CODING_RESULT_INSUFFICIENT_SRC)
4028 break;
4029 }
4030
4031 switch (ccl.status)
4032 {
4033 case CCL_STAT_SUSPEND_BY_SRC:
4034 coding->result = CODING_RESULT_INSUFFICIENT_SRC;
4035 break;
4036 case CCL_STAT_SUSPEND_BY_DST:
4037 break;
4038 case CCL_STAT_QUIT:
4039 case CCL_STAT_INVALID_CMD:
4040 coding->result = CODING_RESULT_INTERRUPT;
4041 break;
4042 default:
4043 coding->result = CODING_RESULT_SUCCESS;
3212 break; 4044 break;
4045 }
4046 coding->consumed_char += consumed_chars;
4047 coding->consumed = src - coding->source;
4048 coding->charbuf_used = charbuf - coding->charbuf;
4049}
3213 4050
3214 default: /* no need for EOL handling */ 4051static int
3215 while (1) 4052encode_coding_ccl (coding)
4053 struct coding_system *coding;
4054{
4055 struct ccl_program ccl;
4056 int multibytep = coding->dst_multibyte;
4057 int *charbuf = coding->charbuf;
4058 int *charbuf_end = charbuf + coding->charbuf_used;
4059 unsigned char *dst = coding->destination + coding->produced;
4060 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4061 unsigned char *adjusted_dst_end = dst_end - 1;
4062 int destination_charbuf[1024];
4063 int i, produced_chars = 0;
4064
4065 setup_ccl_program (&ccl, CODING_CCL_ENCODER (coding));
4066
4067 ccl.last_block = coding->mode & CODING_MODE_LAST_BLOCK;
4068 ccl.dst_multibyte = coding->dst_multibyte;
4069
4070 while (charbuf < charbuf_end && dst < adjusted_dst_end)
4071 {
4072 int dst_bytes = dst_end - dst;
4073 if (dst_bytes > 1024)
4074 dst_bytes = 1024;
4075
4076 ccl_driver (&ccl, charbuf, destination_charbuf,
4077 charbuf_end - charbuf, dst_bytes);
4078 charbuf += ccl.consumed;
4079 if (multibytep)
4080 for (i = 0; i < ccl.produced; i++)
4081 EMIT_ONE_BYTE (destination_charbuf[i] & 0xFF);
4082 else
3216 { 4083 {
3217 src_base = src; 4084 for (i = 0; i < ccl.produced; i++)
3218 ONE_MORE_BYTE (c); 4085 *dst++ = destination_charbuf[i] & 0xFF;
3219 EMIT_CHAR (c); 4086 produced_chars += ccl.produced;
3220 } 4087 }
3221 } 4088 }
3222 4089
3223 label_end_of_loop: 4090 switch (ccl.status)
3224 coding->consumed = coding->consumed_char = src_base - source; 4091 {
3225 coding->produced = dst - destination; 4092 case CCL_STAT_SUSPEND_BY_SRC:
3226 return; 4093 coding->result = CODING_RESULT_INSUFFICIENT_SRC;
4094 break;
4095 case CCL_STAT_SUSPEND_BY_DST:
4096 coding->result = CODING_RESULT_INSUFFICIENT_DST;
4097 break;
4098 case CCL_STAT_QUIT:
4099 case CCL_STAT_INVALID_CMD:
4100 coding->result = CODING_RESULT_INTERRUPT;
4101 break;
4102 default:
4103 coding->result = CODING_RESULT_SUCCESS;
4104 break;
4105 }
4106
4107 coding->produced_char += produced_chars;
4108 coding->produced = dst - coding->destination;
4109 return 0;
3227} 4110}
3228 4111
3229/* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode 4112
3230 format of end-of-line according to `coding->eol_type'. It also 4113
3231 convert multibyte form 8-bit characters to unibyte if 4114/*** 10, 11. no-conversion handlers ***/
3232 CODING->src_multibyte is nonzero. If `coding->mode & 4115
3233 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text 4116/* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3234 also means end-of-line. */
3235 4117
3236static void 4118static void
3237encode_eol (coding, source, destination, src_bytes, dst_bytes) 4119decode_coding_raw_text (coding)
3238 struct coding_system *coding; 4120 struct coding_system *coding;
3239 unsigned char *source, *destination;
3240 int src_bytes, dst_bytes;
3241{ 4121{
3242 unsigned char *src = source; 4122 coding->chars_at_source = 1;
3243 unsigned char *dst = destination; 4123 coding->consumed_char = coding->src_chars;
3244 unsigned char *src_end = src + src_bytes; 4124 coding->consumed = coding->src_bytes;
3245 unsigned char *dst_end = dst + dst_bytes; 4125 coding->result = CODING_RESULT_SUCCESS;
3246 Lisp_Object translation_table; 4126}
3247 /* SRC_BASE remembers the start position in source in each loop. 4127
3248 The loop will be exited when there's not enough source text to 4128static int
3249 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when 4129encode_coding_raw_text (coding)
3250 there's not enough destination area to produce encoded codes 4130 struct coding_system *coding;
3251 (within macro EMIT_BYTES). */ 4131{
3252 unsigned char *src_base; 4132 int multibytep = coding->dst_multibyte;
4133 int *charbuf = coding->charbuf;
4134 int *charbuf_end = coding->charbuf + coding->charbuf_used;
4135 unsigned char *dst = coding->destination + coding->produced;
4136 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4137 int produced_chars = 0;
3253 int c; 4138 int c;
3254 int selective_display = coding->mode & CODING_MODE_SELECTIVE_DISPLAY;
3255 4139
3256 translation_table = Qnil; 4140 if (multibytep)
3257 if (coding->src_multibyte
3258 && *(src_end - 1) == LEADING_CODE_8_BIT_CONTROL)
3259 { 4141 {
3260 src_end--; 4142 int safe_room = MAX_MULTIBYTE_LENGTH * 2;
3261 src_bytes--;
3262 coding->result = CODING_FINISH_INSUFFICIENT_SRC;
3263 }
3264 4143
3265 if (coding->eol_type == CODING_EOL_CRLF) 4144 if (coding->src_multibyte)
3266 { 4145 while (charbuf < charbuf_end)
3267 while (src < src_end) 4146 {
3268 { 4147 ASSURE_DESTINATION (safe_room);
3269 src_base = src; 4148 c = *charbuf++;
3270 c = *src++; 4149 if (ASCII_CHAR_P (c))
3271 if (c >= 0x20) 4150 EMIT_ONE_ASCII_BYTE (c);
3272 EMIT_ONE_BYTE (c); 4151 else if (CHAR_BYTE8_P (c))
3273 else if (c == '\n' || (c == '\r' && selective_display)) 4152 {
3274 EMIT_TWO_BYTES ('\r', '\n'); 4153 c = CHAR_TO_BYTE8 (c);
3275 else 4154 EMIT_ONE_BYTE (c);
4155 }
4156 else
4157 {
4158 unsigned char str[MAX_MULTIBYTE_LENGTH], *p0 = str, *p1 = str;
4159
4160 CHAR_STRING_ADVANCE (c, p1);
4161 while (p0 < p1)
4162 EMIT_ONE_BYTE (*p0);
4163 }
4164 }
4165 else
4166 while (charbuf < charbuf_end)
4167 {
4168 ASSURE_DESTINATION (safe_room);
4169 c = *charbuf++;
3276 EMIT_ONE_BYTE (c); 4170 EMIT_ONE_BYTE (c);
3277 } 4171 }
3278 src_base = src;
3279 label_end_of_loop:
3280 ;
3281 } 4172 }
3282 else 4173 else
3283 { 4174 {
3284 if (!dst_bytes || src_bytes <= dst_bytes) 4175 if (coding->src_multibyte)
3285 { 4176 {
3286 safe_bcopy (src, dst, src_bytes); 4177 int safe_room = MAX_MULTIBYTE_LENGTH;
3287 src_base = src_end; 4178
3288 dst += src_bytes; 4179 while (charbuf < charbuf_end)
4180 {
4181 ASSURE_DESTINATION (safe_room);
4182 c = *charbuf++;
4183 if (ASCII_CHAR_P (c))
4184 *dst++ = c;
4185 else if (CHAR_BYTE8_P (c))
4186 *dst++ = CHAR_TO_BYTE8 (c);
4187 else
4188 CHAR_STRING_ADVANCE (c, dst);
4189 produced_chars++;
4190 }
3289 } 4191 }
3290 else 4192 else
3291 { 4193 {
3292 if (coding->src_multibyte 4194 ASSURE_DESTINATION (charbuf_end - charbuf);
3293 && *(src + dst_bytes - 1) == LEADING_CODE_8_BIT_CONTROL) 4195 while (charbuf < charbuf_end && dst < dst_end)
3294 dst_bytes--; 4196 *dst++ = *charbuf++;
3295 safe_bcopy (src, dst, dst_bytes); 4197 produced_chars = dst - (coding->destination + coding->dst_bytes);
3296 src_base = src + dst_bytes; 4198 }
3297 dst = destination + dst_bytes; 4199 }
3298 coding->result = CODING_FINISH_INSUFFICIENT_DST; 4200 coding->result = CODING_RESULT_SUCCESS;
3299 } 4201 coding->produced_char += produced_chars;
3300 if (coding->eol_type == CODING_EOL_CR) 4202 coding->produced = dst - coding->destination;
4203 return 0;
4204}
4205
4206static int
4207detect_coding_charset (coding, mask)
4208 struct coding_system *coding;
4209 int *mask;
4210{
4211 unsigned char *src = coding->source, *src_base = src;
4212 unsigned char *src_end = coding->source + coding->src_bytes;
4213 int multibytep = coding->src_multibyte;
4214 int consumed_chars = 0;
4215 Lisp_Object attrs, valids;
4216
4217 coding = &coding_categories[coding_category_charset];
4218 attrs = CODING_ID_ATTRS (coding->id);
4219 valids = AREF (attrs, coding_attr_charset_valids);
4220
4221 if (! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
4222 src += coding->head_ascii;
4223
4224 while (1)
4225 {
4226 int c;
4227
4228 ONE_MORE_BYTE (c);
4229 if (NILP (AREF (valids, c)))
4230 break;
4231 }
4232 *mask &= ~CATEGORY_MASK_CHARSET;
4233 return 0;
4234
4235 no_more_source:
4236 *mask &= CATEGORY_MASK_CHARSET;
4237 return 1;
4238}
4239
4240static void
4241decode_coding_charset (coding)
4242 struct coding_system *coding;
4243{
4244 unsigned char *src = coding->source + coding->consumed;
4245 unsigned char *src_end = coding->source + coding->src_bytes;
4246 unsigned char *src_base;
4247 int *charbuf = coding->charbuf;
4248 int *charbuf_end = charbuf + coding->charbuf_size;
4249 int consumed_chars = 0, consumed_chars_base;
4250 int multibytep = coding->src_multibyte;
4251 struct charset *charset;
4252 Lisp_Object attrs, eol_type, charset_list;
4253
4254 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
4255 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
4256
4257 while (1)
4258 {
4259 int c, c1;
4260
4261 src_base = src;
4262 consumed_chars_base = consumed_chars;
4263
4264 if (charbuf >= charbuf_end)
4265 break;
4266
4267 ONE_MORE_BYTE (c1);
4268 if (c == '\r')
3301 { 4269 {
3302 for (src = destination; src < dst; src++) 4270 if (EQ (eol_type, Qdos))
3303 if (*src == '\n') *src = '\r'; 4271 {
4272 if (src == src_end)
4273 goto no_more_source;
4274 if (*src == '\n')
4275 ONE_MORE_BYTE (c);
4276 }
4277 else if (EQ (eol_type, Qmac))
4278 c = '\n';
3304 } 4279 }
3305 else if (selective_display) 4280 else
3306 { 4281 {
3307 for (src = destination; src < dst; src++) 4282 CODING_DECODE_CHAR (coding, src, src_base, src_end, charset, c1, c);
3308 if (*src == '\r') *src = '\n'; 4283 if (c < 0)
4284 goto invalid_code;
3309 } 4285 }
4286 *charbuf++ = c;
4287 continue;
4288
4289 invalid_code:
4290 src = src_base;
4291 consumed_chars = consumed_chars_base;
4292 ONE_MORE_BYTE (c);
4293 *charbuf++ = ASCII_BYTE_P (c) ? c : BYTE8_TO_CHAR (c);
4294 coding->errors++;
3310 } 4295 }
3311 if (coding->src_multibyte)
3312 dst = destination + str_as_unibyte (destination, dst - destination);
3313 4296
3314 coding->consumed = src_base - source; 4297 no_more_source:
3315 coding->produced = dst - destination; 4298 coding->consumed_char += consumed_chars_base;
3316 coding->produced_char = coding->produced; 4299 coding->consumed = src_base - coding->source;
4300 coding->charbuf_used = charbuf - coding->charbuf;
4301}
4302
4303static int
4304encode_coding_charset (coding)
4305 struct coding_system *coding;
4306{
4307 int multibytep = coding->dst_multibyte;
4308 int *charbuf = coding->charbuf;
4309 int *charbuf_end = charbuf + coding->charbuf_used;
4310 unsigned char *dst = coding->destination + coding->produced;
4311 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4312 int safe_room = MAX_MULTIBYTE_LENGTH;
4313 int produced_chars = 0;
4314 struct charset *charset;
4315 Lisp_Object attrs, eol_type, charset_list;
4316 int ascii_compatible;
4317 int c;
4318
4319 CODING_GET_INFO (coding, attrs, eol_type, charset_list);
4320 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
4321 ascii_compatible = ! NILP (CODING_ATTR_ASCII_COMPAT (attrs));
4322
4323 while (charbuf < charbuf_end)
4324 {
4325 unsigned code;
4326
4327 ASSURE_DESTINATION (safe_room);
4328 c = *charbuf++;
4329 if (ascii_compatible && ASCII_CHAR_P (c))
4330 EMIT_ONE_ASCII_BYTE (c);
4331 else if ((code = ENCODE_CHAR (charset, c))
4332 != CHARSET_INVALID_CODE (charset))
4333 EMIT_ONE_BYTE (code);
4334 else
4335 EMIT_ONE_BYTE (coding->default_char);
4336 }
4337
4338 coding->result = CODING_RESULT_SUCCESS;
4339 coding->produced_char += produced_chars;
4340 coding->produced = dst - coding->destination;
4341 return 0;
3317} 4342}
3318 4343
3319 4344
3320/*** 7. C library functions ***/ 4345/*** 7. C library functions ***/
3321 4346
3322/* In Emacs Lisp, a coding system is represented by a Lisp symbol which 4347/* In Emacs Lisp, coding system is represented by a Lisp symbol which
3323 has a property `coding-system'. The value of this property is a 4348 has a property `coding-system'. The value of this property is a
3324 vector of length 5 (called the coding-vector). Among elements of 4349 vector of length 5 (called as coding-vector). Among elements of
3325 this vector, the first (element[0]) and the fifth (element[4]) 4350 this vector, the first (element[0]) and the fifth (element[4])
3326 carry important information for decoding/encoding. Before 4351 carry important information for decoding/encoding. Before
3327 decoding/encoding, this information should be set in fields of a 4352 decoding/encoding, this information should be set in fields of a
3328 structure of type `coding_system'. 4353 structure of type `coding_system'.
3329 4354
3330 The value of the property `coding-system' can be a symbol of another 4355 A value of property `coding-system' can be a symbol of another
3331 subsidiary coding-system. In that case, Emacs gets coding-vector 4356 subsidiary coding-system. In that case, Emacs gets coding-vector
3332 from that symbol. 4357 from that symbol.
3333 4358
@@ -3336,7 +4361,7 @@ encode_eol (coding, source, destination, src_bytes, dst_bytes)
3336 4361
3337 0 -- coding_type_emacs_mule 4362 0 -- coding_type_emacs_mule
3338 1 -- coding_type_sjis 4363 1 -- coding_type_sjis
3339 2 -- coding_type_iso2022 4364 2 -- coding_type_iso_2022
3340 3 -- coding_type_big5 4365 3 -- coding_type_big5
3341 4 -- coding_type_ccl encoder/decoder written in CCL 4366 4 -- coding_type_ccl encoder/decoder written in CCL
3342 nil -- coding_type_no_conversion 4367 nil -- coding_type_no_conversion
@@ -3346,11 +4371,11 @@ encode_eol (coding, source, destination, src_bytes, dst_bytes)
3346 `element[4]' contains information to be set in `coding->flags' and 4371 `element[4]' contains information to be set in `coding->flags' and
3347 `coding->spec'. The meaning varies by `coding->type'. 4372 `coding->spec'. The meaning varies by `coding->type'.
3348 4373
3349 If `coding->type' is `coding_type_iso2022', element[4] is a vector 4374 If `coding->type' is `coding_type_iso_2022', element[4] is a vector
3350 of length 32 (of which the first 13 sub-elements are used now). 4375 of length 32 (of which the first 13 sub-elements are used now).
3351 Meanings of these sub-elements are: 4376 Meanings of these sub-elements are:
3352 4377
3353 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022' 4378 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso_2022'
3354 If the value is an integer of valid charset, the charset is 4379 If the value is an integer of valid charset, the charset is
3355 assumed to be designated to graphic register N initially. 4380 assumed to be designated to graphic register N initially.
3356 4381
@@ -3361,7 +4386,7 @@ encode_eol (coding, source, destination, src_bytes, dst_bytes)
3361 4386
3362 If the value is nil, graphic register N is never used on 4387 If the value is nil, graphic register N is never used on
3363 encoding. 4388 encoding.
3364 4389
3365 sub-element[N] where N is 4 through 11: to be set in `coding->flags' 4390 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3366 Each value takes t or nil. See the section ISO2022 of 4391 Each value takes t or nil. See the section ISO2022 of
3367 `coding.h' for more information. 4392 `coding.h' for more information.
@@ -3371,437 +4396,240 @@ encode_eol (coding, source, destination, src_bytes, dst_bytes)
3371 4396
3372 If `coding->type' takes the other value, element[4] is ignored. 4397 If `coding->type' takes the other value, element[4] is ignored.
3373 4398
3374 Emacs Lisp's coding systems also carry information about format of 4399 Emacs Lisp's coding system also carries information about format of
3375 end-of-line in a value of property `eol-type'. If the value is 4400 end-of-line in a value of property `eol-type'. If the value is
3376 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2 4401 integer, 0 means eol_lf, 1 means eol_crlf, and 2 means eol_cr. If
3377 means CODING_EOL_CR. If it is not integer, it should be a vector 4402 it is not integer, it should be a vector of subsidiary coding
3378 of subsidiary coding systems of which property `eol-type' has one 4403 systems of which property `eol-type' has one of above values.
3379 of the above values.
3380 4404
3381*/ 4405*/
3382 4406
3383/* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL 4407/* Setup coding context CODING from information about CODING_SYSTEM.
3384 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING 4408 If CODING_SYSTEM is nil, `no-conversion' is assumed. If
3385 is setup so that no conversion is necessary and return -1, else 4409 CODING_SYSTEM is invalid, signal an error. */
3386 return 0. */
3387 4410
3388int 4411void
3389setup_coding_system (coding_system, coding) 4412setup_coding_system (coding_system, coding)
3390 Lisp_Object coding_system; 4413 Lisp_Object coding_system;
3391 struct coding_system *coding; 4414 struct coding_system *coding;
3392{ 4415{
3393 Lisp_Object coding_spec, coding_type, eol_type, plist; 4416 int id;
4417 Lisp_Object attrs;
4418 Lisp_Object eol_type;
4419 Lisp_Object coding_type;
3394 Lisp_Object val; 4420 Lisp_Object val;
3395 4421
3396 /* At first, zero clear all members. */ 4422 if (NILP (coding_system))
3397 bzero (coding, sizeof (struct coding_system)); 4423 coding_system = Qno_conversion;
3398 4424
3399 /* Initialize some fields required for all kinds of coding systems. */ 4425 CHECK_CODING_SYSTEM_GET_ID (coding_system, coding->id);
3400 coding->symbol = coding_system;
3401 coding->heading_ascii = -1;
3402 coding->post_read_conversion = coding->pre_write_conversion = Qnil;
3403 coding->composing = COMPOSITION_DISABLED;
3404 coding->cmp_data = NULL;
3405 4426
3406 if (NILP (coding_system)) 4427 attrs = CODING_ID_ATTRS (coding->id);
3407 goto label_invalid_coding_system; 4428 eol_type = CODING_ID_EOL_TYPE (coding->id);
3408 4429
3409 coding_spec = Fget (coding_system, Qcoding_system); 4430 coding->mode = 0;
4431 coding->head_ascii = -1;
4432 coding->common_flags
4433 = (VECTORP (eol_type) ? CODING_REQUIRE_DETECTION_MASK : 0);
3410 4434
3411 if (!VECTORP (coding_spec) 4435 val = CODING_ATTR_SAFE_CHARSETS (attrs);
3412 || XVECTOR (coding_spec)->size != 5 4436 coding->max_charset_id = XSTRING (val)->size - 1;
3413 || !CONSP (XVECTOR (coding_spec)->contents[3])) 4437 coding->safe_charsets = (char *) XSTRING (val)->data;
3414 goto label_invalid_coding_system; 4438 coding->default_char = XINT (CODING_ATTR_DEFAULT_CHAR (attrs));
3415 4439
3416 eol_type = inhibit_eol_conversion ? Qnil : Fget (coding_system, Qeol_type); 4440 coding_type = CODING_ATTR_TYPE (attrs);
3417 if (VECTORP (eol_type)) 4441 if (EQ (coding_type, Qundecided))
3418 { 4442 {
3419 coding->eol_type = CODING_EOL_UNDECIDED; 4443 coding->detector = NULL;
3420 coding->common_flags = CODING_REQUIRE_DETECTION_MASK; 4444 coding->decoder = decode_coding_raw_text;
4445 coding->encoder = encode_coding_raw_text;
4446 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK;
3421 } 4447 }
3422 else if (XFASTINT (eol_type) == 1) 4448 else if (EQ (coding_type, Qiso_2022))
3423 { 4449 {
3424 coding->eol_type = CODING_EOL_CRLF; 4450 int i;
4451 int flags = XINT (AREF (attrs, coding_attr_iso_flags));
4452
4453 /* Invoke graphic register 0 to plane 0. */
4454 CODING_ISO_INVOCATION (coding, 0) = 0;
4455 /* Invoke graphic register 1 to plane 1 if we can use 8-bit. */
4456 CODING_ISO_INVOCATION (coding, 1)
4457 = (flags & CODING_ISO_FLAG_SEVEN_BITS ? -1 : 1);
4458 /* Setup the initial status of designation. */
4459 for (i = 0; i < 4; i++)
4460 CODING_ISO_DESIGNATION (coding, i) = CODING_ISO_INITIAL (coding, i);
4461 /* Not single shifting initially. */
4462 CODING_ISO_SINGLE_SHIFTING (coding) = 0;
4463 /* Beginning of buffer should also be regarded as bol. */
4464 CODING_ISO_BOL (coding) = 1;
4465 coding->detector = detect_coding_iso_2022;
4466 coding->decoder = decode_coding_iso_2022;
4467 coding->encoder = encode_coding_iso_2022;
4468 if (flags & CODING_ISO_FLAG_SAFE)
4469 coding->mode |= CODING_MODE_SAFE_ENCODING;
3425 coding->common_flags 4470 coding->common_flags
3426 = CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK; 4471 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
4472 | CODING_REQUIRE_FLUSHING_MASK);
4473 if (flags & CODING_ISO_FLAG_COMPOSITION)
4474 coding->common_flags |= CODING_ANNOTATE_COMPOSITION_MASK;
4475 if (flags & CODING_ISO_FLAG_FULL_SUPPORT)
4476 {
4477 setup_iso_safe_charsets (attrs);
4478 val = CODING_ATTR_SAFE_CHARSETS (attrs);
4479 coding->max_charset_id = XSTRING (val)->size - 1;
4480 coding->safe_charsets = (char *) XSTRING (val)->data;
4481 }
4482 CODING_ISO_FLAGS (coding) = flags;
3427 } 4483 }
3428 else if (XFASTINT (eol_type) == 2) 4484 else if (EQ (coding_type, Qcharset))
3429 { 4485 {
3430 coding->eol_type = CODING_EOL_CR; 4486 coding->detector = detect_coding_charset;
4487 coding->decoder = decode_coding_charset;
4488 coding->encoder = encode_coding_charset;
3431 coding->common_flags 4489 coding->common_flags
3432 = CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK; 4490 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
3433 } 4491 }
3434 else 4492 else if (EQ (coding_type, Qutf_8))
3435 coding->eol_type = CODING_EOL_LF;
3436
3437 coding_type = XVECTOR (coding_spec)->contents[0];
3438 /* Try short cut. */
3439 if (SYMBOLP (coding_type))
3440 { 4493 {
3441 if (EQ (coding_type, Qt)) 4494 coding->detector = detect_coding_utf_8;
3442 { 4495 coding->decoder = decode_coding_utf_8;
3443 coding->type = coding_type_undecided; 4496 coding->encoder = encode_coding_utf_8;
3444 coding->common_flags |= CODING_REQUIRE_DETECTION_MASK; 4497 coding->common_flags
3445 } 4498 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
3446 else
3447 coding->type = coding_type_no_conversion;
3448 /* Initialize this member. Any thing other than
3449 CODING_CATEGORY_IDX_UTF_16_BE and
3450 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3451 special treatment in detect_eol. */
3452 coding->category_idx = CODING_CATEGORY_IDX_EMACS_MULE;
3453
3454 return 0;
3455 }
3456
3457 /* Get values of coding system properties:
3458 `post-read-conversion', `pre-write-conversion',
3459 `translation-table-for-decode', `translation-table-for-encode'. */
3460 plist = XVECTOR (coding_spec)->contents[3];
3461 /* Pre & post conversion functions should be disabled if
3462 inhibit_eol_conversion is nonzero. This is the case that a code
3463 conversion function is called while those functions are running. */
3464 if (! inhibit_pre_post_conversion)
3465 {
3466 coding->post_read_conversion = Fplist_get (plist, Qpost_read_conversion);
3467 coding->pre_write_conversion = Fplist_get (plist, Qpre_write_conversion);
3468 }
3469 val = Fplist_get (plist, Qtranslation_table_for_decode);
3470 if (SYMBOLP (val))
3471 val = Fget (val, Qtranslation_table_for_decode);
3472 coding->translation_table_for_decode = CHAR_TABLE_P (val) ? val : Qnil;
3473 val = Fplist_get (plist, Qtranslation_table_for_encode);
3474 if (SYMBOLP (val))
3475 val = Fget (val, Qtranslation_table_for_encode);
3476 coding->translation_table_for_encode = CHAR_TABLE_P (val) ? val : Qnil;
3477 val = Fplist_get (plist, Qcoding_category);
3478 if (!NILP (val))
3479 {
3480 val = Fget (val, Qcoding_category_index);
3481 if (INTEGERP (val))
3482 coding->category_idx = XINT (val);
3483 else
3484 goto label_invalid_coding_system;
3485 } 4499 }
3486 else 4500 else if (EQ (coding_type, Qutf_16))
3487 goto label_invalid_coding_system;
3488
3489 /* If the coding system has non-nil `composition' property, enable
3490 composition handling. */
3491 val = Fplist_get (plist, Qcomposition);
3492 if (!NILP (val))
3493 coding->composing = COMPOSITION_NO;
3494
3495 switch (XFASTINT (coding_type))
3496 { 4501 {
3497 case 0: 4502 val = AREF (attrs, coding_attr_utf_16_bom);
3498 coding->type = coding_type_emacs_mule; 4503 CODING_UTF_16_BOM (coding) = (CONSP (val) ? utf_16_detect_bom
4504 : EQ (val, Qt) ? utf_16_with_bom
4505 : utf_16_without_bom);
4506 val = AREF (attrs, coding_attr_utf_16_endian);
4507 CODING_UTF_16_ENDIAN (coding) = (NILP (val) ? utf_16_big_endian
4508 : utf_16_little_endian);
4509 coding->detector = detect_coding_utf_16;
4510 coding->decoder = decode_coding_utf_16;
4511 coding->encoder = encode_coding_utf_16;
3499 coding->common_flags 4512 coding->common_flags
3500 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK; 4513 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
3501 coding->composing = COMPOSITION_NO; 4514 }
3502 if (!NILP (coding->post_read_conversion)) 4515 else if (EQ (coding_type, Qccl))
3503 coding->common_flags |= CODING_REQUIRE_DECODING_MASK; 4516 {
3504 if (!NILP (coding->pre_write_conversion)) 4517 coding->detector = detect_coding_ccl;
3505 coding->common_flags |= CODING_REQUIRE_ENCODING_MASK; 4518 coding->decoder = decode_coding_ccl;
3506 break; 4519 coding->encoder = encode_coding_ccl;
3507
3508 case 1:
3509 coding->type = coding_type_sjis;
3510 coding->common_flags 4520 coding->common_flags
3511 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK; 4521 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK
3512 break; 4522 | CODING_REQUIRE_FLUSHING_MASK);
3513 4523 }
3514 case 2: 4524 else if (EQ (coding_type, Qemacs_mule))
3515 coding->type = coding_type_iso2022; 4525 {
4526 coding->detector = detect_coding_emacs_mule;
4527 coding->decoder = decode_coding_emacs_mule;
4528 coding->encoder = encode_coding_emacs_mule;
3516 coding->common_flags 4529 coding->common_flags
3517 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK; 4530 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
3518 { 4531 if (! NILP (AREF (attrs, coding_attr_emacs_mule_full))
3519 Lisp_Object val, temp; 4532 && ! EQ (CODING_ATTR_CHARSET_LIST (attrs), Vemacs_mule_charset_list))
3520 Lisp_Object *flags; 4533 {
3521 int i, charset, reg_bits = 0; 4534 Lisp_Object tail, safe_charsets;
3522 4535 int max_charset_id = 0;
3523 val = XVECTOR (coding_spec)->contents[4]; 4536
3524 4537 for (tail = Vemacs_mule_charset_list; CONSP (tail);
3525 if (!VECTORP (val) || XVECTOR (val)->size != 32) 4538 tail = XCDR (tail))
3526 goto label_invalid_coding_system; 4539 if (max_charset_id < XFASTINT (XCAR (tail)))
3527 4540 max_charset_id = XFASTINT (XCAR (tail));
3528 flags = XVECTOR (val)->contents; 4541 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
3529 coding->flags 4542 make_number (255));
3530 = ((NILP (flags[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM) 4543 for (tail = Vemacs_mule_charset_list; CONSP (tail);
3531 | (NILP (flags[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL) 4544 tail = XCDR (tail))
3532 | (NILP (flags[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL) 4545 XSTRING (safe_charsets)->data[XFASTINT (XCAR (tail))] = 0;
3533 | (NILP (flags[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS) 4546 coding->max_charset_id = max_charset_id;
3534 | (NILP (flags[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT) 4547 coding->safe_charsets = (char *) XSTRING (safe_charsets)->data;
3535 | (NILP (flags[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT) 4548 }
3536 | (NILP (flags[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN) 4549 }
3537 | (NILP (flags[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS) 4550 else if (EQ (coding_type, Qshift_jis))
3538 | (NILP (flags[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION) 4551 {
3539 | (NILP (flags[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL) 4552 coding->detector = detect_coding_sjis;
3540 | (NILP (flags[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL) 4553 coding->decoder = decode_coding_sjis;
3541 | (NILP (flags[15]) ? 0 : CODING_FLAG_ISO_SAFE) 4554 coding->encoder = encode_coding_sjis;
3542 | (NILP (flags[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA)
3543 );
3544
3545 /* Invoke graphic register 0 to plane 0. */
3546 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0;
3547 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3548 CODING_SPEC_ISO_INVOCATION (coding, 1)
3549 = (coding->flags & CODING_FLAG_ISO_SEVEN_BITS ? -1 : 1);
3550 /* Not single shifting at first. */
3551 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0;
3552 /* Beginning of buffer should also be regarded as bol. */
3553 CODING_SPEC_ISO_BOL (coding) = 1;
3554
3555 for (charset = 0; charset <= MAX_CHARSET; charset++)
3556 CODING_SPEC_ISO_REVISION_NUMBER (coding, charset) = 255;
3557 val = Vcharset_revision_alist;
3558 while (CONSP (val))
3559 {
3560 charset = get_charset_id (Fcar_safe (XCAR (val)));
3561 if (charset >= 0
3562 && (temp = Fcdr_safe (XCAR (val)), INTEGERP (temp))
3563 && (i = XINT (temp), (i >= 0 && (i + '@') < 128)))
3564 CODING_SPEC_ISO_REVISION_NUMBER (coding, charset) = i;
3565 val = XCDR (val);
3566 }
3567
3568 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3569 FLAGS[REG] can be one of below:
3570 integer CHARSET: CHARSET occupies register I,
3571 t: designate nothing to REG initially, but can be used
3572 by any charsets,
3573 list of integer, nil, or t: designate the first
3574 element (if integer) to REG initially, the remaining
3575 elements (if integer) is designated to REG on request,
3576 if an element is t, REG can be used by any charsets,
3577 nil: REG is never used. */
3578 for (charset = 0; charset <= MAX_CHARSET; charset++)
3579 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3580 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION;
3581 for (i = 0; i < 4; i++)
3582 {
3583 if ((INTEGERP (flags[i])
3584 && (charset = XINT (flags[i]), CHARSET_VALID_P (charset)))
3585 || (charset = get_charset_id (flags[i])) >= 0)
3586 {
3587 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = charset;
3588 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) = i;
3589 }
3590 else if (EQ (flags[i], Qt))
3591 {
3592 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = -1;
3593 reg_bits |= 1 << i;
3594 coding->flags |= CODING_FLAG_ISO_DESIGNATION;
3595 }
3596 else if (CONSP (flags[i]))
3597 {
3598 Lisp_Object tail;
3599 tail = flags[i];
3600
3601 coding->flags |= CODING_FLAG_ISO_DESIGNATION;
3602 if ((INTEGERP (XCAR (tail))
3603 && (charset = XINT (XCAR (tail)),
3604 CHARSET_VALID_P (charset)))
3605 || (charset = get_charset_id (XCAR (tail))) >= 0)
3606 {
3607 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = charset;
3608 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) =i;
3609 }
3610 else
3611 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = -1;
3612 tail = XCDR (tail);
3613 while (CONSP (tail))
3614 {
3615 if ((INTEGERP (XCAR (tail))
3616 && (charset = XINT (XCAR (tail)),
3617 CHARSET_VALID_P (charset)))
3618 || (charset = get_charset_id (XCAR (tail))) >= 0)
3619 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3620 = i;
3621 else if (EQ (XCAR (tail), Qt))
3622 reg_bits |= 1 << i;
3623 tail = XCDR (tail);
3624 }
3625 }
3626 else
3627 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i) = -1;
3628
3629 CODING_SPEC_ISO_DESIGNATION (coding, i)
3630 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, i);
3631 }
3632
3633 if (reg_bits && ! (coding->flags & CODING_FLAG_ISO_LOCKING_SHIFT))
3634 {
3635 /* REG 1 can be used only by locking shift in 7-bit env. */
3636 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS)
3637 reg_bits &= ~2;
3638 if (! (coding->flags & CODING_FLAG_ISO_SINGLE_SHIFT))
3639 /* Without any shifting, only REG 0 and 1 can be used. */
3640 reg_bits &= 3;
3641 }
3642
3643 if (reg_bits)
3644 for (charset = 0; charset <= MAX_CHARSET; charset++)
3645 {
3646 if (CHARSET_DEFINED_P (charset)
3647 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3648 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
3649 {
3650 /* There exist some default graphic registers to be
3651 used by CHARSET. */
3652
3653 /* We had better avoid designating a charset of
3654 CHARS96 to REG 0 as far as possible. */
3655 if (CHARSET_CHARS (charset) == 96)
3656 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3657 = (reg_bits & 2
3658 ? 1 : (reg_bits & 4 ? 2 : (reg_bits & 8 ? 3 : 0)));
3659 else
3660 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset)
3661 = (reg_bits & 1
3662 ? 0 : (reg_bits & 2 ? 1 : (reg_bits & 4 ? 2 : 3)));
3663 }
3664 }
3665 }
3666 coding->common_flags |= CODING_REQUIRE_FLUSHING_MASK;
3667 coding->spec.iso2022.last_invalid_designation_register = -1;
3668 break;
3669
3670 case 3:
3671 coding->type = coding_type_big5;
3672 coding->common_flags 4555 coding->common_flags
3673 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK; 4556 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
3674 coding->flags 4557 }
3675 = (NILP (XVECTOR (coding_spec)->contents[4]) 4558 else if (EQ (coding_type, Qbig5))
3676 ? CODING_FLAG_BIG5_HKU 4559 {
3677 : CODING_FLAG_BIG5_ETEN); 4560 coding->detector = detect_coding_big5;
3678 break; 4561 coding->decoder = decode_coding_big5;
3679 4562 coding->encoder = encode_coding_big5;
3680 case 4:
3681 coding->type = coding_type_ccl;
3682 coding->common_flags 4563 coding->common_flags
3683 |= CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK; 4564 |= (CODING_REQUIRE_DECODING_MASK | CODING_REQUIRE_ENCODING_MASK);
3684 {
3685 val = XVECTOR (coding_spec)->contents[4];
3686 if (! CONSP (val)
3687 || setup_ccl_program (&(coding->spec.ccl.decoder),
3688 XCAR (val)) < 0
3689 || setup_ccl_program (&(coding->spec.ccl.encoder),
3690 XCDR (val)) < 0)
3691 goto label_invalid_coding_system;
3692
3693 bzero (coding->spec.ccl.valid_codes, 256);
3694 val = Fplist_get (plist, Qvalid_codes);
3695 if (CONSP (val))
3696 {
3697 Lisp_Object this;
3698
3699 for (; CONSP (val); val = XCDR (val))
3700 {
3701 this = XCAR (val);
3702 if (INTEGERP (this)
3703 && XINT (this) >= 0 && XINT (this) < 256)
3704 coding->spec.ccl.valid_codes[XINT (this)] = 1;
3705 else if (CONSP (this)
3706 && INTEGERP (XCAR (this))
3707 && INTEGERP (XCDR (this)))
3708 {
3709 int start = XINT (XCAR (this));
3710 int end = XINT (XCDR (this));
3711
3712 if (start >= 0 && start <= end && end < 256)
3713 while (start <= end)
3714 coding->spec.ccl.valid_codes[start++] = 1;
3715 }
3716 }
3717 }
3718 }
3719 coding->common_flags |= CODING_REQUIRE_FLUSHING_MASK;
3720 coding->spec.ccl.cr_carryover = 0;
3721 coding->spec.ccl.eight_bit_carryover[0] = 0;
3722 break;
3723
3724 case 5:
3725 coding->type = coding_type_raw_text;
3726 break;
3727
3728 default:
3729 goto label_invalid_coding_system;
3730 } 4565 }
3731 return 0; 4566 else /* EQ (coding_type, Qraw_text) */
3732
3733 label_invalid_coding_system:
3734 coding->type = coding_type_no_conversion;
3735 coding->category_idx = CODING_CATEGORY_IDX_BINARY;
3736 coding->common_flags = 0;
3737 coding->eol_type = CODING_EOL_LF;
3738 coding->pre_write_conversion = coding->post_read_conversion = Qnil;
3739 return -1;
3740}
3741
3742/* Free memory blocks allocated for storing composition information. */
3743
3744void
3745coding_free_composition_data (coding)
3746 struct coding_system *coding;
3747{
3748 struct composition_data *cmp_data = coding->cmp_data, *next;
3749
3750 if (!cmp_data)
3751 return;
3752 /* Memory blocks are chained. At first, rewind to the first, then,
3753 free blocks one by one. */
3754 while (cmp_data->prev)
3755 cmp_data = cmp_data->prev;
3756 while (cmp_data)
3757 { 4567 {
3758 next = cmp_data->next; 4568 coding->detector = NULL;
3759 xfree (cmp_data); 4569 coding->decoder = decode_coding_raw_text;
3760 cmp_data = next; 4570 coding->encoder = encode_coding_raw_text;
4571 coding->common_flags |= CODING_FOR_UNIBYTE_MASK;
3761 } 4572 }
3762 coding->cmp_data = NULL; 4573
4574 return;
3763} 4575}
3764 4576
3765/* Set `char_offset' member of all memory blocks pointed by 4577/* Return raw-text or one of its subsidiaries that has the same
3766 coding->cmp_data to POS. */ 4578 eol_type as CODING-SYSTEM. */
3767 4579
3768void 4580Lisp_Object
3769coding_adjust_composition_offset (coding, pos) 4581raw_text_coding_system (coding_system)
3770 struct coding_system *coding; 4582 Lisp_Object coding_system;
3771 int pos;
3772{ 4583{
3773 struct composition_data *cmp_data; 4584 Lisp_Object spec, attrs, coding_type;
4585 Lisp_Object eol_type, raw_text_eol_type;
4586
4587 spec = CODING_SYSTEM_SPEC (coding_system);
4588 attrs = AREF (spec, 0);
4589
4590 if (EQ (CODING_ATTR_TYPE (attrs), Qraw_text))
4591 return coding_system;
3774 4592
3775 for (cmp_data = coding->cmp_data; cmp_data; cmp_data = cmp_data->next) 4593 eol_type = AREF (spec, 2);
3776 cmp_data->char_offset = pos; 4594 if (VECTORP (eol_type))
4595 return Qraw_text;
4596 spec = CODING_SYSTEM_SPEC (Qraw_text);
4597 raw_text_eol_type = AREF (spec, 2);
4598 return (EQ (eol_type, Qunix) ? AREF (raw_text_eol_type, 0)
4599 : EQ (eol_type, Qdos) ? AREF (raw_text_eol_type, 1)
4600 : AREF (raw_text_eol_type, 2));
3777} 4601}
3778 4602
3779/* Setup raw-text or one of its subsidiaries in the structure
3780 coding_system CODING according to the already setup value eol_type
3781 in CODING. CODING should be setup for some coding system in
3782 advance. */
3783 4603
3784void 4604/* If CODING_SYSTEM doesn't specify end-of-line format but PARENT
3785setup_raw_text_coding_system (coding) 4605 does, return one of the subsidiary that has the same eol-spec as
3786 struct coding_system *coding; 4606 PARENT. Otherwise, return CODING_SYSTEM. */
4607
4608Lisp_Object
4609coding_inherit_eol_type (coding_system, parent)
3787{ 4610{
3788 if (coding->type != coding_type_raw_text) 4611 Lisp_Object spec, attrs, eol_type;
3789 {
3790 coding->symbol = Qraw_text;
3791 coding->type = coding_type_raw_text;
3792 if (coding->eol_type != CODING_EOL_UNDECIDED)
3793 {
3794 Lisp_Object subsidiaries;
3795 subsidiaries = Fget (Qraw_text, Qeol_type);
3796 4612
3797 if (VECTORP (subsidiaries) 4613 spec = CODING_SYSTEM_SPEC (coding_system);
3798 && XVECTOR (subsidiaries)->size == 3) 4614 attrs = AREF (spec, 0);
3799 coding->symbol 4615 eol_type = AREF (spec, 2);
3800 = XVECTOR (subsidiaries)->contents[coding->eol_type]; 4616 if (VECTORP (eol_type))
3801 } 4617 {
3802 setup_coding_system (coding->symbol, coding); 4618 Lisp_Object parent_spec;
4619 Lisp_Object parent_attrs;
4620 Lisp_Object parent_eol_type;
4621
4622 parent_spec
4623 = CODING_SYSTEM_SPEC (buffer_defaults.buffer_file_coding_system);
4624 parent_eol_type = AREF (parent_spec, 2);
4625 if (EQ (parent_eol_type, Qunix))
4626 coding_system = AREF (eol_type, 0);
4627 else if (EQ (parent_eol_type, Qdos))
4628 coding_system = AREF (eol_type, 1);
4629 else if (EQ (parent_eol_type, Qmac))
4630 coding_system = AREF (eol_type, 2);
3803 } 4631 }
3804 return; 4632 return coding_system;
3805} 4633}
3806 4634
3807/* Emacs has a mechanism to automatically detect a coding system if it 4635/* Emacs has a mechanism to automatically detect a coding system if it
@@ -3854,14 +4682,14 @@ setup_raw_text_coding_system (coding)
3854 o coding-category-iso-7-else 4682 o coding-category-iso-7-else
3855 4683
3856 The category for a coding system which has the same code range 4684 The category for a coding system which has the same code range
3857 as ISO2022 of 7-bit environment but uses locking shift or 4685 as ISO2022 of 7-bit environemnt but uses locking shift or
3858 single shift functions. Assigned the coding-system (Lisp 4686 single shift functions. Assigned the coding-system (Lisp
3859 symbol) `iso-2022-7bit-lock' by default. 4687 symbol) `iso-2022-7bit-lock' by default.
3860 4688
3861 o coding-category-iso-8-else 4689 o coding-category-iso-8-else
3862 4690
3863 The category for a coding system which has the same code range 4691 The category for a coding system which has the same code range
3864 as ISO2022 of 8-bit environment but uses locking shift or 4692 as ISO2022 of 8-bit environemnt but uses locking shift or
3865 single shift functions. Assigned the coding-system (Lisp 4693 single shift functions. Assigned the coding-system (Lisp
3866 symbol) `iso-2022-8bit-ss2' by default. 4694 symbol) `iso-2022-8bit-ss2' by default.
3867 4695
@@ -3904,2204 +4732,1296 @@ setup_raw_text_coding_system (coding)
3904 `no-conversion' by default. 4732 `no-conversion' by default.
3905 4733
3906 Each of them is a Lisp symbol and the value is an actual 4734 Each of them is a Lisp symbol and the value is an actual
3907 `coding-system' (this is also a Lisp symbol) assigned by a user. 4735 `coding-system's (this is also a Lisp symbol) assigned by a user.
3908 What Emacs does actually is to detect a category of coding system. 4736 What Emacs does actually is to detect a category of coding system.
3909 Then, it uses a `coding-system' assigned to it. If Emacs can't 4737 Then, it uses a `coding-system' assigned to it. If Emacs can't
3910 decide a single possible category, it selects a category of the 4738 decide only one possible category, it selects a category of the
3911 highest priority. Priorities of categories are also specified by a 4739 highest priority. Priorities of categories are also specified by a
3912 user in a Lisp variable `coding-category-list'. 4740 user in a Lisp variable `coding-category-list'.
3913 4741
3914*/ 4742*/
3915 4743
3916static 4744#define EOL_SEEN_NONE 0
3917int ascii_skip_code[256]; 4745#define EOL_SEEN_LF 1
4746#define EOL_SEEN_CR 2
4747#define EOL_SEEN_CRLF 4
3918 4748
3919/* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded. 4749/* Detect how end-of-line of a text of length CODING->src_bytes
3920 If it detects possible coding systems, return an integer in which 4750 pointed by CODING->source is encoded. Return one of
3921 appropriate flag bits are set. Flag bits are defined by macros 4751 EOL_SEEN_XXX. */
3922 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
3923 it should point the table `coding_priorities'. In that case, only
3924 the flag bit for a coding system of the highest priority is set in
3925 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
3926 range 0x80..0x9F are in multibyte form.
3927 4752
3928 How many ASCII characters are at the head is returned as *SKIP. */ 4753#define MAX_EOL_CHECK_COUNT 3
3929 4754
3930static int 4755static int
3931detect_coding_mask (source, src_bytes, priorities, skip, multibytep) 4756detect_eol (coding, source, src_bytes)
4757 struct coding_system *coding;
3932 unsigned char *source; 4758 unsigned char *source;
3933 int src_bytes, *priorities, *skip; 4759 EMACS_INT src_bytes;
3934 int multibytep;
3935{ 4760{
3936 register unsigned char c; 4761 Lisp_Object attrs, coding_type;
3937 unsigned char *src = source, *src_end = source + src_bytes; 4762 unsigned char *src = source, *src_end = src + src_bytes;
3938 unsigned int mask, utf16_examined_p, iso2022_examined_p; 4763 unsigned char c;
3939 int i; 4764 int total = 0;
4765 int eol_seen = EOL_SEEN_NONE;
4766 int first_eol_seen;
3940 4767
3941 /* At first, skip all ASCII characters and control characters except 4768 attrs = CODING_ID_ATTRS (coding->id);
3942 for three ISO2022 specific control characters. */ 4769 coding_type = CODING_ATTR_TYPE (attrs);
3943 ascii_skip_code[ISO_CODE_SO] = 0;
3944 ascii_skip_code[ISO_CODE_SI] = 0;
3945 ascii_skip_code[ISO_CODE_ESC] = 0;
3946 4770
3947 label_loop_detect_coding: 4771 if (EQ (coding_type, Qccl))
3948 while (src < src_end && ascii_skip_code[*src]) src++; 4772 {
3949 *skip = src - source; 4773 int msb, lsb;
3950 4774
3951 if (src >= src_end) 4775 msb = coding->spec.utf_16.endian == utf_16_little_endian;
3952 /* We found nothing other than ASCII. There's nothing to do. */ 4776 lsb = 1 - msb;
3953 return 0;
3954 4777
3955 c = *src; 4778 while (src + 1 < src_end)
3956 /* The text seems to be encoded in some multilingual coding system.
3957 Now, try to find in which coding system the text is encoded. */
3958 if (c < 0x80)
3959 {
3960 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
3961 /* C is an ISO2022 specific control code of C0. */
3962 mask = detect_coding_iso2022 (src, src_end, multibytep);
3963 if (mask == 0)
3964 { 4779 {
3965 /* No valid ISO2022 code follows C. Try again. */ 4780 c = src[lsb];
3966 src++; 4781 if (src[msb] == 0 && (c == '\n' || c == '\r'))
3967 if (c == ISO_CODE_ESC)
3968 ascii_skip_code[ISO_CODE_ESC] = 1;
3969 else
3970 ascii_skip_code[ISO_CODE_SO] = ascii_skip_code[ISO_CODE_SI] = 1;
3971 goto label_loop_detect_coding;
3972 }
3973 if (priorities)
3974 {
3975 for (i = 0; i < CODING_CATEGORY_IDX_MAX; i++)
3976 { 4782 {
3977 if (mask & priorities[i]) 4783 int this_eol;
3978 return priorities[i]; 4784
4785 if (c == '\n')
4786 this_eol = EOL_SEEN_LF;
4787 else if (src + 3 >= src_end
4788 || src[msb + 2] != 0
4789 || src[lsb + 2] != '\n')
4790 this_eol = EOL_SEEN_CR;
4791 else
4792 this_eol = EOL_SEEN_CRLF;
4793
4794 if (eol_seen == EOL_SEEN_NONE)
4795 /* This is the first end-of-line. */
4796 eol_seen = this_eol;
4797 else if (eol_seen != this_eol)
4798 {
4799 /* The found type is different from what found before. */
4800 eol_seen = EOL_SEEN_LF;
4801 break;
4802 }
4803 if (++total == MAX_EOL_CHECK_COUNT)
4804 break;
3979 } 4805 }
3980 return CODING_CATEGORY_MASK_RAW_TEXT; 4806 src += 2;
3981 } 4807 }
3982 } 4808 }
3983 else 4809 else
3984 { 4810 {
3985 int try; 4811 while (src < src_end)
3986
3987 if (multibytep && c == LEADING_CODE_8_BIT_CONTROL)
3988 c = src[1] - 0x20;
3989
3990 if (c < 0xA0)
3991 {
3992 /* C is the first byte of SJIS character code,
3993 or a leading-code of Emacs' internal format (emacs-mule),
3994 or the first byte of UTF-16. */
3995 try = (CODING_CATEGORY_MASK_SJIS
3996 | CODING_CATEGORY_MASK_EMACS_MULE
3997 | CODING_CATEGORY_MASK_UTF_16_BE
3998 | CODING_CATEGORY_MASK_UTF_16_LE);
3999
4000 /* Or, if C is a special latin extra code,
4001 or is an ISO2022 specific control code of C1 (SS2 or SS3),
4002 or is an ISO2022 control-sequence-introducer (CSI),
4003 we should also consider the possibility of ISO2022 codings. */
4004 if ((VECTORP (Vlatin_extra_code_table)
4005 && !NILP (XVECTOR (Vlatin_extra_code_table)->contents[c]))
4006 || (c == ISO_CODE_SS2 || c == ISO_CODE_SS3)
4007 || (c == ISO_CODE_CSI
4008 && (src < src_end
4009 && (*src == ']'
4010 || ((*src == '0' || *src == '1' || *src == '2')
4011 && src + 1 < src_end
4012 && src[1] == ']')))))
4013 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4014 | CODING_CATEGORY_MASK_ISO_8BIT);
4015 }
4016 else
4017 /* C is a character of ISO2022 in graphic plane right,
4018 or a SJIS's 1-byte character code (i.e. JISX0201),
4019 or the first byte of BIG5's 2-byte code,
4020 or the first byte of UTF-8/16. */
4021 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4022 | CODING_CATEGORY_MASK_ISO_8BIT
4023 | CODING_CATEGORY_MASK_SJIS
4024 | CODING_CATEGORY_MASK_BIG5
4025 | CODING_CATEGORY_MASK_UTF_8
4026 | CODING_CATEGORY_MASK_UTF_16_BE
4027 | CODING_CATEGORY_MASK_UTF_16_LE);
4028
4029 /* Or, we may have to consider the possibility of CCL. */
4030 if (coding_system_table[CODING_CATEGORY_IDX_CCL]
4031 && (coding_system_table[CODING_CATEGORY_IDX_CCL]
4032 ->spec.ccl.valid_codes)[c])
4033 try |= CODING_CATEGORY_MASK_CCL;
4034
4035 mask = 0;
4036 utf16_examined_p = iso2022_examined_p = 0;
4037 if (priorities)
4038 { 4812 {
4039 for (i = 0; i < CODING_CATEGORY_IDX_MAX; i++) 4813 c = *src++;
4814 if (c == '\n' || c == '\r')
4040 { 4815 {
4041 if (!iso2022_examined_p 4816 int this_eol;
4042 && (priorities[i] & try & CODING_CATEGORY_MASK_ISO)) 4817
4043 { 4818 if (c == '\n')
4044 mask |= detect_coding_iso2022 (src, src_end, multibytep); 4819 this_eol = EOL_SEEN_LF;
4045 iso2022_examined_p = 1; 4820 else if (src >= src_end || *src != '\n')
4046 } 4821 this_eol = EOL_SEEN_CR;
4047 else if (priorities[i] & try & CODING_CATEGORY_MASK_SJIS) 4822 else
4048 mask |= detect_coding_sjis (src, src_end, multibytep); 4823 this_eol = EOL_SEEN_CRLF, src++;
4049 else if (priorities[i] & try & CODING_CATEGORY_MASK_UTF_8) 4824
4050 mask |= detect_coding_utf_8 (src, src_end, multibytep); 4825 if (eol_seen == EOL_SEEN_NONE)
4051 else if (!utf16_examined_p 4826 /* This is the first end-of-line. */
4052 && (priorities[i] & try & 4827 eol_seen = this_eol;
4053 CODING_CATEGORY_MASK_UTF_16_BE_LE)) 4828 else if (eol_seen != this_eol)
4054 { 4829 {
4055 mask |= detect_coding_utf_16 (src, src_end, multibytep); 4830 /* The found type is different from what found before. */
4056 utf16_examined_p = 1; 4831 eol_seen = EOL_SEEN_LF;
4832 break;
4057 } 4833 }
4058 else if (priorities[i] & try & CODING_CATEGORY_MASK_BIG5) 4834 if (++total == MAX_EOL_CHECK_COUNT)
4059 mask |= detect_coding_big5 (src, src_end, multibytep); 4835 break;
4060 else if (priorities[i] & try & CODING_CATEGORY_MASK_EMACS_MULE)
4061 mask |= detect_coding_emacs_mule (src, src_end, multibytep);
4062 else if (priorities[i] & try & CODING_CATEGORY_MASK_CCL)
4063 mask |= detect_coding_ccl (src, src_end, multibytep);
4064 else if (priorities[i] & CODING_CATEGORY_MASK_RAW_TEXT)
4065 mask |= CODING_CATEGORY_MASK_RAW_TEXT;
4066 else if (priorities[i] & CODING_CATEGORY_MASK_BINARY)
4067 mask |= CODING_CATEGORY_MASK_BINARY;
4068 if (mask & priorities[i])
4069 return priorities[i];
4070 } 4836 }
4071 return CODING_CATEGORY_MASK_RAW_TEXT;
4072 } 4837 }
4073 if (try & CODING_CATEGORY_MASK_ISO) 4838 }
4074 mask |= detect_coding_iso2022 (src, src_end, multibytep); 4839 return eol_seen;
4075 if (try & CODING_CATEGORY_MASK_SJIS)
4076 mask |= detect_coding_sjis (src, src_end, multibytep);
4077 if (try & CODING_CATEGORY_MASK_BIG5)
4078 mask |= detect_coding_big5 (src, src_end, multibytep);
4079 if (try & CODING_CATEGORY_MASK_UTF_8)
4080 mask |= detect_coding_utf_8 (src, src_end, multibytep);
4081 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE)
4082 mask |= detect_coding_utf_16 (src, src_end, multibytep);
4083 if (try & CODING_CATEGORY_MASK_EMACS_MULE)
4084 mask |= detect_coding_emacs_mule (src, src_end, multibytep);
4085 if (try & CODING_CATEGORY_MASK_CCL)
4086 mask |= detect_coding_ccl (src, src_end, multibytep);
4087 }
4088 return (mask | CODING_CATEGORY_MASK_RAW_TEXT | CODING_CATEGORY_MASK_BINARY);
4089} 4840}
4090 4841
4091/* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4092 The information of the detected coding system is set in CODING. */
4093 4842
4094void 4843static void
4095detect_coding (coding, src, src_bytes) 4844adjust_coding_eol_type (coding, eol_seen)
4096 struct coding_system *coding; 4845 struct coding_system *coding;
4097 unsigned char *src; 4846 int eol_seen;
4098 int src_bytes;
4099{ 4847{
4100 unsigned int idx; 4848 Lisp_Object eol_type, coding_system;
4101 int skip, mask; 4849
4102 Lisp_Object val; 4850 eol_type = CODING_ID_EOL_TYPE (coding->id);
4103 4851 if (eol_seen & EOL_SEEN_LF)
4104 val = Vcoding_category_list; 4852 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 0));
4105 mask = detect_coding_mask (src, src_bytes, coding_priorities, &skip, 4853 else if (eol_type & EOL_SEEN_CRLF)
4106 coding->src_multibyte); 4854 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 1));
4107 coding->heading_ascii = skip; 4855 else if (eol_type & EOL_SEEN_CR)
4108 4856 coding->id = CODING_SYSTEM_ID (AREF (eol_type, 2));
4109 if (!mask) return;
4110
4111 /* We found a single coding system of the highest priority in MASK. */
4112 idx = 0;
4113 while (mask && ! (mask & 1)) mask >>= 1, idx++;
4114 if (! mask)
4115 idx = CODING_CATEGORY_IDX_RAW_TEXT;
4116
4117 val = SYMBOL_VALUE (XVECTOR (Vcoding_category_table)->contents[idx]);
4118
4119 if (coding->eol_type != CODING_EOL_UNDECIDED)
4120 {
4121 Lisp_Object tmp;
4122
4123 tmp = Fget (val, Qeol_type);
4124 if (VECTORP (tmp))
4125 val = XVECTOR (tmp)->contents[coding->eol_type];
4126 }
4127
4128 /* Setup this new coding system while preserving some slots. */
4129 {
4130 int src_multibyte = coding->src_multibyte;
4131 int dst_multibyte = coding->dst_multibyte;
4132
4133 setup_coding_system (val, coding);
4134 coding->src_multibyte = src_multibyte;
4135 coding->dst_multibyte = dst_multibyte;
4136 coding->heading_ascii = skip;
4137 }
4138} 4857}
4139 4858
4140/* Detect how end-of-line of a text of length SRC_BYTES pointed by 4859/* Detect how a text specified in CODING is encoded. If a coding
4141 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF, 4860 system is detected, update fields of CODING by the detected coding
4142 CODING_EOL_CR, and CODING_EOL_UNDECIDED. 4861 system. */
4143
4144 How many non-eol characters are at the head is returned as *SKIP. */
4145 4862
4146#define MAX_EOL_CHECK_COUNT 3 4863void
4147 4864detect_coding (coding)
4148static int 4865 struct coding_system *coding;
4149detect_eol_type (source, src_bytes, skip)
4150 unsigned char *source;
4151 int src_bytes, *skip;
4152{ 4866{
4153 unsigned char *src = source, *src_end = src + src_bytes; 4867 unsigned char *src, *src_end;
4154 unsigned char c; 4868 Lisp_Object attrs, coding_type;
4155 int total = 0; /* How many end-of-lines are found so far. */ 4869
4156 int eol_type = CODING_EOL_UNDECIDED; 4870 coding->consumed = coding->consumed_char = 0;
4157 int this_eol_type; 4871 coding->produced = coding->produced_char = 0;
4872 coding_set_source (coding);
4158 4873
4159 *skip = 0; 4874 src_end = coding->source + coding->src_bytes;
4160 4875
4161 while (src < src_end && total < MAX_EOL_CHECK_COUNT) 4876 /* If we have not yet decided the text encoding type, detect it
4877 now. */
4878 if (EQ (CODING_ATTR_TYPE (CODING_ID_ATTRS (coding->id)), Qundecided))
4162 { 4879 {
4163 c = *src++; 4880 int mask = CATEGORY_MASK_ANY;
4164 if (c == '\n' || c == '\r') 4881 int c, i;
4165 {
4166 if (*skip == 0)
4167 *skip = src - 1 - source;
4168 total++;
4169 if (c == '\n')
4170 this_eol_type = CODING_EOL_LF;
4171 else if (src >= src_end || *src != '\n')
4172 this_eol_type = CODING_EOL_CR;
4173 else
4174 this_eol_type = CODING_EOL_CRLF, src++;
4175 4882
4176 if (eol_type == CODING_EOL_UNDECIDED) 4883 for (src = coding->source; src < src_end; src++)
4177 /* This is the first end-of-line. */ 4884 {
4178 eol_type = this_eol_type; 4885 c = *src;
4179 else if (eol_type != this_eol_type) 4886 if (c & 0x80 || (c < 0x20 && (c == ISO_CODE_ESC
4180 { 4887 || c == ISO_CODE_SI
4181 /* The found type is different from what found before. */ 4888 || c == ISO_CODE_SO)))
4182 eol_type = CODING_EOL_INCONSISTENT; 4889 break;
4183 break;
4184 }
4185 } 4890 }
4186 } 4891 coding->head_ascii = src - (coding->source + coding->consumed);
4187 4892
4188 if (*skip == 0) 4893 if (coding->head_ascii < coding->src_bytes)
4189 *skip = src_end - source; 4894 {
4190 return eol_type; 4895 int detected = 0;
4191}
4192
4193/* Like detect_eol_type, but detect EOL type in 2-octet
4194 big-endian/little-endian format for coding systems utf-16-be and
4195 utf-16-le. */
4196
4197static int
4198detect_eol_type_in_2_octet_form (source, src_bytes, skip, big_endian_p)
4199 unsigned char *source;
4200 int src_bytes, *skip, big_endian_p;
4201{
4202 unsigned char *src = source, *src_end = src + src_bytes;
4203 unsigned int c1, c2;
4204 int total = 0; /* How many end-of-lines are found so far. */
4205 int eol_type = CODING_EOL_UNDECIDED;
4206 int this_eol_type;
4207 int msb, lsb;
4208
4209 if (big_endian_p)
4210 msb = 0, lsb = 1;
4211 else
4212 msb = 1, lsb = 0;
4213 4896
4214 *skip = 0; 4897 for (i = 0; i < coding_category_raw_text; i++)
4898 {
4899 enum coding_category category = coding_priorities[i];
4900 struct coding_system *this = coding_categories + category;
4215 4901
4216 while ((src + 1) < src_end && total < MAX_EOL_CHECK_COUNT) 4902 if (category >= coding_category_raw_text
4217 { 4903 || detected & (1 << category))
4218 c1 = (src[msb] << 8) | (src[lsb]); 4904 continue;
4219 src += 2;
4220 4905
4221 if (c1 == '\n' || c1 == '\r') 4906 if (this->id < 0)
4222 {
4223 if (*skip == 0)
4224 *skip = src - 2 - source;
4225 total++;
4226 if (c1 == '\n')
4227 {
4228 this_eol_type = CODING_EOL_LF;
4229 }
4230 else
4231 {
4232 if ((src + 1) >= src_end)
4233 { 4907 {
4234 this_eol_type = CODING_EOL_CR; 4908 /* No coding system of this category is defined. */
4909 mask &= ~(1 << category);
4235 } 4910 }
4236 else 4911 else
4237 { 4912 {
4238 c2 = (src[msb] << 8) | (src[lsb]); 4913 detected |= detected_mask[category];
4239 if (c2 == '\n') 4914 if ((*(this->detector)) (coding, &mask))
4240 this_eol_type = CODING_EOL_CRLF, src += 2; 4915 break;
4241 else
4242 this_eol_type = CODING_EOL_CR;
4243 } 4916 }
4244 } 4917 }
4918 if (! mask)
4919 setup_coding_system (Qraw_text, coding);
4920 else if (mask != CATEGORY_MASK_ANY)
4921 for (i = 0; i < coding_category_raw_text; i++)
4922 {
4923 enum coding_category category = coding_priorities[i];
4924 struct coding_system *this = coding_categories + category;
4245 4925
4246 if (eol_type == CODING_EOL_UNDECIDED) 4926 if (mask & (1 << category))
4247 /* This is the first end-of-line. */ 4927 {
4248 eol_type = this_eol_type; 4928 setup_coding_system (CODING_ID_NAME (this->id), coding);
4249 else if (eol_type != this_eol_type) 4929 break;
4250 { 4930 }
4251 /* The found type is different from what found before. */ 4931 }
4252 eol_type = CODING_EOL_INCONSISTENT;
4253 break;
4254 }
4255 } 4932 }
4256 } 4933 }
4257 4934
4258 if (*skip == 0) 4935 attrs = CODING_ID_ATTRS (coding->id);
4259 *skip = src_end - source; 4936 coding_type = CODING_ATTR_TYPE (attrs);
4260 return eol_type;
4261}
4262
4263/* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4264 is encoded. If it detects an appropriate format of end-of-line, it
4265 sets the information in *CODING. */
4266
4267void
4268detect_eol (coding, src, src_bytes)
4269 struct coding_system *coding;
4270 unsigned char *src;
4271 int src_bytes;
4272{
4273 Lisp_Object val;
4274 int skip;
4275 int eol_type;
4276 4937
4277 switch (coding->category_idx) 4938 /* If we have not yet decided the EOL type, detect it now. But, the
4939 detection is impossible for a CCL based coding system, in which
4940 case, we detct the EOL type after decoding. */
4941 if (VECTORP (CODING_ID_EOL_TYPE (coding->id))
4942 && ! EQ (coding_type, Qccl))
4278 { 4943 {
4279 case CODING_CATEGORY_IDX_UTF_16_BE: 4944 int eol_seen = detect_eol (coding, coding->source, coding->src_bytes);
4280 eol_type = detect_eol_type_in_2_octet_form (src, src_bytes, &skip, 1); 4945
4281 break; 4946 if (eol_seen != EOL_SEEN_NONE)
4282 case CODING_CATEGORY_IDX_UTF_16_LE: 4947 adjust_coding_eol_type (coding, eol_seen);
4283 eol_type = detect_eol_type_in_2_octet_form (src, src_bytes, &skip, 0);
4284 break;
4285 default:
4286 eol_type = detect_eol_type (src, src_bytes, &skip);
4287 break;
4288 } 4948 }
4949}
4289 4950
4290 if (coding->heading_ascii > skip)
4291 coding->heading_ascii = skip;
4292 else
4293 skip = coding->heading_ascii;
4294 4951
4295 if (eol_type == CODING_EOL_UNDECIDED) 4952static void
4296 return; 4953decode_eol (coding)
4297 if (eol_type == CODING_EOL_INCONSISTENT) 4954 struct coding_system *coding;
4955{
4956 if (VECTORP (CODING_ID_EOL_TYPE (coding->id)))
4298 { 4957 {
4299#if 0 4958 unsigned char *p = CHAR_POS_ADDR (coding->dst_pos);
4300 /* This code is suppressed until we find a better way to 4959 unsigned char *pend = p + coding->produced;
4301 distinguish raw text file and binary file. */ 4960 int eol_seen = EOL_SEEN_NONE;
4302 4961
4303 /* If we have already detected that the coding is raw-text, the 4962 for (; p < pend; p++)
4304 coding should actually be no-conversion. */
4305 if (coding->type == coding_type_raw_text)
4306 { 4963 {
4307 setup_coding_system (Qno_conversion, coding); 4964 if (*p == '\n')
4308 return; 4965 eol_seen |= EOL_SEEN_LF;
4966 else if (*p == '\r')
4967 {
4968 if (p + 1 < pend && *(p + 1) == '\n')
4969 {
4970 eol_seen |= EOL_SEEN_CRLF;
4971 p++;
4972 }
4973 else
4974 eol_seen |= EOL_SEEN_CR;
4975 }
4309 } 4976 }
4310 /* Else, let's decode only text code anyway. */ 4977 if (eol_seen != EOL_SEEN_NONE)
4311#endif /* 0 */ 4978 adjust_coding_eol_type (coding, eol_seen);
4312 eol_type = CODING_EOL_LF;
4313 } 4979 }
4314 4980
4315 val = Fget (coding->symbol, Qeol_type); 4981 if (EQ (CODING_ID_EOL_TYPE (coding->id), Qmac))
4316 if (VECTORP (val) && XVECTOR (val)->size == 3)
4317 { 4982 {
4318 int src_multibyte = coding->src_multibyte; 4983 unsigned char *p = CHAR_POS_ADDR (coding->dst_pos);
4319 int dst_multibyte = coding->dst_multibyte; 4984 unsigned char *pend = p + coding->produced;
4320 4985
4321 setup_coding_system (XVECTOR (val)->contents[eol_type], coding); 4986 for (; p < pend; p++)
4322 coding->src_multibyte = src_multibyte; 4987 if (*p == '\r')
4323 coding->dst_multibyte = dst_multibyte; 4988 *p = '\n';
4324 coding->heading_ascii = skip; 4989 }
4990 else if (EQ (CODING_ID_EOL_TYPE (coding->id), Qdos))
4991 {
4992 unsigned char *p, *pbeg, *pend;
4993 Lisp_Object undo_list;
4994
4995 move_gap_both (coding->dst_pos + coding->produced_char,
4996 coding->dst_pos_byte + coding->produced);
4997 undo_list = current_buffer->undo_list;
4998 current_buffer->undo_list = Qt;
4999 del_range_2 (coding->dst_pos, coding->dst_pos_byte, GPT, GPT_BYTE, Qnil);
5000 current_buffer->undo_list = undo_list;
5001 pbeg = GPT_ADDR;
5002 pend = pbeg + coding->produced;
5003
5004 for (p = pend - 1; p >= pbeg; p--)
5005 if (*p == '\r')
5006 {
5007 safe_bcopy ((char *) (p + 1), (char *) p, pend - p - 1);
5008 pend--;
5009 }
5010 coding->produced_char -= coding->produced - (pend - pbeg);
5011 coding->produced = pend - pbeg;
5012 insert_from_gap (coding->produced_char, coding->produced);
4325 } 5013 }
4326} 5014}
4327 5015
4328#define CONVERSION_BUFFER_EXTRA_ROOM 256 5016static void
4329 5017translate_chars (coding, table)
4330#define DECODING_BUFFER_MAG(coding) \
4331 (coding->type == coding_type_iso2022 \
4332 ? 3 \
4333 : (coding->type == coding_type_ccl \
4334 ? coding->spec.ccl.decoder.buf_magnification \
4335 : 2))
4336
4337/* Return maximum size (bytes) of a buffer enough for decoding
4338 SRC_BYTES of text encoded in CODING. */
4339
4340int
4341decoding_buffer_size (coding, src_bytes)
4342 struct coding_system *coding;
4343 int src_bytes;
4344{
4345 return (src_bytes * DECODING_BUFFER_MAG (coding)
4346 + CONVERSION_BUFFER_EXTRA_ROOM);
4347}
4348
4349/* Return maximum size (bytes) of a buffer enough for encoding
4350 SRC_BYTES of text to CODING. */
4351
4352int
4353encoding_buffer_size (coding, src_bytes)
4354 struct coding_system *coding; 5018 struct coding_system *coding;
4355 int src_bytes; 5019 Lisp_Object table;
4356{
4357 int magnification;
4358
4359 if (coding->type == coding_type_ccl)
4360 magnification = coding->spec.ccl.encoder.buf_magnification;
4361 else if (CODING_REQUIRE_ENCODING (coding))
4362 magnification = 3;
4363 else
4364 magnification = 1;
4365
4366 return (src_bytes * magnification + CONVERSION_BUFFER_EXTRA_ROOM);
4367}
4368
4369/* Working buffer for code conversion. */
4370struct conversion_buffer
4371{ 5020{
4372 int size; /* size of data. */ 5021 int *charbuf = coding->charbuf;
4373 int on_stack; /* 1 if allocated by alloca. */ 5022 int *charbuf_end = charbuf + coding->charbuf_used;
4374 unsigned char *data; 5023 int c;
4375};
4376
4377/* Don't use alloca for allocating memory space larger than this, lest
4378 we overflow their stack. */
4379#define MAX_ALLOCA 16*1024
4380 5024
4381/* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */ 5025 if (coding->chars_at_source)
4382#define allocate_conversion_buffer(buf, len) \ 5026 return;
4383 do { \
4384 if (len < MAX_ALLOCA) \
4385 { \
4386 buf.data = (unsigned char *) alloca (len); \
4387 buf.on_stack = 1; \
4388 } \
4389 else \
4390 { \
4391 buf.data = (unsigned char *) xmalloc (len); \
4392 buf.on_stack = 0; \
4393 } \
4394 buf.size = len; \
4395 } while (0)
4396 5027
4397/* Double the allocated memory for *BUF. */ 5028 while (charbuf < charbuf_end)
4398static void
4399extend_conversion_buffer (buf)
4400 struct conversion_buffer *buf;
4401{
4402 if (buf->on_stack)
4403 { 5029 {
4404 unsigned char *save = buf->data; 5030 c = *charbuf;
4405 buf->data = (unsigned char *) xmalloc (buf->size * 2); 5031 if (c < 0)
4406 bcopy (save, buf->data, buf->size); 5032 charbuf += c;
4407 buf->on_stack = 0; 5033 else
4408 } 5034 *charbuf++ = translate_char (table, c);
4409 else
4410 {
4411 buf->data = (unsigned char *) xrealloc (buf->data, buf->size * 2);
4412 } 5035 }
4413 buf->size *= 2;
4414}
4415
4416/* Free the allocated memory for BUF if it is not on stack. */
4417static void
4418free_conversion_buffer (buf)
4419 struct conversion_buffer *buf;
4420{
4421 if (!buf->on_stack)
4422 xfree (buf->data);
4423} 5036}
4424 5037
4425int 5038static int
4426ccl_coding_driver (coding, source, destination, src_bytes, dst_bytes, encodep) 5039produce_chars (coding)
4427 struct coding_system *coding; 5040 struct coding_system *coding;
4428 unsigned char *source, *destination;
4429 int src_bytes, dst_bytes, encodep;
4430{ 5041{
4431 struct ccl_program *ccl 5042 unsigned char *dst = coding->destination + coding->produced;
4432 = encodep ? &coding->spec.ccl.encoder : &coding->spec.ccl.decoder; 5043 unsigned char *dst_end = coding->destination + coding->dst_bytes;
4433 unsigned char *dst = destination; 5044 int produced;
5045 int produced_chars = 0;
4434 5046
4435 ccl->suppress_error = coding->suppress_error; 5047 if (! coding->chars_at_source)
4436 ccl->last_block = coding->mode & CODING_MODE_LAST_BLOCK;
4437 if (encodep)
4438 {
4439 /* On encoding, EOL format is converted within ccl_driver. For
4440 that, setup proper information in the structure CCL. */
4441 ccl->eol_type = coding->eol_type;
4442 if (ccl->eol_type ==CODING_EOL_UNDECIDED)
4443 ccl->eol_type = CODING_EOL_LF;
4444 ccl->cr_consumed = coding->spec.ccl.cr_carryover;
4445 }
4446 ccl->multibyte = coding->src_multibyte;
4447 if (coding->spec.ccl.eight_bit_carryover[0] != 0)
4448 { 5048 {
4449 /* Move carryover bytes to DESTINATION. */ 5049 /* Characters are in coding->charbuf. */
4450 unsigned char *p = coding->spec.ccl.eight_bit_carryover; 5050 int *buf = coding->charbuf;
4451 while (*p) 5051 int *buf_end = buf + coding->charbuf_used;
4452 *dst++ = *p++; 5052 unsigned char *adjusted_dst_end;
4453 coding->spec.ccl.eight_bit_carryover[0] = 0;
4454 if (dst_bytes)
4455 dst_bytes -= dst - destination;
4456 }
4457 5053
4458 coding->produced = (ccl_driver (ccl, source, dst, src_bytes, dst_bytes, 5054 if (BUFFERP (coding->src_object)
4459 &(coding->consumed)) 5055 && EQ (coding->src_object, coding->dst_object))
4460 + dst - destination); 5056 dst_end = coding->source + coding->consumed;
5057 adjusted_dst_end = dst_end - MAX_MULTIBYTE_LENGTH;
4461 5058
4462 if (encodep) 5059 while (buf < buf_end)
4463 {
4464 coding->produced_char = coding->produced;
4465 coding->spec.ccl.cr_carryover = ccl->cr_consumed;
4466 }
4467 else if (!ccl->eight_bit_control)
4468 {
4469 /* The produced bytes forms a valid multibyte sequence. */
4470 coding->produced_char
4471 = multibyte_chars_in_text (destination, coding->produced);
4472 coding->spec.ccl.eight_bit_carryover[0] = 0;
4473 }
4474 else
4475 {
4476 /* On decoding, the destination should always multibyte. But,
4477 CCL program might have been generated an invalid multibyte
4478 sequence. Here we make such a sequence valid as
4479 multibyte. */
4480 int bytes
4481 = dst_bytes ? dst_bytes : source + coding->consumed - destination;
4482
4483 if ((coding->consumed < src_bytes
4484 || !ccl->last_block)
4485 && coding->produced >= 1
4486 && destination[coding->produced - 1] >= 0x80)
4487 { 5060 {
4488 /* We should not convert the tailing 8-bit codes to 5061 int c = *buf++;
4489 multibyte form even if they doesn't form a valid 5062
4490 multibyte sequence. They may form a valid sequence in 5063 if (dst >= adjusted_dst_end)
4491 the next call. */
4492 int carryover = 0;
4493
4494 if (destination[coding->produced - 1] < 0xA0)
4495 carryover = 1;
4496 else if (coding->produced >= 2)
4497 { 5064 {
4498 if (destination[coding->produced - 2] >= 0x80) 5065 dst = alloc_destination (coding,
4499 { 5066 buf_end - buf + MAX_MULTIBYTE_LENGTH,
4500 if (destination[coding->produced - 2] < 0xA0) 5067 dst);
4501 carryover = 2; 5068 dst_end = coding->destination + coding->dst_bytes;
4502 else if (coding->produced >= 3 5069 adjusted_dst_end = dst_end - MAX_MULTIBYTE_LENGTH;
4503 && destination[coding->produced - 3] >= 0x80
4504 && destination[coding->produced - 3] < 0xA0)
4505 carryover = 3;
4506 }
4507 } 5070 }
4508 if (carryover > 0) 5071 if (c >= 0)
4509 { 5072 {
4510 BCOPY_SHORT (destination + coding->produced - carryover, 5073 if (coding->dst_multibyte
4511 coding->spec.ccl.eight_bit_carryover, 5074 || ! CHAR_BYTE8_P (c))
4512 carryover); 5075 CHAR_STRING_ADVANCE (c, dst);
4513 coding->spec.ccl.eight_bit_carryover[carryover] = 0; 5076 else
4514 coding->produced -= carryover; 5077 *dst++ = CHAR_TO_BYTE8 (c);
5078 produced_chars++;
4515 } 5079 }
5080 else
5081 /* This is an annotation data. */
5082 buf -= c + 1;
4516 } 5083 }
4517 coding->produced = str_as_multibyte (destination, bytes,
4518 coding->produced,
4519 &(coding->produced_char));
4520 } 5084 }
4521 5085 else
4522 switch (ccl->status)
4523 { 5086 {
4524 case CCL_STAT_SUSPEND_BY_SRC: 5087 int multibytep = coding->src_multibyte;
4525 coding->result = CODING_FINISH_INSUFFICIENT_SRC; 5088 unsigned char *src = coding->source;
4526 break; 5089 unsigned char *src_end = src + coding->src_bytes;
4527 case CCL_STAT_SUSPEND_BY_DST: 5090 Lisp_Object eol_type;
4528 coding->result = CODING_FINISH_INSUFFICIENT_DST;
4529 break;
4530 case CCL_STAT_QUIT:
4531 case CCL_STAT_INVALID_CMD:
4532 coding->result = CODING_FINISH_INTERRUPT;
4533 break;
4534 default:
4535 coding->result = CODING_FINISH_NORMAL;
4536 break;
4537 }
4538 return coding->result;
4539}
4540
4541/* Decode EOL format of the text at PTR of BYTES length destructively
4542 according to CODING->eol_type. This is called after the CCL
4543 program produced a decoded text at PTR. If we do CRLF->LF
4544 conversion, update CODING->produced and CODING->produced_char. */
4545 5091
4546static void 5092 eol_type = CODING_ID_EOL_TYPE (coding->id);
4547decode_eol_post_ccl (coding, ptr, bytes)
4548 struct coding_system *coding;
4549 unsigned char *ptr;
4550 int bytes;
4551{
4552 Lisp_Object val, saved_coding_symbol;
4553 unsigned char *pend = ptr + bytes;
4554 int dummy;
4555
4556 /* Remember the current coding system symbol. We set it back when
4557 an inconsistent EOL is found so that `last-coding-system-used' is
4558 set to the coding system that doesn't specify EOL conversion. */
4559 saved_coding_symbol = coding->symbol;
4560
4561 coding->spec.ccl.cr_carryover = 0;
4562 if (coding->eol_type == CODING_EOL_UNDECIDED)
4563 {
4564 /* Here, to avoid the call of setup_coding_system, we directly
4565 call detect_eol_type. */
4566 coding->eol_type = detect_eol_type (ptr, bytes, &dummy);
4567 if (coding->eol_type == CODING_EOL_INCONSISTENT)
4568 coding->eol_type = CODING_EOL_LF;
4569 if (coding->eol_type != CODING_EOL_UNDECIDED)
4570 {
4571 val = Fget (coding->symbol, Qeol_type);
4572 if (VECTORP (val) && XVECTOR (val)->size == 3)
4573 coding->symbol = XVECTOR (val)->contents[coding->eol_type];
4574 }
4575 coding->mode |= CODING_MODE_INHIBIT_INCONSISTENT_EOL;
4576 }
4577
4578 if (coding->eol_type == CODING_EOL_LF
4579 || coding->eol_type == CODING_EOL_UNDECIDED)
4580 {
4581 /* We have nothing to do. */
4582 ptr = pend;
4583 }
4584 else if (coding->eol_type == CODING_EOL_CRLF)
4585 {
4586 unsigned char *pstart = ptr, *p = ptr;
4587 5093
4588 if (! (coding->mode & CODING_MODE_LAST_BLOCK) 5094 if (coding->src_multibyte != coding->dst_multibyte)
4589 && *(pend - 1) == '\r')
4590 { 5095 {
4591 /* If the last character is CR, we can't handle it here 5096 if (coding->src_multibyte)
4592 because LF will be in the not-yet-decoded source text.
4593 Recorded that the CR is not yet processed. */
4594 coding->spec.ccl.cr_carryover = 1;
4595 coding->produced--;
4596 coding->produced_char--;
4597 pend--;
4598 }
4599 while (ptr < pend)
4600 {
4601 if (*ptr == '\r')
4602 { 5097 {
4603 if (ptr + 1 < pend && *(ptr + 1) == '\n') 5098 int consumed_chars;
4604 { 5099
4605 *p++ = '\n'; 5100 while (1)
4606 ptr += 2;
4607 }
4608 else
4609 { 5101 {
4610 if (coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL) 5102 unsigned char *src_base = src;
4611 goto undo_eol_conversion; 5103 int c;
4612 *p++ = *ptr++; 5104
5105 ONE_MORE_BYTE (c);
5106 if (c == '\r')
5107 {
5108 if (EQ (eol_type, Qdos))
5109 {
5110 if (src < src_end
5111 && *src == '\n')
5112 c = *src++;
5113 }
5114 else if (EQ (eol_type, Qmac))
5115 c = '\n';
5116 }
5117 if (dst == dst_end)
5118 {
5119 EMACS_INT offset = src - coding->source;
5120
5121 dst = alloc_destination (coding, src_end - src + 1, dst);
5122 dst_end = coding->destination + coding->dst_bytes;
5123 coding_set_source (coding);
5124 src = coding->source + offset;
5125 src_end = coding->source + coding->src_bytes;
5126 }
5127 *dst++ = c;
5128 produced_chars++;
4613 } 5129 }
5130 no_more_source:
5131 ;
4614 } 5132 }
4615 else if (*ptr == '\n'
4616 && coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL)
4617 goto undo_eol_conversion;
4618 else 5133 else
4619 *p++ = *ptr++; 5134 while (src < src_end)
4620 continue; 5135 {
5136 int c = *src++;
4621 5137
4622 undo_eol_conversion: 5138 if (c == '\r')
4623 /* We have faced with inconsistent EOL format at PTR. 5139 {
4624 Convert all LFs before PTR back to CRLFs. */ 5140 if (EQ (eol_type, Qdos))
4625 for (p--, ptr--; p >= pstart; p--) 5141 {
4626 { 5142 if (src < src_end
4627 if (*p == '\n') 5143 && *src == '\n')
4628 *ptr-- = '\n', *ptr-- = '\r'; 5144 c = *src++;
4629 else 5145 }
4630 *ptr-- = *p; 5146 else if (EQ (eol_type, Qmac))
4631 } 5147 c = '\n';
4632 /* If carryover is recorded, cancel it because we don't 5148 }
4633 convert CRLF anymore. */ 5149 if (dst >= dst_end - 1)
4634 if (coding->spec.ccl.cr_carryover) 5150 {
4635 { 5151 EMACS_INT offset = src - coding->source;
4636 coding->spec.ccl.cr_carryover = 0; 5152
4637 coding->produced++; 5153 dst = alloc_destination (coding, src_end - src + 2, dst);
4638 coding->produced_char++; 5154 dst_end = coding->destination + coding->dst_bytes;
4639 pend++; 5155 coding_set_source (coding);
4640 } 5156 src = coding->source + offset;
4641 p = ptr = pend; 5157 src_end = coding->source + coding->src_bytes;
4642 coding->eol_type = CODING_EOL_LF; 5158 }
4643 coding->symbol = saved_coding_symbol; 5159 EMIT_ONE_BYTE (c);
5160 }
4644 } 5161 }
4645 if (p < pend) 5162 else
4646 { 5163 {
4647 /* As each two-byte sequence CRLF was converted to LF, (PEND 5164 if (!EQ (coding->src_object, coding->dst_object))
4648 - P) is the number of deleted characters. */ 5165 {
4649 coding->produced -= pend - p; 5166 int require = coding->src_bytes - coding->dst_bytes;
4650 coding->produced_char -= pend - p;
4651 }
4652 }
4653 else /* i.e. coding->eol_type == CODING_EOL_CR */
4654 {
4655 unsigned char *p = ptr;
4656 5167
4657 for (; ptr < pend; ptr++) 5168 if (require > 0)
4658 { 5169 {
4659 if (*ptr == '\r') 5170 EMACS_INT offset = src - coding->source;
4660 *ptr = '\n'; 5171
4661 else if (*ptr == '\n' 5172 dst = alloc_destination (coding, require, dst);
4662 && coding->mode & CODING_MODE_INHIBIT_INCONSISTENT_EOL) 5173 coding_set_source (coding);
5174 src = coding->source + offset;
5175 src_end = coding->source + coding->src_bytes;
5176 }
5177 }
5178 produced_chars = coding->src_chars;
5179 while (src < src_end)
4663 { 5180 {
4664 for (; p < ptr; p++) 5181 int c = *src++;
5182
5183 if (c == '\r')
4665 { 5184 {
4666 if (*p == '\n') 5185 if (EQ (eol_type, Qdos))
4667 *p = '\r'; 5186 {
5187 if (src < src_end
5188 && *src == '\n')
5189 c = *src++;
5190 produced_chars--;
5191 }
5192 else if (EQ (eol_type, Qmac))
5193 c = '\n';
4668 } 5194 }
4669 ptr = pend; 5195 *dst++ = c;
4670 coding->eol_type = CODING_EOL_LF;
4671 coding->symbol = saved_coding_symbol;
4672 } 5196 }
4673 } 5197 }
4674 } 5198 }
5199
5200 produced = dst - (coding->destination + coding->produced);
5201 if (BUFFERP (coding->dst_object))
5202 insert_from_gap (produced_chars, produced);
5203 coding->produced += produced;
5204 coding->produced_char += produced_chars;
5205 return produced_chars;
4675} 5206}
4676 5207
4677/* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before 5208/* [ -LENGTH CHAR_POS_OFFSET MASK METHOD COMP_LEN ]
4678 decoding, it may detect coding system and format of end-of-line if 5209 or
4679 those are not yet decided. The source should be unibyte, the 5210 [ -LENGTH CHAR_POS_OFFSET MASK METHOD COMP_LEN COMPONENTS... ]
4680 result is multibyte if CODING->dst_multibyte is nonzero, else 5211 */
4681 unibyte. */
4682 5212
4683int 5213static INLINE void
4684decode_coding (coding, source, destination, src_bytes, dst_bytes) 5214produce_composition (coding, charbuf)
4685 struct coding_system *coding; 5215 struct coding_system *coding;
4686 unsigned char *source, *destination; 5216 int *charbuf;
4687 int src_bytes, dst_bytes;
4688{ 5217{
4689 if (coding->type == coding_type_undecided) 5218 Lisp_Object buffer;
4690 detect_coding (coding, source, src_bytes); 5219 int len;
4691 5220 EMACS_INT pos;
4692 if (coding->eol_type == CODING_EOL_UNDECIDED 5221 enum composition_method method;
4693 && coding->type != coding_type_ccl) 5222 int cmp_len;
5223 Lisp_Object components;
5224
5225 buffer = coding->dst_object;
5226 len = -charbuf[0];
5227 pos = coding->dst_pos + charbuf[1];
5228 method = (enum composition_method) (charbuf[3]);
5229 cmp_len = charbuf[4];
5230
5231 if (method == COMPOSITION_RELATIVE)
5232 components = Qnil;
5233 else
4694 { 5234 {
4695 detect_eol (coding, source, src_bytes); 5235 Lisp_Object args[MAX_COMPOSITION_COMPONENTS * 2 - 1];
4696 /* We had better recover the original eol format if we 5236 int i;
4697 encounter an inconsistent eol format while decoding. */
4698 coding->mode |= CODING_MODE_INHIBIT_INCONSISTENT_EOL;
4699 }
4700 5237
4701 coding->produced = coding->produced_char = 0; 5238 len -= 5;
4702 coding->consumed = coding->consumed_char = 0; 5239 charbuf += 5;
4703 coding->errors = 0; 5240 for (i = 0; i < len; i++)
4704 coding->result = CODING_FINISH_NORMAL; 5241 args[i] = make_number (charbuf[i]);
5242 components = (method == COMPOSITION_WITH_ALTCHARS
5243 ? Fstring (len, args) : Fvector (len, args));
5244 }
5245 compose_text (pos, pos + cmp_len, components, Qnil, Qnil);
5246}
4705 5247
4706 switch (coding->type) 5248static int *
4707 { 5249save_composition_data (buf, buf_end, prop)
4708 case coding_type_sjis: 5250 int *buf, *buf_end;
4709 decode_coding_sjis_big5 (coding, source, destination, 5251 Lisp_Object prop;
4710 src_bytes, dst_bytes, 1); 5252{
4711 break; 5253 enum composition_method method = COMPOSITION_METHOD (prop);
5254 int cmp_len = COMPOSITION_LENGTH (prop);
4712 5255
4713 case coding_type_iso2022: 5256 if (buf + 4 + (MAX_COMPOSITION_COMPONENTS * 2 - 1) > buf_end)
4714 decode_coding_iso2022 (coding, source, destination, 5257 return NULL;
4715 src_bytes, dst_bytes);
4716 break;
4717 5258
4718 case coding_type_big5: 5259 buf[1] = CODING_ANNOTATE_COMPOSITION_MASK;
4719 decode_coding_sjis_big5 (coding, source, destination, 5260 buf[2] = method;
4720 src_bytes, dst_bytes, 0); 5261 buf[3] = cmp_len;
4721 break;
4722 5262
4723 case coding_type_emacs_mule: 5263 if (method == COMPOSITION_RELATIVE)
4724 decode_coding_emacs_mule (coding, source, destination, 5264 buf[0] = 4;
4725 src_bytes, dst_bytes); 5265 else
4726 break; 5266 {
5267 Lisp_Object components;
5268 int len, i;
4727 5269
4728 case coding_type_ccl: 5270 components = COMPOSITION_COMPONENTS (prop);
4729 if (coding->spec.ccl.cr_carryover) 5271 if (VECTORP (components))
4730 { 5272 {
4731 /* Set the CR which is not processed by the previous call of 5273 len = XVECTOR (components)->size;
4732 decode_eol_post_ccl in DESTINATION. */ 5274 for (i = 0; i < len; i++)
4733 *destination = '\r'; 5275 buf[4 + i] = XINT (AREF (components, i));
4734 coding->produced++;
4735 coding->produced_char++;
4736 dst_bytes--;
4737 } 5276 }
4738 ccl_coding_driver (coding, source, 5277 else if (STRINGP (components))
4739 destination + coding->spec.ccl.cr_carryover, 5278 {
4740 src_bytes, dst_bytes, 0); 5279 int i_byte;
4741 if (coding->eol_type != CODING_EOL_LF)
4742 decode_eol_post_ccl (coding, destination, coding->produced);
4743 break;
4744
4745 default:
4746 decode_eol (coding, source, destination, src_bytes, dst_bytes);
4747 }
4748
4749 if (coding->result == CODING_FINISH_INSUFFICIENT_SRC
4750 && coding->mode & CODING_MODE_LAST_BLOCK
4751 && coding->consumed == src_bytes)
4752 coding->result = CODING_FINISH_NORMAL;
4753
4754 if (coding->mode & CODING_MODE_LAST_BLOCK
4755 && coding->result == CODING_FINISH_INSUFFICIENT_SRC)
4756 {
4757 unsigned char *src = source + coding->consumed;
4758 unsigned char *dst = destination + coding->produced;
4759 5280
4760 src_bytes -= coding->consumed; 5281 len = XSTRING (components)->size;
4761 coding->errors++; 5282 i = i_byte = 0;
4762 if (COMPOSING_P (coding)) 5283 while (i < len)
4763 DECODE_COMPOSITION_END ('1'); 5284 FETCH_STRING_CHAR_ADVANCE (buf[4 + i], components, i, i_byte);
4764 while (src_bytes--) 5285 }
5286 else if (INTEGERP (components))
4765 { 5287 {
4766 int c = *src++; 5288 len = 1;
4767 dst += CHAR_STRING (c, dst); 5289 buf[4] = XINT (components);
4768 coding->produced_char++;
4769 } 5290 }
4770 coding->consumed = coding->consumed_char = src - source; 5291 else if (CONSP (components))
4771 coding->produced = dst - destination; 5292 {
4772 coding->result = CODING_FINISH_NORMAL; 5293 for (len = 0; CONSP (components);
5294 len++, components = XCDR (components))
5295 buf[4 + len] = XINT (XCAR (components));
5296 }
5297 else
5298 abort ();
5299 buf[0] = 4 + len;
4773 } 5300 }
5301 return (buf + buf[0]);
5302}
4774 5303
4775 if (!coding->dst_multibyte) 5304#define CHARBUF_SIZE 0x4000
4776 {
4777 coding->produced = str_as_unibyte (destination, coding->produced);
4778 coding->produced_char = coding->produced;
4779 }
4780 5305
4781 return coding->result; 5306#define ALLOC_CONVERSION_WORK_AREA(coding) \
4782} 5307 do { \
5308 int size = CHARBUF_SIZE;; \
5309 \
5310 coding->charbuf = NULL; \
5311 while (size > 1024) \
5312 { \
5313 coding->charbuf = (int *) alloca (sizeof (int) * size); \
5314 if (coding->charbuf) \
5315 break; \
5316 size >>= 1; \
5317 } \
5318 if (! coding->charbuf) \
5319 { \
5320 coding->result = CODING_RESULT_INSUFFICIENT_MEM; \
5321 return coding->result; \
5322 } \
5323 coding->charbuf_size = size; \
5324 } while (0)
4783 5325
4784/* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4785 multibyteness of the source is CODING->src_multibyte, the
4786 multibyteness of the result is always unibyte. */
4787 5326
4788int 5327static void
4789encode_coding (coding, source, destination, src_bytes, dst_bytes) 5328produce_annotation (coding)
4790 struct coding_system *coding; 5329 struct coding_system *coding;
4791 unsigned char *source, *destination;
4792 int src_bytes, dst_bytes;
4793{ 5330{
4794 coding->produced = coding->produced_char = 0; 5331 int *charbuf = coding->charbuf;
4795 coding->consumed = coding->consumed_char = 0; 5332 int *charbuf_end = charbuf + coding->charbuf_used;
4796 coding->errors = 0;
4797 coding->result = CODING_FINISH_NORMAL;
4798
4799 switch (coding->type)
4800 {
4801 case coding_type_sjis:
4802 encode_coding_sjis_big5 (coding, source, destination,
4803 src_bytes, dst_bytes, 1);
4804 break;
4805
4806 case coding_type_iso2022:
4807 encode_coding_iso2022 (coding, source, destination,
4808 src_bytes, dst_bytes);
4809 break;
4810 5333
4811 case coding_type_big5: 5334 while (charbuf < charbuf_end)
4812 encode_coding_sjis_big5 (coding, source, destination,
4813 src_bytes, dst_bytes, 0);
4814 break;
4815
4816 case coding_type_emacs_mule:
4817 encode_coding_emacs_mule (coding, source, destination,
4818 src_bytes, dst_bytes);
4819 break;
4820
4821 case coding_type_ccl:
4822 ccl_coding_driver (coding, source, destination,
4823 src_bytes, dst_bytes, 1);
4824 break;
4825
4826 default:
4827 encode_eol (coding, source, destination, src_bytes, dst_bytes);
4828 }
4829
4830 if (coding->mode & CODING_MODE_LAST_BLOCK
4831 && coding->result == CODING_FINISH_INSUFFICIENT_SRC)
4832 { 5335 {
4833 unsigned char *src = source + coding->consumed; 5336 if (*charbuf >= 0)
4834 unsigned char *dst = destination + coding->produced; 5337 charbuf++;
4835 5338 else
4836 if (coding->type == coding_type_iso2022)
4837 ENCODE_RESET_PLANE_AND_REGISTER;
4838 if (COMPOSING_P (coding))
4839 *dst++ = ISO_CODE_ESC, *dst++ = '1';
4840 if (coding->consumed < src_bytes)
4841 { 5339 {
4842 int len = src_bytes - coding->consumed; 5340 int len = -*charbuf;
4843 5341 switch (charbuf[2])
4844 BCOPY_SHORT (src, dst, len); 5342 {
4845 if (coding->src_multibyte) 5343 case CODING_ANNOTATE_COMPOSITION_MASK:
4846 len = str_as_unibyte (dst, len); 5344 produce_composition (coding, charbuf);
4847 dst += len; 5345 break;
4848 coding->consumed = src_bytes; 5346 default:
5347 abort ();
5348 }
5349 charbuf += len;
4849 } 5350 }
4850 coding->produced = coding->produced_char = dst - destination;
4851 coding->result = CODING_FINISH_NORMAL;
4852 } 5351 }
5352}
4853 5353
4854 if (coding->result == CODING_FINISH_INSUFFICIENT_SRC 5354/* Decode the data at CODING->src_object into CODING->dst_object.
4855 && coding->consumed == src_bytes) 5355 CODING->src_object is a buffer, a string, or nil.
4856 coding->result = CODING_FINISH_NORMAL; 5356 CODING->dst_object is a buffer.
4857 5357
4858 return coding->result; 5358 If CODING->src_object is a buffer, it must be the current buffer.
4859} 5359 In this case, if CODING->src_pos is positive, it is a position of
5360 the source text in the buffer, otherwise, the source text is in the
5361 gap area of the buffer, and CODING->src_pos specifies the offset of
5362 the text from GPT (which must be the same as PT). If this is the
5363 same buffer as CODING->dst_object, CODING->src_pos must be
5364 negative.
4860 5365
4861/* Scan text in the region between *BEG and *END (byte positions), 5366 If CODING->src_object is a string, CODING->src_pos in an index to
4862 skip characters which we don't have to decode by coding system 5367 that string.
4863 CODING at the head and tail, then set *BEG and *END to the region
4864 of the text we actually have to convert. The caller should move
4865 the gap out of the region in advance if the region is from a
4866 buffer.
4867 5368
4868 If STR is not NULL, *BEG and *END are indices into STR. */ 5369 If CODING->src_object is nil, CODING->source must already point to
5370 the non-relocatable memory area. In this case, CODING->src_pos is
5371 an offset from CODING->source.
4869 5372
4870static void 5373 The decoded data is inserted at the current point of the buffer
4871shrink_decoding_region (beg, end, coding, str) 5374 CODING->dst_object.
4872 int *beg, *end; 5375*/
5376
5377static int
5378decode_coding (coding)
4873 struct coding_system *coding; 5379 struct coding_system *coding;
4874 unsigned char *str;
4875{ 5380{
4876 unsigned char *begp_orig, *begp, *endp_orig, *endp, c; 5381 Lisp_Object attrs;
4877 int eol_conversion;
4878 Lisp_Object translation_table;
4879 5382
4880 if (coding->type == coding_type_ccl 5383 if (BUFFERP (coding->src_object)
4881 || coding->type == coding_type_undecided 5384 && coding->src_pos > 0
4882 || coding->eol_type != CODING_EOL_LF 5385 && coding->src_pos < GPT
4883 || !NILP (coding->post_read_conversion) 5386 && coding->src_pos + coding->src_chars > GPT)
4884 || coding->composing != COMPOSITION_DISABLED) 5387 move_gap_both (coding->src_pos, coding->src_pos_byte);
4885 {
4886 /* We can't skip any data. */
4887 return;
4888 }
4889 if (coding->type == coding_type_no_conversion
4890 || coding->type == coding_type_raw_text
4891 || coding->type == coding_type_emacs_mule)
4892 {
4893 /* We need no conversion, but don't have to skip any data here.
4894 Decoding routine handles them effectively anyway. */
4895 return;
4896 }
4897 5388
4898 translation_table = coding->translation_table_for_decode; 5389 if (BUFFERP (coding->dst_object))
4899 if (NILP (translation_table) && !NILP (Venable_character_translation))
4900 translation_table = Vstandard_translation_table_for_decode;
4901 if (CHAR_TABLE_P (translation_table))
4902 { 5390 {
4903 int i; 5391 if (current_buffer != XBUFFER (coding->dst_object))
4904 for (i = 0; i < 128; i++) 5392 set_buffer_internal (XBUFFER (coding->dst_object));
4905 if (!NILP (CHAR_TABLE_REF (translation_table, i))) 5393 if (GPT != PT)
4906 break; 5394 move_gap_both (PT, PT_BYTE);
4907 if (i < 128)
4908 /* Some ASCII character should be translated. We give up
4909 shrinking. */
4910 return;
4911 } 5395 }
4912 5396
4913 if (coding->heading_ascii >= 0) 5397 coding->consumed = coding->consumed_char = 0;
4914 /* Detection routine has already found how much we can skip at the 5398 coding->produced = coding->produced_char = 0;
4915 head. */ 5399 coding->chars_at_source = 0;
4916 *beg += coding->heading_ascii; 5400 coding->result = CODING_RESULT_SUCCESS;
5401 coding->errors = 0;
4917 5402
4918 if (str) 5403 ALLOC_CONVERSION_WORK_AREA (coding);
4919 { 5404
4920 begp_orig = begp = str + *beg; 5405 attrs = CODING_ID_ATTRS (coding->id);
4921 endp_orig = endp = str + *end; 5406
4922 } 5407 do
4923 else
4924 { 5408 {
4925 begp_orig = begp = BYTE_POS_ADDR (*beg); 5409 coding_set_source (coding);
4926 endp_orig = endp = begp + *end - *beg; 5410 coding->annotated = 0;
5411 (*(coding->decoder)) (coding);
5412 if (!NILP (CODING_ATTR_DECODE_TBL (attrs)))
5413 translate_chars (CODING_ATTR_DECODE_TBL (attrs), coding);
5414 coding_set_destination (coding);
5415 produce_chars (coding);
5416 if (coding->annotated)
5417 produce_annotation (coding);
4927 } 5418 }
5419 while (coding->consumed < coding->src_bytes
5420 && ! coding->result);
4928 5421
4929 eol_conversion = (coding->eol_type == CODING_EOL_CR 5422 if (EQ (CODING_ATTR_TYPE (CODING_ID_ATTRS (coding->id)), Qccl)
4930 || coding->eol_type == CODING_EOL_CRLF); 5423 && SYMBOLP (CODING_ID_EOL_TYPE (coding->id))
5424 && ! EQ (CODING_ID_EOL_TYPE (coding->id), Qunix))
5425 decode_eol (coding);
4931 5426
4932 switch (coding->type) 5427 coding->carryover_bytes = 0;
5428 if (coding->consumed < coding->src_bytes)
4933 { 5429 {
4934 case coding_type_sjis: 5430 int nbytes = coding->src_bytes - coding->consumed;
4935 case coding_type_big5: 5431 unsigned char *src;
4936 /* We can skip all ASCII characters at the head. */
4937 if (coding->heading_ascii < 0)
4938 {
4939 if (eol_conversion)
4940 while (begp < endp && *begp < 0x80 && *begp != '\r') begp++;
4941 else
4942 while (begp < endp && *begp < 0x80) begp++;
4943 }
4944 /* We can skip all ASCII characters at the tail except for the
4945 second byte of SJIS or BIG5 code. */
4946 if (eol_conversion)
4947 while (begp < endp && endp[-1] < 0x80 && endp[-1] != '\r') endp--;
4948 else
4949 while (begp < endp && endp[-1] < 0x80) endp--;
4950 /* Do not consider LF as ascii if preceded by CR, since that
4951 confuses eol decoding. */
4952 if (begp < endp && endp < endp_orig && endp[-1] == '\r' && endp[0] == '\n')
4953 endp++;
4954 if (begp < endp && endp < endp_orig && endp[-1] >= 0x80)
4955 endp++;
4956 break;
4957 5432
4958 case coding_type_iso2022: 5433 coding_set_source (coding);
4959 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, 0) != CHARSET_ASCII) 5434 coding_set_destination (coding);
4960 /* We can't skip any data. */ 5435 src = coding->source + coding->consumed;
4961 break; 5436
4962 if (coding->heading_ascii < 0) 5437 if (coding->mode & CODING_MODE_LAST_BLOCK)
4963 { 5438 {
4964 /* We can skip all ASCII characters at the head except for a 5439 /* Flush out unprocessed data as binary chars. We are sure
4965 few control codes. */ 5440 that the number of data is less than the size of
4966 while (begp < endp && (c = *begp) < 0x80 5441 coding->charbuf. */
4967 && c != ISO_CODE_CR && c != ISO_CODE_SO 5442 int *charbuf = coding->charbuf;
4968 && c != ISO_CODE_SI && c != ISO_CODE_ESC 5443
4969 && (!eol_conversion || c != ISO_CODE_LF)) 5444 while (nbytes-- > 0)
4970 begp++; 5445 {
5446 int c = *src++;
5447 *charbuf++ = (c & 0x80 ? - c : c);
5448 }
5449 produce_chars (coding);
4971 } 5450 }
4972 switch (coding->category_idx) 5451 else
4973 { 5452 {
4974 case CODING_CATEGORY_IDX_ISO_8_1: 5453 /* Record unprocessed bytes in coding->carryover. We are
4975 case CODING_CATEGORY_IDX_ISO_8_2: 5454 sure that the number of data is less than the size of
4976 /* We can skip all ASCII characters at the tail. */ 5455 coding->carryover. */
4977 if (eol_conversion) 5456 unsigned char *p = coding->carryover;
4978 while (begp < endp && (c = endp[-1]) < 0x80 && c != '\r') endp--; 5457
4979 else 5458 coding->carryover_bytes = nbytes;
4980 while (begp < endp && endp[-1] < 0x80) endp--; 5459 while (nbytes-- > 0)
4981 /* Do not consider LF as ascii if preceded by CR, since that 5460 *p++ = *src++;
4982 confuses eol decoding. */
4983 if (begp < endp && endp < endp_orig && endp[-1] == '\r' && endp[0] == '\n')
4984 endp++;
4985 break;
4986
4987 case CODING_CATEGORY_IDX_ISO_7:
4988 case CODING_CATEGORY_IDX_ISO_7_TIGHT:
4989 {
4990 /* We can skip all characters at the tail except for 8-bit
4991 codes and ESC and the following 2-byte at the tail. */
4992 unsigned char *eight_bit = NULL;
4993
4994 if (eol_conversion)
4995 while (begp < endp
4996 && (c = endp[-1]) != ISO_CODE_ESC && c != '\r')
4997 {
4998 if (!eight_bit && c & 0x80) eight_bit = endp;
4999 endp--;
5000 }
5001 else
5002 while (begp < endp
5003 && (c = endp[-1]) != ISO_CODE_ESC)
5004 {
5005 if (!eight_bit && c & 0x80) eight_bit = endp;
5006 endp--;
5007 }
5008 /* Do not consider LF as ascii if preceded by CR, since that
5009 confuses eol decoding. */
5010 if (begp < endp && endp < endp_orig
5011 && endp[-1] == '\r' && endp[0] == '\n')
5012 endp++;
5013 if (begp < endp && endp[-1] == ISO_CODE_ESC)
5014 {
5015 if (endp + 1 < endp_orig && end[0] == '(' && end[1] == 'B')
5016 /* This is an ASCII designation sequence. We can
5017 surely skip the tail. But, if we have
5018 encountered an 8-bit code, skip only the codes
5019 after that. */
5020 endp = eight_bit ? eight_bit : endp + 2;
5021 else
5022 /* Hmmm, we can't skip the tail. */
5023 endp = endp_orig;
5024 }
5025 else if (eight_bit)
5026 endp = eight_bit;
5027 }
5028 } 5461 }
5029 break; 5462 coding->consumed = coding->src_bytes;
5463 }
5030 5464
5031 default: 5465 if (BUFFERP (coding->dst_object))
5032 abort (); 5466 {
5467 record_insert (coding->dst_pos, coding->produced_char);
5033 } 5468 }
5034 *beg += begp - begp_orig;
5035 *end += endp - endp_orig;
5036 return;
5037}
5038 5469
5039/* Like shrink_decoding_region but for encoding. */ 5470 return coding->result;
5471}
5040 5472
5041static void 5473static void
5042shrink_encoding_region (beg, end, coding, str) 5474consume_chars (coding)
5043 int *beg, *end;
5044 struct coding_system *coding; 5475 struct coding_system *coding;
5045 unsigned char *str;
5046{ 5476{
5047 unsigned char *begp_orig, *begp, *endp_orig, *endp; 5477 int *buf = coding->charbuf;
5048 int eol_conversion; 5478 /* -1 is to compensate for CRLF. */
5049 Lisp_Object translation_table; 5479 int *buf_end = coding->charbuf + coding->charbuf_size - 1;
5480 unsigned char *src = coding->source + coding->consumed;
5481 int pos = coding->src_pos + coding->consumed_char;
5482 int end_pos = coding->src_pos + coding->src_chars;
5483 int multibytep = coding->src_multibyte;
5484 Lisp_Object eol_type;
5485 int c;
5486 int start, end, stop;
5487 Lisp_Object object, prop;
5050 5488
5051 if (coding->type == coding_type_ccl 5489 eol_type = CODING_ID_EOL_TYPE (coding->id);
5052 || coding->eol_type == CODING_EOL_CRLF 5490 if (VECTORP (eol_type))
5053 || coding->eol_type == CODING_EOL_CR 5491 eol_type = Qunix;
5054 || (coding->cmp_data && coding->cmp_data->used > 0))
5055 {
5056 /* We can't skip any data. */
5057 return;
5058 }
5059 if (coding->type == coding_type_no_conversion
5060 || coding->type == coding_type_raw_text
5061 || coding->type == coding_type_emacs_mule
5062 || coding->type == coding_type_undecided)
5063 {
5064 /* We need no conversion, but don't have to skip any data here.
5065 Encoding routine handles them effectively anyway. */
5066 return;
5067 }
5068 5492
5069 translation_table = coding->translation_table_for_encode; 5493 object = coding->src_object;
5070 if (NILP (translation_table) && !NILP (Venable_character_translation))
5071 translation_table = Vstandard_translation_table_for_encode;
5072 if (CHAR_TABLE_P (translation_table))
5073 {
5074 int i;
5075 for (i = 0; i < 128; i++)
5076 if (!NILP (CHAR_TABLE_REF (translation_table, i)))
5077 break;
5078 if (i < 128)
5079 /* Some ASCII character should be translated. We give up
5080 shrinking. */
5081 return;
5082 }
5083 5494
5084 if (str) 5495 /* Note: composition handling is not yet implemented. */
5085 { 5496 coding->common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
5086 begp_orig = begp = str + *beg; 5497
5087 endp_orig = endp = str + *end; 5498 if (coding->common_flags & CODING_ANNOTATE_COMPOSITION_MASK
5088 } 5499 && find_composition (pos, end_pos, &start, &end, &prop, object)
5500 && end <= end_pos
5501 && (start >= pos
5502 || (find_composition (end, end_pos, &start, &end, &prop, object)
5503 && end <= end_pos)))
5504 stop = start;
5089 else 5505 else
5506 stop = end_pos;
5507
5508 while (buf < buf_end)
5090 { 5509 {
5091 begp_orig = begp = BYTE_POS_ADDR (*beg); 5510 if (pos == stop)
5092 endp_orig = endp = begp + *end - *beg; 5511 {
5093 } 5512 int *p;
5094 5513
5095 eol_conversion = (coding->eol_type == CODING_EOL_CR 5514 if (pos == end_pos)
5096 || coding->eol_type == CODING_EOL_CRLF); 5515 break;
5516 p = save_composition_data (buf, buf_end, prop);
5517 if (p == NULL)
5518 break;
5519 buf = p;
5520 if (find_composition (end, end_pos, &start, &end, &prop, object)
5521 && end <= end_pos)
5522 stop = start;
5523 else
5524 stop = end_pos;
5525 }
5097 5526
5098 /* Here, we don't have to check coding->pre_write_conversion because 5527 if (! multibytep)
5099 the caller is expected to have handled it already. */ 5528 c = *src++;
5100 switch (coding->type) 5529 else
5101 { 5530 c = STRING_CHAR_ADVANCE (src);
5102 case coding_type_iso2022: 5531 if ((c == '\r') && (coding->mode & CODING_MODE_SELECTIVE_DISPLAY))
5103 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, 0) != CHARSET_ASCII) 5532 c = '\n';
5104 /* We can't skip any data. */ 5533 if (! EQ (eol_type, Qunix))
5105 break;
5106 if (coding->flags & CODING_FLAG_ISO_DESIGNATE_AT_BOL)
5107 { 5534 {
5108 unsigned char *bol = begp; 5535 if (c == '\n')
5109 while (begp < endp && *begp < 0x80)
5110 { 5536 {
5111 begp++; 5537 if (EQ (eol_type, Qdos))
5112 if (begp[-1] == '\n') 5538 *buf++ = '\r';
5113 bol = begp; 5539 else
5540 c = '\r';
5114 } 5541 }
5115 begp = bol;
5116 goto label_skip_tail;
5117 } 5542 }
5118 /* fall down ... */ 5543 *buf++ = c;
5119 5544 pos++;
5120 case coding_type_sjis:
5121 case coding_type_big5:
5122 /* We can skip all ASCII characters at the head and tail. */
5123 if (eol_conversion)
5124 while (begp < endp && *begp < 0x80 && *begp != '\n') begp++;
5125 else
5126 while (begp < endp && *begp < 0x80) begp++;
5127 label_skip_tail:
5128 if (eol_conversion)
5129 while (begp < endp && endp[-1] < 0x80 && endp[-1] != '\n') endp--;
5130 else
5131 while (begp < endp && *(endp - 1) < 0x80) endp--;
5132 break;
5133
5134 default:
5135 abort ();
5136 } 5545 }
5137 5546
5138 *beg += begp - begp_orig; 5547 coding->consumed = src - coding->source;
5139 *end += endp - endp_orig; 5548 coding->consumed_char = pos - coding->src_pos;
5140 return; 5549 coding->charbuf_used = buf - coding->charbuf;
5550 coding->chars_at_source = 0;
5141} 5551}
5142 5552
5143/* As shrinking conversion region requires some overhead, we don't try
5144 shrinking if the length of conversion region is less than this
5145 value. */
5146static int shrink_conversion_region_threshhold = 1024;
5147 5553
5148#define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \ 5554/* Encode the text at CODING->src_object into CODING->dst_object.
5149 do { \ 5555 CODING->src_object is a buffer or a string.
5150 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \ 5556 CODING->dst_object is a buffer or nil.
5151 { \
5152 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5153 else shrink_decoding_region (beg, end, coding, str); \
5154 } \
5155 } while (0)
5156 5557
5157static Lisp_Object 5558 If CODING->src_object is a buffer, it must be the current buffer.
5158code_convert_region_unwind (dummy) 5559 In this case, if CODING->src_pos is positive, it is a position of
5159 Lisp_Object dummy; 5560 the source text in the buffer, otherwise. the source text is in the
5160{ 5561 gap area of the buffer, and coding->src_pos specifies the offset of
5161 inhibit_pre_post_conversion = 0; 5562 the text from GPT (which must be the same as PT). If this is the
5162 return Qnil; 5563 same buffer as CODING->dst_object, CODING->src_pos must be
5163} 5564 negative and CODING should not have `pre-write-conversion'.
5164 5565
5165/* Store information about all compositions in the range FROM and TO 5566 If CODING->src_object is a string, CODING should not have
5166 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a 5567 `pre-write-conversion'.
5167 buffer or a string, defaults to the current buffer. */
5168 5568
5169void 5569 If CODING->dst_object is a buffer, the encoded data is inserted at
5170coding_save_composition (coding, from, to, obj) 5570 the current point of that buffer.
5571
5572 If CODING->dst_object is nil, the encoded data is placed at the
5573 memory area specified by CODING->destination. */
5574
5575static int
5576encode_coding (coding)
5171 struct coding_system *coding; 5577 struct coding_system *coding;
5172 int from, to;
5173 Lisp_Object obj;
5174{ 5578{
5175 Lisp_Object prop; 5579 int error = 0;
5176 int start, end; 5580 Lisp_Object attrs;
5177 5581
5178 if (coding->composing == COMPOSITION_DISABLED) 5582 attrs = CODING_ID_ATTRS (coding->id);
5179 return;
5180 if (!coding->cmp_data)
5181 coding_allocate_composition_data (coding, from);
5182 if (!find_composition (from, to, &start, &end, &prop, obj)
5183 || end > to)
5184 return;
5185 if (start < from
5186 && (!find_composition (end, to, &start, &end, &prop, obj)
5187 || end > to))
5188 return;
5189 coding->composing = COMPOSITION_NO;
5190 do
5191 {
5192 if (COMPOSITION_VALID_P (start, end, prop))
5193 {
5194 enum composition_method method = COMPOSITION_METHOD (prop);
5195 if (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH
5196 >= COMPOSITION_DATA_SIZE)
5197 coding_allocate_composition_data (coding, from);
5198 /* For relative composition, we remember start and end
5199 positions, for the other compositions, we also remember
5200 components. */
5201 CODING_ADD_COMPOSITION_START (coding, start - from, method);
5202 if (method != COMPOSITION_RELATIVE)
5203 {
5204 /* We must store a*/
5205 Lisp_Object val, ch;
5206 5583
5207 val = COMPOSITION_COMPONENTS (prop); 5584 if (BUFFERP (coding->dst_object))
5208 if (CONSP (val)) 5585 {
5209 while (CONSP (val)) 5586 set_buffer_internal (XBUFFER (coding->dst_object));
5210 { 5587 coding->dst_multibyte
5211 ch = XCAR (val), val = XCDR (val); 5588 = ! NILP (current_buffer->enable_multibyte_characters);
5212 CODING_ADD_COMPOSITION_COMPONENT (coding, XINT (ch));
5213 }
5214 else if (VECTORP (val) || STRINGP (val))
5215 {
5216 int len = (VECTORP (val)
5217 ? XVECTOR (val)->size : XSTRING (val)->size);
5218 int i;
5219 for (i = 0; i < len; i++)
5220 {
5221 ch = (STRINGP (val)
5222 ? Faref (val, make_number (i))
5223 : XVECTOR (val)->contents[i]);
5224 CODING_ADD_COMPOSITION_COMPONENT (coding, XINT (ch));
5225 }
5226 }
5227 else /* INTEGERP (val) */
5228 CODING_ADD_COMPOSITION_COMPONENT (coding, XINT (val));
5229 }
5230 CODING_ADD_COMPOSITION_END (coding, end - from);
5231 }
5232 start = end;
5233 } 5589 }
5234 while (start < to
5235 && find_composition (start, to, &start, &end, &prop, obj)
5236 && end <= to);
5237
5238 /* Make coding->cmp_data point to the first memory block. */
5239 while (coding->cmp_data->prev)
5240 coding->cmp_data = coding->cmp_data->prev;
5241 coding->cmp_data_start = 0;
5242}
5243
5244/* Reflect the saved information about compositions to OBJ.
5245 CODING->cmp_data points to a memory block for the information. OBJ
5246 is a buffer or a string, defaults to the current buffer. */
5247 5590
5248void 5591 coding->consumed = coding->consumed_char = 0;
5249coding_restore_composition (coding, obj) 5592 coding->produced = coding->produced_char = 0;
5250 struct coding_system *coding; 5593 coding->result = CODING_RESULT_SUCCESS;
5251 Lisp_Object obj; 5594 coding->errors = 0;
5252{
5253 struct composition_data *cmp_data = coding->cmp_data;
5254 5595
5255 if (!cmp_data) 5596 ALLOC_CONVERSION_WORK_AREA (coding);
5256 return;
5257 5597
5258 while (cmp_data->prev) 5598 do {
5259 cmp_data = cmp_data->prev; 5599 coding_set_source (coding);
5600 consume_chars (coding);
5260 5601
5261 while (cmp_data) 5602 if (!NILP (CODING_ATTR_ENCODE_TBL (attrs)))
5262 { 5603 translate_chars (CODING_ATTR_ENCODE_TBL (attrs), coding);
5263 int i;
5264 5604
5265 for (i = 0; i < cmp_data->used && cmp_data->data[i] > 0; 5605 coding_set_destination (coding);
5266 i += cmp_data->data[i]) 5606 (*(coding->encoder)) (coding);
5267 { 5607 } while (coding->consumed_char < coding->src_chars);
5268 int *data = cmp_data->data + i;
5269 enum composition_method method = (enum composition_method) data[3];
5270 Lisp_Object components;
5271 5608
5272 if (method == COMPOSITION_RELATIVE) 5609 if (BUFFERP (coding->dst_object))
5273 components = Qnil; 5610 insert_from_gap (coding->produced_char, coding->produced);
5274 else
5275 {
5276 int len = data[0] - 4, j;
5277 Lisp_Object args[MAX_COMPOSITION_COMPONENTS * 2 - 1];
5278 5611
5279 for (j = 0; j < len; j++) 5612 return (coding->result);
5280 args[j] = make_number (data[4 + j]);
5281 components = (method == COMPOSITION_WITH_ALTCHARS
5282 ? Fstring (len, args) : Fvector (len, args));
5283 }
5284 compose_text (data[1], data[2], components, Qnil, obj);
5285 }
5286 cmp_data = cmp_data->next;
5287 }
5288} 5613}
5289 5614
5290/* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the 5615/* Work buffer */
5291 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5292 coding system CODING, and return the status code of code conversion
5293 (currently, this value has no meaning).
5294 5616
5295 How many characters (and bytes) are converted to how many 5617/* List of currently used working buffer. */
5296 characters (and bytes) are recorded in members of the structure 5618Lisp_Object Vcode_conversion_work_buf_list;
5297 CODING.
5298 5619
5299 If REPLACE is nonzero, we do various things as if the original text 5620/* A working buffer used by the top level conversion. */
5300 is deleted and a new text is inserted. See the comments in 5621Lisp_Object Vcode_conversion_reused_work_buf;
5301 replace_range (insdel.c) to know what we are doing.
5302 5622
5303 If REPLACE is zero, it is assumed that the source text is unibyte.
5304 Otherwise, it is assumed that the source text is multibyte. */
5305 5623
5306int 5624/* Return a working buffer that can be freely used by the following
5307code_convert_region (from, from_byte, to, to_byte, coding, encodep, replace) 5625 code conversion. MULTIBYTEP specifies the multibyteness of the
5308 int from, from_byte, to, to_byte, encodep, replace; 5626 buffer. */
5309 struct coding_system *coding;
5310{
5311 int len = to - from, len_byte = to_byte - from_byte;
5312 int nchars_del = 0, nbytes_del = 0;
5313 int require, inserted, inserted_byte;
5314 int head_skip, tail_skip, total_skip = 0;
5315 Lisp_Object saved_coding_symbol;
5316 int first = 1;
5317 unsigned char *src, *dst;
5318 Lisp_Object deletion;
5319 int orig_point = PT, orig_len = len;
5320 int prev_Z;
5321 int multibyte_p = !NILP (current_buffer->enable_multibyte_characters);
5322 5627
5323 deletion = Qnil; 5628Lisp_Object
5324 saved_coding_symbol = coding->symbol; 5629make_conversion_work_buffer (multibytep)
5630 int multibytep;
5631{
5632 struct buffer *current = current_buffer;
5633 Lisp_Object buf;
5325 5634
5326 if (from < PT && PT < to) 5635 if (NILP (Vcode_conversion_work_buf_list))
5327 { 5636 {
5328 TEMP_SET_PT_BOTH (from, from_byte); 5637 if (NILP (Vcode_conversion_reused_work_buf))
5329 orig_point = from; 5638 Vcode_conversion_reused_work_buf
5639 = Fget_buffer_create (build_string (" *code-conversion-work*"));
5640 Vcode_conversion_work_buf_list
5641 = Fcons (Vcode_conversion_reused_work_buf, Qnil);
5330 } 5642 }
5331 5643 else
5332 if (replace)
5333 { 5644 {
5334 int saved_from = from; 5645 int depth = Flength (Vcode_conversion_work_buf_list);
5335 int saved_inhibit_modification_hooks; 5646 char str[128];
5336 5647
5337 prepare_to_modify_buffer (from, to, &from); 5648 sprintf (str, " *code-conversion-work*<%d>", depth);
5338 if (saved_from != from) 5649 Vcode_conversion_work_buf_list
5339 { 5650 = Fcons (Fget_buffer_create (build_string (str)),
5340 to = from + len; 5651 Vcode_conversion_work_buf_list);
5341 from_byte = CHAR_TO_BYTE (from), to_byte = CHAR_TO_BYTE (to);
5342 len_byte = to_byte - from_byte;
5343 }
5344
5345 /* The code conversion routine can not preserve text properties
5346 for now. So, we must remove all text properties in the
5347 region. Here, we must suppress all modification hooks. */
5348 saved_inhibit_modification_hooks = inhibit_modification_hooks;
5349 inhibit_modification_hooks = 1;
5350 Fset_text_properties (make_number (from), make_number (to), Qnil, Qnil);
5351 inhibit_modification_hooks = saved_inhibit_modification_hooks;
5352 } 5652 }
5353 5653
5354 if (! encodep && CODING_REQUIRE_DETECTION (coding)) 5654 buf = XCAR (Vcode_conversion_work_buf_list);
5355 { 5655 set_buffer_internal (XBUFFER (buf));
5356 /* We must detect encoding of text and eol format. */ 5656 current_buffer->undo_list = Qt;
5657 Ferase_buffer ();
5658 Fset_buffer_multibyte (multibytep ? Qt : Qnil);
5659 set_buffer_internal (current);
5660 return buf;
5661}
5357 5662
5358 if (from < GPT && to > GPT) 5663static struct coding_system *saved_coding;
5359 move_gap_both (from, from_byte);
5360 if (coding->type == coding_type_undecided)
5361 {
5362 detect_coding (coding, BYTE_POS_ADDR (from_byte), len_byte);
5363 if (coding->type == coding_type_undecided)
5364 {
5365 /* It seems that the text contains only ASCII, but we
5366 should not leave it undecided because the deeper
5367 decoding routine (decode_coding) tries to detect the
5368 encodings again in vain. */
5369 coding->type = coding_type_emacs_mule;
5370 coding->category_idx = CODING_CATEGORY_IDX_EMACS_MULE;
5371 /* As emacs-mule decoder will handle composition, we
5372 need this setting to allocate coding->cmp_data
5373 later. */
5374 coding->composing = COMPOSITION_NO;
5375 }
5376 }
5377 if (coding->eol_type == CODING_EOL_UNDECIDED
5378 && coding->type != coding_type_ccl)
5379 {
5380 detect_eol (coding, BYTE_POS_ADDR (from_byte), len_byte);
5381 if (coding->eol_type == CODING_EOL_UNDECIDED)
5382 coding->eol_type = CODING_EOL_LF;
5383 /* We had better recover the original eol format if we
5384 encounter an inconsistent eol format while decoding. */
5385 coding->mode |= CODING_MODE_INHIBIT_INCONSISTENT_EOL;
5386 }
5387 }
5388 5664
5389 /* Now we convert the text. */ 5665Lisp_Object
5666code_conversion_restore (info)
5667 Lisp_Object info;
5668{
5669 int depth = Flength (Vcode_conversion_work_buf_list);
5670 Lisp_Object buf;
5390 5671
5391 /* For encoding, we must process pre-write-conversion in advance. */ 5672 if (depth > 0)
5392 if (! inhibit_pre_post_conversion
5393 && encodep
5394 && SYMBOLP (coding->pre_write_conversion)
5395 && ! NILP (Ffboundp (coding->pre_write_conversion)))
5396 { 5673 {
5397 /* The function in pre-write-conversion may put a new text in a 5674 buf = XCAR (Vcode_conversion_work_buf_list);
5398 new buffer. */ 5675 Vcode_conversion_work_buf_list = XCDR (Vcode_conversion_work_buf_list);
5399 struct buffer *prev = current_buffer; 5676 if (depth > 1 && !NILP (Fbuffer_live_p (buf)))
5400 Lisp_Object new; 5677 Fkill_buffer (buf);
5678 }
5401 5679
5402 record_unwind_protect (code_convert_region_unwind, Qnil); 5680 if (saved_coding->dst_object == Qt
5403 /* We should not call any more pre-write/post-read-conversion 5681 && saved_coding->destination)
5404 functions while this pre-write-conversion is running. */ 5682 xfree (saved_coding->destination);
5405 inhibit_pre_post_conversion = 1;
5406 call2 (coding->pre_write_conversion,
5407 make_number (from), make_number (to));
5408 inhibit_pre_post_conversion = 0;
5409 /* Discard the unwind protect. */
5410 specpdl_ptr--;
5411 5683
5412 if (current_buffer != prev) 5684 return save_excursion_restore (info);
5413 { 5685}
5414 len = ZV - BEGV;
5415 new = Fcurrent_buffer ();
5416 set_buffer_internal_1 (prev);
5417 del_range_2 (from, from_byte, to, to_byte, 0);
5418 TEMP_SET_PT_BOTH (from, from_byte);
5419 insert_from_buffer (XBUFFER (new), 1, len, 0);
5420 Fkill_buffer (new);
5421 if (orig_point >= to)
5422 orig_point += len - orig_len;
5423 else if (orig_point > from)
5424 orig_point = from;
5425 orig_len = len;
5426 to = from + len;
5427 from_byte = CHAR_TO_BYTE (from);
5428 to_byte = CHAR_TO_BYTE (to);
5429 len_byte = to_byte - from_byte;
5430 TEMP_SET_PT_BOTH (from, from_byte);
5431 }
5432 }
5433 5686
5434 if (replace)
5435 {
5436 if (! EQ (current_buffer->undo_list, Qt))
5437 deletion = make_buffer_string_both (from, from_byte, to, to_byte, 1);
5438 else
5439 {
5440 nchars_del = to - from;
5441 nbytes_del = to_byte - from_byte;
5442 }
5443 }
5444 5687
5445 if (coding->composing != COMPOSITION_DISABLED) 5688int
5446 { 5689decode_coding_gap (coding, chars, bytes)
5447 if (encodep) 5690 struct coding_system *coding;
5448 coding_save_composition (coding, from, to, Fcurrent_buffer ()); 5691 EMACS_INT chars, bytes;
5449 else 5692{
5450 coding_allocate_composition_data (coding, from); 5693 int count = specpdl_ptr - specpdl;
5451 }
5452 5694
5453 /* Try to skip the heading and tailing ASCIIs. */ 5695 saved_coding = coding;
5454 if (coding->type != coding_type_ccl) 5696 record_unwind_protect (code_conversion_restore, save_excursion_save ());
5455 {
5456 int from_byte_orig = from_byte, to_byte_orig = to_byte;
5457 5697
5458 if (from < GPT && GPT < to) 5698 coding->src_object = Fcurrent_buffer ();
5459 move_gap_both (from, from_byte); 5699 coding->src_chars = chars;
5460 SHRINK_CONVERSION_REGION (&from_byte, &to_byte, coding, NULL, encodep); 5700 coding->src_bytes = bytes;
5461 if (from_byte == to_byte 5701 coding->src_pos = -chars;
5462 && (encodep || NILP (coding->post_read_conversion)) 5702 coding->src_pos_byte = -bytes;
5463 && ! CODING_REQUIRE_FLUSHING (coding)) 5703 coding->src_multibyte = chars < bytes;
5464 { 5704 coding->dst_object = coding->src_object;
5465 coding->produced = len_byte; 5705 coding->dst_pos = PT;
5466 coding->produced_char = len; 5706 coding->dst_pos_byte = PT_BYTE;
5467 if (!replace)
5468 /* We must record and adjust for this new text now. */
5469 adjust_after_insert (from, from_byte_orig, to, to_byte_orig, len);
5470 return 0;
5471 }
5472 5707
5473 head_skip = from_byte - from_byte_orig; 5708 if (CODING_REQUIRE_DETECTION (coding))
5474 tail_skip = to_byte_orig - to_byte; 5709 detect_coding (coding);
5475 total_skip = head_skip + tail_skip; 5710
5476 from += head_skip; 5711 decode_coding (coding);
5477 to -= tail_skip;
5478 len -= total_skip; len_byte -= total_skip;
5479 }
5480 5712
5481 /* For conversion, we must put the gap before the text in addition to 5713 unbind_to (count, Qnil);
5482 making the gap larger for efficient decoding. The required gap 5714 return coding->result;
5483 size starts from 2000 which is the magic number used in make_gap. 5715}
5484 But, after one batch of conversion, it will be incremented if we
5485 find that it is not enough . */
5486 require = 2000;
5487 5716
5488 if (GAP_SIZE < require) 5717int
5489 make_gap (require - GAP_SIZE); 5718encode_coding_gap (coding, chars, bytes)
5490 move_gap_both (from, from_byte); 5719 struct coding_system *coding;
5720 EMACS_INT chars, bytes;
5721{
5722 int count = specpdl_ptr - specpdl;
5723 Lisp_Object buffer;
5491 5724
5492 inserted = inserted_byte = 0; 5725 saved_coding = coding;
5726 record_unwind_protect (code_conversion_restore, save_excursion_save ());
5493 5727
5494 GAP_SIZE += len_byte; 5728 buffer = Fcurrent_buffer ();
5495 ZV -= len; 5729 coding->src_object = buffer;
5496 Z -= len; 5730 coding->src_chars = chars;
5497 ZV_BYTE -= len_byte; 5731 coding->src_bytes = bytes;
5498 Z_BYTE -= len_byte; 5732 coding->src_pos = -chars;
5733 coding->src_pos_byte = -bytes;
5734 coding->src_multibyte = chars < bytes;
5735 coding->dst_object = coding->src_object;
5736 coding->dst_pos = PT;
5737 coding->dst_pos_byte = PT_BYTE;
5499 5738
5500 if (GPT - BEG < BEG_UNCHANGED) 5739 encode_coding (coding);
5501 BEG_UNCHANGED = GPT - BEG;
5502 if (Z - GPT < END_UNCHANGED)
5503 END_UNCHANGED = Z - GPT;
5504 5740
5505 if (!encodep && coding->src_multibyte) 5741 unbind_to (count, Qnil);
5506 { 5742 return coding->result;
5507 /* Decoding routines expects that the source text is unibyte. 5743}
5508 We must convert 8-bit characters of multibyte form to
5509 unibyte. */
5510 int len_byte_orig = len_byte;
5511 len_byte = str_as_unibyte (GAP_END_ADDR - len_byte, len_byte);
5512 if (len_byte < len_byte_orig)
5513 safe_bcopy (GAP_END_ADDR - len_byte_orig, GAP_END_ADDR - len_byte,
5514 len_byte);
5515 coding->src_multibyte = 0;
5516 }
5517 5744
5518 for (;;)
5519 {
5520 int result;
5521 5745
5522 /* The buffer memory is now: 5746/* Decode the text in the range FROM/FROM_BYTE and TO/TO_BYTE in
5523 +--------+converted-text+---------+-------original-text-------+---+ 5747 SRC_OBJECT into DST_OBJECT by coding context CODING.
5524 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5525 |<---------------------- GAP ----------------------->| */
5526 src = GAP_END_ADDR - len_byte;
5527 dst = GPT_ADDR + inserted_byte;
5528 5748
5529 if (encodep) 5749 SRC_OBJECT is a buffer, a string, or Qnil.
5530 result = encode_coding (coding, src, dst, len_byte, 0);
5531 else
5532 {
5533 if (coding->composing != COMPOSITION_DISABLED)
5534 coding->cmp_data->char_offset = from + inserted;
5535 result = decode_coding (coding, src, dst, len_byte, 0);
5536 }
5537 5750
5538 /* The buffer memory is now: 5751 If it is a buffer, the text is at point of the buffer. FROM and TO
5539 +--------+-------converted-text----+--+------original-text----+---+ 5752 are positions in the buffer.
5540 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5541 |<---------------------- GAP ----------------------->| */
5542 5753
5543 inserted += coding->produced_char; 5754 If it is a string, the text is at the beginning of the string.
5544 inserted_byte += coding->produced; 5755 FROM and TO are indices to the string.
5545 len_byte -= coding->consumed;
5546 5756
5547 if (result == CODING_FINISH_INSUFFICIENT_CMP) 5757 If it is nil, the text is at coding->source. FROM and TO are
5548 { 5758 indices to coding->source.
5549 coding_allocate_composition_data (coding, from + inserted);
5550 continue;
5551 }
5552 5759
5553 src += coding->consumed; 5760 DST_OBJECT is a buffer, Qt, or Qnil.
5554 dst += coding->produced;
5555 5761
5556 if (result == CODING_FINISH_NORMAL) 5762 If it is a buffer, the decoded text is inserted at point of the
5557 { 5763 buffer. If the buffer is the same as SRC_OBJECT, the source text
5558 src += len_byte; 5764 is deleted.
5559 break;
5560 }
5561 if (! encodep && result == CODING_FINISH_INCONSISTENT_EOL)
5562 {
5563 unsigned char *pend = dst, *p = pend - inserted_byte;
5564 Lisp_Object eol_type;
5565 5765
5566 /* Encode LFs back to the original eol format (CR or CRLF). */ 5766 If it is Qt, a string is made from the decoded text, and
5567 if (coding->eol_type == CODING_EOL_CR) 5767 set in CODING->dst_object.
5568 {
5569 while (p < pend) if (*p++ == '\n') p[-1] = '\r';
5570 }
5571 else
5572 {
5573 int count = 0;
5574 5768
5575 while (p < pend) if (*p++ == '\n') count++; 5769 If it is Qnil, the decoded text is stored at CODING->destination.
5576 if (src - dst < count) 5770 The called must allocate CODING->dst_bytes bytes at
5577 { 5771 CODING->destination by xmalloc. If the decoded text is longer than
5578 /* We don't have sufficient room for encoding LFs 5772 CODING->dst_bytes, CODING->destination is relocated by xrealloc.
5579 back to CRLF. We must record converted and 5773 */
5580 not-yet-converted text back to the buffer
5581 content, enlarge the gap, then record them out of
5582 the buffer contents again. */
5583 int add = len_byte + inserted_byte;
5584
5585 GAP_SIZE -= add;
5586 ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
5587 GPT += inserted_byte; GPT_BYTE += inserted_byte;
5588 make_gap (count - GAP_SIZE);
5589 GAP_SIZE += add;
5590 ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
5591 GPT -= inserted_byte; GPT_BYTE -= inserted_byte;
5592 /* Don't forget to update SRC, DST, and PEND. */
5593 src = GAP_END_ADDR - len_byte;
5594 dst = GPT_ADDR + inserted_byte;
5595 pend = dst;
5596 }
5597 inserted += count;
5598 inserted_byte += count;
5599 coding->produced += count;
5600 p = dst = pend + count;
5601 while (count)
5602 {
5603 *--p = *--pend;
5604 if (*p == '\n') count--, *--p = '\r';
5605 }
5606 }
5607 5774
5608 /* Suppress eol-format conversion in the further conversion. */ 5775void
5609 coding->eol_type = CODING_EOL_LF; 5776decode_coding_object (coding, src_object, from, from_byte, to, to_byte,
5777 dst_object)
5778 struct coding_system *coding;
5779 Lisp_Object src_object;
5780 EMACS_INT from, from_byte, to, to_byte;
5781 Lisp_Object dst_object;
5782{
5783 int count = specpdl_ptr - specpdl;
5784 unsigned char *destination;
5785 EMACS_INT dst_bytes;
5786 EMACS_INT chars = to - from;
5787 EMACS_INT bytes = to_byte - from_byte;
5788 Lisp_Object attrs;
5610 5789
5611 /* Set the coding system symbol to that for Unix-like EOL. */ 5790 saved_coding = coding;
5612 eol_type = Fget (saved_coding_symbol, Qeol_type); 5791 record_unwind_protect (code_conversion_restore, save_excursion_save ());
5613 if (VECTORP (eol_type)
5614 && XVECTOR (eol_type)->size == 3
5615 && SYMBOLP (XVECTOR (eol_type)->contents[CODING_EOL_LF]))
5616 coding->symbol = XVECTOR (eol_type)->contents[CODING_EOL_LF];
5617 else
5618 coding->symbol = saved_coding_symbol;
5619 5792
5620 continue; 5793 if (NILP (dst_object))
5621 } 5794 {
5622 if (len_byte <= 0) 5795 destination = coding->destination;
5623 { 5796 dst_bytes = coding->dst_bytes;
5624 if (coding->type != coding_type_ccl 5797 }
5625 || coding->mode & CODING_MODE_LAST_BLOCK)
5626 break;
5627 coding->mode |= CODING_MODE_LAST_BLOCK;
5628 continue;
5629 }
5630 if (result == CODING_FINISH_INSUFFICIENT_SRC)
5631 {
5632 /* The source text ends in invalid codes. Let's just
5633 make them valid buffer contents, and finish conversion. */
5634 if (multibyte_p)
5635 {
5636 unsigned char *start = dst;
5637 5798
5638 inserted += len_byte; 5799 coding->src_object = src_object;
5639 while (len_byte--) 5800 coding->src_chars = chars;
5640 { 5801 coding->src_bytes = bytes;
5641 int c = *src++; 5802 coding->src_multibyte = chars < bytes;
5642 dst += CHAR_STRING (c, dst);
5643 }
5644 5803
5645 inserted_byte += dst - start; 5804 if (STRINGP (src_object))
5646 } 5805 {
5647 else 5806 coding->src_pos = from;
5648 { 5807 coding->src_pos_byte = from_byte;
5649 inserted += len_byte; 5808 }
5650 inserted_byte += len_byte; 5809 else if (BUFFERP (src_object))
5651 while (len_byte--) 5810 {
5652 *dst++ = *src++; 5811 set_buffer_internal (XBUFFER (src_object));
5653 } 5812 if (from != GPT)
5654 break; 5813 move_gap_both (from, from_byte);
5655 } 5814 if (EQ (src_object, dst_object))
5656 if (result == CODING_FINISH_INTERRUPT)
5657 {
5658 /* The conversion procedure was interrupted by a user. */
5659 break;
5660 }
5661 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5662 if (coding->consumed < 1)
5663 {
5664 /* It's quite strange to require more memory without
5665 consuming any bytes. Perhaps CCL program bug. */
5666 break;
5667 }
5668 if (first)
5669 { 5815 {
5670 /* We have just done the first batch of conversion which was 5816 TEMP_SET_PT_BOTH (from, from_byte);
5671 stopped because of insufficient gap. Let's reconsider the 5817 del_range_both (from, from_byte, to, to_byte, 1);
5672 required gap size (i.e. SRT - DST) now. 5818 coding->src_pos = -chars;
5673 5819 coding->src_pos_byte = -bytes;
5674 We have converted ORIG bytes (== coding->consumed) into
5675 NEW bytes (coding->produced). To convert the remaining
5676 LEN bytes, we may need REQUIRE bytes of gap, where:
5677 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5678 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5679 Here, we are sure that NEW >= ORIG. */
5680 float ratio = coding->produced - coding->consumed;
5681 ratio /= coding->consumed;
5682 require = len_byte * ratio;
5683 first = 0;
5684 } 5820 }
5685 if ((src - dst) < (require + 2000)) 5821 else
5686 { 5822 {
5687 /* See the comment above the previous call of make_gap. */ 5823 coding->src_pos = from;
5688 int add = len_byte + inserted_byte; 5824 coding->src_pos_byte = from_byte;
5689
5690 GAP_SIZE -= add;
5691 ZV += add; Z += add; ZV_BYTE += add; Z_BYTE += add;
5692 GPT += inserted_byte; GPT_BYTE += inserted_byte;
5693 make_gap (require + 2000);
5694 GAP_SIZE += add;
5695 ZV -= add; Z -= add; ZV_BYTE -= add; Z_BYTE -= add;
5696 GPT -= inserted_byte; GPT_BYTE -= inserted_byte;
5697 } 5825 }
5698 } 5826 }
5699 if (src - dst > 0) *dst = 0; /* Put an anchor. */
5700 5827
5701 if (encodep && coding->dst_multibyte) 5828 if (CODING_REQUIRE_DETECTION (coding))
5829 detect_coding (coding);
5830 attrs = CODING_ID_ATTRS (coding->id);
5831
5832 if (! NILP (CODING_ATTR_POST_READ (attrs))
5833 || EQ (dst_object, Qt))
5702 { 5834 {
5703 /* The output is unibyte. We must convert 8-bit characters to 5835 coding->dst_object = make_conversion_work_buffer (1);
5704 multibyte form. */ 5836 coding->dst_pos = BEG;
5705 if (inserted_byte * 2 > GAP_SIZE) 5837 coding->dst_pos_byte = BEG_BYTE;
5706 { 5838 coding->dst_multibyte = 1;
5707 GAP_SIZE -= inserted_byte;
5708 ZV += inserted_byte; Z += inserted_byte;
5709 ZV_BYTE += inserted_byte; Z_BYTE += inserted_byte;
5710 GPT += inserted_byte; GPT_BYTE += inserted_byte;
5711 make_gap (inserted_byte - GAP_SIZE);
5712 GAP_SIZE += inserted_byte;
5713 ZV -= inserted_byte; Z -= inserted_byte;
5714 ZV_BYTE -= inserted_byte; Z_BYTE -= inserted_byte;
5715 GPT -= inserted_byte; GPT_BYTE -= inserted_byte;
5716 }
5717 inserted_byte = str_to_multibyte (GPT_ADDR, GAP_SIZE, inserted_byte);
5718 } 5839 }
5719 5840 else if (BUFFERP (dst_object))
5720 /* If we shrank the conversion area, adjust it now. */
5721 if (total_skip > 0)
5722 { 5841 {
5723 if (tail_skip > 0) 5842 coding->dst_object = dst_object;
5724 safe_bcopy (GAP_END_ADDR, GPT_ADDR + inserted_byte, tail_skip); 5843 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
5725 inserted += total_skip; inserted_byte += total_skip; 5844 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
5726 GAP_SIZE += total_skip; 5845 coding->dst_multibyte
5727 GPT -= head_skip; GPT_BYTE -= head_skip; 5846 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
5728 ZV -= total_skip; ZV_BYTE -= total_skip;
5729 Z -= total_skip; Z_BYTE -= total_skip;
5730 from -= head_skip; from_byte -= head_skip;
5731 to += tail_skip; to_byte += tail_skip;
5732 } 5847 }
5733
5734 prev_Z = Z;
5735 if (! EQ (current_buffer->undo_list, Qt))
5736 adjust_after_replace (from, from_byte, deletion, inserted, inserted_byte);
5737 else 5848 else
5738 adjust_after_replace_noundo (from, from_byte, nchars_del, nbytes_del, 5849 {
5739 inserted, inserted_byte); 5850 coding->dst_object = Qnil;
5740 inserted = Z - prev_Z; 5851 coding->dst_multibyte = 1;
5852 }
5853
5854 decode_coding (coding);
5741 5855
5742 if (!encodep && coding->cmp_data && coding->cmp_data->used) 5856 if (BUFFERP (coding->dst_object))
5743 coding_restore_composition (coding, Fcurrent_buffer ()); 5857 set_buffer_internal (XBUFFER (coding->dst_object));
5744 coding_free_composition_data (coding);
5745 5858
5746 if (! inhibit_pre_post_conversion 5859 if (! NILP (CODING_ATTR_POST_READ (attrs)))
5747 && ! encodep && ! NILP (coding->post_read_conversion))
5748 { 5860 {
5861 struct gcpro gcpro1, gcpro2;
5862 EMACS_INT prev_Z = Z, prev_Z_BYTE = Z_BYTE;
5749 Lisp_Object val; 5863 Lisp_Object val;
5750 5864
5751 if (from != PT) 5865 GCPRO2 (coding->src_object, coding->dst_object);
5752 TEMP_SET_PT_BOTH (from, from_byte); 5866 val = call1 (CODING_ATTR_POST_READ (attrs),
5753 prev_Z = Z; 5867 make_number (coding->produced_char));
5754 record_unwind_protect (code_convert_region_unwind, Qnil); 5868 UNGCPRO;
5755 /* We should not call any more pre-write/post-read-conversion 5869 CHECK_NATNUM (val);
5756 functions while this post-read-conversion is running. */ 5870 coding->produced_char += Z - prev_Z;
5757 inhibit_pre_post_conversion = 1; 5871 coding->produced += Z_BYTE - prev_Z_BYTE;
5758 val = call1 (coding->post_read_conversion, make_number (inserted));
5759 inhibit_pre_post_conversion = 0;
5760 /* Discard the unwind protect. */
5761 specpdl_ptr--;
5762 CHECK_NUMBER (val);
5763 inserted += Z - prev_Z;
5764 }
5765
5766 if (orig_point >= from)
5767 {
5768 if (orig_point >= from + orig_len)
5769 orig_point += inserted - orig_len;
5770 else
5771 orig_point = from;
5772 TEMP_SET_PT (orig_point);
5773 } 5872 }
5774 5873
5775 if (replace) 5874 if (EQ (dst_object, Qt))
5776 { 5875 {
5777 signal_after_change (from, to - from, inserted); 5876 coding->dst_object = Fbuffer_string ();
5778 update_compositions (from, from + inserted, CHECK_BORDER); 5877 }
5878 else if (NILP (dst_object) && BUFFERP (coding->dst_object))
5879 {
5880 set_buffer_internal (XBUFFER (coding->dst_object));
5881 if (dst_bytes < coding->produced)
5882 {
5883 destination
5884 = (unsigned char *) xrealloc (destination, coding->produced);
5885 if (! destination)
5886 {
5887 coding->result = CODING_RESULT_INSUFFICIENT_DST;
5888 unbind_to (count, Qnil);
5889 return;
5890 }
5891 if (BEGV < GPT && GPT < BEGV + coding->produced_char)
5892 move_gap_both (BEGV, BEGV_BYTE);
5893 bcopy (BEGV_ADDR, destination, coding->produced);
5894 coding->destination = destination;
5895 }
5779 } 5896 }
5780 5897
5781 { 5898 unbind_to (count, Qnil);
5782 coding->consumed = to_byte - from_byte;
5783 coding->consumed_char = to - from;
5784 coding->produced = inserted_byte;
5785 coding->produced_char = inserted;
5786 }
5787
5788 return 0;
5789} 5899}
5790 5900
5791Lisp_Object 5901
5792run_pre_post_conversion_on_str (str, coding, encodep) 5902void
5793 Lisp_Object str; 5903encode_coding_object (coding, src_object, from, from_byte, to, to_byte,
5904 dst_object)
5794 struct coding_system *coding; 5905 struct coding_system *coding;
5795 int encodep; 5906 Lisp_Object src_object;
5907 EMACS_INT from, from_byte, to, to_byte;
5908 Lisp_Object dst_object;
5796{ 5909{
5797 int count = specpdl_ptr - specpdl; 5910 int count = specpdl_ptr - specpdl;
5798 struct gcpro gcpro1; 5911 EMACS_INT chars = to - from;
5799 int multibyte = STRING_MULTIBYTE (str); 5912 EMACS_INT bytes = to_byte - from_byte;
5800 5913 Lisp_Object attrs;
5801 record_unwind_protect (Fset_buffer, Fcurrent_buffer ());
5802 record_unwind_protect (code_convert_region_unwind, Qnil);
5803 GCPRO1 (str);
5804 temp_output_buffer_setup (" *code-converting-work*");
5805 set_buffer_internal (XBUFFER (Vstandard_output));
5806 /* We must insert the contents of STR as is without
5807 unibyte<->multibyte conversion. For that, we adjust the
5808 multibyteness of the working buffer to that of STR. */
5809 Ferase_buffer ();
5810 current_buffer->enable_multibyte_characters = multibyte ? Qt : Qnil;
5811 insert_from_string (str, 0, 0,
5812 XSTRING (str)->size, STRING_BYTES (XSTRING (str)), 0);
5813 UNGCPRO;
5814 inhibit_pre_post_conversion = 1;
5815 if (encodep)
5816 call2 (coding->pre_write_conversion, make_number (BEG), make_number (Z));
5817 else
5818 {
5819 TEMP_SET_PT_BOTH (BEG, BEG_BYTE);
5820 call1 (coding->post_read_conversion, make_number (Z - BEG));
5821 }
5822 inhibit_pre_post_conversion = 0;
5823 str = make_buffer_string (BEG, Z, 1);
5824 return unbind_to (count, str);
5825}
5826 5914
5827Lisp_Object 5915 saved_coding = coding;
5828decode_coding_string (str, coding, nocopy) 5916 record_unwind_protect (code_conversion_restore, save_excursion_save ());
5829 Lisp_Object str;
5830 struct coding_system *coding;
5831 int nocopy;
5832{
5833 int len;
5834 struct conversion_buffer buf;
5835 int from, to_byte;
5836 Lisp_Object saved_coding_symbol;
5837 int result;
5838 int require_decoding;
5839 int shrinked_bytes = 0;
5840 Lisp_Object newstr;
5841 int consumed, consumed_char, produced, produced_char;
5842
5843 from = 0;
5844 to_byte = STRING_BYTES (XSTRING (str));
5845
5846 saved_coding_symbol = coding->symbol;
5847 coding->src_multibyte = STRING_MULTIBYTE (str);
5848 coding->dst_multibyte = 1;
5849 if (CODING_REQUIRE_DETECTION (coding))
5850 {
5851 /* See the comments in code_convert_region. */
5852 if (coding->type == coding_type_undecided)
5853 {
5854 detect_coding (coding, XSTRING (str)->data, to_byte);
5855 if (coding->type == coding_type_undecided)
5856 {
5857 coding->type = coding_type_emacs_mule;
5858 coding->category_idx = CODING_CATEGORY_IDX_EMACS_MULE;
5859 /* As emacs-mule decoder will handle composition, we
5860 need this setting to allocate coding->cmp_data
5861 later. */
5862 coding->composing = COMPOSITION_NO;
5863 }
5864 }
5865 if (coding->eol_type == CODING_EOL_UNDECIDED
5866 && coding->type != coding_type_ccl)
5867 {
5868 saved_coding_symbol = coding->symbol;
5869 detect_eol (coding, XSTRING (str)->data, to_byte);
5870 if (coding->eol_type == CODING_EOL_UNDECIDED)
5871 coding->eol_type = CODING_EOL_LF;
5872 /* We had better recover the original eol format if we
5873 encounter an inconsistent eol format while decoding. */
5874 coding->mode |= CODING_MODE_INHIBIT_INCONSISTENT_EOL;
5875 }
5876 }
5877 5917
5878 if (coding->type == coding_type_no_conversion 5918 coding->src_object = src_object;
5879 || coding->type == coding_type_raw_text) 5919 coding->src_chars = chars;
5880 coding->dst_multibyte = 0; 5920 coding->src_bytes = bytes;
5921 coding->src_multibyte = chars < bytes;
5881 5922
5882 require_decoding = CODING_REQUIRE_DECODING (coding); 5923 attrs = CODING_ID_ATTRS (coding->id);
5883 5924
5884 if (STRING_MULTIBYTE (str)) 5925 if (! NILP (CODING_ATTR_PRE_WRITE (attrs)))
5885 { 5926 {
5886 /* Decoding routines expect the source text to be unibyte. */ 5927 Lisp_Object val;
5887 str = Fstring_as_unibyte (str);
5888 to_byte = STRING_BYTES (XSTRING (str));
5889 nocopy = 1;
5890 coding->src_multibyte = 0;
5891 }
5892 5928
5893 /* Try to skip the heading and tailing ASCIIs. */ 5929 coding->src_object = make_conversion_work_buffer (coding->src_multibyte);
5894 if (require_decoding && coding->type != coding_type_ccl) 5930 set_buffer_internal (XBUFFER (coding->src_object));
5895 { 5931 if (STRINGP (src_object))
5896 SHRINK_CONVERSION_REGION (&from, &to_byte, coding, XSTRING (str)->data, 5932 insert_from_string (src_object, from, from_byte, chars, bytes, 0);
5897 0); 5933 else if (BUFFERP (src_object))
5898 if (from == to_byte) 5934 insert_from_buffer (XBUFFER (src_object), from, chars, 0);
5899 require_decoding = 0; 5935 else
5900 shrinked_bytes = from + (STRING_BYTES (XSTRING (str)) - to_byte); 5936 insert_1_both (coding->source + from, chars, bytes, 0, 0, 0);
5901 }
5902 5937
5903 if (!require_decoding) 5938 if (EQ (src_object, dst_object))
5904 {
5905 coding->consumed = STRING_BYTES (XSTRING (str));
5906 coding->consumed_char = XSTRING (str)->size;
5907 if (coding->dst_multibyte)
5908 { 5939 {
5909 str = Fstring_as_multibyte (str); 5940 set_buffer_internal (XBUFFER (src_object));
5910 nocopy = 1; 5941 del_range_both (from, from_byte, to, to_byte, 1);
5942 set_buffer_internal (XBUFFER (coding->src_object));
5911 } 5943 }
5912 coding->produced = STRING_BYTES (XSTRING (str));
5913 coding->produced_char = XSTRING (str)->size;
5914 return (nocopy ? str : Fcopy_sequence (str));
5915 }
5916
5917 if (coding->composing != COMPOSITION_DISABLED)
5918 coding_allocate_composition_data (coding, from);
5919 len = decoding_buffer_size (coding, to_byte - from);
5920 allocate_conversion_buffer (buf, len);
5921 5944
5922 consumed = consumed_char = produced = produced_char = 0; 5945 val = call2 (CODING_ATTR_PRE_WRITE (attrs),
5923 while (1) 5946 make_number (1), make_number (chars));
5947 CHECK_NATNUM (val);
5948 if (BEG != GPT)
5949 move_gap_both (BEG, BEG_BYTE);
5950 coding->src_chars = Z - BEG;
5951 coding->src_bytes = Z_BYTE - BEG_BYTE;
5952 coding->src_pos = BEG;
5953 coding->src_pos_byte = BEG_BYTE;
5954 coding->src_multibyte = Z < Z_BYTE;
5955 }
5956 else if (STRINGP (src_object))
5924 { 5957 {
5925 result = decode_coding (coding, XSTRING (str)->data + from + consumed, 5958 coding->src_pos = from;
5926 buf.data + produced, to_byte - from - consumed, 5959 coding->src_pos_byte = from_byte;
5927 buf.size - produced); 5960 }
5928 consumed += coding->consumed; 5961 else if (BUFFERP (src_object))
5929 consumed_char += coding->consumed_char; 5962 {
5930 produced += coding->produced; 5963 set_buffer_internal (XBUFFER (src_object));
5931 produced_char += coding->produced_char; 5964 if (from != GPT)
5932 if (result == CODING_FINISH_NORMAL 5965 move_gap_both (from, from_byte);
5933 || (result == CODING_FINISH_INSUFFICIENT_SRC 5966 if (EQ (src_object, dst_object))
5934 && coding->consumed == 0))
5935 break;
5936 if (result == CODING_FINISH_INSUFFICIENT_CMP)
5937 coding_allocate_composition_data (coding, from + produced_char);
5938 else if (result == CODING_FINISH_INSUFFICIENT_DST)
5939 extend_conversion_buffer (&buf);
5940 else if (result == CODING_FINISH_INCONSISTENT_EOL)
5941 { 5967 {
5942 Lisp_Object eol_type; 5968 del_range_both (from, from_byte, to, to_byte, 1);
5943 5969 coding->src_pos = -chars;
5944 /* Recover the original EOL format. */ 5970 coding->src_pos_byte = -bytes;
5945 if (coding->eol_type == CODING_EOL_CR)
5946 {
5947 unsigned char *p;
5948 for (p = buf.data; p < buf.data + produced; p++)
5949 if (*p == '\n') *p = '\r';
5950 }
5951 else if (coding->eol_type == CODING_EOL_CRLF)
5952 {
5953 int num_eol = 0;
5954 unsigned char *p0, *p1;
5955 for (p0 = buf.data, p1 = p0 + produced; p0 < p1; p0++)
5956 if (*p0 == '\n') num_eol++;
5957 if (produced + num_eol >= buf.size)
5958 extend_conversion_buffer (&buf);
5959 for (p0 = buf.data + produced, p1 = p0 + num_eol; p0 > buf.data;)
5960 {
5961 *--p1 = *--p0;
5962 if (*p0 == '\n') *--p1 = '\r';
5963 }
5964 produced += num_eol;
5965 produced_char += num_eol;
5966 }
5967 /* Suppress eol-format conversion in the further conversion. */
5968 coding->eol_type = CODING_EOL_LF;
5969
5970 /* Set the coding system symbol to that for Unix-like EOL. */
5971 eol_type = Fget (saved_coding_symbol, Qeol_type);
5972 if (VECTORP (eol_type)
5973 && XVECTOR (eol_type)->size == 3
5974 && SYMBOLP (XVECTOR (eol_type)->contents[CODING_EOL_LF]))
5975 coding->symbol = XVECTOR (eol_type)->contents[CODING_EOL_LF];
5976 else
5977 coding->symbol = saved_coding_symbol;
5978
5979
5980 } 5971 }
5981 } 5972 else
5982
5983 coding->consumed = consumed;
5984 coding->consumed_char = consumed_char;
5985 coding->produced = produced;
5986 coding->produced_char = produced_char;
5987
5988 if (coding->dst_multibyte)
5989 newstr = make_uninit_multibyte_string (produced_char + shrinked_bytes,
5990 produced + shrinked_bytes);
5991 else
5992 newstr = make_uninit_string (produced + shrinked_bytes);
5993 if (from > 0)
5994 bcopy (XSTRING (str)->data, XSTRING (newstr)->data, from);
5995 bcopy (buf.data, XSTRING (newstr)->data + from, produced);
5996 if (shrinked_bytes > from)
5997 bcopy (XSTRING (str)->data + to_byte,
5998 XSTRING (newstr)->data + from + produced,
5999 shrinked_bytes - from);
6000 free_conversion_buffer (&buf);
6001
6002 if (coding->cmp_data && coding->cmp_data->used)
6003 coding_restore_composition (coding, newstr);
6004 coding_free_composition_data (coding);
6005
6006 if (SYMBOLP (coding->post_read_conversion)
6007 && !NILP (Ffboundp (coding->post_read_conversion)))
6008 newstr = run_pre_post_conversion_on_str (newstr, coding, 0);
6009
6010 return newstr;
6011}
6012
6013Lisp_Object
6014encode_coding_string (str, coding, nocopy)
6015 Lisp_Object str;
6016 struct coding_system *coding;
6017 int nocopy;
6018{
6019 int len;
6020 struct conversion_buffer buf;
6021 int from, to, to_byte;
6022 int result;
6023 int shrinked_bytes = 0;
6024 Lisp_Object newstr;
6025 int consumed, consumed_char, produced, produced_char;
6026
6027 if (SYMBOLP (coding->pre_write_conversion)
6028 && !NILP (Ffboundp (coding->pre_write_conversion)))
6029 str = run_pre_post_conversion_on_str (str, coding, 1);
6030
6031 from = 0;
6032 to = XSTRING (str)->size;
6033 to_byte = STRING_BYTES (XSTRING (str));
6034
6035 /* Encoding routines determine the multibyteness of the source text
6036 by coding->src_multibyte. */
6037 coding->src_multibyte = STRING_MULTIBYTE (str);
6038 coding->dst_multibyte = 0;
6039 if (! CODING_REQUIRE_ENCODING (coding))
6040 {
6041 coding->consumed = STRING_BYTES (XSTRING (str));
6042 coding->consumed_char = XSTRING (str)->size;
6043 if (STRING_MULTIBYTE (str))
6044 { 5973 {
6045 str = Fstring_as_unibyte (str); 5974 coding->src_pos = from;
6046 nocopy = 1; 5975 coding->src_pos_byte = from_byte;
6047 } 5976 }
6048 coding->produced = STRING_BYTES (XSTRING (str));
6049 coding->produced_char = XSTRING (str)->size;
6050 return (nocopy ? str : Fcopy_sequence (str));
6051 } 5977 }
6052 5978
6053 if (coding->composing != COMPOSITION_DISABLED) 5979 if (BUFFERP (dst_object))
6054 coding_save_composition (coding, from, to, str); 5980 {
6055 5981 coding->dst_object = dst_object;
6056 /* Try to skip the heading and tailing ASCIIs. */ 5982 coding->dst_pos = BUF_PT (XBUFFER (dst_object));
6057 if (coding->type != coding_type_ccl) 5983 coding->dst_pos_byte = BUF_PT_BYTE (XBUFFER (dst_object));
5984 coding->dst_multibyte
5985 = ! NILP (XBUFFER (dst_object)->enable_multibyte_characters);
5986 }
5987 else if (EQ (dst_object, Qt))
5988 {
5989 coding->dst_object = Qnil;
5990 coding->destination = (unsigned char *) xmalloc (coding->src_chars);
5991 coding->dst_bytes = coding->src_chars;
5992 coding->dst_multibyte = 0;
5993 }
5994 else
6058 { 5995 {
6059 SHRINK_CONVERSION_REGION (&from, &to_byte, coding, XSTRING (str)->data, 5996 coding->dst_object = Qnil;
6060 1); 5997 coding->dst_multibyte = 0;
6061 if (from == to_byte)
6062 return (nocopy ? str : Fcopy_sequence (str));
6063 shrinked_bytes = from + (STRING_BYTES (XSTRING (str)) - to_byte);
6064 } 5998 }
6065 5999
6066 len = encoding_buffer_size (coding, to_byte - from); 6000 encode_coding (coding);
6067 allocate_conversion_buffer (buf, len);
6068 6001
6069 consumed = consumed_char = produced = produced_char = 0; 6002 if (EQ (dst_object, Qt))
6070 while (1)
6071 { 6003 {
6072 result = encode_coding (coding, XSTRING (str)->data + from + consumed, 6004 if (BUFFERP (coding->dst_object))
6073 buf.data + produced, to_byte - from - consumed, 6005 coding->dst_object = Fbuffer_string ();
6074 buf.size - produced); 6006 else
6075 consumed += coding->consumed; 6007 {
6076 consumed_char += coding->consumed_char; 6008 coding->dst_object
6077 produced += coding->produced; 6009 = make_unibyte_string ((char *) coding->destination,
6078 produced_char += coding->produced_char; 6010 coding->produced);
6079 if (result == CODING_FINISH_NORMAL 6011 xfree (coding->destination);
6080 || (result == CODING_FINISH_INSUFFICIENT_SRC 6012 }
6081 && coding->consumed == 0))
6082 break;
6083 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6084 extend_conversion_buffer (&buf);
6085 } 6013 }
6086 6014
6087 coding->consumed = consumed; 6015 unbind_to (count, Qnil);
6088 coding->consumed_char = consumed_char; 6016}
6089 coding->produced = produced;
6090 coding->produced_char = produced_char;
6091 6017
6092 newstr = make_uninit_string (produced + shrinked_bytes);
6093 if (from > 0)
6094 bcopy (XSTRING (str)->data, XSTRING (newstr)->data, from);
6095 bcopy (buf.data, XSTRING (newstr)->data + from, produced);
6096 if (shrinked_bytes > from)
6097 bcopy (XSTRING (str)->data + to_byte,
6098 XSTRING (newstr)->data + from + produced,
6099 shrinked_bytes - from);
6100 6018
6101 free_conversion_buffer (&buf); 6019Lisp_Object
6102 coding_free_composition_data (coding); 6020preferred_coding_system ()
6021{
6022 int id = coding_categories[coding_priorities[0]].id;
6103 6023
6104 return newstr; 6024 return CODING_ID_NAME (id);
6105} 6025}
6106 6026
6107 6027
@@ -6110,19 +6030,12 @@ encode_coding_string (str, coding, nocopy)
6110 6030
6111DEFUN ("coding-system-p", Fcoding_system_p, Scoding_system_p, 1, 1, 0, 6031DEFUN ("coding-system-p", Fcoding_system_p, Scoding_system_p, 1, 1, 0,
6112 doc: /* Return t if OBJECT is nil or a coding-system. 6032 doc: /* Return t if OBJECT is nil or a coding-system.
6113See the documentation of `make-coding-system' for information 6033See the documentation of `define-coding-system' for information
6114about coding-system objects. */) 6034about coding-system objects. */)
6115 (obj) 6035 (obj)
6116 Lisp_Object obj; 6036 Lisp_Object obj;
6117{ 6037{
6118 if (NILP (obj)) 6038 return ((NILP (obj) || CODING_SYSTEM_P (obj)) ? Qt : Qnil);
6119 return Qt;
6120 if (!SYMBOLP (obj))
6121 return Qnil;
6122 /* Get coding-spec vector for OBJ. */
6123 obj = Fget (obj, Qcoding_system);
6124 return ((VECTORP (obj) && XVECTOR (obj)->size == 5)
6125 ? Qt : Qnil);
6126} 6039}
6127 6040
6128DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system, 6041DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system,
@@ -6162,7 +6075,7 @@ DEFUN ("check-coding-system", Fcheck_coding_system, Scheck_coding_system,
6162If valid, return CODING-SYSTEM, else signal a `coding-system-error' error. 6075If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
6163It is valid if it is a symbol with a non-nil `coding-system' property. 6076It is valid if it is a symbol with a non-nil `coding-system' property.
6164The value of property should be a vector of length 5. */) 6077The value of property should be a vector of length 5. */)
6165 (coding_system) 6078 (coding_system)
6166 Lisp_Object coding_system; 6079 Lisp_Object coding_system;
6167{ 6080{
6168 CHECK_SYMBOL (coding_system); 6081 CHECK_SYMBOL (coding_system);
@@ -6171,70 +6084,142 @@ The value of property should be a vector of length 5. */)
6171 while (1) 6084 while (1)
6172 Fsignal (Qcoding_system_error, Fcons (coding_system, Qnil)); 6085 Fsignal (Qcoding_system_error, Fcons (coding_system, Qnil));
6173} 6086}
6087
6174 6088
6175Lisp_Object 6089Lisp_Object
6176detect_coding_system (src, src_bytes, highest, multibytep) 6090detect_coding_system (src, src_bytes, highest, multibytep, coding_system)
6177 unsigned char *src; 6091 unsigned char *src;
6178 int src_bytes, highest; 6092 int src_bytes, highest;
6179 int multibytep; 6093 int multibytep;
6094 Lisp_Object coding_system;
6180{ 6095{
6181 int coding_mask, eol_type; 6096 unsigned char *src_end = src + src_bytes;
6182 Lisp_Object val, tmp; 6097 int mask = CATEGORY_MASK_ANY;
6183 int dummy; 6098 int detected = 0;
6099 int c, i;
6100 Lisp_Object attrs, eol_type;
6101 Lisp_Object val;
6102 struct coding_system coding;
6103
6104 if (NILP (coding_system))
6105 coding_system = Qundecided;
6106 setup_coding_system (coding_system, &coding);
6107 attrs = CODING_ID_ATTRS (coding.id);
6108 eol_type = CODING_ID_EOL_TYPE (coding.id);
6184 6109
6185 coding_mask = detect_coding_mask (src, src_bytes, NULL, &dummy, multibytep); 6110 coding.source = src;
6186 eol_type = detect_eol_type (src, src_bytes, &dummy); 6111 coding.src_bytes = src_bytes;
6187 if (eol_type == CODING_EOL_INCONSISTENT) 6112 coding.src_multibyte = multibytep;
6188 eol_type = CODING_EOL_UNDECIDED; 6113 coding.consumed = 0;
6189 6114
6190 if (!coding_mask) 6115 if (XINT (CODING_ATTR_CATEGORY (attrs)) != coding_category_undecided)
6191 { 6116 {
6192 val = Qundecided; 6117 mask = 1 << XINT (CODING_ATTR_CATEGORY (attrs));
6193 if (eol_type != CODING_EOL_UNDECIDED)
6194 {
6195 Lisp_Object val2;
6196 val2 = Fget (Qundecided, Qeol_type);
6197 if (VECTORP (val2))
6198 val = XVECTOR (val2)->contents[eol_type];
6199 }
6200 return (highest ? val : Fcons (val, Qnil));
6201 } 6118 }
6202 6119 else
6203 /* At first, gather possible coding systems in VAL. */
6204 val = Qnil;
6205 for (tmp = Vcoding_category_list; CONSP (tmp); tmp = XCDR (tmp))
6206 { 6120 {
6207 Lisp_Object category_val, category_index; 6121 coding_system = Qnil;
6208 6122 for (; src < src_end; src++)
6209 category_index = Fget (XCAR (tmp), Qcoding_category_index);
6210 category_val = Fsymbol_value (XCAR (tmp));
6211 if (!NILP (category_val)
6212 && NATNUMP (category_index)
6213 && (coding_mask & (1 << XFASTINT (category_index))))
6214 { 6123 {
6215 val = Fcons (category_val, val); 6124 c = *src;
6216 if (highest) 6125 if (c & 0x80 || (c < 0x20 && (c == ISO_CODE_ESC
6126 || c == ISO_CODE_SI
6127 || c == ISO_CODE_SO)))
6217 break; 6128 break;
6218 } 6129 }
6130 coding.head_ascii = src - coding.source;
6131
6132 if (src < src_end)
6133 for (i = 0; i < coding_category_raw_text; i++)
6134 {
6135 enum coding_category category = coding_priorities[i];
6136 struct coding_system *this = coding_categories + category;
6137
6138 if (category >= coding_category_raw_text
6139 || detected & (1 << category))
6140 continue;
6141
6142 if (this->id < 0)
6143 {
6144 /* No coding system of this category is defined. */
6145 mask &= ~(1 << category);
6146 }
6147 else
6148 {
6149 detected |= detected_mask[category];
6150 if ((*(coding_categories[category].detector)) (&coding, &mask)
6151 && highest)
6152 {
6153 mask &= detected_mask[category];
6154 break;
6155 }
6156 }
6157 }
6219 } 6158 }
6220 if (!highest)
6221 val = Fnreverse (val);
6222 6159
6223 /* Then, replace the elements with subsidiary coding systems. */ 6160 if (!mask)
6224 for (tmp = val; CONSP (tmp); tmp = XCDR (tmp)) 6161 val = Fcons (make_number (coding_category_raw_text), Qnil);
6162 else if (mask == CATEGORY_MASK_ANY)
6163 val = Fcons (make_number (coding_category_undecided), Qnil);
6164 else if (highest)
6225 { 6165 {
6226 if (eol_type != CODING_EOL_UNDECIDED 6166 for (i = 0; i < coding_category_raw_text; i++)
6227 && eol_type != CODING_EOL_INCONSISTENT) 6167 if (mask & (1 << coding_priorities[i]))
6228 { 6168 {
6229 Lisp_Object eol; 6169 val = Fcons (make_number (coding_priorities[i]), Qnil);
6230 eol = Fget (XCAR (tmp), Qeol_type); 6170 break;
6231 if (VECTORP (eol)) 6171 }
6232 XSETCAR (tmp, XVECTOR (eol)->contents[eol_type]); 6172 }
6233 } 6173 else
6174 {
6175 val = Qnil;
6176 for (i = coding_category_raw_text - 1; i >= 0; i--)
6177 if (mask & (1 << coding_priorities[i]))
6178 val = Fcons (make_number (coding_priorities[i]), val);
6234 } 6179 }
6180
6181 {
6182 int one_byte_eol = -1, two_byte_eol = -1;
6183 Lisp_Object tail;
6184
6185 for (tail = val; CONSP (tail); tail = XCDR (tail))
6186 {
6187 struct coding_system *this
6188 = (NILP (coding_system) ? coding_categories + XINT (XCAR (tail))
6189 : &coding);
6190 int this_eol;
6191
6192 attrs = CODING_ID_ATTRS (this->id);
6193 eol_type = CODING_ID_EOL_TYPE (this->id);
6194 XSETCAR (tail, CODING_ID_NAME (this->id));
6195 if (VECTORP (eol_type))
6196 {
6197 if (EQ (CODING_ATTR_TYPE (attrs), Qutf_16))
6198 {
6199 if (two_byte_eol < 0)
6200 two_byte_eol = detect_eol (this, coding.source, src_bytes);
6201 this_eol = two_byte_eol;
6202 }
6203 else
6204 {
6205 if (one_byte_eol < 0)
6206 one_byte_eol =detect_eol (this, coding.source, src_bytes);
6207 this_eol = one_byte_eol;
6208 }
6209 if (this_eol == EOL_SEEN_LF)
6210 XSETCAR (tail, AREF (eol_type, 0));
6211 else if (this_eol == EOL_SEEN_CRLF)
6212 XSETCAR (tail, AREF (eol_type, 1));
6213 else if (this_eol == EOL_SEEN_CR)
6214 XSETCAR (tail, AREF (eol_type, 2));
6215 }
6216 }
6217 }
6218
6235 return (highest ? XCAR (val) : val); 6219 return (highest ? XCAR (val) : val);
6236} 6220}
6237 6221
6222
6238DEFUN ("detect-coding-region", Fdetect_coding_region, Sdetect_coding_region, 6223DEFUN ("detect-coding-region", Fdetect_coding_region, Sdetect_coding_region,
6239 2, 3, 0, 6224 2, 3, 0,
6240 doc: /* Detect coding system of the text in the region between START and END. 6225 doc: /* Detect coding system of the text in the region between START and END.
@@ -6251,7 +6236,6 @@ highest priority. */)
6251{ 6236{
6252 int from, to; 6237 int from, to;
6253 int from_byte, to_byte; 6238 int from_byte, to_byte;
6254 int include_anchor_byte = 0;
6255 6239
6256 CHECK_NUMBER_COERCE_MARKER (start); 6240 CHECK_NUMBER_COERCE_MARKER (start);
6257 CHECK_NUMBER_COERCE_MARKER (end); 6241 CHECK_NUMBER_COERCE_MARKER (end);
@@ -6263,20 +6247,13 @@ highest priority. */)
6263 6247
6264 if (from < GPT && to >= GPT) 6248 if (from < GPT && to >= GPT)
6265 move_gap_both (to, to_byte); 6249 move_gap_both (to, to_byte);
6266 /* If we an anchor byte `\0' follows the region, we include it in
6267 the detecting source. Then code detectors can handle the tailing
6268 byte sequence more accurately.
6269 6250
6270 Fix me: This is not an perfect solution. It is better that we
6271 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6272 */
6273 if (to == Z || (to == GPT && GAP_SIZE > 0))
6274 include_anchor_byte = 1;
6275 return detect_coding_system (BYTE_POS_ADDR (from_byte), 6251 return detect_coding_system (BYTE_POS_ADDR (from_byte),
6276 to_byte - from_byte + include_anchor_byte, 6252 to_byte - from_byte,
6277 !NILP (highest), 6253 !NILP (highest),
6278 !NILP (current_buffer 6254 !NILP (current_buffer
6279 ->enable_multibyte_characters)); 6255 ->enable_multibyte_characters),
6256 Qnil);
6280} 6257}
6281 6258
6282DEFUN ("detect-coding-string", Fdetect_coding_string, Sdetect_coding_string, 6259DEFUN ("detect-coding-string", Fdetect_coding_string, Sdetect_coding_string,
@@ -6296,285 +6273,414 @@ highest priority. */)
6296 CHECK_STRING (string); 6273 CHECK_STRING (string);
6297 6274
6298 return detect_coding_system (XSTRING (string)->data, 6275 return detect_coding_system (XSTRING (string)->data,
6299 /* "+ 1" is to include the anchor byte 6276 STRING_BYTES (XSTRING (string)),
6300 `\0'. With this, code detectors can
6301 handle the tailing bytes more
6302 accurately. */
6303 STRING_BYTES (XSTRING (string)) + 1,
6304 !NILP (highest), 6277 !NILP (highest),
6305 STRING_MULTIBYTE (string)); 6278 STRING_MULTIBYTE (string),
6279 Qnil);
6306} 6280}
6307 6281
6308/* Return an intersection of lists L1 and L2. */
6309 6282
6310static Lisp_Object 6283static INLINE int
6311intersection (l1, l2) 6284char_encodable_p (c, attrs)
6312 Lisp_Object l1, l2; 6285 int c;
6286 Lisp_Object attrs;
6313{ 6287{
6314 Lisp_Object val; 6288 Lisp_Object tail;
6289 int id;
6290 struct charset *charset;
6315 6291
6316 for (val = Qnil; CONSP (l1); l1 = XCDR (l1)) 6292 for (tail = CODING_ATTR_CHARSET_LIST (attrs);
6293 CONSP (tail); tail = XCDR (tail))
6317 { 6294 {
6318 if (!NILP (Fmemq (XCAR (l1), l2))) 6295 charset = CHARSET_FROM_ID (XINT (XCAR (tail)));
6319 val = Fcons (XCAR (l1), val); 6296 if (CHAR_CHARSET_P (c, charset))
6297 break;
6320 } 6298 }
6321 return val; 6299 return (! NILP (tail));
6322} 6300}
6323 6301
6324 6302
6325/* Subroutine for Fsafe_coding_systems_region_internal. 6303/* Return a list of coding systems that safely encode the text between
6304 START and END. If EXCLUDE is non-nil, it is a list of coding
6305 systems not to check. The returned list doesn't contain any such
6306 coding systems. In any case, If the text contains only ASCII or is
6307 unibyte, return t. */
6308
6309DEFUN ("find-coding-systems-region-internal",
6310 Ffind_coding_systems_region_internal,
6311 Sfind_coding_systems_region_internal, 2, 3, 0,
6312 doc: /* Internal use only. */)
6313 (start, end, exclude)
6314 Lisp_Object start, end, exclude;
6315{
6316 Lisp_Object coding_attrs_list, safe_codings;
6317 EMACS_INT start_byte, end_byte;
6318 unsigned char *p, *pbeg, *pend;
6319 int c;
6320 Lisp_Object tail, elt;
6321
6322 if (STRINGP (start))
6323 {
6324 if (!STRING_MULTIBYTE (start)
6325 && XSTRING (start)->size != STRING_BYTES (XSTRING (start)))
6326 return Qt;
6327 start_byte = 0;
6328 end_byte = STRING_BYTES (XSTRING (start));
6329 }
6330 else
6331 {
6332 CHECK_NUMBER_COERCE_MARKER (start);
6333 CHECK_NUMBER_COERCE_MARKER (end);
6334 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
6335 args_out_of_range (start, end);
6336 if (NILP (current_buffer->enable_multibyte_characters))
6337 return Qt;
6338 start_byte = CHAR_TO_BYTE (XINT (start));
6339 end_byte = CHAR_TO_BYTE (XINT (end));
6340 if (XINT (end) - XINT (start) == end_byte - start_byte)
6341 return Qt;
6342
6343 if (start < GPT && end > GPT)
6344 {
6345 if ((GPT - start) < (end - GPT))
6346 move_gap_both (start, start_byte);
6347 else
6348 move_gap_both (end, end_byte);
6349 }
6350 }
6326 6351
6327 Return a list of coding systems that safely encode the multibyte 6352 coding_attrs_list = Qnil;
6328 text between P and PEND. SAFE_CODINGS, if non-nil, is a list of 6353 for (tail = Vcoding_system_list; CONSP (tail); tail = XCDR (tail))
6329 possible coding systems. If it is nil, it means that we have not 6354 if (NILP (exclude)
6330 yet found any coding systems. 6355 || NILP (Fmemq (XCAR (tail), exclude)))
6356 {
6357 Lisp_Object attrs;
6331 6358
6332 WORK_TABLE is a copy of the char-table Vchar_coding_system_table. An 6359 attrs = AREF (CODING_SYSTEM_SPEC (XCAR (tail)), 0);
6333 element of WORK_TABLE is set to t once the element is looked up. 6360 if (EQ (XCAR (tail), CODING_ATTR_BASE_NAME (attrs))
6361 && ! EQ (CODING_ATTR_TYPE (attrs), Qundecided))
6362 coding_attrs_list = Fcons (attrs, coding_attrs_list);
6363 }
6334 6364
6335 If a non-ASCII single byte char is found, set 6365 if (STRINGP (start))
6336 *single_byte_char_found to 1. */ 6366 p = pbeg = XSTRING (start)->data;
6367 else
6368 p = pbeg = BYTE_POS_ADDR (start_byte);
6369 pend = p + (end_byte - start_byte);
6337 6370
6338static Lisp_Object 6371 while (p < pend && ASCII_BYTE_P (*p)) p++;
6339find_safe_codings (p, pend, safe_codings, work_table, single_byte_char_found) 6372 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
6340 unsigned char *p, *pend;
6341 Lisp_Object safe_codings, work_table;
6342 int *single_byte_char_found;
6343{
6344 int c, len, idx;
6345 Lisp_Object val;
6346 6373
6347 while (p < pend) 6374 while (p < pend)
6348 { 6375 {
6349 c = STRING_CHAR_AND_LENGTH (p, pend - p, len); 6376 if (ASCII_BYTE_P (*p))
6350 p += len; 6377 p++;
6351 if (ASCII_BYTE_P (c)) 6378 else
6352 /* We can ignore ASCII characters here. */ 6379 {
6353 continue; 6380 c = STRING_CHAR_ADVANCE (p);
6354 if (SINGLE_BYTE_CHAR_P (c)) 6381
6355 *single_byte_char_found = 1; 6382 charset_map_loaded = 0;
6356 if (NILP (safe_codings)) 6383 for (tail = coding_attrs_list; CONSP (tail);)
6357 continue; 6384 {
6358 /* Check the safe coding systems for C. */ 6385 elt = XCAR (tail);
6359 val = char_table_ref_and_index (work_table, c, &idx); 6386 if (NILP (elt))
6360 if (EQ (val, Qt)) 6387 tail = XCDR (tail);
6361 /* This element was already checked. Ignore it. */ 6388 else if (char_encodable_p (c, elt))
6362 continue; 6389 tail = XCDR (tail);
6363 /* Remember that we checked this element. */ 6390 else if (CONSP (XCDR (tail)))
6364 CHAR_TABLE_SET (work_table, make_number (idx), Qt); 6391 {
6392 XSETCAR (tail, XCAR (XCDR (tail)));
6393 XSETCDR (tail, XCDR (XCDR (tail)));
6394 }
6395 else
6396 {
6397 XSETCAR (tail, Qnil);
6398 tail = XCDR (tail);
6399 }
6400 }
6401 if (charset_map_loaded)
6402 {
6403 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
6365 6404
6366 /* If there are some safe coding systems for C and we have 6405 if (STRINGP (start))
6367 already found the other set of coding systems for the 6406 pbeg = XSTRING (start)->data;
6368 different characters, get the intersection of them. */ 6407 else
6369 if (!EQ (safe_codings, Qt) && !NILP (val)) 6408 pbeg = BYTE_POS_ADDR (start_byte);
6370 val = intersection (safe_codings, val); 6409 p = pbeg + p_offset;
6371 safe_codings = val; 6410 pend = pbeg + pend_offset;
6411 }
6412 }
6372 } 6413 }
6414
6415 safe_codings = Qnil;
6416 for (tail = coding_attrs_list; CONSP (tail); tail = XCDR (tail))
6417 if (! NILP (XCAR (tail)))
6418 safe_codings = Fcons (CODING_ATTR_BASE_NAME (XCAR (tail)), safe_codings);
6419
6373 return safe_codings; 6420 return safe_codings;
6374} 6421}
6375 6422
6376 6423
6377/* Return a list of coding systems that safely encode the text between 6424DEFUN ("check-coding-systems-region", Fcheck_coding_systems_region,
6378 START and END. If the text contains only ASCII or is unibyte, 6425 Scheck_coding_systems_region, 3, 3, 0,
6379 return t. */ 6426 doc: /* Check if the region is encodable by coding systems.
6380 6427
6381DEFUN ("find-coding-systems-region-internal", 6428START and END are buffer positions specifying the region.
6382 Ffind_coding_systems_region_internal, 6429CODING-SYSTEM-LIST is a list of coding systems to check.
6383 Sfind_coding_systems_region_internal, 2, 2, 0, 6430
6384 doc: /* Internal use only. */) 6431The value is an alist ((CODING-SYSTEM POS0 POS1 ...) ...), where
6385 (start, end) 6432CODING-SYSTEM is a member of CODING-SYSTEM-LIst and can't encode the
6386 Lisp_Object start, end; 6433whole region, POS0, POS1, ... are buffer positions where non-encodable
6434characters are found.
6435
6436If all coding systems in CODING-SYSTEM-LIST can encode the region, the
6437value is nil.
6438
6439START may be a string. In that case, check if the string is
6440encodable, and the value contains indices to the string instead of
6441buffer positions. END is ignored. */)
6442 (start, end, coding_system_list)
6443 Lisp_Object start, end, coding_system_list;
6387{ 6444{
6388 Lisp_Object work_table, safe_codings; 6445 Lisp_Object list;
6389 int non_ascii_p = 0; 6446 EMACS_INT start_byte, end_byte;
6390 int single_byte_char_found = 0; 6447 int pos;
6391 unsigned char *p1, *p1end, *p2, *p2end, *p; 6448 unsigned char *p, *pbeg, *pend;
6449 int c;
6450 Lisp_Object tail, elt;
6392 6451
6393 if (STRINGP (start)) 6452 if (STRINGP (start))
6394 { 6453 {
6395 if (!STRING_MULTIBYTE (start)) 6454 if (!STRING_MULTIBYTE (start)
6396 return Qt; 6455 && XSTRING (start)->size != STRING_BYTES (XSTRING (start)))
6397 p1 = XSTRING (start)->data, p1end = p1 + STRING_BYTES (XSTRING (start)); 6456 return Qnil;
6398 p2 = p2end = p1end; 6457 start_byte = 0;
6399 if (XSTRING (start)->size != STRING_BYTES (XSTRING (start))) 6458 end_byte = STRING_BYTES (XSTRING (start));
6400 non_ascii_p = 1; 6459 pos = 0;
6401 } 6460 }
6402 else 6461 else
6403 { 6462 {
6404 int from, to, stop;
6405
6406 CHECK_NUMBER_COERCE_MARKER (start); 6463 CHECK_NUMBER_COERCE_MARKER (start);
6407 CHECK_NUMBER_COERCE_MARKER (end); 6464 CHECK_NUMBER_COERCE_MARKER (end);
6408 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end)) 6465 if (XINT (start) < BEG || XINT (end) > Z || XINT (start) > XINT (end))
6409 args_out_of_range (start, end); 6466 args_out_of_range (start, end);
6410 if (NILP (current_buffer->enable_multibyte_characters)) 6467 if (NILP (current_buffer->enable_multibyte_characters))
6468 return Qnil;
6469 start_byte = CHAR_TO_BYTE (XINT (start));
6470 end_byte = CHAR_TO_BYTE (XINT (end));
6471 if (XINT (end) - XINT (start) == end_byte - start_byte)
6411 return Qt; 6472 return Qt;
6412 from = CHAR_TO_BYTE (XINT (start)); 6473
6413 to = CHAR_TO_BYTE (XINT (end)); 6474 if (start < GPT && end > GPT)
6414 stop = from < GPT_BYTE && GPT_BYTE < to ? GPT_BYTE : to; 6475 {
6415 p1 = BYTE_POS_ADDR (from), p1end = p1 + (stop - from); 6476 if ((GPT - start) < (end - GPT))
6416 if (stop == to) 6477 move_gap_both (start, start_byte);
6417 p2 = p2end = p1end; 6478 else
6418 else 6479 move_gap_both (end, end_byte);
6419 p2 = BYTE_POS_ADDR (stop), p2end = p2 + (to - stop); 6480 }
6420 if (XINT (end) - XINT (start) != to - from) 6481 pos = start;
6421 non_ascii_p = 1; 6482 }
6483
6484 list = Qnil;
6485 for (tail = coding_system_list; CONSP (tail); tail = XCDR (tail))
6486 {
6487 elt = XCAR (tail);
6488 list = Fcons (Fcons (elt, Fcons (AREF (CODING_SYSTEM_SPEC (elt), 0),
6489 Qnil)),
6490 list);
6422 } 6491 }
6423 6492
6424 if (!non_ascii_p) 6493 if (STRINGP (start))
6494 p = pbeg = XSTRING (start)->data;
6495 else
6496 p = pbeg = BYTE_POS_ADDR (start_byte);
6497 pend = p + (end_byte - start_byte);
6498
6499 while (p < pend && ASCII_BYTE_P (*p)) p++, pos++;
6500 while (p < pend && ASCII_BYTE_P (*(pend - 1))) pend--;
6501
6502 while (p < pend)
6425 { 6503 {
6426 /* We are sure that the text contains no multibyte character. 6504 if (ASCII_BYTE_P (*p))
6427 Check if it contains eight-bit-graphic. */ 6505 p++;
6428 p = p1; 6506 else
6429 for (p = p1; p < p1end && ASCII_BYTE_P (*p); p++);
6430 if (p == p1end)
6431 { 6507 {
6432 for (p = p2; p < p2end && ASCII_BYTE_P (*p); p++); 6508 c = STRING_CHAR_ADVANCE (p);
6433 if (p == p2end) 6509
6434 return Qt; 6510 charset_map_loaded = 0;
6511 for (tail = list; CONSP (tail); tail = XCDR (tail))
6512 {
6513 elt = XCDR (XCAR (tail));
6514 if (! char_encodable_p (c, XCAR (elt)))
6515 XSETCDR (elt, Fcons (make_number (pos), XCDR (elt)));
6516 }
6517 if (charset_map_loaded)
6518 {
6519 EMACS_INT p_offset = p - pbeg, pend_offset = pend - pbeg;
6520
6521 if (STRINGP (start))
6522 pbeg = XSTRING (start)->data;
6523 else
6524 pbeg = BYTE_POS_ADDR (start_byte);
6525 p = pbeg + p_offset;
6526 pend = pbeg + pend_offset;
6527 }
6435 } 6528 }
6529 pos++;
6436 } 6530 }
6437 6531
6438 /* The text contains non-ASCII characters. */ 6532 tail = list;
6439 work_table = Fcopy_sequence (Vchar_coding_system_table); 6533 list = Qnil;
6440 safe_codings = find_safe_codings (p1, p1end, Qt, work_table, 6534 for (; CONSP (tail); tail = XCDR (tail))
6441 &single_byte_char_found);
6442 if (p2 < p2end)
6443 safe_codings = find_safe_codings (p2, p2end, safe_codings, work_table,
6444 &single_byte_char_found);
6445
6446 if (EQ (safe_codings, Qt))
6447 ; /* Nothing to be done. */
6448 else if (!single_byte_char_found)
6449 { 6535 {
6450 /* Append generic coding systems. */ 6536 elt = XCAR (tail);
6451 Lisp_Object args[2]; 6537 if (CONSP (XCDR (XCDR (elt))))
6452 args[0] = safe_codings; 6538 list = Fcons (Fcons (XCAR (elt), Fnreverse (XCDR (XCDR (elt)))),
6453 args[1] = Fchar_table_extra_slot (Vchar_coding_system_table, 6539 list);
6454 make_number (0));
6455 safe_codings = Fappend (2, args);
6456 } 6540 }
6457 else 6541
6458 safe_codings = Fcons (Qraw_text, 6542 return list;
6459 Fcons (Qemacs_mule,
6460 Fcons (Qno_conversion, safe_codings)));
6461 return safe_codings;
6462} 6543}
6463 6544
6464 6545
6546
6465Lisp_Object 6547Lisp_Object
6466code_convert_region1 (start, end, coding_system, encodep) 6548code_convert_region (start, end, coding_system, dst_object, encodep, norecord)
6467 Lisp_Object start, end, coding_system; 6549 Lisp_Object start, end, coding_system, dst_object;
6468 int encodep; 6550 int encodep, norecord;
6469{ 6551{
6470 struct coding_system coding; 6552 struct coding_system coding;
6471 int from, to; 6553 EMACS_INT from, from_byte, to, to_byte;
6554 Lisp_Object src_object;
6472 6555
6473 CHECK_NUMBER_COERCE_MARKER (start); 6556 CHECK_NUMBER_COERCE_MARKER (start);
6474 CHECK_NUMBER_COERCE_MARKER (end); 6557 CHECK_NUMBER_COERCE_MARKER (end);
6475 CHECK_SYMBOL (coding_system); 6558 if (NILP (coding_system))
6559 coding_system = Qno_conversion;
6560 else
6561 CHECK_CODING_SYSTEM (coding_system);
6562 src_object = Fcurrent_buffer ();
6563 if (NILP (dst_object))
6564 dst_object = src_object;
6565 else if (! EQ (dst_object, Qt))
6566 CHECK_BUFFER (dst_object);
6476 6567
6477 validate_region (&start, &end); 6568 validate_region (&start, &end);
6478 from = XFASTINT (start); 6569 from = XFASTINT (start);
6570 from_byte = CHAR_TO_BYTE (from);
6479 to = XFASTINT (end); 6571 to = XFASTINT (end);
6572 to_byte = CHAR_TO_BYTE (to);
6480 6573
6481 if (NILP (coding_system)) 6574 setup_coding_system (coding_system, &coding);
6482 return make_number (to - from); 6575 coding.mode |= CODING_MODE_LAST_BLOCK;
6483 6576
6484 if (setup_coding_system (Fcheck_coding_system (coding_system), &coding) < 0) 6577 if (encodep)
6485 error ("Invalid coding system: %s", XSYMBOL (coding_system)->name->data); 6578 encode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
6579 dst_object);
6580 else
6581 decode_coding_object (&coding, src_object, from, from_byte, to, to_byte,
6582 dst_object);
6583 if (! norecord)
6584 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
6486 6585
6487 coding.mode |= CODING_MODE_LAST_BLOCK; 6586 if (coding.result != CODING_RESULT_SUCCESS)
6488 coding.src_multibyte = coding.dst_multibyte 6587 error ("Code conversion error: %d", coding.result);
6489 = !NILP (current_buffer->enable_multibyte_characters); 6588
6490 code_convert_region (from, CHAR_TO_BYTE (from), to, CHAR_TO_BYTE (to), 6589 return (BUFFERP (dst_object)
6491 &coding, encodep, 1); 6590 ? make_number (coding.produced_char)
6492 Vlast_coding_system_used = coding.symbol; 6591 : coding.dst_object);
6493 return make_number (coding.produced_char);
6494} 6592}
6495 6593
6594
6496DEFUN ("decode-coding-region", Fdecode_coding_region, Sdecode_coding_region, 6595DEFUN ("decode-coding-region", Fdecode_coding_region, Sdecode_coding_region,
6497 3, 3, "r\nzCoding system: ", 6596 3, 4, "r\nzCoding system: ",
6498 doc: /* Decode the current region from the specified coding system. 6597 doc: /* Decode the current region from the specified coding system.
6499When called from a program, takes three arguments: 6598When called from a program, takes four arguments:
6500START, END, and CODING-SYSTEM. START and END are buffer positions. 6599 START, END, CODING-SYSTEM, and DESTINATION.
6600START and END are buffer positions.
6601
6602Optional 4th arguments DESTINATION specifies where the decoded text goes.
6603If nil, the region between START and END is replace by the decoded text.
6604If buffer, the decoded text is inserted in the buffer.
6605If t, the decoded text is returned.
6606
6501This function sets `last-coding-system-used' to the precise coding system 6607This function sets `last-coding-system-used' to the precise coding system
6502used (which may be different from CODING-SYSTEM if CODING-SYSTEM is 6608used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6503not fully specified.) 6609not fully specified.)
6504It returns the length of the decoded text. */) 6610It returns the length of the decoded text. */)
6505 (start, end, coding_system) 6611 (start, end, coding_system, destination)
6506 Lisp_Object start, end, coding_system; 6612 Lisp_Object start, end, coding_system, destination;
6507{ 6613{
6508 return code_convert_region1 (start, end, coding_system, 0); 6614 return code_convert_region (start, end, coding_system, destination, 0, 0);
6509} 6615}
6510 6616
6511DEFUN ("encode-coding-region", Fencode_coding_region, Sencode_coding_region, 6617DEFUN ("encode-coding-region", Fencode_coding_region, Sencode_coding_region,
6512 3, 3, "r\nzCoding system: ", 6618 3, 4, "r\nzCoding system: ",
6513 doc: /* Encode the current region into the specified coding system. 6619 doc: /* Encode the current region by specified coding system.
6514When called from a program, takes three arguments: 6620When called from a program, takes three arguments:
6515START, END, and CODING-SYSTEM. START and END are buffer positions. 6621START, END, and CODING-SYSTEM. START and END are buffer positions.
6622
6623Optional 4th arguments DESTINATION specifies where the encoded text goes.
6624If nil, the region between START and END is replace by the encoded text.
6625If buffer, the encoded text is inserted in the buffer.
6626If t, the encoded text is returned.
6627
6516This function sets `last-coding-system-used' to the precise coding system 6628This function sets `last-coding-system-used' to the precise coding system
6517used (which may be different from CODING-SYSTEM if CODING-SYSTEM is 6629used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6518not fully specified.) 6630not fully specified.)
6519It returns the length of the encoded text. */) 6631It returns the length of the encoded text. */)
6520 (start, end, coding_system) 6632 (start, end, coding_system, destination)
6521 Lisp_Object start, end, coding_system; 6633 Lisp_Object start, end, coding_system, destination;
6522{ 6634{
6523 return code_convert_region1 (start, end, coding_system, 1); 6635 return code_convert_region (start, end, coding_system, destination, 1, 0);
6524} 6636}
6525 6637
6526Lisp_Object 6638Lisp_Object
6527code_convert_string1 (string, coding_system, nocopy, encodep) 6639code_convert_string (string, coding_system, dst_object,
6528 Lisp_Object string, coding_system, nocopy; 6640 encodep, nocopy, norecord)
6529 int encodep; 6641 Lisp_Object string, coding_system, dst_object;
6642 int encodep, nocopy, norecord;
6530{ 6643{
6531 struct coding_system coding; 6644 struct coding_system coding;
6645 EMACS_INT chars, bytes;
6532 6646
6533 CHECK_STRING (string); 6647 CHECK_STRING (string);
6534 CHECK_SYMBOL (coding_system);
6535
6536 if (NILP (coding_system)) 6648 if (NILP (coding_system))
6537 return (NILP (nocopy) ? Fcopy_sequence (string) : string); 6649 {
6650 if (! norecord)
6651 Vlast_coding_system_used = Qno_conversion;
6652 if (NILP (dst_object))
6653 return (nocopy ? Fcopy_sequence (string) : string);
6654 }
6538 6655
6539 if (setup_coding_system (Fcheck_coding_system (coding_system), &coding) < 0) 6656 if (NILP (coding_system))
6540 error ("Invalid coding system: %s", XSYMBOL (coding_system)->name->data); 6657 coding_system = Qno_conversion;
6658 else
6659 CHECK_CODING_SYSTEM (coding_system);
6660 if (NILP (dst_object))
6661 dst_object = Qt;
6662 else if (! EQ (dst_object, Qt))
6663 CHECK_BUFFER (dst_object);
6541 6664
6665 setup_coding_system (coding_system, &coding);
6542 coding.mode |= CODING_MODE_LAST_BLOCK; 6666 coding.mode |= CODING_MODE_LAST_BLOCK;
6543 string = (encodep 6667 chars = XSTRING (string)->size;
6544 ? encode_coding_string (string, &coding, !NILP (nocopy)) 6668 bytes = STRING_BYTES (XSTRING (string));
6545 : decode_coding_string (string, &coding, !NILP (nocopy))); 6669 if (encodep)
6546 Vlast_coding_system_used = coding.symbol; 6670 encode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
6671 else
6672 decode_coding_object (&coding, string, 0, 0, chars, bytes, dst_object);
6673 if (! norecord)
6674 Vlast_coding_system_used = CODING_ID_NAME (coding.id);
6547 6675
6548 return string; 6676 if (coding.result != CODING_RESULT_SUCCESS)
6549} 6677 error ("Code conversion error: %d", coding.result);
6550 6678
6551DEFUN ("decode-coding-string", Fdecode_coding_string, Sdecode_coding_string, 6679 return (BUFFERP (dst_object)
6552 2, 3, 0, 6680 ? make_number (coding.produced_char)
6553 doc: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result. 6681 : coding.dst_object);
6554Optional arg NOCOPY non-nil means it is OK to return STRING itself
6555if the decoding operation is trivial.
6556This function sets `last-coding-system-used' to the precise coding system
6557used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6558not fully specified.) */)
6559 (string, coding_system, nocopy)
6560 Lisp_Object string, coding_system, nocopy;
6561{
6562 return code_convert_string1 (string, coding_system, nocopy, 0);
6563} 6682}
6564 6683
6565DEFUN ("encode-coding-string", Fencode_coding_string, Sencode_coding_string,
6566 2, 3, 0,
6567 doc: /* Encode STRING to CODING-SYSTEM, and return the result.
6568Optional arg NOCOPY non-nil means it is OK to return STRING itself
6569if the encoding operation is trivial.
6570This function sets `last-coding-system-used' to the precise coding system
6571used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6572not fully specified.) */)
6573 (string, coding_system, nocopy)
6574 Lisp_Object string, coding_system, nocopy;
6575{
6576 return code_convert_string1 (string, coding_system, nocopy, 1);
6577}
6578 6684
6579/* Encode or decode STRING according to CODING_SYSTEM. 6685/* Encode or decode STRING according to CODING_SYSTEM.
6580 Do not set Vlast_coding_system_used. 6686 Do not set Vlast_coding_system_used.
@@ -6587,23 +6693,52 @@ code_convert_string_norecord (string, coding_system, encodep)
6587 Lisp_Object string, coding_system; 6693 Lisp_Object string, coding_system;
6588 int encodep; 6694 int encodep;
6589{ 6695{
6590 struct coding_system coding; 6696 code_convert_string (string, coding_system, Qt, encodep, 0, 1);
6697}
6591 6698
6592 CHECK_STRING (string);
6593 CHECK_SYMBOL (coding_system);
6594 6699
6595 if (NILP (coding_system)) 6700DEFUN ("decode-coding-string", Fdecode_coding_string, Sdecode_coding_string,
6596 return string; 6701 2, 4, 0,
6702 doc: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
6597 6703
6598 if (setup_coding_system (Fcheck_coding_system (coding_system), &coding) < 0) 6704Optional third arg NOCOPY non-nil means it is OK to return STRING itself
6599 error ("Invalid coding system: %s", XSYMBOL (coding_system)->name->data); 6705if the decoding operation is trivial.
6600 6706
6601 coding.composing = COMPOSITION_DISABLED; 6707Optional fourth arg BUFFER non-nil meant that the decoded text is
6602 coding.mode |= CODING_MODE_LAST_BLOCK; 6708inserted in BUFFER instead of returned as a astring. In this case,
6603 return (encodep 6709the return value is BUFFER.
6604 ? encode_coding_string (string, &coding, 1) 6710
6605 : decode_coding_string (string, &coding, 1)); 6711This function sets `last-coding-system-used' to the precise coding system
6712used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6713not fully specified. */)
6714 (string, coding_system, nocopy, buffer)
6715 Lisp_Object string, coding_system, nocopy, buffer;
6716{
6717 return code_convert_string (string, coding_system, buffer,
6718 0, ! NILP (nocopy), 0);
6719}
6720
6721DEFUN ("encode-coding-string", Fencode_coding_string, Sencode_coding_string,
6722 2, 4, 0,
6723 doc: /* Encode STRING to CODING-SYSTEM, and return the result.
6724
6725Optional third arg NOCOPY non-nil means it is OK to return STRING
6726itself if the encoding operation is trivial.
6727
6728Optional fourth arg BUFFER non-nil meant that the encoded text is
6729inserted in BUFFER instead of returned as a astring. In this case,
6730the return value is BUFFER.
6731
6732This function sets `last-coding-system-used' to the precise coding system
6733used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6734not fully specified.) */)
6735 (string, coding_system, nocopy, buffer)
6736 Lisp_Object string, coding_system, nocopy, buffer;
6737{
6738 return code_convert_string (string, coding_system, buffer,
6739 nocopy, ! NILP (nocopy), 1);
6606} 6740}
6741
6607 6742
6608DEFUN ("decode-sjis-char", Fdecode_sjis_char, Sdecode_sjis_char, 1, 1, 0, 6743DEFUN ("decode-sjis-char", Fdecode_sjis_char, Sdecode_sjis_char, 1, 1, 0,
6609 doc: /* Decode a Japanese character which has CODE in shift_jis encoding. 6744 doc: /* Decode a Japanese character which has CODE in shift_jis encoding.
@@ -6611,60 +6746,75 @@ Return the corresponding character. */)
6611 (code) 6746 (code)
6612 Lisp_Object code; 6747 Lisp_Object code;
6613{ 6748{
6614 unsigned char c1, c2, s1, s2; 6749 Lisp_Object spec, attrs, val;
6615 Lisp_Object val; 6750 struct charset *charset_roman, *charset_kanji, *charset_kana, *charset;
6751 int c;
6752
6753 CHECK_NATNUM (code);
6754 c = XFASTINT (code);
6755 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
6756 attrs = AREF (spec, 0);
6757
6758 if (ASCII_BYTE_P (c)
6759 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
6760 return code;
6616 6761
6617 CHECK_NUMBER (code); 6762 val = CODING_ATTR_CHARSET_LIST (attrs);
6618 s1 = (XFASTINT (code)) >> 8, s2 = (XFASTINT (code)) & 0xFF; 6763 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
6619 if (s1 == 0) 6764 charset_kanji = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
6765 charset_kana = CHARSET_FROM_ID (XINT (XCAR (val)));
6766
6767 if (c <= 0x7F)
6768 charset = charset_roman;
6769 else if (c >= 0xA0 && c < 0xDF)
6620 { 6770 {
6621 if (s2 < 0x80) 6771 charset = charset_kana;
6622 XSETFASTINT (val, s2); 6772 c -= 0x80;
6623 else if (s2 >= 0xA0 || s2 <= 0xDF)
6624 XSETFASTINT (val, MAKE_CHAR (charset_katakana_jisx0201, s2, 0));
6625 else
6626 error ("Invalid Shift JIS code: %x", XFASTINT (code));
6627 } 6773 }
6628 else 6774 else
6629 { 6775 {
6630 if ((s1 < 0x80 || (s1 > 0x9F && s1 < 0xE0) || s1 > 0xEF) 6776 int s1 = c >> 8, s2 = c & 0x7F;
6631 || (s2 < 0x40 || s2 == 0x7F || s2 > 0xFC)) 6777
6632 error ("Invalid Shift JIS code: %x", XFASTINT (code)); 6778 if (s1 < 0x81 || (s1 > 0x9F && s1 < 0xE0) || s1 > 0xEF
6633 DECODE_SJIS (s1, s2, c1, c2); 6779 || s2 < 0x40 || s2 == 0x7F || s2 > 0xFC)
6634 XSETFASTINT (val, MAKE_CHAR (charset_jisx0208, c1, c2)); 6780 error ("Invalid code: %d", code);
6781 SJIS_TO_JIS (c);
6782 charset = charset_kanji;
6635 } 6783 }
6636 return val; 6784 c = DECODE_CHAR (charset, c);
6785 if (c < 0)
6786 error ("Invalid code: %d", code);
6787 return make_number (c);
6637} 6788}
6638 6789
6790
6639DEFUN ("encode-sjis-char", Fencode_sjis_char, Sencode_sjis_char, 1, 1, 0, 6791DEFUN ("encode-sjis-char", Fencode_sjis_char, Sencode_sjis_char, 1, 1, 0,
6640 doc: /* Encode a Japanese character CHAR to shift_jis encoding. 6792 doc: /* Encode a Japanese character CHAR to shift_jis encoding.
6641Return the corresponding code in SJIS. */) 6793Return the corresponding code in SJIS. */)
6642 (ch) 6794 (ch)
6643 Lisp_Object ch; 6795 Lisp_Object ch;
6644{ 6796{
6645 int charset, c1, c2, s1, s2; 6797 Lisp_Object spec, attrs, charset_list;
6646 Lisp_Object val; 6798 int c;
6799 struct charset *charset;
6800 unsigned code;
6647 6801
6648 CHECK_NUMBER (ch); 6802 CHECK_CHARACTER (ch);
6649 SPLIT_CHAR (XFASTINT (ch), charset, c1, c2); 6803 c = XFASTINT (ch);
6650 if (charset == CHARSET_ASCII) 6804 CHECK_CODING_SYSTEM_GET_SPEC (Vsjis_coding_system, spec);
6651 { 6805 attrs = AREF (spec, 0);
6652 val = ch; 6806
6653 } 6807 if (ASCII_CHAR_P (c)
6654 else if (charset == charset_jisx0208 6808 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
6655 && c1 > 0x20 && c1 < 0x7F && c2 > 0x20 && c2 < 0x7F) 6809 return ch;
6656 { 6810
6657 ENCODE_SJIS (c1, c2, s1, s2); 6811 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
6658 XSETFASTINT (val, (s1 << 8) | s2); 6812 charset = char_charset (c, charset_list, &code);
6659 } 6813 if (code == CHARSET_INVALID_CODE (charset))
6660 else if (charset == charset_katakana_jisx0201 6814 error ("Can't encode by shift_jis encoding: %d", c);
6661 && c1 > 0x20 && c2 < 0xE0) 6815 JIS_TO_SJIS (code);
6662 { 6816
6663 XSETFASTINT (val, c1 | 0x80); 6817 return make_number (code);
6664 }
6665 else
6666 error ("Can't encode to shift_jis: %d", XFASTINT (ch));
6667 return val;
6668} 6818}
6669 6819
6670DEFUN ("decode-big5-char", Fdecode_big5_char, Sdecode_big5_char, 1, 1, 0, 6820DEFUN ("decode-big5-char", Fdecode_big5_char, Sdecode_big5_char, 1, 1, 0,
@@ -6673,27 +6823,37 @@ Return the corresponding character. */)
6673 (code) 6823 (code)
6674 Lisp_Object code; 6824 Lisp_Object code;
6675{ 6825{
6676 int charset; 6826 Lisp_Object spec, attrs, val;
6677 unsigned char b1, b2, c1, c2; 6827 struct charset *charset_roman, *charset_big5, *charset;
6678 Lisp_Object val; 6828 int c;
6679 6829
6680 CHECK_NUMBER (code); 6830 CHECK_NATNUM (code);
6681 b1 = (XFASTINT (code)) >> 8, b2 = (XFASTINT (code)) & 0xFF; 6831 c = XFASTINT (code);
6682 if (b1 == 0) 6832 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
6683 { 6833 attrs = AREF (spec, 0);
6684 if (b2 >= 0x80) 6834
6685 error ("Invalid BIG5 code: %x", XFASTINT (code)); 6835 if (ASCII_BYTE_P (c)
6686 val = code; 6836 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
6687 } 6837 return code;
6838
6839 val = CODING_ATTR_CHARSET_LIST (attrs);
6840 charset_roman = CHARSET_FROM_ID (XINT (XCAR (val))), val = XCDR (val);
6841 charset_big5 = CHARSET_FROM_ID (XINT (XCAR (val)));
6842
6843 if (c <= 0x7F)
6844 charset = charset_roman;
6688 else 6845 else
6689 { 6846 {
6690 if ((b1 < 0xA1 || b1 > 0xFE) 6847 int b1 = c >> 8, b2 = c & 0x7F;
6691 || (b2 < 0x40 || (b2 > 0x7E && b2 < 0xA1) || b2 > 0xFE)) 6848 if (b1 < 0xA1 || b1 > 0xFE
6692 error ("Invalid BIG5 code: %x", XFASTINT (code)); 6849 || b2 < 0x40 || (b2 > 0x7E && b2 < 0xA1) || b2 > 0xFE)
6693 DECODE_BIG5 (b1, b2, charset, c1, c2); 6850 error ("Invalid code: %d", code);
6694 XSETFASTINT (val, MAKE_CHAR (charset, c1, c2)); 6851 charset = charset_big5;
6695 } 6852 }
6696 return val; 6853 c = DECODE_CHAR (charset, (unsigned )c);
6854 if (c < 0)
6855 error ("Invalid code: %d", code);
6856 return make_number (c);
6697} 6857}
6698 6858
6699DEFUN ("encode-big5-char", Fencode_big5_char, Sencode_big5_char, 1, 1, 0, 6859DEFUN ("encode-big5-char", Fencode_big5_char, Sencode_big5_char, 1, 1, 0,
@@ -6702,43 +6862,42 @@ Return the corresponding character code in Big5. */)
6702 (ch) 6862 (ch)
6703 Lisp_Object ch; 6863 Lisp_Object ch;
6704{ 6864{
6705 int charset, c1, c2, b1, b2; 6865 Lisp_Object spec, attrs, charset_list;
6706 Lisp_Object val; 6866 struct charset *charset;
6707 6867 int c;
6708 CHECK_NUMBER (ch); 6868 unsigned code;
6709 SPLIT_CHAR (XFASTINT (ch), charset, c1, c2); 6869
6710 if (charset == CHARSET_ASCII) 6870 CHECK_CHARACTER (ch);
6711 { 6871 c = XFASTINT (ch);
6712 val = ch; 6872 CHECK_CODING_SYSTEM_GET_SPEC (Vbig5_coding_system, spec);
6713 } 6873 attrs = AREF (spec, 0);
6714 else if ((charset == charset_big5_1 6874 if (ASCII_CHAR_P (c)
6715 && (XFASTINT (ch) >= 0x250a1 && XFASTINT (ch) <= 0x271ec)) 6875 && ! NILP (CODING_ATTR_ASCII_COMPAT (attrs)))
6716 || (charset == charset_big5_2 6876 return ch;
6717 && XFASTINT (ch) >= 0x290a1 && XFASTINT (ch) <= 0x2bdb2)) 6877
6718 { 6878 charset_list = CODING_ATTR_CHARSET_LIST (attrs);
6719 ENCODE_BIG5 (charset, c1, c2, b1, b2); 6879 charset = char_charset (c, charset_list, &code);
6720 XSETFASTINT (val, (b1 << 8) | b2); 6880 if (code == CHARSET_INVALID_CODE (charset))
6721 } 6881 error ("Can't encode by Big5 encoding: %d", c);
6722 else 6882
6723 error ("Can't encode to Big5: %d", XFASTINT (ch)); 6883 return make_number (code);
6724 return val;
6725} 6884}
6885
6726 6886
6727DEFUN ("set-terminal-coding-system-internal", 6887DEFUN ("set-terminal-coding-system-internal",
6728 Fset_terminal_coding_system_internal, 6888 Fset_terminal_coding_system_internal,
6729 Sset_terminal_coding_system_internal, 1, 1, 0, 6889 Sset_terminal_coding_system_internal, 1, 1, 0,
6730 doc: /* Internal use only. */) 6890 doc: /* Internal use only. */)
6731 (coding_system) 6891 (coding_system)
6732 Lisp_Object coding_system;
6733{ 6892{
6734 CHECK_SYMBOL (coding_system); 6893 CHECK_SYMBOL (coding_system);
6735 setup_coding_system (Fcheck_coding_system (coding_system), &terminal_coding); 6894 setup_coding_system (Fcheck_coding_system (coding_system),
6895 &terminal_coding);
6896
6736 /* We had better not send unsafe characters to terminal. */ 6897 /* We had better not send unsafe characters to terminal. */
6737 terminal_coding.flags |= CODING_FLAG_ISO_SAFE; 6898 terminal_coding.mode |= CODING_MODE_SAFE_ENCODING;
6738 /* Character composition should be disabled. */ 6899 /* Characer composition should be disabled. */
6739 terminal_coding.composing = COMPOSITION_DISABLED; 6900 terminal_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
6740 /* Error notification should be suppressed. */
6741 terminal_coding.suppress_error = 1;
6742 terminal_coding.src_multibyte = 1; 6901 terminal_coding.src_multibyte = 1;
6743 terminal_coding.dst_multibyte = 0; 6902 terminal_coding.dst_multibyte = 0;
6744 return Qnil; 6903 return Qnil;
@@ -6749,15 +6908,12 @@ DEFUN ("set-safe-terminal-coding-system-internal",
6749 Sset_safe_terminal_coding_system_internal, 1, 1, 0, 6908 Sset_safe_terminal_coding_system_internal, 1, 1, 0,
6750 doc: /* Internal use only. */) 6909 doc: /* Internal use only. */)
6751 (coding_system) 6910 (coding_system)
6752 Lisp_Object coding_system;
6753{ 6911{
6754 CHECK_SYMBOL (coding_system); 6912 CHECK_SYMBOL (coding_system);
6755 setup_coding_system (Fcheck_coding_system (coding_system), 6913 setup_coding_system (Fcheck_coding_system (coding_system),
6756 &safe_terminal_coding); 6914 &safe_terminal_coding);
6757 /* Character composition should be disabled. */ 6915 /* Characer composition should be disabled. */
6758 safe_terminal_coding.composing = COMPOSITION_DISABLED; 6916 safe_terminal_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
6759 /* Error notification should be suppressed. */
6760 terminal_coding.suppress_error = 1;
6761 safe_terminal_coding.src_multibyte = 1; 6917 safe_terminal_coding.src_multibyte = 1;
6762 safe_terminal_coding.dst_multibyte = 0; 6918 safe_terminal_coding.dst_multibyte = 0;
6763 return Qnil; 6919 return Qnil;
@@ -6768,7 +6924,7 @@ DEFUN ("terminal-coding-system",
6768 doc: /* Return coding system specified for terminal output. */) 6924 doc: /* Return coding system specified for terminal output. */)
6769 () 6925 ()
6770{ 6926{
6771 return terminal_coding.symbol; 6927 return CODING_ID_NAME (terminal_coding.id);
6772} 6928}
6773 6929
6774DEFUN ("set-keyboard-coding-system-internal", 6930DEFUN ("set-keyboard-coding-system-internal",
@@ -6779,9 +6935,10 @@ DEFUN ("set-keyboard-coding-system-internal",
6779 Lisp_Object coding_system; 6935 Lisp_Object coding_system;
6780{ 6936{
6781 CHECK_SYMBOL (coding_system); 6937 CHECK_SYMBOL (coding_system);
6782 setup_coding_system (Fcheck_coding_system (coding_system), &keyboard_coding); 6938 setup_coding_system (Fcheck_coding_system (coding_system),
6783 /* Character composition should be disabled. */ 6939 &keyboard_coding);
6784 keyboard_coding.composing = COMPOSITION_DISABLED; 6940 /* Characer composition should be disabled. */
6941 keyboard_coding.common_flags &= ~CODING_ANNOTATE_COMPOSITION_MASK;
6785 return Qnil; 6942 return Qnil;
6786} 6943}
6787 6944
@@ -6790,7 +6947,7 @@ DEFUN ("keyboard-coding-system",
6790 doc: /* Return coding system specified for decoding keyboard input. */) 6947 doc: /* Return coding system specified for decoding keyboard input. */)
6791 () 6948 ()
6792{ 6949{
6793 return keyboard_coding.symbol; 6950 return CODING_ID_NAME (keyboard_coding.id);
6794} 6951}
6795 6952
6796 6953
@@ -6838,14 +6995,14 @@ usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
6838 operation = args[0]; 6995 operation = args[0];
6839 if (!SYMBOLP (operation) 6996 if (!SYMBOLP (operation)
6840 || !INTEGERP (target_idx = Fget (operation, Qtarget_idx))) 6997 || !INTEGERP (target_idx = Fget (operation, Qtarget_idx)))
6841 error ("Invalid first argument"); 6998 error ("Invalid first arguement");
6842 if (nargs < 1 + XINT (target_idx)) 6999 if (nargs < 1 + XINT (target_idx))
6843 error ("Too few arguments for operation: %s", 7000 error ("Too few arguments for operation: %s",
6844 XSYMBOL (operation)->name->data); 7001 XSYMBOL (operation)->name->data);
6845 target = args[XINT (target_idx) + 1]; 7002 target = args[XINT (target_idx) + 1];
6846 if (!(STRINGP (target) 7003 if (!(STRINGP (target)
6847 || (EQ (operation, Qopen_network_stream) && INTEGERP (target)))) 7004 || (EQ (operation, Qopen_network_stream) && INTEGERP (target))))
6848 error ("Invalid argument %d", XINT (target_idx) + 1); 7005 error ("Invalid %dth argument", XINT (target_idx) + 1);
6849 7006
6850 chain = ((EQ (operation, Qinsert_file_contents) 7007 chain = ((EQ (operation, Qinsert_file_contents)
6851 || EQ (operation, Qwrite_region)) 7008 || EQ (operation, Qwrite_region))
@@ -6859,8 +7016,8 @@ usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
6859 for (; CONSP (chain); chain = XCDR (chain)) 7016 for (; CONSP (chain); chain = XCDR (chain))
6860 { 7017 {
6861 Lisp_Object elt; 7018 Lisp_Object elt;
6862 elt = XCAR (chain);
6863 7019
7020 elt = XCAR (chain);
6864 if (CONSP (elt) 7021 if (CONSP (elt)
6865 && ((STRINGP (target) 7022 && ((STRINGP (target)
6866 && STRINGP (XCAR (elt)) 7023 && STRINGP (XCAR (elt))
@@ -6890,67 +7047,611 @@ usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
6890 return Qnil; 7047 return Qnil;
6891} 7048}
6892 7049
6893DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal, 7050DEFUN ("set-coding-system-priority", Fset_coding_system_priority,
6894 Supdate_coding_systems_internal, 0, 0, 0, 7051 Sset_coding_system_priority, 1, MANY, 0,
6895 doc: /* Update internal database for ISO2022 and CCL based coding systems. 7052 doc: /* Put higher priority to coding systems of the arguments. */)
6896When values of any coding categories are changed, you must 7053 (nargs, args)
6897call this function. */) 7054 int nargs;
6898 () 7055 Lisp_Object *args;
7056{
7057 int i, j;
7058 int changed[coding_category_max];
7059 enum coding_category priorities[coding_category_max];
7060
7061 bzero (changed, sizeof changed);
7062
7063 for (i = j = 0; i < nargs; i++)
7064 {
7065 enum coding_category category;
7066 Lisp_Object spec, attrs;
7067
7068 CHECK_CODING_SYSTEM_GET_SPEC (args[i], spec);
7069 attrs = AREF (spec, 0);
7070 category = XINT (CODING_ATTR_CATEGORY (attrs));
7071 if (changed[category])
7072 /* Ignore this coding system because a coding system of the
7073 same category already had a higher priority. */
7074 continue;
7075 changed[category] = 1;
7076 priorities[j++] = category;
7077 if (coding_categories[category].id >= 0
7078 && ! EQ (args[i], CODING_ID_NAME (coding_categories[category].id)))
7079 setup_coding_system (args[i], &coding_categories[category]);
7080 }
7081
7082 /* Now we have decided top J priorities. Reflect the order of the
7083 original priorities to the remaining priorities. */
7084
7085 for (i = j, j = 0; i < coding_category_max; i++, j++)
7086 {
7087 while (j < coding_category_max
7088 && changed[coding_priorities[j]])
7089 j++;
7090 if (j == coding_category_max)
7091 abort ();
7092 priorities[i] = coding_priorities[j];
7093 }
7094
7095 bcopy (priorities, coding_priorities, sizeof priorities);
7096 return Qnil;
7097}
7098
7099DEFUN ("coding-system-priority-list", Fcoding_system_priority_list,
7100 Scoding_system_priority_list, 0, 1, 0,
7101 doc: /* Return a list of coding systems ordered by their priorities. */)
7102 (highestp)
7103 Lisp_Object highestp;
6899{ 7104{
6900 int i; 7105 int i;
7106 Lisp_Object val;
6901 7107
6902 for (i = CODING_CATEGORY_IDX_EMACS_MULE; i < CODING_CATEGORY_IDX_MAX; i++) 7108 for (i = 0, val = Qnil; i < coding_category_max; i++)
6903 { 7109 {
6904 Lisp_Object val; 7110 enum coding_category category = coding_priorities[i];
7111 int id = coding_categories[category].id;
7112 Lisp_Object attrs;
7113
7114 if (id < 0)
7115 continue;
7116 attrs = CODING_ID_ATTRS (id);
7117 if (! NILP (highestp))
7118 return CODING_ATTR_BASE_NAME (attrs);
7119 val = Fcons (CODING_ATTR_BASE_NAME (attrs), val);
7120 }
7121 return Fnreverse (val);
7122}
7123
7124static Lisp_Object
7125make_subsidiaries (base)
7126 Lisp_Object base;
7127{
7128 Lisp_Object subsidiaries;
7129 char *suffixes[] = { "-unix", "-dos", "-mac" };
7130 int base_name_len = STRING_BYTES (XSYMBOL (base)->name);
7131 char *buf = (char *) alloca (base_name_len + 6);
7132 int i;
7133
7134 bcopy (XSYMBOL (base)->name->data, buf, base_name_len);
7135 subsidiaries = Fmake_vector (make_number (3), Qnil);
7136 for (i = 0; i < 3; i++)
7137 {
7138 bcopy (suffixes[i], buf + base_name_len, strlen (suffixes[i]) + 1);
7139 ASET (subsidiaries, i, intern (buf));
7140 }
7141 return subsidiaries;
7142}
7143
7144
7145DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal,
7146 Sdefine_coding_system_internal, coding_arg_max, MANY, 0,
7147 doc: /* For internal use only. */)
7148 (nargs, args)
7149 int nargs;
7150 Lisp_Object *args;
7151{
7152 Lisp_Object name;
7153 Lisp_Object spec_vec; /* [ ATTRS ALIASE EOL_TYPE ] */
7154 Lisp_Object attrs; /* Vector of attributes. */
7155 Lisp_Object eol_type;
7156 Lisp_Object aliases;
7157 Lisp_Object coding_type, charset_list, safe_charsets;
7158 enum coding_category category;
7159 Lisp_Object tail, val;
7160 int max_charset_id = 0;
7161 int i;
7162
7163 if (nargs < coding_arg_max)
7164 goto short_args;
7165
7166 attrs = Fmake_vector (make_number (coding_attr_last_index), Qnil);
7167
7168 name = args[coding_arg_name];
7169 CHECK_SYMBOL (name);
7170 CODING_ATTR_BASE_NAME (attrs) = name;
7171
7172 val = args[coding_arg_mnemonic];
7173 if (! STRINGP (val))
7174 CHECK_CHARACTER (val);
7175 CODING_ATTR_MNEMONIC (attrs) = val;
7176
7177 coding_type = args[coding_arg_coding_type];
7178 CHECK_SYMBOL (coding_type);
7179 CODING_ATTR_TYPE (attrs) = coding_type;
7180
7181 charset_list = args[coding_arg_charset_list];
7182 if (SYMBOLP (charset_list))
7183 {
7184 if (EQ (charset_list, Qiso_2022))
7185 {
7186 if (! EQ (coding_type, Qiso_2022))
7187 error ("Invalid charset-list");
7188 charset_list = Viso_2022_charset_list;
7189 }
7190 else if (EQ (charset_list, Qemacs_mule))
7191 {
7192 if (! EQ (coding_type, Qemacs_mule))
7193 error ("Invalid charset-list");
7194 charset_list = Vemacs_mule_charset_list;
7195 }
7196 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
7197 if (max_charset_id < XFASTINT (XCAR (tail)))
7198 max_charset_id = XFASTINT (XCAR (tail));
7199 }
7200 else
7201 {
7202 charset_list = Fcopy_sequence (charset_list);
7203 for (tail = charset_list; !NILP (tail); tail = Fcdr (tail))
7204 {
7205 struct charset *charset;
7206
7207 val = Fcar (tail);
7208 CHECK_CHARSET_GET_CHARSET (val, charset);
7209 if (EQ (coding_type, Qiso_2022)
7210 ? CHARSET_ISO_FINAL (charset) < 0
7211 : EQ (coding_type, Qemacs_mule)
7212 ? CHARSET_EMACS_MULE_ID (charset) < 0
7213 : 0)
7214 error ("Can't handle charset `%s'",
7215 XSYMBOL (CHARSET_NAME (charset))->name->data);
7216
7217 XCAR (tail) = make_number (charset->id);
7218 if (max_charset_id < charset->id)
7219 max_charset_id = charset->id;
7220 }
7221 }
7222 CODING_ATTR_CHARSET_LIST (attrs) = charset_list;
7223
7224 safe_charsets = Fmake_string (make_number (max_charset_id + 1),
7225 make_number (255));
7226 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
7227 XSTRING (safe_charsets)->data[XFASTINT (XCAR (tail))] = 0;
7228 CODING_ATTR_SAFE_CHARSETS (attrs) = safe_charsets;
7229
7230 val = args[coding_arg_decode_translation_table];
7231 if (! NILP (val))
7232 CHECK_CHAR_TABLE (val);
7233 CODING_ATTR_DECODE_TBL (attrs) = val;
7234
7235 val = args[coding_arg_encode_translation_table];
7236 if (! NILP (val))
7237 CHECK_CHAR_TABLE (val);
7238 CODING_ATTR_ENCODE_TBL (attrs) = val;
7239
7240 val = args[coding_arg_post_read_conversion];
7241 CHECK_SYMBOL (val);
7242 CODING_ATTR_POST_READ (attrs) = val;
7243
7244 val = args[coding_arg_pre_write_conversion];
7245 CHECK_SYMBOL (val);
7246 CODING_ATTR_PRE_WRITE (attrs) = val;
7247
7248 val = args[coding_arg_default_char];
7249 if (NILP (val))
7250 CODING_ATTR_DEFAULT_CHAR (attrs) = make_number (' ');
7251 else
7252 {
7253 CHECK_CHARACTER (val);
7254 CODING_ATTR_DEFAULT_CHAR (attrs) = val;
7255 }
7256
7257 val = args[coding_arg_plist];
7258 CHECK_LIST (val);
7259 CODING_ATTR_PLIST (attrs) = val;
7260
7261 if (EQ (coding_type, Qcharset))
7262 {
7263 val = Fmake_vector (make_number (256), Qnil);
7264
7265 for (tail = charset_list; CONSP (tail); tail = XCDR (tail))
7266 {
7267 struct charset *charset = CHARSET_FROM_ID (XINT (XCAR (tail)));
7268
7269 for (i = charset->code_space[0]; i <= charset->code_space[1]; i++)
7270 if (NILP (AREF (val, i)))
7271 ASET (val, i, XCAR (tail));
7272 }
7273 ASET (attrs, coding_attr_charset_valids, val);
7274 category = coding_category_charset;
7275 }
7276 else if (EQ (coding_type, Qccl))
7277 {
7278 Lisp_Object valids;
7279
7280 if (nargs < coding_arg_ccl_max)
7281 goto short_args;
7282
7283 val = args[coding_arg_ccl_decoder];
7284 CHECK_CCL_PROGRAM (val);
7285 if (VECTORP (val))
7286 val = Fcopy_sequence (val);
7287 ASET (attrs, coding_attr_ccl_decoder, val);
7288
7289 val = args[coding_arg_ccl_encoder];
7290 CHECK_CCL_PROGRAM (val);
7291 if (VECTORP (val))
7292 val = Fcopy_sequence (val);
7293 ASET (attrs, coding_attr_ccl_encoder, val);
7294
7295 val = args[coding_arg_ccl_valids];
7296 valids = Fmake_string (make_number (256), make_number (0));
7297 for (tail = val; !NILP (tail); tail = Fcdr (tail))
7298 {
7299 val = Fcar (tail);
7300 if (INTEGERP (val))
7301 ASET (valids, XINT (val), 1);
7302 else
7303 {
7304 int from, to;
7305
7306 CHECK_CONS (val);
7307 CHECK_NUMBER (XCAR (val));
7308 CHECK_NUMBER (XCDR (val));
7309 from = XINT (XCAR (val));
7310 to = XINT (XCDR (val));
7311 for (i = from; i <= to; i++)
7312 ASET (valids, i, 1);
7313 }
7314 }
7315 ASET (attrs, coding_attr_ccl_valids, valids);
7316
7317 category = coding_category_ccl;
7318 }
7319 else if (EQ (coding_type, Qutf_16))
7320 {
7321 Lisp_Object bom, endian;
7322
7323 if (nargs < coding_arg_utf16_max)
7324 goto short_args;
7325
7326 bom = args[coding_arg_utf16_bom];
7327 if (! NILP (bom) && ! EQ (bom, Qt))
7328 {
7329 CHECK_CONS (bom);
7330 CHECK_CODING_SYSTEM (XCAR (bom));
7331 CHECK_CODING_SYSTEM (XCDR (bom));
7332 }
7333 ASET (attrs, coding_attr_utf_16_bom, bom);
7334
7335 endian = args[coding_arg_utf16_endian];
7336 ASET (attrs, coding_attr_utf_16_endian, endian);
7337
7338 category = (CONSP (bom)
7339 ? coding_category_utf_16_auto
7340 : NILP (bom)
7341 ? (NILP (endian)
7342 ? coding_category_utf_16_be_nosig
7343 : coding_category_utf_16_le_nosig)
7344 : (NILP (endian)
7345 ? coding_category_utf_16_be
7346 : coding_category_utf_16_le));
7347 }
7348 else if (EQ (coding_type, Qiso_2022))
7349 {
7350 Lisp_Object initial, reg_usage, request, flags;
7351 struct charset *charset;
7352 int i, id, max_id = -1;
7353
7354 if (nargs < coding_arg_iso2022_max)
7355 goto short_args;
7356
7357 initial = Fcopy_sequence (args[coding_arg_iso2022_initial]);
7358 CHECK_VECTOR (initial);
7359 for (i = 0; i < 4; i++)
7360 {
7361 val = Faref (initial, make_number (i));
7362 if (! NILP (val))
7363 {
7364 CHECK_CHARSET_GET_ID (val, id);
7365 ASET (initial, i, make_number (id));
7366 }
7367 else
7368 ASET (initial, i, make_number (-1));
7369 }
7370
7371 reg_usage = args[coding_arg_iso2022_reg_usage];
7372 CHECK_CONS (reg_usage);
7373 CHECK_NATNUM (XCAR (reg_usage));
7374 CHECK_NATNUM (XCDR (reg_usage));
6905 7375
6906 val = SYMBOL_VALUE (XVECTOR (Vcoding_category_table)->contents[i]); 7376 request = Fcopy_sequence (args[coding_arg_iso2022_request]);
6907 if (!NILP (val)) 7377 for (tail = request; ! NILP (tail); tail = Fcdr (tail))
6908 { 7378 {
6909 if (! coding_system_table[i]) 7379 int id;
6910 coding_system_table[i] = ((struct coding_system *) 7380
6911 xmalloc (sizeof (struct coding_system))); 7381 val = Fcar (tail);
6912 setup_coding_system (val, coding_system_table[i]); 7382 CHECK_CONS (val);
7383 CHECK_CHARSET_GET_ID (XCAR (val), id);
7384 CHECK_NATNUM (XCDR (val));
7385 if (XINT (XCDR (val)) >= 4)
7386 error ("Invalid graphic register number: %d", XINT (XCDR (val)));
7387 XCAR (val) = make_number (id);
6913 } 7388 }
6914 else if (coding_system_table[i]) 7389
7390 flags = args[coding_arg_iso2022_flags];
7391 CHECK_NATNUM (flags);
7392 i = XINT (flags);
7393 if (EQ (args[coding_arg_charset_list], Qiso_2022))
7394 flags = make_number (i | CODING_ISO_FLAG_FULL_SUPPORT);
7395
7396 ASET (attrs, coding_attr_iso_initial, initial);
7397 ASET (attrs, coding_attr_iso_usage, reg_usage);
7398 ASET (attrs, coding_attr_iso_request, request);
7399 ASET (attrs, coding_attr_iso_flags, flags);
7400 setup_iso_safe_charsets (attrs);
7401
7402 if (i & CODING_ISO_FLAG_SEVEN_BITS)
7403 category = ((i & (CODING_ISO_FLAG_LOCKING_SHIFT
7404 | CODING_ISO_FLAG_SINGLE_SHIFT))
7405 ? coding_category_iso_7_else
7406 : EQ (args[coding_arg_charset_list], Qiso_2022)
7407 ? coding_category_iso_7
7408 : coding_category_iso_7_tight);
7409 else
7410 {
7411 int id = XINT (AREF (initial, 1));
7412
7413 category = (((i & (CODING_ISO_FLAG_LOCKING_SHIFT
7414 | CODING_ISO_FLAG_SINGLE_SHIFT))
7415 || EQ (args[coding_arg_charset_list], Qiso_2022)
7416 || id < 0)
7417 ? coding_category_iso_8_else
7418 : (CHARSET_DIMENSION (CHARSET_FROM_ID (id)) == 1)
7419 ? coding_category_iso_8_1
7420 : coding_category_iso_8_2);
7421 }
7422 }
7423 else if (EQ (coding_type, Qemacs_mule))
7424 {
7425 if (EQ (args[coding_arg_charset_list], Qemacs_mule))
7426 ASET (attrs, coding_attr_emacs_mule_full, Qt);
7427
7428 category = coding_category_emacs_mule;
7429 }
7430 else if (EQ (coding_type, Qshift_jis))
7431 {
7432
7433 struct charset *charset;
7434
7435 if (XINT (Flength (charset_list)) != 3)
7436 error ("There should be just three charsets");
7437
7438 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
7439 if (CHARSET_DIMENSION (charset) != 1)
7440 error ("Dimension of charset %s is not one",
7441 XSYMBOL (CHARSET_NAME (charset))->name->data);
7442
7443 charset_list = XCDR (charset_list);
7444 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
7445 if (CHARSET_DIMENSION (charset) != 1)
7446 error ("Dimension of charset %s is not one",
7447 XSYMBOL (CHARSET_NAME (charset))->name->data);
7448
7449 charset_list = XCDR (charset_list);
7450 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
7451 if (CHARSET_DIMENSION (charset) != 2)
7452 error ("Dimension of charset %s is not two",
7453 XSYMBOL (CHARSET_NAME (charset))->name->data);
7454
7455 category = coding_category_sjis;
7456 Vsjis_coding_system = name;
7457 }
7458 else if (EQ (coding_type, Qbig5))
7459 {
7460 struct charset *charset;
7461
7462 if (XINT (Flength (charset_list)) != 2)
7463 error ("There should be just two charsets");
7464
7465 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
7466 if (CHARSET_DIMENSION (charset) != 1)
7467 error ("Dimension of charset %s is not one",
7468 XSYMBOL (CHARSET_NAME (charset))->name->data);
7469
7470 charset_list = XCDR (charset_list);
7471 charset = CHARSET_FROM_ID (XINT (XCAR (charset_list)));
7472 if (CHARSET_DIMENSION (charset) != 2)
7473 error ("Dimension of charset %s is not two",
7474 XSYMBOL (CHARSET_NAME (charset))->name->data);
7475
7476 category = coding_category_big5;
7477 Vbig5_coding_system = name;
7478 }
7479 else if (EQ (coding_type, Qraw_text))
7480 category = coding_category_raw_text;
7481 else if (EQ (coding_type, Qutf_8))
7482 category = coding_category_utf_8;
7483 else if (EQ (coding_type, Qundecided))
7484 category = coding_category_undecided;
7485 else
7486 error ("Invalid coding system type: %s",
7487 XSYMBOL (coding_type)->name->data);
7488
7489 CODING_ATTR_CATEGORY (attrs) = make_number (category);
7490
7491 eol_type = args[coding_arg_eol_type];
7492 if (! NILP (eol_type)
7493 && ! EQ (eol_type, Qunix)
7494 && ! EQ (eol_type, Qdos)
7495 && ! EQ (eol_type, Qmac))
7496 error ("Invalid eol-type");
7497
7498 aliases = Fcons (name, Qnil);
7499
7500 if (NILP (eol_type))
7501 {
7502 eol_type = make_subsidiaries (name);
7503 for (i = 0; i < 3; i++)
6915 { 7504 {
6916 xfree (coding_system_table[i]); 7505 Lisp_Object this_spec, this_name, this_aliases, this_eol_type;
6917 coding_system_table[i] = NULL; 7506
7507 this_name = AREF (eol_type, i);
7508 this_aliases = Fcons (this_name, Qnil);
7509 this_eol_type = (i == 0 ? Qunix : i == 1 ? Qdos : Qmac);
7510 this_spec = Fmake_vector (make_number (3), attrs);
7511 ASET (this_spec, 1, this_aliases);
7512 ASET (this_spec, 2, this_eol_type);
7513 Fputhash (this_name, this_spec, Vcoding_system_hash_table);
7514 Vcoding_system_list = Fcons (this_name, Vcoding_system_list);
7515 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (this_name), Qnil),
7516 Vcoding_system_alist);
6918 } 7517 }
6919 } 7518 }
6920 7519
7520 spec_vec = Fmake_vector (make_number (3), attrs);
7521 ASET (spec_vec, 1, aliases);
7522 ASET (spec_vec, 2, eol_type);
7523
7524 Fputhash (name, spec_vec, Vcoding_system_hash_table);
7525 Vcoding_system_list = Fcons (name, Vcoding_system_list);
7526 Vcoding_system_alist = Fcons (Fcons (Fsymbol_name (name), Qnil),
7527 Vcoding_system_alist);
7528
7529 {
7530 int id = coding_categories[category].id;
7531
7532 if (id < 0 || EQ (name, CODING_ID_NAME (id)))
7533 setup_coding_system (name, &coding_categories[category]);
7534 }
7535
6921 return Qnil; 7536 return Qnil;
7537
7538 short_args:
7539 return Fsignal (Qwrong_number_of_arguments,
7540 Fcons (intern ("define-coding-system-internal"),
7541 make_number (nargs)));
6922} 7542}
6923 7543
6924DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal, 7544DEFUN ("define-coding-system-alias", Fdefine_coding_system_alias,
6925 Sset_coding_priority_internal, 0, 0, 0, 7545 Sdefine_coding_system_alias, 2, 2, 0,
6926 doc: /* Update internal database for the current value of `coding-category-list'. 7546 doc: /* Define ALIAS as an alias for CODING-SYSTEM. */)
6927This function is internal use only. */) 7547 (alias, coding_system)
6928 () 7548 Lisp_Object alias, coding_system;
6929{ 7549{
6930 int i = 0, idx; 7550 Lisp_Object spec, aliases, eol_type;
6931 Lisp_Object val;
6932 7551
6933 val = Vcoding_category_list; 7552 CHECK_SYMBOL (alias);
7553 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
7554 aliases = AREF (spec, 1);
7555 while (!NILP (XCDR (aliases)))
7556 aliases = XCDR (aliases);
7557 XCDR (aliases) = Fcons (alias, Qnil);
6934 7558
6935 while (CONSP (val) && i < CODING_CATEGORY_IDX_MAX) 7559 eol_type = AREF (spec, 2);
7560 if (VECTORP (eol_type))
6936 { 7561 {
6937 if (! SYMBOLP (XCAR (val))) 7562 Lisp_Object subsidiaries;
6938 break; 7563 int i;
6939 idx = XFASTINT (Fget (XCAR (val), Qcoding_category_index)); 7564
6940 if (idx >= CODING_CATEGORY_IDX_MAX) 7565 subsidiaries = make_subsidiaries (alias);
6941 break; 7566 for (i = 0; i < 3; i++)
6942 coding_priorities[i++] = (1 << idx); 7567 Fdefine_coding_system_alias (AREF (subsidiaries, i),
6943 val = XCDR (val); 7568 AREF (eol_type, i));
7569
7570 ASET (spec, 2, subsidiaries);
6944 } 7571 }
6945 /* If coding-category-list is valid and contains all coding 7572
6946 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not, 7573 Fputhash (alias, spec, Vcoding_system_hash_table);
6947 the following code saves Emacs from crashing. */
6948 while (i < CODING_CATEGORY_IDX_MAX)
6949 coding_priorities[i++] = CODING_CATEGORY_MASK_RAW_TEXT;
6950 7574
6951 return Qnil; 7575 return Qnil;
6952} 7576}
6953 7577
7578DEFUN ("coding-system-base", Fcoding_system_base, Scoding_system_base,
7579 1, 1, 0,
7580 doc: /* Return the base of CODING-SYSTEM.
7581Any alias or subsidiary coding systems are not base coding system. */)
7582 (coding_system)
7583 Lisp_Object coding_system;
7584{
7585 Lisp_Object spec, attrs;
7586
7587 if (NILP (coding_system))
7588 return (Qno_conversion);
7589 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
7590 attrs = AREF (spec, 0);
7591 return CODING_ATTR_BASE_NAME (attrs);
7592}
7593
7594DEFUN ("coding-system-plist", Fcoding_system_plist, Scoding_system_plist,
7595 1, 1, 0,
7596 doc: "Return the property list of CODING-SYSTEM.")
7597 (coding_system)
7598 Lisp_Object coding_system;
7599{
7600 Lisp_Object spec, attrs;
7601
7602 if (NILP (coding_system))
7603 coding_system = Qno_conversion;
7604 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
7605 attrs = AREF (spec, 0);
7606 return CODING_ATTR_PLIST (attrs);
7607}
7608
7609
7610DEFUN ("coding-system-aliases", Fcoding_system_aliases, Scoding_system_aliases,
7611 1, 1, 0,
7612 doc: /* Return the list of aliases of CODING-SYSTEM.
7613A base coding system is what made by `define-coding-system'.
7614Any alias nor subsidiary coding systems are not base coding system. */)
7615 (coding_system)
7616 Lisp_Object coding_system;
7617{
7618 Lisp_Object spec;
7619
7620 if (NILP (coding_system))
7621 coding_system = Qno_conversion;
7622 CHECK_CODING_SYSTEM_GET_SPEC (coding_system, spec);
7623 return AREF (spec, 2);
7624}
7625
7626DEFUN ("coding-system-eol-type", Fcoding_system_eol_type,
7627 Scoding_system_eol_type, 1, 1, 0,
7628 doc: /* Return eol-type of CODING-SYSTEM.
7629An eol-type is integer 0, 1, 2, or a vector of coding systems.
7630
7631Integer values 0, 1, and 2 indicate a format of end-of-line; LF, CRLF,
7632and CR respectively.
7633
7634A vector value indicates that a format of end-of-line should be
7635detected automatically. Nth element of the vector is the subsidiary
7636coding system whose eol-type is N. */)
7637 (coding_system)
7638 Lisp_Object coding_system;
7639{
7640 Lisp_Object spec, eol_type;
7641 int n;
7642
7643 if (NILP (coding_system))
7644 coding_system = Qno_conversion;
7645 if (! CODING_SYSTEM_P (coding_system))
7646 return Qnil;
7647 spec = CODING_SYSTEM_SPEC (coding_system);
7648 eol_type = AREF (spec, 2);
7649 if (VECTORP (eol_type))
7650 return Fcopy_sequence (eol_type);
7651 n = EQ (eol_type, Qunix) ? 0 : EQ (eol_type, Qdos) ? 1 : 2;
7652 return make_number (n);
7653}
7654
6954#endif /* emacs */ 7655#endif /* emacs */
6955 7656
6956 7657
@@ -6961,20 +7662,11 @@ init_coding_once ()
6961{ 7662{
6962 int i; 7663 int i;
6963 7664
6964 /* Emacs' internal format specific initialize routine. */ 7665 for (i = 0; i < coding_category_max; i++)
6965 for (i = 0; i <= 0x20; i++) 7666 {
6966 emacs_code_class[i] = EMACS_control_code; 7667 coding_categories[i].id = -1;
6967 emacs_code_class[0x0A] = EMACS_linefeed_code; 7668 coding_priorities[i] = i;
6968 emacs_code_class[0x0D] = EMACS_carriage_return_code; 7669 }
6969 for (i = 0x21 ; i < 0x7F; i++)
6970 emacs_code_class[i] = EMACS_ascii_code;
6971 emacs_code_class[0x7F] = EMACS_control_code;
6972 for (i = 0x80; i < 0xFF; i++)
6973 emacs_code_class[i] = EMACS_invalid_code;
6974 emacs_code_class[LEADING_CODE_PRIVATE_11] = EMACS_leading_code_3;
6975 emacs_code_class[LEADING_CODE_PRIVATE_12] = EMACS_leading_code_3;
6976 emacs_code_class[LEADING_CODE_PRIVATE_21] = EMACS_leading_code_4;
6977 emacs_code_class[LEADING_CODE_PRIVATE_22] = EMACS_leading_code_4;
6978 7670
6979 /* ISO2022 specific initialize routine. */ 7671 /* ISO2022 specific initialize routine. */
6980 for (i = 0; i < 0x20; i++) 7672 for (i = 0; i < 0x20; i++)
@@ -6996,24 +7688,12 @@ init_coding_once ()
6996 iso_code_class[ISO_CODE_SS3] = ISO_single_shift_3; 7688 iso_code_class[ISO_CODE_SS3] = ISO_single_shift_3;
6997 iso_code_class[ISO_CODE_CSI] = ISO_control_sequence_introducer; 7689 iso_code_class[ISO_CODE_CSI] = ISO_control_sequence_introducer;
6998 7690
6999 setup_coding_system (Qnil, &keyboard_coding);
7000 setup_coding_system (Qnil, &terminal_coding);
7001 setup_coding_system (Qnil, &safe_terminal_coding);
7002 setup_coding_system (Qnil, &default_buffer_file_coding);
7003
7004 bzero (coding_system_table, sizeof coding_system_table);
7005
7006 bzero (ascii_skip_code, sizeof ascii_skip_code);
7007 for (i = 0; i < 128; i++)
7008 ascii_skip_code[i] = 1;
7009
7010#if defined (MSDOS) || defined (WINDOWSNT)
7011 system_eol_type = CODING_EOL_CRLF;
7012#else
7013 system_eol_type = CODING_EOL_LF;
7014#endif
7015
7016 inhibit_pre_post_conversion = 0; 7691 inhibit_pre_post_conversion = 0;
7692
7693 for (i = 0; i < 256; i++)
7694 {
7695 emacs_mule_bytes[i] = 1;
7696 }
7017} 7697}
7018 7698
7019#ifdef emacs 7699#ifdef emacs
@@ -7021,11 +7701,24 @@ init_coding_once ()
7021void 7701void
7022syms_of_coding () 7702syms_of_coding ()
7023{ 7703{
7024 Qtarget_idx = intern ("target-idx"); 7704 staticpro (&Vcoding_system_hash_table);
7025 staticpro (&Qtarget_idx); 7705 Vcoding_system_hash_table = Fmakehash (Qeq);
7706
7707 staticpro (&Vsjis_coding_system);
7708 Vsjis_coding_system = Qnil;
7026 7709
7027 Qcoding_system_history = intern ("coding-system-history"); 7710 staticpro (&Vbig5_coding_system);
7028 staticpro (&Qcoding_system_history); 7711 Vbig5_coding_system = Qnil;
7712
7713 staticpro (&Vcode_conversion_work_buf_list);
7714 Vcode_conversion_work_buf_list = Qnil;
7715
7716 staticpro (&Vcode_conversion_reused_work_buf);
7717 Vcode_conversion_reused_work_buf = Qnil;
7718
7719 DEFSYM (Qcharset, "charset");
7720 DEFSYM (Qtarget_idx, "target-idx");
7721 DEFSYM (Qcoding_system_history, "coding-system-history");
7029 Fset (Qcoding_system_history, Qnil); 7722 Fset (Qcoding_system_history, Qnil);
7030 7723
7031 /* Target FILENAME is the first argument. */ 7724 /* Target FILENAME is the first argument. */
@@ -7033,111 +7726,157 @@ syms_of_coding ()
7033 /* Target FILENAME is the third argument. */ 7726 /* Target FILENAME is the third argument. */
7034 Fput (Qwrite_region, Qtarget_idx, make_number (2)); 7727 Fput (Qwrite_region, Qtarget_idx, make_number (2));
7035 7728
7036 Qcall_process = intern ("call-process"); 7729 DEFSYM (Qcall_process, "call-process");
7037 staticpro (&Qcall_process);
7038 /* Target PROGRAM is the first argument. */ 7730 /* Target PROGRAM is the first argument. */
7039 Fput (Qcall_process, Qtarget_idx, make_number (0)); 7731 Fput (Qcall_process, Qtarget_idx, make_number (0));
7040 7732
7041 Qcall_process_region = intern ("call-process-region"); 7733 DEFSYM (Qcall_process_region, "call-process-region");
7042 staticpro (&Qcall_process_region);
7043 /* Target PROGRAM is the third argument. */ 7734 /* Target PROGRAM is the third argument. */
7044 Fput (Qcall_process_region, Qtarget_idx, make_number (2)); 7735 Fput (Qcall_process_region, Qtarget_idx, make_number (2));
7045 7736
7046 Qstart_process = intern ("start-process"); 7737 DEFSYM (Qstart_process, "start-process");
7047 staticpro (&Qstart_process);
7048 /* Target PROGRAM is the third argument. */ 7738 /* Target PROGRAM is the third argument. */
7049 Fput (Qstart_process, Qtarget_idx, make_number (2)); 7739 Fput (Qstart_process, Qtarget_idx, make_number (2));
7050 7740
7051 Qopen_network_stream = intern ("open-network-stream"); 7741 DEFSYM (Qopen_network_stream, "open-network-stream");
7052 staticpro (&Qopen_network_stream);
7053 /* Target SERVICE is the fourth argument. */ 7742 /* Target SERVICE is the fourth argument. */
7054 Fput (Qopen_network_stream, Qtarget_idx, make_number (3)); 7743 Fput (Qopen_network_stream, Qtarget_idx, make_number (3));
7055 7744
7056 Qcoding_system = intern ("coding-system"); 7745 DEFSYM (Qcoding_system, "coding-system");
7057 staticpro (&Qcoding_system); 7746 DEFSYM (Qcoding_aliases, "coding-aliases");
7058 7747
7059 Qeol_type = intern ("eol-type"); 7748 DEFSYM (Qeol_type, "eol-type");
7060 staticpro (&Qeol_type); 7749 DEFSYM (Qunix, "unix");
7750 DEFSYM (Qdos, "dos");
7751 DEFSYM (Qmac, "mac");
7061 7752
7062 Qbuffer_file_coding_system = intern ("buffer-file-coding-system"); 7753 DEFSYM (Qbuffer_file_coding_system, "buffer-file-coding-system");
7063 staticpro (&Qbuffer_file_coding_system); 7754 DEFSYM (Qpost_read_conversion, "post-read-conversion");
7755 DEFSYM (Qpre_write_conversion, "pre-write-conversion");
7756 DEFSYM (Qdefault_char, "default-char");
7757 DEFSYM (Qundecided, "undecided");
7758 DEFSYM (Qno_conversion, "no-conversion");
7759 DEFSYM (Qraw_text, "raw-text");
7064 7760
7065 Qpost_read_conversion = intern ("post-read-conversion"); 7761 DEFSYM (Qiso_2022, "iso-2022");
7066 staticpro (&Qpost_read_conversion);
7067 7762
7068 Qpre_write_conversion = intern ("pre-write-conversion"); 7763 DEFSYM (Qutf_8, "utf-8");
7069 staticpro (&Qpre_write_conversion);
7070 7764
7071 Qno_conversion = intern ("no-conversion"); 7765 DEFSYM (Qutf_16, "utf-16");
7072 staticpro (&Qno_conversion); 7766 DEFSYM (Qutf_16_be, "utf-16-be");
7767 DEFSYM (Qutf_16_be_nosig, "utf-16-be-nosig");
7768 DEFSYM (Qutf_16_le, "utf-16-l3");
7769 DEFSYM (Qutf_16_le_nosig, "utf-16-le-nosig");
7770 DEFSYM (Qsignature, "signature");
7771 DEFSYM (Qendian, "endian");
7772 DEFSYM (Qbig, "big");
7773 DEFSYM (Qlittle, "little");
7073 7774
7074 Qundecided = intern ("undecided"); 7775 DEFSYM (Qshift_jis, "shift-jis");
7075 staticpro (&Qundecided); 7776 DEFSYM (Qbig5, "big5");
7076 7777
7077 Qcoding_system_p = intern ("coding-system-p"); 7778 DEFSYM (Qcoding_system_p, "coding-system-p");
7078 staticpro (&Qcoding_system_p);
7079
7080 Qcoding_system_error = intern ("coding-system-error");
7081 staticpro (&Qcoding_system_error);
7082 7779
7780 DEFSYM (Qcoding_system_error, "coding-system-error");
7083 Fput (Qcoding_system_error, Qerror_conditions, 7781 Fput (Qcoding_system_error, Qerror_conditions,
7084 Fcons (Qcoding_system_error, Fcons (Qerror, Qnil))); 7782 Fcons (Qcoding_system_error, Fcons (Qerror, Qnil)));
7085 Fput (Qcoding_system_error, Qerror_message, 7783 Fput (Qcoding_system_error, Qerror_message,
7086 build_string ("Invalid coding system")); 7784 build_string ("Invalid coding system"));
7087 7785
7088 Qcoding_category = intern ("coding-category"); 7786 /* Intern this now in case it isn't already done.
7089 staticpro (&Qcoding_category); 7787 Setting this variable twice is harmless.
7090 Qcoding_category_index = intern ("coding-category-index"); 7788 But don't staticpro it here--that is done in alloc.c. */
7091 staticpro (&Qcoding_category_index); 7789 Qchar_table_extra_slots = intern ("char-table-extra-slots");
7092
7093 Vcoding_category_table
7094 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX), Qnil);
7095 staticpro (&Vcoding_category_table);
7096 {
7097 int i;
7098 for (i = 0; i < CODING_CATEGORY_IDX_MAX; i++)
7099 {
7100 XVECTOR (Vcoding_category_table)->contents[i]
7101 = intern (coding_category_name[i]);
7102 Fput (XVECTOR (Vcoding_category_table)->contents[i],
7103 Qcoding_category_index, make_number (i));
7104 }
7105 }
7106 7790
7107 Qtranslation_table = intern ("translation-table"); 7791 DEFSYM (Qtranslation_table, "translation-table");
7108 staticpro (&Qtranslation_table);
7109 Fput (Qtranslation_table, Qchar_table_extra_slots, make_number (1)); 7792 Fput (Qtranslation_table, Qchar_table_extra_slots, make_number (1));
7793 DEFSYM (Qtranslation_table_id, "translation-table-id");
7794 DEFSYM (Qtranslation_table_for_decode, "translation-table-for-decode");
7795 DEFSYM (Qtranslation_table_for_encode, "translation-table-for-encode");
7110 7796
7111 Qtranslation_table_id = intern ("translation-table-id"); 7797 DEFSYM (Qchar_coding_system, "char-coding-system");
7112 staticpro (&Qtranslation_table_id);
7113
7114 Qtranslation_table_for_decode = intern ("translation-table-for-decode");
7115 staticpro (&Qtranslation_table_for_decode);
7116 7798
7117 Qtranslation_table_for_encode = intern ("translation-table-for-encode"); 7799 Fput (Qchar_coding_system, Qchar_table_extra_slots, make_number (2));
7118 staticpro (&Qtranslation_table_for_encode);
7119 7800
7120 Qsafe_chars = intern ("safe-chars"); 7801 DEFSYM (Qvalid_codes, "valid-codes");
7121 staticpro (&Qsafe_chars);
7122 7802
7123 Qchar_coding_system = intern ("char-coding-system"); 7803 DEFSYM (Qemacs_mule, "emacs-mule");
7124 staticpro (&Qchar_coding_system);
7125 7804
7126 /* Intern this now in case it isn't already done. 7805 Vcoding_category_table
7127 Setting this variable twice is harmless. 7806 = Fmake_vector (make_number (coding_category_max), Qnil);
7128 But don't staticpro it here--that is done in alloc.c. */ 7807 staticpro (&Vcoding_category_table);
7129 Qchar_table_extra_slots = intern ("char-table-extra-slots"); 7808 /* Followings are target of code detection. */
7130 Fput (Qsafe_chars, Qchar_table_extra_slots, make_number (0)); 7809 ASET (Vcoding_category_table, coding_category_iso_7,
7131 Fput (Qchar_coding_system, Qchar_table_extra_slots, make_number (2)); 7810 intern ("coding-category-iso-7"));
7811 ASET (Vcoding_category_table, coding_category_iso_7_tight,
7812 intern ("coding-category-iso-7-tight"));
7813 ASET (Vcoding_category_table, coding_category_iso_8_1,
7814 intern ("coding-category-iso-8-1"));
7815 ASET (Vcoding_category_table, coding_category_iso_8_2,
7816 intern ("coding-category-iso-8-2"));
7817 ASET (Vcoding_category_table, coding_category_iso_7_else,
7818 intern ("coding-category-iso-7-else"));
7819 ASET (Vcoding_category_table, coding_category_iso_8_else,
7820 intern ("coding-category-iso-8-else"));
7821 ASET (Vcoding_category_table, coding_category_utf_8,
7822 intern ("coding-category-utf-8"));
7823 ASET (Vcoding_category_table, coding_category_utf_16_be,
7824 intern ("coding-category-utf-16-be"));
7825 ASET (Vcoding_category_table, coding_category_utf_16_le,
7826 intern ("coding-category-utf-16-le"));
7827 ASET (Vcoding_category_table, coding_category_utf_16_be_nosig,
7828 intern ("coding-category-utf-16-be-nosig"));
7829 ASET (Vcoding_category_table, coding_category_utf_16_le_nosig,
7830 intern ("coding-category-utf-16-le-nosig"));
7831 ASET (Vcoding_category_table, coding_category_charset,
7832 intern ("coding-category-charset"));
7833 ASET (Vcoding_category_table, coding_category_sjis,
7834 intern ("coding-category-sjis"));
7835 ASET (Vcoding_category_table, coding_category_big5,
7836 intern ("coding-category-big5"));
7837 ASET (Vcoding_category_table, coding_category_ccl,
7838 intern ("coding-category-ccl"));
7839 ASET (Vcoding_category_table, coding_category_emacs_mule,
7840 intern ("coding-category-emacs-mule"));
7841 /* Followings are NOT target of code detection. */
7842 ASET (Vcoding_category_table, coding_category_raw_text,
7843 intern ("coding-category-raw-text"));
7844 ASET (Vcoding_category_table, coding_category_undecided,
7845 intern ("coding-category-undecided"));
7132 7846
7133 Qvalid_codes = intern ("valid-codes"); 7847 {
7134 staticpro (&Qvalid_codes); 7848 Lisp_Object args[coding_arg_max];
7849 Lisp_Object plist[14];
7850 int i;
7135 7851
7136 Qemacs_mule = intern ("emacs-mule"); 7852 for (i = 0; i < coding_arg_max; i++)
7137 staticpro (&Qemacs_mule); 7853 args[i] = Qnil;
7854
7855 plist[0] = intern (":name");
7856 plist[1] = args[coding_arg_name] = Qno_conversion;
7857 plist[2] = intern (":mnemonic");
7858 plist[3] = args[coding_arg_mnemonic] = make_number ('=');
7859 plist[4] = intern (":coding-type");
7860 plist[5] = args[coding_arg_coding_type] = Qraw_text;
7861 plist[6] = intern (":ascii-compatible-p");
7862 plist[7] = args[coding_arg_ascii_compatible_p] = Qt;
7863 plist[8] = intern (":default-char");
7864 plist[9] = args[coding_arg_default_char] = make_number (0);
7865 plist[10] = intern (":docstring");
7866 plist[11] = build_string ("Do no conversion.\n\
7867\n\
7868When you visit a file with this coding, the file is read into a\n\
7869unibyte buffer as is, thus each byte of a file is treated as a\n\
7870character.");
7871 plist[12] = intern (":eol-type");
7872 plist[13] = args[coding_arg_eol_type] = Qunix;
7873 args[coding_arg_plist] = Flist (14, plist);
7874 Fdefine_coding_system_internal (coding_arg_max, args);
7875 }
7138 7876
7139 Qraw_text = intern ("raw-text"); 7877 setup_coding_system (Qno_conversion, &keyboard_coding);
7140 staticpro (&Qraw_text); 7878 setup_coding_system (Qno_conversion, &terminal_coding);
7879 setup_coding_system (Qno_conversion, &safe_terminal_coding);
7141 7880
7142 defsubr (&Scoding_system_p); 7881 defsubr (&Scoding_system_p);
7143 defsubr (&Sread_coding_system); 7882 defsubr (&Sread_coding_system);
@@ -7146,6 +7885,7 @@ syms_of_coding ()
7146 defsubr (&Sdetect_coding_region); 7885 defsubr (&Sdetect_coding_region);
7147 defsubr (&Sdetect_coding_string); 7886 defsubr (&Sdetect_coding_string);
7148 defsubr (&Sfind_coding_systems_region_internal); 7887 defsubr (&Sfind_coding_systems_region_internal);
7888 defsubr (&Scheck_coding_systems_region);
7149 defsubr (&Sdecode_coding_region); 7889 defsubr (&Sdecode_coding_region);
7150 defsubr (&Sencode_coding_region); 7890 defsubr (&Sencode_coding_region);
7151 defsubr (&Sdecode_coding_string); 7891 defsubr (&Sdecode_coding_string);
@@ -7160,14 +7900,20 @@ syms_of_coding ()
7160 defsubr (&Sset_keyboard_coding_system_internal); 7900 defsubr (&Sset_keyboard_coding_system_internal);
7161 defsubr (&Skeyboard_coding_system); 7901 defsubr (&Skeyboard_coding_system);
7162 defsubr (&Sfind_operation_coding_system); 7902 defsubr (&Sfind_operation_coding_system);
7163 defsubr (&Supdate_coding_systems_internal); 7903 defsubr (&Sset_coding_system_priority);
7164 defsubr (&Sset_coding_priority_internal); 7904 defsubr (&Sdefine_coding_system_internal);
7905 defsubr (&Sdefine_coding_system_alias);
7906 defsubr (&Scoding_system_base);
7907 defsubr (&Scoding_system_plist);
7908 defsubr (&Scoding_system_aliases);
7909 defsubr (&Scoding_system_eol_type);
7910 defsubr (&Scoding_system_priority_list);
7165 7911
7166 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list, 7912 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list,
7167 doc: /* List of coding systems. 7913 doc: /* List of coding systems.
7168 7914
7169Do not alter the value of this variable manually. This variable should be 7915Do not alter the value of this variable manually. This variable should be
7170updated by the functions `make-coding-system' and 7916updated by the functions `define-coding-system' and
7171`define-coding-system-alias'. */); 7917`define-coding-system-alias'. */);
7172 Vcoding_system_list = Qnil; 7918 Vcoding_system_list = Qnil;
7173 7919
@@ -7192,7 +7938,7 @@ system bound to the corresponding coding-category is selected. */);
7192 int i; 7938 int i;
7193 7939
7194 Vcoding_category_list = Qnil; 7940 Vcoding_category_list = Qnil;
7195 for (i = CODING_CATEGORY_IDX_MAX - 1; i >= 0; i--) 7941 for (i = coding_category_max - 1; i >= 0; i--)
7196 Vcoding_category_list 7942 Vcoding_category_list
7197 = Fcons (XVECTOR (Vcoding_category_table)->contents[i], 7943 = Fcons (XVECTOR (Vcoding_category_table)->contents[i],
7198 Vcoding_category_list); 7944 Vcoding_category_list);
@@ -7222,23 +7968,27 @@ the value of `buffer-file-coding-system' is used. */);
7222 Vcoding_system_for_write = Qnil; 7968 Vcoding_system_for_write = Qnil;
7223 7969
7224 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used, 7970 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used,
7225 doc: /* Coding system used in the latest file or process I/O. */); 7971 doc: /*
7972Coding system used in the latest file or process I/O. */);
7226 Vlast_coding_system_used = Qnil; 7973 Vlast_coding_system_used = Qnil;
7227 7974
7228 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion, 7975 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion,
7229 doc: /* *Non-nil means always inhibit code conversion of end-of-line format. 7976 doc: /*
7977*Non-nil means always inhibit code conversion of end-of-line format.
7230See info node `Coding Systems' and info node `Text and Binary' concerning 7978See info node `Coding Systems' and info node `Text and Binary' concerning
7231such conversion. */); 7979such conversion. */);
7232 inhibit_eol_conversion = 0; 7980 inhibit_eol_conversion = 0;
7233 7981
7234 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system, 7982 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system,
7235 doc: /* Non-nil means process buffer inherits coding system of process output. 7983 doc: /*
7984Non-nil means process buffer inherits coding system of process output.
7236Bind it to t if the process output is to be treated as if it were a file 7985Bind it to t if the process output is to be treated as if it were a file
7237read from some filesystem. */); 7986read from some filesystem. */);
7238 inherit_process_coding_system = 0; 7987 inherit_process_coding_system = 0;
7239 7988
7240 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist, 7989 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist,
7241 doc: /* Alist to decide a coding system to use for a file I/O operation. 7990 doc: /*
7991Alist to decide a coding system to use for a file I/O operation.
7242The format is ((PATTERN . VAL) ...), 7992The format is ((PATTERN . VAL) ...),
7243where PATTERN is a regular expression matching a file name, 7993where PATTERN is a regular expression matching a file name,
7244VAL is a coding system, a cons of coding systems, or a function symbol. 7994VAL is a coding system, a cons of coding systems, or a function symbol.
@@ -7255,7 +8005,8 @@ and the variable `auto-coding-alist'. */);
7255 Vfile_coding_system_alist = Qnil; 8005 Vfile_coding_system_alist = Qnil;
7256 8006
7257 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist, 8007 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist,
7258 doc: /* Alist to decide a coding system to use for a process I/O operation. 8008 doc: /*
8009Alist to decide a coding system to use for a process I/O operation.
7259The format is ((PATTERN . VAL) ...), 8010The format is ((PATTERN . VAL) ...),
7260where PATTERN is a regular expression matching a program name, 8011where PATTERN is a regular expression matching a program name,
7261VAL is a coding system, a cons of coding systems, or a function symbol. 8012VAL is a coding system, a cons of coding systems, or a function symbol.
@@ -7270,7 +8021,8 @@ See also the function `find-operation-coding-system'. */);
7270 Vprocess_coding_system_alist = Qnil; 8021 Vprocess_coding_system_alist = Qnil;
7271 8022
7272 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist, 8023 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist,
7273 doc: /* Alist to decide a coding system to use for a network I/O operation. 8024 doc: /*
8025Alist to decide a coding system to use for a network I/O operation.
7274The format is ((PATTERN . VAL) ...), 8026The format is ((PATTERN . VAL) ...),
7275where PATTERN is a regular expression matching a network service name 8027where PATTERN is a regular expression matching a network service name
7276or is a port number to connect to, 8028or is a port number to connect to,
@@ -7292,23 +8044,28 @@ Also used for decoding keyboard input on X Window system. */);
7292 8044
7293 /* The eol mnemonics are reset in startup.el system-dependently. */ 8045 /* The eol mnemonics are reset in startup.el system-dependently. */
7294 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix, 8046 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix,
7295 doc: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */); 8047 doc: /*
8048*String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7296 eol_mnemonic_unix = build_string (":"); 8049 eol_mnemonic_unix = build_string (":");
7297 8050
7298 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos, 8051 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos,
7299 doc: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */); 8052 doc: /*
8053*String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7300 eol_mnemonic_dos = build_string ("\\"); 8054 eol_mnemonic_dos = build_string ("\\");
7301 8055
7302 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac, 8056 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac,
7303 doc: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */); 8057 doc: /*
8058*String displayed in mode line for MAC-like (CR) end-of-line format. */);
7304 eol_mnemonic_mac = build_string ("/"); 8059 eol_mnemonic_mac = build_string ("/");
7305 8060
7306 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided, 8061 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided,
7307 doc: /* *String displayed in mode line when end-of-line format is not yet determined. */); 8062 doc: /*
8063*String displayed in mode line when end-of-line format is not yet determined. */);
7308 eol_mnemonic_undecided = build_string (":"); 8064 eol_mnemonic_undecided = build_string (":");
7309 8065
7310 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation, 8066 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation,
7311 doc: /* *Non-nil enables character translation while encoding and decoding. */); 8067 doc: /*
8068*Non-nil enables character translation while encoding and decoding. */);
7312 Venable_character_translation = Qt; 8069 Venable_character_translation = Qt;
7313 8070
7314 DEFVAR_LISP ("standard-translation-table-for-decode", 8071 DEFVAR_LISP ("standard-translation-table-for-decode",
@@ -7321,11 +8078,12 @@ Also used for decoding keyboard input on X Window system. */);
7321 doc: /* Table for translating characters while encoding. */); 8078 doc: /* Table for translating characters while encoding. */);
7322 Vstandard_translation_table_for_encode = Qnil; 8079 Vstandard_translation_table_for_encode = Qnil;
7323 8080
7324 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist, 8081 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_table,
7325 doc: /* Alist of charsets vs revision numbers. 8082 doc: /* Alist of charsets vs revision numbers.
7326While encoding, if a charset (car part of an element) is found, 8083While encoding, if a charset (car part of an element) is found,
7327designate it with the escape sequence identifying revision (cdr part of the element). */); 8084designate it with the escape sequence identifying revision (cdr part
7328 Vcharset_revision_alist = Qnil; 8085of the element). */);
8086 Vcharset_revision_table = Qnil;
7329 8087
7330 DEFVAR_LISP ("default-process-coding-system", 8088 DEFVAR_LISP ("default-process-coding-system",
7331 &Vdefault_process_coding_system, 8089 &Vdefault_process_coding_system,
@@ -7335,7 +8093,8 @@ the cdr part is used for encoding a text to be sent to a process. */);
7335 Vdefault_process_coding_system = Qnil; 8093 Vdefault_process_coding_system = Qnil;
7336 8094
7337 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table, 8095 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table,
7338 doc: /* Table of extra Latin codes in the range 128..159 (inclusive). 8096 doc: /*
8097Table of extra Latin codes in the range 128..159 (inclusive).
7339This is a vector of length 256. 8098This is a vector of length 256.
7340If Nth element is non-nil, the existence of code N in a file 8099If Nth element is non-nil, the existence of code N in a file
7341\(or output of subprocess) doesn't prevent it to be detected as 8100\(or output of subprocess) doesn't prevent it to be detected as
@@ -7347,7 +8106,8 @@ Only 128th through 159th elements has a meaning. */);
7347 8106
7348 DEFVAR_LISP ("select-safe-coding-system-function", 8107 DEFVAR_LISP ("select-safe-coding-system-function",
7349 &Vselect_safe_coding_system_function, 8108 &Vselect_safe_coding_system_function,
7350 doc: /* Function to call to select safe coding system for encoding a text. 8109 doc: /*
8110Function to call to select safe coding system for encoding a text.
7351 8111
7352If set, this function is called to force a user to select a proper 8112If set, this function is called to force a user to select a proper
7353coding system which can encode the text in the case that a default 8113coding system which can encode the text in the case that a default
@@ -7357,14 +8117,16 @@ The default value is `select-safe-coding-system' (which see). */);
7357 Vselect_safe_coding_system_function = Qnil; 8117 Vselect_safe_coding_system_function = Qnil;
7358 8118
7359 DEFVAR_LISP ("char-coding-system-table", &Vchar_coding_system_table, 8119 DEFVAR_LISP ("char-coding-system-table", &Vchar_coding_system_table,
7360 doc: /* Char-table containing safe coding systems of each characters. 8120 doc: /*
8121Char-table containing safe coding systems of each characters.
7361Each element doesn't include such generic coding systems that can 8122Each element doesn't include such generic coding systems that can
7362encode any characters. They are in the first extra slot. */); 8123encode any characters. They are in the first extra slot. */);
7363 Vchar_coding_system_table = Fmake_char_table (Qchar_coding_system, Qnil); 8124 Vchar_coding_system_table = Fmake_char_table (Qchar_coding_system, Qnil);
7364 8125
7365 DEFVAR_BOOL ("inhibit-iso-escape-detection", 8126 DEFVAR_BOOL ("inhibit-iso-escape-detection",
7366 &inhibit_iso_escape_detection, 8127 &inhibit_iso_escape_detection,
7367 doc: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection. 8128 doc: /*
8129If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
7368 8130
7369By default, on reading a file, Emacs tries to detect how the text is 8131By default, on reading a file, Emacs tries to detect how the text is
7370encoded. This code detection is sensitive to escape sequences. If 8132encoded. This code detection is sensitive to escape sequences. If
@@ -7411,4 +8173,3 @@ emacs_strerror (error_number)
7411} 8173}
7412 8174
7413#endif /* emacs */ 8175#endif /* emacs */
7414
generated by cgit v1.2.1 (git 2.18.0) at 2026-03-05 07:22:19 +0000