Chinese word-segmented writing

Chinese word-segmented writing, or Chinese word-separated writing (), is a style of written Chinese where texts are written with spaces between words like written English. Chinese sentences are traditionally written as strings of characters, with no marks between words. Hence, word segmentation according to the context (done either consciously or unconsciously) is a task for the reader.

There are many advantages or reasons of word-segmented writing. An important reason lies in the existence of ambiguous texts where only the author knows the intended meaning and the correct segmentation. For example, "美國會不同意。 美国会不同意。" may mean "美國 會 不同意。 美国 会 不同意。" (The US will not agree.) or "美 國會 不同意。 美 国会 不同意。" (The US Congress does not agree).

History

In ancient China, texts were written without punctuation marks, which led to the reader needing to spend a considerable amount of time finding the boundary of a sentence. It was not until the early 1900s when the present punctuation marks were adopted.

In the 1950s, there was a proposal for the employment of word-segmented writing in a discussion among the Chinese linguists, however it was not passed.

In 1987, the idea of Chinese word-segmented writing was put forward again by Chen Liwei in an international conference on Chinese information processing.

Chinese word-segmented writing was first put into application no later than 1998, when a paper entitled Written Chinese Word Segmentation Revisited: Ten advantages of word-segmented writing was published in a key academic journal in China. The whole paper, seven pages altogether, was written word-segmentedly, with the abstract presented as: 摘要: 单词 的 切分 对 现代 汉语 的 运用、研究 和 计算机 信息 处理 等 都 具有 相当 重要 的 意义。本文 阐述 书面 汉语 分词 连写 的 十 大 好处, 并 讨论 一些 实施 方面 的 问题。文章 全文 分词 连写。

In 2018, a one-paragraph short article was published on Wikiversity entitled Word segmentation of Hanzi, with the Chinese text word-segmented as follows: 历史上，中国古文 是 没有 标点符号的。读者 需要 付出 额外的 精力 专注于 断句，而且 稍有差池 便会 造成 误读。所谓 差之毫厘 失之千里。引入 标点符号 是 一次 重大的 文字改革，使得 汉字文本的 阅读效率 有了 很大的 提高。但 中文的 改革 才 刚刚 起步， 远未达到 尽善尽美的 程度。至少 在 阅读效率 方面 仍然 存在着 一个 显而易见的 障碍 - 断词 （汉字的 分词连写）。

The first book written in word segmentation was 语言理论 (Language theories) published in 2000.

Chinese character word-segmented writing

Chinese is usually written in Chinese characters, so Chinese word segmented writing mainly refers to the segmentation of Chinese character text. The following are some methods or skills.

Textual context

The most important purpose of word-segmented writing is to express the intended meaning of the writer accurately and clearly. For example, the traditional non-word-segmented text "乒乓球拍卖完了。" has two possible meanings, which can be expressed in word-segmented writing as "乒乓 球拍 卖完了。" (Ping pong bats are sold out) and "乒乓球 拍卖 完了。" (The ping pong balls have been auctioned). The author is to make a selection to correctly express the intended meaning without ambiguity.

Dictionaries

If not sure whether a character string is a legal word, the writer can check its existence in a reliable word dictionary, such as Xiandai Hanyu Cidian and CEDICT. Or check whether it is a linguistically qualified word according to lexical, morphological and syntactical knowledge.

Prosody

In spoken language, there is usually a pause between two words (and pause is not allowed within a word), so it is natural to put a pause (represented by a space) between the words in written language.

Methods to identify word boundaries can also be found in Word#Word boundaries.

Whitespace

The space between two words should be set at half the width of a Chinese character, shorter than the distance between two lines. Because the average length of a Chinese word is about 2 characters, if a space is of full width of a Chinese character, longer than the inter-line distance, the lines of words will appear scattered, not compact.

Proper noun marker

To further help the reader, the proper nouns should be marked as well, such as by underlines. In fact this is already done in the Holy Bible (Union Version with modern punctuation).

Pinyin segmentation

Pinyin is usually used to mark the pronunciation of Chinese characters, but in elementary Chinese teaching or teaching Chinese as a foreign language, Pinyin is sometimes used to express Chinese directly. Therefore, Pinyin writing is also a kind of Chinese writing, and it can also be an important reference for Chinese character word segmentation. "Basic Rules of Chinese Pinyin Orthography" is the Chinese national standard for Pinyin writing and word segmentation. Its main content "5. General rules" is excerpted as follows:

Pinyin orthography

The general rules are

1. Use words as the basic writing units for Pinyin expressions. For example: rén (人, person), pǎo (跑, run), māma (妈妈, mother), yuèdú (阅读, read), túshūɡuǎn (图书馆, library).
2. A two-syllable and three-syllable expression of a concept is written consecutively (without spaces). For example: huánbǎo (环保, environmental protection), ɡōnɡɡuān (公关, public relations), chánɡyònɡcí (常用词, commonly used words), duìbuqǐ (对不起, sorry).
3. Names with four or more syllables that represent a concept are written-segmentedly by words or syllables (segments divided by speech pauses inside the phrase). Those that cannot be divided into words or syllables are written consecutively. For example: wúfènɡ ɡānɡɡuǎn (无缝钢管, seamless steel pipe), huánjìnɡ bǎohù guīhuà (环境保护规划, environmental protection planning), Zhōnɡɡuó Shèhuì Kēxuéyuàn (中国社会科学院, Chinese Academy of Social Sciences), yánjiūshēnɡyuàn (研究生院, graduate school), hónɡshízìhuì (红十字会, Red Cross Society)
4. Single-syllable repeating words are to be written consecutively; double-syllable repeating words are written separately. For example: rénrén (人人, everyone), kànkan (看看, look), hónɡhónɡ de (红红的, very red), yánjiū yánjiū (研究研究, research research), xuěbái xuěbái (雪白雪白, snow white snow white). Repeating words in AABB structure are written consecutively. For example: láiláiwǎnɡwǎnɡ (来来往往, coming and going), qīnɡqīnɡchǔchǔ (清清楚楚, crystal clear), fānɡfānɡmiànmiàn (方方面面, all aspects).
5. Monosyllabic prefixes (副 vice, 总 general/chief, 非 non, 反 anti, 超 super, 老 old, 阿 A, 可 able, 无 non, 半 semi, etc.) or monosyllable suffixes (子 zi, 儿 er, 头 man, 性 -ity, 者 person, 员 member, 家 expert, 手 specialist, 化 -ize, 们 plural, etc.) are written consecutively with the main word. For example: fùbùzhǎnɡ (副部长, vice minister), zǒnɡɡōnɡchénɡshī (总工程师, chief engineer), fùzǒnɡɡōnɡchénɡshī (副总工程师, vice chief engineer), fēijīnshǔ (非金属, non-metallic), kēxuéxìnɡ (科学性, scientific / scientificity), chénɡwùyuán (乘务员, flight attendant), xiàndàihuà (现代化, modernization), háizimen (孩子们, children).
6. For the convenience of reading and understanding, a hyphen can be used between some parallel words or morphemes, or in some abbreviations. For example: bā-jiǔ tiān (八九天, eight or nine days), rén-jī duìhuà (人机对话, human-computer dialogue), Jīnɡ-Zànɡ Gāosù Gōnɡlù (京藏高速公路, Beijing-Tibet Expressway).

In addition to the general rules, there are specific rules for nouns, verbs, adjectives, pronouns, numerals, quantifiers, adverbs, prepositions, conjunctions, auxiliary words, interjections, onomatopoeias, idioms, sayings, as well as names of people and places.

Example

Below is an example with a longer text from the Chinese version of the United Nations Universal Declaration of Human Rights:

Article 1 of the Universal Declaration of Human Rights in simplified Chinese characters:

The pinyin transcription can be word-segmented into Rénrén shēng ér zìyóu, zài zūnyán hé quánlì shàng yīlǜ píngděng. Tāmen fùyǒu lǐxìng hé liángxīn, bìng yīng yǐ xiōngdì guānxì de jīngshén xiāng duìdài. Accordingly, the Chinese character text can be segmented into 人人 生 而 自由，在 尊严 和 权利 上 一律 平等。 他们 赋有 理性 和 良心， 并 应 以 兄弟 关系 的 精神 相 对待。

Computer-based word segmentation

Before word-segmented writing was popularized, computer-based word segmentation was often used for language information processing. Although the quality of such systems has improved over time, manual post-editing is still required.

References