Classical Chinese Prompts Spark Debate Over AI Token Savings

Because Classical Chinese can convey more meaning with fewer characters, some users see it as a possible way to reduce the cost of generative AI. But companies including Claude developer Anthropic largely optimize their tokenizers for English-language data, meaning Chinese characters may not require fewer tokens. A lower character count does not necessarily reduce API charges and could come at the expense of a model's comprehension and reasoning quality.

A recent screenshot of a Classical Chinese conversation with Claude sparked debate among engineers. Tests cited in the discussion suggested that the same prompt in English is generally more tokenizer-efficient and may use fewer tokens than its Classical Chinese equivalent. The report provided no standardized test date, model version, token count or cost. Security concerns were also raised that mainstream models receive less safety training in Classical Chinese, potentially making safeguards easier to bypass.