Moving off arabic_reshaper¶
For years, the way to get Arabic into Pillow and matplotlib was:
import arabic_reshaper
from bidi.algorithm import get_display
draw.text((x, y), get_display(arabic_reshaper.reshape(text)), font=font)
arabic_reshaper.reshape swaps each letter for its joined shape (Arabic “presentation
forms”, such as ﻣ for an initial meem), and get_display reverses the line into the
order it is shown in. Both steps were needed because Pillow and matplotlib couldn’t
shape Arabic themselves.
They can now, which breaks that recipe:
Pillow shapes Arabic when it has Raqm. Many Linux images already have it, and Pillow is working on Raqm in all its wheels.
matplotlib 3.11 and later shape Arabic in their own wheels.
OpenCV 5.1 and later shape Arabic in
cv2.putText. Pre-shaped text comes out as?with its built-in font, or backwards with a font of your own.
Given pre-shaped text, they shape it a second time. The letters come out unjoined or in the wrong order, often only on some machines, so it can look fine on yours and broken in production.
Find it¶
khatt --doctor shows which shapers your environment has. Give it your project, and it
also lists every place that pre-shapes Arabic:
$ khatt --doctor src/
...
pre-shaping Arabic:
src/render.py:44: return get_display(arabic_reshaper.reshape(text))
warning: 1 place(s) in src/ pre-shape Arabic (listed above). ...
It exits with status 1 when it finds any, so it can guard a project in CI. As a pre-commit hook:
repos:
- repo: https://github.com/DrEssam97/khatt
rev: v0.14.0
hooks:
- id: khatt-doctor
args: ["src/"] # the folder to scan; the default is the whole project
From Python: khatt.find_reshaping("src/"), or khatt.doctor("src/").
Fix it¶
Pass the text as typed to something that shapes it once.
With khatt, the output is the same whether or not Pillow has Raqm:
import khatt
khatt.draw_text(image, (x, y), text, font="amiri", size_px=48)
Or keep your ImageDraw code and change only the call: khatt.pillow.text(draw, xy, text, ...) takes draw.text’s arguments, and turns already-reshaped strings back
before drawing them. For a library you can’t edit, wrap its calls in
with khatt.pillow.routing():.
Or, where you know the library shapes Arabic (Pillow with Raqm, matplotlib 3.11+,
OpenCV 5.1+), drop both calls and give it text itself.
When Pillow shapes Arabic everywhere¶
Pillow is working on Raqm in all its wheels, Windows and macOS included. On the day that
ships, every get_display(arabic_reshaper.reshape(text)) passed to ImageDraw.text
starts drawing backwards, on every machine at once, after a routine pip install -U.
Three steps get you ready now:
Run
khatt --doctor .(or add the hook above) to list every place that pre-shapes.Replace those calls with
khatt.draw_text, which shapes the text once whatever Pillow has, or give Pillow the plain text where you know it has Raqm.Run
khatt.unreshape()over any reshaped strings you have stored.
Text that was already reshaped¶
Strings saved after arabic_reshaper (in a database, a JSON file, subtitles) can be
turned back:
plain = khatt.unreshape(saved)
Presentation forms go back to letters: a lam-alef ligature to lam and alef,
ﷲtoالله. Symbols typed on purpose, such asﷺand the ornate brackets﴿ ﴾, stay.Lines that went through
get_displayare put back in typed order. Whether a line did is told from its letter shapes; where they can’t tell (short words, lam-alef), it counts as having gone throughget_display, the usual recipe. Passdisplay_order=Falseif you only reshaped.A mostly Arabic line that starts with a Latin word looks the same, once shown, as one that ends with it. Those come back right to left; pass
direction="ltr"if yours start with the Latin word.arabic_reshaperdrops harakat by default, so they can’t come back.Multi-line text that went through
get_displayin one piece can’t be put back: python-bidi moves its line breaks. Text shown one line at a time is fine.khatt.is_reshaped(text)says whether a string has presentation forms. khatt warns (khatt.ReshapedTextWarning) when it is asked to draw one.