Arabic Joining Rendering Problem

I'm been thinking about this weird problem that FarsiWeb fonts look bad at small sizes. The effect was like some glyphs being placed one pixel above where they should be and so an unwanted shift when they join to the neighboring glyphs. I discovered what's going on when accidentally rendered a sample text in huge font with inverse colors. A reworked sample is the image exhibited in this post. See it at full size here. If you don't see the problem, try enlarging the photo.

The problem is two-fold, and by far one of the hardest problems I have faced so far in Persian computing. In short: A) The glyphs in the font have tails spanning out of the glyph extents. B) When rendered with anti-aliasing on, the overlapped section becomes darker, due to being rendered twice.

I need to elaborate on why glyphs have those long tails in the first place. Since anti-aliased rendering is a relatively new phenomenon and many older renderers have had rounding bugs, etc, Arabic font designer have mostly opted for have glyphs that overlap, to make sure no white gap shows up at the joint. I have witnessed such a problem myself with DVI/PS/PDF viewers quite a few times while working on FarsiTeX. The solution again, have been to mechanically add some extra tail to make sure no gap is rendered. But now with anti-aliased rendering, it's causing problem.

The overlapping glyphs is what Keith Packard calls a broken font. Seems like Xft already takes extra care of joining glyphs, pre-adding the joints such that they don't look weird. So all we need is fonts that are not broken, which is, well, reasonable. Seeking for another fix may fail miserably, since otherwise you need to either 1) use conjoint operators to add the glyphs to a temporary surface and compose it to destination afterward, but keithp says even that doesn't quite work, or 2) merge all the glyph paths and draw them in one operation. This one sucks performance-wise, sine each glyph will be drawn from path every time, instead of the usual fast path of getting FreeType render glyph alpha masks that are cached on the server by Xft...

But there's one more reason to have broken fonts: We use that in our justification algorithm in Persian TeX systems. To implement proper Arabic justification in TeX, there are two ways: 1) use rules (black boxes), this is the easiest, but suffers from the same broken-viewers problem: the rules will be rendered with solid black edges, while neighboring edge from the glyphs is antialiased. 2) use cleaders to repeat a narrow joining glyph. This eats a lot of resources in the final output and rendered, but at least looks better. But since cleaders insert an integer number of those glyphs, we need to somehow fill the remainder, and we do that by having glyphs with joining tails extended out of their box. Is there another way to do justification without broken fonts? Maybe. With a PostScript backend for example, one may be able to write PS hacks to convert an stretchable rule or glue into an scaled joining glyph. Or we may be able to add support for that to pdfTeX. They have recently added a bunch of types, like floating point registers, etc. They are quite open to useful extensions. It may even be solved by PangoTeX, by the time it happens, Pango would definitely have support for Arabic justification :-).

There is a partial third solution: Use hinting to make sure the baseline of the font occupies a full number of aligned pixels, such that double-painting doesn't make any difference. But that seems to be harder to achieve than just cutting the tails.

¶ 6:18 AM