High-DPI, Subpixel Text Positioning, Hinting

What happens when an unstoppable bullet hits an impenetrable wall?

A while back I wrote a document about the interactions of high-density displays and font rendering options.  I just went ahead and made it public, so, enjoy!

High-DPI, Subpixel Text Positioning, Hinting

Update (July 18, 2013): It was brought to my attention that the document had disappeared. I have no idea how that happened, but I've uploaded it again. It also means that the URL has changed. :(

Labels: fonts, hinting, textlayout, typography

¶ 5:08 PM   5 comments

 Can I has intel tablet?

So I missed the intel party it seems. I assume the tablets are GLES2-ready and have a fair pixel density. If that is the case, I can make good use of one for OpenGL-based text rendering I'm experimenting with. If there are leftovers at the summit, I'd happily take one!

kthxbye :)

Labels: desktopsummit, guadec, intel, pango, textlayout

¶ 4:45 AM   0 comments

 Pango vs HarfBuzz

Since the rewritten HarfBuzz is shaping up fast and getting lots of Buzz these days, I get asked the same question again and again: "Will HarfBuzz replace Pango?" This post tries to answer that.


Short answer: No, not at all! Pango is here to stay. It will change, but only get better.


Long answer:

Pango provides two levels of API: A low-level and a high-level.

Low level API: What I can the "three pillars of pango":

• pango_itemize(): Breaks text into runs that each have the same font, Unicode script, language, direction, and other characteristics.
• pango_shape(): Shapes a single run of text, given the font, script, language, direction, and other properties. Shaping means converting Unicode text to positioned glyphs.
• pango_break(): Does line breaking and other text segmentation (cursor positions, cluster boundaries, word boundaries, and sentence boundaries).

High-level API: Pango's high-level API consists of the PangoLayout object, aka "here's a piece of text render it in this box I don't care what you do."

Of these, HarfBuzz only does shaping. That is, hb_shape() is functionally equivalent to pango_shape().


API implications: Here is how moving to HarfBuzz affects the Pango API:

• Everything in pango-ot.h will be deprecated and be a thin wrapper around hb-ot.h. This is already done in the harfbuzz-ng-external branch of Pango.
• There will be new API in Pango, perhaps in pango-hb.h to help extracting various HarfBuzz structures from their Pango equivalents.
• pango_shape() will be a thin wrapper around hb_shape() (read below).

Pango Modules: pango_shape() calls into Pango shaper modules to get the actual shaping done. There are two kinds Pango shaper modules depending on what they do (the API is the same, so Pango doesn't differentiate between the two classes):

• Bridge modules: The basic-win32.c, basic-atsui.c modules call into another, platform native, shaping system to get the work done. The external (not integrated in Pango yet) modules basic-graphite.c and basic-m17n.c also do the same for the SIL Graphite and m17n shaping libraries.
• On Linux, since there currently is no native shaping engine, Pango has multiple shaping modules, one per script, to do the actual shaping (arabic-fc, syriac-fc, indic-fc, thai-fc, ..., and basic-fc for all the non-complex scripts).

Now, as HarfBuzz becomes the shaping engine on Linux, all those script-specific modules will be removed and basic-fc will simply call into hb_shape(). That's indeed what the basic-fc.c in the harfbuzz-ng-external does.

Later on, when we add support for native win32, CoreText, Graphite, and m17n to HarfBuzz, all those other modules will also be replaced by HarfBuzz-calling equivalents.

Which one to use: Pango or HarfBuzz? Depends.

PangoLayout is designed to be the 'render this text in this box I don't care how' kind of API. That's a perfect fit for GUI toolkits like GTK+, but not suitable for lots of other uses, for example:

• Web browsers
• Word processors
• Designer tools
• Font design tools
• Terminal emulators
• Batch document processors
• TeX engines

while in many of those cases PangoLayout can be made to work (with much pain, mind you), Pango still provides the lower level API and lots of other bits and pieces to get something going. What it doesn't give full control on however is font selection, which happens to be a deal-breaker for many of those usecases (browsers following CSS rules, etc).

So, each of those kinds of applications need to assess the pros and cons of using Pango vs using HarBuzz and providing all the other bits themselves. For example, HarfBuzz doesn't provide:

• An itemizer
• A Unicode Bidirection Algorithm implementation
• A Unicode Line Breaking implementation
• Glyph rasterization
• Glyph metrics information
• etc

There's also a hybrid use possible: to borrow those pieces from Pango on platforms that it's feasable, but drive HarfBuzz directly. It all depends. When in doubt, ask! We have a mailing list.

That said, Firefox will use HarfBuzz as soon as it's ready (there are patches circulating around). Google is using old HarfBuzz for their Webkit and will port to the new one. I'm also attending the Webkit-GTK hackfest in December to port that to the new HarfBuzz. We'll work towards sharing the HarfBuzz-dealing code among Webkit backends.

This is already a long post. Let me finish now. Hope I made it a tiny bit more clear.

Labels: gnome, harfbuzz, pango, textlayout

¶ 8:01 PM   7 comments

 HarfBuzz HackFest

Here is a quick update re HarfBuzz:

During May and August I finished rewriting the OpenType Layout engine to use mmap()ed font files. This is in Pango 1.26.x already. Pango and fontconfig also received a lot more optimization love. That deserves a long and separate blogpost. The net result is that the text stack's memory usage is considerably lower now. All this goodness will be in the upcoming Fedora 12.

In October, I attended the 33rd Internationalization and Unicode Conference in San Jose to present the free software text stack (useless slides) as well as present and promote HarfBuzz (useless slides). That was a very fruitful event and I received lots of interest from many major industry players. With the liberal license that we are releasing HarfBuzz under, we expect broad adoption, which is exactly what we are looking for.

This week, Jonathan Kew and myself are having a small HarfBuzz HackFest here in Mozilla's Toronto office. Here's what we have got done so far:

• Jonathan has a version of Firefox using harfbuzz-ng (the codename for the rewrite) that has advanced layout features controlable through CSS. Very very cool stuff. He updated it to the latest harfbuzz-ng code.
• I ripped harfbuzz-ng out of the Pango tree and into a standalone module. Finally! Took a couple hours of git surgery plus ten minutes to put together an autotools build system. Git clone URL is this. The harfbuzz-ng-external branch in Pango uses that as an external module. The plan is to reach a stable 1.0 release of harfbuzz-ng before next stable GNOME and most probably, Pango will require harfbuzz unconditionally (that is, on all platforms). Note that harfbuzz is NOT tied to FreeType, so you can use it with any rasterizer you have around.
• We fixed all portability issues Jonathan had faced when compiling harfbuzz-ng with MSVC.
• Jonathan is working on the shaper side, while I'm working on the API and pulling it all together.
  
• I added glue code for using harfbuzz-ng with glib, ICU, and FreeType.
• Lots of API and design review.

At the rate this is developing, by the end of the week we should have basic shaper (Latin, Cyrillic, CJK, ...) and Arabic+Syriac working perfectly and tackling Indic family. We're closer to 1.0 than you may think!

Labels: hackfest, harfbuzz, pango, textlayout

¶ 11:55 AM   5 comments

 On Nokia, Qt, and what not...

Not sure how to feel about the Nokia-Qt deal. If they can use it, congrats to my fellow hackers at Qt.

Philip:I've been working in my capacity as the Pango maintainer with the Qt guys for almost two years now. We are moving all our secret-recipe shaping code to a single module hosted on freedestkop.org and called HarfBuzz. (more on that module later)

We arrange annual conferences called Text Layout Summit to meet and discuss. The first one happened at GNOME Boston Summit 2006, second one at aKademy 2007, and the next instance is scheduled to happen at the Libre Graphics Meeting 2008.

Really, we work together all the time. You just need to want to. And meeting face to face gives you that needed initial push. That's the main reason that I'm working hard on getting GNOME Foundation and KDE e.V. co-hosting aKademy and GUADEC in 2009.

Labels: gnome, harfbuzz, kde, textlayout

¶ 11:19 AM   4 comments

 Text Layout Summit, First Day

I left my apartment Tuesday 3pm and arrived in London 7:30am. First look at London, fabulous. Everything's kinda different, yet not much different. From the British accent, to the driving on the left, to the license plates, to measurements in yards... It is a nice weather here, sunny, warm, still lots of gray clouds. Kinda like Whistler's dark gray London has got to be...

Entering UK was very smooth except for the hour-long line at passport control. Am on the bus to Stansted airport to catch my RyanAir flight to Glasgow. It's 10am now; Text Layout crew should be almost done with the keynote. Totally missing on all the presentations in the morning :(.

Umm, was hacking all the time in the flight. Hope that doesn't ruin the rest of my day. It's just 5am EST.

Time passes...

Arrived in aKademy / Text Layout summit at 3pm local time. Scotland is a funny place with it's own details too. Really like to post photos I took on the way but it's too hard right now. It's about 24 hours with no sleep. Am doing Ok though, we'll see. Everyone's here and they were so nice they put all the interesting talks for the afternoon so I'll be here. Thanks you!

Quite interesting to meet Eric Mader and Kenichi Handa, attaching faces to names and all that. Also Simon Hausmann (Qt) is here too, so we probably sit down and write a Pango module to use the Qt's HarfBuzz shaper stuff tomorrow instead of going to the BBQ picnic that is in the schedule! I'll write about HarfBuzz in detail later.

On another note, Richard Stallman and Bjarne Stroustrup will be talking in Toronto while I'm away :(.

Got to go talk. Later.

More time passes...

6:30pm now, calling it a night, heading for dinner. Looking forward to lots of hacking tomorrow...

Labels: akademy, glasgow, harfbuzz, textlayout, uk

¶ 1:48 PM   0 comments

 UK

My UK visa arrived this morning at 9. I sent the package out to Ottawa the day before yesterday, in the afternoon! Very, very, nice of them.

So I'm going from my deferred schedule to original schedule. all things going as planned (and not much is left), I'll be at Text Layout 2007 / aKademy, LUG Radio Live!, and last but not least, GUADEC.

Interviews:

Interview with Hans Reiser in prison. Quite touching, particularly the closing.

Mehdi, an Iranian Free Software evangelist interviewed me a few weeks ago, it's up on Hezardastan (Persian).

Labels: akademy, guadec, lrl, reiser, textlayout, travel, uk

¶ 6:14 PM   3 comments