How to use custom application fonts with Pango

I am at the Libre Graphics Meeting in Toronto this week, which means that I got to talk to GIMP and Inkscape developers after many years, and was reminded that Pango still does not make it easy to use custom (aka. application) fonts, and it still does not allow turning OpenType features on or off.  So I decided to give Pango some love.

OpenType features is bug 738505.  Akira and Matthias wrote the initial patch, but there are certain complexities in handling the attributes that needs to be fixed before this can go in.  I'll see if I can get myself to do that tomorrow.

Custom fonts is a different issue.  And by custom fonts I mean when an application wants to use a font file that it ships, but is not installed in the system or user font directories.  Most of the times when people have this request, they also don't want any system fonts, ie, they only want their custom fonts.  A font viewer is a basic example of this that we never had a good solution for.

Webfonts, and other embedded fonts, are another use case.  Eg, a document might bundle fonts that it uses.  Many times, the document would want to refer to the font using font family name, so in that case you want the font to be added to the system fonts and go through the font matching process.

Back in 2006 when we considered pangocairo the only backend we care about, I proposed that we add pangocairo API to use a certain cairo_face_t.  The thinking was that by doing it in the pangocairo layer, we don't have to do it in individual platform backends (pangofc, pangowin32, pangoatsui / pangocoretext).  That, however, never happened, because I was too lazy to implement it, but also because it was actually a lot of tricky work, to make the generic pangofc layer understand this custom object...

I have since changed my mind on how this should be done.  Over the years different groups asked how they do this, I've had suggested different variations of the same solution: custom Fontconfig config; and Win32 / CoreText APIs on other platforms.  Ie, do it at font host layer, not Pango layer.  It's a legitimate approach, if not most convenient.

Here is one way to do it, using fontconfig API:

Before calling into Pango, do this:

• Create a new fontconfig configuration object using FcConfigCreate() and make it current using FcConfigSetCurrent(); this ensures that default configuration and fonts are not loaded,
• Add fonts to it using FcConfigAppFontAddFile() or FcConfigAppFontAddDir(),
• Use Pango as you otherwise would.

This way Pango will only see your fonts.  If you want to add your fonts but also see the system fonts, you can skip the first step.  That will make your custom fonts visible to all Pango users within the process, including Gtk+ and its font selection dialog.  If, instead, you want to limit it to a particular document, we need something more involved; we talk about further down.

Recently I needed to do something similar in Noto's test suite, which is written in Python.  Since we currently don't have fontconfig Python bindings, I either would have had to be bothered to do a ctypes binding of the few functions I needed, or find an even simpler solution.  Which is:

Before calling into Pango, do this:

• Set the FONTCONFIG_FILE environment variable to point to your custom fonts.conf XML,
• Add your custom font directory to the above-mentioned XML.

That's it.  Here's are the two steps for Noto: 1 and 2.

Now, this sounds easy, and works for very limited usecases.  But for it to be useful in apps like The GIMP or Inkscape, there are a few issues that need to be handled.  And I'm writing this post to raise enough interest from Akira, Matthias, Khaled, and others, to help fix these so we can have a great custom-font experience.  Writing a tutorial when these are all fixed would be great, but for now, this post is documentation enough!

First, the custom XML currently has to have a cachedir element or fontconfig warns.  That's annoying to say the least.  Filed here.  For now, you can use "<cachedir prefix="xdg">fontconfig</cachedir>" and that should work with recent-enough versions of fontconfig that understand the prefix="xdg".  Older version will ignore it and try to create a fontconfig directory under the current directory and use as cache, so beware of that.

Python / JS / etc bindings for fontconfig will be useful if we are pushing in this direction.  Filed here.

Another problem with the suggested setups above is that it works for non-GTK use-cases, like a game, test suite, etc.  But in most usecases, you definitely don't want your menus and other GUI elements to use the custom fonts.  In the case of a document editor or graphics editor, you might want to add custom fonts per document.  You can do that by using a separate PangoFcFontMap instance for each custom need.  I outlined that like this before:

1. FcConfigCreate()
2. FcConfigAppFontAddFile()
3. pango_cairo_font_map_new_for_font_type()
4. Use that PangoFontMap to create your context, etc.
5. Every time you want to use Pango with that context, FcConfigSetCurrent the above config.  Then reset it back to whatever it was before.

This will allow you to use Pango with custom-only fonts in parts of the application.  If you also want the system fonts to be visible in this private font map, you can use the obscurely named FcInitLoadConfigAndFonts() instead of FcConfigCreate().

That's not awfully bad.  Except that the last step is very cumbersome and a recipe for disaster.  That step can be avoided if we have API to attach a custom FcConfig to a PangoFcFontMap.  Owen and I talked about it years ago and I filed a bug, but never happened... until yesterday.

I added pango_fc_font_map_set_config() yesterday, and just added pango_fc_font_map_config_changed().  So you can attach your private FcConfig to a private PangoFcFontMap, inform the font map every time you change the FcConfig (eg, add more fonts to it), and you should be able to use that font map to create layouts and use normally, without switching FcConfig's all the time.  However, remember: I wrote the code, but I didn't test it.  So a brave first user is wanted.

Then there's the set of issues with Fontconfig; the fact that to add font or configuration to an FcConfig, you need to have those in files, and can't add from memory blobs.  Fixing that should be fairly easy, and I filed those last year.  Would be great if we can fix them soon.  Here's one for font blobs, and another for configuration XML.

Another issue with using custom fonts is that every time you add a custom font, Fontconfig has to scan it.  Ie, you get no benefit from the Fontconfig font cache.  This can be really slow for huge fonts.  And became five times slower when the Adobe CFF rasterizer was integrated in FreeType.  I tried a few different approaches to speed it up.  Removing the (prematurely added) FC_HASH helped.  But there's more that can be done.  That bug is here.

Now, here's the one remaining issue that I don't have a full answer for: by sidestepping the default fontconfig configuration, you will miss some essential features.  Right now those are: synthetic italics, synthetic bold, and scaling of bitmap fonts.  The reason is that these are encoded in configuration files instead of in the library.  That sounds slightly over-engineered, and I like to improve it, but as of right now, that is the way it is.  You also will lose system-wide and user's configuration.

Sometimes that's not a problem.  For example, in the Noto test suite, we don't want any system or user configuration or any synthetic emboldening or italics, so the FcConfigCreate() approach suites those kinds of scenarios perfectly well.  Same about the preview part of a font-viewer, or other use cases where being system-independent is a goal.  But in other situations it might not be desirable.

I can, of course, suggest that anyone creating a custom FcConfig to add the essential configuration to it; Eg. bitmap scaling, synthetic bold and italic, essential aliases, user's configuration.  But that still leaves out generic aliases, system config, etc.  And I don't like that approach, because that spreads our default configuration all around Fontconfig clients and will create a mess that we would have to clean up every time we change how we do configuration.  It's a leaky abstraction.

Quite fortunately, there's an almost-perfect solution for that.  The obscure FcInitLoadConfig() function creates a new FcConfig and loads the default configuration files, but does not load any fonts referred to from those configuration files.  As such, you can go ahead and add your own fonts to it, but still benefit from the configuration.  Don't complain, if the configuration does undesirable font aliasing or other stuff.  I'm hugely relieved that this might do exactly what we need.  I have not tested this, so if you test, please kindly let me know how it works.  I cannot think of obvious undesirable behaviors from this approach.

So, to wrap up, assuming you would use Pango to be released soon, this is how you do custom fonts with PangoCairoFc:

1. Create a custom FcConfig:
   
   • Use FcConfigCreate() if you don't want any system / user fonts or configuration whatsoever, neither you want any synthetic or other manipulations,
   • Use FcInitLoadConfig() if you want system / user configuration, but no system / user fonts,
   •  Use FcInitLoadConfigAndFonts() if you want system / user configuration and fonts visible in your private font map,
2. Call FcConfigParseAndLoad() to add any custom configuration you want to add.  Such configuration can add custom font directories, or you can use next step,
3. Call FcConfigAppFontAddFile() or FcConfigAppFontAddDir() to add custom fonts,
4. Call pango_cairo_font_map_new_for_font_type(CAIRO_FONT_TYPE_FT), to create a private PangoCairoFcFontMap, which is a subclass of PangoFcFontMap, which is a subclass of PangoFontMap,
5. Call pango_fc_font_map_set_config() to attach your custom FcConfig to your private PangoFcFontMap,
6. Call pango_fc_font_map_config_changed() whenever you add new fonts to your custom FcConfig,
7. Use that PangoFontMap to create your context, etc, and use normally.

You can optimize the logic to use the default font map if there are no custom fonts involved, and do the above otherwise.

This can trivially be adapted to the PangoFT2 backend by the way.

If your application has its own font dialog, then you can implement that using the private font map as well, and things should work.  But if you use the GTK+ font dialog, currently you can't attach your private font map to the font dialog instance.  I filed a request for that.

While there, let me address the issue of having or not having to restart applications when new fonts are installed.  As I covered in my State of Text Rendering in 2009 and presented at the Gran Canaria Desktop Summit, online font addition/removal should Just Work in GNOME.  I implemented that in 2008.  Indeed, it does in simple applications.  They are a bit slow (~4 seconds before new font shows up), so I filed a bug to shorten the delay.  However, a lot is not working:

• I implemented this in the GTK+ font-chooser dialog in 2009, to update online as fonts were added / removed.  But font-chooser was rewrote in 2011 and is completely broken now.  So I filed a new bug (UPDATE: this is fixed now),
• I wasn't loud enough in making this feature heard or document how custom font dialog widgets can implement this.  Take a look at this deprecated font dialog implementation in GTK+.  Or watch the new bug for discussion and how this will evolve,
• To implement this for non-GTK+-based applications, you need to catch the XSETTINGS signal yourself and respond; search for Fontconfig/Timestamp in GTK+ source,
• If you create private fontmaps and FcConfig's that include system / user configs or fonts (ie, you used FcInitLoadConfig() or FcInitLoadConfigAndFonts()), then you also need something similar to the above code in GTK+, that is, catch the signal, recreate your FcConfig (and add your custom fonts to it again), and attach the new FcConfig to your PangoFcFontMap.

As for pangocairo API to use a cairo_font_face_t, I still think that would be useful, but much less so than before.  Also, it won't be very useful until cairo adds API to create a cairo_font_face_t from a font file or memory blob.  Right now the only way to do that is to write code for the FreeType, Win32, and ATSUI / CoreText font backends of cairo separately.

Even if you try to do that, you will hit a very well-known problem with AddFontMemResourceEx(); which is: if you try to add a custom font with a family name that matches that of an existing font on the system, when you try to use the font, you might end up using the version installed on the system.  Ie. you can add fonts to the system, but you can't reliably address them.  The hack that Firefox, HarfBuzz, and others use to work around this is to modify the font data before calling AddFontMemResourceEx() to set a unique font family name on it, so they can be sure there will be no collision.  Here's the HarfBuzz code for that.

Anyway, I hope this write up helps developers, after confusing them, implement custom fonts in their applications, and motivate others to help me fix remaining issues and generate examples and better documentation.

While writing this, I ended up doing some bug triage and closing obsolete issues around Pango, Fontconfig, and cairo, which is always nice :D.

Anyway, this alone makes me really happy that I attended LGM.  The HarfBuzz Documentation Sprint was also very productive, but takes a couple more weeks to get to a stage that we can show off what we produced.

Labels: fonts, gtk+, pango, pangocairo

¶ 7:15 PM   3 comments

 Can I has intel tablet?

So I missed the intel party it seems. I assume the tablets are GLES2-ready and have a fair pixel density. If that is the case, I can make good use of one for OpenGL-based text rendering I'm experimenting with. If there are leftovers at the summit, I'd happily take one!

kthxbye :)

Labels: desktopsummit, guadec, intel, pango, textlayout

¶ 4:45 AM   0 comments

 Arrived at the Desktop Summit

Just got to the Desktop Summit in Berlin. It's lovely seeing everyone after two years.

I'm running Text Layout Summit August 9th to 12th, so come find me for some font and text chat!

Labels: berlin, gnome, harfbuzz, pango

¶ 8:24 AM   2 comments

 June

June 18th was my last day at Red Hat.

I spent last week road-tripping to Eastern Canada with my friends.

In a couple of weeks I will start working for Google Canada in the Waterloo office. I will be working for a large part on HarfBuzz as part of the Chrome / ChromeOS team.

Unfortunately the setup also means that I have to skip this year's GUADEC as I can't get a visa on time :(.

I will keep my current GNOME duties, namely maintaining the text stack (fribidi, fontconfig, harfbuzz, pango, etc) as well as vte. The break may give me some time hacking on things I couldn't get the time to hack on before even. We'll see.

That's all for June.

Labels: gnome, pango, vte

¶ 4:39 PM   15 comments

 Pango vs HarfBuzz

Since the rewritten HarfBuzz is shaping up fast and getting lots of Buzz these days, I get asked the same question again and again: "Will HarfBuzz replace Pango?" This post tries to answer that.


Short answer: No, not at all! Pango is here to stay. It will change, but only get better.


Long answer:

Pango provides two levels of API: A low-level and a high-level.

Low level API: What I can the "three pillars of pango":

• pango_itemize(): Breaks text into runs that each have the same font, Unicode script, language, direction, and other characteristics.
• pango_shape(): Shapes a single run of text, given the font, script, language, direction, and other properties. Shaping means converting Unicode text to positioned glyphs.
• pango_break(): Does line breaking and other text segmentation (cursor positions, cluster boundaries, word boundaries, and sentence boundaries).

High-level API: Pango's high-level API consists of the PangoLayout object, aka "here's a piece of text render it in this box I don't care what you do."

Of these, HarfBuzz only does shaping. That is, hb_shape() is functionally equivalent to pango_shape().


API implications: Here is how moving to HarfBuzz affects the Pango API:

• Everything in pango-ot.h will be deprecated and be a thin wrapper around hb-ot.h. This is already done in the harfbuzz-ng-external branch of Pango.
• There will be new API in Pango, perhaps in pango-hb.h to help extracting various HarfBuzz structures from their Pango equivalents.
• pango_shape() will be a thin wrapper around hb_shape() (read below).

Pango Modules: pango_shape() calls into Pango shaper modules to get the actual shaping done. There are two kinds Pango shaper modules depending on what they do (the API is the same, so Pango doesn't differentiate between the two classes):

• Bridge modules: The basic-win32.c, basic-atsui.c modules call into another, platform native, shaping system to get the work done. The external (not integrated in Pango yet) modules basic-graphite.c and basic-m17n.c also do the same for the SIL Graphite and m17n shaping libraries.
• On Linux, since there currently is no native shaping engine, Pango has multiple shaping modules, one per script, to do the actual shaping (arabic-fc, syriac-fc, indic-fc, thai-fc, ..., and basic-fc for all the non-complex scripts).

Now, as HarfBuzz becomes the shaping engine on Linux, all those script-specific modules will be removed and basic-fc will simply call into hb_shape(). That's indeed what the basic-fc.c in the harfbuzz-ng-external does.

Later on, when we add support for native win32, CoreText, Graphite, and m17n to HarfBuzz, all those other modules will also be replaced by HarfBuzz-calling equivalents.

Which one to use: Pango or HarfBuzz? Depends.

PangoLayout is designed to be the 'render this text in this box I don't care how' kind of API. That's a perfect fit for GUI toolkits like GTK+, but not suitable for lots of other uses, for example:

• Web browsers
• Word processors
• Designer tools
• Font design tools
• Terminal emulators
• Batch document processors
• TeX engines

while in many of those cases PangoLayout can be made to work (with much pain, mind you), Pango still provides the lower level API and lots of other bits and pieces to get something going. What it doesn't give full control on however is font selection, which happens to be a deal-breaker for many of those usecases (browsers following CSS rules, etc).

So, each of those kinds of applications need to assess the pros and cons of using Pango vs using HarBuzz and providing all the other bits themselves. For example, HarfBuzz doesn't provide:

• An itemizer
• A Unicode Bidirection Algorithm implementation
• A Unicode Line Breaking implementation
• Glyph rasterization
• Glyph metrics information
• etc

There's also a hybrid use possible: to borrow those pieces from Pango on platforms that it's feasable, but drive HarfBuzz directly. It all depends. When in doubt, ask! We have a mailing list.

That said, Firefox will use HarfBuzz as soon as it's ready (there are patches circulating around). Google is using old HarfBuzz for their Webkit and will port to the new one. I'm also attending the Webkit-GTK hackfest in December to port that to the new HarfBuzz. We'll work towards sharing the HarfBuzz-dealing code among Webkit backends.

This is already a long post. Let me finish now. Hope I made it a tiny bit more clear.

Labels: gnome, harfbuzz, pango, textlayout

¶ 8:01 PM   7 comments

 HarfBuzz HackFest

Here is a quick update re HarfBuzz:

During May and August I finished rewriting the OpenType Layout engine to use mmap()ed font files. This is in Pango 1.26.x already. Pango and fontconfig also received a lot more optimization love. That deserves a long and separate blogpost. The net result is that the text stack's memory usage is considerably lower now. All this goodness will be in the upcoming Fedora 12.

In October, I attended the 33rd Internationalization and Unicode Conference in San Jose to present the free software text stack (useless slides) as well as present and promote HarfBuzz (useless slides). That was a very fruitful event and I received lots of interest from many major industry players. With the liberal license that we are releasing HarfBuzz under, we expect broad adoption, which is exactly what we are looking for.

This week, Jonathan Kew and myself are having a small HarfBuzz HackFest here in Mozilla's Toronto office. Here's what we have got done so far:

• Jonathan has a version of Firefox using harfbuzz-ng (the codename for the rewrite) that has advanced layout features controlable through CSS. Very very cool stuff. He updated it to the latest harfbuzz-ng code.
• I ripped harfbuzz-ng out of the Pango tree and into a standalone module. Finally! Took a couple hours of git surgery plus ten minutes to put together an autotools build system. Git clone URL is this. The harfbuzz-ng-external branch in Pango uses that as an external module. The plan is to reach a stable 1.0 release of harfbuzz-ng before next stable GNOME and most probably, Pango will require harfbuzz unconditionally (that is, on all platforms). Note that harfbuzz is NOT tied to FreeType, so you can use it with any rasterizer you have around.
• We fixed all portability issues Jonathan had faced when compiling harfbuzz-ng with MSVC.
• Jonathan is working on the shaper side, while I'm working on the API and pulling it all together.
  
• I added glue code for using harfbuzz-ng with glib, ICU, and FreeType.
• Lots of API and design review.

At the rate this is developing, by the end of the week we should have basic shaper (Latin, Cyrillic, CJK, ...) and Arabic+Syriac working perfectly and tackling Indic family. We're closer to 1.0 than you may think!

Labels: hackfest, harfbuzz, pango, textlayout

¶ 11:55 AM   5 comments

 Text-on-path with cairo

Cairotwisted is the name a piece of code I wrote using pangocairo to lay text on a path. Over the years many people have found it useful, so here is a public post to make sure it's easier to find on the interwebs.


The code is shipped in Pango tarball under examples/cairotwisted.c and can be browsed here. Click on the thumbnail to see the full size image output.

At the core of it there is a function to map one path onto another one. This is done by parameterizing the second path and computing it's gradient (see the code).

Parameterizing a Bezier curve uniformly is not easy though. I currently flatten curves to lines and use that, but that has its own down sides as the error introduced in the gradient can magnify out of bound. Any ideas?

Labels: cairo, cairotwisted, pango, pangocairo

¶ 1:14 PM   4 comments

 Online font installation

Just submitted patches for GTK+ and gnome-settings-daemon to monitor all fontconfig configuration and react to changes.

What this means is that with the patches in place:

• Modifying ~/.fonts.conf takes effect in all running applications immediately (ok, withing three seconds),
• Installing missing fonts will make hexboxes automagically turn into real text rendered using the newly installed font.

Interestingly, no changes in Pango were needed.

Next step, detect missing fonts and make PackageKit show a notification offering to install needed fonts.

Labels: fonts, gtk+, packagekit, pango

¶ 4:42 AM   2 comments

 The Most Beautiful Persian OpenType Font

Glad I asked for it last month!

Apparently the Supreme Council of ICT of Iran has released the first ever OpenType Persian Nastaliq font. This is of great importance because previously all Persian Nastaliq solutions were Windows-only systems requiring custom rendering engine installed and only worked with select applications. Not anymore!

Anyway, very nice outcome of a governmental-funded project in Iran. I quite appreciate that. The font can be downloaded here.

Now to the interesting part. Again, Pango 1.18.3 and gedit:

The font itself is far from perfect in its OpenType tables, but it's a great start. I've been wanting that for so long... The only issue is that the font doesn't hold a clear license. Got to sort that out and voila! I know, I'm clearly excited. Very excited.

Labels: gedit, gnome, harfbuzz, IranNastaliq, Nastaliq, pango, Persian

¶ 5:27 PM   14 comments

 Episode VI: Return of Federico

Federico:

Funny, how I read your post after Boston Summit, and I've had in fact discussed both issues with Owen Taylor while there. Lets see:

First, increasing cairo scaled font cache size: You saw a 230 KB process size increase when increasing cache size from 256 to 2048. Well, here's a little secret I decided not to disclose until someone notices it, and you kinda qualify now: Cairo doesn't free glyph renderings after uploading them to the X server! You definitely need them if you are using the image surface, but most processes don't. So, you've got the smart X server hashing and reusing glyph renderings, and cairo-using processes keeping a copy around, for no good. Fix that and happily increase cache size to 2048 without 230 KB size increase!

Next, time spent in HarfBuzz, and particularly recreating and enlarging buffers all the time. As my summit hacking project I took on optimizing HarfBuzz. In short, I did:

• Make output buffer copy/swap'ing during GSUB processing lazy, such that if a lookup doesn't affect a glyph string, no glyph copying takes place.
• Compile all of HarfBuzz as a single file, to let compiler do more optimizations. This increased HarfBuzz binary size from 100 KB to 150 KB. Compiling with -Os brings it down to 70 KB. May be worth profiling with -Os too.
  
• Last but not least, cache one HB_Buffer.

All in all, in my measurements, these three made repeated text layout 10 to 20 percent faster for 1) very long paragraphs, and 2) using fonts with many many looksup like Nafees Nastaliq (more than 100). They made no difference for regular small text+font combinations.

Mandatory screenshot of Nafees Nastaliq after I fixed a bug to tolerate a font bug in the GDEF table. I'm surprised how good actually it worked with the synthesized GDEF Pango put together for it, but now it works perfect (again):

Labels: Boston Summit, federico, harfbuzz, optimization, pango

¶ 1:39 PM   5 comments

 Debian Patch Review

Thanks everyone for comments on JDS patch review yesterday. Thanks to lool for the link, I reviewed Debian's pango and vte patches this morning on the way to office.

• 10_scan-module-files-in-dirs.patch: This is filed as bug 355985 already, thanks.
• 11_module-files-append-module-files-d.patch and 12_module-files-append-compat-module-files-d.patch: They should be reconsidered when the bug above is fixed.
• 04_dsp_non_alias.patch: Similar to the wrong vte patch from JDS. It helps if someone who knows the problem gives detailed description of it upstream.
• 12_python_reaper.patch: Filed as bug 320127, but please see discussion in bug 112172.
• 41_from_upstream_fix_nvi_scrolling.patch: Chris tells me that a better fix has been committed upstream.
• 60_fix-ctrl-dash.patch: Filed as upstream bug 448259. Hopefully someone will review and commit it soon.

All in all, seems like someone's been doing their homework. Nice!

Who's next? It could be you!

Labels: debian, pango, vte

¶ 5:49 PM   3 comments

 JDS Patch Review

(Note: After receiving two comments and rereading my previous post, I removed it in accordance to CoC which I have volunteerly signed. Well, we're cool now.)

Seeing people commenting on JDS patches for their packages, I couldn't help but checking pango, cairo, and vte patches they ship. Having reviewed them, I thought I should provide some feedback :). I'm posting here, hoping that other GNOME hackers pick this up too. Later we can move on to collectively review other vendors' patches too, and help the over-busy maintainers push the right patches upstream and drop/fix the wrong ones. Not every project has a sebuild, right? ;) Brian Cameron has been doing a great job doing that for opensolaris already, but there's more room.

• pango-01-fullwidth-space.diff: I'm interested to have a Pango bug requesting what this tries to fix, as I'm not sure about what the patch tries to achieve. Other than that, both changes in the patch are wrong. The first one is adding an entry for 0x0020 after 0x2000 in a table that is supposed to remain sorted. Moreover, characters below 0x2000 are not looked up in that table at all, that's why the table starts at 0x2000. The other change too adds an entry, for 0x3000, but does not remove the old entry for 0x3000. Both can confuse the bsearch routine.
• pango-02-pua.diff: Is a very weird attempt that I believe is trying to fix bug 443206 that was fixed upstream two weekes ago. Other than that, the patch touches public pango API, something that should be avoided at all costs. It adds a new enum value PANGO_SCRIPT_OTHER that makes no sense. Just use the upstream patch please, or wait for next stable release.
• pango-03-no-xrender.diff: Removes check for xrender from pango's configure check for pangoxft. If that check is not necessary, and there's a use in compiling without it, please file a bug agaist Pango and I'll be more than happy to commit it upstream.
• cairo-01-uninstalled-pc.diff: Submitted to upstream bugzilla already. Only if someone could tell me how exactly the -uninstalled pkg-config files are used...
• cairo-02-8bit-fix.diff: This works around the infamous cairo 8-bit issues. The patch has been posted upstream. Carl has been working on getting this fixed, but it's not easy, and a clean fix is even harder.
• cairo-03-full-hinting.diff: I'm interested to hear what the patch tries to achieve. I assume this relies on custom changes to fontconfig.
• cairo-04-mediaLib.diff: In the works in upstream bugzilla already.
• cairo-05-remove-buggy_repeat.diff: Committed upstream already.
• vte-01-fcconfig.diff: According to Owen, patch is clearly wrong, and I agree. The issue is more of a fontconfig misconfiguration IMO, but not enough detail is given, so I can't test. Needs someone debugging what's going on, instead of blindly cut it out.
• vte-03-cut-copy-paste-handle.diff: The patch looks interesting. I don't remember seeing any upstream report about it, and I'm not sure what problem it tries to solve, but I like to know!
• vte-04-selection-perf-improve.diff: Well, this was submitted upstream and rejected. It removes vte's regular expression matching functionality and hardcodes hand-written matching of common URL schemes. There are many other ways to improve vte's selection performance without sacrificing functionality. Needs someone to profile/debug it.

All in all, very good for cairo, but can do better for vte and pango.

Labels: cairo, jds, pango, vte

¶ 5:46 PM   7 comments

 Foundations of Gtk+ Development

Yesterday I shortly reviewed the new Gtk+ book by Andrew Krause, Foundations of Gtk+ Development, that I received as an ebook thanks to Apress.

I just searched Pango and cairo in it and read those parts. All in all, pretty good. Doesn't get into Pango at all, just defined PangoFontDescription and Pango markup and uses those with Gtk+ widgets all over the place. As for cairo, it has a short (2 3 page) intro to using cairo to draw paths.

The ebook is quite searchable which is a huge advantage over the dead=tree version. So far so good, but I doubt that I'll do a more in depth review staring at the monitor (or printing the 600+ page beast). Of course the fact that I lost my last pair of glasses this past weekend doesn't help either...

Labels: andrewcrause, apress, book, cairo, gtk+, pango

¶ 11:23 AM   3 comments

 Bug Fixing...

...in seventeen minutes.

Labels: bug, pango

¶ 10:03 PM   0 comments

 Pango OpenType Update

All other modules are ported to new OpenType API now. I really hope I've not broken any of them in the way.

In other news, Pango's Arabic module is now extended to support the N'Ko script. I believe Pango is the first Unicode rendering system to support N'Ko!

Seems like my optimization in Pango's line-breaking algorithm introduced a bug that I hope is fixed now.

Also came across this hilarious bug titled "My microwave doesn't work any more because of gnome"!

Labels: bug, funny, microwave, n'ko, opentype, pango

¶ 8:42 AM   5 comments

 Advanced OpenType Features in Pango

Last week I worked on bringing Pango's OpenType engine into shape. Something that has been long overdue. Not anymore.

Pango can now select arbitrary OpenType script and language systems, and in near future arbitrary OpenType features too. The basic, arabic, and syriac modules are ported to use these features. More modules to follow.

Finishing off that feature is addition of another one, that pango will read env vars PANGO_LANGUAGE or LANGUAGE and use that for making better guesses to tag runs of text with languages.

First window is rendering of Pango's test-mixed.txt using the command line pango-view tool.

Second window is the same, but with env var PANGO_LANGUAGE="en:ur". As you can notice, an Arabic font more suitable in style for Urdu is selected instead of my default Persian fonts.

Third window is vertical rendering of the same text. If you look at the last, Japanese, line, in the vertical window, the brackets around the text are correctly rotated, unlike previous versions of Pango.

This and previous works are all in Pango 1.17.0.

Labels: opentype, pango

¶ 8:48 PM   8 comments

 PangoCairo arbitrary shape rendering

On Tuesday I met Travis Griggs, the author of cairo Smalltalk bindings, who was presenting about cairo at IT360°. He made his slide deck using cairo and Pango, using a background and font that resembled Egyptian papyrus scroll using the cairo beetle logo instead of bullet points. I asked him how he did that and of course the hard way. I then explained how pango supports shape attributes that can be used by a custom made PangoRenderer to render arbitrary shapes.

Last night I thought there should be an easier way to do that, and so I added pango_cairo_context_set_shape_renderer().

Obligatory screenshot:

Source code.

Lets see how long I can go with a Pango feature a day. Today excluded as I'll be driving to LGM. Weekends too as I've finally got myself spending the weekend with species other than computers.

Labels: cairo, gnome, pango, pangocairo

¶ 7:36 AM   3 comments

 Avoiding extra work

When working on justification support in Pango, I found a loop that could use some optimization.

The loop in question is to decide how much of a piece of text (called item. For normal text it typically is the rest of the paragraph) we can fit into the current line. The implementation was removing one character at a time from the end until we find a break spot and the width of remaining chars fits the available width.

Think about it. If the entire item fits the available width the loop is not reached at all, but otherwise, unless the paragraph fits in exactly two lines, we are doing a lot of extra work. Say the paragraph takes 10 lines. Then for the first line we have to remove 9 lines worth of text before finding the right break point. For next line we have to remove 8 lines... and so on. It's quadratic in the number of lines, so to speak.

What I did was to start inserting chars from the beginning of the item until no more fits.

For a really long paragraph (pango/pango-view/test-long-paragraph.txt) the number of times the loop contents are run came down from ~1,000,000 to about 13,000. Not bad.

This was all in my quest to make gedit run faster when viewing a REALLY LONG paragraph. I was under the impression that some silly quadratic algorithms in Pango are responsible for the slow speed, but apparently I'm wrong. In short, Gtk+ recreates the Pango layout on EVERY CURSOR MOVEMENT OR CURSOR BLINKING. This is mostly a result of having a very extensible and powerful text object in Gtk+ that supports multiple marks, some of them may be cursors. I've already attached a patch to improve it for cursor blinking, but that's not very interesting. I want to be able to put my finger on the left or right arrow and don't see a gedit that is frozen as a result. Way to go...

Labels: gtk+, pango, performance

¶ 8:25 PM   13 comments

 Justified text with Pango

...is not unsupported anymore.

The first paragraph has positive letter spacing, and then justified. As you see the third line didn't hvae any space chars and so is justified by applying more letter spacing. Now the Latin part has unfairly long words and short lines, but Arabic works much better than I expected.

It's a very simple greedy algorithm. Mathias Hasselmann start writing patches, and hours later and three ~100 line patches, I've got it all working and nicely interacting with letter-spacing too.

Six year old bugzilla entry here.

Labels: pango, screenshot, typography

¶ 9:34 PM   17 comments

 Metrics hinting and kerning do mix

Metrics hinting is the act of round the advance width of a glyph to whole pixels. It's essential for getting good looking text redering on displays with not-so-high resolutions.

Kerning is the act of adjusting the space between certain pairs of adjacent glyphs. This, too, is essential for high quality text rendering.

Both Pango and Qt have been doing metrics hinting, and doing kerning, practically since forever. But both had this bug that the kerning adjustment were not rounded to whole pixels. So, a single fractional kerning pair could end up canceling all the metrics hinting effort... This was most evident with fonts that have more than a few kerning pairs, like DejaVu Sans. Most apparent in the string "rrrrrrrrrrrrrrrr" with that font.

The bug is no more. Fixed in Qt last week, and in Pango today. Pango commit here.

Before/after:


Further reading: there is a lot more to do in this area. An informative post from David Turner, and an excellent paper from Owen Taylor.

I want to implement some of that stuff in cairo/pango for next major releases, but more important is that all these details are being taken care of in the HarfBuzz unified shaper project that Pango and Qt developers are developing together. Exciting times.

Labels: gridfitting, harfbuzz, hinting, kerning, pango, qt

¶ 12:38 AM   3 comments