Happy Holidays

Dear MTA,

Putting “Happy Holidays” on the front display panels of your buses is cute and all, but could you at least cycle them with the number and destination of the bus?  Thanks.

Best wishes, Pip!

Waited nearly an hour for the bus on the way home, but that’s probably the fault of the traffic diversions and general congestion stemming from the Christmas tree lighting.  Doesn’t really get the MTA off the hook for this morning, though.


An Ode to screen

Just a listing of the many ways that screen is indispensable to my way of using my computer.

The biggest thing is, of course, the fact that screen is detachable.  Start screen, start a program, detach screen while in the middle of doing something, log out, login later, reattach, program is just as I left it.  This works at a distance, too.  I can leave my home and go elsewhere (work, friend’s house, etc.) and be able to ssh to my home computer, reattach screen, and pick up exactly where I left off.

Second only to detachability is screen’s multiplexing capability.  A screen session can contain many different windows, each running a different program.  This allows me to have an entire workspace within screen.  I can have an editor in one window, working on some source code; a shell in another window, where I might be doing trial runs of the program; a web browser in a third window, which could be looking at some documentation; and so on.  My normal screen setup has 15 windows, which I use for various purposes.

I can specify what programs to run from screen’s config file.  So when I start screen, my default workspace is already seeded with all the programs I use regularly (text editor, IRC client, web browser, mail client, etc.).  I use this capability just to start some programs that I never even interact with, for example SETI@Home.  Running it within screen ensures that it’s running at all times (I always have a screen session running) and only one instance is ever running (I do everything within one screen session).

I can attach to the same session multiple times.  So my customary graphical workspace is three xterms, all attached to the same screen session.  This gives me more visual real estate, while allowing me to be very flexible.  The windows displayed by each of the xterms change depending on what I’m doing at the moment.

Screen understands several different character encodings.  I run all of my programs in UTF-8 mode.  When I’m attached to my local xterms, screen passes the UTF-8 characters straight through, because the xterms can handle it.  On friends’ computers, screen translates the UTF-8 into ISO-8859-1, showing all the characters it can and filling in question marks for those it can’t.  Likewise for my serial terminal, which uses a CP437 charset.  (I’ll admit that that last took some work on my part.)

I can also input most Unicode characters via screen’s digraph support.  Press the right escape characters, enter an RFC1345 digraph, and whatever program I’m currently using gets a UTF-8 character.  (This also took a little work—I had to patch screen to get digraph support for non-ISO-8859-1 characters.)

Screen keeps a separate scrollback buffer for each window.  I have it set to keep a very large number of lines, which has come in useful on several occasions, especially since you can search through the scrollback buffer.  (“What was the exact output of that command?” ::search::  “Ah, that was it.”)

I’ve used screen’s monitoring capabilities a lot.  It can watch a particular window and notify you when there’s new activity (“Oh, something happened in that log file.”) or when it’s been silent for a time (“Oh, that long-running compile is done.”)

Screen supports having a caption line across the bottom of the screen.  I use it to give me an omnipresent clock, as well as showing me info on the current window.  The capability also exists to run arbitrary programs and put their output in the caption.  On my laptop, I do this with information on the current state of its battery.

Instances of screen can be password protected, to prevent others from getting at your programs.  I find this feature useful when using screen in a semi-public area where I might need to leave the computer for a time.

There are some features that, while not mind-blowing, are just nice to have around.  Normally, when the last program in a window exits the window closes.  With zombie control, the window remains, and you can restart the program with a single keypress.  Very useful for windows dedicated to particular programs.

While I don’t use it regularly, screen’s multiuser support has been useful on a couple of occasions.  When doing some collaborative programming, I created a single, multiuser screen session, and all of us connected to it.  It proved very useful for sharing information among the group of people.  (“Just go over to window 7, where I’ll show you how feature X works…”)

There are, of course, plenty of other useful aspects to screen.  These are just the ones that I rely on or have found myself relying on.  I encourage anyone who uses a command line regularly to give screen a try.

For further information:


Verizon's Web Site

I recently tried to set up a new account on Verizon’s web site.  In order to do this, you have to choose a login and a password.  Reasonable enough.  The page says that the password must be “Minimum 6 characters with at least 1 number”.  I run my password generating script (dd if=/dev/random bs=1 count=6 | uuencode -) and get VB3:Q-0".  Seems reasonable, so I enter it.  No.  “The password should contain one number, only three repeating characters, no spaces or email addresses and no other special characters.  Please try again.” Grrr.  So I run the program a few more times, get a password that’s just numbers and letters, randomize its case, and enter it.

And go on to create my account?  No.  Verizon’s is also one of the many websites that refuses to acknowledge the validity of the plus sign in an email address.  When submitting my email to a site, I usually use , just to track usage of that address.  Many stupid sites don’t like that.  Bah.  And the page was SSL, so I couldn’t easily mess with the parameters to see if they put their trust in client-side validity checks.

Now, of course, I’ve attempted to log in and the page is sitting at a “please wait” box.  Probably only works with IE anyway.  I’ll have to go through the site and try to find a contact email address to yell at.

Verizon sucks.  (Like everyone doesn’t already know that.)


Car Engine Advice Sought

I’ve been having intermittent problems with my car that have now stumped two mechanics, so I’m turning to the Internet for advice.  I have a 1996 Jeep Grand Cherokee, with Selec-Track four-wheel-drive and a 6 cylinder engine.

The problem I’m having is that the engine will sometimes just stall (the motive power goes away and the engine RPMs drop to zero).  It only seems to happen right after starting the car (so the engine hasn’t warmed up yet), and it’s much more common in wet weather (rain, snow, heavy fog).  Often times, I can feel the engine losing power and can prevent a stall by stepping on the gas pedal.  If the engine actually stalls, I have to wait at least a minute to restart it; if I try restarting it immediately, I hear the chug-chug-chug of the starter, but the engine won’t start running.

I’ve had a few times when the stalling seemed to hit as I was braking, just as the automatic transmission was downshifting from second to first gear.  I’ve had the car sall at times apparently unrelated to shifting, though.

This morning, it was particularly bad.  I barely got out of the parking lot at home (after several stop/wait/restart cycles) before the engine stopped being useful—it would only run for 30–45 seconds before stalling, and I couldn’t prevent the stall.  I had the car towed to a mechanic, but by the time they took a look at it, the engine started and ran normally.  They couldn’t find anything wrong with the engine, and there were no codes from the onboard diagnostics.

The coolant system has a slow leak (my regular mechanic tracked it to the water pump, I think), but I’ve had that for over a year now and I doubt it’s related.  In the time since the problem first manifested, I’ve had a system tune-up (including replacing the spark plug wiring, apparently a common contributor to problems such as these), had the oil changed, had the transmission fluid changed, and had a (presumably unrelated) badly-leaking coolant hose replaced.

Any suggestions on what to look for to fix this will be gladly accepted.  I’ll be cross-posting this to several social media sites; responses there are fine, or just send an email to

Update 2010-01-02: I’ve gotten a number of suggestions for problem areas that could be causing this problem.  The most-suggested item was the fuel filter, which I haven’t replaced since I got the car five years ago, so I’m taking care of that to start with.  I’ll see how well the car behaves with a new filter.  Other suggestions have included the idle valve, O2 sensor, or fuel pump.  If replacing the fuel filter doesn’t work, I’ll check out some of the other possibilities.

Update from late 2010: The problem turned out to be the engine computer.  It was failing to set appropriate engine parameters under various circumstances, particularly when it was wet.  It took a lot of trial an error to get to this diagnosis, and my mechanic said this was the first he’d ever seen this particular problem.


Cairo and Pango in Python

Cairo is a pretty useful vector-based graphics library.  I usually use it from Python.  But Cairo’s built-in text handling can be a little limited; their official stance on the subject is if you want more functionality you should use an external library “like pangocairo that is part of the Pango text layout and rendering library.”  Here’s how to use Pango with Cairo in Python.

This guide takes the form of building a simple Python program that demonstrates a few basic aspects of Pango use.  The full program is in pango_test.py.

There used to be native Python modules for Pango, but they seem to be defunct now.  There are some current modules based on CFFI, but they’re incomplete and I found them difficult to work with.  The approach I like the best is to use the PyGObject module to access Pango, just as a GTK program would.  You’ll also need Pycairo installed, of course.

§ Module Imports

So let’s set up the imports for our program.

import math

import cairo

import gi
gi.require_version('Pango', '1.0')
gi.require_version('PangoCairo', '1.0')
from gi.repository import Pango, PangoCairo

We’ll need the math module for some minor things later.  The cairo module should be obvious.

The gi module is the entry point to PyGObject.  You can check the documentation for more details, but basically you tell it which object versions you want to use, and then import specific libraries’ object definitions.  If you don’t specify the versions, PyGObject will use the latest version available and will print a warning to the console about it (since APIs are not guaranteed to be identical across different GObject release versions).

§ Cairo Surface Creation

Next, let’s set up a Cairo SVG surface to work with.  This should be routine for anyone familiar with Cairo.  We’ll make a surface and draw a white rectangle on it to serve as a background.

surface = cairo.SVGSurface('test.svg', 480, 480)
ctx = cairo.Context(surface)
ctx.set_source_rgb(1, 1, 1)
ctx.rectangle(0, 0, 480, 480)
ctx.fill()

§ PangoCairo Layout Creation

Now we get into the first Pango-specific thing.  The Pango library does text layout and rendering, and there’s a PangoCairo library that serves to translate Pango’s rendering into something Cairo can understand.  (Under the hood, it uses Cairo’s low-level glyphs API, but you don’t really need to worry about that.)

PangoCairo uses a “layout” to manage text rendering.  Roughly speaking, you establish a layout, tell it how to render text (what font, how to justify the text, how to space characters and lines, and so on) and what text to render, and then it gives you the rendered result.

So we start by creating a layout to use.  It takes the current Cairo context as a parameter.

layout = PangoCairo.create_layout(ctx)

§ Font Loading

Next, we’ll load a font.  Fonts are a little complicated in Pango.  The library uses a description of a font as a reference for locating the font on your system.  The description consists of information like the font name, its style (bold, italic, thin, etc.), and its size.  The easiest way to load a font is via Pango.font_description_from_string().  It takes as its parameter a string describing a font.  It will try to match the description given; if it can’t, it’ll fall back to something it thinks is a suitable substitute.

The description string has a syntax of its own, described in the documentation.  You can generally assume that you can just put the font name, style, and a number for the point size in the string and it should work.  I’ve found that sometimes adding the optional comma after the font name can help, as in the example below.  We’re going to use Times New Roman for the font.  But “Roman” is also a style keyword for Pango, so the string “Times New Roman 45” will be interpreted as a request for the Roman style of a font named “Times New”.  Putting in the comma, as in “Times New Roman, 45” lets Pango parse things correctly.

Linux users without the Microsoft fonts installed might want to use Liberation Serif instead of Times New Roman.

Note that the size given is relative to the base unit of the Cairo surface.  For a standard SVG surface, as we’re using, the size will be in points.  If we were using an ImageSurface, the below code would load Times New Roman at a size of 45 pixels tall.

After loading the font description, we tell the layout to use the font we’ve described.

font_description = Pango.font_description_from_string('Times New Roman, 45')
layout.set_font_description(font_description)

We’ll go over an alternate mechanism for loading fonts later.

§ Context Translation

Just to illustrate something, we’ll translate the Cairo context to change where the origin is.  This isn’t strictly necessary—you can just call Context.move_to() before outputting the text—but this demonstrates how you need to handle context translations when working with Pango.

Internally, the PangoCairo layout keeps a PangoContext—which, as it might sound, is the Pango version of a Cairo context.  If the transformation matrix of the Cairo context changes, you need to call PangoCairo.update_layout() to keep the PangoContext in sync.  You also need to call PangoCairo.update_layout() if the context’s target surface changes.

ctx.translate(16, 16)
PangoCairo.update_layout(ctx, layout)

§ Text-drawing Function

We’ll want to render a few strings in the program, and there are a few things we want to do for each string.  So we’ll make a function to render a given bit of text.  The function needs the Cairo context, the PangoCairo layout, and, of course, the text to render.

def draw_text_and_bounds(ctx, layout, text): 

The function will also return some information about the bounds of the text it rendered, but we’ll get to that shortly.

§ Set the Text to Render

Telling the layout what text it needs to render is a simple function call.

    layout.set_text(text)

§ Text Bounds

Example of Text Bounds

Pango gives you a few different measures of what the bounds of the text are.  In Cairo’s toy text API, the point of origin for rendering text is the left side of the text, on the text baseline.  In contrast, Pango’s point of origin is the upper left corner of the box defining the text’s logical extents.

The logical extents of the text are the full area the text occupies, based on the font.  The extents include vertical whitespace to encompass a full line of text, which means having room for ascenders and descenders (whether or not the text being rendered has them) and extra vertical room the font might include (some fonts add vertical whitespace above—and occasionally below—all of the characters).  The width of the logical extents more or less corresponds to Cairo’s TextExtents’ x_advance.  In the image to the right, the logical extents of each string are shown in red.  Note that the first string has two spaces at the end.

Pango also has ink extents for a given piece of text.  The ink extents are the area covered by the specific glyphs being rendered.  This more or less corresponds to the area covered by Cairo’s TextExtents’ x_bearing, y_bearing, width, and height.  In the image to the right, the ink extents are shown in blue.

The function Layout.get_extents() returns two values: the ink extents and the logical extents, in that order.  Each value is a PangoRectangle, which has x, y, width, and height members.  The coordinates are all with respect to the upper left corner of the logical extents.  That means the x and y values for the logical rectangle should always both be zero.  The y value for the ink rectangle will almost always be positive.  The x value for the ink rectangle might be zero, might be positive, and might be negative.  (In the example on the right, the second line has a slightly-negative ink x value, while the third line has a positive x value.)

The values in the rectangles returned by Layout.get_extents() are in Pango units.  You must divide them by Pango.SCALE to get the equivalent Cairo units.  (While Cairo works with floats, Pango works with integers.  In order to get a reasonable amount of precision, Pango scales up its coordinates by a factor of Pango.SCALE in order to facilitate meaningful subpixel operations.)

In Cairo, you always know where the text’s baseline is, because that’s where Cairo starts from when drawing text.  With Pango, you call Layout.get_baseline() to get the distance from the top of the logical extents to the baseline.  As with other Pango functions, the result is in Pango units and must be divided by Pango.SCALE to get Cairo units.  The baseline is shown in orange in the example on the right.

With that in mind, the following code will draw boxes around the text’s various extents, as well as put a small dot at the text’s origin (the upper left corner of the logical extents).

    ink_rect, logical_rect = layout.get_extents()
    baseline = layout.get_baseline()

    ctx.set_line_width(2)

    # Baseline in orange.  Offset to account for stroke thickness. 
    ctx.set_source_rgb(1, 0.5, 0.25)
    ctx.move_to(-8, baseline / Pango.SCALE + 1)
    ctx.line_to((logical_rect.x + logical_rect.width) / Pango.SCALE + 8,
                baseline / Pango.SCALE + 1)
    ctx.stroke()

    # logical rect in red
    ctx.set_source_rgb(0.75, 0, 0)
    ctx.rectangle(logical_rect.x / Pango.SCALE - 1,
                  logical_rect.y / Pango.SCALE - 1,
                  logical_rect.width / Pango.SCALE + 2,
                  logical_rect.height / Pango.SCALE + 2)
    ctx.stroke()

    # ink rect in blue
    ctx.set_source_rgb(0, 0, 0.75)
    ctx.rectangle(ink_rect.x / Pango.SCALE - 1,
                  ink_rect.y / Pango.SCALE - 1,
                  ink_rect.width / Pango.SCALE + 2,
                  ink_rect.height / Pango.SCALE + 2)
    ctx.stroke()

    # Baseline in orange
    ctx.set_source_rgb(1, 0.5, 0.25)
    ctx.move_to(-8, baseline / Pango.SCALE)
    ctx.line_to((logical_rect.x + logical_rect.width) / Pango.SCALE + 8,
                baseline / Pango.SCALE)
    ctx.stroke()
    
    # Origin in dark blue
    ctx.set_source_rgb(0, 0, 0.5)
    ctx.arc(0, 0, 2 * math.sqrt(2), 0, 2 * math.pi)
    ctx.fill()

§ Render the Text

The PangoCairo.show_layout() function is what takes care of outputting the rendered text to the Cairo surface, by way of the Cairo context.  As noted previously, PangoCairo uses the Cairo context’s current position as the upper left corner of the text’s logical bounds.  Since this program relies on using Context.translate() to shift the origin to the appropriate rendering location, we’ll simply draw the text at that (translated) origin.

    ctx.set_source_rgb(0, 0, 0)
    ctx.move_to(0, 0)
    PangoCairo.show_layout(ctx, layout)

§ Return the Logical Bounds

To facilitate text positioning, we’ll have the function return the logical bounds of the text having been rendered.  Keep in mind that the bounds are in Pango units, not Cairo units!

    return logical_rect

§ Actually Render Some Text

Now that the function is set up, let’s draw some text.  This will draw the text “Aa Ee Rr”, followed by two spaces, in the previously-loaded font (45 point Times New Roman).

logical_rect = draw_text_and_bounds(ctx, layout, 'Aa Ee Rr  ')

Following that, we’ll shift the origin down by the logical height of the rendered text, plus a bit of a buffer.  We’ll then render the text “Bb Gg Jj” (which has some descenders) in italic Times New Roman.

ctx.translate(0, 16 + logical_rect.height / Pango.SCALE)
PangoCairo.update_layout(ctx, layout)
font_description = Pango.font_description_from_string('Times New Roman, Italic 45')
layout.set_font_description(font_description)
logical_rect = draw_text_and_bounds(ctx, layout, 'Bb Gg Jj')

Next, we’ll shift the origin down a little farther for the next line of text.

ctx.translate(0, 16 + logical_rect.height / Pango.SCALE)
PangoCairo.update_layout(ctx, layout)

§ Alternate Font Loading Method

When you use Pango.font_description_from_string(), you’re not necessarily guaranteed to get the font you asked for.  If Pango can’t satisfy the request literally, it’ll use a fallback font instead.  (In my experience, the fallback font ignores all additional font characteristics like point size and font weight, which can give substantially less-than-ideal results.)

An alternate approach involves going through PangoFontMap.  You can load a list of all of the fonts on the system with PangoCairo.font_map_get_default(), which returns a PangoFontMap object.  From there, you can use PangoFontMap.get_family() to retrieve a font family by name (as a PangoFontFamily object) and then use PangoFontFamily.get_face() to retrieve a specific font variant within the family.  Face names will be things like “Regular”, “Bold”, “Bold Italic”, and so on.  You can call PangoFontFamily.list_faces() to return a list of all of the faces available for a family.

If you use this approach and the requested family or face is unavailable, the relevant function will return None instead of an object.  You can use this to be precise about your font loading rather than relying on Pango’s fallback mechanism.

Once you have a font face, you get its description with PangoFontFace.describe(), which returns a PangoFontDescription object.  After setting the font description size (in Pango units, so you’ll probably need to multiply by Pango.SCALE), you can set the font on a layout using Layout.set_font_description(), just as when we loaded a font description from a string earlier.

Here we’ll load 45 point italic Times New Roman using this alternate method.

font_map = PangoCairo.font_map_get_default()
font_family = font_map.get_family('Times New Roman')
font_face = font_family.get_face('Italic')
font_description = font_face.describe()
font_description.set_size(45 * Pango.SCALE)
layout.set_font_description(font_description)

Note that the final program, linked above and below, contains some extra error handling here in case the specified font family or face is not available.

Also note that the font description being loaded is still a Pango-created summary of the font.  In my experience, this approach can be a bit more precise than Pango.font_description_from_string(), but I’ve still run into problems.  I have one font family on my system where the specific font names are a little messed up and several different weights of the italic faces have the same name.  I can select the exactly-correct PangoFontFace object using the above process, but the summary it generates as a PangoFontDescription leads to the wrong weight being loaded.

Anyway, now the font’s been loaded, let’s render some more text.  Just for fun, we’ll do Hebrew, which is written right-to-left.

logical_rect = draw_text_and_bounds(ctx, layout, 'אָלֶף־בֵּית עִבְרִי')

Note that some of the diacritics here are actually a little outside the logical text bounds (unless you’ve changed to a font that handles them differently).

Note also that even though the text runs right-to-left, Pango still uses the left side of the rendered text as its origin.  This ensure consistent placement regardless of the text direction.  (And, indeed, you can mix together text with different writing directions and Pango will lay everything out in a reasonable way.)  If you were aligning several lines of right-to-left text (and didn’t want to just pass a string with newlines to Pango and let the library figure it out), you would need to use the text extents to determine how to position Pango’s logical origin to the left of the text’s right margin.

§ Conclusion

The completed program, with some comments and a little error handling, is at pango_test.py.

If you’re used to Cairo’s toy text API, Pango isn’t too dissimilar, although there is another object to keep track of, and font loading is a bit different.  The main thing I had to get used to—and it didn’t take too long—was the different origin for text rendering and the different way Pango has that it thinks about text extents.