safe-cp: A safer file copy script

A while back I wrote a shell script to do a “safe” file copy on a Linux system.  I use it somewhat frequently for managing servers.  It basically copies a file in a completely atomic way.  Programs accessing the file always see either the complete original file or the complete new file; never anything else.

Because it’s a little involved, I though it would be nice to share it publicly, so here it is.

The script is in safe-cp.sh.  You copy a file with safe-cp.sh source-file destination-file.  More details are available by running safe-cp.sh --help.

The program ensures three things:

  1. If the destination file existed before running the program, a file always exists with that name.  There is never a time when the old file has been removed but the new one has not yet been created.

  2. The destination file is always complete and whole.  Either the entirety of the old version of the file is available or the entirety of the new version of the file is available.  There is never a time when the destination file is either empty or contains an incomplete copy of either the old or new data.

  3. The contents of the original file always exist in the filesystem.  After the copy, the old data exists—with the same inode—in a copy of the original file with a different name.  This ensures that if a program has an open filehandle to the file, its view of the data is unchanged.  The program won’t see the new data unless it closes and reopens the file, but it will never see inconsistent data.

You’d think the guarantees in the last point would be a given because of how Unix handles unlinking files with open filehandles, but I’ve seen NFS mounts that didn’t behave correctly in this regard.

Fundamentally, the script works by making sure a copy of the new file exists in the same filesystem as the destination, hard-linking the old file to a new filename, then renaming the copy of the new file to the destination name.  That final rename is atomic, while the hardlinking ensures the original file and inode are preserved.

You can read through the script for the precise details, of course.


Cairo and Pango in Python

Cairo is a pretty useful vector-based graphics library.  I usually use it from Python.  But Cairo’s built-in text handling can be a little limited; their official stance on the subject is if you want more functionality you should use an external library “like pangocairo that is part of the Pango text layout and rendering library.”  Here’s how to use Pango with Cairo in Python.

This guide takes the form of building a simple Python program that demonstrates a few basic aspects of Pango use.  The full program is in pango_test.py.

There used to be native Python modules for Pango, but they seem to be defunct now.  There are some current modules based on CFFI, but they’re incomplete and I found them difficult to work with.  The approach I like the best is to use the PyGObject module to access Pango, just as a GTK program would.  You’ll also need Pycairo installed, of course.

§ Module Imports

So let’s set up the imports for our program.

import math

import cairo

import gi
gi.require_version('Pango', '1.0')
gi.require_version('PangoCairo', '1.0')
from gi.repository import Pango, PangoCairo

We’ll need the math module for some minor things later.  The cairo module should be obvious.

The gi module is the entry point to PyGObject.  You can check the documentation for more details, but basically you tell it which object versions you want to use, and then import specific libraries’ object definitions.  If you don’t specify the versions, PyGObject will use the latest version available and will print a warning to the console about it (since APIs are not guaranteed to be identical across different GObject release versions).

§ Cairo Surface Creation

Next, let’s set up a Cairo SVG surface to work with.  This should be routine for anyone familiar with Cairo.  We’ll make a surface and draw a white rectangle on it to serve as a background.

surface = cairo.SVGSurface('test.svg', 480, 480)
ctx = cairo.Context(surface)
ctx.set_source_rgb(1, 1, 1)
ctx.rectangle(0, 0, 480, 480)
ctx.fill()

§ PangoCairo Layout Creation

Now we get into the first Pango-specific thing.  The Pango library does text layout and rendering, and there’s a PangoCairo library that serves to translate Pango’s rendering into something Cairo can understand.  (Under the hood, it uses Cairo’s low-level glyphs API, but you don’t really need to worry about that.)

PangoCairo uses a “layout” to manage text rendering.  Roughly speaking, you establish a layout, tell it how to render text (what font, how to justify the text, how to space characters and lines, and so on) and what text to render, and then it gives you the rendered result.

So we start by creating a layout to use.  It takes the current Cairo context as a parameter.

layout = PangoCairo.create_layout(ctx)

§ Font Loading

Next, we’ll load a font.  Fonts are a little complicated in Pango.  The library uses a description of a font as a reference for locating the font on your system.  The description consists of information like the font name, its style (bold, italic, thin, etc.), and its size.  The easiest way to load a font is via Pango.font_description_from_string().  It takes as its parameter a string describing a font.  It will try to match the description given; if it can’t, it’ll fall back to something it thinks is a suitable substitute.

The description string has a syntax of its own, described in the documentation.  You can generally assume that you can just put the font name, style, and a number for the point size in the string and it should work.  I’ve found that sometimes adding the optional comma after the font name can help, as in the example below.  We’re going to use Times New Roman for the font.  But “Roman” is also a style keyword for Pango, so the string “Times New Roman 45” will be interpreted as a request for the Roman style of a font named “Times New”.  Putting in the comma, as in “Times New Roman, 45” lets Pango parse things correctly.

Linux users without the Microsoft fonts installed might want to use Liberation Serif instead of Times New Roman.

Note that the size given is relative to the base unit of the Cairo surface.  For a standard SVG surface, as we’re using, the size will be in points.  If we were using an ImageSurface, the below code would load Times New Roman at a size of 45 pixels tall.

After loading the font description, we tell the layout to use the font we’ve described.

font_description = Pango.font_description_from_string('Times New Roman, 45')
layout.set_font_description(font_description)

We’ll go over an alternate mechanism for loading fonts later.

§ Context Translation

Just to illustrate something, we’ll translate the Cairo context to change where the origin is.  This isn’t strictly necessary—you can just call Context.move_to() before outputting the text—but this demonstrates how you need to handle context translations when working with Pango.

Internally, the PangoCairo layout keeps a PangoContext—which, as it might sound, is the Pango version of a Cairo context.  If the transformation matrix of the Cairo context changes, you need to call PangoCairo.update_layout() to keep the PangoContext in sync.  You also need to call PangoCairo.update_layout() if the context’s target surface changes.

ctx.translate(16, 16)
PangoCairo.update_layout(ctx, layout)

§ Text-drawing Function

We’ll want to render a few strings in the program, and there are a few things we want to do for each string.  So we’ll make a function to render a given bit of text.  The function needs the Cairo context, the PangoCairo layout, and, of course, the text to render.

def draw_text_and_bounds(ctx, layout, text): 

The function will also return some information about the bounds of the text it rendered, but we’ll get to that shortly.

§ Set the Text to Render

Telling the layout what text it needs to render is a simple function call.

    layout.set_text(text)

§ Text Bounds

Example of Text Bounds

Pango gives you a few different measures of what the bounds of the text are.  In Cairo’s toy text API, the point of origin for rendering text is the left side of the text, on the text baseline.  In contrast, Pango’s point of origin is the upper left corner of the box defining the text’s logical extents.

The logical extents of the text are the full area the text occupies, based on the font.  The extents include vertical whitespace to encompass a full line of text, which means having room for ascenders and descenders (whether or not the text being rendered has them) and extra vertical room the font might include (some fonts add vertical whitespace above—and occasionally below—all of the characters).  The width of the logical extents more or less corresponds to Cairo’s TextExtents’ x_advance.  In the image to the right, the logical extents of each string are shown in red.  Note that the first string has two spaces at the end.

Pango also has ink extents for a given piece of text.  The ink extents are the area covered by the specific glyphs being rendered.  This more or less corresponds to the area covered by Cairo’s TextExtents’ x_bearing, y_bearing, width, and height.  In the image to the right, the ink extents are shown in blue.

The function Layout.get_extents() returns two values: the ink extents and the logical extents, in that order.  Each value is a PangoRectangle, which has x, y, width, and height members.  The coordinates are all with respect to the upper left corner of the logical extents.  That means the x and y values for the logical rectangle should always both be zero.  The y value for the ink rectangle will almost always be positive.  The x value for the ink rectangle might be zero, might be positive, and might be negative.  (In the example on the right, the second line has a slightly-negative ink x value, while the third line has a positive x value.)

The values in the rectangles returned by Layout.get_extents() are in Pango units.  You must divide them by Pango.SCALE to get the equivalent Cairo units.  (While Cairo works with floats, Pango works with integers.  In order to get a reasonable amount of precision, Pango scales up its coordinates by a factor of Pango.SCALE in order to facilitate meaningful subpixel operations.)

In Cairo, you always know where the text’s baseline is, because that’s where Cairo starts from when drawing text.  With Pango, you call Layout.get_baseline() to get the distance from the top of the logical extents to the baseline.  As with other Pango functions, the result is in Pango units and must be divided by Pango.SCALE to get Cairo units.  The baseline is shown in orange in the example on the right.

With that in mind, the following code will draw boxes around the text’s various extents, as well as put a small dot at the text’s origin (the upper left corner of the logical extents).

    ink_rect, logical_rect = layout.get_extents()
    baseline = layout.get_baseline()

    ctx.set_line_width(2)

    # Baseline in orange.  Offset to account for stroke thickness. 
    ctx.set_source_rgb(1, 0.5, 0.25)
    ctx.move_to(-8, baseline / Pango.SCALE + 1)
    ctx.line_to((logical_rect.x + logical_rect.width) / Pango.SCALE + 8,
                baseline / Pango.SCALE + 1)
    ctx.stroke()

    # logical rect in red
    ctx.set_source_rgb(0.75, 0, 0)
    ctx.rectangle(logical_rect.x / Pango.SCALE - 1,
                  logical_rect.y / Pango.SCALE - 1,
                  logical_rect.width / Pango.SCALE + 2,
                  logical_rect.height / Pango.SCALE + 2)
    ctx.stroke()

    # ink rect in blue
    ctx.set_source_rgb(0, 0, 0.75)
    ctx.rectangle(ink_rect.x / Pango.SCALE - 1,
                  ink_rect.y / Pango.SCALE - 1,
                  ink_rect.width / Pango.SCALE + 2,
                  ink_rect.height / Pango.SCALE + 2)
    ctx.stroke()

    # Baseline in orange
    ctx.set_source_rgb(1, 0.5, 0.25)
    ctx.move_to(-8, baseline / Pango.SCALE)
    ctx.line_to((logical_rect.x + logical_rect.width) / Pango.SCALE + 8,
                baseline / Pango.SCALE)
    ctx.stroke()
    
    # Origin in dark blue
    ctx.set_source_rgb(0, 0, 0.5)
    ctx.arc(0, 0, 2 * math.sqrt(2), 0, 2 * math.pi)
    ctx.fill()

§ Render the Text

The PangoCairo.show_layout() function is what takes care of outputting the rendered text to the Cairo surface, by way of the Cairo context.  As noted previously, PangoCairo uses the Cairo context’s current position as the upper left corner of the text’s logical bounds.  Since this program relies on using Context.translate() to shift the origin to the appropriate rendering location, we’ll simply draw the text at that (translated) origin.

    ctx.set_source_rgb(0, 0, 0)
    ctx.move_to(0, 0)
    PangoCairo.show_layout(ctx, layout)

§ Return the Logical Bounds

To facilitate text positioning, we’ll have the function return the logical bounds of the text having been rendered.  Keep in mind that the bounds are in Pango units, not Cairo units!

    return logical_rect

§ Actually Render Some Text

Now that the function is set up, let’s draw some text.  This will draw the text “Aa Ee Rr”, followed by two spaces, in the previously-loaded font (45 point Times New Roman).

logical_rect = draw_text_and_bounds(ctx, layout, 'Aa Ee Rr  ')

Following that, we’ll shift the origin down by the logical height of the rendered text, plus a bit of a buffer.  We’ll then render the text “Bb Gg Jj” (which has some descenders) in italic Times New Roman.

ctx.translate(0, 16 + logical_rect.height / Pango.SCALE)
PangoCairo.update_layout(ctx, layout)
font_description = Pango.font_description_from_string('Times New Roman, Italic 45')
layout.set_font_description(font_description)
logical_rect = draw_text_and_bounds(ctx, layout, 'Bb Gg Jj')

Next, we’ll shift the origin down a little farther for the next line of text.

ctx.translate(0, 16 + logical_rect.height / Pango.SCALE)
PangoCairo.update_layout(ctx, layout)

§ Alternate Font Loading Method

When you use Pango.font_description_from_string(), you’re not necessarily guaranteed to get the font you asked for.  If Pango can’t satisfy the request literally, it’ll use a fallback font instead.  (In my experience, the fallback font ignores all additional font characteristics like point size and font weight, which can give substantially less-than-ideal results.)

An alternate approach involves going through PangoFontMap.  You can load a list of all of the fonts on the system with PangoCairo.font_map_get_default(), which returns a PangoFontMap object.  From there, you can use PangoFontMap.get_family() to retrieve a font family by name (as a PangoFontFamily object) and then use PangoFontFamily.get_face() to retrieve a specific font variant within the family.  Face names will be things like “Regular”, “Bold”, “Bold Italic”, and so on.  You can call PangoFontFamily.list_faces() to return a list of all of the faces available for a family.

If you use this approach and the requested family or face is unavailable, the relevant function will return None instead of an object.  You can use this to be precise about your font loading rather than relying on Pango’s fallback mechanism.

Once you have a font face, you get its description with PangoFontFace.describe(), which returns a PangoFontDescription object.  After setting the font description size (in Pango units, so you’ll probably need to multiply by Pango.SCALE), you can set the font on a layout using Layout.set_font_description(), just as when we loaded a font description from a string earlier.

Here we’ll load 45 point italic Times New Roman using this alternate method.

font_map = PangoCairo.font_map_get_default()
font_family = font_map.get_family('Times New Roman')
font_face = font_family.get_face('Italic')
font_description = font_face.describe()
font_description.set_size(45 * Pango.SCALE)
layout.set_font_description(font_description)

Note that the final program, linked above and below, contains some extra error handling here in case the specified font family or face is not available.

Also note that the font description being loaded is still a Pango-created summary of the font.  In my experience, this approach can be a bit more precise than Pango.font_description_from_string(), but I’ve still run into problems.  I have one font family on my system where the specific font names are a little messed up and several different weights of the italic faces have the same name.  I can select the exactly-correct PangoFontFace object using the above process, but the summary it generates as a PangoFontDescription leads to the wrong weight being loaded.

Anyway, now the font’s been loaded, let’s render some more text.  Just for fun, we’ll do Hebrew, which is written right-to-left.

logical_rect = draw_text_and_bounds(ctx, layout, 'אָלֶף־בֵּית עִבְרִי')

Note that some of the diacritics here are actually a little outside the logical text bounds (unless you’ve changed to a font that handles them differently).

Note also that even though the text runs right-to-left, Pango still uses the left side of the rendered text as its origin.  This ensure consistent placement regardless of the text direction.  (And, indeed, you can mix together text with different writing directions and Pango will lay everything out in a reasonable way.)  If you were aligning several lines of right-to-left text (and didn’t want to just pass a string with newlines to Pango and let the library figure it out), you would need to use the text extents to determine how to position Pango’s logical origin to the left of the text’s right margin.

§ Conclusion

The completed program, with some comments and a little error handling, is at pango_test.py.

If you’re used to Cairo’s toy text API, Pango isn’t too dissimilar, although there is another object to keep track of, and font loading is a bit different.  The main thing I had to get used to—and it didn’t take too long—was the different origin for text rendering and the different way Pango has that it thinks about text extents.


The “Pip! is Nonbinary” FAQ

Over the past year or so, I’ve come to an understanding of myself having a nonbinary gender identity.  This is a set of frequently asked questions (with answers) I’ve fielded about that transition.

§ What does “nonbinary gender identity” mean?

It means I don’t feel I fully belong to the group of people labeled “men” and I also don’t feel I fully belong to the group of people labeled “women”.  That puts me outside of the traditional gender binary, or “nonbinary” for short.

Nonbinary is a pretty broad label.  It encompasses a diversity of gender identities, including people whose gender identity fluctuates over time (genderfluid, bigender), people who don’t feel any gender identity applies to them (agender), people who feel like a blend of gender identities (androgynous, among others, and note that not everyone has a 50%/50% blend), and many others.  For me, it’s more of a default setting, defined by the absence of a strong male or female identity.

§ Does that mean you’re transgender?

Yes.  My gender identity as I currently understand it is different from the gender identity I wore as I was growing up.  (The latter is sometimes called AGAB, short for “assigned gender at birth”.)  That makes me transgender, or trans for short.  (Note that “trans” is an adjective, not a noun.  I am trans, and I am a trans person.  It is incorrect—both grammatically and in a dehumanizing way—to say that anyone is “a trans.”)

Recall from the previous question that “nonbinary” encompasses many different gender identities.  Some nonbinary people do not see themselves as trans, though any nonbinary person may if they so choose.  I am one of those who do.

§ Is your name changing?

Yes.  My name is now Piper.  I also go by Pip.

My old names aren’t terribly antithetical to me.  I used them for many years.  They’ll always be part of my past identity.  But I feel a bit of distance now between my old name and how I see myself.  Just as I expected people to adjust when I started going by a nickname at age 15, and when I changed my name after I got married, I hope that people can now adjust to me going by Piper or Pip.

§ How should I refer to you when talking about you with someone who might not know your current name?

Use my current name, but you can add some clarification on the first use.  For example:

“I was talking with Piper the other day (who you might know as ‘old name here’, but they go by Piper now).”

§ Are your pronouns changing?

Yes and no.  The short answer is that you can use whatever pronouns you feel best fit me, but if you’re not sure what to use, “they/them” is a good default.

The longer answer is that I don’t feel a strong enough affinity to any particular gender identity to actively claim one set of pronouns to the exclusion of others.  Using a singular “they” for a specific person still feels a bit unusual to me, even though I’ve been doing it for a while now, and even though the singular “they” predates not just Shakespeare but the singular “you”.  As a result, I currently prefer to let people choose what pronouns they feel comfortable using for me.  (But, again, if you’re not sure, give “they/them” a try.)

This can result in some unavoidable confusion when two or more people are talking about me and each uses a different set of pronouns.  I’m sorry about that, but this still seems to be the best approach for me at the moment.

§ What honorific should people use for you instead of “Mister”?

This doesn’t come up very often, but please use “Mx.” (pronounced “mix”), as in “Mx. Piper” or “Mx. Gold”.

§ What if I accidentally use the wrong name or pronouns for you?

If you’re trying to adapt to this change but you use my old name out of habit, don’t stress about it.  Just briefly correct yourself—“Sorry; I meant Piper.”—and move on.  I don’t expect anyone to instantly change they way they refer to me after years of using the old way.  The new things will come with practice, and practice includes sometimes making mistakes, learning from them, and continuing on.

Also, as noted above, you can use whatever pronouns you feel best fit me, so you can’t even use the wrong ones by my definition.

§ Is your email address changing?

Yes.  The old one will continue to work (indefinitely, or as long as that’s feasible), but I’m now using .

It might be some time before I have my email address (and displayed name and account name) updated everywhere I have an account online.

§ Are you going to look different than you used to?

Yes.  I’m wearing more overtly-feminine clothes than I used to, and I’ve adopted some other forms of presentation that are more socially associated with women than with men.

This is all a process of finding which things feel comfortable for me, which feel uncomfortable, and which don’t feel relevant one way or another.

§ Why are you doing this now (as opposed to earlier in your life)?

Although I was raised as a boy and have lived for many years as a man, I’ve felt drawn to more feminine aspects of gender identity and presentation for most of my life.  But my discomfort at being confined to a male identity (also known as gender dysphoria) was never strong enough to outweigh the social pressures against coming out as a trans woman.

The biggest thing that changed was a greater awareness, on both my part and society’s, of nonbinary gender identities.  Just having nonbinary as an option felt incredibly freeing to me.  It gave me a space to explore my gender identity and presentation without feeling I had to stay in a 100% male role or transition to a 100% female identity.

(I do still have to sort myself into binary categories on occasion, like when a public facility doesn’t have gender-neutral or single-occupancy bathrooms.  I usually take an ad hoc approach and try to pick the option that I think will confuse the fewest number of people.)

§ Why are you doing this now (as opposed to never)?

As I noted above, I’ve always felt drawn to more of a female identity than I felt I was allowed to express.  Having to live with that was a persistent, low-grade pressure at the back of my mind.  I was constantly trying to decide how much I could subtly step into feminine things without explicitly claiming anything other than a male identity.

Once I started exploring a nonbinary identity, I found that I was much more comfortable and happy with the way I lived and presented myself.  And once I started feeling happier with nonbinary expressions, the times when I was restricted to male-only presentation felt more and more uncomfortable.

In short, I’m happier now, even with all of the uncertainty and opposition in some parts of society to trans people.  Staying closeted (or going back to being closeted) would be tantamount to saying, “Well, I guess I’ll just live with being unhappy for the rest of my life.”  I’m not going to do that.

§ Other questions?

I’m happy to answer or discuss most honest, respectful questions.  Feel free to reach out to me.  Friends and family should have my contact information already.  For everyone else, my email address is linked at the bottom of this page.  Be aware that it does often take me a while to get around to responding to emails, even from people I know.


Garmin Lily 2 Watch Faces

I recently got a Garmin Lily 2 smartwatch.  When I was deciding what watch to get, I went looking to see what different watch faces the Lily 2 had.  I couldn’t find such a list online at the time.  Now that I have a Lily 2, I’m making that list for other people’s reference.

The Lily 2 is a fitness tracking smartwatch from Garmin.  It’s the smallest and lightest smartwatch they offer, and its features are a bit limited in comparison to other Garmin smartwatches.  One of those limitations is in the area of watch faces.  The Lily 2 does not support Garmin Connect IQ watch faces.  It has its own set of predefined faces that you can choose from.

Each watch face shows at least the time and has a spot for additional information.  That spot shows one piece of information at a time.  You can cycle through the available pieces of information by tapping on the watch face.  Some watch faces also have pieces of information (e.g. current heart rate) that they always show, in addition to the cyclable information slot.

The pieces of information available for the cyclable slot are:

  • Steps
  • Heart Rate
  • Body Battery
  • Calories
  • Weather
  • Battery (watch battery level)
  • Garmin Logo

You can restrict the actually-displayed pieces of information in the settings, if you don’t want to go through all of them all the time.  I only show weather and device battery; I use widgets to show the other information when I want to see it.

Although it’s not an option for the cyclable information, some watch faces show the current date.  Those that do so always use the same formatting for it, showing the abbreviated day of the week and the day of the month.

In the images below, the spot for cyclable information will be shown as the Garmin logo.  That serves to distinguish it from the always-shown information, since no watch face has an always-shown Garmin logo.

  • Digital clock, with seconds
  • Analog clock, with seconds
  • Digital clock
  • Date
  • Digital clock
  • Date
  • Calories
  • Steps
  • Digital clock
  • Date
  • Heart Rate
  • Digital clock
  • Body Battery
  • Date
  • Analog clock, with seconds
  • Digital clock
  • Date
  • Heart Rate

Note: See below for a description of the grey arc at the top. 

  • Digital clock
  • Date
  • Weather
  • Calories

The next to last face shown above has a grey arc at the top of the face.  When the cyclable display is showing a bit of information that can be viewed as a percentage (device battery, body battery, steps (as a percent of the daily goal), etc.), a section of that arc is illuminated in proportion to the percentage.

I have a Lily 2 Classic, but I believe the (non-Classic) Lily 2 has the same faces.  This information is accurate as of March 2024 and firmware 3.11.  Newer firmware might add, remove, and change the watch faces available.


Éowyn Challenge – Mordor!

Back in 2022, I started “Walking to Mordor and Back”.  Briefly, that involves seeing how long it takes me to walk as far as Frodo did in The Lord of the Rings.  For more information, see my previous post on the topic.

As of December 20, I’ve reached Mount Doom in Mordor!  That’s a total of 1,779 miles walked in just under two years since January 1, 2022.  The same journey took Frodo about six months, so he definitely made better time than I did.