not quite minimalistic enough  

2019-05-31

Hey! A Python packaging rant!

burn.exe Considered Harmful, or, Why burn Should Burn

The WiX Toolset is a suite of applications for creating and working with Windows Installer packages (.msi files). Today, it is the most elegant way of creating installation packages for Windows, because it provides reproducibility, a logical structure, and, because it uses XML files instead of a GUI, the package definition can be trivially version controlled. All around, it is the optimal solution to a difficult problem.

Windows Installer has some support for installation UI, mostly to determine the installation directory and to select which optional features to include, but other things are possible. It is not the prettiest of UI libraries, neither in visual appearance nor in design technique, but it works, and, after all, an installation package’s UI does not have to be pretty as long as it works.

Recently, support was added to WiX for “bundles” with “bootstrapper applications”. Bundles are collections of MSI packages that are installed in a specific sequence. Conditions can be defined to decide whether any given package is installed at all. The final distributed output is an .exe file. This is a problem.

First, some background

The MSI file format is designed to make software package management, if not simple, then at least tractable. This situation is particular to Windows, because on other platforms with an integrated package manager (yum, apt, pkg, etc.), there is typically only one source of packages (the distribution’s own repository), and it is well understood that third-party packages do not usually work correctly. On Windows, with hundreds of thousands, if not millions, of developers producing their own packages, this model is clearly not viable. Microsoft’s answer was the MSI file format, because a well-designed MSI has a good chance of “being a good citizen” on the system it is installed on, and, possibly more important, the quality of a package can be assessed before it is installed (e.g. by running the ICEs against it or simply by looking through the tables for obvious issues).

An additional complication that mostly affects Windows, though it exists in some manner on all operating systems, is the Registry database. Installing a package usually both places files on the system and adds certain entries to the Registry, such as file type associations and non-user-specific application configuration data.

The price that the designer of an MSI package has to pay is a certain amount of restrictions. There are rules that a correctly designed package has to obey. Some of them are only checked when the package is validated, some are inherent in the structure of the MSI database and cannot be circumvented.

Windows Installer defines a set of “actions”, which are discrete steps in installing a package. Each action acts upon the system, using information from a set of database tables in the MSI file, to transform the system from one known state to another.

If you are not yet familiar with MSI, it is worthwhile to read the previous paragraph again. MSI, alone among all distribution package formats as far as I know, does not follow an imperative approach that describes how the target system should be changed by the installation, but instead uses a declarative model that describes how the system should be when the installation is finished. The knowledge of how to achieve the desired system state, which may involve, e.g., interdependent modifications that have to happen in a specific order, is provided by Windows Installer itself. The package author does not need to be overly concerned about it.

Enter the “custom action”. “Custom” actions are, as the name suggests, provided by the package author. They can basically do whatever they want, including things that the MSI developers deliberately did not make it possible to do.

Immediately after the first version of Windows Installer was released and Microsoft documented the MSI file format, various companies that had made a good living producing software for producing installation packages (Wise, InstallShield, etc.) started offering versions of their software that no longer created opaque SETUP.EXE files, but MSI files. Unfortunately, they either did not understand the reasons for MSI’s strict design or bowed to user demand; they introduced their own custom actions that, in a nutshell, turned the MSI model inside out by using Windows Installer only as an entrypoint to a “legacy” installation package written in their, by now obsolete, installation scripting language. InstallShield is a very good (or very bad) example of this. For several years, users lived in fear of seeing the words “InstallScript” anywhere, particularly in an installation error message. (InstallScript was a separate component from the actual installation package and had to be installed already, in the correct version, etc.; it was “DLL Hell” with setup components.)

There are some operations involved in installing certain kinds of packages that cannot be represented simply as state changes (like adding a file or a registry key can) but need to interact with other software already on the system, such as registering a plugin for something where the “something” only offers a registration helper application that must be called. For such things, custom actions are, regrettably, necessary. However, if something can be done without a custom action, it should be, no matter how convenient the custom action-based approach may appear in comparison.

Back to WiX

The central advantage of the MSI package format is that it is, essentially, data. It does not contain executable code, other than where the package author decides to include custom actions. By inspecting the Features table, you can learn what parts of the package are optional. The Directory table tells you if the package wants to install files anywhere outside its installation base (such as the Windows directory). In the Property table you find all the package metadata, such as version numbers. You can even adapt a package to your particular requirements by designing a transform (.MST file) that will be applied during installation but leave the original MSI file entirely untouched, giving you the opportunity – unheard of before the introduction of MSI – to actually change the result of the installation in ways not necessarily foreseen by the package author.

Most of all, the nature of the MSI package enables seamless deployment. You don’t have to dig up long-forgotten lists of command-line arguments supported by the setup.exe package of the day, you do not even need the ability to run an executable as part of your deployment at all. You simply tell your client management solution (or even a simple group policy object) to install a package, and that package is installed, perhaps with a transform or two.

burn’s bundles clearly were intended to do away with all that, and they succeeded fully, from start to finish. There is no seamless deployment anymore, because now you have to work with command line options again. There is no trivial transformation anymore, because that requires that the bootstrapper application pass your arguments (including the transform file name) to the actual MSI package it is intended for (and none other). It dares the package developer to design UI for prettiness instead of functionality by providing several predefined “themes” and support for custom ones. Finally, it abuses a feature intended for hiding required system components to prevent its bundled MSIs from showing up in the Control Panel’s applications list.

The WiX installer bundle itself is a perfect example of how not to do installation UI: It is entirely nonstandard, moves and blinks to maximally confuse the user, and is quite ugly on top of that. In an environment that does not support deployment of .exe packages (the vast majority of domains that only have group policy available), it is also essentially nondeployable. It does not even provide a download size advantage even though some parts of the bundle are optional and might be downloaded only if required, because the WiX project always ships the full set in a bundle of ~24 MiB (including some redundant files that would only appear once in an integrated package, but they apparently do not make much of a difference in size).

Conclusion

There is very little that can be done with bundles that a well-designed single MSI package cannot do, and what little there is, no installation package ever should do.

Do not use burn. It burns.

2019-05-30

Some are more equal than others.

I just updated the signature on a number of Excel .xlam add-ins after the signing certificate expired. When I opened each, Excel behaved in three different ways:

All for the same kind of file, signed with the same since-expired cert.

Huh?

2019-04-29

A window to the world

Changing the console resolution in FreeBSD 12 with UEFI boot

There are many pages on the Web that explain how to change the console resolution in the FreeBSD boot process, either for BIOS or UEFI boot. All of them generally agree that the way to do it is to put a line into /boot/loader.rc.local, either mode X or, more recently for UEFI, gop set X.

With FreeBSD 12, all of that went out the window.

FreeBSD 12 now defaults to the “Lua loader”, that is, the last stage of the boot process is now implemented in Lua rather than Forth. I’m not sure why, but probably because Lua is just the embedded language all the cool kids are using these days.

The thing is, while the manpages on loader(8) and related topics confidently imply that loader.rc.local is a platform-independent source of configuration data, in reality it is not.

Workaround

The simplest way to work around the issue is to switch back to the Forth-based loader:

cd /boot
ln -f loader_4th.efi loader.efi

This will restore loader.rc.local to its former glory, i.e. it will be read and applied.

Do not, as I just did, try to cp loader_4th.efi loader.efi; cp overwrites existing destination files and it just so happens that loader.efi is loader_lua.efi (a hard link), so the cp command will overwrite the Lua loader with the Forth one.

Fix

The alternative is to replace loader.rc.local with its newfangled Lua replacement:

cd /boot/lua
cat >local.lua <<<EOF
loader.perform("whatever you want")
EOF

Put your preferred loader command inside the quotes; if you want to run multiple commands, each needs its own loader.perform() call, I think.

By the way, even though gop set X works for me on the loader command line, it does not work in either loader.rc.local or lua/local.lua. The older mode X command works everywhere.

2019-04-18

Ein guter Anfang.

Warum ist denn im neuen Outlook-Icon die E-Mail verpixelt? Hat Microsoft endlich den Datenschutz fĂĽr sich entdeckt?

Outlook

2019-03-29

Reihe ins Nichts.

Danach kommt gar nichts mehr.

OEIS

2019-03-14

Linguistics fail

It’s not exactly German, is it?

Luxembourgish, translated from German by Microsoft

2019-03-06

They do this to sell consulting services.

Problem

You want your OTRS installation to send your agents notifications whenever a ticket is created through the web service interface.

Analysis

The default notification configuration does not do this, perhaps because the triggering event is NotificationNewTicket, whatever that is.

Solution

Create a new ticket notification, or change the existing one, and set the “Event” field to TicketCreate.

2019-03-05

en_?? ?

Problem

When replying to or forwarding a message in Outlook, the attribution header uses a 12-hour clock (AM/PM).

You are of the opinion that this format is a remnant from the Dark Ages, when it was, well, dark and people had to be told whether it was day or night, and so you would really like Outlook to stop that.

Analysis

Outlook is being excessively clever here, as usual. It has an option to override the language used for formatting the attribution header, and this option overrides not just the language but the entire locale with the (im)moral equivalent of en-US.

Solution

In Options/Advanced/International options, turn off “Use English for message headers on replies and forwards and for forward notifications. [sic]".

Outlook will now use the format defined in Windows regional settings. It will also use the language defined there.

2019-02-25

Double speed

ZFS backup via SSH tunnel. Throughput without SSH compression is ~4 Mb/s. With compression, it is >30 Mb/s (of compressed data!), and about 400 MiB/min net (about 1.7:1 compression). How come?

2019-02-19

Ein Quell der Freude
#include <algorithm>
std::vector<int> spaces = std::accumulate(lines.begin() + 1, lines.end(),
        std::vector<int>(max_length, 1), [](auto acc, auto line) {
    std::vector<int> tmp(acc.size(), 0);
    std::transform(line.begin(), line.end(), acc.begin(), tmp.begin(), 
            [](wchar_t ch, int b) {
        return b && (ch == L' '); 
    });
    return tmp;
});