Massiv ups for Phusion Passenger

Posted by max on June 03, 2009

I hadn’t worked with a Rails application server since back when Mongrel was the cool thing. Boy, is Phusion Passenger (mod_rails) a wonderful surprise! It is exceedingly rare to see software have such clearly presented installation and support documentation. The installer executable is non-confusingly verbose, telling you what it’s checking for and precisely what to do to fix what’s broken. Bravo, Hongli Lai and Ninh Bui.

Their site design is nifty-looking, too.

Popularity: 35% [?]

HOWTO: Time Machine tweaking: fixing ACLs and extended attributes using fsaclctl and chflags

Posted by max on March 03, 2009

I recently needed to reinstall OS X on my laptop. Once I reinstalled, I my primary username (“max”) didn’t have the same UID that it had had on the prior install. This didn’t cause any problems until I wanted to browse my Time Machine backups while not as root.

Much to my chagrin, I wasn’t able to change ownership of the old backups of /Users/max with a simple chown -R. What was going on here?

Two things. Volumes configured for Time Machine use two mechanisms to keep users (including you too, root) from mucking around too much and making the backups un-useful.

(It’s actually fairly difficult to corrupt a Time Machine backup, due to its novel and clever use of hardlinks, described at some length in this excellent article by John Siracusa, which I actually mentioned in an earlier post. One of the great things about Time Machine is that the files are actually files, not wrapped in some gargantuan monolithic binary clump comprehensible only to a particular flavor of backup software.)

If you need to do major mucking, like I did, this is what you have to deal with:

  • Filesystem ACLs, manipulated with fsaclctl. (Yeah, it’s a damn mouthful. Think “filesystem ACL control” and you’ll be somewhat OK.
  • UFS+ extended filesystem attributes, manipulated with chflags

You might not need to do major mucking. You might want to do parts of this if, for example, you discover that Time Machine just backed up a bunch of Linux ISOs you really don’t want to waste your storage space backing up.

Naturally, you’ll need to be root or using sudo in order to actually change these.

First, you need to disable ACLs on the filesystem being managed by Time Machine. It’s easy. This doesn’t destroy any ACL metadata stored, it just stops (temporarily, if you wish) from being used or enforced.

fsaclctl -p /Volumes/backup_vol -d

where “backup_vol” is whatever the name of your Time Machine volume is.

Next, if you had the same situation as I did where you need to change ownership of a whole branch of your filesystem, you’ll want to use chflags, but probably selectively. Here’s what I did:

find /Volumes/backup_vol/Backups.backupdb/machinename/"Macintosh HD"/Users/max -flags +uchg -print0 | xargs -0 chflags -R nouchg

where machinename is the short hostname of the backed-up machine in question. “Macintosh HD” is the default main volume name, which may or may not be the case for you; don’t forget to enclose the name in quotes if it’s got spaces in it.

For those less Unix-inclined of you, that finds every file in my backed up /Users/max branch that has the “uchg” flag set (no user changes permitted) and unsets the flag. For my purposes, I don’t believe I need to reset the “uchg” flag back, so I didn’t store the changed filenames in order to do so.

Finally I changed the whole branch to belong to me again:

chown -Rh max /Volumes/backup_vol/Backups.backupdb/machinename/"Macintosh HD"/Users/max

There were a couple of symlinks that the above choked on, but they weren’t important enough to me to track down exactly why the ownership of the symlinks wasn’t changing. This may not, obviously, be the case every time. I could have programmatically found them, deleted them, and re-linked them, but it wasn’t worth the effort.

Don’t forget to turn the filesystem ACLs back on:

fsaclctl -p /Volumes/backup_vol -e

Popularity: 49% [?]

/etc/hosts entries on OS X Leopard (10.5.5)

Posted by max on November 21, 2008

I was having a hell of a time making entries in /etc/hosts in Leopard. I’d never had any issue in Tiger. Without getting into the details of the fairly complicated way that Leopard handles name resolution via DNS, directory services, and flatfiles (you can look elsewhere for that), it appears that Leopard is pretty picky aout where precisely you put entries in /etc/hosts.

A virgin /etc/hosts file looks like this in Leopard:

# Host Database
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##       localhost broadcasthost
::1             localhost
fe80::1%lo0     localhost

I thought it would make sense to put in my new IPv4 entries above the IPv6 entries (i.e., between the broadcasthost line and the ::1 localhost line. Right? Wrong. If you put them there, your resolver will happily go on ignoring them until the end of time.

Put them at the bottom of the file and they’ll work.

Addendum 19 May 2009: I’m reading the O’Reilly book “Mac OS X for Unix geeks,” and here’s how to do it the OS X way:

$ sudo dscl . /Hosts/ ip_address

Note that /Hosts/ is not part of the filesystem — it’s part of the Directory Services local database.

The directory entries take precedence over /etc/hosts.

Popularity: 51% [?]

How to learn (and not get burned by) LVM

Posted by max on July 02, 2008

LVM2 is a very useful thingy indeed. I sort of misunderstood its utility for a while. It’s not just for managing storage on mongo SAN clusters; it can be very useful for managing storage on a workgroup box with several drives.

However, on Linux, beware: You cannot boot from an LVM logical volume. You need to have a regular filesystem (ext2/ext3/xfs/jfs/reiserfs/etc.) partition for /boot. I’m not quite sure why I didn’t grasp this when planning my first LVM layout.

The Red Hat documentation on LVM2 is excellent, but is not the first thing that comes up on searches, because it’s buried in the weirdo RHEL system documentation. There’s a similar, RedHat-generated doc which is also good at the CentOS project, available here.

It takes some getting used to to understand LVM, and if you’re like me, it doesn’t hurt to practice. There is some degree of overlap with what RAID can do for you, and using them both together makes one’s head spin at first. Drawing a map of your planned layout is extremely helpful.

For wrangling your storage layouts, the Parted Magic live CD can be very helpful. Unfortunately, not all installers want to respect what you’ve laid out with it. Fedora 9 played nice. Ubuntu 8.10 (alternate install CD — essential for RAID installs) was not; it kept not respecting the /dev/md software RAID devices that I’d previously set up, then would create some new bogus md devices.

Popularity: 61% [?]

DHCP on VMware Fusion

Posted by max on March 04, 2008

VMware Fusion is a wonderful thing. I am happily running three Linux instances at once under my MacBook with 4GB of RAM (though, as best I can tell, the mobo can actually only address 3GB of RAM).

One glitch that was driving me crazy, though, was that it seemed like DHCP leases were being allocated flakily to the guest machines. When you’re doing database replication, you can’t have master/slave IP addresses changing all the time. (And you certainly don’t want two machines with the same IP address!)

Some digging around turned this up. I don’t remember exactly where — I believe snippets of it were in an official VMware forum — but it was harder to find than it should have been. Hence my posting, to add another path to help those who need it.

On OS X, the file you need to change should be in /Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf. (vmnet8 is the virtual interface for NAT networking in the guest machines.) You should have a subnet clause in it already like this:

subnet netmask {
    option broadcast-address;
    option domain-name-servers;
    option domain-name "localdomain";
    option routers;

Now, for each virtual machine, set up a new host clause, e.g. :

host ubuntu64-vm {
    hardware ethernet 00:0F:FF:EF:00:00;

Be sure to assign an IP address (fixed-address) outside the range specified in the subnet clause at top.

Get the Ethernet MAC address from ifconfig eth0 inside the guest machine, or from ~/Documents/Virtual Machines/GUESTMACHINENAME/GUESTMACHINENAME.vmx (it’s the line with ethernet0.generatedAddress).

Finally, restart the requisite VMware daemons by running

sudo "/Library/Application Support/VMware Fusion/" --restart

Popularity: 100% [?]

Airport Extreme is a big disappointment. Or, (10?) Things I Hate About Apple

Posted by max on October 05, 2007

I bought the elegant little Apple Airport Extreme back in the summer. Network disk (pseudo-NAS) support! Printer sharing! Fantastic! Besides the fact that our last wireless router was puking to death again (why does that always happen to us after <2 years? Do I emit gamma rays from my head?), I was excited to enable USB printer networking and shared network drives.

Only in both cases, they don't work. At least not in a way that is useful to me.

1) Printer sharing

There are two defects here: one design, one implementation. After trying to figure out why I couldn't do anything that my fancy-ass new multifunction printer that Canon sells as a loss leader for its pricey ink cartridges, I discovered that it it normal that non-printing functions don’t work unless the unit is directly connected to the computer. Networks are for printing only. No scanning, no ink maintenance fancy stuff, just printing. That’s the design problem, and it’s a major bummer, but it wasn’t a deal killer. (OK, well, nothing was a deal killer; I bought the damn thing.)

Nope, what ended network printer sharing for me was that the shared network printer would periodically disappear or otherwise become unavailable for users on the network. Had to delete the printer and re-add it, or at the very least power down the router and printer and power up again. That’s crap. So now the way I provide reliable-ish network sharing is to connect it directly to my computer and share it as a network printer.

That’s crap.

Grade: F. I would blame Canon, because they make great hardware and execrable software, but I had a similar problem with my old HP Laserjet 6MP printer.

2) Drive sharing

I finally got my ass in gear and moved my old 250GB USB drive from my slowly-decommissioning Windows machine onto the Airport Extreme. After many slow reboots of the router (yeah, any change requires a reboot; couldn’t it at least boot faster, damnit?! I’m not even getting a DHCP lease from my ISP — it’s static.), I discover that the Airport Extreme is unable to share NTFS volumes. Well, that’s not so multi-platform, now, is it? FAT32 is insecure and doesn’t work for big volumes, say, any hard drive created after about 1998. HFS+ would have been an option if this drive were originally on a Mac, but it wasn’t. So my best options are:

  • Copy all of the data off the drive and reformat to HFS+; remount on the Airport Extreme after getting their wonky SMB networking
  • Abandon the project, as the USB drive is already 5 years old and isn’t worth spending a lot of time monkeying with

Guess which one I picked?

Grade: D+. You’ll note that the spec sheet mentions nothing about not supporting NTFS. I’d like to blame M$ for that one, but there are plenty of NAS vendors that offer NTFS support.

While I’m at it, let me complain about Apple file sharing. You seem to be able to choose between a yucky slow SMB implementation and AFP (which does appear to be much faster, at least for authenticating). I have grown frustrated with both and am now very, very happily using instead sshfs, which is great both on the LAN and across the Internet. Too bad ‘Doze doesn’t support user-space filesystems; so no sshfs for Windows.

(To run SSHFS, I use Google’s MacFUSE project and their SSHFS executable. I don’t remember exactly why, but the MacFusion GUI doesn’t work on my machine.)

Popularity: 19% [?]

Installing Ruby 1.8.6 on Ubuntu Feisty Fawn (7.04)

Posted by max on October 04, 2007

Ubuntu seems to be the big distro these days. (Remember all of those other “it” distros? Slackware? The original Red Hat? Mandrake? I am vaguely tired of the trendiness of distros.)

Anyway, there is no package yet for Ruby 1.8.6 on Ubuntu Feisty Fawn (7.04), which is unfortunate, since mongrel_cluster doesn’t work properly under Ruby 1.8.5.

The install steps I got at Urban Puddle (here) were nice, but Rails still wasn’t happy: I kept getting that awful `require’: no such file to load – rubygems (LoadError) error.

Here’s what I did to fix it. YMMV.

Building Ruby (from Urban Puddle):

tar xjvf ruby-1.8.6.tar.bz2
cd ruby-1.8.6
apt-get build-dep ruby1.8
./configure --prefix=/usr
sudo make install

Next, to fix it:

cd /usr/src/rubygems/rubygems-0.9.4
sudo ruby setup.rb

Et voilà :

max@somehost:~ $ irb
irb(main):001:0> require 'rubygems'
=> true

I’m not quite sure why the Ruby source doesn’t ship with the rubygems source. Is it because rubygems is still a < 1.0 release?

Popularity: 18% [?]

Beware Trac and Unicode

Posted by max on September 25, 2007

I just blew a frightening amount of time (mostly unbilled) tracking down the most aggravating problem I’ve seen in a while.

The way that the “production” version of Trac (which is at 0.10) works with Unicode is a bit dim. Somewhere laced in there it likes to use the default encoding of latin-1 instead of utf-8. (It’s not in the standard location, though.)

The errors I was getting when trying to sync Trac to the SVN repository were like this:

2007-09-16 22:51:59,995 Trac[cache] WARNING: Revision 11 already cached: 'latin-1' codec can't encode character u'\ufffd' in position 43: ordinal not in range(256)

I tried a number of approaches, including rebuilding swig, subversion, python, and some others. None solved the damn problem, though I obviously encountered new ones along the way. Amy managed to refocus me properly on the goal by verifying that the problem was indeed caused by Unicode characters in filenames the Subversion repository, and by picking at the Trac code that inserts SVN data into its database. In attacking that problem I converted the Trac database to the utf-8 character set from latin-1, which would have been a problem. That still didn’t fix it, though.

It wasn’t until I tried resyncing using trac 0.11dev’s trac-admin tool that it finally worked.

Trac’s site does warn that 0.10 still has some problems with Unicode. There’s even a ticket pointing out a fix which I really didn’t want to apply myself, knowing it would get rebroken at some point. (One commenter noted that “this should be backported to the 0.10 line”, but unfortunately that doesn’t seem to have happened.)

Update: I’m fairly confident that, had I started with a utf8 MySQL database to begin with instead of latin1, I would have avoided all of these woes. But that wasn’t my starting point. (Fun fact: Python calls them “latin-1″ and “utf-8″, while MySQL calls them “latin1″ and “utf8″. Strangely enuf, that difference didn’t cause me any problems.)

Popularity: 12% [?]