Massiv ups for Phusion Passenger

Posted by max on June 03, 2009

I hadn’t worked with a Rails application server since back when Mongrel was the cool thing. Boy, is Phusion Passenger (mod_rails) a wonderful surprise! It is exceedingly rare to see software have such clearly presented installation and support documentation. The installer executable is non-confusingly verbose, telling you what it’s checking for and precisely what to do to fix what’s broken. Bravo, Hongli Lai and Ninh Bui.

Their site design is nifty-looking, too.

Popularity: 35% [?]

HOWTO: Time Machine tweaking: fixing ACLs and extended attributes using fsaclctl and chflags

Posted by max on March 03, 2009

I recently needed to reinstall OS X on my laptop. Once I reinstalled, I my primary username (“max”) didn’t have the same UID that it had had on the prior install. This didn’t cause any problems until I wanted to browse my Time Machine backups while not as root.

Much to my chagrin, I wasn’t able to change ownership of the old backups of /Users/max with a simple chown -R. What was going on here?

Two things. Volumes configured for Time Machine use two mechanisms to keep users (including you too, root) from mucking around too much and making the backups un-useful.

(It’s actually fairly difficult to corrupt a Time Machine backup, due to its novel and clever use of hardlinks, described at some length in this excellent article by John Siracusa, which I actually mentioned in an earlier post. One of the great things about Time Machine is that the files are actually files, not wrapped in some gargantuan monolithic binary clump comprehensible only to a particular flavor of backup software.)

If you need to do major mucking, like I did, this is what you have to deal with:

  • Filesystem ACLs, manipulated with fsaclctl. (Yeah, it’s a damn mouthful. Think “filesystem ACL control” and you’ll be somewhat OK.
  • UFS+ extended filesystem attributes, manipulated with chflags

You might not need to do major mucking. You might want to do parts of this if, for example, you discover that Time Machine just backed up a bunch of Linux ISOs you really don’t want to waste your storage space backing up.

Naturally, you’ll need to be root or using sudo in order to actually change these.

First, you need to disable ACLs on the filesystem being managed by Time Machine. It’s easy. This doesn’t destroy any ACL metadata stored, it just stops (temporarily, if you wish) from being used or enforced.

fsaclctl -p /Volumes/backup_vol -d

where “backup_vol” is whatever the name of your Time Machine volume is.

Next, if you had the same situation as I did where you need to change ownership of a whole branch of your filesystem, you’ll want to use chflags, but probably selectively. Here’s what I did:

find /Volumes/backup_vol/Backups.backupdb/machinename/"Macintosh HD"/Users/max -flags +uchg -print0 | xargs -0 chflags -R nouchg

where machinename is the short hostname of the backed-up machine in question. “Macintosh HD” is the default main volume name, which may or may not be the case for you; don’t forget to enclose the name in quotes if it’s got spaces in it.

For those less Unix-inclined of you, that finds every file in my backed up /Users/max branch that has the “uchg” flag set (no user changes permitted) and unsets the flag. For my purposes, I don’t believe I need to reset the “uchg” flag back, so I didn’t store the changed filenames in order to do so.

Finally I changed the whole branch to belong to me again:

chown -Rh max /Volumes/backup_vol/Backups.backupdb/machinename/"Macintosh HD"/Users/max

There were a couple of symlinks that the above choked on, but they weren’t important enough to me to track down exactly why the ownership of the symlinks wasn’t changing. This may not, obviously, be the case every time. I could have programmatically found them, deleted them, and re-linked them, but it wasn’t worth the effort.

Don’t forget to turn the filesystem ACLs back on:

fsaclctl -p /Volumes/backup_vol -e

Popularity: 49% [?]

/etc/hosts entries on OS X Leopard (10.5.5)

Posted by max on November 21, 2008

I was having a hell of a time making entries in /etc/hosts in Leopard. I’d never had any issue in Tiger. Without getting into the details of the fairly complicated way that Leopard handles name resolution via DNS, directory services, and flatfiles (you can look elsewhere for that), it appears that Leopard is pretty picky aout where precisely you put entries in /etc/hosts.

A virgin /etc/hosts file looks like this in Leopard:

# Host Database
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##       localhost broadcasthost
::1             localhost
fe80::1%lo0     localhost

I thought it would make sense to put in my new IPv4 entries above the IPv6 entries (i.e., between the broadcasthost line and the ::1 localhost line. Right? Wrong. If you put them there, your resolver will happily go on ignoring them until the end of time.

Put them at the bottom of the file and they’ll work.

Addendum 19 May 2009: I’m reading the O’Reilly book “Mac OS X for Unix geeks,” and here’s how to do it the OS X way:

$ sudo dscl . /Hosts/ ip_address

Note that /Hosts/ is not part of the filesystem — it’s part of the Directory Services local database.

The directory entries take precedence over /etc/hosts.

Popularity: 51% [?]

Where to catch up on OS X tech stuff

Posted by max on September 30, 2008

I’ve long felt much shakier about the OS X internals than I have about Linux or Solaris. It’s not been that clear to me where to turn to get solid OS X technical overviews without getting too distracted by fluff (OS X user sites) or by more detail than I necessarily needed at once (Apple’s OS documentation.)

Enter Ars Technica’s John Siracusa. The breadth he uses to review OS X Leopard (10.5) is amazing — he covers everything from kernel details to user-interface design decisions. The review is here, but he’s got plenty of other excellent articles covering great OS X topics like the Finder, HFS+, and the evolution of OS X file metadata.

Siracusa is one of the best technical writers I’ve read in a long time.

Popularity: 53% [?]

/usr/bin/file rules, even when it’s wrong

Posted by max on July 11, 2008

‘file’ is a great utility. Even when it’s wrong, it can at least be good for an audible laugh:

$ dd if=/dev/urandom of=fakefile.bullshit bs=4096 count=400
400+0 records in
400+0 records out
1638400 bytes (1.6 MB) copied, 0.307462 s, 5.3 MB/s
$ file fakefile.bullshit
fakefile.bullshit: DOS executable (COM)

Popularity: 60% [?]

How to learn (and not get burned by) LVM

Posted by max on July 02, 2008

LVM2 is a very useful thingy indeed. I sort of misunderstood its utility for a while. It’s not just for managing storage on mongo SAN clusters; it can be very useful for managing storage on a workgroup box with several drives.

However, on Linux, beware: You cannot boot from an LVM logical volume. You need to have a regular filesystem (ext2/ext3/xfs/jfs/reiserfs/etc.) partition for /boot. I’m not quite sure why I didn’t grasp this when planning my first LVM layout.

The Red Hat documentation on LVM2 is excellent, but is not the first thing that comes up on searches, because it’s buried in the weirdo RHEL system documentation. There’s a similar, RedHat-generated doc which is also good at the CentOS project, available here.

It takes some getting used to to understand LVM, and if you’re like me, it doesn’t hurt to practice. There is some degree of overlap with what RAID can do for you, and using them both together makes one’s head spin at first. Drawing a map of your planned layout is extremely helpful.

For wrangling your storage layouts, the Parted Magic live CD can be very helpful. Unfortunately, not all installers want to respect what you’ve laid out with it. Fedora 9 played nice. Ubuntu 8.10 (alternate install CD — essential for RAID installs) was not; it kept not respecting the /dev/md software RAID devices that I’d previously set up, then would create some new bogus md devices.

Popularity: 61% [?]

sometimes 37 signals is so right on

Posted by amy on April 20, 2008

They just went to a four-day work week. “Urgency is poisonous!” says Jason. I couldn’t agree more.

Popularity: 65% [?]

It is hard to write a book, and other whining.

Posted by amy on March 08, 2008

Dear Internets,

I have been neglecting you. That’s because I started a new job last month, at a place called Smartleaf, doing Ruby on Rails development for 20 hours a week. It is fantastic. It is a lot of fun. It’s great people, good food, easy commute, and coding. I haven’t been an employee for years, so it’s quite a change for me, and in a good way. (It turns out that you can write better code without children yelling in your ear.)

But I’ve got these kids and this life and this job and I’m kinda busy.

Also, I’m supposed to be writing a book about computer science for practicing programmers. It’s meant to be an overview and sourcebook for people who didn’t get CS degrees but are working as programmers. Which, it turns out, is most of us. And when I signed on to write it, I hadn’t actually anticipated that a great job opportunity would drop into my lap, the way this one did, so I thought I’d have four or five hours a day to work on it. But suddenly I have about three minutes a day to work on it, and I’m having a really hard time. The subject matter is not easy. It is not just something I can write in my sleep. I don’t know the stuff I plan to write about. And even though I’ve lately become a big believer in the idea that you can accomplish an awful lot in just fifteen minutes, I fear that it is turning out not to be the case that you can write a book about computer science in just fifteen minutes a day.

Actually I emailed Steve Yegge about the book (partly because he was an inspiration for my wanting to write it, and partly because I wanted to use some quotes from his blog in it and wanted his approval) and he wrote back and said something like (I’m paraphrasing here): Hah, good luck. Many have tried, all have failed to write such a book. Which was a little bit depressing and discouraging, but honestly, what do you expect from Stevey?

Still, my editor is getting restive (hi Daniel!) and with good reason. Maybe we should put the book on hold, he asked, and see if it’s actually going to happen? Ick, I said. That is a terrible idea. I am going to write the damn book. Of course I can do it.

But I don’t know how. I don’t know how to do it, internets. I need your help, hive mind. I asked my friend John if he wanted to write the book with me, and he was like “hah, no.” All he does is send me links to stuff that I should read for the book, and recommend books that would be helpful, and talk about how much he wants to read the book. I asked Max to make me work on it for an hour every morning before the kids wake up, but the kids somehow magically sense when I am awake and wake up with me. And I have never been a morning person and I just can’t think at 6:30 a.m. And I have two hours on Saturday morning to myself when the kids go out to the French library with Max, so today I managed to clean the bedroom, set the robot to vacuum, and dump all the clean laundry on the bed in an enormous mountain so that I am forced to fold it before bedtime. I also went for a short walk and stopped in at the bookstore. And finally started typing this and as I am writing the family is returning, and a baby is running down the hall crying for me. So that was my morning.

And I realize how obnoxious it is that, out of amazing good fortune, I have a book contract, an actual book contract, with people who are my programming heroes, and here I am complaining about it, and basically just screwing it up.

So this is what happens: i get some bursts of energy to work on the book — like I spent two weeks reading and learning a bunch of stuff about C, and figuring out all the different ways to get segfaults, and I wrote a few pages of notes for a chapter on C, and I found awesome documents like the ANSI C Rationale on the internet, and I also read a bunch of stuff about machine architecture and cache memory and pipelining and bitwise operations and analog computing and I took a bunch of notes, and I thought a lot, and I wrote some stuff, and I made what progress I made by basically doing nothing but that, work, and cooking dinner. I mean, I basically ignored my children. I’d be like “uh huh, that’s great, but I’m really trying to understand the semantics of arrays in C right now, sweetie pie.” Oh, and I didn’t get quite enough sleep. After a couple of weeks of that I needed to do things like make plans for our trip to Europe in May (for my oldest friend’s wedding, not just because we’re decadent francophilic wine-and-cheese freaks, although we are that too, but man, that Euro is PAINFUL now) and get the taxes ready to send off to the accountant, and fight with the insurance company about various claims they refuse to pay for various ridiculous reasons. And read to my kids, and spend ten minutes alone with my husband, and parent help at Ari’s preschool, and everything else. And honestly, our lives aren’t even that busy. Our kids are not in a ton of activities. We don’t schlep around to baby music and movement classes. Ari is in no organized sports. He goes to preschool three mornings a week. Max and I each work 20 hours a week, and we have about 18 hours of babysitting.

But somehow I just can’t manage it.

What this tells me, first of all, is that nobody out there is managing it. We have far, far more time than most other parents with two children under five, and it’s still not ‘enough’. There’s an endless pile of stuff that I think should be getting done that isn’t getting done. This tells me that my ideas about what has to actually get done are just wrong, because I’m not getting a lot of these things done but things go on just the same. So I should probably just drop that stuff off my list. Like “inventory our crap for insurance purposes.” And “clean out the basement storage room”. And “start tomato seeds.”

But honestly, I’ve already dropped all that off my list. I’ve even dropped baking bread off my list, for the time being, and I’m someone who has an actual grain mill sitting on her counter. I grind my own damn wheat to make my own damn bread, and I haven’t done it in a couple of months.

I don’t watch TV or movies. The entire season 3 of Lost is sitting on my dresser waiting to be watched, but it’s an enormous commitment and I just don’t see how I can make the time. I’ve cut my newsfeeds down. I do no recreational reading except that I sometimes give an hour to the New Yorker.

Look, internets, I really want to write this book. It’s a lot of fun, except when it’s so painful I want to die or puke or something. I’m learning a lot. I think I could help other programmers figure out how to learn the stuff about computer science that they want to learn, and help them figure out why they should care. But it turns out, unsurprisingly, to be really, really, really hard to write a book. At least this kind of book. Well, I’m sure any book, really.

But am I just a big lazy whiner? I honestly don’t know. I thought maybe the book would be a year-long project, and it looks to be more like a five-year-project, especially at the pace I’m going. Now it’s true that Don Knuth is still working on his opus, but I’m not Don Knuth. Does the world need what I have to offer, or am I torturing myself and my family for an unnecessary project?

I keep thinking, okay, well, I didn’t make much progress that way, I’ll try some other way. And so I try all the hacks I know of to motivate, and make time, and make progress. And I keep trying in different ways. And when I drop it for a few days and fall off the wagon and stop meeting my goals, I just try again. So I’m sorta impressed with myself about this, that I keep trying, even in the face of basically constant failure and what feels like a pathetic lack of progress. But boy does it take an emotional toll. Writing this book (or mostly not managing to write it, as the case may be) is incredibly emotionally and intellectually taxing. Not to mention time-consuming.

I know we don’t have a ton of blog readers out there, really. But if anyone out there wants to offer some encouragement, or has any suggestions, I’d love to hear from you.

Popularity: 73% [?]

DHCP on VMware Fusion

Posted by max on March 04, 2008

VMware Fusion is a wonderful thing. I am happily running three Linux instances at once under my MacBook with 4GB of RAM (though, as best I can tell, the mobo can actually only address 3GB of RAM).

One glitch that was driving me crazy, though, was that it seemed like DHCP leases were being allocated flakily to the guest machines. When you’re doing database replication, you can’t have master/slave IP addresses changing all the time. (And you certainly don’t want two machines with the same IP address!)

Some digging around turned this up. I don’t remember exactly where — I believe snippets of it were in an official VMware forum — but it was harder to find than it should have been. Hence my posting, to add another path to help those who need it.

On OS X, the file you need to change should be in /Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf. (vmnet8 is the virtual interface for NAT networking in the guest machines.) You should have a subnet clause in it already like this:

subnet netmask {
    option broadcast-address;
    option domain-name-servers;
    option domain-name "localdomain";
    option routers;

Now, for each virtual machine, set up a new host clause, e.g. :

host ubuntu64-vm {
    hardware ethernet 00:0F:FF:EF:00:00;

Be sure to assign an IP address (fixed-address) outside the range specified in the subnet clause at top.

Get the Ethernet MAC address from ifconfig eth0 inside the guest machine, or from ~/Documents/Virtual Machines/GUESTMACHINENAME/GUESTMACHINENAME.vmx (it’s the line with ethernet0.generatedAddress).

Finally, restart the requisite VMware daemons by running

sudo "/Library/Application Support/VMware Fusion/" --restart

Popularity: 100% [?]

No Ranting, No Railing, Just Grading

Posted by amy on January 15, 2008

A few of our students, people new to ruby and rails, are pretty turned off by the ranting and railing in the rails community that’s been on super-duper heavy display lately, what with the Zed thing and the Dreamhost thing and all. (No links, google Zed Shaw and Dreamhost Rails for your ownselves.)

These students want to know: what’s up with the rails community being so childish, ranty, obscene, profane, and macho? If rails wants to be treated like a grownup framework for grownups, shouldn’t rails act like a grownup? How are we supposed to take rails seriously when the guy in charge of rails thinks people who are offended by swearing are not worth his time? And why should we want to join a community that appears so unwelcoming?

To which I say the following:

First, my job is not to defend rails and all the people associated with it (Right, say students, your job is actually to finish grading our assignments. Could you finish?!? I’m working on it, I swear). If Rails is a ponzi scheme I have not been let in on it; I get no kickbacks from duping students into learning it. I am enthusiastic but not passionate about rails ( I believe, as I’ve said before, that my only true passion may be cheese), and I hope I’ve managed to share some of my enthusiasm with our students, without appearing cultish.

Second, I could point out that the Ruby and Rails communities are not isomorphic. There is a lot going on in Ruby that is not rails. But it’s also true that many people come to Ruby through Rails, so if the rails community is unwelcoming, then fewer people may make it over the hump to discover the Ruby in which Rails is built. That’s too bad, because they’d be missing out on something truly lovely. Still, there are other truly lovely languages out there too. “Let a hundred flowers bloom, let a hundred schools of thoughts contend,” except without all the reeducation camps.

Third, it’s possible that Rails will hit a wall due to personality problems. It’s also possible that, being young, Rails will grow out of its personality problems. If you don’t like the personality of the rails community right now, just wait, it’ll probably change. Or jump right in there with your own personality problems, mix it up a little.

Whatever happens, though, I think that Rails has a few good tricks that the rest of the web development community has and will continue to learn from. I think that Rails, like all frameworks, will someday go out of fashion, and then, out of date, and then, eventually, fade into obscurity. And I think more good and interesting and beautiful things will come out of Ruby.

In any case, as I wrote to one of the students who was a bit depressed by the recent rails flame wars:

For me, I know that if it had not been for rails getting popular, I would not have gotten into Ruby, and if I had not gotten into Ruby, it might have been much, much longer before I’d realized that life could get a whole lot better than:

Iterator iter = collection_thingy.getIterator();
while (iter.hasNext()) {
Thingy thingy = (Thingy);

No one can take each away from me.

Popularity: 76% [?]