On a side-note, PHP (5.0.4, but probably other versions too) can cache gethostbyname information.
In short, once PHP looks up an address, it may not actually perform another lookup as you may expect. In my particular case (I think) the problem was a change to resolv.conf didn't take effect inside PHP (although nslookup/ping etc worked fine). Stop/Starting Apache fixed it (although a simple 'restart' (kill -HUP) didn't).
In short, if you change resolv.conf, stop and restart Apache.