Yak Shaving: So There I Am, Shaving a Yak…

An important term to know of in the software development industry is yak shaving. The term is properly defined in Wiktionary, but Wiktionary's etymology section for the term is odd, and it doesn't really convey why hairy cattle is being referred to.

On the web, there are many different anecdotes illustrating yak shaving. In my opinion, the following is the best one. It was posted by the late Bill Gaiennie in 2009 on a website which now only exists in the WayBackMachine. I think it deserves to exist on a live website, so here it is:

I simply wanted to snap some pictures of my dogs running around the park last weekend. As I was about to round up the dogs to head to the park, I went to grab my camera, but then realized that I had left it at work.

So I jumped in the car to run by my office to grab the camera, but realized that I didn’t have a key to get into the building. I knew that a co-worker always had a key, so I started to drive to his house, but then realized that the last time I borrowed something from him, an exotic suit, I never returned it because I had accidentally ripped a pretty big hole in the jacket. I knew that my co-worker wouldn’t trust me with the office key if I didn’t return the suit, sans hole, but I had not been able to get it repaired because it was made out of yak hair, something I wasn’t even sure I could get.

I contacted a tailor who would be able to repair the suit, but wouldn’t be able to do it unless I was able to provide a supply of yak hair so that she could weave it into a fabric, and then from there use the yak fabric to repair the yak suit jacket….

So there I was at the zoo, shaving a yak, all so I could take a few pictures of my dogs at the park.

Disable Fedora's Postgres packages when using PGDG packages

In a poll in the Copenhagen Postgres User Group, I asked folks which software source they use when installing Postgres. For some strange reason, the poll is gone from LinkedIn, but I'm rather sure the majority uses packages from postgresql.org, i.e. the "PGDG" packages.

A word of caution for those who use PGDG package on cutting-edge distributions like Fedora: Your distribution may sometimes get ahead of PGDG, and then your installation will install your distribution's Postgres when updating software. This may wreak havoc, because your distribution's Postgres will have data in a different directory than PGDG's. Consequently:

On RPM-based systems, you should add a line like this in in /etc/yum.repos.d/fedora.repo / /etc/yum.repos.d/fedora-updates.repo:

exclude=postgresql* postgis*

That way there's no ambiguity about which flavor of Postgres packages your system tracks.

 

Overcoming stubborn jstack

Situation: Need to peek into what the different threads of a Java process are doing. But when running jstack, something the following happens after a little while:

target process 1985458 doesn't respond within 10500ms or HotSpot VM not loaded

You could try adding the "-F" flag to jstack, but often it will not work, because of jstack's timeout.

One way to handle this is to create a core dump of the Java process and then running "jhsdb jstack ..." on the resulting core file. Since jhsdb does not have a timeout, this makes a difference.

To generate the core file on Linux, you first need to install gdb on the server, if it's not already around. Then, assuming the Java process ID is 54321:

gdb -p 54321

In the gdb shell, run:

generate-core-file

You should now have a file "core.54321".

Things which may cause trouble for core file generation:

• When running gdb, your current working directory is on a file system which is too small, or you don't have write permissions.
• Your login session has limits on the size of core files. For this, run "ulimit -c unlimited".

My experience is that core file generation will not cause the process to fail (if you answer yes to leave the process running when exiting gdb), but it will freeze, while the core file is being created.

Now, having produced a core file, analyze it without a timeout (it can take a long time, depending on the size of the Java process). One long command line:

jhsdb jstack --exe /usr/lib/jvm/java-17-openjdk-amd64/bin/java --core core.54321 > output.txt

Of course, you may need to adjust the path to the Java which was used by process 54321.

By the way, you may re-use the core file to get a memory dump (one long line):

jhsdb jmap --exe /usr/lib/jvm/java-17-openjdk-amd64/bin/java --core /tmp/core.54321 > 54321-jmap.txt

Fixing LDAP auth for Postgres with AD

I've spent hours trying to find out why I couldn't get Postgres LDAP auth to work against a Samba active directory server in one setup (it worked well against a Samba active directory in another...).

I kept getting this in Postgres' logs:

2025-01-04 19:03:32.037 CET [58282] LOG:  could not search LDAP with scope 2 for filter "(sAMAccountName=troels)" on server "dcsrv.test": Operations error
2025-01-04 19:03:32.037 CET [58282] DETAIL:  LDAP diagnostics: 00002020: Operation unavailable without authentication

Adding this line in /etc/ldap/ldap.conf made things work:

REFERRALS off

In Red Hat derived Linux distributions, the path to ldap.conf is /etc/openldap/ldap.conf.

For why this make a difference, search for "referrals" in the Python FAQ.

(I'm getting the impression it would be nice if "off" was the default for REFERRALS.)