DTrace flamegraphs for FreeBSD and node.js

Introduction

Recently BSD UNIXes are being acknowledged by the application development community as an interesting operating system to deploy to.

This is not surprising given that FreeBSD had jails, the original container system, about 17 years ago and a lot of network focused businesses such as netflix see it as the best way to deliver content.

This developer interest has led to hosting providers supporting FreeBSD.
e.g. Amazon, Azure, Joyent and you can get a 2 months free instance at Digital Ocean.

DTrace is another vital feature for anyone who has had to deal with production issues and has been in FreeBSD since version 9. As of FreeBSD 11 the operating system now contains some great work by Fedor Indutny so you can profile node applications and create flamegraphs of node.js processes without any additional runtime flags or restarting of processes. To understand this graphic further the 'father of flamegraphs' Brendan Gregg has provided a great deep dive that also has some entertaining historical context.

Instructions

In order to configure your FreeBSD instance to utilize this feature make the following changes to the configuration of the server.

  1. Load DTrace when the OS starts - edit /boot/loader.conf or /boot/loader.local.conf add the line dtraceall_load="YES" You can also load it without rebooting by running kldload dtraceall

  2. Next we need to allow the kernel to load DTrace Object Files greater that 128k - edit /etc/sysctl.conf and add
    kern.dtrace.helper_actions_max=16000 Again you can run this without a reboot with # sysctl -w kern.dtrace.helper_actions_max=16000

  3. Install the build tools
    # pkg install python gcc git gmake

  4. Now install node.js for FreeBSD from ports to get the latest stable release.
    # portsnap fetch
    # portsnap extract
    # cd /usr/ports/www/node6
    # make install clean - Choose the following options

  5. npm is also required
    # cd /usr/ports/www/npm3
    # make install clean
    You may also want to configure npm to install globals to the home directory so you don't have to su to often.

  6. Install stackvis The visualisation tool managed by Dave Pacheco
    # npm install -g stackvis

  7. Get the OS source to compile the dtrace libs
    # curl http://ftp.freebsd.org/pub/FreeBSD/releases/amd64/11.0-RELEASE/src.txz > src.txz

  8. Then extract them into the default location
    # tar -C / -xvf src.txz

  9. We now need to update stackvis. This is a little bit messy while this patch waits to land and stackvis will be updated.
    # cd /usr/local/lib/node_modules/stackvis/node_modules
    # rm -fr dtrace-provider
    # git clone https://github.com/no9/node-dtrace-provider.git
    # mv node-dtrace-provider dtrace-provider
    # cd dtrace-provider
    # git submodule init
    # git submodule update
    # npm install

You will now be able to follow the flamegraphs tutorial mentioned earlier.

Summary

I hope you find this article useful. The ability to look at a runtime in this manor has saved me twice this year and I hope it will save you in the future too.
My next post on freeBSD and node.js will be looking at some scenarios on utilising the ZFS features.

Attribution

The patch to the node-dtrace-provider is only a small part of this story - All the credit goes to Fedor Indutny, Brendon Gregg and Dave Pacheco for sharing so much information and experience on this topic and making it interesting to get involved in.