Ubuntu packages

About the packages

The Derecho project provides experimental .deb files for Derecho itself, all its dependencies, and all the dependencies of the Derecho SOSP demo. You can download the .deb files at files/ubuntu.tar.bz2, or find the source code which built those .deb files in our packaging github repo. If you’re using an Ubuntu from before 18.04 or running Ubuntu via the Windows Subsystem for Linux, your Derecho experience will likely be better via the provided VM.

Downloading the package archive

There are lots of ways to download the Ubuntu archive. My personal favorite is wget:

$ wget https://derecho.cs.cornell.edu/files/ubuntu.tar.bz2
--2019-10-25 19:49:06--  https://derecho.cs.cornell.edu/files/ubuntu.tar.bz2
Resolving derecho.cs.cornell.edu... 128.253.49.36
Connecting to derecho.cs.cornell.edu|128.253.49.36|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24452372 (23M) [application/x-bzip2]
Saving to: ‘ubuntu.tar.bz2’

ubuntu.tar.bz2  100%[======>]  23.32M   111MB/s    in 0.2s    

2019-10-25 19:49:06 (111 MB/s) - ‘ubuntu.tar.bz2’ saved [24452372/24452372]

Installing the files

First, unpack the downloaded .tar.bz2:

$ tar -xvf ubuntu.tar.bz2

Next, cd into the new ubuntu/ directory:

$ cd ubuntu

In here should be a very large number of .deb files. Running ls should return:

$ ls
demo-dependencies_0.9-1_amd64.deb  mutils-containers_0.9-1_amd64.deb
derecho_0.9-1_amd64.deb            mutils-tasks_0.9-1_amd64.deb
grpc_1.24.2-1_amd64.deb            mxnet_1.5.1-1_amd64.deb
libfabrics_1.7.0-1_amd64.deb       spdlog_1.2.1-1_amd64.deb
mutils_0.9-1_amd64.deb

You should use apt to install each of these files. But first, we’re going to see just what would be installed via apt by running:

$ echo ./* | xargs apt -y install --dry-run

Study the list of packages apt will install, and especially check what apt will remove. If you see nothing clearly amiss, then go ahead and run:

$ echo ./* | xargs sudo apt -y install

This will install every package in the repository, and all of their dependencies. This turns out to be a lot of packages! On a fresh Ubuntu 18.04 install this can be over 300 packages, and takes more than 20 minutes to download and install on a very fast connection—likely much longer over conference wifi!

Once you’re all done, head over to the Tutorial instructions to continue

Installing one at a time

You may also install these packages individually, but bear in mind that some of these packages depend on each other. For example, you cannot install derecho itself without first having installed every mutils package, spdlog, and libfabrics. The best and easiest way to install the packages is to pass them all to apt all at once, and let it sort out the correct installation order.

Conflicts with installed packages

Derecho requires a custom-built version of spdlog and libfabric which conflict with the versions available in Ubuntu. If you have installed libfabric1 or libspdlog-dev on your system, running the apt -y install command above will automatically remove them and replace them with our custom packages. This is sadly an unavoidable part of working with Derecho. The recommended path forward is to note down which packages apt wants to remove which you would prefer to keep, and to remember to apt install them once you’ve finished using Derecho for the day.

Installing in the Windows Subsystem for Linux

Please note that Derecho does not run well under the Windows Subsystem for Linux (1.x) due to networking extensions missing from the Windows-provided Linux Kernel interface. For the same reason, Derecho will likely not run on Kernels older than Linux 3.16. If you do not have an Ubuntu machine matching these specifications, we recommend giving our VM a try instead. That being said, you’re welcome to continue with WSL, and we’ll try to provide support—but if you start seeing crashes accompanied by SETSOCKOPT errors, it might be time to give our VM a try.

Installing in some other Distro

These packages have only been tested in Ubuntu 18.04 and 19.10. You are welcome to install them on any distro, but we cannot guarantee that the dependencies will be installable (or the resulting installation stable) on other distros. To work with Derecho on non-Ubuntu Linux, we recommend building from source. We also do provide ebuilds for Gentoo, which can serve as a template for building on a ports-style (“source-based”) package manager.

Recommended: Removing Derecho

Because the Derecho packages are not explicitly versioned in a PPA (or other repository structure), updates to these packages will not automatically be detected by your system package manager. Thus once you’re done using Derecho for a while, we recommend removing the packages you’ve installed here. You can do that with:

$ sudo apt remove demo-dependencies derecho grpc libfabrics mutils mutils-containers mutils-tasks mxnet spdlog

As before, please study the list of packages apt will remove. This will include some surprises! If you don’t like the list that apt is suggesting, then please go ahead and remove them and simply re-install the packages you’d prefer to keep.

Why do it this way?

Once you’ve installed the demo-dependencies package, other packages which depend on libfabric1 will now instead depend on Derecho’s custom libfabrics. When you remove Derecho’s libfabrics, apt isn’t smart enough to automatically re-install the Ubuntu-provided libfabric1. Instead, it will remove everything which depended on libfabrics or libfabric1, and you’ll have to manually re-install them. Sorry!

Running the tutorial

If these instructions have worked for you, then head on over to the Tutorial instructions to continue. If they have not for some reason, you should try the VM