Setup Package Cache Server on Ubuntu 18.04

If you’re like me and have several Debian/Ubuntu machines on your network there’s going to come a time when you need to upgrade them. Doing so, will use up a lot of bandwidth while every machine will likely be downloading the same packages. This may or may not upset your significant other who’s binge-watching Gilmore Girls on Netflix.

Since you’ve slowed the Internet down to a crawl, this it might  be a good excuse to leave the computer and get outside for some fresh air. HA, who am I kidding, we got stuff to do. Let’s setup a cache!

Here, I’ll be using Ubuntu 18.04 LTS and setting up apt-cacher-ng.

While we could setup Squid to function in the same way, and cache way more than just debian/ubuntu packages, using apt-cacher-ng is a quick win and requires hardly any configuration to get going. Maybew I’ll cover how to setup Squid in a future post.

First, we’ll make sure everything is up to date, then install apt-cacher-ng.

sudo apt-get update
sudo apt-get dist-upgrade -y
sudo apt-get install apt-cacher-ng -y

Let’s go over a few config options. We won’t go over every single one, just the ones that might be relevant. Open /etc/apt-cacher-ng/acng.conf using your favorite text editor and let’s start.

CacheDir. This is where acng will actually do its caching and store packages as they’re downloaded. You may want to change this if you’d like to save packages to a different partition with more space.

CacheDir: /var/cache/apt-cacher-ng

Port. Here, you can change the TCP port apt-cacher will listen on. Note, this should be higher than 1024. Otherwise, you would need to run acng as root.

Port: 3142

BindAddress. If you have a multi-honed server with several IP addresses, you might want acng to only listen on one. Just provide the IP here to do so. By default, it will listen on all interfaces (0.0.0.0).

BindAddress

ReportPage. If you’d like to see some misc statistics about the caching of packages (hit/miss ratio, space usage, etc…) set the page name here. To disable it, just comment out this line.

ReportPage: acng-report.html

Here’s how the page looks from my server:

ExThreshold. The number of days before deleting unreferenced files. You may want to tweak this to cache packages for longer periods of time.

ExTreshhold: 4

MaxDlSpeed. Here, you can limit how much bandwidth acng will use up. Very handy in some environments. Units are KiB/s.

MaxDlSpeed: 250

There are several other options layed out in the config file, feel free to read more on them and tweak as needed. For now, let’s move onto setting up our hosts to point at our new cache server for packages.

Log into one of your Ubuntu/Debian machines and create a new file at /etc/apt/apt.conf.d/21acng. Replace SERVER_IP with the IP address of the cache server you setup. If you specified a BindAddress above, use that one instead.

echo '"Acquire::HTTP::proxy "http://SERVER_IP:3142";' | sudo tee /etc/apt/apt.conf.d/21acng

Now, let’s try it out! From this client machine, run:

sudo apt-get update

Back on the caching server, you can see what’s happening by tailing the log file.

sudo tail -f /var/log/apt-cacher-ng/apt-cacher.log
.......
............
1532529680|O|227|10.137.5.1|ppa.launchpad.net/openjdk-r/ppa/ubuntu/dists/bionic/InRelease
1532529680|O|228|10.137.5.1|packages.cloud.google.com/apt/dists/cloud-sdk-bionic/InRelease
1532529680|O|217|10.137.5.1|dl.google.com/linux/chrome/deb/dists/stable/Release
.........
..............

Don’t forget, you can access the statistics page by opening a web browser to http://SERVER_IP:3142/acng-report.html

Nginx and Letsencrypt on SmartOS

acme_tiny is a nice, small utility for creating and renewing your doman SSL/TLS certificates.

It’s less than 200 lines of bash and just works. Here’s how to set it up and have your certificate automatically renewed once a month.

First, let’s get things installed; nginx and the acme-tiny client and setup the necessary directories.

pkgin in nginx py36-acme-tiny
mkdir -p /opt/local/etc/acme /opt/local/www/acme

Add the following stanza towards the top of your nginx config and reload nginx.

vi /opt/local/etc/nginx/nginx.conf
  location ^~ /.well-known/acme-challenge/ {
     alias /opt/local/www/acme/;
     try_files $uri =404;
  }
nginx -s reload

Now, let’s setup the necessary keys so we can request a new certificate for our site.

cd /opt/local/etc/acme
openssl genrsa 4096 > account.key
openssl genrsa 4096 > domain.key
openssl req -new -sha256 -key domain.key -subj "/CN=shaner.life" > domain.csr

Now we can create the initial request for our domain.

acme_tiny   --account-key /opt/local/etc/acme/account.key \
            --csr /opt/local/etc/acme/domain.csr \
            --acme-dir /opt/local/www/acme \
            > /opt/local/etc/acme/signed.crt
curl -s 'https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem' > /opt/local/etc/acme/intermediate.pem
cat /opt/local/etc/acme/signed.crt /opt/local/etc/acme/intermediate.pem > /opt/local/etc/acme/fullchain.pem

If you get an error complaining about the expired terms of service pdf don’t fret, we just need to update the acme script with the name of the updated version.

vi /opt/local/bin/acme_tiny
 # search for 'pdf' and replace the url with the one from the error you recieved during your (failed) certificate request.

If you got an error complaining about not being able to access the acme-challenge token, double check your nginx config and be sure to restart nginx.

One last step before we can setup nginx with our new certs. We need to create a diffie-helman key. This will take a while, go grab some coffee and when you get back it should be done.

openssl dhparam 4096 > /opt/local/etc/nginx/dhparam.pem

Now that we have our dh key, certificate and private key. Let’s setup nginx to use them. Here are the relevant SSL bits that can be added to your server stanza in _opt/local/etc/nginx.conf_ .

server_tokens off;
ssl_certificate /opt/local/etc/acme/fullchain.pem;
ssl_certificate_key /opt/local/etc/acme/domain.key;
ssl_dhparam dhparam.pem;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers 'ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DH+3DES:!ADH:!AECDH:!MD5';

Let’s automate this thing. Here, we’re creating a script to be called by a cron job for automatic renewal of our certificate. We do this on a somewhat frequent schedule as lets-encrypt only issues certificates that are valid for 3 months.

cat > /opt/local/etc/acme/renew.sh <<EOF
#!/bin/bash
acme_tiny --account-key /opt/local/etc/acme/account.key --csr /opt/local/etc/acme/domain.csr --acme-dir /opt/local/www/acme > /opt/local/etc/acme/signed.crt
curl -s 'https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem' > /opt/local/etc/acme/intermediate.pem
cat /opt/local/etc/acme/signed.crt /opt/local/etc/acme/intermediate.pem > /opt/local/etc/acme/fullchain.pem
cp fullchain.pem /opt/local/etc/nginx/ssl/fullchain.pem 
nginx -s reload
EOF

Okay, with our script in place, let’s create the cron job to run once a month (on the first).

crontab -e 
0 0 1 * * /opt/local/etc/acme/renew.sh >/dev/null 2>&1

All set! Good Job 🙂

Install elasticsearch 5.x on SmartOS

Whenever possible, I like running software in containers instead of kvms. Aside from the obvious performance gains, server density increases significantly since you’re not kidnapping huge chunks of DRAM from the OS and holding it hostage.

I recently had a need to setup an elasticsearch 5.x cluster on a SmartOS machine. It was mostly straight-forward except for a couple gotchas. Here’s how you do it.

1. Create the zone/container. Here, we’re just using a stock ubuntu 16.04 LTS image. Note, the key here is the max_lwps field. Elasticsearch requires at least 2048.

vmadm create << EOF
{
  "brand": "lx",
  "image_uuid": "7b5981c4-1889-11e7-b4c5-3f3bdfc9b88b",
  "autoboot": true,
  "alias": "elastic1",
  "hostname": "es01",
  "dns_domain": "example.com",
  "resolvers": [
    "192.168.1.1",
    "8.8.8.8"
  ],
  "max_physical_memory": 8192,
  "max_swap": 4096,
  "quota": 60,
  "max_lwps": 2048,
  "nics": [
    {
      "nic_tag": "admin",
      "ip": "192.168.1.100",
      "netmask": "255.255.255.0",
      "gateway": "192.168.1.1",
      "primary": true
    }
  ]
}
EOF

2. Login to the container and install elasticsearch

zlogin `vmadm list | awk '/elastic/{print $1}'`
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" &amp;gt; /etc/apt/sources.list.d/elastic-5.x.list
apt-get update ; apt-get install elasticsearch

3. Disable system_call_filter install during bootstrap phase

echo "bootstrap.system_call_filter: false" >> /etc/elasticsearch/elasticsearch.yml

4. Enable and fire it up!

systemctl enable elasticsearch ; systemctl start elasticsearch

5. Tail the logs to make sure everything started up OK.

tail -f /var/log/elasticsearch/elasticsearch.log