A goat in a forest

Photo credit: Léonie Lejon (Unsplash license)

Edit: The article was updated. Thanks to the replies from Martin Tournoij and Kellen Scarlett.

Web analytics applications

I had been using Google Analytics for a while. It's easy to setup. Just add a script and is ready to go. Then I checked out the open source alternatives. Matomo was originally based on Piwik, but Piwik Pro went commercial only and is not open source anymore. Matomo is open source and is written in PHP.

I had tried Matomo with Caddy web server. Here's my comment. I had no problem to install it. But after installation, it said it had problem with the HTTP headers or proxy settings and I can't login after install. Also, it is hard to protect Matomo in Caddy, it has lots of paths that is required to process by PHP and also lots of paths (e.g. tmp, plugins, config) that should be protected and not accessible from Internet. Currently, it is better to be used with Nginx or Apache (some .htaccess files are provided by Matomo). There is no permanent free hosting plan offered by Matomo now.

Plausible is another trending open source web analytics application. It is written in Elixir and the Phoenix framework. However, it requires two database products (PostgreSQL and Clickhouse DB). Like Matomo, there is no permanent free hosting plan provided by Plausible now.

Enter GoatCounter

GoatCounter was created by Martin Tournoij. This application is open source and is using the European Public License. GoatCounter.com offered a free hosting plan (for non-commercial use only) and other commercial plans. Since it is open source, I would like to self-host it.

GoatCounter is written in Go language, it is delivered as a single executable. For storage it supports a local SQLite database or connect to PostgreSQL. Check out this web page for building and installation instruction. GoatCounter can be used as webserver on it's own and it supports ACME. For my blog, I had already occupied port 80 and 443 by Caddy to forward request to an existing application (Meiliserach). So, I would make GoatConter to listen on a TCP port other than 80 or 443. Then use virtual hosting in Caddy.

It is worth to mention about privacy and web analytics. Whatever web analytics application you are going to use, you should check if the application or how you collect the data is compliant to GDPR. For my blog, it requires users to accept the Cookie Consent before sending analytics data to GoatCounter.

Setup GoatCounter

Step 1: Create user

Create a group and an user account for GoatCounter. Then change the current user to it
1
2
3
# groupadd --system goatcounter
# useradd --system --gid goatcounter --create-home --home-dir /home/goatcounter goatcounter
# su - goatcounter

Step 2: Get GoatCounter

I fetch the package from official GitHub page.

Step 3: Create a domain configuration

Setup a domain configuration to use in GoatCounter. It would use SQLite database by default.
1
$ ./goatcounter create -email webmaster@example.org -domain analytics.example.org

It would prompt you for the password for the above account.

Step 4: Startup GoatCounter

1
$ ./goatcounter serve -listen 127.0.0.1:8000 -tls none -debug all

That's it! GoatCounter is running now, as simple as that. If everything is fine, just terminate or kill the process. We would use a systemd service to start GoatCounter. Also, we would start it without the debug flag.

Step 5: Setup systemd service for GoatCounter

Create this as /etc/systemd/system/goatcounter.service
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
[Unit]
Description=GoatCounter
After=network.target

[Service]
User=goatcounter
Group=goatcounter
WorkingDirectory=/home/goatcounter
ExecStart=/home/goatcounter/goatcounter serve -listen 127.0.0.1:8000 -tls none
SuccessExitStatus=0

[Install]
WantedBy=multi-user.target

Then enable and start the service with administrative right.

1
2
# systemctl enable goatcounter
# systemctl start goatcounter

Next, we are going to setup a web server to serve the GoatCounter application.

A quick take on web servers

When it comes to web server, the popular products in the market are Apache and Nginx. These are written in C. Apache, is a rock solid web server, it is in business for 25 years. More functionality is added as modules. I feel the Apache configurations a bit sort of boilerplate.

Nginx is another big player in the web server market. According to an article in Wikipedia, Nginx may have surpassed Apache in terms of market share. Anyway, I think Apache and Nginx combined already secured about 60% to 70% of the market share. Nginx was started in 15 years ago. Compared with Apache, I think Nginx is smaller and faster.

Caddy was started in 5 years ago, but it has more "Stars" than the two big players in GitHub. Caddy was written in Go language. The author of Caddy is Matt Holt. In 2017, commercial use of Caddy v1 binaries distributed by official Caddy project would need a commercial license. In 2019, this restriction was lifted. Now all development effort was put into Caddy v2. Caddy v2 was released in May 2020.

Caddy web server notable features:

  • Automatically managed HTTPS
    • For public web sites, certificates could be issued from Let's Encrypt automatically
    • For internal IP or internal domains, certificates could be issued from a locally-trusted, embedded CA
    • Web server stays up even there are TLS/OCSP/certificate issues
  • Written in Golang. It has better memory safety than application written in C
  • Simplicity, it had a small configuration file
  • Configuration could be change dynamically via the JSON API
  • Markdown rendering
  • Experimental HTTP/3 support
  • Single binary

For full details, check out the Caddy GitHub website. Note Caddy is a registered trademark of Light Code Labs, LLC.

Configure Caddy

In this article, we would use Caddyfile configuration, instead of JSON configuration.

Step 1: Get Caddy

Either use the package from Offical page or get it from GitHub

Step 2: Create a user for Caddy

If you are not using a package, you may need to create the caddy user manually.
1
2
# groupadd --system caddy
# useradd --system --gid caddy --home-dir / --shell /sbin/nologin caddy

Step 3: Automatic HTTPS

We are going to use the automatic HTTPS feature of Caddy. We just need to provide the email address and the public hostname of the web site. Caddy would use ACME (Automatic Certificate Managment Environment) to obtain a certifiate from Let's Encrypt.

By default, it would try the HTTP-01 challenge first, which relies on HTTP port 80 to be accessible by the Internet. If it is not accessible, it would try to use HTTPS TLS-ALPN-01 Challenge.

Make sure the HTTPS port (TCP port 443) is accessible by the Internet. The TLS-ALPN-01 Challenge would take place in about two to three minutes after Caddy is started.

Step 4: Create the Caddy configuration file

Below is a simple configuration file (/etc/caddy/Caddyfile) for Caddy v2.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
{
  # Replace by your contact email
  email webmaster@example.org
}

# Replace by your domain name and IP address
analytics.example.org {

  bind 172.16.0.2

  # Update it with the IP and port you had configured in GoatCounter
  reverse_proxy {
   to 127.0.0.1:8000
  transport http {
   }
  }

  log {
    # https://caddyserver.com/docs/caddyfile/directives/log
    output file /var/log/caddy/access.log {
      roll_size 100
      roll_keep 30
    }
    level info
  }

  @compressExtensions {
    path *.html / */ *.css *.js
  }
  encode @compressExtensions gzip

}

The configuration is small and simple. By default, Caddy performs HTTP to HTTPS redirection. For logging, the roll_size is 100MiB, which rotates the log file at 100MiB. roll_keep is to specify how many old log files to keep. We have also enabled gzip compression for some of the file extensions.

Next, create the necessary directories.

1
2
# mkdir -p /var/log/caddy
# chown caddy:caddy /var/log/caddy

Step 5: Start the Caddy web server

Here's the command to validate a caddy configuration file:
1
# caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile

If everything is alright, start the Caddy web server. If you are running on Linux, you should start it as a systemd service. Refer to the documentation for details. In here, I create the file as /etc/systemd/system/caddy.service as my Caddy web server is not installed by a package.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target

[Service]
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile --adapter caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

Next, enable and start the web server

1
2
# systemctl enable caddy
# systemctl start caddy

The Caddy web server is now configured. It would forward requests to the GoatCounter application.

Configure GoatCounter by using web console

Access the GoatCounter by using the browser. You need to enter the admin user and password when you create the GoatCounter by command line in the previous step. Next, you need to include the JavaScript site code into your website or the template of your static site generator. Configure your domain (for example, it's www.kappawingman.com for me) in the web console. The GoatCounter web analytic application is now ready to use.


Share this post on: TwitterRedditEmailHackerNewsLinkedInFacebookIndienewsFor fed.brid.gy




This website supports Webmention

Webmentions are included inside the static pages.

It needs to regenerate the web pages for the latest Webmentions.

Reactions: 2

Martin Tournoij ❤️ WE ARE LAUNCHING A NEW STARTUP BUSINESS/ECOSYSTEM ❤️


What if I don't have Webmention in my web site?

You could use Comment Parade to send message. Enter https://commentpara.de as the URL when asked to sign in.