# Recommended LogFormat for other_vhosts_access.log

## Problem

ISPConfig is a control panel that allows multiple users to maintain multiple web sites. Allowing all users access to the Apache access and error logs is not good for the following reasons:
- Information pertaining to sites owned by others would be a security breach.
- Determining what logs pertain to the user's site(s) would be very difficult.

ISPConfig instead sets each vhost configuration to create a separate log of only the entries pertaining to that site. However, this does not allow the systems administrator to detect global problems easily.

Debian's Apache package ships a ready-made answer to this: `other-vhosts-access-log`, a `conf-available` fragment enabled by default, which writes one combined log covering every vhost — `/var/log/apache2/other_vhosts_access.log`. With it, an administrator can review all entries for all web sites in one place, and more importantly, use fail2ban to detect attacks across multiple sites without having to create a new fail2ban jail every time a new site is added.

A combined log like this needs to record which site was visited, thus the `CustomLog` line, adding that information.

## The format

`/etc/apache2/conf-available/other-vhosts-access-log.conf` defines and uses the following stock nickname, and is enabled by default:

```apache
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
```

If you haven't touched `other_vhosts_access.log` before, this is almost certainly the format it's already writing — confirm with `a2query -c other-vhosts-access-log`.

RECOMMENDED: the following format makes the log more fail2ban-friendly:

```apache
LogFormat "%t %v:%p %h %l %u \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" \"%{Host}i\" %D" fail2ban
```

This reorders one field from `vhost_combined` and adds two:

- `%t` moves to the front — fail2ban's default date-detection (`datepattern`) works best when the timestamp is the first token on the line. In `vhost_combined`, `%t` sits after `%v:%p %h %l %u`, so a filter's `failregex` has to skip over the vhost/IP/ident fields before it can anchor on the date. Putting `%t` first lets fail2ban's auto date-matching work with a simpler, more reliable regex.
- Added `"%{Host}i"` — this is the literal `Host:` header the client sent, as opposed to `%v`, which is the vhost Apache resolved the request to. They can differ (e.g., a bot hitting your IP directly with a bogus or missing Host header, or probing for other vhosts on the same IP). Logging both lets a jail flag mismatches or malformed Host headers as suspicious. Logs `-` when absent (HTTP/1.0).
- Added `%D` — request duration in microseconds, something `vhost_combined` has no equivalent of at all. Useful for filters aimed at slow-request/slowloris-style abuse, or for confirming after the fact that a mitigation actually made a given kind of request cheap. (`%T` would only give whole seconds — too coarse for this.)

So the advantage isn't in the fields `vhost_combined` already has — it's `%t` repositioned for easier fail2ban parsing, plus two fields (Host header, request duration) that give fail2ban filters detection signal `vhost_combined` simply doesn't carry.

Both lines are packaged together as a ready-to-deploy drop-in: `conf-available/fail2ban-log.conf`, which mirrors `/etc/apache2/conf-available/` the same way `filter.d/`, `jail.d/`, and `action.d/` mirror fail2ban's own layout — copy it across directly.

## Why append rather than restructure

The obvious alternative is to move to Apache's stock "combined" field order (client IP first). The usual reason to do that is so fail2ban's stock apache filters work — but they cannot work here, and testing on fail2ban 1.0.2 shows why. The three stock access-log filters (`apache-badbots`, `apache-pass`, `apache-fakegooglebot`) anchor BOTH ends: `^<HOST>` at the start and `\"$"` right after the User-Agent. Verified:

- exact combined, no vhost, nothing appended → `apache-badbots` matches
- combined + fields appended after the UA → no match (`$` anchor)
- vhost-first (the recommended layout above) → no match (`^<HOST>` anchor)

A shared log for many vhosts must carry a vhost field somewhere, and either placement breaks those anchors. Stock compatibility is therefore unreachable no matter which order you choose, so it is not a reason to restructure.

Keeping the vhost-first order instead has a concrete payoff: filters written against it, such as `apache-vhost-wp-probe` and `apache-vhost-dokuwiki-crawl` in this project, work with NO CHANGES on top of the fields the stock filters need, because neither anchors at end-of-line, so trailing fields are ignored. Verified against captured samples:

| Filter | Sample | Result |
|---|---|---|
| `apache-vhost-wp-probe` | new-format WordPress sample | 16/16 |
| `apache-vhost-dokuwiki-crawl` | new-format DokuWiki @403 | 10/10 |
| `apache-vhost-dokuwiki-crawl` | new-format DokuWiki @200 | 0 (correct) |

**CHECKED (2026-09-13):** the one thing that could change this recommendation is whether the per-site vhost logs already use stock "combined" — if so, restructuring this shared log to match would let one filter set serve both. On Devaun they do (no per-vhost `CustomLog`/`LogFormat` overrides at all — confirmed with the grep below — and the per-site ISPConfig logs and the main `/var/log/apache2/access.log` are both plain stock combined). It does not change the recommendation, because this shared log still needs a vhost field that a single-vhost per-site log never needed in the first place; either placement of that field breaks the stock anchors regardless of what the per-site logs do. It does mean stock filters (e.g. `apache-badbots`) already work unmodified against the per-site logs with no conversion, on this host or any other using plain "combined" per-vhost logs — worth using directly if you want per-domain bot filtering independent of this shared log. Re-check this on a new host with:

```bash
grep -rh "CustomLog\|LogFormat" /etc/apache2/sites-enabled/ | sort -u
```

If you do restructure, the filters need their `datepattern` changed — see the note at the bottom of this file.

## Deploying it

`other-vhosts-access-log` is enabled by default, so the normal starting point is: it's already running with the stock `vhost_combined` format, and you're switching it to the `fail2ban` format above.

```bash
sudo cp conf-available/fail2ban-log.conf /etc/apache2/conf-available/fail2ban-log.conf
sudo a2disconf other-vhosts-access-log # disable the stock log
sudo a2enconf fail2ban-log # enable the fail2ban log
sudo apache2ctl configtest
sudo service apache2 reload   # SysV; use "systemctl reload apache2" where systemd runs
# OPTIONAL, and has issues
sudo logrotate -f /etc/logrotate.d/apache2 # rotate existing logs, removing other-vhosts-access_log entries
```

`scripts/setWebServer` runs the Apache-side steps above plus installs the WordPress-probe filter/jail in one pass — see the file itself before running it, since it hard-codes which jail it installs.

Install the filter/jail files as usual, then `sudo fail2ban-client reload`.

**Do not** add the `LogFormat` line to `/etc/apache2/apache2.conf`, and do not edit the stock `/etc/apache2/conf-available/other-vhosts-access-log.conf`. Both are Debian-packaged (dpkg conffiles); a future `apt upgrade apache2` can silently overwrite either one, or prompt to and get answered wrong during an unattended run. Keeping the definition in its own `conf-available/` file, enabled with `a2enconf`, is the mechanism Debian's Apache packaging provides specifically to avoid that.

You do NOT need to stop Apache, and you do NOT need to remove `other_vhosts_access.log`. Filters ignore old lines and will begin to acquire information on the new ones, so the mixed-format file works fine while switching over from the stock format — old lines simply stop being relevant as they age out of `findtime`. It's fine to leave the old lines in place: fail2ban ignores any line it doesn't understand, and the log will still rotate on its normal schedule.

DO NOT `rm` the log while Apache is running. Apache holds the file open and keeps writing to the now-unlinked inode: the disk space is not freed and no new file appears until Apache reopens its logs. If you want a clean file, rotate instead — this also signals Apache to reopen:

```bash
sudo logrotate -f /etc/logrotate.d/apache2
```

## If you restructure to client-IP-first order

This is a different axis from the deployment mechanics above — it is about the field *order* within the shared multi-vhost log, and only applies if you deliberately move away from the vhost-first layout. It is unrelated to whether a host needs the `fail2ban`-nicknamed log at all: a single-vhost server has no need for a shared multi-vhost log in the first place and can use Apache's stock `combined` format directly. (See `WorkingNotes/Backlog.md` for the fail2ban filter variants still needed for that case.)

If you do restructure this log, the date is no longer at the start of the line, and the filters' `datepattern` must change to the stock idiom:

```ini
datepattern = ^[^\[]*\[({DATE})
```

and every `failregex` must gain `\[\]` where the timestamp was, becoming e.g.:

```ini
failregex = ^<HOST> \S+ \S+ \[\] "(?:GET|POST|HEAD) /+(?:wp-login\.php| ...
```

This is not cosmetic. Tested on fail2ban 1.0.2: with the stock idiom, fail2ban keeps the `^[^\[]*` prefix (the client IP) and removes only the captured date, leaving empty brackets behind — which is exactly why the stock `apache-pass` filter is written with `\[\]`. An explicit `datepattern` that includes the brackets AND a leading `^[^\[]*` instead removes the whole match, taking the client IP with it, and then nothing matches.