Document Invoice Ninja incidents and troubleshooting (2026-05-15)

Added troubleshooting procedures for three recent issues: client portal
500 errors from uninitialized pdf_variables, MariaDB healthcheck
failures from password desync, and duplicate Stripe gateways. Included
root causes, detailed fixes, and notes on the upstream pdf_variables
bug in Invoice Ninja 5.13.22.
This commit is contained in:
Jared Swanson 2026-05-15 16:20:42 -04:00
parent b988d43085
commit e7e6c47354
1 changed files with 79 additions and 0 deletions

View File

@ -183,12 +183,91 @@ ssh jared@15.204.247.153 "docker exec invoiceninja-app-1 supervisorctl status"
Should show `queue-worker_00` and `queue-worker_01` as RUNNING.
### Client Portal 500 Errors on Invoice View (pdf_variables bug — IN 5.13.22)
**Symptom**: All clients get HTTP 500 when viewing invoices via the client portal.
**Laravel log signature** (check `docker/app/storage/logs/laravel.log` or `docker logs invoiceninja-app-1`):
```
Undefined property: stdClass::$product_columns in PdfSlot.php:149
```
**Root cause**: The `pdf_variables` field in company settings was never properly initialized (left as empty object `{}`). `PdfSlot.php` crashes when it tries to access named columns on the empty object. This is an unreported upstream bug in Invoice Ninja 5.13.22 with no upstream fix as of 2026-05-15.
**Fix** — run once to populate the defaults:
```bash
docker exec invoiceninja-app-1 php artisan tinker --execute="
\$company = App\Models\Company::first();
\$settings = \$company->settings;
\$defaults = App\DataMapper\CompanySettings::getEntityVariableDefaults();
\$settings->pdf_variables = (object) \$defaults;
\$company->settings = \$settings;
\$company->save();
echo 'Done: ' . json_encode(\$company->fresh()->settings->pdf_variables->product_columns);
"
```
A successful run prints the populated `product_columns` array. No restart required — the fix takes effect immediately.
### MariaDB Health Check Unhealthy
**Symptom**: `docker ps` shows `invoiceninja-db-1` with status `(unhealthy)`.
**Root cause**: The healthcheck user's password in the container's `.my-healthcheck.cnf` is out of sync with the actual MariaDB user password (can happen after env changes or manual password resets).
**Fix** — sync the password:
```bash
docker exec invoiceninja-db-1 mariadb -uroot -p<ROOT_PASS> -e \
"ALTER USER 'healthcheck'@'localhost' IDENTIFIED BY '<password_from_.my-healthcheck.cnf>'"
```
Replace `<ROOT_PASS>` with the value of `MYSQL_ROOT_PASSWORD` from `~/services/invoiceninja/env`, and `<password_from_.my-healthcheck.cnf>` with the password found in that file inside the container:
```bash
docker exec invoiceninja-db-1 cat /root/.my-healthcheck.cnf
```
After running the ALTER USER, the container transitions to `healthy` within one health-check interval (typically 30 s).
### Duplicate Stripe Payment Gateway
**Symptom**: Two Stripe gateways appear in Settings > Payment Gateways, or payments route through an old/invalid gateway config.
**Fix**: Delete the older/unused gateway entry from the Invoice Ninja admin UI (Settings > Payment Gateways → delete). Keep only the active gateway.
**Also check**: After any gateway cleanup, verify `accepted_credit_cards` is non-zero. A value of `0` means no card types are accepted; set it to `7` (Visa + Mastercard + Amex) or the desired bitmask via Settings > Payment Gateways > Edit.
## Migration History
This deployment uses the **official Invoice Ninja Docker stack** (as of 2025-10-14). Previous custom stack was replaced to align with upstream recommendations and simplify maintenance.
**Pre-migration backup location**: `~/backups/invoiceninja-pre-official/` on server (retained until deployment proven stable)
## Incidents
### 2026-05-15 — Client Portal 500 / MariaDB Unhealthy / Duplicate Gateway
**Symptoms observed:**
- All clients received HTTP 500 on invoice view in the client portal
- `invoiceninja-db-1` had been showing `(unhealthy)` for 29 days
- Two Stripe payment gateways were present; `accepted_credit_cards` was `0`
**Root causes & fixes:**
| Issue | Root cause | Fix applied |
|-------|------------|-------------|
| Portal 500 errors | `pdf_variables` field never initialized (`{}`); `PdfSlot.php:149` crashes on `->product_columns` | Populated defaults via `tinker` (see troubleshooting section above) |
| MariaDB unhealthy | Healthcheck user password out of sync with `.my-healthcheck.cnf` | `ALTER USER 'healthcheck'@'localhost' IDENTIFIED BY '...'` |
| Duplicate Stripe gateway | Old gateway left over from previous config | Deleted stale gateway from admin UI |
| `accepted_credit_cards: 0` | Never set after initial setup | Set to `7` (Visa + Mastercard + Amex) in gateway settings |
**Notes:**
- The `pdf_variables` bug is unreported upstream as of 2026-05-15 (Invoice Ninja 5.13.22). Watch future releases for a permanent fix.
- The MariaDB health check had been broken for 29 days before discovery — consider adding an external health alert for this container.
## References
- Official Docker Repository: https://github.com/invoiceninja/dockerfiles